I have my MiniPac working with both mame and nes, now. I went through the retroarch.cfg line by line. I removed a bunch of extra stuff after I read online that joystick/gamepad configuration may take precedence over keyboard and mess stuff up. That shader hotkey is in there, make sure you remove it if you have buttons bound to it.
Also, you can’t use raw numbers like that. The comments in the cfg file right above output will tell you all the possible options. I think what you want is:
input_player1_start = num1
input_player1_select = num5
input_player2_start = num2
input_player2_select = num6
If all else fails and things still aren’t working right, quit out of EmulationStation and run: startx from the command line to get into XWindows. From there, you can run: xev from the terminal and it output the keypresses that your joysticks and buttons output.
Hope that helps!