Theoretically this could work, provided that you only use 2 types of controllers.
You’ll have to redo the ESconfig, with both controllers plugged in at the same time. Set the one up as Player 1 and the other up as Player 2, that way both controllers would be able to navigate through the ES Menus. Unless ESconfig takes controller ID’s into account (which it might) it should work. If not, you’d have to have both controllers plugged in at all times for this to work.
From then on it’s pretty straightforward. You’ll just need to configure a retroarch.cfg file for each emulator.