Well, I may have just solved my problem… I edited the following file.

/opt/retropie/configs/all/autoconf.cfg

I changed the following line from 0 to 1.

es_swap_a_b = 1

I then configured the controller in ES, and mapped A (X) and B (O) backwards.

Now the controller is mapped properly in the games, and ES is now mapped properly once it does the swap…

Weird, but seems to be all figured out… Wanted to post this in case anybody else runs into this issue like I did.