512 is way too high for emulationstation, specially when playing PSX games. The GPU needs very little memory with the games, as most of the emulation is handled by the CPU, and the games load from the CPU memory split, not the GPU. 128 mb to 256 mb should be all you need, everything will run fine with that.
Giving the GPU 512 mb of ram causes some PSX games not to load, and others to run very choppy. I am actually running my Pi with 600 mb for the CPU and 400 mb with the GPU, because I am using Kodi. I have done a ton of testing with Kodi 15 RC 3 and found that it works best with around 400 mb of ram, as it’s very GPU heavy unlike the emulators. More ram is always nice for Kodi, but I limit it to 400 because anything more than that and I start having issues with the PSX games and some N64 games running smoothly/locking up the pi. This for me seems to be the happy balance as all of my emulators run great and Kodi runs great too. With less memory I was having more crashes with Kodi, specially streaming HQ 1080p movies…
But yea keep it some where between 128 and 256 for the GPU giving the cpu the rest should keep all of those systems running just fine.
Just edit /boot/config.txt and comment on the gpu_memory line and change that to 128 and or 256 and that should set it.
As Alucard said, the performance issues with emulators in most cases are not memory related, they are CPU related, or simply sometimes emulators that are not optimized. PSX games run pretty much 100% for me. I rarely have any slowdown on any games, and I have around 150 psx games on my system. The N64 emulator needs some work and could probably be optimized a lot better for Pi, I think that’s the main issue with that.