What you are talking about has nothing to do with “shaders” in the retropie sense… What epsxe is doing is applying high end filters to your games, it’s using things like FSAA and anisotropic filtering, along with higher resolution, to greatly enhance your psx games. And as you noted, the results can be quite stunning… The raspberry pi’s gpu on the other hand is simply not powerful enough to pull off effects like this… There is an option in the RGUI menu for pcsx to put it in “enhanced resolution” or something along those lines. This does greatly improve the look, sharpness and smoothness of the games, but again the pi really isn’t powerful enough to use this feature, and most games are unplayable/unstable with it enabled.
The best solution I found was using one of the built-in retroarch shaders. I use the hq2x_lcd3x.glslp for PSX and really all my retroarch emulators. What it does is sharpens the screen a bit, as well as smooths things out greatly by using a crt scan line filter. On PSX it helps a good bit with the pixelation and greatly improves the look of the games. It’s not going to “enhance” the games to like what your talking about, but it does help and really makes things look a lot better. It has gives the games a nice “retro” look and feel to them by using the scan lines, and thus I use it for all my emulators. Hope this helps some