Best possible input lag with retroarch?

[ngseven]

Hi there,

I am using retroarch with lr-fba on a RPI 2 with the retropie 3.1 SD card image to emulate Neo Geo games. I am wondering whether the 4 frames (at 60 Hz monitor refresh) input lag are improvable. I have determined the input lag with a camera recording at 60 Hz and 120 Hz and counted the number of frames passing between pressing a key (or a button on my Neo Geo X arcade stick USB joystick) and the emulated Neo Geo reacting. I never managed to get below the 4 frames input lag, regardless of the settings I tried: using or not using overlays (I never used shaders), different video drivers (gl, sdl, vg), hard GPU sync on/off (in fact, hard GPU sync does not seem to work since the output of –verbose retroarch suggests that the feature is not compiled into the code), choosing different values for usbhid.mousepoll in /boot/cmdline.txt and so on. Also, I tried a joystick hard-wired to GPIO.

Now, what would interest me is if it is possible to get below 4 frames input lag with retroarch/lr-fba and if yes, how this can be done. This would be nice for the fighter games on the Neo Geo…

Cheers,

NGseven

[gizmo98]

Have you already disabled threaded video?

[ngseven]

Yes, I did.

So in your opinion, it could be better? Do you know what should be achievable?

Maybe the problem is the display, it’s an LG IPS234 monitor. Didn’t find any measurements on the internet about the monitor’s latency. On the other hand, I checked the input lag of my PC running MAME with that monitor, and that combination only had 3 frames input lag…

[ngseven]

With respect to the stuff I wrote above, does anyone know if its possible to get hard GPU sync to work in retropie? Do I have to compile retroarch from sources with certain dependencies for that?

[dankcushions]

retroarch does have an input handler that checks for hotkeys/etc, before it passes your inputs to the emulator, which might add a bit of latency, but it’s pretty simple so i doubt it. you could install retroarch and lr-fba on your PC to see if it’s the same result there?

there’s a couple of other retropie emulators that run neogeo games that might have different results:
lr-mame2003
gngeo
lr-fba-next
(more?)

also, you might get more info at libretro.com/forums as the developers hang out there. let us know what you find :)

[ngseven]

Hi dankcushions, thanks a lot for your suggestions! I’ll try some things out and post it here if I find a solution!

[ngseven]

Update: I managed to get retroarch and fba-libretro to run on my PC, using the same monitor and USB keyboard as on my PI. Using hard GPU sync on, input lag went down to 2 frames! With hard GPU sync off, I got the same input lag of 4 frames or more as on my PI.

So, at the moment I think that the problem is that hard GPU sync does not work in retroarch on retropie, as I already conjectured above.

Hence, it would be interesting how I could get hard GPU sync to work in retroarch on retropie, e.g., by compiling retroarch with certain options. However, I remember that certain dependencies would have to be included, and I am not a Linux expert, so any suggestions on how to successfully activate hard GPU sync would be very welcome.

Of course, the problem might be then that the processing power of the PI 2 does not suffice for Neo Geo games with hard GPU sync switched on…

[ngseven]

Update: I looked at the sources of retroarch, and as far as I can see, hard GPU sync is not compiled in if OpenGL ES is used. Hence, I tried to compile retroarch with –enable-opengl but without –enable-gles, but this is not meant to be done since GLES is automatically used on platforms with a videocore GPU. I fiddled a bit with the scripts and the sources, but I cannot get retroarch to run with opengl but without open gl es.

Hence, I think one might solve the problem if one would succeed in compiling retroarch on the pi without GLES, but as I said, no success yet.

Any hint on how to do it would be very welcome.

[dankcushions]

there’s an alternative video driver that uses open gl 2.0 rather than open gles. it’s only available as a complete system image I think, so you’d have to install retropie on top of that via the scripts.

seems like a lot of effort but yeah! http://dri.freedesktop.org/wiki/VC4/

EDIT: having re-read your post I’m not sure this would change the compilation stuff anyway!

[petrockblog]

there is a dispmanx backend without gles you can try (–enable-dispmanx)

There is no finished opengl for the rpi yet (still in development).

[ngseven]

Thanks a lot for your help! I’ll see what I can do, maybe the stuff will work when a full OpenGL implementation for the pi exists.

[furntree]

Hey ngseven! I have been working on this for a few days now and have had some interesting results.

I ended up using this realtime kernel disk image of Raspbian but am not sure if it was totally necessary. I am an old Linux audio geek so I like realtime kernels!

http://docs.emlid.com/navio/Downloads/Real-time-Linux-RPi2/

Then I install SDL-1.2.15-raspberrypi for dispmanx which is basically a fast way to get the rpi to render 2d graphics.

https://github.com/vanfanel/SDL-1.2.15-raspberrypi

Apparently you don’t need to do this part ^

After that is installed I downloaded the latest Retroarch and configured using these arguments to enable the drivers I wanted.

./configure --disable-x11 --enable-floathard --enable-neon --disable-ffmpeg --enable-udev --disable-sdl --disable-pulse --disable-oss --enable-dispmanx

After this you can run Retroarch and find we can run the following drivers:

Input: linuxraw
Joypad: linuxraw
Video: dispmanx
Audio: alsa

I also turned vsync off etc.

With this configuration I was able to get 2 frame input delay! Down from 6 frames with stock Retropie! This is using libretro-fba

2 frame input delay should be about half the latency of inputs on a PS3 fighting game title which I believe is about 4 frames.

I recorded the input vs monitor output at 60fps and played it back frame by frame for testing: https://dl.dropboxusercontent.com/u/225000/test%20%2802%29.mp4

Let me know your findings and if any of this helps you!

Edit: I haven’t tried Xbox360 controllers yet and ALSA gives me a little crackle here and there. Will see if I can iron out the bugs.

[petrockblog]

RetroArch as you have built it will not use SDL 1 btw so you don’t need to install SDL first.

[furntree]

Thanks buzz! If I remember correctly I was trying to get around a compilation error so may have installed it unnecessarily! Will update my process thank you! :)

[dankcushions]

cool progress! is performance near enough the same? I’m curious as to whether dispmanx improves performance on some of the more intense 2d games, as I presume it should as it’s closer to the hardware than the regular gl driver?

[furntree]

dank: I haven’t noticed any problems. There is about a 10-20% CPU overhead using the RT kernel as you are forced to use an older compatible USB driver.

http://community.emlid.com/t/rt-kernel-unusual-dwc-otg-high-8k-s-interrupt-rate-using-quite-some-cpu/1002

I honestly haven’t noticed a problem with it though. I have tested Genesis on Picodrive which is pretty intensive without issues.

I tried setting display mode 2 for 480p@60hz however I am getting overscan on the top and bottom? The retroarch viewport settings do not seem to do anything when running dispmanx but the aspect ratio does.

[ngseven]

Hi furntree,

thanks a lot for your help and the stuff you tried out! After a lot of frustration with trying out stuff, I switched to PC/Windows for retroarch, but with this new stuff you told us, I’ll check your hints out when I have time. Sounds very interesting!

[dankcushions]

[quote=110919]dank: I haven’t noticed any problems. There is about a 10-20% CPU overhead using the RT kernel as you are forced to use an older compatible USB driver.

http://community.emlid.com/t/rt-kernel-unusual-dwc-otg-high-8k-s-interrupt-rate-using-quite-some-cpu/1002

I honestly haven’t noticed a problem with it though. I have tested Genesis on Picodrive which is pretty intensive without issues.

I tried setting display mode 2 for 480p@60hz however I am getting overscan on the top and bottom? The retroarch viewport settings do not seem to do anything when running dispmanx but the aspect ratio does.
[/quote]interesting! it sounds from your previous post like the RT kernal isn’t maybe necessary, though? even so, i’m not sure the display driver is the limiting factor in most of these 2d systems – probably the CPU, but still :)

EDIT: apart from cutting latency of course – that’s definitely awesome regardless!

overscan: not sure about that – might be one for the libretro forums? i’ve always run out full 1080p on my display, and used integer scaling (although lately i’ve favoured just letting it stretch to full height) i think using non-native display modes puts you at the mercy of your display’s handling of those resolutions, perhaps?

[furntree]

I haven’t tested on normal Raspbian yet honestly only because my results have been so good with RT. Another benefit which may be partially related to RT kernel is that I can successfully run audio latency all the way down to 32ms in Retroarch which is pretty quick! (16ms crackles and distorts)

It took awhile but I finally got a standalone copy of pcsx-rearmed (non retroarch) to compile properly and have been running FF7 what seems like perfectly. I am not getting a lot of the graphical and speed issues I was having in other PCSX versions on other releases. I even had double res textures option running full speed but then the game crashed…

Am going to try and get as many cores/emus as possible running the best it can then try and release a SD card image for you dudes to try. I think it will be really helpful, especially to the competitive fighting game players to have something that is basically as good as anything else with minimal input lag out of the box.

Also I figured out the overscan issue is just my TV not handling 640×480 very well. I have had much better success running 1024×768 to maintain aspect ratio!

[petrockblog]

please can you post me your retroarch.cfg and the console output from running retroarch –verbose (with a nes core & rom) ? The latest retroarch is segfaulting for me with the dispmanx driver enabled in retroarch.cfg (older codebase works). Wondering what revision you are running etc.

I am currently git bisecting it to post a bug report upstream.

[petrockblog]

ok – looks like it’s something config related triggering it – don’t need a config now as I have a working one and will diff/compare. Cheers.

[petrockblog]

ok – it crashes with video_threaded = true so I will report this upstream. BTW when you were comparing the drivers, did you test the GL driver with video_threaded=false and the same options for dispmanx. I would not have though there would be major differences in performance.

[furntree]

I do not have a NES core working currently but to be fair I haven’t tried too hard either. Regardless here is my output this is kind of the best I can do over SSH. I can not get the entire console output to go into a file for some reason:

Didn't read whole file: /sys/devices/system/cpu/present.
Didn't read whole file: /sys/devices/system/cpu/possible.
RetroArch [INFO] :: === Build =======================================RetroArch [INFO] :: [CPUID]: Features:
Built: Nov 30 2015
RetroArch [INFO] :: Version: 1.3.0
RetroArch [INFO] :: Git: 3cd7014
RetroArch [INFO] :: =================================================
RetroArch [INFO] :: [CPUID]: Features:
RetroArch [INFO] :: Looking for config in: "/home/pi/.config/retroarch/retroarch.cfg".
RetroArch [INFO] :: Config: loading config from: /home/pi/.config/retroarch/retroarch.cfg.
RetroArch [INFO] :: Threaded rarch_task started
RetroArch [INFO] :: Loading dynamic libretro core from: "/home/pi/.config/retroarch/cores/picodrive_libretro.so"
RetroArch [INFO] :: Environ SET_VARIABLES.
RetroArch [INFO] :: Version of libretro API: 1
RetroArch [INFO] :: Compiled against API: 1
RetroArch [INFO] :: Environ PERFORMANCE_LEVEL: 0.
RetroArch [INFO] :: Environ GET_LOG_INTERFACE.
RetroArch [INFO] :: Environ SET_DISK_CONTROL_INTERFACE.
RetroArch [INFO] :: Environ GET_VARIABLE picodrive_input1:
RetroArch [INFO] ::     3 button pad
RetroArch [INFO] :: Environ GET_VARIABLE picodrive_input2:
RetroArch [INFO] ::     3 button pad
RetroArch [INFO] :: Environ GET_VARIABLE picodrive_sprlim:
RetroArch [INFO] ::     disabled
RetroArch [INFO] :: Environ GET_VARIABLE picodrive_ramcart:
RetroArch [INFO] ::     disabled
RetroArch [INFO] :: Environ GET_VARIABLE picodrive_region:
RetroArch [INFO] ::     Auto
RetroArch [INFO] :: Environ GET_VARIABLE picodrive_drc:
RetroArch [INFO] ::     enabled
RetroArch [INFO] :: libretro core requires content, but nothing was provided.
RetroArch [INFO] :: Does not have enough samples for monitor refresh rate estimation. Requires to run for at least 4096 frames.

Here is my config file: http://fpaste.org/296043/93866914/

Just to confirm you are using RPI 2?

[petrockblog]

Yep RPI2 – I found the cause – Please see my posts above. I’m constructing a report for upstream.

[petrockblog]

https://github.com/libretro/RetroArch/issues/2503

[furntree]

Good find. Was able to reproduce instantly by enabling threaded video. Is there really a benefit to having this enabled?

Edit – Saw the new commit. Trying it now.

[petrockblog]

threaded video makes a big difference for performance for some stuff – at least with the gl driver (and for people with rpi1). We changed the default to “false” for a retropie release and there were lots of complaints regarding performance. It is easily switchable via our “config editor” from the retropie menu.

from retroarch.cfg template:

Use threaded video driver. Using this might improve performance at possible cost of latency and more video stuttering.

I am adding the dispmanx driver to our retroarch build (you will need to build from source of course in retropie-setup). I will add an option for switching config in the config editor.

[petrockblog]

another thing that possibly could affect latency (or not) is the cpu governor on retropie-setup – it’s set to “ondemand” by default, but you can switch it to “performance” which might help. this can be changed via retropie-setup -> setup / configuration -> configure runcommand launch script

[furntree]

Wow thanks so much buzz! Did not think I would ever make such a major impact!

One thing I did test while doing all of this was loading my own compiled Retroarch build (dispmanx enabled and on) through EmulationStation and it would just close and bring me back to the frontend.

Figuring that one out is a bit above my head but I just wanted to put it out there.

[dankcushions]

so i’ve just downloaded the latest retropie_setup script (with the updated retroarch.sh) and rebuilt retroarch from source. dispmanx appears as a video driver in the RGUI, and if i select it and restart the game it works, but…
– it has what appears to be the video_smooth option set to true, regardless of the config setting (possibly the smoothing/scaling is hard-coded to the driver??)
– it crashes when you go into the RGUI menu unless you turn off video_threaded to false (and also do the same, or comment out the same entry in /all/retroarch.cfg) – does this happen for anyone else?
EDIT: i seem to get these display crashes in all cores and with video_threaded = “false” :( just need to load the RGUI to trigger after a few seconds.

this is all in FBA-next running neogeo games, btw.

[MattRigg]

Hey furntree! Great job on fiddling with this. I’m in a similar position where input lag is killing the experience for me (getting a 7 frame lag on the NES/SNES cores (plus the 2 frames my TV adds!))

Any chance you could upload an image of what you’ve got working so far? I’m curious to know if the input lag can be reduced to a level that I would personally find acceptable. 9 frames just isn’t good enough for me. If I can get it down to 2 (plus the 2 from my TV) I’d be more willing to put more time into getting RetroPie up and running. As it stands, the stock image/script installer just don’t cut it imo.

Don’t get me wrong, I appreciate the effort that’s gone into this project and understand that the technical problems are actually related to Libretro/RetroArch, but I don’t really want to invest more time into this if it isn’t possible to reduce the input lag.

[dankcushions]

I think it’s basically the dispmanx driver that is the difference. if you go into /opt/retropie/configs/all/retroarch.cfg

and change
video_driver = “gl”
to
video_driver = “dispmanx”

that should enable it for all libretro systems (might cause issues with some cores – if so comment out the line here and apply it to the specific system retroarch.cfgs instead). please let us know what you find! :)

[MattRigg]

Hey dankcushions! Ok, cool. I’ll have another fiddle around with it at some point and see what happens.

Out of pure curiosity, I ended up downloading the RT kernel Raspbian that furntree mentioned, compiled RetroArch using his custom flags then proceeded to download/compile FCEUmm and see what happens. Performing the same SMB3 test I did in my other post I ended up shaving 1 or 2 frames off the input lag (mostly 2, sometimes 1, but this could’ve been a camera/TV refresh problem – my test isn’t exactly scientific!).

While that brings my lag down from 9 frames to 7 (or, 7 to 5 if you deduct the 2 frame lag my TV causes), it’s still a couple of frames off what furntree has accomplished. Granted, I’m testing a different core and I’m not exactly sure what his setup is. This is why I wanted an image of his SD. Firstly I’d want to run my own test using his setup. Secondly, it’d be interesting to use his setup as a starting point then, working backwards, start changing the settings to match my own and see which (if any) start causing lag.

btw, you’ll have to excuse my lack of knowledge on all of this. I’m still relatively new to the world of Linux. I’ve completed a few other Linux-based projects (NAS server, Hyperion box etc) but they were all mainly just a case of copying/pasting commands and hoping for the best!

[dankcushions]

sounds like between you the approach is about as scientific as we’ve had! :) i think you’re doing a great job and it’d be awesome to get some numbers on all these different eventualities as it would perhaps give me an excuse as to why i don’t seem to be able to get past about world 6 in super mario bros :)

the dispmanx stuff is interesting but it’s a bit of a niche driver that definitely has some issues in retroarch, but perhaps they’ll be more likely to be fixed (or the other drivers improved) if it’s proven to have the least latency.

[furntree]

Hey guys didn’t see this popped up again!

I actually gave up on this project because I couldn’t find a reliable way of getting it to work with low latency input. The best I was able to get was by using dispmanx as the display driver and linuxraw instead of udev for joystick inputs. The problem with linuxraw is…..no Xbox 360 pad support.

I also used the realtime kernel, but beyond that I didn’t really do anything too fancy. Keep in mind in that video I am running fba libretro running Street Fighter Alpha 2 test mode. Games you test may or may not have inherent latency to them.

If you are looking for a pocket sized computer to bring around with you for fighting games like I am. You might want to look into the UP Board running Windows:

Homepage

Best of luck!

[Author]

Posts