The current version of emulationstation looks for the gamelist.xml file first in the rom folder and if it doesn’t exist it looks in the ~/.emulationstation/gamelists/(system) folder and finally /etc/emulationstation/gamelists/(system). The images can be anywhere since the gamelist tells ES where to look.

I prefer the gamelist and images in the rom folder since it makes it easier to copy files to and from the retropie. I actually wrote the scraper to run on my desktop and then I copied the files over to make upgrading systems easier.

ES’s author, Aloshi, has the opposite opinion and prefers that the gamelist and images not clutter up the rom folder so the internal scraper places things in the ~/.emulationstation folder.

My script can be invoked with command line flags to tell it where to find the roms and where you will be placing the gamelist and images. This is how it works when my script is invoked from the retropie-setup script. But keep in mind the gamelist can only be in one of three places if you expect ES to find it.

Something like this(with (system) replaced by the system) but you need to double check the paths since I’m doing this from memory. The image path is specified twice since this script was originally designed to be run from a different system, this allowed the host and target system to have a different folder location.

scraper -output_file ~/.emulationstation/gamelists/(system)/gamelist.xml -img_dir ~/.emulationstation/downloaded_images/(system) -img_path ~/.emulationstation/downloaded_images/(system) -rom_dir ~/retropie/roms/(system)

[quote=110013]Sselph,

I’m having a little issue getting your scraper working. I’ve ran through the install guide on github.com and my pi2 has downloaded release version v1.0.4.

I’ve scrapped the Megadrive and Snes rom folders and it created the image folders which have many thumbnail size pictures but the gamelist.xml that got created in the roms fokder is empty.

For mame the folder again it created the images folder, but this time no gamelist.xml file.

Is there a command I can run to generate the gamelist.xml properly?

Not sure why it’s gone wrong.

I’m using Retropie 3.2.1.

Thanks

[/quote]

This is happening for me now on any system – started doing this a little over a week ago.

Hi cyborguk,

The gamelist.xml is placed in the current working directory. This is the same directory where it is looking for roms and placing images so if those are working it should be fine. You can try adding -output_file /home/pi/RetroPie/roms/(rom_dir)/gamelist.xml to make it explicit. Permissions don’t seem to be the issue since it is able to create an images folder and write to it and it should give an error in that case. The other option is the scraper didn’t find any hashes which doesn’t appear to be the case since it downloaded images. I haven’t changed anything in my script in over a month except adding and removing some hashes in my database.

This was happening to me using the Windows version of the scraper, I’ll try scraping using my Pi tonight and see whether it makes a difference :)

*edit* appears to be only certain Sega CD games which it affects (strange!).

Ran this:

pi@retropie ~/RetroPie/roms/megadrive $ scraper -thumb_only -output_file /home/pi/RetroPie/roms/megadrive/gamelist.xml

No luck, images folder is created but no gamelist.xml

Formatted another SD card and installed 3.2.1 on it and put a few games in the Megadrive folder and installed the Sselph scraper manually as the experimental one doesn’t work and this time it worked.

So my main SD card doesn’t work, what could be the issue?

OK, so I figured out my issue. Not sure if it’s the same issue that others are having, but, I came across this:
https://github.com/RetroPie/RetroPie-Setup/issues/807

Emulationstation seems to load all gamelists at startup and writes them back at exit. So if emulationstation is running and you modify a gamelist any modification will be overwritten.

So, it appears that EmulationStation cannot be running while you run the scraper. If you’re running it from SSH while EmulationStation is up, or if you try to run it from the RetroPie setup script / Experimental through EmulationStation it will scrape everything, but your gamelist.xml will get overwritten. So you need to shut down EmulationStation and drop to the console, run scraper (or sudo ~/RetroPie-Setup/retropie_setup.sh) from the command-line, and then when you restart EmulationStation again, everything should load properly.

Ideally, the scraper should be able to handle this by telling EmulationStation to reload the gamelist somehow. I’m assuming it’s possible, since the built-in scraper can do it. In the meantime, I’d suggest adding this bit of info to the README. Excellent job on the scraper, though. Works much better than the built-in one. I’d love to see this become the default, or even perhaps just merging the searching algorithm in with the default one.

if you exit ES with f4 I think it doesn’t save any gameslists. I might look at changing the exit so you have exit + save and exit without saving options or so

Hi zigurana,

Yeah this is most likely what is happening.

As for adding my code to ES, I have submitted https://github.com/Aloshi/EmulationStation/pull/355 to ES and it is in their unstable branch. This code is somewhat out of date but it is there. Now that Go can create C modules it might be possible to refactor my code or create a small wrapper to allow ES to import it as an optional library so that it is kept more up to date. If you are a programmer and want to look in to this feel free and I’ll be happy to answer any questions.

The other solution to this is that ES was moving to using sqlite for storage and I already have some unchecked in code to also use sqlite. When this happens ES would no longer be dumping everything to the xml on exit which would probably solve the issue.