Adding scrapers to ES source? Questions (Was Hey Jools,,,,)

[dabone]

I’ve been trying to add systems to the scraper, and as Jools said over on github, the issue tracker is not really the place for advice on this.

Where should discussions like this take place?

(And I’m still wondering why my code change doesn’t work on the master, but does on the unstable.)

Later,
dabone

[zerojay]

This is fine, really. What were you trying to add?

[dabone]

I’ve been trying to add the coco to the scraper, but I’ve hit a wall.

I’ve been adding it to Platform.cpp, Platform.h and GamesDBScraper.cpp.

In PlatformId.h I add line 27

COCO,

In PlatformId.cpp I add line 27

“coco”,

And in GamesDBScraper.cpp I add line 27

(COCO, “TRS-80 Color Computer”)

The strange thing is that this works under the original es from Aloshi, and the unstable branch from retropie, but not in the main. I also did this with the vectrex with the same results, it’s like something else has been added as a check for the platforms but I’m not seeing it in the change logs.

Any suggestions?

[petrockblog]

Please post a unified diff against RetroPie master branch (via pastebin or similar)

[dabone]

Diff for the changes

It sees the changes, but when I scrape, it returns no data.

[petrockblog]

ok. I think the problem you are having is because we modified the getCleanName function to not strip parentheses and your COCO roms I suspect have data in parentheses? (the games db is quite fussy on input)

A quick fix is to add a removeParenthesis to the scraper code where it calls cleanName, but I think I will add a new FileData function for DisplayName and CleanName, or rename your roms of course.

[dabone]

Yep, that was it, parentheses in the names.

Works fine with renamed roms, so I don’t feel so dumb now.

Has anyone ever worked out a fix for games like pac-man?
Because of the “-” the game can’t scrape.

Thank you.

Later,
dabone

[petrockblog]

either it could be worked around in ES by replacing – with ” “, but tbh the gamesdb search probably needs improving.

Why not just use sselph’s scraper which uses checksums and does a much better job ?

[dabone]

Because it doesn’t work on other systems, like atari 5200, vectrex, coco, atari 800, apple2, amiga, etc….. There is still a real need for the original scraper.

The computer ones are the worse, because of multiple different releases of the same program.

My pi setup is very computer centric, being housed in a commodore 64 unit, using the original keyboard, and having a custom coco joystick interface for coco and apple2 games, dual db9 joysticks for atari 2600,800,commodore,amiga and then 2 wireless 360 controllers for newer systems. Currently I’m using a 120GB ssd as storage.

Later,
dabone

[herbfargus]

Perhaps you should submit a request to sselph to improve his scraper as he pulls from the same source- he just does a better job. You could even help him add the hashes for the games.

[petrockblog]

a good idea, then everyone can benefit.

[dabone]

My point was that the computer games should be able to scrape by name, not hashes.

[herbfargus]

hashes are more accurate.

[zerojay]

The MAME stuff in sselph’s scraper seaches by name so it shouldn’t be too difficult to get something similar working for computer systems if needed. Poke him and see what he says.

[Author]

Posts