[Floob]

Just had a major meltdown with some atarilynx rom scraping which seemed fine before. Can you see where the issue may be?

github.com/sselph/scraper/ds.(*Hasher).Hash(0x1080aa90, 0x10f1d320, 0x23, 0x0, 0x0, 0x0, 0x0)
        /home/sselph/go/src/github.com/sselph/scraper/ds/hasher.go:32 +0x170 fp=0x1a462a4c sp=0x1a4629e0
github.com/sselph/scraper/ds.(*Hasher).Hash(0x1080aa90, 0x10f1d320, 0x23, 0x0, 0x0, 0x0, 0x0)
        /home/sselph/go/src/github.com/sselph/scraper/ds/hasher.go:32 +0x170 fp=0x1a462ab8 sp=0x1a462a4c
github.com/sselph/scraper/ds.(*Hasher).Hash(0x1080aa90, 0x10f1d320, 0x23, 0x0, 0x0, 0x0, 0x0)
        /home/sselph/go/src/github.com/sselph/scraper/ds/hasher.go:32 +0x170 fp=0x1a462b24 sp=0x1a462ab8
...additional frames elided...
created by main.CrawlROMs
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:173 +0x5e4

goroutine 1 [chan send]:
main.CrawlROMs(0x11522cc0, 0x10a48010, 0x1, 0x1, 0x10810140, 0x1080aa88, 0x0, 0x0)
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:184 +0xf98
main.Scrape(0x10a48010, 0x1, 0x1, 0x10810140, 0x1080aa88, 0x0, 0x0)
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:285 +0x194
main.main()
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:414 +0xf54

goroutine 5 [syscall]:
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:21 +0x1c
created by os/signal.init·1
        /usr/local/go/src/os/signal/signal_unix.go:27 +0x40

goroutine 15 [chan receive]:
main.func·003()
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:187 +0x60
created by main.CrawlROMs
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:184 +0x938

goroutine 14 [chan receive]:
main.func·002()
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:177 +0x94
created by main.CrawlROMs
        /home/sselph/go/src/github.com/sselph/scraper/scraper.go:180 +0x6b8

goroutine 10 [select]:
net.func·019()
        /usr/local/go/src/net/dnsclient_unix.go:241 +0x310

[Floob]

[quote=101375]Hmm those errors are from the mame scraper trying to parse the result of getting the URL and getting a response it can’t parse. Since it happens with different roms and in bursts might be some throttling or issues with the website.

[/quote]

Just tried it again, and its fine now. Must have been a temporary bottleneck like you said.

[Floob]

Could a backupdb query work like this?

http://www.progettoemma.net/gioco.php?game=wyvernf0

with the image being:
http://www.progettoemma.net/snap/wyvernf0/0000.png

Just a thought. I’m more than impressed with what it does already!

[Floob]

[quote=101371]It uses mamedb.com. It strips off the file extension and pulls the url http://www.mamedb.com/game/wyvernf0

[/quote]

Ah – ok, that explains it. Thanks.

[Floob]

Also, when processing mame4all roms I seem to periodically get these errors

I dont think its rom specific though, as its a consecutive batch, then next scrape they are fine and others complain?

/07/05 01:47:12 INFO: Starting: bosco.zip
2015/07/05 01:47:12 ERR: error processing bosco.zip: ILM Bad HTML
2015/07/05 01:47:12 INFO: Starting: bouldash.zip
2015/07/05 01:47:12 ERR: error processing bouldash.zip: ILM Bad HTML
2015/07/05 01:47:12 INFO: Starting: bouldash.zip
2015/07/05 01:47:12 ERR: error processing bouldash.zip: ILM Bad HTML
2015/07/05 01:47:12 INFO: Starting: bouldash.zip
2015/07/05 01:47:12 ERR: error processing bouldash.zip: ILM Bad HTML
2015/07/05 01:47:12 INFO: Starting: brain.zip
2015/07/05 01:47:13 ERR: error processing brain.zip: ILM Bad HTML
2015/07/05 01:47:13 INFO: Starting: brain.zip
2015/07/05 01:47:13 ERR: error processing brain.zip: ILM Bad HTML
2015/07/05 01:47:13 INFO: Starting: brain.zip
2015/07/05 01:47:13 ERR: error processing brain.zip: ILM Bad HTML
2015/07/05 01:47:13 INFO: Starting: breakers.zip
2015/07/05 01:47:13 ERR: error processing breakers.zip: ILM Bad HTML
2015/07/05 01:47:13 INFO: Starting: breakers.zip
2015/07/05 01:47:13 ERR: error processing breakers.zip: ILM Bad HTML
2015/07/05 01:47:13 INFO: Starting: breakers.zip
2015/07/05 01:47:14 ERR: error processing breakers.zip: ILM Bad HTML
2015/07/05 01:47:14 INFO: Starting: brkthru.zip
2015/07/05 01:47:14 ERR: error processing brkthru.zip: ILM Bad HTML
2015/07/05 01:47:14 INFO: Starting: brkthru.zip
2015/07/05 01:47:14 ERR: error processing brkthru.zip: ILM Bad HTML
2015/07/05 01:47:14 INFO: Starting: brkthru.zip
2015/07/05 01:47:14 ERR: error processing brkthru.zip: ILM Bad HTML
2015/07/05 01:47:14 INFO: Starting: brubber.zip
2015/07/05 01:47:15 ERR: error processing brubber.zip: ILM Bad HTML
2015/07/05 01:47:15 INFO: Starting: brubber.zip

[Floob]

Scrap that – it found it this time – just no image returned.

One it didnt find was wyvernf0.zip
http://www.progettoemma.net/index.php?gioco=wyvernf0&lang=en

[Floob]

Ah I see – thats great. I’ll give it a go.

Can you remind me how the mame lookup works – which database does it check?

For example, I’ve got ddp2.zip which is:
http://www.progettoemma.net/index.php?gioco=ddp2&lang=en

but nothing scraped?

[Floob]

Thanks very much for the update. Its great!
Loving the extra description detail on mame roms.

I’ve added an error I found on your issue list, it may just be me doing something odd though.

I like the PSX support, although as I have very few PSX games, and I dont use a .cue for single track games I’ll probably still use the ad-hoc in built scraper for those.

Thanks again for all the work you put into this, it makes Emulation Station so much nicer to use.

I’ll try to sort a new video for all these updates!

[Floob]

Can you send the 3 files I listed above? I’ll see if its something straight forward.

This output will also help me
“ls /opt/retropie/configs/all/retroarch-joypads/”

[Floob]

Can you let me know which version of RetroPie you are running or how you built it?

[Floob]

@davidgourde

Can you post the following files for me (using attachment or pastebin)

/home/pi/.emulationstation/es_temporaryinput.cfg
/home/pi/.emulationstation/es_input.cfg
/opt/retropie/configs/all/retroarch-joypads/(the only file in here?)

Make sure your retroarch.cfg files are untouched as they were when you built the image.

[Floob]

Some interesting points

– Built-in input mapping –
It is now possible to bind your gamepad from within the program without having to go through the chores of previous versions. Go to Settings -> Input Settings and select ‘Bind All’ to bind each and every button one at a time, or bind the actions individually. If you are using a gamepad which is supported by us, it will be autodetected and autoconfigured to align with our gamepad abstraction, the RetroPad.

– Ability to remap controls –
An often-heard complaint of before was the inability to change the controls. We have made this possible now. Once a game is loaded, go to the ‘Quick Menu’ and select ‘Core Input Remapping Options’. From there you can change the controls per core, and you can also save these changes to a config fil

[Floob]

Ok, here is the makeup of the “mslug.zip” for fba-libretro.
Does yours match this if you open up the zip?

NOTE: The roms starting with 201 need to be in the mslug.zip, and the rest will need to be in your neogeo.zip that should be in the same directory.

<game name="mslug" romof="neogeo">
		<description>Metal Slug - Super Vehicle-001</description>
		<year>1996</year>
		<manufacturer>Nazca</manufacturer>
		<rom name="201-p1.p1" size="2097152" crc="08d8daa5"/>
		<rom name="201-s1.s1" size="131072" crc="2f55958d"/>
		<rom name="201-c1.c1" size="4194304" crc="72813676"/>
		<rom name="201-c2.c2" size="4194304" crc="96f62574"/>
		<rom name="201-c3.c3" size="4194304" crc="5121456a"/>
		<rom name="201-c4.c4" size="4194304" crc="f4ad59a3"/>
		<rom name="201-m1.m1" size="131072" crc="c28b3253"/>
		<rom name="201-v1.v1" size="4194304" crc="23d22ed1"/>
		<rom name="201-v2.v2" size="4194304" crc="472cf9db"/>
		<rom name="asia-s3.rom" merge="asia-s3.rom" size="131072" crc="91b64be3"/>
		<rom name="sp-s2.sp1" merge="sp-s2.sp1" size="131072" crc="9036d879"/>
		<rom name="sp-s.sp1" merge="sp-s.sp1" size="131072" crc="c7f2fa45"/>
		<rom name="sp-u2.sp1" merge="sp-u2.sp1" size="131072" crc="e72943de"/>
		<rom name="sp-e.sp1" merge="sp-e.sp1" size="131072" crc="2723a5b5"/>
		<rom name="vs-bios.rom" merge="vs-bios.rom" size="131072" crc="f0e8f27d"/>
		<rom name="sp-j2.sp1" merge="sp-j2.sp1" size="131072" crc="acede59c"/>
		<rom name="sp1.jipan.1024" merge="sp1.jipan.1024" size="131072" crc="9fb0abe4"/>
		<rom name="sp-45.sp1" merge="sp-45.sp1" size="524288" crc="03cc9f6a"/>
		<rom name="japan-j3.bin" merge="japan-j3.bin" size="131072" crc="dff6d41f"/>
		<rom name="neo-po.bin" merge="neo-po.bin" size="131072" crc="16d0c132"/>
		<rom name="neo-epo.bin" merge="neo-epo.bin" size="131072" crc="d27a71f1"/>
		<rom name="neodebug.bin" merge="neodebug.bin" size="131072" crc="698ebb7d"/>
		<rom name="sp-1v1_3db8c.bin" merge="sp-1v1_3db8c.bin" size="131072" crc="162f0ebe"/>
		<rom name="uni-bios_3_1.rom" merge="uni-bios_3_1.rom" size="131072" crc="0c58093f"/>
		<rom name="uni-bios_3_0.rom" merge="uni-bios_3_0.rom" size="131072" crc="a97c89a9"/>
		<rom name="uni-bios_2_3.rom" merge="uni-bios_2_3.rom" size="131072" crc="27664eb5"/>
		<rom name="uni-bios_2_3o.rom" merge="uni-bios_2_3o.rom" size="131072" crc="601720ae"/>
		<rom name="uni-bios_2_2.rom" merge="uni-bios_2_2.rom" size="131072" crc="2d50996a"/>
		<rom name="uni-bios_2_1.rom" merge="uni-bios_2_1.rom" size="131072" crc="8dabf76b"/>
		<rom name="uni-bios_2_0.rom" merge="uni-bios_2_0.rom" size="131072" crc="0c12c2ad"/>
		<rom name="uni-bios_1_3.rom" merge="uni-bios_1_3.rom" size="131072" crc="b24b44a0"/>
		<rom name="uni-bios_1_2.rom" merge="uni-bios_1_2.rom" size="131072" crc="4fa698e9"/>
		<rom name="uni-bios_1_2o.rom" merge="uni-bios_1_2o.rom" size="131072" crc="e19d3ce9"/>
		<rom name="uni-bios_1_1.rom" merge="uni-bios_1_1.rom" size="131072" crc="5dda0d84"/>
		<rom name="uni-bios_1_0.rom" merge="uni-bios_1_0.rom" size="131072" crc="0ce453a0"/>
		<rom name="neopen.sp1" merge="neopen.sp1" size="131072" crc="cb915e76"/>
		<rom name="sm1.sm1" merge="sm1.sm1" size="131072" crc="94416d67"/>
		<rom name="sfix.sfix" merge="sfix.sfix" size="131072" crc="c2ea0cfd"/>

		<rom name="000-lo.lo" merge="000-lo.lo" size="131072" crc="5a86cff2"/>
	</game>

[Floob]

This may do it
https://github.com/AndrewFromMelbourne/raspi2png
or
https://github.com/info-beamer/tools/tree/master/screenshot

[Floob]

It looks like the romset isnt correct. Which game are you trying?

[Floob]

Which beta version of RetroPie are you using? Did you write an image to the SD, or did you build from binaries?

Does this not work for you?
“cd /home/pi/RetroPie/BIOS/”

[Floob]

I put my bios here:

/home/pi/RetroPie/BIOS/

Details here:
https://github.com/RetroPie/RetroPie-Setup/wiki/Playstation-1

I basically followed these steps as well as copying the BIOS file

[Floob]

The romsets needed for the arcade based emulators are here:

MAME versions and romsets on RetroPie

Dat files found here:
http://smartretro.co.uk/forums/viewtopic.php?f=3&t=68

Guide to create romsets here:
https://github.com/RetroPie/RetroPie-Setup/wiki/Managing-ROMs

[Floob]

Which mame are you using – was it mame4all, or the libretro version. Or advance mame?

[Floob]

Maybe tomorrow unplug your keyboard and boot into ES again – then re-post that temp file as it should catch the teensy settings then.

[Floob]

You could put this file in:
/opt/retropie/configs/all/retroarch-joypads

I think you can call it anything – try “joypadtest.cfg”
It may well need a different driver name

input_device = "2 Serial/Keyboard/Mouse/Joystick"
input_driver = "udev"
input_up_axis = "-1"
input_right_axis = "+0"
input_down_axis = "+1"
input_left_axis = "-0"
input_a_btn = "13"
input_b_btn = "14"
input_x_btn = "15"
input_y_btn = "999"
input_l_btn = "3"
input_r_btn = "4"
input_start_btn = "7"
input_exit_emulator_btn = "7"
input_select_btn = "6"
input_enable_hotkey_btn = "6"

Although this implies you may be better outputting keyboard keys from the Teensy:
https://forums.adafruit.com/viewtopic.php?f=8&t=58901

What you want is yellow text at the bottom of the screen when you start a retroarch based game, indicating it has found the joypad.

Yes, you could be right about the / marks in the name – do you have a file here?
/home/pi/.emulationstation/es_temporaryinput.cfg ?

[Floob]

Ok, now try

“sudo lsusb -D /dev/bus/usb/001/005″

and look for the iProduct value.

And can you screenshot the jstest output as well?

[Floob]

To test the principle lets just try the buttons first.

whats your output when you type lsusb ?

[Floob]

Here is an example file that should be placed here:
/opt/retropie/configs/all/retroarch-joypads

http://pastebin.com/raw.php?i=vYP9xSeF

You can get the input_device value by looking at the “iproduct” value.
use “lsusb -v”
Or for a specific device, like this
“sudo lsusb -D /dev/bus/usb/001/004”

because mine is the 4th device on the usb bus

Anyway… not sure if it will work but you can try.

[Floob]

For lr-fba I get around the issue by using a different hotkey enable (left shoulder in my case)

FBA Retroarch core – coin controls

Maybe others have got around it a different way.

[Floob]

Maybe you pasted in the lines

input_enable_hotkey_btn = “7”
input_exit_emulator_btn = “6”

if you fixed those and added
input_enable_hotkey = alt

it should work – but I dont know why your controls are set in there, these should be done in the auto config files, with overrides in the system version.
A dull explanation of that is here:

[Floob]

Does the jstest output tell you the button mappings?

Is it like this?
http://www.instructables.com/id/SNES-Teensy-USBPS3-Gamepad/

[Floob]

Thanks dragonjarvis. Its really a case of describing as best you can what steps you took to make the problem happen. Especially including how you built the RetroPie installation, i.e. from source, binaries or just using an image.

If others can replicate the issue its much easier to debug.

[Floob]

Theres quite a lot of edits in your file:
/opt/retropie/configs/all/retroarch.cfg

And some corrupted characters at the end as well. I wouldnt edit that file if I were you. Just use the stock one – if you want to tweak controls use the system based retroarch, or the controller auto file.

Check my vids for an overview on how the configs work.

Try this instead.
– Write a clean RetroPie 3 beta 4 image
(Dont update binaries, script, or anything else)
– Boot it up and try just one rom and see how it goes.
To test one rom you dont even need to expand the filesystem.

[Floob]

For RetroPie 3 beta 4, you just use the roms/fba folder – and press x as the game starts to make sure you choose lr-fba (as opposed to pifba).

Sounds like you are sorted now though.
Here is the full list of games that will work with fba-libretro
https://github.com/libretro/fba-libretro/blob/master/svn-current/trunk/gamelist.txt

[Floob]

I’m going to look at mine soon (although mine are MadCatz Arcade SE and TE).
I have to get the internals swapped out (although its fine in my PC if I use a VIA USB chipset).
I’m not sure if the Qanba has the same issue. Does it work fine in your PC?

[Floob]

You can find version info here:

MAME versions and romsets on RetroPie

[Floob]

Can you test this with a MegaDrive rom – or any libretro core really – just not mame or pifba ones that may not be using retroarch,

Can you post as an attachment
/opt/retropie/configs/all/retroarch.cfg
and
/opt/retropie/configs/{systemname}/retroarch.cfg

[Floob]

I dont see why not, I’m pretty sure Picodrive emulates the Mega-CD.

[Floob]

It won’t be read on boot up. I could find that setting when I’m back at a PC, but it’s what the video manager tool should sort for you as well.
When you save settings from RGUI they aren’t saved in a file retroarch will automatically read – for good reason. Post your saved config file and I’ll happily rip out the setting for you tomorrow.

[Author]

Posts