I have worked out how to fix it in advmame.
You need to ssh into the RetroPie. Edit /opt/retropie/configs/mame-advmame/advmame-0.94.0.rc (or advmame-1.4.rc).
Find the following lines:
input_map[p1_dialx] auto
input_map[p1_dialy] auto
And replace with:
input_map[p1_dialx] joystick[0,1,0]
input_map[p1_dialy] joystick[0,1,1]
Then find:
input_map[p2_dialx] auto
input_map[p2_dialy] auto
And replace with:
input_map[p2_dialx] joystick[1,1,0]
input_map[p2_dialy] joystick[1,1,1]
That should get rid of the spin. Then you should assign keys to the dial in the game, and probably adjust the Analog Controls, particularly the sensitivity.