this is the retropie forum so we can only answer questions about that, although retropie uses emulation station as its frontend so i’m not really sure what you mean when you say you’ve tried both.
console games don’t need you to insert coins :) only mame/fba/neogeo(unless in AES mode). every mame/fba/neogeo game should require coins to play unless you have enabled free play in the dip switches. you install a retropie 3.5 image and add a valid game.zip to your arcade folder, and it will ask you to insert coins every time, unless you change something.