the file you need to look at is retroarch.cfg located in home/pi/RetroPie/configs/all/retroarch.cfg
To determine buttons run:
sudo apt-get install joystick
then
jstest /dev/input/js0
Here is the joypad section of my retroarch.cfg configured for my Xbox pad for example:
input_player1_joypad_index = "0"
input_player1_b_btn = "1"
input_player1_y_btn = "3"
input_player1_select_btn = "8"
input_player1_start_btn = "9"
input_player1_up_axis = "-5"
input_player1_down_axis = "+5"
input_player1_left_axis = "-4"
input_player1_right_axis = "+4"
input_player1_a_btn = "0"
input_player1_x_btn = "2"
input_player1_l_btn = "4"
input_player1_r_btn = "5"
input_player1_l2_btn = "6"
input_player1_r2_btn = "7"
input_player1_l3_btn = "11"
input_player1_r3_btn = "12"
input_player1_l_x_plus_axis = "+0"
input_player1_l_x_minus_axis = "-0"
input_player1_l_y_plus_axis = "+1"
input_player1_l_y_minus_axis = "-1"
input_player1_r_x_plus_axis = "+2"
input_player1_r_x_minus_axis = "-2"
input_player1_r_y_plus_axis = "+3"
input_player1_r_y_minus_axis = "-3"
input_enable_hotkey_btn = "8"
input_exit_emulator_btn = "9"
input_save_state_btn = "5"
input_load_state_btn = "4"
input_state_slot_increase_btn = "7"
input_state_slot_decrease_btn ="6"
input_menu_toggle_btn = "3"
For player 2 controls copy player 1 controls and change ‘joypad_index’ to 1 and ‘input_player1’ to 2 e.g. :
input_player2_joypad_index = "1"
input_player2_b_btn = "1"
input_player2_y_btn = "3"
etc
Here is the skeleton config for retroarch.cfg:
https://github.com/libretro/RetroArch/blob/master/retroarch.cfg