Standard Arduino kit button caps?
You can use a different type of buttons on with your Arduino, there are some that keep the state or just push button.

The most used are the "arcade" style or the push button on a breadboard.

Just remember to have a pull up/down resistor and add some code to avoid debouncing when the button has been pressed.
https://www.arduino.cc/en/Tutorial/Debounce

If you want some more custom, you can use a normal push button and add small piece of wood/plastic/metal into the hole and having low friction you will be able to press the button with a nice look and feel.