Connect arduino to COM pin of a NO NC button
Your solution is a good one, provided that all inputs will always be inputs.

The advantage of the push button is that when it's not pushed, the pin is only weakly pulled to ground and the pin can then be used for other tasks, such as programming.

The bounce effect you describe is quite accurate, I do not know of any switches so badly built that they bounce between pins, so the bouncing can be very easily "helped" by adding a capacitor, if and when you find you still notice it. If the input circuitry isn't also performing weak pull-up it's quite possible the capacitance of the wires already takes care of a part of the bouncing. (Because the inputs of the Atmels used in Arduinos, when they are only input consume very little current)