Buttons, LOW vs no connection
There must be a difference between a LOW reading and no connection, correct? 


No, the input circuitry measures a voltage. The exact details are in the datasheet, but how could it know there is nothing connected? There is not some sort of sensor that detects the insertion of a wire into the input. Therefore it is trying to read a voltage all the time.

With nothing connected then stray voltages from nearby circuitry will influence the reading.




  ... and just really finicky when i moved the board ...


Yep, you are influencing the voltage by capacitance from your hand.

You need to have a definite voltage for "no switch pressed" which can be a resistor to ground (eg. 10k) or a resistor to +5V.

I have a tutorial on this.



It sounds like you have wired your switch like this:



Because you don't have a definite voltage there, the input could "randomly" switch between LOW and HIGH.



You could add a pull-down resistor:





Or a pull-up resistor:





Or use the INPUT_PULLUP mode to create a pull-up resistor in the processor hardware. (There is no INPUT_PULLDOWN mode).

Then you will get consistent results.