My Arduino keeps telling me the button is being pushed
When using a pull-up mode on an input pin, that means that when nothing is connected, the pin floats up to HIGH. The connected button should have one side connected to your input pin, and the other to GND, so that when you press the button, the input pin is pulled to GND (that is, LOW).

So your comparison to trigger on a button press should be if (buttonState == LOW)