Direct port manipulation gives random result: Delay necessary?
As weird as this is going to sound, your code is too fast.

The pin (and the part of the breadboard that it's plugged into) have some capacitance. It takes time for the resistive pullup in the AVR to pull the pin high.

The Arduino pinMode() and digitalRead() functinons are slow enough that, by the time they get around to reading the pin, the pullup has done its job. Your code is accessing registers directly, which is considerably faster -- so it's possible for the pin to still be low when you read it.

The amount of delay you need is pretty minimal, though. 50 microseconds (not milliseconds!) should be sufficient.