Shift Register 74HC595 not working
Solution: Remove ground from Q0 as Gerben correctly pointed out in comments.

Also: rotate LEDs 90 degrees so that the anode and cathode are on different rows.

What was tripping me up is in the Arduino tutorial diagram is a little unclear on this (I now understand why their way is correct and my way (in the question) was wrong: