Magnetism and my breadboard
Three things.  First, switch bounce, explained here, is a thing.  It's a two part article -- read both parts.  I wish I'd written it.  Second, you appear to be using LEDs as pull-ups, with no current limiting on them when you push a switch -- I'm surprised you're not burning up LEDs and doing bad things to your power supply at the same time.  Third, I don't see a ground connection on your chip.