74161N counter isn't counting with clock input
Answer: sanitize your inputs, switches bounce many times (they are springy they can bounce on and off many many times before they come to a resting state). This will do bad things to digital electronics, such as missed counts or even metastability. Use a Schmidt trigger and an RC filter or other circuit (google switch bounce circuit) to get rid of the switch bounce. http://www.labbookpages.co.uk/electronics/debounce.html

Suggestion: 
Get a cheap oscilloscope or logic analyzer on ebay. Would you rather be short 20-50$ or have those hours of your life back. Scopes make it dead simple to troubleshoot problems like this.