Digital Counter signal oscillation
Your issue is called switch bounce -- where the mechanics of the switch can actually open or close multiple times during a cycle. 

Short answer: yes, and RC low pass filter can fix this up, but you will be sacrificing the speed with which the counter can recognize a switch action.

Long answer:  there are better solutions, using RC's, drivers, and sometimes diodes (to make the math work), depending on how picky your needs are.  There are other approaches that replace SPST switches with SPDT switches, and use them to drive SR flip flops.  There are also debounce IC's available.  

If the simple solution isn't good enough, the complicated solutions are called for   

As to what values, try RC on the order of 20-50 msec, to start, and adjust if the behavior isn't right. (i.e., \$0.01\ \mu F \$  and \$20 k\Omega\$)