7490 Counting But a Little bit Buggy
Looks like you have some kind of wiring error, or perhaps a floating input if you are using CMOS parts. Assuming you're using a 74x47 decoder-driver. 

The 7447 family are not fully decoded so if you feed a number greater than 0x1001 to them they will show something that may look odd. 



Edit: If the wiring is fine then you most likely have a power issue. Make sure you have a 0.1uF ceramic bypass capacitor very near the 7490 power and ground pins. Try reducing the display current by 10:1 and see if the issue goes away (you can put a series resistor in the anode just for a test- the segments will be uneven but it's easier than changing all 7 resistors). 

Bad debouncing should not cause this issue because those counts should never come up if the counter is working properly. If you can stop it when it is displaying an invalid digit, test the inputs with a meter and see what they read. 

Edit2: You seem to have no resistors in the segment lines. This is very bad and can damage your display or driver chip. It also can upset the supply causing your problem. Try about 500 ohms in each line (7 resistors)- eg. 470 ohms or 560 ohms. Also you MUST have a 0.1uF or higher ceramic capacitor from Vdd to GND and it should be near the 7490 (a couple cm at most). And another across the 555 power. And tie unused inputs to GND or to +5 through 1K (74xx) or directly (all other types such as 74LSxx).