Up/Down Counter Not Counting Down
The flip flops contain the true count state and it seems that you should be using them to drive your display.

What happens if you disconnect wires to U14 A &amp; B and connect
U14A to U15A pin 5 and
U14B to U2A pin 5.   

ie take the two Q outputs as outputs.