TTL-based computer/video frequency limit
The chips can handle the frequency just fine (read the datasheets!). The big problem with doing a large design on breadboard sockets is signal integrity, caused by:

poor power distribution (high resistance, limited decoupling)
long signal connections
high parasitic capacitance
high levels of crosstalk and ground bounce

All of these effects contribute to having signals taking a long time to settle after any change, leading to a long clock period.