Why are there age limits to becoming a pilot in the armed forces?
Largely for the same reason there is an upper bound on how old you can be to start training as an FAA Air Traffic Controller: Training people is expensive and time-consuming (flight training moreso than most military specialties), and we want to make sure we're going to get a certain number of years where the pilot is operating at peak performance out of that investment.
Combat flying - particularly in fighters - is a young man's game: It is physically demanding, and it's much easier to maintain the kind of physical conditioning required when you're younger.

Pilot contracts are also typically longer than other enlistment contracts (I believe the going term these days is 10 years of active service, after you "earn your wings"): A pilot who enters training at the maximum age (currently 28 for the US Air Force) will be 38-39 upon leaving the service, if they re-enlist for another 9 year contract they will be in their late 40s (when the natural process of aging will likely begin to limit the duties they can perform, particularly for more demanding combat roles).
Conversely if they entered at age 35 they would be 45-46 at the end of their first contract, and may already be limited by the natural effects of aging. They would be well into their 50s if they re-enlist, and while we've had active-duty pilots in their 50s they are by far the exception rather than the rule.