What are the factors that limit how long an aircraft can stay aloft?
Maintenance is probably the biggest issue. What types of maintenance is needed and the failures that not performing those steps will lead to depends on the design of the aircraft.

For example, on a piston engine used in a small aircraft:

The engine will need oil replaced at some point, and unless a system was created to cycle in new oil in-flight this is probably the first thing that will break down after crew and fuel (as you mentioned). Oil breaking down will lead to early failure of the engine.

Fuel and air filters will get clogged, leading to the engine shutting down.

Which leads to engine wear. Eventually piston rings, ignition systems, etc. will wear out.