What hardware limitations limit the number of stations connected to and access point chip?
Well, how much your hardware can deal with depends on how much the hardware actually does itself, and what it doesn't and has to be done by the host machine.

So, no general answers can be given: at the higher end of commercial APs, you'll typically find pretty capable wifi chips, doing a lot of the low-latency stuff like acknowledging receptions after checking checksums etc, while still having a beefy applications processor attached to do all things crypto and user management, and of course, actual data movement through a high-speed interface (typically, PCIe these days). These in themselves execute relatively complex firmware, and are limited by their respective RAM etc.

You'll also want to model what "limits the number of stations" actually is: number of stations authed for that AP? number of stations authed to that multi-BSSID network? number of stations > 100 kB/s? number of stations &lt; 10 ms RTT to some gateway?