ZigBee vs WiFi for wireless communication
ZigBee (and more generally 802.15.4) targets different uses compared to WiFi (control vs data streaming), thus has lower data rate (250 kbps max) but also much lower power consumption (few tens of mW transmitting and receiving).

For sure you can make star networks, but also more complex tree and mesh networks. In fact, meshing is one of the main selling points of ZigBee.

But concerning reconnect times, it depends on how good the reception is. If you use standard power modules, they are limited to 1/100 of the transmission power of 802.11 thus you might need to use high power modules or use routers to relay messages. You may need to tune the frequency of the reconnect attempts since nodes are typically configured to operate in a duty-cycled manner, waiting for a certain time between transmissions in order to reduce power demand.

Latency is generally low, as the collision avoidance scheme is lighter (read "less sophisticate") than the RTS/CTS used in WiFi. But if you have to send large amounts of data, the throughput will limit your responsiveness as well.

If you want to retain IP networking, consider using either 6LoWPAN on bare 802.15.4 or ZigBee IP.

And if you want to stick to Atmel you can also consider replacing the whole microcontroller with a one-chip MCU with transceiver. Link