Can I connect an Arduino wifi module directly to my android phone? I would like to use my arduino even where there's no wifi router. I've only found people saying that it needs to connect to a router, but isn't the phone like a router when it's in tethering mode? Do I still need a mobile internet connection or can I do it directly?
A phone running in wifi tethering (mobile hotspot) mode is no different to a wifi access point or router. So yes, you can use that.

Chances are though that you will need a mobile internet connection in order to activate mobile hotspot mode in the first place.