How does the GPS work on Android phones? Today I used the GPS on my Android for the first time, and I was wondering how it actually works.

I know that Android phones have a GPS chip in them, but does the phone establish a direct connection to the GPS server or is there something else going on? If there is a direct server connection, does that mean that every Android user can make a direct server connection?
GPS itself does not need a network connection to any server to work. It just needs a signal from a least 4 GPS satellites to calculate a good position fix.

There is also the GPS Almanac, which basically provides future data on where the satellites are expected to be in a given time in the future, usually up to 7 days. This can be used to speed up the position calculation. The Almanac is broadcasted by the GPS satellites and it takes up to 15 minutes to download the full Almanac via the GPS system. That's why there are also Servers on the Internet that provide the Almanac data in an more convenient, faster way. Maybe that's what you meant by "GPS Server". Android makes use of that too.