What exactly is the APN field used for, and why do some providers require it while others don't? I've switched providers and now have to enter an APN (access point name) in order to use the service. With my older two providers I did not need to do this and it worked out of the box. What is the rationale for requiring APNs to be entered on some cell phones, and exactly what features do they enable?

Why do some providers require them while others don't, and what do they do?
APN settings are necessary for data service on GSM and related networks (UMTS, LTE) to work.  It defines what network to connect to and the configuration of the network (e.g. the gateway GPRS support node address, which is the interface between the mobile network and the public internet).

APN settings are always required for data service on GSM and related networks.  The reason why you didn't need to enter them before would be either that you purchased the device from the carrier, in which case it would have been pre-configured with their APN settings, that the ROM (either stock or custom) included your previous carrier in their APN database, or your phone was able to accept automatic configuration from the carrier.

Most custom ROMs and some stock ones (in the case of direct-from-the-manufacturer devices) include an extensive database of different carrier's APNs and Android can automatically choose the correct one (Based on information from the SIM card) so you don't have to enter stuff manually.