What is the reference point for AGL?
FAR 91.159 speaks for itself.  The reference point is directly below your current location.

There are other regulations that use "AGL" (sometimes stated as "Above the surface") as a reference (see FAR 91.155 or 91.119).

As a side note, FAR 91.159 is used to mitigate the risk of midair collisions but not at the expense of prohibiting necessary close to the surface operations (e.g., helicopter EMS, telephone line patrol, crop dusting).