X.Org vs. XQuartz - MacPorts
In general you are confusing several things.

XQuartz is a set of libraries to allow X11 applications to be compiled and run on OSX. It is based on the X11 sources from X.org the changes were originally done by Apple.

Quartz is the set of technologies that are in the OSX Core Graphics that deal with 2d from wikipedia 


  Quartz is often synonymous with Core Graphics


Thus Quartz is so called as it provides the X11 libraries that work by calling Quartz provided APIs

In the macports variants.conf what you are saying is don't use X11 (i.e. XQuartz) but use Apple's graphics directly and not via X11 so not calling any X.org code