Is connecting a 12V circuit to +12V and GND the same as connecting it to GND and -12V?
the voltage is the potential difference to ground


Voltage is the potential difference between two nodes and has nothing specifically to do with ground unless one of those nodes happens to be ground.

You can connect a circuit that requires 12 volts between ground and -12 volts for sure but make sure the polarity is right i.e. positive terminal of the target circuit connects to ground and the negative terminal of the target circuit connects to -12 volts.


  In other words, say I had a 12V LED, could I connect the anode to GND
  and the cathode to -12V and still have it work same as when I connect
  the anode to +12V and the cathode to GND?


Yes.