Why did my pushbutton LED circuit work when my wiring was wrong? I had a Coursera Arduino homewhere wherein you create a simple pushbutton-controlled LED circuit.

I used the example "button.ino" script that's builtin to the Arduino IDE, without modification. I'll reproduce it at the end of this question.

Here's what I did (wrong): 

My circuit went from 5V Power to the top pin of a push button. The bottom of the pushbutton was (correctly) wired to Pin 2 on one side and a resistor was used to wire the other bottom pin to the anode of LED. The cathode of the LED was wired to ground. 

The reason I think this is wrong is that I forgot that I was supposed to wire the LED to a pin corresponding to ledPin in the example script. The example script uses Pin 13, which has the builtin LED. I was using an external LED. Yet, the circuit worked perfectly.

Why did it work? Doesn't that mean the 5V power output was getting turned off (I'd guess you call that being set to LOW) when Pin 13 was set to LOW? Why would that be the case?

/*
  Button

 Turns on and off a light emitting diode(LED) connected to digital  
 pin 13, when pressing a pushbutton attached to pin 2. 


 The circuit:
 * LED attached from pin 13 to ground 
 * pushbutton attached to pin 2 from +5V
 * 10K resistor attached to pin 2 from ground

 * Note: on most Arduinos there is already an LED on the board
 attached to pin 13.


 created 2005
 by DojoDave &lt;http://www.0j0.org&gt;
 modified 30 Aug 2011
 by Tom Igoe

 This example code is in the public domain.

 http://www.arduino.cc/en/Tutorial/Button
 */

// constants won't change. They're used here to 
// set pin numbers:
const int buttonPin = 2;     // the number of the pushbutton pin
const int ledPin =  13;      // the number of the LED pin

// variables will change:
int buttonState = 0;         // variable for reading the pushbutton status

void setup() {
  // initialize the LED pin as an output:
  pinMode(ledPin, OUTPUT);      
  // initialize the pushbutton pin as an input:
  pinMode(buttonPin, INPUT);     
}

void loop(){
  // read the state of the pushbutton value:
  buttonState = digitalRead(buttonPin);

  // check if the pushbutton is pressed.
  // if it is, the buttonState is HIGH:
  if (buttonState == HIGH) {     
    // turn LED on:    
    digitalWrite(ledPin, HIGH);  
  } 
  else {
    // turn LED off:
    digitalWrite(ledPin, LOW); 
  }
}
The LED wired to the push button was turned ON by the push button closing the circuit, without any intervention from the micro controller.
That's a pretty simple circuit: a voltage source, a switch, an LED and a resistor, all in series. When you close the circuit the LED turns on. Simple.

The code on your microcontroller is watching the state change on the button and turning ON/OFF the on board LED: it has nothing to do with the LED in your circuit/breadboard.