Delay function in Arduino microcontroller
The delay() function does not return until the requsted interval has expired. During that time it constantly reads the clock, watching for the end of the interval. This means that your program can't do anything else during that time.

An alternative is to for your program set a timer that will interrupt whatever the program is doing when the time expires by calling a function to (in this case) toggle the LED state. Meanwhile, once the timer is set, your program can continue on to do something more useful. 

In a blinky program, where blinking an LED is the whole point, there isn't anything else needing doing, but in general, you'll want your programs to be able to maintain a display, react to buttons or a keyboard, and read a few sensors, for example, and appear to do them simultaneously.

Update:

A clock or timer is a hardware register the program can read, which is automatically counted up or down at a fixed rate, such as once every millisecond. This counting is done by the hardware without the program having to do anything. To delay for say, 2 seconds, the delay() function will read the clock register, add 2000 to it (the number of milliseconds of the required delay) to find the ending time, then keep reading the clock register until it equals or exceeds that time.

In the second case, above, a timer register would be set to 2000 (again, the number of milliseconds to delay), but in this case the hardware decrements that register every millisecond, and will interrupt the program when the timer register gets to 0.