What's The fundamental difference between interrupts and delay in embedded programming?
Those are two conceptually totally different things. 


"Interrupt function"
This is more commonly called an interrupt handler or Interrupt Service Routine (ISR). The execution is triggered by the reception of an hardware interrupt. What actions are taken in the ISR normally depends on the reason the interrupt was generated (the kind of interrupt source that caused it).
"delay function"
This is typically a function accepting 1 parameter which specifies the delay that the code execution should be blocked for.


Maybe you are confused here, because a delay can also be implemented using a interrupt driven approach. You can implement a delay using a (timer) interrupt. This is often preferred to calling a delay function like 

delay(1000); /// delay for 1000ms


because this would block the main loop for 1 second. 

Using an interrupt based approach, you can set some kind of flag in the interrupt service routine (which fires after a specific delay using a hardware timer/counter). That flag can just be checked in the main loop to determine if the delay expired, effectively not blocking other code from execution.