How can I create a backup of my iMac when it won't start and recovery mode fails
Have you tried starting your iMac in Target Disk Mode? 

To do this, switch off your iMac and then boot it up again making sure to hold the T key down until you see an icon appear on the screen.

If that works, then you may be able to use this mode to access your Hard Drive from another Mac (or even a Windows PC). However, if using a Windows computer it has to have the right connections and also be able to read a Mac's drive, usually by having extra drivers such as MacDrive, TransMac, MacDisk, or HFSExplorer installed.

Speaking of connections, you didn't specify what model of iMac you have. Typically you would use an appropriate cable connecting the two computers via your FireWire or Thunderbolt port. USB is not supported.

Summary 

In a nutshell, if you can get your iMac to start in Target Disk Mode, you can connect it to another computer (almost always another Mac) and your iMac's hard drive will then appear as an external drive on the other computer. If so, copy whatever files you can from it to another drive.

I should warn you though that it sounds as if your hard drive has failed, and doing this may not work. Also, even if it does, I wouldn't try reformatting it to continue using it. You really should replace the drive or use an external drive as your bootable drive.

Also, if you can boot from an external drive, another option is to just do that and try copying across your data that way (i.e. From the internal drive onto another drive). This avoids the need to try using Target Disk Mode.