How can I rebind Caps lock to delete/backspace?
You may want to try Seil (formerly PCKeyboardHack).
I use it to bind Caps Lock to Right-Ctrl (not plain Ctrl), so that I can bind Caps Lock to Meta in Emacs (while Ctrl remains Ctrl, and other keys are kept to the usual behavior).