Strange behaviour from delete key on a non-apple keyboard
That's completely normal. The key called delete on Windows is called forward delete on OS X, even though it's often labeled like delete ⌦. It deletes a character forward in text views. It doesn't delete items in many applications like Finder, presumably because it would be too easy to press accidentally.

Many shortcut recorders don't allow using just forward delete as a shortcut. You can often edit the property lists that store the shortcuts directly though. See How to remap delete and shift+delete keys to delete and permanently delete a file in Mac OS X Lion?.