Deleting from the outliner
It's missing from the Keymap, but you can add it manually. Go to Preferences>>Keymap>>Outliner. Click on Add New then type in outliner.id_delete for the identifier for the new item. Assign the Delete key or what you want and choose Any or Release for the press method, because for some reason it doesn't work with "Press". Now you will delete the item which below of the mouse cursor, not the selected.