File browser alternatives with remote connections
You can use sshfs or Transmit to mount SSH locations as local volumes.

↩ renames files in Finder. ⌘↓ or ⌘O opens files, ⌘⌫ moves files to the trash, and ⌥⌘V moves files copied with ⌘C.

You could also take a look at Forklift or Path Finder. Both support making return open files but not making forward delete move files to to the trash.