Command+TAB (⌘⇥) to switch to most recent window (even on different space)
There is the default shortcut ⌘` (that's a backtick [or ⌘` for QWERTZ keyboard layouts] to toggle focus between application windows.

I don't think it's possible to adjust the behaviour of ⌘⇥ so it meets your criteria.