OS X 10.8.5: Can I move icons around in the dock and not have them stay there when the apps are closed?
You can remove all closed apps from the Dock (even if they are set to 'Keep in Dock') with the following command:

defaults delete com.apple.dock static-only


You can assign a hotkey to run this, or set up a launch agent to run this regularly.