Do I have to keep the .dmg files on my computer?
A lot of applications (bundles) for OS X don't need an installer in contrary to almost all Windows apps. They are distributed inside an .dmg.

To properly "install" them just open the .dmg and copy the app file to /Applications or /Applications/Utilities. A lot of images already contain a soft link to the Applications folder to be more convenient for the user. 



If the .dmg contains such a soft link just drop the app on it (like indicated by the grey arrow in the screenshot above).

You have to enter an admin password to start the task if you aren't logged in with a user with admin permissions. You always have to enter an admin password if you copy the file into /Applications/Utilities/.

All subsequent installations (like LaunchDaemons or application support files) are done by the app itself after starting it for the first time.

After copying the app to one of the folders mentioned above, you can unmount the "installer" volume and delete the .dmg file.