Setting "open with" default app forever
Press ⌘I, select an application, and press the Change All button:



Another option is to use duti. Download the installer and run /usr/local/bin/duti ~/.duti after saving a file like this as ~/.duti:

org.videolan.vlc .mkv all
net.sourceforge.skim-app.skim .pdf all
com.apple.TextEdit public.plain-text all
com.macromates.TextMate.preview public.unix-executable all