How to fully 'uninstall' an app on OS X
If the app is a driver or some kind of system hack, then you better read it's documentation or contact it's tech support team to find out how to uninstall it.

For most apps, simply quit the app and drag it to the trash.

Most mac apps won't create much extra "junk". It will leave a few configuration files/etc around, but they're usually really small and you want them to be there, incase you ever decide to install the app again.

If you really want to get rid of it all:

Open Finder, and bring up the Go menu. Hold down the Option key to make the Library menu item visible, and click on it.

Type the name of the app into the top right search box (eg: "TextMate"), and when it suggests "Filename contains: textmate" in the menu, click on that. By default, it will search "This Mac", click on "Library" to only search the folder you were in before doing the search.

Make sure everything you see is actually related to the app, and move it to the trash. When a file is selected, the "Path Bar" at the bottom of the window will tell you where the file is, which can help figure out what app it belongs to.

Be careful, modifying anything in a Library folder can screw up your system. Don't do it at all unless you have a backup to restore to incase something goes wrong. It's usually pretty safe though, most files in the Library folder in your user's home folder will be re-created as necessary, if something important has been deleted.

You might also want to search your entire system instead of just ~/Library, but if the app installs anything elsewhere in the system then that's a pretty good sign that you should be checking the documentation for the correct uninstall procedure and/or contacting their tech support, to make sure it is actually safe to delete those files without screwing up your system.

If you are the type of person who regularly installs random apps from the internet and then deletes most of them... then you should either test these apps in a virtual machine or at least using the "Guest" account feature (where pretty much everything is deleted as soon as you log out).