Difference between installing an application on OS X and Windows
Installing Apps
There are several ways to install applications on OS X.

The App Store: Another way to install applications is from the App Store. Apps from the App Store are first downloaded as a .dmg to a system directory, and then installed to the /Applications/ directory.
Apps downloaded from the App Store are sandboxed, which means that they can only read from and write to specific folders. They're also managed a bit better when it comes to the installation and uninstall process.

From the Web/Drag-and-Drop: OS X applications are sometimes distributed as standalone folders with a .app extension or inside of an installer with a .pkg extension. Sometime programs are distributed on a .dmg, which is a disk image containing the .app and perhaps a Readme. Apps downloaded outside of the App Store might not run unless you secondary-click on them to run, and then grant permission. This is a security feature. (It's similar in appearance to the Windows UAC, but it works differently.)
Applications are typically installed into /Applications/ but each user can have their own applications directory inside of their home folder, located at /Users/{user}/Applications/, or ~/Applications/ for short. Apps placed into the personal ~/Applications directory will show up in Launchpad and Spotlight searches.

Package Managers/The Command Line: It's fairly common (especially for developers) to install apps via the command line with homebrew, macports, or even as Ruby gems. These sometimes land in other directories, such as /usr/local/bin/ If you're not sure where a package manager is putting your files, that information is readily available on the internet.


User Data
The closest thing you have to the windows registry on OS X is the notion of a &quot;property list.&quot; Property lists are small files saved by each application, and unlike the windows registry, they aren't centralized.  Property list files end in a .plist extension. (Interestingly enough, unlike Windows, file extensions don't define what a file is on OS X as much as they do on Windows. Finder does rely extensions for information on how to display a file, but the file meta-data is treated with more significance on OS X than on Windows.)
Sometimes they're saved in  /Users/{user}/Library/Preferences/, but they can be saved in a  variety of places, depending on the app and the kind of information being stored. For example, Safari stores your pinned tabs and last session in ~/Library/Safari/LastSession.plist
Uninstalling Apps
I had questions about uninstalling apps when I got my first Mac. The simplest answer is that you can just delete the .app bundle from the /Applications/ directory. As noted above, there may be some plist files left behind.
On older version of OS X, there were third-party uninstaller apps that tried to clean up for you. They are still available, but I don't know how well they work, because newer versions of OS X are stricter about security. (You can try AppZapper if you'd like.)
Apple has a knowledge base article that covers installing and deleting apps from outside the App Store.