Odin and ChainFire on Samsung Galaxy S III
Rooting will add a few files to your system and will most likely remove the startup process that resets your recovery back to stock. The files it will usually add is the su binary, which is what will actually elevate your privileges, busybox, which will give you some basic command prompt commands, and a super user app, (usually either Superuser or SuperSU). Most stock ROMs have a file that is run during the bootup process that will reflash the stock recovery on every boot (for security). Most rooting processes will remove/rename/change this file to prevent your custom recovery from being overwritten.

Most rooting processes have easy ways to unroot. But essentially, you would just remove the files that it added and then flash your stock recovery back on the device. Many toolkits have an easy unroot option.

As to how an app can use root... when an app attempts to use the su binary, Superuser or SuperSU steps in and pops up a dialog asking you for permission to allow this app root access. You can approve or deny it and there is usually a checkbox asking if you want the Superuser/SuperSU app to remember your choice. An app is not able to get elevated privileges without you approving it through the Superuser/SuperSU app.