How to choose capacitor for an IC
What you are referring to is called a decoupling capacitor and is used to decouple the IC supply pins from the bus. In other words, it prevents a sensitive IC from being "starved" if another device on the bus turns on quickly and draws significant current, which would drop the bus voltage for a period of time. The capacitor supplies the extra current required to start up the device, as well as to prevent its chip from suffering the effects of a sudden loaded bus. This is generally required for high-speed devices that switch very quickly, as this tends to draw significant current. The capacitor is not necessarily chosen by its capacitance, but by its ESR (equivalent series resistance) and its ESL (equivalent series inductance). Ideally you would determine the speed at which the device would turn on, and pick the capacitor with the lowest ESR/ESL for that speed. The most common decoupling capacitor value is probably 0.1uF but for faster circuits you may require 0.01uF or 0.001uF (again, depending on their ESR and ESL at those speeds). If multiple devices with different speeds exist on the same bus, you may need more than one decoupling capacitor, one for each speed.

99 times out of 100 the datasheets will tell you exactly what value decoupling capacitors to use on which pins, so read the datasheet. This tutorial from Analog Devices is also a great resource.