What are these capacitors doing here?
Those are decoupling capacitors. They filter out ripple voltages.  The reason for 0.1 uF is that they tend to filter out wide range of frequencies  better than in pF or nF range capacitors and 1uF or higher capacitors are larger in size so they are avoided because of that 0.1 uF has usually used.
Finally 0.1uF are cheaper and easily available.