How Modern Overclocking Works
Do we actually increase the frequency at which the chip operates?


Yes, we do!

Modern CPUs have a unit called PLL - Phase Locked Loop -  which make the multiple GHz out of relatively cheap crystals running at something like 33.3 or 100 MHz. These units are programmable in a wide range of output frequencies. That is used to slow down the core(s) when there is less work to do to save power - or to overclock them. 

You can increase the clock frequency further when the voltage is higher - but at the price of massive additional generated heat. And the silicon will "wear out" faster, as bad things like Electromigration will increase too.