Why do charge pumps use capacitors?
The name gives you the answer : you need to store charge and move that stored charge around. 

Charge storage devices are either batteries (not very practical) or capacitors (practical, cheap and common).

The only alternative I can see is to store charge on an insulator, move the insulator, and reclaim the charge from it elsewhere. A rubber belt makes a suitable insulator for the job. Thus a Van de Graaff Generator (the machine not the band) is a charge pump of sorts.

But capacitors and rectifiers (or even spark gaps used as switches) are a more practical approach, as in the Cockroft-Walton multiplier.