The raiders drove into the Metro Bank branch at Babbage Gate, Oakgrove in Milton Keynes at 04:20 BST, police said.
The cash machine was put in a dark 4x4, and the thieves drove off in that vehicle and a white VW Golf.
Police are appealing for anyone with mobile phone footage of the raid to get in touch.
Ray Cardy, who walked past the bank shortly after the raid, said: "The dumper truck had been driven through the front doors, turned round and smashed the ATM from behind.
"The remains of it were on the footpath - the dumper's engine was still running."
A Metro Bank spokeswoman declined to reveal how much money had been taken.
Four balaclava-clad men smashed a dumper truck into the front of a bank and escaped with its cash machine.