The raiders targeted the Central England Co-operative property on Chesterfield Road in Duckmanton at about 04:00 BST.
The owner of nearby business WB Motors, Ian Womack, said the raiders had torn down the front of the shop.
Mr Womack said: "It looks like they devastated the property to be fair - the whole front has fallen down."
He said: "There's a large piece of machinery - a JCB - was taken from a building site around the corner - and half the shop front has been pulled down by the JCB to take out the cash machine.
"It will be weeks before it is sorted out. It was not just the cash machine that was stolen - the whole front wall was pulled out."
Mr Womack said the Co-op was cordoned off by police when he arrived at work, adding that CCTV images from his forecourt "would probably identify the culprits".
He said an abandoned vehicle found nearby would also provide "some interesting evidence".
A Co-op spokesman said: "While the building is structurally sound, at present we cannot say when we will be able to open the store again for trading."
The culprits fled the scene in a white Vauxhall van and police are appealing for witnesses to contact them.
The cash machine and van were found later by officers near Buttermilk Lane, police said.
A police spokesman said several vehicles were involved, but officers were "still trying to piece together exactly what happened".
Thieves used a JCB to smash into the front of a filling station store during a cash machine raid in Derbyshire.