The former army officer promised to be "even more dangerous than when I was in uniform".
He also vowed to turn his tiny West African nation into an "economic superpower" over the next five years.
President Jammeh first seized power in 1994 but was re-elected in December in a widely criticised poll.
"You cannot be in your offices every day doing nothing... and at the end of the day you expect to be paid," he said on a televised address on Wednesday.
"This has to stop. You either do your work or leave or go to jail," the president said.
"I will wipe out almost 82% of those in the workforce in the next five years starting this Friday unless they change their attitudes," he said - without elaborating.
Mr Jammeh also promised "zero tolerance" on corruption and drugs.
The Gambia - a popular destination for foreign tourists - has recently become a key transit point for cocaine trafficked from Latin America.
The president also promised to focus more on the empowerment of women and create more jobs for the youth.
Mr Jammeh - who first came to power in a bloodless coup in 1994 - has been criticised by international rights groups for suppressing any dissent.
On Tuesday, former Information Minister Amadou Scatred Janneh, a US citizen, was sentenced to life in jail for plotting a coup and distributing T-shirts with the slogan "End to Dictatorship Now".
Yahya Jammeh has been sworn in for a fourth term as Gambia's president and promised to "wipe out 82%" of workers, accusing them of being lazy.