HM Inspectorate of Probation found delays in getting offenders to start their work, and said too many were let off if they did not attend.
In one case, an offender had completed just 16 hours in the 17 months since he had been sentenced.
The Ministry of Justice said it was acting on the report's recommendations.
Anything from 40 to 300 hours of unpaid work - or "community payback" - may be given for offences like shoplifting, theft, drug offences, or criminal damage.
Some 70,000 offenders were ordered to do unpaid work such as garden maintenance, decorating or litter-picking in 2014.
Inspectors looked at 100 cases and interviewed 86 offenders from different probation divisions for their report.
The report outlined 15 recommendations for improvement.
Some of the problems identified were:
Seven years ago the government, then led by Gordon Brown, placed an order for 10,000 orange bibs.
They were to be worn by offenders doing unpaid work under a re-branded scheme known as "community payback".
It was intended to give the public confidence that community sentences, involving clearing communal paths and gardens, picking up litter and painting over graffiti, were a "tough" alternative to imprisonment.
When the coalition came to power in 2010 it promised to make unpaid work more physical and intense, with unemployed offenders required to complete their sentences more quickly.
The difficulty has always been organisation. Getting people who lead chaotic lives - many of them with drink or drug problems - to work on time is a task in itself.
If sanctions are applied, they will end up in prison; if not, the sentence cannot be enforced. As the inspection report reveals, it is a problem the authorities still have not solved.
Inspectors did praise instances of high-quality work, and cases where offenders believed they had gained skills from using good tools and equipment.
They also highlighted that many offenders viewed their sentence positively and "were determined to... desist from future offending".
Among its recommendations were that community rehabilitation companies should reduce how often offenders are turned away from work when they have reported on time, and to try to create objectives for unpaid work that "match the circumstances of the offender".
The chief inspector of probation, Paul Wilson, said it appeared in most cases that unpaid work was viewed simply as a punishment, which was a "wasted opportunity".
"Although we found some high-quality management and delivery, much of it was simply not good enough, lacking in focus on the basic requirement to deliver and enforce the sentence of the court."
He called for "urgent remedial action".
A Ministry of Justice spokesperson said it wanted "much more effective rehabilitation" - which had to start with making sure offenders turned up on time and faced consequences if they did not.
"It's totally unacceptable for offenders to skip work enforced by the courts.
"We accept all the recommendations for the Ministry of Justice in this report and are already taking action to implement them."
The way that offenders sentenced to unpaid community work in England and Wales are managed and supervised has been criticised by inspectors.