A judge at the High Court in London ruled in favour of environmental lawyers ClientEarth.
The group called air pollution a "public health crisis" and said the government has failed to tackle it.
The ruling in the judicial review called the government's plan "woefully inadequate".
The Department for Environment, Food and Rural Affairs (Defra) told the BBC that it accepted the court's judgment.
ClientEarth won a separate, Supreme Court ruling against the government in April 2015.
That judgment ordered ministers to come up with a plan to bring down air pollution to within legal limits as soon as possible.
But ClientEarth was dissatisfied with those proposals, and took the government to the High Court in a judicial review.
In the ruling on Wednesday morning, Mr Justice Garnham said the 2015 Air Quality Plan (AQP), devised when Liz Truss MP was Environment Secretary, failed to comply with the Supreme Court judgment and EU directives.
Furthermore, the ruling said the government had erred in law by fixing compliance dates based on over-optimistic modelling of pollution levels.
"It seems to me plain that by the time the plan was introduced the assumptions underlying the Secretary of State's assessment of the extent of likely future non-compliance had already been shown to be markedly optimistic," said Mr Justice Garnham.
He said the Secretary of State "fell into error" by fixing on a projected compliance date of 2020 (and 2025 for London).
In particular, he drew attention to the cost implications of bringing in Clean Air Zones no earlier than 2020.
"The evidence demonstrates clearly that Clean Air Zones, the measure identified in the plan as the primary means of reducing nitrogen dioxide emissions, could be introduced more quickly than 2020," Mr Justice Garnham said.
He continued: "I have also seen a document setting out questions for Defra from HM Treasury dated 18 August 2015; one question asks why the money required 'needs to be spent over two years?'
"The answer given is 'spend does not have to be over two years, nine years is more realistic given that London does not need to be in compliance until 2025'."
A raft of recent studies and reports have linked air pollution to heart disease and lung problems, including asthma.
In February, the Royal Colleges of Physicians and of Paediatrics and Child Health said outdoor air pollution was contributing to some 40,000 early deaths a year in the UK.
ClientEarth chief executive James Thornton commented: "This is an urgent public health crisis over which the Prime Minister must take personal control.
"I challenge Theresa May to take immediate action now to deal with illegal levels of pollution and prevent tens of thousands of additional early deaths in the UK. The High Court has ruled that more urgent action must be taken."
A spokesperson for Defra said its plans had always followed the best available evidence, adding: "Whilst our huge investment in green transport initiatives and plans to introduce Clean Air Zones around the country will help tackle this problem, we accept the court's judgment.
"We will now carefully consider this ruling, and our next steps, in detail."
The mayor of London, Sadiq Khan, said: "Today's High Court ruling brings sharply into focus the scale of the country's air pollution crisis and lays the blame at the door of the Government for its complacency in failing to tackle the problem quickly and credibly.
"In so doing they have let down millions of people the length and breadth of the country."
Follow Paul on Twitter.
Campaigners have won the latest battle in legal action against the UK Government over levels of air pollution.