Dal Babu, a chief superintendent until 2013, said many Muslims did not trust the "Prevent" strategy and many saw it as a form of spying.
Prevent is a multi-agency programme which aims to stop people being drawn into terrorist-related activity.
The Home Office defended the scheme, saying 130,000 people had been trained to identify and prevent extremism.
With an annual budget of Â£40m, the Prevent programme is one of four strands of Contest, the acronym given to the government's multi-pronged counter-terrorism strategy.
The Home Office says there are now Prevent programmes in place in all key sectors, including local government, health, education, prisons, immigration and charities.
But Mr Babu, who retired from the Metropolitan Police two years ago, said cases like those of the three London schoolgirls who have gone to Syria had caught the authorities unaware.
He said because police counter-terrorism units were mainly white, with few Muslim officers, they did not fully understand issues of race, Islam and gender.
He said: "Sadly, Prevent has become a toxic brand and most Muslims are suspicious of what Prevent is doing."
He told BBC Radio 4's Today programme: "Prevent, when it was introduced, was a good idea. It is about engagement of local communities.
"But over the years it has become less and less trusted.
"Cameras were implemented, without the community understanding them, in Muslim areas of Birmingham."
Mr Babu, a highly respected figure in policing circles, helped found the National Association of Muslim Police during a 30-year career in which he became one of the most senior Asian officers in the UK.
He said that organisations who have taken Prevent funding have a considerable credibility gap within the Muslim community.
"A lot of public money is given to organisations that say they can stop extremism, but we don't know if they are successful.
"You need a strategy for dealing with extremism. But how is that strategy measured, has it been successful?
"We should not put Muslims in a separate box when safeguarding young people.
"You need to look at children services. There is also a spectacular lack of diversity in hierarchy of  children services.
"They are all white. I am not aware of a single non-white chair of an independent safeguarding board.
"In diverse areas like Manchester, Birmingham, Thames Valley, London and Yorkshire you need to make sure you have services that reflect the community."
Sir Peter Fahy, Chief Constable of Greater Manchester Police defended the Prevent strategy, saying: "Overall it has been a very successful programme.
"If you ask the Muslim communities whether they have confidence in policing, they have a higher confidence in policing than the mainstream population [does]."
"It's very difficult to measure the success of a programme that's about prevention because obviously the success rate here is that no bombs have gone off."
He said that parents had to take "prime responsibility" for stopping young people being attracted by Islamic State.
He added: "But there is a recognition that the Muslim leaders themselves have allowed too many extremists to take the ground. They are the prime people that need to find a solution to this."
Mizahur Rahman, who underwent a deradicalisation programme after serving a prison term for soliciting to murder, told the BBC that the Prevent programme was never going to work as there was an atmosphere of fear and intimidation.
He told BBC Radio 5 live on Friday: "The Prevent mentors, deradicalisers or whatever you want to call them, are going to be seen as agents of the government, intelligence gatherers.
"There is no way to have a trusting relationship with someone in that position.
"I never felt comfortable at all, despite what they said. I felt if I asked him something I would go to prison."
The Home Office spokesperson defended the programme in a statement: "This Government fundamentally revised the Prevent strategy in 2011 to ensure it challenges terrorist ideology, supports people who are vulnerable to being drawn into terrorism and works with sectors and institutions where there are risks of radicalisation.
"We will soon be coming forward with a new extremism strategy that deals with the whole spectrum of extremism."
A Scottish government spokesman said that its programme, run through the Scottish Preventing Violent Extremism Unit, took a different approach, "based on community engagement and the strength of the relationship between our Muslim communities and our police service".
The government's anti-terror strategy has become "a toxic brand", a Muslim former senior police officer has said.