She singled out four forces for having no black police officers - but two of those constabularies hit back, saying they had at least one black officer.
Mrs May said the proportion of black and Asian officers was "simply not good enough".
Critics said funding cuts were restricting recruitment of officers.
The home secretary also said that stop and search reforms "must continue".
And she challenged police over the number of female officers, who currently make up 28% of the police workforce - despite representing 51% of the population.
During a speech to the National Black Police Association conference, Mrs May said "diversity profiles" showed no force had a black and minority ethnic representation reflecting its local population.
Mrs May said she hoped the figures would act as a "wake-up call".
She quoted data suggesting four forces - Cheshire, Durham, Dyfed-Powys and North Yorkshire - did not have a single black officer.
The four forces do have officers from other ethnic minorities, according to the most recent Home Office figures, which are based on information officers volunteer themselves.
But both Durham Constabulary and Dyfed-Powys Police said they did have black officers.
Det Con Waheed Mughal, chair of Durham Black and Asian Police Association, said: "I can confirm that the force does have a British Black Caribbean officer who serves in the north of the county as well as at least 19 other black and minority ethnic officers."
Julia Mulligan, police and crime commissioner for North Yorkshire, said the force had needed to recruit BME officers from other areas of the UK because of small numbers within the force's area.
A Cheshire Police spokesman said: "Despite our current efforts, more needs to be done particularly to recruit officers of Afro-Caribbean descent."
Met Assistant Commissioner Helen King also questioned figures being quoted, and said that four chief officers in London were black or minority ethnic. She added that, of London's 32 boroughs, five were lead by BME chief superintendents.
By Danny Shaw, BBC home affairs correspondent
Although a dispute about the precise figures on ethnic diversity may have blunted Theresa May's message, the broad point she is making remains: for police to have the confidence of the public they must be more racially representative of the communities they serve.
There have been improvements since the Macpherson report into the murder of Stephen Lawrence in 1999 branded the Metropolitan Police as "institutionally racist".
However, targets for BME officers set after the report have not been met.
Although the proportion of new BME officers appears to be rising, recruitment is not at a high enough level - because of budget cuts - to make a significant difference.
Sir Bernard Hogan-Howe wants a change to the law which would, in effect, allow forces to impose BME recruitment quotas.
The home secretary is against that and there is little wider appetite for such a fundamental reform.
Watch children give their views on black and ethnic minority police officers
Chief Constable Giles York, from the National Police Chiefs Council, said the rate of recruitment of officers from black and minority ethnic communities was increasing.
But he added: "There's much more to do and, with reduced budgets constraining recruitment, it is difficult to move at the pace we need to."
Steve White, chairman of the Police Federation of England and Wales, also pointed out that it was difficult for forces to recruit officers from ethnic minorities while chief constables were having to make savings.
Jack Dromey, Labour's shadow police minister, said some forces would not be able to recruit until 2019 "at the earliest" because of budget cuts.
He said: "The police must act, but it is extremely difficult for forces to improve diversity and to carry on protecting communities whilst the Tories continue on their mission to slash police funding and hollow out the police service."
Franstine Jones, from the National Black Police Association, said: "I think police forces need to recognise their black talent, because you have got officers who have got the skills, who have got the knowledge, it is just that in the police service they don't get the opportunity to be developed."
This 1975 BBC piece reports on a Metropolitan Police Service campaign to attract more ethnic minority officers. It contains language some may find offensive.
Mrs May's speech in Birmingham also disputed claims that a rise in knife crime had been caused by a reduction in police stop-and-searches, calling it a "knee-jerk reaction on the back of a false link".
Changes were made after figures showed only about 10% of searches had led to an arrest, with black people six times more likely to be stopped than white people.
This later prompted Metropolitan Police Commissioner Sir Bernard Hogan-Howe to say he believed a rise in knife crime in London could be connected to large reductions in stops-and-searches by his officers.
But Mrs May said it was "simply not true that knife crime is rising because the police are no longer stopping and searching those carrying knives".
The public will not have confidence in the police unless there are more black and ethnic minority officers, Home Secretary Theresa May has said.