Banbury's Horton General has been trying "very hard" since April to recruit new obstetricians.
But Victoria Prentis, Banbury MP, said the "contingency" midwife plan, which she is to discuss with health secretary Jeremy Hunt, was "terrifying".
Campaigners staged a rally earlier against the changes.
"Our bottom line in Banbury is we want our maternity unit to be the best it can be," Mrs Prentis said.
Oxford University Hospitals NHS Foundation Trust has told Horton staff the plan could take effect from the end of September if it did not successfully recruit the obstetricians, who are specially-trained maternity doctors.
Paul Brennan, director of clinical services, said the Oxford trust was trying "very hard" to fill the posts, which have been advertised since April.
He described the changes as "contingency arrangements in case recruitment continues to prove unsuccessful".
But Sophie Hammond, who was among campaigners at the rally, said "lives would be put at risk".
"The thought that these key consultants and this access to emergency care and surgery could be taken away from here doesn't bear thinking about," she said.
Mr Brennan said they had worked to make the obstetrician posts "more attractive to applicants".
"This includes enhanced pay and additional training at the John Radcliffe Hospital," he explained.
"We believe we have got a good offer for doctors wanting a role that offers some responsibility and also some variety, and are again re-advertising the posts."
According to the trust, women who do not want to give birth at the unit will be offered alternatives at other hospitals.
A decision is expected in late August.
A hospital's maternity services may become midwife-led because it is struggling to recruit doctors, an NHS trust has admitted.