It meant that Mr Bayrou's centrist party, MoDem, surrendered all three of its cabinet posts within 24 hours.
President Macron came to power with a promise to fight political sleaze.
MoDem, allied to Mr Macron's La République en Marche (LREM) party, is facing an inquiry into claims that it used EU funds to pay party workers.
Defence Minister and ex-MEP Sylvie Goulard was the first MoDem minister to hand in her resignation on Tuesday.
After Mr Bayrou said he was standing down early on Wednesday, it became clear that Marielle de Sarnez, European affairs minister, was also leaving the government to take over as head of MoDem in the National Assembly.
The resignations were not confined to Mr Bayrou's party. Mr Macron's close ally Richard Ferrand stepped down on Monday, amid unrelated allegations he had used insider information to secure a lucrative property deal for his wife while he was head of a mutual health insurance fund.
Mr Ferrand and Ms Goulard have both denied any wrongdoing.
However, the allegations have cast a shadow over the new government. It was François Bayrou who outlined details of a bill to clean up politics.
Three of the figures brought in to replace the four who resigned were women from civil society rather than politics:
The relationship between the outspoken Francois Bayrou and Mr Macron's government has been tense for weeks.
The French papers were full of anonymous sources from the president's party denouncing Mr Bayrou as having an "outsized ego" who "stops [them] from governing". "We can't keep moving forward," one said, "when we're always going back every five minutes to check whether Bayrou has made a mess again."
Mr Bayrou's departure shows how difficult it can be to prevent scandal, even in those with a well-known political past.  With so many new faces entering parliament this month after a rapid selection process, there's a risk those problems could multiply.
The president's party has a clear majority in parliament, even without the support of MoDem.  But having vowed to bridge the left-right divide in French politics, Mr Macron may instead be facing a growing chasm with his centrist allies.
The bill to clean up politics included stopping politicians hiring members of their own family, a ban of up to 10 years for MPs and senators convicted of corruption or fraud, and reform of party financing.
Mr Macron will no doubt be hoping it quells concerns over his fledgling administration, which faces one of its first hurdles on Thursday when it presents controversial new anti-terrorism legislation.
Documents leaked to French newspaper Le Monde have already raised fears amongst civil liberties campaigners it may make France's emergency measures, which give authorities extra powers including the ability to carry out searches at any time, permanent.
Interior Minister Gérard Collomb told French newspaper Le Figaro [in French] the legislation would involve four flagship methods: creating protection areas around potential targets, closing places of worship which incite terrorism, replacing house arrest with personalised measures, and placing a judge in overall charge of searches and the resulting documents.
France's Justice Minister, François Bayrou, resigned on Wednesday hours before President Emmanuel Macron announced his new government line-up.