The remark was directed at a woman in the audience who had criticised Mr Martelly and his administration.
According to Haitian media, Mr Martelly told her to "go get a man and go into the bushes" at the rally in Miragoane.
A presidential adviser said Mr Martelly's words had been distorted.
Mr Martelly, who is in the final year of his five-year-term, is a former singer of Haitian dance music known for his flamboyant style and sometimes colourful language.
The row was triggered by President Martelly's reaction to a woman at a campaign rally in the port city of Miragoane on 29 July.
The woman complained about the failure of the government to provide electricity to her area, prompting Mr Martelly's hostile response, which in turn was greeted by cheers and clapping from the audience.
In response to the incident, Women's Affairs Minister Yves Rose Morquette, Social Affairs Minister Victor Benoit and Secretary of State for Alphabetisation Mozart Clerisson announced on Wednesday that they would step down from their posts.
They all belong to the Fusion of Haitian Social Democrats party, which is part of the governing coalition.
Party leader Edmonde Supplice Beauzile said Mr Martelly's behaviour was "unbecoming" to a head of state.
Ms Supplice Beauzile is running for president in the upcoming presidential election in October, in which Mr Martelly will not be able to stand again.
She said that "by insulting this woman, he has insulted all Haitian women".
"Such behaviour is a shame for the country," she added.
She also urged voters to "punish" candidates for Mr Martelly's party standing in Sunday's legislative election.
Three officials in Haiti's governing coalition announced their resignation on Wednesday in protest at a remark President Michel Martelly made at a campaign rally last week.