UKIP's leader told the Sunday Telegraph the risk of attacks like those in Cologne, Germany, would be "the nuclear bomb" of the referendum campaign.
But Treasury minister Andrea Leadsom, who backs Brexit, said he was wrong.
"Obviously that is just an outrageous thing to say," she told the BBC.
Allegations that hundreds of women were groped, robbed and intimidated at Cologne's central station on New Year's Eve emerged in January.
Victims described chaos outside the city's main station, as the men - some of whom were identified in an official report as migrants or asylum seekers of North African and Arab origin - carried out dozens of attacks with little apparent response from the authorities.
Mr Farage claimed women could be at risk because of "very big cultural" issues between British society and migrants if the UK voted to remain in the EU, according to the Sunday Telegraph.
Asked whether mass attacks on the scale seen in Cologne could happen in Britain, he replied: "It depends if they get EU passports. It depends if we vote for Brexit or not. It is an issue."
The UKIP leader made the comments ahead of a live TV debate on ITV on Thursday. During last year's TV general election debates, Mr Farage sparked controversy by claiming foreign HIV patients were costing the NHS Â£25,000 each per year.
"The nuclear bomb this time would be about Cologne," he told the Telegraph.
This issue covers immigration and free movement within Europe.
Rather than backing her fellow Brexit campaigner's comments, Tory MP Ms Leadsom told Pienaar's Politics on BBC Radio 5 Live: "I don't like that sort of campaigning at all."
"However we know for a fact that there was an appalling experience for women over in Germany over the Christmas period.
"I do not approve of that sort of campaigning, I do not believe in outright blatant scaremongering so I think it's really, really regrettable.
"I haven't seen it and I wouldn't support suggesting if you vote to remain you'd be raped."
Justice Secretary Michael Gove, a prominent Brexit campaigner, also refused to associate himself with Mr Farage's comments.
"He's made those remarks and I haven't made remarks like that and I won't make remarks like that," he told ITV's Peston on Sunday.
Sal Brinton, president of the Liberal Democrats, said Farage had "sunk to new depths in his scaremongering with these remarks which are completely unacceptable".
And Ryan Coetzee, director of strategy for the official remain campaign, Britain Stronger In Europe, tweeted to say Mr Farage's comments were "disgraceful", adding: "Don't vote for this man's idea of Britain.
But Mr Farage also told the Telegraph the "honesty and straightforwardness" of the prime minister was now being questioned by voters.
Many believed Mr Cameron, who he dubbed "Dishonest Dave", had broken his promise to cut migration and renegotiate the UK's terms of membership of the EU, he said.
Mr Cameron's ally, Chancellor George Osborne, took aim at the UKIP leader in an interview published in the Sunday Times.
"This is a battle between Farage's mean vision of Britain and the outward-facing, generous Britain that the mainstream of this country celebrates," he said.
"I say: we don't want Farage's Britain. That means voting to remain."
Meanwhile former PM Sir John Major hit out at the "squalid" and "deceitful" campaign to get Britain out of the EU.
He told Andrew Marr he was "angry about the way the British people are being misled" by fellow Conservative Boris Johnson and Vote Leave.
Mr Johnson stood by Vote Leave's figures and called for an end to "blue-on-blue" conflict.
Nigel Farage has been accused of "blatant scaremongering" by a fellow Leave campaigner for suggesting there could be a risk of sex attacks on women by migrants if Britain stays in the EU.