A Saudi-led coalition of nine countries has been heavily criticised over the civilian death toll from a bombing campaign in Yemen.
The parliament called on EU foreign policy chief Federica Mogherini to "launch an initiative aimed at imposing an EU arms embargo".
The motion passed by 449 votes to 36.
There were 78 abstentions.
Although the vote is not legally binding, lawmakers hope it will pressure EU governments to agree to an embargo, following a petition of 750,000 European citizens.
Scottish MEP Alyn Smith, who led the motion, said EU-made weapons were being exported to Saudi Arabia "in breach of international law".
Mr Smyth said: "Having grown up in Saudi Arabia I am sensitive to the realities of the Saudis and appreciate that the Saudis have concerns in their neighbourhood.
"But our duty is to the civilians in Yemen, and given widespread and very valid concerns over the conduct of the war by Saudi forces, our call for an EU-wide arms embargo is proportionate and necessary."
Anna MacDonald, director of campaign group Control Arms, said the group welcomed "this first step towards preventing European arms from being used to further human suffering in Yemen and elsewhere".
Nearly 6,000 people have been killed since the Saudi-led coalition entered the conflict in March 2015, almost half of them civilians, according to the United Nations.
The US is the largest international supplier of arms to Saudi Arabia. The UK and France are the main European suppliers, while Germany has also licensed arms exports to the kingdom.
Correction 26 February 2016:  This report has been amended to change incorrect figures given for the European Parliament vote.
The European Parliament has adopted a resolution calling for an arms embargo on Saudi Arabia over alleged breaches of international law in Yemen.