Canadian officials said an attack had been planned with support from al-Qaeda elements in Iran, although there was no evidence of state sponsorship.
Chiheb Esseghaier, 30, and Raed Jaser, 35, are due to appear in court on Tuesday for a bail hearing.
Iranian Foreign Minister Ali Akbar Salehi said to suggest a link between al-Qaeda and Iran was "ridiculous".
"This is the most hilarious thing I've heard in my 64 years," Mr Salehi told the Iranian Isna news agency.
"It is really ridiculous to link al-Qaeda to Iran. I hope that the Canadian authorities think a bit more rationally and pay attention to the consciousness of the people and world public opinion."
Al-Qaeda - a militant Salafist Islamic movement - preaches a radical anti-Shia ideology that places it firmly at odds with Shia Iran.
However, analysts say that despite this enmity, al-Qaeda and Iran have tolerated one another where it suits them.
Last September Canada severed diplomatic ties with Iran, closing its embassy in Tehran and expelling all remaining Iranian diplomats from Canada.
At the time Foreign Minister John Baird said Canada viewed Iran "as the most significant threat to global peace and security in the world today".
Canadian authorities said the two suspects were arrested in Montreal and Toronto on Monday. They are not Canadian citizens but their nationalities have not yet been officially confirmed.
According to Canadian media reports, Mr Esseghaier is a Tunisian national while Mr Jaser is thought to be a Palestinian with citizenship in the United Arab Emirates.
Canada's Globe and Mail reported that the pair had been under investigation since last year following a tip-off by a concerned imam in the Toronto Muslim community.
The imam was worried that young people in his community were being corrupted by an extremist, the report said.
It is alleged that the two had planned to derail a passenger train in the greater Toronto area, but it is not clear when.
The Royal Canadian Mounted Police (RCMP) said the surveillance operation leading to the arrests was "a result of extensive collaborative efforts".
It said FBI agents from the US were involved in helping to foil the attack, although a US justice department official said there was no connection between the plot and last week's Boston Marathon bombings.
The RCMP said the two men, who were not Canadian citizens, had planned to derail a train operated by VIA Rail and "kill and hurt people".
RCMP Chief Superintendent Jennifer Strachan said the attack was "definitely in the planning stage but not imminent".
"We are alleging that these two individuals took steps and conducted activities to initiate a terrorist attack," she said.
"They watched trains and railways in the Greater Toronto area."
Chief Supt Strachan did not say if the route being targeted was a cross-border route with the US.
However, the Globe and Mail, citing sources, said it involved a Toronto-New York City train, and New York Republican Representative Peter King said the attack was intended "to cause significant loss of human life including New Yorkers".
VIA Rail, which operates passenger rail services across Canada, said that "at no time" were passengers or members of the public in imminent danger.
Canadian Public Safety Minister Vic Toews said the arrests showed that terrorism continued to be a real threat to Canada.
"Canada will not tolerate terrorist activity and we will not be used as a safe haven for terrorists or those who support terrorist activity," he said in parliament.
Analysts say Iran's links with al-Qaeda are shadowy and complex.
Some of the group's senior figures - including Osama Bin Laden's son, Saad Bin Laden, and former security chief Saif al-Adel - are said to have fled to Iran after the US-led invasion of Afghanistan in 2001.
They were allegedly held under house arrest by Iran's Islamic Revolutionary Guard, although Tehran never acknowledged their presence.
According to the US, Saif al-Adel's father-in-law, Mustafa Hamid, is the link between al-Qaeda and the Iranian government.
After the fall of the Taliban, he is said to have negotiated the safe relocation of several senior al-Qaeda members and their families to Iran. In mid-2003, he was arrested by the Iranian authorities.
Iran has denied any links to two men arrested in Canada on suspicion of planning an attack on a train.