Presiding officer Tricia Marwick shed tears after veteran Labour MSP Duncan McNeil bid farewell to the parliament.
And she was again visibly emotional after Mr Salmond told the chamber: "It's goodbye from me... for now".
Ms Marwick, Mr Salmond and Mr McNeil are among those standing down as MSPs ahead of the election on 5 May.
Mr Salmond, who served as first minister between 2007 and 2014, used his valedictory address to repeat a belief he expressed in his very first speech in the parliament that Scotland was "not divided, but diverse".
He said both the country and parliament were on a journey, and added: "Yes we are a country of different views, but we are not divided. There is in fact a broad consensus on the need for this parliament to assume greater responsibility for the governance of Scotland.
"And we are definitely stronger - so much stronger - as a result of that".
He welcomed the transfer of powers to Holyrood under the Scotland Bill, but again insisted the bill did not fulfil the pledges which he said had been made by unionist politicians in the last days of the referendum campaign.
There were lighters moments as Mr Salmond, who was speaking before the Chancellor delivered his UK budget, joked that he was aware that he faced competition from a "major attraction down south today".
He then added: "However, on balance, I feel that the champion chase at Cheltenham racecourse will not be overshadowed by my remarks."
And the former SNP leader joked that he wished everyone standing in the forthcoming election good luck, "albeit with varying degrees of enthusiasm".
He concluded his address by saying: "Let me leave you with these final thoughts - there is no greater honour in public life than to be a member of this parliament.
"There is no greater task than to mould the public purpose of Scotland. There is no greater cause than to serve the people of this country.
"And so with that it is goodbye from me... for now".
Ms Marwick thanked Mr Salmond for his contribution both as an MSP and as first minister, and added that he had "served the parliament and Scotland with distinction".
The presiding officer had earlier been moved to tears by a powerful final speech by Scottish Labour backbencher Duncan McNeil, who is stepping down as an MSP after serving in the parliament since it was established in 1999.
Mr McNeil used his address to warn that Holyrood had things to learn from Westminster, particularly with regard to the committee system.
He said: "It will be this parliament's responsibility to ensure there is accountability, there is scrutiny and even opposition when that is necessary.
"We must ensure we are capable of meeting that challenge, or we will face the consequences."
He recalled how former Labour MP Robin Cook came to Holyrood to look at the lessons that could be learned when it came to reforming Westminster.
But Mr McNeil said: "It saddens me to say we now have to do a bit of learning from them and how they run their business."
Former Scottish Conservative MSP Annabel Goldie is also bowing out of Holyrood.
In her speech, she joked she would miss her dealings with Finance Secretary John Swinney, especially "his huge outbursts of faux indignation".
Former first minister Alex Salmond has made his final speech in the Scottish Parliament amid emotional scenes in the Holyrood chamber.