In a speech, Ian Murray said the party could take inspiration from dominant figures such as John Smith, Donald Dewar and Gordon Brown.
However, Mr Murray added that a new generation must now be responsible for safeguarding the party's future.
Party members are set to vote for new leaders at Holyrood and Westminster.
The Shadow Scottish Secretary has given his backing to Scottish leadership contender Kezia Dugdale, ahead of her MSP rival Ken Macintosh.
He is also backing Yvette Cooper to become the next Labour Party leader.
In his speech at Morningside Parish Church in Edinburgh, he said: "Our lowest points can be the catalyst for our greatest victories.
"We will elect a new leader in a matter of weeks and the hard work will begin.
"A fresh team, a fresh approach, a new generation. I'll play my part in whatever way I can.
"A party which takes inspiration from those who went before - Smith, Dewar, McConnell, Brown. All of them made their own unique and lasting contribution to our party and to our country.
"But the Scottish Labour Party can no longer turn to the big beasts. It falls to a new generation to take the Scottish Labour Party forward."
For many years, Scottish Labour figures played a dominant role on the UK political stage and in government.
Gordon Brown, Alistair Darling, John Reid, the list goes on.
But these figures have since stepped down from frontline politics, while the party has fallen out of favour with voters.
Today, Labour supporters gathered at Edinburgh's Morningside Church - where former Labour leader John Smith's funeral was held 21 years ago - not to dwell on the past, but to look forward.
Speaking at the event, Ian Murray - now Scotland's only Labour MP - told them that while the legacy of these "big beasts" should serve to inspire, the party must now turn to a new generation of party talent.
His comments were, in part, a nod to one of the youthful audience members, Kezia Dugdale, who Mr Murray has backed to become Scottish Labour's new leader over Ken Macintosh, an MSP since 1999.
At the same time, he also gave his support to Yvette Cooper for the UK Labour leadership at a time when veteran MP Jeremy Corbyn is seen by some as a favourite to get the gig.
Ian Murray and his Labour colleagues are all too aware of the challenge ahead, and with polls indicating victory for the SNP in the 2016 Scottish Parliament election, the party knows it will have to work extra hard to win back the trust of voters.
Mr Murray added: "I want us to look back 20 years from now, in government in the UK and in Scotland, and be able to say that when the burden of responsibility passed to the next generation we were up to the challenge.
"This will be the work of the new generation. It has to be the work of the new generation because we cannot go back."
Mr Murray was the only Labour candidate to win in Scotland at May's election, when the SNP took 56 of the 59 seats.
Recent polling suggested the SNP is also on course for a landslide in next year's Holyrood election.
Scottish Labour can no longer turn to the "big beasts" as it recovers from the general election defeat, according to the party's only MP in Scotland.