The RSPCA said it believed the adult grey seal swam inland along the River Glaven in Norfolk on New Year's Eve.
It then managed to make its way into a woman's garden in Blakeney, just off a coastal road, through the fence. It was returned to the sea on New Year's Day.
The RSPCA said: "The lady certainly had a shock when she spotted him lazing around in her shrubbery."
The seal was spotted by the holiday home owner in the afternoon.
Animal collection officer Naemi Kilbey said the animal was "in very good condition".
"Despite being well and truly lost, he luckily didn't have any obvious injuries or illness," she said.
"So we loaded him up in the van and took him straight down to the beach at Blakeney to release him.
"It was so lovely to start the new year watching him swim back out to sea where he belongs."
The RSPCA said it was "not uncommon" for seals to venture up rivers away from the sea looking for food.
It said they normally managed to find their way back.
"But it looks like this guy's internal navigation system got a bit frazzled and he ended up a little lost," Ms Kilbey added.
The RSPCA advises members of the public to keep a distance from any seals on land and not to interfere unless necessary.

Hannah Witheridge, from Hemsby in Norfolk, died along with David Miller, 24, from Jersey, in an attack on Koh Tao island in 2014.
Burmese migrants Zaw Lin and Wai Phyo were convicted of the murders and sentenced to death in December.
Norfolk coroner Jacqueline Lake ruled Ms Witheridge's death was caused by "severe head injuries".
More on this and other Norfolk stories
In October 2014, about 200 people gathered inside St Mary's Church in Hemsby for the speech and language therapy student's funeral.
In a statement read to the inquest, Ms Witheridge's mother Susan told how her daughter was thinking of going on holiday to mainland Europe or Australia but made a late decision to go to Thailand.
"None of the family was happy with her going there, but she had made her mind up," said Mrs Witheridge.
She said the death "changed our lives forever".
She added that on being told the news Ms Witheridge's father, Tony, had burst into the kitchen and "collapsed crying and sobbing".
Amnesty International has called for an independent inquiry over allegations Thai police tortured both defendants during the inquiry.
On their conviction in December, Michael Miller, David Miller's brother, urged those who had doubted the accused's guilt to respect the court's decision.

Stephen Probert, 56 and his partner Joanne Winder, 40, both died following a crash on the A470 near Llanrwst., Conwy county, on Easter Sunday 2015.
Lawrence Lougher, 29, of Addlestone, Surrey, was found guilty of causing the deaths of the Colwyn Bay couple by careless driving.
He was jailed for 12 months at Caernarfon Crown Court.
The court heard Lougher failed to check for the overtaking motorcycle when turning his BMW across the road.
Mr Probert died at the scene and Ms Winder died at Ysbyty Gwynedd two days later.
The court previously heard that Lougher was originally told he would not face prosecution, but there had been a review.
As well as the custodial sentence, he was banned from driving for two and a half years.
A seal was found in the back garden of a holiday home, 2km (1.2 miles) away from the sea.
[NEXT_CONCEPT]
A 23-year-old British woman who died in Thailand was unlawfully killed, an inquest has found.
[NEXT_CONCEPT]
A man has been jailed for causing the deaths of a motorcyclist and his passenger.