Ms Tiffney was last seen leaving her home in Edinburgh's Dean Village in May 2002.
A murder charge against her son Sean Flynn was found not proven by a jury at the High Court in Perth in 2005.
The remains were found in an area off the A198, near the entrance to Gosford House in Longniddry.
A cyclist called the emergency services when they made the discovery at about 18:30 on Sunday.
There are 40 police officers working on the find.
It is thought Ms Tiffney was murdered and her body disposed of in the Gosford House area.
The bones are still in situ and it is understood it will take a few days to recover them.
Det Supt Pat Campbell, of Police Scotland, said: "The recovery will be a painstaking process.
"We are consulting with a number of experts to ensure that the remains are recovered in a sensitive and careful manner and no evidence that helps us to establish what happened is damaged or missed.
"I understand the anxiety of those who may be waiting for news of a missing loved one, but whilst the identity of the remains is unknown we cannot speculate."
He added: "The death is being treated as unexplained and once the remains are recovered, there will be significant, detailed forensic analysis required to establish the cause of death and whether any crime has been committed."
Police investigating the discovery of human remains in East Lothian have spoken to the family of missing woman Louise Tiffney.