The 37-year-old, the fifth highest Test run-scorer in history, was given a guard of honour by India's fielders in the second Test in Colombo.
He hit three fours before edging Ravichandran Ashwin to gully, leaving Sri Lanka 33-2 in pursuit of 413.
Sangakkara ends his 134-Test career with 12,400 runs at an average of 57.40, with 38 centuries.
The left-hander also played 404 one-day internationals and 56 Twenty20 internationals and also appeared in English county cricket for Warwickshire, Durham and Surrey, who he joined on a two-year contract in January.
After breaking into the Sri Lanka side in 2000 as a wicketkeeper-batsman, he eventually relinquished the wicketkeeping gloves in Test cricket in 2008. He gave the MCC Spirit of Cricket Cowdrey Lecture at Lord's in 2011.
Sri Lanka ended day four at the P Sara Oval on 72-2, with captain Angelo Mathews unbeaten on 23.
England coach Trevor Bayliss, former coach of Sri Lanka, on BBC Test Match Special: "He's one of the legends of the game. He's one of those local heroes that a lot of people in Sri Lanka look up to, and he's been a great role model. He's had so many great innings over a period of time. The poise and time that he's got at the crease has been one of his hallmarks."
Ex-England batsman Geoffrey Boycott on BBC Test Match Special: "Sangakkara has been a beautiful player; a stylish, excellent batsman. Nearly all the great players have footwork and he has it. And I don't think there's any doubt he could carry on for two or three years. But you get tired mentally."
Sri Lanka batsman Kumar Sangakkara made 18 in his final international innings before retirement.