Chasing 268 in the first Test, Australia were all out for 161, their innings finishing with a record 154 consecutive deliveries without a run.
Australia - the world's top-ranked Test side - had a first-innings lead of 86.
But Kusal Mendis' superb 176 - his first Test century - and 5-54 from Rangana Herath proved decisive.
Australia captain Steve Smith made a gritty 55 before tail-enders Steve O'Keefe and Peter Nevill resisted for 178 balls, scoring four.
The second Test of the three-match series takes place in Galle, starting on Thursday, 4 August.
O'Keefe and Nevill's ninth-wicket stand came at just 0.13 runs an over, before debutant Dhananjaya de Silva had Nevill caught behind. The previous record for a partnership lasting 100 balls or more was 0.64 by South Africa's Hashim Amla and AB de Villiers against India last December.
Australia survived just 15 further deliveries before Herath claimed his ninth wicket of the match to secure Sri Lanka's first win over the Baggy Greens since 1999, and second in 27 attempts.
Sri Lanka beat Australia by 106 runs in Pallekele to claim only their second Test victory over the tourists, who set a Test match record in defeat.