The 1-0 win, courtesy of Nicky Cadden's penalty, kept Livi a point clear of Airdrieonians, who have a game in hand and beat Peterhead 4-2 away.
Queen's Park moved above Albion Rovers into fifth place by beating Brechin.
The Spiders, who won 2-0, are now five points behind third-placed Brechin, with Alloa, who are fourth, inactive.
The Wasps, like Stranraer, East Fife and Stenhousemuir, were not in league action, though Stenny beat Forres Mechanics in the Scottish Cup.
At Cliftonhill, Ross Dunlop fouled Dale Carrick in the box and Cadden converted the spot-kick in the penultimate minute.
There was further misery for the hosts as Alan Reid was sent off in stoppage time following his second booking.
Airdrie raced into a three-goal lead at Balmoor with Andy Ryan's strike, Sean McIntosh's header and Iain Russell's finish all coming within the opening 20 minutes.
Nicky Riley replied for the hosts before half-time and Leighton McIntosh's penalty following Jack Leitch's foul on Riley further reduced Peterhead's arrears.
But Ryan's second effort of the match deep into stoppage time clinched the visitors' win.
Brechin City lost ground on the top two after losing at Hampden.
Ross Millen opened the scoring from the penalty spot after a first-half foul by goalkeeper Graeme Smith on Dario Zanatta and Ryan McGeever headed home a late Sean Burns corner.
Livingston maintained their slender advantage at the top of Scottish League One with a narrow and late victory over Albion Rovers.