Nathan Smith's overhead kick hit the bar and bounced in off luckless keeper Dean Snedker's back to give the Gulls an early lead before Nathan Blissett's towering header doubled the advantage.
Kelvin Langmead then headed in before Arthur Gnahoua shot home the equaliser.
But Smith poked in an 82nd-minute winner after a goalmouth scramble to earn Torquay's fourth successive win.
It leaves the Gulls four points from safety with a game in hand, but Colin Gordon's Harriers are now 11 points off 20th-placed Boreham Wood with nine games left.
Former Harriers striker Blissett's goal was his fourth in six games since arriving on loan from Bristol Rovers.
Torquay United, 22nd in the National League, edged a thrilling relegation battle with bottom club Kidderminster.