Tom Rogic, who levelled in extra time for Celtic, missed the vital spot-kick, blazing high and wide.
Rangers striker Kenny Miller fired in the opening goal, with Celtic replying through an Erik Sviatchenko header.
A wonderful long-range strike from Barrie McKay had Rangers in front on 96 minutes before Rogic swept home to set up a tense Hampden shootout.
James Tavernier and Nicky Clark failed to convert from the spot for Rangers, while Callum McGregor and captain Scott Brown could not hit the target for Celtic before Rogic's miss made it 5-4 on penalties to the Championship winners.
Minutes before the penalties, a thunderous free-kick from Celtic striker Leigh Griffiths crashed off the crossbar and bounced off goalkeeper Wes Foderingham before slipping narrowly wide.
Foderingham also made a good stop to deny Rogic from close range after the Australian substitute had scored to make it 2-2.
A pulsating, engrossing and always combative Old Firm game ended with several players out on their feet, barely able to move.
The game had been a relentless affair from the outset. The pitch seemed a clutter of players in the opening spell, as play ranged from one end to the other.
Defenders had to rely on their wits and hope that their composure would survive the sense of being under siege.
Celtic's back four were certainly soon aware of the nature of the threat they would face, when McKay - who was lively and impressively dangerous throughout - speared a ball through to Miller, whose shot was well stopped by Craig Gordon.
Rangers began to impose their game plan, passing the ball swiftly and adroitly and taking the set-pieces short, particularly corners.
Yet Celtic could be incisive on the break, and Brown ought to have opened the scoring from the edge of the area, only to tug his shot wide.
Amid the breathlessness, Rangers found good fortune on their side. The breakthrough came when Tavernier delivered a poor corner kick that led to the ball breaking to Andy Halliday. His cross in turn was also mis-hit, but the ball cannoned off Brown into the path of Miller, who finished coolly.
With Halliday and Jason Holt pushing forward in central midfield, Rangers were composed and assertive. The strategy was effective but Celtic could still rely on individuals to be bold and they should have equalised when Griffiths' shot bounced off the post, with the rebound landing at the feet of Patrick Roberts.
With an open goal in front of him, the on-loan winger skewed wide.
The miss would have galled Celtic, but so too would the way Rangers often passed the ball around them, even at close-quarters.
Kieran Tierney, so solid, dependable and with such sound defensive instincts, was a one-man resistance for the Premiership leaders for a period before half-time.
Fiery words would certainly have been spilled inside the Celtic dressing room during the interval. It was effective, since the team was aggressive, intense and irrepressible at the start of the second half.
Celtic won four corners in succession and scored with the last of them, when Roberts' delivery was headed high into the net by Sviatchenko.
Sviatchenko and Nir Bitton both came close with headers before players tired and became more conservative.
No quarter could be given, though, and every aspect of the game was contested.
Celtic certainly believed that a throw-in early in extra should have gone their way. Instead, Rangers worked the ball to the edge of the area, where McKay hammered a shot into the top corner.
For all that the players were drained and sore, the rivals could not be separated and the excellent Tierney made space to cross and Rogic converted low into the corner.
Penalties followed a frantic finish and Halliday, McKay, Nicky Law, Lee Wallace and Gedion Zelalem held their nerve to score and set up the the first final to be contested by two teams outside the top flight.
Match ends, Rangers 2(5), Celtic 2(4).
Penalty Shootout ends, Rangers 2(5), Celtic 2(4).
Penalty missed! Bad penalty by Tomas Rogic (Celtic) left footed shot is high and wide to the left. Tomas Rogic should be disappointed.
Goal!  Rangers 2(5), Celtic 2(4). Nicky Law (Rangers) converts the penalty with a right footed shot to the bottom right corner.
Goal!  Rangers 2(4), Celtic 2(4). Mikael Lustig (Celtic) converts the penalty with a right footed shot to the top left corner.
Goal!  Rangers 2(4), Celtic 2(3). Gedion Zelalem (Rangers) converts the penalty with a right footed shot to the bottom right corner.
Goal!  Rangers 2(3), Celtic 2(3). Leigh Griffiths (Celtic) converts the penalty with a left footed shot to the top left corner.
Goal!  Rangers 2(3), Celtic 2(2). Lee Wallace (Rangers) converts the penalty with a left footed shot to the bottom left corner.
Penalty saved! Scott Brown (Celtic) fails to capitalise on this great opportunity,  right footed shot saved  in the bottom right corner.
Penalty saved! Nicky Clark (Rangers) fails to capitalise on this great opportunity,  right footed shot saved  in the top left corner.
Goal!  Rangers 2(2), Celtic 2(2). Nir Bitton (Celtic) converts the penalty with a right footed shot to the high centre of the goal.
Goal!  Rangers 2(2), Celtic 2(1). Barrie McKay (Rangers) converts the penalty with a right footed shot to the bottom right corner.
Penalty missed! Still  Rangers 2(1), Celtic 2(1). Callum McGregor (Celtic) hits the bar with a left footed shot.
Penalty missed! Bad penalty by James Tavernier (Rangers) right footed shot is too high. James Tavernier should be disappointed.
Goal!  Rangers 2(1), Celtic 2(1). Charlie Mulgrew (Celtic) converts the penalty with a left footed shot to the bottom right corner.
Goal!  Rangers 2(1), Celtic 2. Andy Halliday (Rangers) converts the penalty with a left footed shot to the top left corner.
Penalty Shootout begins Rangers 2, Celtic 2.
Second Half Extra Time ends, Rangers 2, Celtic 2.
Barrie McKay (Rangers) wins a free kick in the defensive half.
Foul by Patrick Roberts (Celtic).
Attempt missed. Erik Sviatchenko (Celtic) header from the centre of the box is close, but misses to the right. Assisted by Patrick Roberts with a cross following a corner.
Corner,  Celtic. Conceded by Wes Foderingham.
Attempt saved. Leigh Griffiths (Celtic) left footed shot from outside the box is saved in the top left corner.
Andy Halliday (Rangers) is shown the yellow card for a bad foul.
Foul by Andy Halliday (Rangers).
Tomas Rogic (Celtic) wins a free kick in the attacking half.
Andy Halliday (Rangers) wins a free kick in the defensive half.
Foul by Tomas Rogic (Celtic).
Corner,  Rangers. Conceded by Kieran Tierney.
Attempt saved. Callum McGregor (Celtic) left footed shot from outside the box is saved in the bottom left corner. Assisted by Tomas Rogic.
Attempt missed. Jason Holt (Rangers) right footed shot from outside the box misses to the right. Assisted by Nicky Law.
Corner,  Celtic. Conceded by Wes Foderingham.
Attempt saved. Tomas Rogic (Celtic) left footed shot from the centre of the box is saved in the centre of the goal. Assisted by Callum McGregor with a through ball.
Attempt missed. Patrick Roberts (Celtic) left footed shot from the centre of the box misses to the left.
Goal!  Rangers 2, Celtic 2. Tomas Rogic (Celtic) left footed shot from the centre of the box to the bottom right corner. Assisted by Kieran Tierney.
Second Half Extra Time begins Rangers 2, Celtic 1.
First Half Extra Time ends, Rangers 2, Celtic 1.
James Tavernier (Rangers) wins a free kick in the defensive half.
Foul by Patrick Roberts (Celtic).
Attempt saved. Erik Sviatchenko (Celtic) header from the centre of the box is saved in the bottom left corner. Assisted by Patrick Roberts with a cross.
Rangers set up a Scottish Cup final with Hibernian after a thrilling tussle with Celtic was settled on penalties.