Two goals by Timmy Adamson in the first half appeared to have Crusaders in control but the Crues striker was later sent-off for two yellow cards.
Dean Youle headed Bangor back into contention on 68 minutes with Andy Hall equalising with a superb free-kick.
Sub Ethan Boylan scored two goals in extra-time to put Bangor through.
He ran onto a flick on to shoot past keeper Sean O'Neill in the 99th minute.
Boylan then produced a lovely finish for 4-2, lifting the ball over the stranded O'Neill to make sure of the victory.
Adamson had given Crusaders the lead in the 24th minute after Stephen Cockcroft's cross had caused hesitation between defender Colin Nixon and keeper Paul Cairnduff.
The Crues front man scored again on 36 minutes after Gavin Whyte had set the chance up.
Adamson, who had been booked in the first half for a hand ball, was sent-off when a foul early in the second half resulted in a second yellow card.
Bangor, seven points clear in the Irish League's second division, play the Crues again in the semi-finals of the County Antrim Shield.
Bangor manager Garth Scates: "We were excellent from the start and had a couple of chances in the first half which we did not take.
"We knew if we kept going forward we would get a chance to get into the game.
"The sending-off helped and we went three at the back. What a team performance, they worked tirelessly."
Crusaders manager Stephen Baxter: "The sending-off was stupid. Timmy is not a young lad, he is 32 and should have known better.
"We made one or two changes with Saturday's match against Cliftonville in mind and it has come back to haunt us.
"But there are more important things going on in the world than a game of football so we are not going to lose any sleep over it."
Championship One leaders Bangor came from two goals down to beat Premiership title challengers Crusaders in the League Cup quarter-finals.