Two goals by Timmy Adamson in the first half appeared to have Crusaders in control but the Crues striker was later sent-off for two yellow cards.
Dean Youle headed Bangor back into contention on 68 minutes with Andy Hall equalising with a superb free-kick.
Sub Ethan Boylan scored two goals in extra-time to put Bangor through.
He ran onto a flick on to shoot past keeper Sean O'Neill in the 99th minute.
Boylan then produced a lovely finish for 4-2, lifting the ball over the stranded O'Neill to make sure of the victory.
Adamson had given Crusaders the lead in the 24th minute after Stephen Cockcroft's cross had caused hesitation between defender Colin Nixon and keeper Paul Cairnduff.
The Crues front man scored again on 36 minutes after Gavin Whyte had set the chance up.
Adamson, who had been booked in the first half for a hand ball, was sent-off when a foul early in the second half resulted in a second yellow card.
Bangor, seven points clear in the Irish League's second division, play the Crues again in the semi-finals of the County Antrim Shield.
Bangor manager Garth Scates: "We were excellent from the start and had a couple of chances in the first half which we did not take.
"We knew if we kept going forward we would get a chance to get into the game.
"The sending-off helped and we went three at the back. What a team performance, they worked tirelessly."
Crusaders manager Stephen Baxter: "The sending-off was stupid. Timmy is not a young lad, he is 32 and should have known better.
"We made one or two changes with Saturday's match against Cliftonville in mind and it has come back to haunt us.
"But there are more important things going on in the world than a game of football so we are not going to lose any sleep over it."

The gaffe was picked up by a number of viewers, among them Sanjeev Bhaskar, who suggested the broadcaster had "misunderstood diversity issues".
ITV News blamed the mistake on "an error in the production process".
Sir Lenny said he was "very thankful" to receive a knighthood from the Queen for services to drama and charity.
The Comic Relief co-founder has often spoken out on the subject of diversity in the media, a topic he was again asked to comment about on Friday.
"We've made progress as far as on-screen representation is concerned," said the 57-year-old following the investiture ceremony at Windsor Castle.
"But there's a lot of work to be done in the furthering of diversity behind the camera."
"ITV News apologises for the error broadcast in the lunchtime news package today regarding Sir Lenny Henry's knighthood at the palace," said an ITV News spokesperson.
"This was the result of an error in the production process in a piece intended to celebrate Sir Lenny's significant achievements in British entertainment."
BBC News has contacted Ainsley Harriott for comment.

Clydesdale Bank's Scottish £5 note was the first fully polymer banknote to enter circulation in Great Britain.
The International Bank Note Society has nominated the fivers, made from a polymer substrate, for the award alongside banknotes from countries including New Zealand and China.
They are the only UK note on the shortlist.
Clydesdale chief executive David Duffy said: "We are proud that our polymer £5 note has been nominated.
"We take our responsibility as an issuer of banknotes seriously."
Two million of the £5 notes are currently in circulation.
The bank introduces around £400m of new notes every year.
A new polymer £10 note, featuring Robert Burns, will start to be made later in 2016 and is expected to enter circulation in 2017.
More than 20 countries around the world have adopted polymer notes.
The Bank of England is to begin issuing them later this year.
In 1999, Northern Bank of Northern Ireland issued a polymer £5 commemorative note celebrating the year 2000.
Championship One leaders Bangor came from two goals down to beat Premiership title challengers Crusaders in the League Cup quarter-finals.
[NEXT_CONCEPT]
ITV has apologised for showing a news item about Sir Lenny Henry receiving a knighthood that briefly featured footage of TV cook Ainsley Harriott.
[NEXT_CONCEPT]
Britain's first fully polymer £5 note is in the running for the International Banknote of the Year award.