They will be joined at next year's Olympics by South Africa who defeated Equatorial Guinea in Bata 1-0.
It is the first time that any Zimbabwe team has qualified for a global football tournament.
Rudo Neshamba's 8th minute goal was enough to send Zimbabwe to Brazil.
It's a great achievement for us, I'm so happy I can't even explain how I'm feeling right now
The result meant the tie finished level at 2-2 on aggregate, with Zimbabwe qualifying on the away-goals rule.
It was disappointment for Cameroon who represented Africa at the Women's World Cup in Canada, reaching the second round.
Neshamba (pictured) was again Zimbabwe's match-winner, having scored their crucial away-goal in the first leg in Yaounde.
Her captain, Felistas Muzongondi, said it was a famous day in their footballing history.
"It's a great achievement for us, I'm so happy I can't even explain how I'm feeling right now," Muzongondi told BBC Sport.
"This is our first time in history to beat Cameroon, and we were expecting this because we prepared very well."
Zimbabwe's skipper also admitted it was a very tough match.
"There was so much intensity so we had to put much pressure on them because they have so much more physical fitness than us."
Zimbabwe booked their place in Rio despite severe financial difficulties facing football in their country.
They were only able to continue in this qualifying tournament after Ivory Coast withdrew.
Also on Sunday, South Africa defeated Equatorial Guinea 1-0 in Bata to win the tie by the same score on aggregate after the first leg had ended goal-less.
Jermaine Seoposenwe's second half goal proved crucial for Banyana Banyana and was enough to win them their place in Rio.
The South African Football Association (Safa) President, Dr Danny Jordaan congratulated Banyana Banyana for their achievement.
"Today is a historic day for women football; not only have they qualified for the Rio Olympics but have done so in style - winning away from home in 'hostile' Bata environment," Jordaan said in a statement.
"Few teams survive trips to Equatorial Guinea as Nigeria would attest but coach Vera Pauw plotted one of the greatest wins in South African football history," Jordaan added.
South Africa's progress to the Olympics makes up for their disappointment of missing out on qualification for the Women's World Cup earlier this year.
Zimbabwe's women beat Cameroon 1-0 in the second leg of their final Olympic qualifier in Harare to book a place at Rio 2016.