Andorra had stunned Chris Coleman's side by taking a sixth-minute lead through Ildefons Lima's penalty, Andorra's first competitive goal since 2010.
Bale headed Wales level 16 minutes later, though that did not spare Coleman the jeers of the furious travelling Welsh fans as the score remained 1-1 at half-time.
Wales searched desperately for a winner, and their reprieve eventually came after 81 minutes when Bale curled in a beautiful free-kick to deny Andorra a first competitive point for nine years.
Although a victory of any sort was welcome, the Welsh performance was distinctly unconvincing as they aim to qualify for the finals stage of a major tournament for the first time since the 1958 World Cup.
There had been optimism leading up to this campaign, with Euro 2016 expanded from 16 to 24 teams for the first time and Wales drawn alongside Bosnia, Belgium, Israel, Cyprus and Andorra in their qualifying group.
Wales could hardly have asked for a kinder opening fixture - a trip to an Andorran side who are ranked 199th in the world, 158 places below them in the standings, and have lost their last 44 competitive games.
But to Wales' horror, the prospect of a first competitive win for Andorra since 2004 seemed soberingly real as the hosts took an unexpected lead.
With Wales unable to clear from a long Andorran throw, Neil Taylor was adjudged to have pulled the shirt of Ivan Lorenzo.
Prompted by Mitja Zganec - the additional assistant referee behind the goal - Slovenian referee Slavko Vincic pointed to the spot and Andorra captain Lima calmly converted by sending Wayne Hennessey the wrong way.
While Wales' travelling contingent of around 1,500 were momentarily silenced, the other half of the 3,300-capacity Estadio Nacional roared in disbelieving rapture.
The lead lasted only 16 minutes, as Bale headed into the bottom corner from a tantalising cross from left-back Ben Davies.
Bale had inevitably been the focus of pre-match attention, with the £85m Real Madrid winger expected to start as centre-forward.
Coleman instead picked the world's most expensive footballer on his favoured right flank, but he showed the goalscoring instinct of a natural striker to nod Wales level.
The visitors enjoyed near total domination of possession, albeit a little unsteadily at times as players familiarised themselves with the erratic bounce of Andorra's controversial 3G pitch.
With the artificial surface's rubber pellets flying up every time the ball hit the ground, Wales made hard work of creating genuine scoring opportunities.
Having been booed off the field at half time, Coleman sought to take the initiative by switching to a 3-5-2 formation at the beginning of the second half.
There was more urgency to the Welsh performance after the restart, and Bale was the catalyst for the change in pace.
He had a free-kick from distance saved by Ferran Pol and a long-range effort which fizzed just wide, while his cross was headed wide by Andy King.
There were times when Wales' desperation to take the lead led to a loss of shape and rhythm, as players struggled to adapt to tactical changes.
Coleman continued to tinker - replacing his only striker Simon Church with midfielder Joe Ledley to leave Bale as the most advanced player in a makeshift attack.
And when the breakthrough eventually came, it was no surprise that Bale was once again the saviour.
After substitute George Williams was fouled 25 yards from goal, Bale whipped the ball brilliantly beyond Pol to spare Coleman and his side a major embarrassment.
In the only other match in Group B on Tuesday, Bosnia lost 2-1 at home to Cyprus.
Match ends, Andorra 1, Wales 2.
Second Half ends, Andorra 1, Wales 2.
Marc Vales González (Andorra) is shown the yellow card for a bad foul.
Gareth Bale (Wales) wins a free kick on the left wing.
Foul by Marc Vales González (Andorra).
Substitution, Wales. Emyr Huws replaces Aaron Ramsey because of an injury.
Foul by George Williams (Wales).
David Maneiro (Andorra) wins a free kick in the defensive half.
Delay over. They are ready to continue.
Delay in match Aaron Ramsey (Wales) because of an injury.
Foul by George Williams (Wales).
Ildefons Lima (Andorra) wins a free kick in the defensive half.
Joe Ledley (Wales) wins a free kick on the left wing.
Oscar Sonejee (Andorra).
James Chester (Wales) wins a free kick in the defensive half.
Foul by Marcio Vieira (Andorra).
Substitution, Andorra. Juli Sanchez Soto replaces Josep Ayala.
Offside, Wales. Aaron Ramsey tries a through ball, but Gareth Bale is caught offside.
Joe Allen (Wales) wins a free kick in the defensive half.
Foul by Gabi Riera (Andorra).
Substitution, Andorra. Oscar Sonejee replaces Cristian Martinez.
Goal!  Andorra 1, Wales 2. Gareth Bale (Wales) from a free kick with a left footed shot to the top right corner.
Gabi Riera (Andorra) is shown the yellow card.
George Williams (Wales) wins a free kick in the attacking half.
Foul by Ildefons Lima (Andorra).
Substitution, Wales. George Williams replaces Andy King.
Marcio Vieira (Andorra) is shown the yellow card for a bad foul.
Joe Allen (Wales) wins a free kick on the left wing.
Foul by Marcio Vieira (Andorra).
Delay over. They are ready to continue.
Delay in match Cristian Martinez (Andorra) because of an injury.
Corner,  Wales. Conceded by Josep Ayala.
Attempt blocked. Gareth Bale (Wales) header from the centre of the box is blocked. Assisted by Ben Davies with a headed pass.
Corner,  Wales. Conceded by Ildefons Lima.
Foul by Joe Allen (Wales).
Cristian Martinez (Andorra) wins a free kick in the defensive half.
David Maneiro (Andorra) is shown the yellow card.
Aaron Ramsey (Wales) wins a free kick on the right wing.
Cristian Martinez (Andorra).
Corner,  Wales. Conceded by David Maneiro.
Gareth Bale's late free-kick saved Wales from embarrassment in Andorra as his two goals made it a winning start to their Euro 2016 qualifying campaign.