Jonny May's first-minute try gave Gloucester the ideal start and Matt Kvesic added another for the home side.
But Christian Wade, Tom Varndell and Nathan Hughes all crossed to earn Wasps a 24-14 half-time lead.
Greig Laidlaw gave Gloucester hope with three second-half penalties, but Andy Goode kicked 12 points to secure Wasps' first away league win of the season.
A week on from breaking the Premiership's all-time individual points-scoring record against London Irish in Wasps' home Premiership debut at the Ricoh Arena, Coventry-born Goode converted all three Wasps tries.
He also added a penalty in either half, while centre Elliot Daly kicked a late three-pointer to stretch his side's winning margin, earn them their first Premiership victory on the road this campaign and lift them up a place to sixth in the table.
Laidlaw's 13 points with the boot, having also converted both his side's first-half tries, earned the Cherry and Whites a losing bonus point.
From the first line-out 45 metres out from the visitors' line, Gloucester moved the ball swiftly through the hands to allow May to force his way over, Laidlaw converting expertly.
But, eight minutes later, Wasps were level with an excellent try of their own.
Skipper James Haskell made a strong burst before the ball was moved wide to winger Varndell, who proved too strong in a one-on-one with Steve McColl before feeding Wade to score.
Media playback is not supported on this device
Goode converted then added a penalty to put Wasps 10-7 in front and their lead was extended when the former England number 10's speculative kick bounced kindly for Varndell to score, the visitors' fly-half again converting.
Flanker Kvesic crashed over for Gloucester's second try, which Laidlaw again converted, but the last of the game's five touchdowns went to Wasps three minutes from half-time when Hughes finished off a line-out drive, Goode converting for a 24-14 interval lead.
After replacing McColl at half-time with Billy Burns, Gloucester had the better of the second half, reducing the deficit to just a point with three Laidlaw penalties.
But Wasps brought on Jake Cooper-Woolley and Bradley Davies to shore up their pack, immediately exerting pressure to gain another penalty, which the reliable Goode kicked.
With five minutes remaining, Daly's long-range penalty completed the scoring to confirm a second successive home defeat for Gloucester following last week's heavy loss to Bath.
Gloucester director of rugby David Humphreys:
Media playback is not supported on this device
"We wanted to make Kingsholm a difficult place to play but it's not happening. We were flat and lacked edge in the first half but we had strong words at half-time and were much better afterwards.
"Kingsholm is a great place to play - with our second sell-out crowd - but we are not where we want to be. We know we have to get better and another home defeat will be a worry but if we show the energy that we did in the second half we will win matches."
"This was an important group of four matches. We are two down and it's not getting any easier. Going down to Exeter will be a huge challenge for us."
Wasps director of rugby Dai Young:
"I'm really pleased to get this monkey off our back as you have to win on the road and beat the sides below you in the table if you want to stay in the mix. We put in a poor performance at Exeter but apart from that we haven't been a million miles away.
"We were in control for the first 37 minutes but I was really disappointed with our second-half performance. We knew Gloucester would come back at us but we turned the ball over seven times at first phases.
"We found a way to win as we were composed in the last five minutes, which was pleasing as we've thrown away a lot of games at that stage in the past."
Gloucester: McColl; Sharples, Atkinson, Twelvetrees (capt), May; Hook, Laidlaw; Wood, Hibbard, Afoa; Savage, Palmer; Kalamafoni, Kvesic, G Evans.
Replacements: Dawidiuk, Y Thomas, Knight, Hudson, Moriarty, Robson, Burns, Purdy.
Wasps: Masi; Wade, Daly, Jacobs, Varndell; Goode, Simpson; Mullan, Shervington, Cittadini; Gaskell, Myall; Johnson; Haskell (capt), Hughes.
Replacements: Festuccia, McIntyre, Cooper-Woolley, B Davies, Jones, C Davies, Lozowski, Miller.
Referee: Dean Richards (RFU).
Wasps scored three first-half tries at Kingsholm to set up their second successive win over the festive period.