Watford's Etienne Capoue scored early in the second half, but Freddie Hinds and Bobby Reid responded for City.
Hornets defender Jose Holebas was sent off in the 88th minute before Niclas Eliasson increased the visitors' lead in stoppage time.
Adrian Mariappa scored a 95th-minute consolation as Watford fell to their first defeat under boss Marco Silva.
The Portuguese made six changes to his side after their league win over Bournemouth on Saturday, while City manager Lee Johnson made nine.
The Watford side was still strong with summer signing Will Hughes handed his debut and Troy Deeney making his first start of the season.
After falling behind, City responded excellently through 18-year-old Hinds' superb strike from distance and Reid's goal eight minutes later.
Brazilian Richarlison hit a post for Watford before Holebas received a second yellow card for a trip on Eliasson.
Eliasson netted a breakaway goal to put the game beyond Watford.
Match ends, Watford 2, Bristol City 3.
Second Half ends, Watford 2, Bristol City 3.
Goal!  Watford 2, Bristol City 3. Adrian Mariappa (Watford) header from very close range to the bottom right corner. Assisted by Christian Kabasele with a headed pass following a set piece situation.
Richarlison (Watford) wins a free kick on the right wing.
Foul by Lloyd Kelly (Bristol City).
Goal!  Watford 1, Bristol City 3. Niclas Eliasson (Bristol City) left footed shot from the centre of the box to the bottom left corner. Assisted by Callum O'Dowda.
Corner,  Watford. Conceded by Niclas Eliasson.
Adrian Mariappa (Watford) wins a free kick in the defensive half.
Foul by Callum O'Dowda (Bristol City).
Foul by Tom Cleverley (Watford).
Callum O'Dowda (Bristol City) wins a free kick on the left wing.
Attempt saved. Callum O'Dowda (Bristol City) right footed shot from the centre of the box is saved in the centre of the goal. Assisted by Matty Taylor with a cross.
Second yellow card to José Holebas (Watford) for a bad foul.
Foul by José Holebas (Watford).
Bobby Reid (Bristol City) wins a free kick in the defensive half.
Corner,  Watford. Conceded by Aden Flint.
Richarlison (Watford) wins a free kick in the defensive half.
Foul by Lloyd Kelly (Bristol City).
Attempt missed. Richarlison (Watford) header from the right side of the box is high and wide to the right. Assisted by Isaac Success with a cross.
Foul by José Holebas (Watford).
Niclas Eliasson (Bristol City) wins a free kick on the right wing.
Foul by Isaac Success (Watford).
Korey Smith (Bristol City) wins a free kick in the defensive half.
Delay over. They are ready to continue.
Delay in match Lloyd Kelly (Bristol City) because of an injury.
Foul by Adrian Mariappa (Watford).
Callum O'Dowda (Bristol City) wins a free kick in the defensive half.
Substitution, Watford. Tom Cleverley replaces Etienne Capoue.
Substitution, Bristol City. Matty Taylor replaces Freddy Hinds.
Foul by Sebastian Prödl (Watford).
Aden Flint (Bristol City) wins a free kick in the defensive half.
Corner,  Watford. Conceded by Frank Fielding.
Attempt saved. Andre Gray (Watford) right footed shot from the centre of the box is saved in the centre of the goal. Assisted by Troy Deeney.
Corner,  Bristol City. Conceded by Ben Watson.
Corner,  Bristol City. Conceded by José Holebas.
Attempt saved. Callum O'Dowda (Bristol City) left footed shot from outside the box is saved in the centre of the goal. Assisted by Hördur Bjorgvin Magnusson with a headed pass.
Substitution, Watford. Andre Gray replaces Will Hughes.
Richarlison (Watford) hits the right post with a header from the right side of the box. Assisted by Isaac Success.
Foul by Troy Deeney (Watford).
Aden Flint (Bristol City) wins a free kick in the defensive half.
Championship side Bristol City came from behind to beat Premier League Watford in the EFL Cup second round.