Media playback is not supported on this device
After an even first 30 minutes, Danny Wright tapped in from James Dayton's free-kick to put the Robins in front.
But Tubbs levelled after the restart, latching onto Maxime Biamou's header and slotting in from 10 yards out.
Robert Dickie headed against the bar for the Robins late on, before Deacon fired in to win the game in added time.
Sutton have won just one of their past eight games in the National League - including a 6-0 home defeat by Dover on Tuesday - but Deacon's late heroics earned them a place in third round for the first time in 22 years.
The U's have twice gone even further in the competition, reaching the fourth round in 1970, and again in 1989 after they famously beat First Division Coventry City in round three.
They will receive £27,000 for getting through the second round, and could earn a lucrative tie with Premier League or Championship opposition when the third round draw takes place on Monday.
Match ends, Sutton United 2, Cheltenham Town 1.
Second Half ends, Sutton United 2, Cheltenham Town 1.
Offside, Cheltenham Town. Robert Dickie tries a through ball, but Daniel Wright is caught offside.
Goal!  Sutton United 2, Cheltenham Town 1. Roarie Deacon (Sutton United) right footed shot from the right side of the box to the bottom left corner. Assisted by Nicky Bailey.
Substitution, Sutton United. Adam May replaces Matt Tubbs.
Daniel Spence (Sutton United) is shown the yellow card for a bad foul.
Foul by Daniel Spence (Sutton United).
James Dayton (Cheltenham Town) wins a free kick on the right wing.
Attempt saved. Roarie Deacon (Sutton United) right footed shot from outside the box is saved in the bottom left corner. Assisted by Simon Downer with a headed pass.
Attempt saved. Daniel O'Shaughnessy (Cheltenham Town) header from the centre of the box is saved in the bottom right corner. Assisted by Kyle Storer.
Foul by Craig Eastmond (Sutton United).
James Dayton (Cheltenham Town) wins a free kick in the defensive half.
Attempt blocked. Harry Pell (Cheltenham Town) right footed shot from outside the box is blocked. Assisted by Kyle Storer.
Nicky Bailey (Sutton United) is shown the yellow card for a bad foul.
Foul by Nicky Bailey (Sutton United).
Billy Waters (Cheltenham Town) wins a free kick in the attacking half.
Delay over. They are ready to continue.
Delay in match Nicky Bailey (Sutton United) because of an injury.
Craig Eastmond (Sutton United) wins a free kick in the defensive half.
Foul by Daniel Wright (Cheltenham Town).
Robert Dickie (Cheltenham Town) hits the bar with a header from the centre of the box. Assisted by James Dayton with a cross following a corner.
Corner,  Cheltenham Town. Conceded by Jamie Collins.
Substitution, Cheltenham Town. Jon Smith replaces Jack Munns.
Ross Worner (Sutton United) wins a free kick in the defensive half.
Foul by Billy Waters (Cheltenham Town).
Delay over. They are ready to continue.
Delay in match Ross Worner (Sutton United) because of an injury.
Foul by Craig Eastmond (Sutton United).
Jack Munns (Cheltenham Town) wins a free kick in the defensive half.
Simon Downer (Sutton United) is shown the yellow card.
Gomis (Sutton United) wins a free kick on the right wing.
Foul by Daniel O'Shaughnessy (Cheltenham Town).
Attempt missed. Roarie Deacon (Sutton United) right footed shot from outside the box is close, but misses to the left. Assisted by Daniel Spence.
Simon Downer (Sutton United) wins a free kick in the attacking half.
Foul by James Dayton (Cheltenham Town).
Foul by Roarie Deacon (Sutton United).
James Dayton (Cheltenham Town) wins a free kick in the defensive half.
Attempt missed. Roarie Deacon (Sutton United) left footed shot from the left side of the box is just a bit too high.
Attempt saved. Roarie Deacon (Sutton United) right footed shot from the left side of the box is saved in the bottom left corner. Assisted by Craig Eastmond.
Foul by Maxime Biamou (Sutton United).
Roarie Deacon's late winner helped non-league Sutton United beat League Two side Cheltenham Town in the FA Cup second round.