Media playback is not supported on this device
Sean Geddes' two goals helped the Conference North side through at the expense of the League One Sky Blues, Cup winners in 1987.
League Two Portsmouth, winners in 2008, left it late to draw 2-2 at home with Conference club Aldershot.
Northern Premier Blyth Spartans beat Conference Altrincham 4-1 to take their place in the next round.
Media playback is not supported on this device
Worcester took full advantage after Coventry goalkeeper Lee Burge was sent off in the first half for lashing out at visiting  striker Daniel Nti.
Geddes scored the resulting penalty and added a second goal 10 minutes into the second half.
Coventry's Reda Johnson missed a penalty before the break and, though he scored late on, Worcester held on.
"We were the better side," Worcester manager Carl Heeley told BBC Sport. "We're nine games unbeaten now, so we're a good footballing side. But to come to a League One club and to beat them on their own patch - it's a brilliant day."
Coventry boss Steven Pressley said: "This defeat ranks as one of the worst in the club's history."
Blyth's Robbie Dale maintained his record of scoring in every round of this season's competition with two against Altrincham, while Danny Maguire also scored twice for team from the seventh tier of English football.
Aldershot were nine minutes away from a famous win over their Hampshire rivals, but Danny Hollands' header earned the former Premier League side a replay.
Bradford City came from behind against non-league opposition, overcoming an early FC Halifax Town goal to win 2-1 thanks to two goals in quick succession early in the second half.
Two deflected goals from Gary McSheffrey were enough to give Scunthorpe United victory at Forest Green Rovers, while League One pair Chesterfield and Colchester put six past Braintree and Gosport Borough respectively.
Maidstone United, of the Ryman Premier Division, held Stevenage of League Two to a 0-0 draw, while a lacklustre encounter between Notts County and Accrington Stanley ended in the same scoreline.
Rob Ramshaw hit a hat-trick as Gateshead eased to a 4-0 win away at Norton United, while Wrexham are also into the second round after a 3-0 victory at home to fellow Conference side Woking.
The second-round draw takes place on Monday at 19:00 GMT. You can watch it live on BBC Two and the BBC Sport website.
Worcester City beat Coventry City 2-1 to produce the biggest shock of Sunday's FA Cup first-round ties.