After the game Alan Kernaghan resigned as manager of the Glens.
Jamie Gardiner gave the hosts the lead in the first half, with Jordan Campbell adding a second in the 62nd minute.
Ciaran Caldwell pulled a goal back, Stephen Park restored the two-goal cushion and Nacho Novo netted what proved to be a consolation late on.
Annagh lie bottom of their league, having shipped 12 goals in three games.
Glentoran manager Kernaghan had gone into the game under a degree of pressure after a section of Glens supporters had voiced their discontent following a 4-1 league reverse at Coleraine on Saturday.
Gardiner completed a fine breakaway by chipping Elliott Morris for the opener in the 19th minute and the County Armagh side had their illustrious visitors in real trouble when Campbell added the second.
Caldwell's deflected 25-yard effort three minutes later gave the Glens hope, but that optimism proved short-lived when Park made it 3-1, Novo, Glentoran skipper for the evening, pulling one back from six yards out on 85 minutes.
"The first goal was important for us, it gave us a real lift, and hopefully this result will kick-start our season. We fear nobody," Annagh manager Paul Matchet told BBC Radio Ulster's Sportsound programme after the game.
Glens manager Kernaghan argued: "If we had taken the chances we created, we could have won the game by seven or eight. I am frustrated and I feel the fans' frustration."
Elsewhere, holders Cliftonville began their bid for a fifth straight success in the competition with a thumping 11-1 victory over 2011 winners Lisburn Distillery at Solitude.
David McDaid and Stephen Garrett both scored hat-tricks, with Chris Ramsey, Jaimie McGovern, Chris Curran, Jude Winchester and Gary Donnelly also getting their names on the scoresheet.
Media playback is not supported on this device
The Reds' north Belfast rivals Crusaders were comfortable 3-1 winners over Loughgall, while Portadown hammered Donegal Celtic 6-1 at Shamrock Park.
Ballymena United survived an early scare when Chris McFall gave Newington the lead midway through the first half, but Cathair Friel, Allan Jenkins, a Tony Kane penalty and Johnny McMurray replied for David Jeffrey's Braidmen.
James McLaughlin scored twice as Coleraine saw off Limavady United 3-0 in the north-west derby, while Carrick Rangers got the better of Ballyclare Comrades 2-0 in the East Antrim derby.
Declan O'Brien plundered four goals in Glenavon's 5-0 triumph over Tobermore United, while Andrew Waterworth grabbed his first two strikes of the season in Linfield's 5-0 win against Larne.
David Healy's outfit are chasing a first League Cup success since 2008.
Ballinamallard United were 3-0 victors over Dundela, Ards defeated Newry City 3-1  and Dungannon Swifts romped hime 6-2 over Sport and Leisure Swifts.
Annagh United 3   (Gardiner 19, Campbell 62, Park 72)
Glentoran 2   (Caldwell 65, Novo 85)
Ards 3  (McComb pen 64, 90; McAllister 83)
Newry City 1 (Walker pen 70)
Ballyclare Comrades 0
Carrick Rangers 2   (Kelly 57, Sally 90+3)
Crusaders   3   (Caddell 6, Heatley 23, Carvill 84)
Loughgall  1  (Uprichard 65, Copeland s/o 43)
Ballinamallard Utd   3  (Mayse 22, McCartney 61, Laverty 79)
Dundela   0
Dungannon Swifts   6  (Lavery 20, Fitzpatrick 33, McElroy pen 37, Burke 48, Teggart 57, Hazley 90)
S&L Swifts  2   (Johnson pen 78, McKissock 80)
Glenavon    5   (O'Brien 13, 42, 52, 54; Hall 33)
Tobermore Utd   0
Linfield 5   (Waterworth 14, 34; Burns 20, Clarke 44, Fallon 84)
Larne    0
Institute   0   (Scoltock s/o 92)
H&W Welders 2  (AET)  (McKee pen 92, 119; Rainey s/o 84)
Ballymena Utd   (Friel 30, Jenkins 46, Kane pen 50, McMurray 84)
Newington 1  (McFall 22; Burns s/o 77)
Portadown  6  (Henderson 17, Shannon 26, Kirwin 28, Withers pen 52, Ferris 85, Rooney 88)
Donegal Celtic  1  (Conlon 65)
Cliftonville   11  (Ramsey 29, McDaid 39, 40, pen 46; McGovern 60, Garrett 68, 82, 87; Curran 75, Winchester 77, Donnelly 79)
Lisburn Distillery 1   (McSorley 71)
Coleraine    3   (McCauley 10; McLaughlin 64, 82)
Limavady Utd  0
Struggling Championship One team Annagh United caused a major shock by beating Premiership side Glentoran 3-2 in Tuesday's League Cup second round tie.