Sexton suffered a suspected groin injury, before O'Connell was carried off at the end of the first half.
Leading 9-6 at half-time, Rob Kearney's try increased Ireland's lead before O'Mahony had to be taken off.
Conor Murray added a further late try as the gutsy Irish set up a quarter-final with Argentina next Sunday.
Relive the action from the game on BBC Radio 5 live
Topping Pool D means Joe Schmidt's side avoid a last-eight encounter with the world champions but the injuries to three such influential members of the Irish team will be a massive concern for the rest of the campaign.
There will also be fears that the outstanding Sean O'Brien could be facing a citing after he appeared to punch France lock Pascal Pape in the opening minute of the contest.
With O'Connell retiring from international duty after this tournament, there will be fears that his Ireland career could be over after 108 caps.
The French, blunt as an attacking force for the most part, must regroup to face the All Blacks in their quarter-final at the Millennium Stadium next Saturday evening.
Ireland showed remarkable resolve to stand up to a hugely physical France side.
The French, as promised, targeted ex-Racing Metro player Sexton and the Ireland fly-half was forced off in the 25th minute after a heavy tackle by Louis Picamoles, although it looked as though the Irish playmaker injured his groin kicking the ball seconds earlier.
O'Connell's international career may have come to an end after he fell awkwardly as he was cleared out at a ruck in first-half injury-time, while O'Mahony was forced off 14 minutes into the second half with another leg injury.
However, that came five minutes after a break by the magnificent Robbie Henshaw helped set up Kearney's crucial try.
Sexton's replacement Ian Madigan produced a tremendous display after his introduction as he conclusively won his battle with France fly-half Frederic Michalak.
The Irish also managed to out-scrum the French, while the performances of second-half replacements Iain Henderson and Chris Henry demonstrated the depth in Joe Schmidt's squad.
However, that depth may be tested to its limits next weekend against the Pumas.
A huge early tackle from Jamie Heaslip on France full-back Scott Spedding set the tone for a bone-crunching contest.
Media playback is not supported on this device
Michalak and Spedding missed early chances to put France ahead but Sexton did nudge the Irish ahead from 45 metres after their first move into opposition territory.
Two long-range Spedding penalties - either side of another Sexton three points - levelled the contest before the tearful Ireland fly-half was forced off.
The composed Madigan's first action was to restore Ireland's lead in the 28th minute but the Irish suffered second devastating injury blow in first-half injury-time after Pascal Pape clattered into O'Connell at a ruck.
Late in the half, Ireland centre Keith Earls had wasted a glorious try-scoring after to take a routine pass following a searing Tommy Bowe break.
But the determined Irish were not to be denied after the break.
Not for the first time in the contest, Henshaw charged past France's ineffective battering ram centre Mathieu Bastareaud to put Ireland on the offensive and the resultant scrum was finished off by full-back Kearney finish.
After O'Mahony's departure, a penalty from France replacement Morgan Parra cut the Irish lead to 14-9 but the excellent Murray's 72nd-minute try sealed Ireland's remarkable victory after Rory Best's initial drive for the line.
There were many man of the match contenders in the Ireland team but the Tullow Tank produced a titanic display in the most bruising of contests.
The Ireland flanker made 19 carries in the contest and made as many metres in the game as impressive wing Tommy Bowe.
Given his injury woes, Joe Schmidt will be praying that O'Brien avoids a citing for his early indiscretion.
Ireland coach Joe Schmidt: "It felt like a dog fight at times, a real slugfest between the two packs and then it would blow wide open with both sets of backs playing some really expansive rugby with the ball going end to end.
"Over the last couple of years, it's been a one-score game between the two sides. We knew it was going to go right to the wire. You have to be patient, calm and focused and take your opportunity when it comes and the boys did that.
"There are a lot of sore bodies in that changing room. The lads were just physically and emotionally drained at the end of it and from this point on that is what you've got to give when you get the chance to wear that jersey."
France will play New Zealand in the quarter-finals at the Millennium Stadium, Cardiff, on Saturday 17 October, with a 20:00 BST kick-off. Ireland face Argentina at the same venue at 13:00 BST on Sunday 18 October.
Ireland: Rob Kearney; Tommy Bowe, Keith Earls, Robbie Henshaw, Dave Kearney; Johnny Sexton, Conor Murray; Cian Healy, Rory Best, Mike Ross; Devin Toner, Paul O'Connell; Peter O'Mahony, Sean O'Brien, Jamie Heasip.
Replacements: Richardt Strauss for Best (73), Jack McGrath for Healy (57), Nathan White for Ross (65), Iain Henderson for O'Connell (40), Chris Henry for O'Mahony (55), Eoin Reddan for Murray (76), Ian Madigan for Sexton (25), Luke Fitzgerald for Earls (62).
France: Scott Spedding; Noa Nakaitaci, Mathieu Bastareaud, Wesley Fofana, Brice Dulin; Frederic Michalak, Sebastien Tillous-Borde; Eddy Ben Arous, Guilhem Guirado, Rabah Slimani; Pascal Pape, Yoann Maestri; Thierry Dusautior (capt), Damien Chouly, Louis Picamoles.
Replacements: Benjamin Kayser for Guiardo (58), Vincent Debaty for Ben Arous (43), Nicolas Mas for Slimani (63), Alexandre Flanquart for Pape (73), Bernard Le Roux for Chouly (55), Morgan Parra for Tillous-Borde (55), Remi Tales for Michalak (55), Alexandre Dumoulin for Bastareaud (62)
Attendance: 72,163
Referee: Nigel Owens (Wales)
Ireland beat France to avoid a quarter-final encounter with New Zealand, but suffered injuries to Johnny Sexton, Paul O'Connell and Peter O'Mahony.