Darren Jones brought down Jobi McAnuff in the penalty area and O's top-scorer Simpson took his season's tally to 22.
The win means the hosts rise two places to sixth in the League Two table as new player-manager Kevin Nolan enjoyed a second successive victory.
Newport produced a battling performance, but remain 20th after a night of few chances.
Nolan sent in the telling cross towards McAnuff for the game's deciding moment and referee Andy Davies awarded a spot-kick for the challenge by Jones.
Newport manager Warren Feeney told BBC Radio Wales: "That's possibly one of the worst decisions I've seen in the game.
"People say players have bad performances, but you've got to watch what you say about these people (referees).
"I wouldn't mind if he's consistent, but through the whole game, I thought he was poor from the start.
"I'm gutted for the players because I don't think they deserved that."
A contentious Jay Simpson penalty was enough to earn promotion-chasing Leyton Orient victory over Newport County.