Roberts' low finish came after team-mate Christian Burgess was cautioned twice inside a minute, firstly after a tussle with Francois Zoko and then for fouling Ryan Dickson.
Zoko had earlier pounced on poor Pompey defending from a corner to put the Glovers ahead at the far post.
Portsmouth stay in the play-off places, with Yeovil still next to bottom.
Portsmouth manager Paul Cook told BBC Radio Solent:
Media playback is not supported on this device
"It's a fantastic point, but never ever at any stage were we playing well in the game or we were in control of the game.
"I thought Yeovil were excellent on the day. It was a difficult game for us, but the reality of it was we never played well and we have dug ourselves a point and at the end of the season I think we will look at that point as a good point.
"We just weren't at it today like we were last week. As a manger it always comes down to me, I accept that, but when you're trying to change a culture of a club it is hard."
Gary Roberts rescued a point for 10-man Portsmouth with an equaliser at relegation-threatened Yeovil Town.