After a dour first period, James Constable's driven effort gave the out-of-form away side a surprise lead.
The advantage only lasted three minutes though as Parkin fired home from the centre of the area.
With time running out, Parkin scrambled home the winner before the visitors' Ben Strevens was sent off in injury time for spitting.
Forest Green assistant Jamie Day told BBC Radio Gloucestershire:
Media playback is not supported on this device
"It's exciting, it keeps everyone on their toes. I thought we played well, I'm disappointed that we went behind again, like we have done in recent weeks."
"We showed character to come back into the game and to then get a winner was first class from the boys."
"There's two teams I think fighting it out at the top, so credit to Cheltenham that they've done really well to go to Tranmere and get a result and we've just got to keep going and worry about ourselves."
Jon Parkin netted a late winner as National League promotion-chasers Forest Green beat 10-man Eastleigh.