The youngest of six children, when he was still a child, his family moved to the Bronx, New York, where Rosenthal's father found work as a house painter.
According to his son, Andrew Rosenthal, Rosenthal was a member of the Communist Party youth league briefly as a teenager in the late 1930s.