Born in Manhattan, Rose attended Townsend High School and briefly attended City College (now part of the City University of New York) before serving in the U.S. Army in 1942-46, where he became a first lieutenant.
Rose was married twice, to Barbara Langbart in 1943, with whom he had four children, and to Ellen McLaughlin (not the playwright and actor) in 1963, with whom he had two children.