Westheimer was born on September 6, 1916, the son of Milton F. Westheimer, a Baltimore investment banker, and Helen Gutman Westheimer.
He began work in the toy department at Macy's in New York City for $35 per week.