Hart was born Howard Lester Phillips, Jr. on 16 October 1940 in St. Louis, Missouri, the only son of Eleanor Laidley and Howard Lester Phillips.
Hart attended Kent School and graduated from Brent School in the Philippines in 1958.