After his birth in Dallas, Cole lived with his mother in Texas until the age of four, when he became ill with a severe case of scarlet fever.
He enlisted in the United States Coast Guard during World War II, where he met his wife, fellow Coast Guard member Nancy Corbett.