Potts was born on 10 December 1908 in Buzzard Roost, California (a temporary logging camp in Shasta County) while his parent were traveling to Oregon.
During World War II, Potts served on active duty in the United States Navy, and then remained in the Navy Reserve for 20 years.