Carswell attended North Side High School, where he played football, with his high school highlight being the winning touchdown he scored on Armistice Day in a game against the Wichita Falls team in 1933.
On a double date while still at TCU, Horace met a co-ed featured in one of the yearbook's beauty pages, by the name of Virginia Ede.