Nevers parents, George and Mary Ann Nevers, were immigrants to the United States from New Brunswick, Canada.
In 1921, Nevers attended Santa Rosa Junior College and was the star of the school's football team.