Frey was born in Mankato, Minnesota, in 1871, the son of an immigrant German father, Leopold, and an immigrant French mother, Julia.
In 1893, Frey was elected president of the local International Molders and Foundry Workers Union of North America (IMFWU).