Phillips played football at Lamar University in Beaumont, Texas but enlisted in the U.S. Marine Corps shortly after the attack on Pearl Harbor.
His college coaching stints included serving as an assistant coach at Texas A&M University (for Bear Bryant), the University of Houston (for Bill Yeoman), Southern Methodist University (for Hayden Fry), and Oklahoma State University with Jim Stanley.