Frost and his wife Shirley were married for 47 years and raised two children: Louis O. Frost IV and Deborah.
Upon his retirement, Frost was interviewed by the Florida Times-Union: "It has been rewarding to see people whose careers I've helped and also to see people that we've defended turn the corner and turn their lives around.