Stephen Lee was born to 1957 Nobel Prize winner in Physics Tsung-Dao Lee and Hui-Chun Jeannette Chin (), who died in 1996. Lee has one brother, James Lee (; born 1952), who is the dean of the School of Humanities and Social Science at the Hong Kong University of Science and Technology and chair professor of the Division of Social Science at the same university.