 In 1921, aged 14, Lee went to Saratoga Springs, New York, and began a two-year career as a jockey.Lee returned to his parents' home in Harlem in 1923 with no idea what he was going to do next.He considered returning to music, but an old school friend suggested that he try boxing.