Roberts was born on September 14, 1879 in Chillicothe, Ohio, the son of Andrew Jackson Roberts (1852-1927), a graduate of Oberlin College, and Ellen Wayles Hemings (1856-1940), the daughter of Madison Hemings and Mary Hughes McCoy, a free woman of color.
Roberts attended college at the University of Southern California (USC) where he majored in pre-law.