Born in New York City, he was the son of James Boulter and Caroline (née Phelps) Stokes; brother of Anson Phelps Stokes and Olivia Eggleston Phelps Stokes.
As a young man Stokes started his career working in the family business, Phelps, Dodge & Company, a mercantile establishment founded by his grandfather Phelps[REF] and his uncle, William Earle Dodge, Sr., in the 1830s [REF].