Glasgow's maternal uncle, Joseph Reid Anderson, graduated fourth in his class of 49 from West Point in 1836.
Anne Gholson married Francis T. Glasgow on July 14, 1853, and they had ten children together.