 Howard's mother was Elizabeth Mackay Howard, daughter of Helen and Aeneas Mackay, a United States Army officer stationed at Jefferson Barracks, St. Louis, as was also her maternal grandfather, Captain Thomas Legate.Captain Legate moved to Galena, Illinois in 1828 as Superintendent of the lead mines.As a cadet in the academy, Howard was often, by special invitation, a guest at the White House, and was usually appointed Grant's aide whenever the President visited Annapolis.