He married Margaret Chambers on January 18, 1816 in Bardstown, Kentucky.
Brandon died, at the age of 61, on March 28, 1850 and was buried in a private family cemetery at his Columbian Springs Plantation in Wilkinson County, Mississippi.