He was eldest son of Andrew Murray (1782-1847) of Murrayshall, Perthshire, who served as sheriff of Aberdeenshire, and his wife Janet, daughter of Oliver Thomson of Leckiebank.
Murray Graham was a relation of Thomas Graham, 1st Baron Lynedoch (died 1843), to a part of whose estates he succeeded in 1859 on the death of his heir Robert Graham of Balgowan, and whose surname he adopted.