 Doughton was a member and deacon of the Laurel Springs Baptist Church.After his death in 1954 at the age of 90 he was buried next to his wife Lillie in the church cemetery.In the 1990s Rufus Doughton's former home in Laurel Springs was restored, and it is now a popular bed-and-breakfast for tourists to the region.