Born in Gordon, Georgia, Williams was a noted Savannah, Georgia antiques dealer and historic preservationist who played an active role in the preservation of Savannah's historic district.
In 1969, Williams purchased a home originally built for General Hugh Weedon Mercer, great-grandfather of famed American songwriter Johnny Mercer.