John Dillard settled in the area around what is now Dillard with his son, James Dillard, circa 1823.
The small town is now known primarily for the Dillard House, a regionally well-known inn, restaurant and conference center that serves traditional southern fare.