He was the eighth child of the Rev. David Williams (1751-1836), born in the Hungerford almshouse in Wiltshire; his father, an uncle of John Williams (1792-1858) the archdeacon of Cardigan, was warden of the almshouse and curate of Heytesbury.
In 1840 Williams was elected a fellow of the College of Physicians.