 He was consecrated Bishop of Tirunelveli in 1877.In 1844, Caldwell married Eliza Mault (1822–99), with whom he had seven children.She was the younger daughter of the veteran Travancore missionary, Reverend Charles Mault (1791–1858) of the London Missionary Society.