 Calthorpe first came to reside at his family's ancestral home, Ampton Hall, in 1736, and immediately set about improving his mansion and estate by enlarging the former, and enclosing, planting, and ornamenting the latter; dividing his time in attendance on his official duties in London, and in agricultural and horticultural pursuits, when resident in the country.In 1754, Calthorpe offered himself as a candidate for the borough of Hindon in Wiltshire, but declined, although sure of his election, in favour of James Dawkins.Upon Dawkin's death in 1757, Calthorpe was elected for the borough on 23 January 1758.