He was born on 8 February 1754 at Barnack in Northamptonshire, where his father, Thomas Rennell (1720-1798), a prebendary of Winchester, was rector.
At Cambridge he made the acquaintance of Thomas James Mathias, and contributed to the notes of his Pursuits of Literature (1794-7).