After the Norman conquest of England in 1066 William FitzOsbern, 1st Earl of Hereford held a manor of 10 hides of land at Fritwell.
The de Lisle manor house was probably built late in the 16th century and rebuilt in 1619.