Ruffley was born in Bolton, Lancashire.
During this time, he was an adviser to the Grant Maintained Schools Foundation (1993-96), as well as governor of St Marylebone School (1992-94) and Pimlico School (1994-96).