Over the years several of Charles ' senior descendants have made names for themselves , including Augustus FitzRoy , 3rd Duke of Grafton who became Prime Minister of Great Britain , cricket pioneer Charles Lennox , 1st Duke of Richmond , parliamentary reformer Charles Lennox , 3rd Duke of Richmond . Over the years several of Charles ' senior descendants have made names for themselves , including Augustus FitzRoy , 3rd Duke of Grafton , who became Prime Minister of Great Britain , cricket pioneer Charles Lennox , 1st Duke of Richmond , parliamentary reformer Charles Lennox , 3rd Duke of Richmond .