 He was born in London, England, the son of Thomas Penn and his wife Lady Juliana Fermor Penn (the daughter of Thomas Fermor, first earl of Pomfret), elder brother to Granville Penn, and a grandson of William Penn, founder of Pennsylvania.He studied at Eton College.On the death of his father in 1775, he succeeded to his father's interests, and inherited three quarters of the proprietorship of Pennsylvania.