 He was born William Elphinstone in Stirlingshire, the third son of Charles Elphinstone, 10th Lord Elphinstone, and his wife Lady Clementina Fleming.His younger brother was George Elphinstone, 1st Viscount Keith.He went to sea at the age of 15 and joined the East India Company's maritime service in 1757.