Early literary friends were Dr. William King of Oxford and his pupil Lord Orrery, with whom Ellis frequently exchanged visits.
Chief among the circle of his literary friends and admirers was Samuel Johnson, who once said to Boswell, "It is wonderful, sir, what is to be found in London.