Born at Leicester, he was elder son of the physician Richard Ludlam (1680-1728), who practised there; Thomas Ludlam, the clergyman, was his youngest brother.
In 1760 Ludlam unsuccessfully contested the Lucasian chair of mathematics with Edward Waring.