James Freney was a native of County Kilkenny, and from a respectable family who had been wealthy and powerful in the region since the 13th century, having their seat at Ballyreddy Castle.
He received a good education locally, including tuition in the Robbins household-and in 1742 moved to Waterford where he opened a pub with his wife Anne.