Elrington was advanced on 25 September 1820 to the bishopric of Limerick, and on 21 December 1822 he was translated to Leighlin and Ferns.
A portrait of the bishop was painted in 1820 for his brother, Major Elrington, by Thomas Foster; engraved by William Ward, it was published in 1836 by Graves & Co.