The 51-year-old man, who has not been named, was arrested in North Yorkshire on Wednesday following a complaint made to Durham Constabulary.
The force said the man had been bailed pending further inquiries.
The vicar has been suspended for the duration of the investigation by the Bishop of Leeds, the Diocese of Leeds said.
It added in a statement: "Appropriate pastoral and safe guarding arrangements have been put in place.
"No further statement will be made at this time."
A vicar arrested in connection with historical sexual abuse has been suspended by the Church of England.