The 65-year-old said he was going to "miss it dreadfully" but there always comes a point when things must end.
Sir Eric also served as secretary for communities and local government between 2010 and 2015.
He is one of several high-profile MPs who are standing down, including George Osborne and Alan Johnson.
Prime Minister Theresa May announced on Tuesday she intended to call a snap general election, a move that was backed by MPs the following day. The election will take place on 8 June.
Labour MP for Nottingham North Graham Allen also announced on Saturday that he is to step down as an MP, due to ill health.
Speaking to the BBC, Sir Eric said: "I think it's always better to leave when people are asking why you're going, rather than why you're staying.
"I'd always decided this was going to be my last Parliament having served 25 years and came to the view that it was time Brentwood and Ongar had a new MP."
He said he had told Mrs May of his decision, and wrote on Twitter that he would be continuing in his role as the prime minister's special envoy on post-Holocaust issues.
Which MPs are quitting - and who might stand?
In a letter sent to his local Conservative association, Sir Eric wrote: "It has been an enormous honour to represent the constituency for 25 years; Brentwood and Ongar is a wonderful place to live.
"Throughout that quarter of century, I have enjoyed success and some setbacks, but have always been sustained by the friendship back home in the patch."
He added that he was grateful for the invitation to serve a further five years but that after "much heart searching" he had concluded it was the "right time for Brentwood and Ongar to have a new representative in Westminster".
Sir Eric, who was born in Yorkshire into a Labour-supporting family, joined the Keighley Young Conservatives in 1968, going on to become chairman. He was knighted in 2015.
Formerly a Eurosceptic, in 2016 he joined the group of Conservative MPs supporting the campaign to remain in the European Union subject to David Cameron's renegotiations.
Sir Eric was responsible for the Localism Act of 2011 that changed the powers of local government in England.
The measures effected by the Act include more elected mayors and referendums.
Sir Eric Pickles, the former Conservative Party chairman, has announced he is standing down as Brentwood and Ongar MP after 25 years.