The think tank's UK growth forecast for 2016 has been cut from 2.3% to 1.9%, and from 2.6% to just 0.4% for 2017.
Meanwhile its forecast for GDP growth for 2018 was slashed from 2.4% to 1.4%.
Its report said the Brexit vote would have "severe confidence effects on spending and business investment", which would lead to anaemic GDP growth over the next three years.
However, the drop in the value of the pound could bolster exports by 3.4% next year, the Item Club said, with imports falling by 0.3%.
Overall, the move would see net exports adding 1.1% to GDP in 2017, it added.
Brexit 'will be horrible for UK economy' - fund manager
Peter Spencer, chief economic advisor to the EY Item Club, said the UK economy was set to suffer a severe loss of momentum in the second half of this year.
"Heightened uncertainty is likely to hold back business investment, while consumer spending will be restrained by a weaker jobs market and higher inflation," he said.
"Longer-term, the UK may have to adjust to a permanent reduction in the size of the economy, compared to the trend that seemed possible prior to the vote. But amongst the gloom, the weaker pound provides one silver lining to exporters, particularly those selling to the US and emerging markets."
A new survey from Deloitte found 82% of chief financial officers of both FTSE 350 and large privately owned companies expected to cut capital spending in the next year. That was the biggest proportion on record and more than double the 34% in the first quarter, Deloitte said.
Its survey was conducted between 28 June 28 and 11 July, after the EU referendum and just before Theresa May became the new prime minister.
Almost all the CFOs surveyed said the level of uncertainty facing their businesses was higher than normal.
Ian Stewart, Deloitte's chief economist, said: "Perceptions of uncertainty have soared to levels last associated with the euro crisis five years ago."
Leading City fund manager Richard Buxton, chief executive of Old Mutual Global Investors, has said the Brexit vote would have a "horrible" impact on the UK economy, which could "judder to a halt".
The Item Club's Mr Spencer said that there were some positives ahead, including the UK retaining full access to the EU's single market in the short-term.
As well as the fall in the exchange rate offering export opportunities, he said the predicted increases in inflation and unemployment would allow the economy to rebalance away from consumption.
The Item Club also predicted that unemployment would rise from 5% to 7.1% by late 2019.
The UK economy may face "severe loss of momentum" after the vote to leave the EU, according to the EY Item Club.