The discussion centred mainly on the gas market, energy security and the risks of having to rely on supplies from Russia in the distant future.
But, actually, the report says, the short-term risks in the gas market of things such as having to pay more for imports and security of supply are minimal.
The £500m is actually made up of technical factors involving the electricity market, specifically if the UK fails to negotiate continued membership of the Internal Energy Market (IEM).
That cost comes from things such as losing access to more efficient cross-border energy trading systems.
But the report is clear such costs could be avoided if the UK remained part of the IEM, as non-EU members such as Norway have managed.
The report is also concerned about the possibility of it becoming more expensive to finance big investment projects such as new power stations, but that is not part of the £500m.
And it mentions potential problems involving the UK finding "itself excluded from EU 'solidarity principles' in which European nations agree to supply to their neighbours in the event of a gas supply crisis".
It stresses the uncertainty surrounding the forecasts, but concludes: "The impact of Brexit on the UK energy system is very likely to be negative."
But it does not say at least £500m, it says up to £500m.
Reality Check verdict: There is considerable uncertainty about the figure and much of the effect could be avoided by staying in the IEM.
READ MORE: The facts behind claims in the EU debate
A Vivid Economics report prepared for the National Grid shows leaving the European Union would put an extra £500m a year on to UK energy bills, Energy Secretary Amber Rudd has told BBC Radio 4's Today programme.