The new measure could lead to delays of up to three years before new drugs are made available to give NHS bosses the chance to try to renegotiate the price with drug firms.
The plan was agreed by the National Institute for Health and Care Excellence amid mounting pressure on NHS funds.
It will be introduced from next month.
NICE chief executive Sir Andrew Dillon said the move was needed given the "significant financial challenge facing the NHS".
Patient groups and the drug industry have already objected to the plan - a consultation has been run over the past three months.
Currently, drugs that are assessed as being cost-effective by NICE are automatically recommend for use in the NHS.
Once that happens, the health service has 90 days to start offering the drugs.
But crucially that process just assesses the cost versus benefit of the drug on the basis of the impact to an individual.
It does not take into account how many people may take the drug and therefore the total cost to the NHS.
Under the new plan, widely used drugs and expensive but effective drugs that breach the £20m a year threshold will face a secondary process.
NHS England, which pushed for the change, will be able to halt the 90-day deadline and begin talks with the drug manufacturer to try to get the price down.
They will be able to apply for an extension of up to three years. At that point, NICE will review what is happening.
During this period, NICE will have the power to allow restricted use to patients deemed most in need.
The move comes as the drugs bill is on the rise. Last year £16.8bn was spent on drugs by the NHS, up from £13bn in 2011.
There is concern a breakthrough in fields such as dementia could end up costing the NHS billions of pounds.
Last year NHS bosses capped the number of patients that could be given a new drug for hepatitis C, to keep the annual cost at £200m.
The new arrangements will be applied only to new drugs.
Information provided by the industry suggested that if they had been in place they would have affected the rollout of drugs such as Ezetimibe, used by 159,000 people with heart disease, and Lucentis, which is used by 1,200 people with diabetes to prevent sight loss.
As well as introducing the cap, NICE has also agreed to a fast-track process for cheaper drugs.
A new £20m-a-year cap on the cost of new drugs will be introduced in the NHS in England in an attempt to save money, health chiefs have announced.