But thanks to the controversy around gifts to David Cameron from his mother, IHT is now under the political spotlight.
Just 40,100 families - that's around 8% of estates - will pay any IHT at all in 2016/17, according to the Office for Budget Responsibility (OBR).
Thanks to rising house prices, that number is set to rise over the next few years, but much more slowly than was originally expected.
That's because the chancellor, George Osborne, has finally been able to make changes to IHT - changes that were blocked by the Liberal Democrats during the years of coalition government.
So how does IHT work - and what reforms are already in the pipeline?
If you pass on property, or anything else of monetary value, to your descendants, you will pay IHT, but only if the total value exceeds £325,000 in 2016/17.
Anything above that amount will incur tax at 40%.
Since married or civil partners can transfer assets free of tax between each other, one partner automatically inherits the other's allowance.
So in practice, the IHT allowance is often doubled to £650,000.
In other words, if a father passes wealth to a mother, who subsequently also dies, she can pass on up to £650,000 without having to pay IHT.
From April 2017, the government will introduce a new Transferable Main Residence Allowance (TMRA), to help people pass on property to their descendants.
Initially this will be set at £100,000, rising to £175,000 by 2020/21.
When added to the IHT threshold of £325,000, it will allow each individual to pass on £425,000 with no tax payable - or £850,000 per couple.
By 2021, the tax-free limit will be £500,000 each, or £1m for married or civil partners.
Further details of TMRA can be found here
Even if a parent sets up a trust in favour of a child, inheritance tax is still payable. In most cases, IHT will be charged at 20% on money or property when it goes into the trust, if it exceeds the IHT allowance.
If the parent dies within seven years, an additional 20% is charged, to equal the 40% non-trust rate.
In addition, assets within a trust are usually re-assessed every ten years, to take account of changing property valuations, for example.
Under the existing rules, you can pass on money, property or possessions without paying any tax, as long as you survive for seven years after giving it to the recipient.
If you die within the seven year period, and you gift more than the allowance, a taper system exists. If you die six years after making the gift, for example, you will only pay 8% IHT. See table below.
In addition you can give up to £250 a year to as many people as you like without paying tax.
Usually it is the estate which is liable for IHT. However if you are the recipient of a gift, and the giver has died within 7 years, and has already given away more than £325,000, you could be liable to pay IHT yourself.
More details on gifts
Anyone can give away up to £3,000 a year, and pay no tax. This is known as the annual exemption. If unused, this allowance can be carried over to the following year, up to a maximum of £6,000.
In addition, if you can show that the gift was funded out of income - as opposed to savings - you will not pay IHT. But you may have to prove that the living standards of the deceased person were not reduced as a result.
The rules around wedding - or civil partnership - gifts are different again. Providing the gifts are made at the time of the wedding, there are allowances as follows:
Very few people pay inheritance tax (IHT) and it raises relatively little money for the Treasury.