European Commission getting rid of excess carbon emission permits
What were these excess permits of carbon emissions? Is it the amount of permits that are
still up for sale (or to be distributed for free); and hence, aren't used?
Page 92 of the ETS handbook states that since 2009, because of the economic crisis of 2008, a higher than expected imports of international carbon credits and a large uptake of renewables led to a significant surplus of allowances in the EU ETS.
So, yes, excess permits are the ones that aren't used by the holder in the near future. According to the ETS handbook, page 92, in early 2012 there were 1 billions surplus allowances. And 2 billions by the start of 2013.
From the handbook:

The presence of the surplus and resulting low price may discourage participants in the
EU ETS from taking actions now to reduce emissions, which may result in a deviation from the most
cost-efficient path to long-term emission reduction goals. The large surplus has led to persistently low
carbon prices in the EU ETS, reducing the price incentive for low-carbon investments.

One mechanism of tackling this surplus in allowances is the creation of a market stability reserve (MSR).
And how did the European Comission get rid of them? Is it by decreasing the cap of overall supply of permits within the EU?
The MSR acts to combat this imbalance of allowances. From the handbook, page 95:

Using the total number of allowances in the market as an indicator, market imbalances due to
unexpected shocks that impact demand, such as the economic crisis, can be addressed. This allows
the EU ETS to maintain its objective to reduce emissions in a cost-effective and economically efficient
manner, even under unexpected circumstances.

Hence, this way, each year the number of allowances injected into the market by auction will take the market demand into account.
So, yes, the amount of overall supply of permits, by auction, will be decreased if there is an excess. This will act as a carbon reduction incentive, by keeping prices competitive.
From this diagram of the ETS handbook, company B, with a shortage of allowances, has two options. Either buy from company A's surplus allowances or buy them at an ETS auction.

Also, what is the unit of an allowance?
The website of the European Commission states that each allowance gives the holder the right to emit one tonne of carbon dioxide (CO2), or
the equivalent amount of other powerful greenhouse gases, nitrous oxide (N2O) and perfluorocarbons (PFCs).
But isn't this ultimately limited by the overall EU cap that is decided politically?
Yes, indeed the total amount of allowances injected into the market by the ETS is determined politically. This overall cap will decrease accordingly, to realize the European Green Deal; no net emissions of green house gasses by 2050.