Participants had to be rejected - where and how do I explain that in my paper?
It is very simple: you just need to (1) be honest and (2) explain clearly what you did, just like in the question you are asking here.

The explanation why there are two different values of N belongs in the "Methods" or "Participants" section. But it will most likely be necessary to add (N=...) in a few other places in the paper (eg. in figures), to make sure that there is no confusion over which of the two was used.

To mention N in the abstract, if you have to choose between the two, the lower value would be preferred (in general, people like modesty and statisticians like to "err on the side of caution").