Confusion regarding the mechanism of acidic/basic buffer solutions
While learning about acidic and basic buffer solutions we were told that they can resist pH change if we add small amounts of acid or base by neutralizing the $\ce{H+/OH−}$ ions from the added acid or base.

Yes, the pH changes less when you add small amounts of acid or base to a buffer, and more if you add it to pure water or an unbuffered aqueous solution.

But I am confused about the fact that if we add a small amount of base to an acidic buffer then after neutralization of the excess $\ce{OH−}$ ions from the base by the H+ ions from the buffer, shouldn't the concentration of $\ce{H+}$ ions in the solution decrease,

Yes, that is what happens. As a consequence, the weak acid/base reaction of the buffer  substances is no longer at equilibrium, and there will be a net reaction to replenish some of the $\ce{H+}$ that was lost.

thereby making the solution acidic and reducing the pH. (vice-versa for the addition of acid to a basic buffer)

This is probably a typo. When you add a small amount of base, the pH would go up, buffered or not. In the presence of buffer, it goes up a little, and in the absence of buffer, a lot.

(vice-versa for the addition of acid to a basic buffer)

Buffers stabilize the pH near the $\mathrm{p}K_\mathrm{a}$ of the conjugate weak acid in &quot;both directions&quot;, so even if the pH of a solution is slightly acidic, it is able to buffer against addition of acid or base.