Can sodium bicarbonate be considered an acid?
The statement you read that "baking soda is a base" comes from the fact that a solution of sodium bicarbonate (baking soda) and water has a pH of around $8.3$.  

However, sodium bicarbonate is amphoteric with respect to Brønsted–Lowry acid/base theory, which means that it can act as either an acid or a base. More specifically, the bicarbonate ion is amphiprotic, meaning that it can either donate a proton, acting as an acid, or accept a proton, making it act as a base. The reason that a sodium bicarbonate solution is slightly basic is that it's ability to take a proton from a water molecule is greater than its ability to donate a proton to a water molecule. In other words, the chemical reaction causing the solution to be basic is:  

$$\ce{HCO3- + H2O &lt;--&gt; H2CO3 + OH-}$$  

The other equilibrium reaction at play, the reaction "trying" to make the system more acidic, is:  

$$\ce{HCO3- + H2O &lt;--&gt; CO3^2- + H3O+}$$  

Again, in an otherwise pure aqueous system, it is the first reaction that dominates, and thus giving the observed slightly basic solution.  

Note that the $\ce{H2CO3}$ produced in the first reaction is also part of yet another equilibrium process:  

$$\ce{H2CO3 &lt;--&gt; CO2 ^ + H2O}$$  

Where the $\ce{CO2}$ is liberated as a gas. This is the cause of the well-known "fizzing" when baking soda is added to vinegar (acetic acid).