Whether you brewed your tea for too long, stored your tea improperly or used water that was just a tad too hot, your tea is bitter, and you need a solution. To fix bitter tea: 1  Add a pinch of baking soda. Just a pinch will do or else you'll wind up with tea that tastes like baking soda.  Add a dash of honey or agave syrup.