after sealing the chimney, fireplace still not insulated
If this is a "Through the wall" fireplace (exposed brick on the exterior wall, which I think is what you are describing) then yes, it conducts heat very efficiently out of your house. Insulation and stucco over the exterior brick would probably be the most effective solution other than removing it if you don't use it.