Is it bad to publish only in specialized math journals?
I don't think that "generalist vs specialist" is the right question to ask.  Rather, I think that you should think of journals as being in various tiers of quality.  Notice that I'll express some opinions on specific journals below, and other mathematicians might disagree with them.  It is worthwhile to talk to a good number of senior people to help get a sense as to what tier various journals occupy.  I don't think that impact factors are what really matters here, but rather the subjective opinions of whoever will be evaluating your cv (at least in the US, this is mostly other mathematicians).

Some subject-specific journals are really excellent (e.g. Geometry and Topology, Algebra and Number Theory, Journal of Algebraic Geometry), some are decent but not excellent (I think that the Journal of Algebra fits this bill), others are pretty terrible (I won't make a list here so as to avoid starting a debate).

It is true that the very best generalist journals (Annals, Inventiones, JAMS, Acta, Publications IHES, Duke, JEMS) occupy a higher tier than even the best subject-specific journals.  However, it's not clear to me that the excellent but not top generalist journals (eg Math Zeitschrift) are clearly better than the best subject-specific journals.

You should try very hard to have your papers in the best journals that you can get to accept them.  Editors at strong journals are used to getting papers that are not directly related to their research, so you shouldn't stress out about sending your papers to them.

As to your final question (about how many papers in one tier are needed to get the equivalent quality of one paper in a higher tier), the correct answer is probably "infinity".  When I evaluate someone's cv, I am trying to figure out how good their best work is.  One paper in the Annals trumps any number of papers in lesser journals.