Will not collaborating hurt the career of a math postdoc?
I am a math professor at a research-oriented university, and I have been on multiple hiring committees for tenure-track positions. If I were reading your file, your situation would raise two questions:


Are you doing research that is of broad interest? 
Do you work well with others? Are you the kind of person that has ideas to share, that I would want to chat math with in the hallway, that would get graduate students interested in your subject?


Your application should offer evidence that the answer to both of these questions is yes. The first point will hopefully be addressed by your letter writers. It will also look good if your CV lists many invited conference or seminar talks.

The second point will again hopefully be addressed by your letter writers. If possible, I would suggest that at least one of them should be written by someone with whom you have talked math a great deal, and not just by your advisor and senior scientists who have read your papers. If you have engaged in any leadership activities (i.e., started a seminar, organized a conference, informally helped to mentor graduate students, etc.) I would make sure that your letter writers include someone who will attest to this.

And finally, like the other commenters, I would definitely recommend striking up some collaborations, although there is certainly nothing wrong with also doing work on your own.