How to keep balance in the thin line between non attachment to reputation and "not caring about anything"
I don't exactly have an answer or advice to this, as I have a similar experience to a certain extent, but I understand that even though others might see the external us and base an opinion on it, the mind development that's going on for Buddhists may always be misunderstood by those not familiar with it and thus, dukkha arises for the one making the judgement. Maybe it also arises in us too, if we react to it even just a little bit. It's hard to be unaffected, I find. I guess that depends on where each one of is in the practice.  As the Buddha describes, it is 'Patisothagami' or "it is a path that goes against the stream", so we're bound to hit resistance both from the stream and the other fish...to use a popular analogy. 

Maybe here equanimity is needed with compassion in order to keep us within the sphere of 'normal' worldliness whilst striving for something beyond it.

I hope the reflection is useful and thank you for bringing it up.

Best wishes

Paul