When building up your community, remember to always abide by the PEAS principles: Prejudice, Exclusion, Apathy, Stigma.

Prejudice: when someone wants to dictate what you can do/say, they rarely declare their intentions outright. Learn to pick up on signs that this is their goal (e.g. acting offended at a harmless and popular joke) and discard their opinion accordingly. Don't give them a chance to ingratiate themselves to you or masquerade among your fan-base in the hopes that their behaviour will change - it won't.
Exclusion: no one can make everyone happy, and there exist people who will never be happy with anything you do. It is okay to disregard those people. They aren't your fans and, despite their claims, never were or will be.
Apathy: not all criticisms are made equal. Even critiques that are well-meaning and/or actionable are often delivered in excess due to the fact that those providing them have no way of knowing how many others are doing so at the same time. It may sound callous, but if you are approaching the Internet as a professional looking to generate an income, apathy is a necessity in order to avoid being overwhelmed by general feedback (both good and bad).
Stigma: every community has rules, and every creator should have standards for their fan-base. Behaviours that do not comply with these rules and standards should be denounced and heavily discouraged. Examples must be made out of those who do not try to fit in, if only for the benefit of affirming to those who do that the integrity of their shared space is being looked after and that their efforts aren't for nought.