Should I remove features that occur very rarely to build a model?
Short answer: 

yes, if they occur so rarely, they can only lead to overfit, so it's better to ignore them as features.

Longer answer:

Usually one puts all those unique occurrence in a single feature/token, and that's the way I suggest you to proceed. So if you have two features that appears only once, you can join them to create a feature that appears twice.

But anyway, 3000 short text are too few to create a NLP model that can work well. 
To mitigate your lack of data, you can use pre-trained word embeddings like the ones available here. Doing so you can also keep most of the single occurring words/features, because in those word embeddings they already have a defined semantic meaning.