Would there be any reason to pretrain BERT on specific texts? So the official BERT English model is trained on Wikipedia and BookCurpos (source).
Now, for example, let's say I want to use BERT for Movies tag recommendation. Is there any reason for me to pretrain a new BERT model from scratch on movie-related dataset?
Can my model become more accurate since I trained it on movie-related texts rather than general texts? Is there an example of such usage?
To be clear, the question is on the importance of context (not size) of the dataset.
Sure, if you have a large and good quality in-domain dataset, the results may certainly  be better than with the generic pretrained BERT.
This has already been done before: BioBERT is a BERT model pretrained on biomedical texts:

[...] a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement).

Of course, other factors may be taken into account in the decision to pretrain such a model, e.g. computational budget.