Creating a valid dataset for obtaining results
There are two potential biases:

With this automatic method, you might have a few erroneous labels. For example it happens regularly here on DataScienceSE that a user tags a question &quot;python&quot; but actually the question is not specific to python at all. Same thing for the opposite case: for example it's possible that some content contains some python code but doesn't mention &quot;python&quot; anywhere.
The distribution between positive and negative classes is arbitrary. Let's assume you use 50% positive / 50% negative: if later you want to apply your classifier on a new data science website where only 10% of the content is about python, it's likely to predict a lot of false positive cases so the true performance on this data will be much lower than on your test set.

It's rare to have a perfect dataset, so realistically in my opinion the first issue is probably acceptable because the noise in the labels should be very limited. The second issue could be a bit more serious but this depends on what is the end application. Keep in mind that a trained model is meant to be applied to the same kind of data as it was trained/tested on.