Does reinforcing correct predictions increase model accuracy further?
Your suggestion is risky. It might make improvements to your classifier, but it may also reduce generalisation.

The two conflicting factors in play are:


Adding data points to a model which has capacity to learn more detail can improve its performance.
Training from a different distribution of data points than your target population can reduce its performance.


From the first point, you need your model to be able to accept the new data. This can be harder to achieve than you might think at first - if you have tuned some regularisation parameters by using cross-validation, then you may have at least in part saturated the model's capacity in order to prevent over-fitting. That means fitting to new data could require re-tuning your hyper-parameters as well - but you probably won't need to start from scratch, just search nearby.

It is hard to tell the impact from the second point. There is not going to be a general answer here, too much depends on how the data is arranged for your problem. My gut feeling is that you will notice an improvement to the model initially, but that there will be diminishing returns due to the self-selection of already-correct data points.  

You should definitely keep back a cross-validation set of data distributed as it is in production, that has been properly and fully labelled in order to assess this work. This will be your way to assess whether generalisation is improving using your approach.

Worst case: You may need to go back and re-label the mistakes in order to significantly improve performance of your model. So I would suggest part of your auto-labelling process should store the misclassified items somewhere so you can re-visit them and spend the extra effort. 

Sadly, even collecting more properly labelled data is not a guaranteed fix - some models already have enough data. You can check whether that might applies in your case by training with different sizes of training set (from the available training data you have so far), and seeing what the trend in performance is when you increase the training set size. In fact, you should probably do this first, before you invest significant effort in collecting more data.