Labeling Dataset Algorithmically
Are there any statistical/otherwise tests that I can run to understand the quality of the labeling I get from my code.


Yes. You can treat your automated labelling as a model in its own right (it essentially is an "expert" model, that takes in additional features). Collect ground truth data with known accurate labels, and use a metric such as accuracy, AUROC, F1 score to determine how well your expert model works. Usually you should pick one metric that makes sense to you based on the problem you are trying to solve. Most statistics libraries will allow you to assess these metrics on arbitrary data, so it should be possible to find the relevant part of the API and feed it your synthetic labels to compare with ground truth data you have collected.

But wait! That is exactly what you would need to do in order to train and assess an ML model. About the best you can hope is that you will need less ground truth data to assess your expert than you would to train the ML version from scratch. For complex ML models such as deep neural networks, this could well be true.

Note that you may hit a limitation from this approach. Say that you assess from ground truth data that your expert is 85% accurate. You will not be able to use it to train an ML solution that is better than 85% accurate, because the ML will learn to copy the expert's predictions as best it can.

One thing to watch out for - if you analyse why the expert got some values wrong and then adjust it to give correct answers for those labels, you will need to remove the labels you used to do this from your test data - and ideally collect more, or take more from some unused pool, to replace them. Otherwise you will fit your expert to the test data and get incorrect reporting of how well the expert generalises to unseen data. Same reasoning means you should not use tweets and labels that you analysed when developing your expert model in order to test it.