Question on reservoir sampling I have a general question on reservoir sampling. When I use this method to sample a very large dataset for training machine learning classification algorithms, I am curious as to how to make my pipeline robust to fluctuations in  class distribution across samples. 

For example, suppose I am working on a binary classification problem. I want to sample a relatively small subset of data on which to evaluate my algorithms, so I used reservoir sampling. However, it seems to me (thought I could be wrong) that by chance I may draw a sample wherein the class distribution is significantly different from that seen in the larger population of data. If this is a correct inference, can anybody tell me how to remedy the situation? If reservoir sampling is not the answer, what other procedure are out there that I can leverage in this case (apart from using distributed environment)? 

I am learning, so some explanation, hint, and direction would be greatly appreciated.
Consider the class to be the variable that you are sampling.

Your "reservoir sample" should still be as good as uniformly drawn from your data.

Yes, there may be fluctuations, in particular if you have small samples. If you sample a single observation, the class distribution in that sample will be 100% of one class, there is no way around that. ;-)