Machine learning algorithm for predicting binary decisions on a large, underrepresented dataset
When you have many more "No" samples than "Yes" samples, this is known as the "class imbalance" problem.  Dealing with class imbalance is challenging: many machine learning algorithms don't work as well in this situation.

There are many techniques for dealing with class imbalance, including using class weights, oversampling the "No" samples, or undersampling the "Yes" samples.  I suggest doing a little bit of reading about techniques for dealing with class imbalance on Stats.SE, then trying them to see if they help.  Their effectiveness may depend on the data set and on which type of classifier you use.