Tuning a classifier for high precision, with no regard for recall
To make your learner cost sensitive, you can increase your training data by more "no" instances. If there are 10 times more "no" instance in your training data, the errors on "no's" will hurt much more and your learner will come up with a dicision scheme which will be biased that way. The variance on the "yes'es" will be lowered on the other hand (pay attention on overfitting here). 
After the training, you use your original data for testing and you should get good results here.