Determining threshold in an area with very few samples of positive label
You need a cost sensitive learning mechanism for these type of tasks. You basically create a cost matrix that determines the cost of getting predictions wrong. For example while predicting whether a patient has cancer or not, the cost of predicting he doesn't have a cancer while he does is much more than predicting than he has a cancer and actually doesn't. 
Here's a good article : https://mlr.mlr-org.com/articles/tutorial/cost_sensitive_classif.html

Here's one for python :
https://towardsdatascience.com/fraud-detection-with-cost-sensitive-machine-learning-24b8760d35d9