Is an $F_1$ score of 0.1 always bad? I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans originating in 2015Q4), but for now I'm just doing in-time training (training &amp; testing on 2018Q1) -- and even this I've found challenging. The dataset contains ~400k observations, of which ~99% are non-delinquent and ~1% are delinquent. My idea so far has been to use precision, recall, and $F_1$ as performance metrics.
I am working in Python. Things I've tried:

Models: logistic regression &amp; random forest.
Model selection: GridSearchCV to tune hyperparameters with $F_1$ scoring (results were not significantly different when optimizing for log-loss, ROC-AUC, Cohen's Kappa).
Handing imbalanced data: I tried random undersampling with various ratios and settled on a ratio of ~0.2. I also tried messing with the class weights parameter.

Unfortunately, my validation &amp; testing $F_1$ scores are only around 0.1, (precision &amp; recall are usually both close to 0.1). This seems very poor, since with many problems you can achieve $F_1$ scores of 0.9+. At the same time I've heard there's no such thing as a &quot;good $F_1$&quot; range, i.e. it is task-dependent. Indeed, a dummy classifier which predicts proportional to the class frequencies only achieves precision, recall, and $F_1$ of 0.01.
I've tried to find references on what a &quot;good&quot; score for this type of task is, but I can't seem to find much. Others' often report ROC-AUC or Brier Score, but I think these are hard to interpret in terms of business value added. Some report $F_1$ but see overly optimistic results due to data leakage or reporting testing performance on undersampled data. Finally, I've seen some people weight confusion matrix results by expected business costs as opposed to reporting $F_1$, which seems like it may be a better route.
My questions are: (1) is an $F_1$ score of 0.1 always bad?, (2) does it even make sense to optimize for $F_1$ or should I used another metric?, (3) if $F_1$ is appropriate and a score of 0.1 is bad, how might I improve my performance?
From a credit scoring point of view : a $F_1$ score of $0.1$ seems pretty bad but not impossible with an unbalanced data-set. It might be enough for your needs (once you weight your errors by the cost). And it might not be possible to go higher (not enough data to predict an event that appears random). In credit scoring there is always a 'random' part in the target (sudden death, divorce ...) depending on the population and the goal of the loans.

You might want to investigate your features and your target. Basically : statistically, on an univariate approach, do you have features that appears predictive of the target ? (Age of the person ? revenue ? purpose of the loan ?). You might also need to investigate the target : do you have some questionnaire that would allow to get an insight on why the person defaulted ? (If the majority of default come from random event, you might not be able to modelise it).

The main problem with $F_1$ score in credit scoring is not data imbalance, but cost imbalance. Type I and Type II errors have far differents consequences. Given that you already gave the loans I am not even sure there is a cost associated with false positive (saying someone will default when it won't). It might be interesting to weight precision and recall (i.e. use $F_\beta$ as defined here). Another problem is that it is usually good for a binary decision. Depending on what you want to use the model for (measuring risk of already granted loans ? granting new loans ? pricing new loans ?) there might be alternatives that better capture model discrimination (AUC - see its statistical interpretation) or individual % chance of default (Brier Score).

Assuming that there is no specific problem with your current modelling (Feature engineering, imbalance treatment, 'power' of your model). There are some credit-scoring specific things you can do. Work on your target definition (what if you do 90+ days delinquant in the 5 years after origination ?). Try to collect more data about your clients and their behavior (purpose of the loan, others products they use at your bank... etc.).