Is an $F_1$ score of 0.1 always bad?
From a credit scoring point of view : a $F_1$ score of $0.1$ seems pretty bad but not impossible with an unbalanced data-set. It might be enough for your needs (once you weight your errors by the cost). And it might not be possible to go higher (not enough data to predict an event that appears random). In credit scoring there is always a 'random' part in the target (sudden death, divorce ...) depending on the population and the goal of the loans.

You might want to investigate your features and your target. Basically : statistically, on an univariate approach, do you have features that appears predictive of the target ? (Age of the person ? revenue ? purpose of the loan ?). You might also need to investigate the target : do you have some questionnaire that would allow to get an insight on why the person defaulted ? (If the majority of default come from random event, you might not be able to modelise it).

The main problem with $F_1$ score in credit scoring is not data imbalance, but cost imbalance. Type I and Type II errors have far differents consequences. Given that you already gave the loans I am not even sure there is a cost associated with false positive (saying someone will default when it won't). It might be interesting to weight precision and recall (i.e. use $F_\beta$ as defined here). Another problem is that it is usually good for a binary decision. Depending on what you want to use the model for (measuring risk of already granted loans ? granting new loans ? pricing new loans ?) there might be alternatives that better capture model discrimination (AUC - see its statistical interpretation) or individual % chance of default (Brier Score).

Assuming that there is no specific problem with your current modelling (Feature engineering, imbalance treatment, 'power' of your model). There are some credit-scoring specific things you can do. Work on your target definition (what if you do 90+ days delinquant in the 5 years after origination ?). Try to collect more data about your clients and their behavior (purpose of the loan, others products they use at your bank... etc.).