Training model on a Balanced vs Imbalanced dataset?
It's an interesting problem, but I think the answer might be disappointing:

Usually problems about maximizing recall are considered in the context of a trade-off with precision, i.e. the goal is to sacrifice some precision by predicting more positives, whether true (TP) or false (FP). Usually in a binary problem this is possible because we focus on one class of importance and consider the other as irrelevant, so its performance doesn't matter.

Here we have a binary problem where we want to maximize recall for both classes, so we cannot sacrifice one for the sake of the other: any gain in recall for a class is likely to cause a loss in recall for the other class, since predicting more instances as positive for class A would mean more negative instances for class B, and conversely. If we were talking micro-recall we could still use the higher importance in proportion of class A, but with macro-recall we can't. Note that this is a typical case where accuracy could be used, since it would give the same weight to both classes and would be a much simpler metric.

So the only way to improve macro-recall is to increase true positives. Let's look at the options:


  1) A: 10*N observations, B: N observations


Pro: performance for class A is maximized. Cons: class B proportionally disadvantaged, so possible loss in its performance.


  2) A: 5*N observations, B: N observations


Cons: class A a bit less advantaged; pro: class B a bit less disadvantaged.


  3) A: N observations, B: N observations


Pro: maximum performance for class B; cons: class A underperforms.

Actually the best option is probably to use all of N*10 instances for A and repeat 10 times the N instances for B, so that:


the learning method can benefit from all the available training data for A
class B is not proportionally disadvantaged.


But my guess is that it's unlikely to make a very big difference anyway. When it's a matter of increasing true positives, it's usually the features and the ML method which can have an impact.