Is it possible to train a machine learning trainer?
I would try using a Genetic Algorithm to estimate optimal parameters for training. You would need to figure out the objective function of parameters of training algorithm $A_2$ which minimalised fitting $A_1(D)$. Those parameters could be regularization coefficients or numer of layers/neurons, or any other parameters describing your training algorithm.

There is also algorithm called bayesian regularization. It is very similiar to Levenberg - Marquardt training, but it is computing optimal regularization parameters $\alpha$ and $\beta$ by computing effective numbers of network's parameters. 

I hope I understood your question well.
Regards, Max