which scoring function for validation_curve (regression)? Is there any thumb of rule which scoring function should be used for e.g. the validation_curve?

Atm I try to study the difference between several optimizers:

validation_curve(grid_best 
                                          , X_train 
                                          , y_train
                                          , param_name = 'Adam', 'SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adamax', 'Nadam']
                                          , param_range = param_range 
                                          , cv=tscv
                                          , scoring="explained_variance" 
                                          , verbose = 1
                                          , n_jobs = n_jobs
                )




I use the explained_variance but I think the function has to be minimized cause the values are mostly below zero. That's why I think the explained variance does not make sense here?

edit:

When I use r2 I get the following curve:



Is that normal?
I'd use r2 for validating regression. It'd be much easier to compare different models. Here's wiki, here's sklearn.