h2o, different stopping metric leads to different optimal for hyperparameters I want to choose the "optimal" hyperparameters for gbm. So I run the following code using the h2o package

# create hyperparameter grid
hyper_params = list(ntrees = c(10,20,50,100,200,500),
                    max_depth = c(5,10,20,30),
                    min_rows = c(5,10,20,30),
                    learn_rate = c(0.01,0.05,0.08,0.1),
                    balance_classes=c(T,F))

# random subset of hyperparameters
search_criteria = list(strategy = "RandomDiscrete", max_models = 192, seed = 42,
                       stopping_rounds = 15, stopping_tolerance = 1e-3, stopping_metric = "mean_per_class_error")

# run grid
gbm.grid &lt;- h2o.grid("gbm", grid_id = "gbm.grid", x = names(td.train.h2o)[!names(td.train.h2o)%like%"s_sd_segment"], y = "s_sd_segment", 
                     seed = 42, distribution = "multinomial",
                     training_frame = td.train.hyper.h2o, nfolds = 3,
                     hyper_params = hyper_params, search_criteria = search_criteria)

# sort results
gbm.sorted.grid &lt;- h2o.getGrid(grid_id = "gbm.grid", sort_by = "mean_per_class_error", decreasing=FALSE)

hyperparameters_gbm &lt;- as.data.table(gbm.sorted.grid@summary_table)


This gives as optimal combination for the hyperparameters 

learn_rate  max_depth   min_rows    ntrees
0.08    10  5   200


Then I am trying to do the same but with different stopping_metric. So in the above i use mean_per_class_error, and in the following i use logloss, so i run the following code:

# create hyperparameter grid
hyper_params = list(ntrees = c(10,20,50,100,200,500),
                    max_depth = c(5,10,20,30),
                    min_rows = c(5,10,20,30),
                    learn_rate = c(0.01,0.05,0.08,0.1),
                    balance_classes=c(T,F))

# random subset of hyperparameters
search_criteria = list(strategy = "RandomDiscrete", max_models = 192, seed = 42,
                       stopping_rounds = 15, stopping_tolerance = 1e-3, stopping_metric = "logloss")

# run grid
gbm.grid &lt;- h2o.grid("gbm", grid_id = "gbm.grid", x = names(td.train.h2o)[!names(td.train.h2o)%like%"s_sd_segment"], y = "s_sd_segment", 
                     seed = 42, distribution = "multinomial",
                     training_frame = td.train.hyper.h2o, nfolds = 3,
                     hyper_params = hyper_params, search_criteria = search_criteria)

# sort results
gbm.sorted.grid &lt;- h2o.getGrid(grid_id = "gbm.grid", sort_by = "logloss", decreasing=FALSE)

hyperparameters_gbm &lt;- as.data.table(gbm.sorted.grid@summary_table)


which gives as optimal combination for hyperparameters

learn_rate  max_depth   min_rows    ntrees
0.1 20  5   500


I know that i use as argument strategy = "RandomDiscrete", but still for instance the optimal combination for gbm using stopping_metric = "mean_per_class_error" is the "50th optimal combination" for the gbm using stopping_metric = "logloss", and the "2nd optimal combination" for gbm using stopping_metric = "logloss", is the "14th optimal combination" for gbm using 
stopping_metric = "mean_per_class_error"

Why could that happen ?
First of all, you are using different metrics to determine how well you are doing, that means it's not weird that different metrics find different hyperparameter settings that work better. Second of all, some hyperparameters might not matter for the problem you are solving, which means all the signal you are getting from those hyperparameters is noise. Third of all, most of the machine learning algorithms are stochastic, meaning there is randomness involved in training them and sometimes in evaluating them, this means even starting the same grid or random search could lead to different hyperparameters. That said the probability of that is only high if the real performances are close to each other.