How to determine best parameters after grid search?
I too was in your place when I started using Neural networks. There are so many hyper-parameters to choose, and each take on many values. Like Emre said, you need to check the model which is giving best metric score on your data (Cross validation set). The parameter values of that model will be your optimized values. You can also check this link