Bounds for MLP hyperparameter search
In designing an MLP architecture, we can restrict ourselves to 4-8 layers with 8-128 (power of 2) neurons per layer. In addition, we can assume recommended ReLU activations with He normal weight initialization and Adam or SGD with Nesterov momentum optimizers (see ipython notebook for comparison). Your loss function will depend on the problem: cross-entropy for classification and MSE for regression. 

In order to avoid local optima in parameter search, it's better to use random search or bayesian optimization. For additional answers, see this post.