Hyperparameter tuning of neural networks using Bayesian Optimization
Neural networks are optimized by gradient descent which assumes the loss function for parameters is a differentiable function, in other words smooth. Given the nature of the differentiable loss function, Bayesian Optimization could be used for neural networks hyperparameter optimization. In fact, gradient descent can be used to learn the hyperparameters themselves as evidenced in the paper - "Gradient Descent: The Ultimate Optimizer".