Bayesian optimisation in deeplearning
Currently I am fiddling around with Ax/BoTorch to optimize a ResNet50 for Object classification (and later on detection via SSD). I found it quite hard to apply the samples from the docs to my own model, so this is a very first prototype. Just have a look at this GIST...

https://gist.github.com/pinkerltm/74f98485d1408849476fb02686d0c228

So far I can tell, that it is somehow optimizing the learning rate into the right direction, but I think I have to wire some more Hyperparameters properly and perhaps optimize longer to get a result which is compareable to simply defining the best parameters manually so far, as Accuracy with the optimized Learning rate only gets around 67% after several 100 epochs while at least 71% should be easily possible with this model.

I am also very interested in sharing experiences and findings on this. I will edit my answer here accordingly so that it should provide at least an simple example of optimizing a CNN Classification Network with Ax/BoTorch.