Hyperparameter optimization for Pytorch model
What is the best way to perform hyperparameter optimization for a Pytorch model? Implement e.g. Random Search myself? Use Skicit Learn? Or is there anything else I am not aware of?

Many researchers use RayTune. It's a scalable hyperparameter tuning framework, specifically for deep learning. You can easily use it with any deep learning framework (2 lines of code below), and it provides most state-of-the-art algorithms, including HyperBand, Population-based Training, Bayesian Optimization, and BOHB.
import torch.optim as optim
from ray import tune
from ray.tune.examples.mnist_pytorch import get_data_loaders, ConvNet, train, test


def train_mnist(config):
    train_loader, test_loader = get_data_loaders()
    model = ConvNet()
    optimizer = optim.SGD(model.parameters(), lr=config[&quot;lr&quot;])
    for i in range(10):
        train(model, optimizer, train_loader)
        acc = test(model, test_loader)
        tune.report(mean_accuracy=acc)


analysis = tune.run(
    train_mnist, config={&quot;lr&quot;: tune.grid_search([0.001, 0.01, 0.1])})

print(&quot;Best config: &quot;, analysis.get_best_config(metric=&quot;mean_accuracy&quot;))

# Get a dataframe for analyzing trial results.
df = analysis.dataframe()

[Disclaimer: I contribute actively to this project!]
