Setting a minimum learning rate on "Reduce On Plateau"
I'm currently using PyTorch's ReduceLROnPlateau learning rate scheduler using:
learning_rate = 1e-3
optimizer = optim.Adam(model.params, lr = learning_rate)
model.optimizer = optimizer
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(model.optimizer, factor=0.9, patience = 5000, verbose=True)

My issue is that the loss will get really low within a few minutes, then jumps really high and start decreasing steadily. Hence, ReduceLROnPlateau will just continuously decrease the learning rate to a point that the model can't learn anything (because it takes the lowest loss into acount).
Is there a way to set a minimum learning rate using this scheduler so that it won't go to almost 0? Say to set it at minimum of 1e8

Yes, from the documentation:

min_lr (float or list) – A scalar or a list of scalars. A lower bound on the learning rate of all param groups or each group respectively. Default: 0.

You can simply go for:
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
    model.optimizer,
    factor=0.9,
    patience=5000,
    verbose=True,
    min_lr=1e-8,
)

