Pytorch schedule learning rate
I am trying to re-implement one paper, which suggests to adjust the learning rate as below:

The learning rate is decreased by a factor of the regression value with patience epochs 10 on the change value of 0.0001.

Should I use the torch.optim.lr_scheduler.ReduceLROnPlateau()?
I am not sure what value should I pass to each parameter.

Is the change value in the statement denotes to the parameter threshold?

Is the factor in the statement denotes to the parameter factor?



torch.optim.lr_scheduler.ReduceLROnPlateau is indeed what you are looking for. I summarized all of the important stuff for you.
mode=min: lr will be reduced when the quantity monitored has stopped decreasing
factor: factor by which the learning rate will be reduced
patience: number of epochs with no improvement after which learning rate will be reduced
threshold: threshold for measuring the new optimum, to only focus on significant changes (change value). Say we have threshold=0.0001, if loss is 18.0 on epoch n and loss is 17.9999 on epoch n+1 then we have met our criteria to multiply the current learning rate by the factor.
criterion = torch.nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min',
    factor=0.1, patience=10, threshold=0.0001, threshold_mode='abs')

for epoch in range(20):
    # training loop stuff
    loss = criterion(...)
    scheduler.step(loss)

You can check more details in the documentation: https://pytorch.org/docs/stable/optim.html#torch.optim.lr_scheduler.ReduceLROnPlateau
