Perceptron with weights of bounded condition number
Let N be a (linear) single-layer perceptron with weight matrix w of dimension nxn.
I want to train N under the Boolean constraint that the condition number k(w) of the weights w remain below a given threshold k_0 at each step of the optimisation.
Is there a standard way to implement this constraint (in pytorch, say)?

After each optimizer step, go through the list of parameters and recondition all matrices:
(code looked at for a few seconds, but not tested)
def recondition_(x, max_cond): # would need to be fixed for non-square x
    u, s, vh = torch.linalg.svd(x)
    curr_cond = s[0] / s[-1]
    if curr_cond &gt; max_cond:
        ratio = curr_cond / max_cond
        mult = torch.linspace(0, math.log(ratio), len(s)).exp()
        s = mult * s
        x[:] = torch.mm(u, torch.mm(torch.diag(s), vh))

Training loop:
...
optimizer.step()

with torch.no_grad():
    for p in model.parameters():
        if p.dim() == 2:
            recondition_(p, max_cond)
...

