Use PyTorch to speed up linear least squares optimization with bounds?
I'm using scipy.optimize.lsq_linear to run some linear least squares optimizations and all is well, but a little slow.  My A matrix is typically about 100 x 10,000 in size and sparse (sparsity usually ~50%).  The bounds on the solution are critical.  Given my tolerance lsq_linear typically solves the problems in about 10 seconds and speeding this up would be very helpful for running many optimizations.
I've read about speeding up linear algebra operations using GPU acceleration in PyTorch.  It looks like PyTorch handles sparse arrays (torch calls them tensors), which is good.  However, I've been digging through the PyTorch documentation, particularly the torch.optim and torch.linalg packages, and I haven't found anything that appears to be able to do a linear least squares optimization with bounds.

Is there a torch method that can do linear least squares optimization with bounds like scipy.optimize.lsq_linear?
Is there another way to speed up lsq_linear or to perform the optimization in a faster way?

For what it's worth, I think I've pushed lsq_linear pretty far.  I don't think I can decrease the number of matrix elements, increase sparsity or decrease optimiation tolerances much farther without sacrificing the results.

Not easily, no.
I'd try to profile lsq_linear on your problem to see if it's pure python overhead (which can probably be trimmed some) or linear algebra. In the latter case, I'd start with vendoring the lsq_linear code and swapping relevant linear algebra routines. YMMV though.
