I am solving the following matrix equation for $\mathbf{x}$:$$(J^{\mathbf{T}}J)\mathbf{x}=J^{\mathbf{T}}\mathbf{r}$$$J$ is $m\times n$ matrix$\mathbf{x}$ is vector of size $n$$\mathbf{r}$ is vector of size $m$Furthermore, $m\gg n$.It is sometimes the case that $n$ is quite large (e.g. 1000) and $J$ contains values like 500. Standard multiplication is then impossible to do even with double-precision numbers as the values exceed $10^{128}$.I found that scaling $J$ by a nonzero value has predictable effect on the result (it has the same effect as multiplying both sides of the above equation by scalars), so I am thinking about computing a scale factor so that the above equation can be solved without overflows. However, too small scale factor can also endanger accuracy of the matrix multiplication.Furthermore, $J$ can contain both very small and very large values.So the question is how to choose some scaling factor $s$ so that we can substitute $J$ by $sJ$ and avoid too big numbers and also keep high accuracy of the multiplication.My ideas:make average/median of elements in $J$ equal to e.g. $10^3$scale $J$ according to sum of its biggest columnfind some preconditioner matrix $A$ and substitute $J$ for $AJ$ ... the preconditioned solution can be obtained using inverse transform, but $A$ can get very large as it has to be $m\times m$Any ideas?
Avoid arithmetic overflow in matrix multiplication
linear algebra;linear solver;stability;preconditioning