Stochastic gradient descent in matrix factorization, sensitive to label's scale?
The larger your target scores, the larger latent variables should be (well, it's not only a magnitude that matters, but also a variance, but it still applies to your case). There's no problem with larger coefficients of latent vectors unless you use regularization (and, likely, you do). In case of regularization your optimal solution will tend towards smaller values, and'd sometimes prefer to sacrifice some accuracy for lower regularization penalty.

Gradient Descent doesn't suffer from problem of large coefficients (unless you run into some sort of numerical issues): if the learning rate is tuned properly (there are lots of stuff on it, google), it should arrive to equivalent parameters. Otherwise nobody guarantees you convergence :-)

The common rule of thumb when doing regression (and your instance of matrix factorization is a kind of regression) is to standardize your data: make it having zero mean and unit variance.