How can we define missing rating in recommender system?
For the Collaborative Filtering while using Matrix Factorization you don't necessary use a vanilla SVD over a dense matrix. As for the most cases the input dataset of ratings is vary sparse and your U/I rating matrix will have most of values set to 0.0. So, when SVD is done for input date the final recommendation will suffer of that.

And this was the main breakthrough in the publication of using SGD for MF in Netflix Prize contest in 2006. All is described by the author here. In short - you calculate your lost function only over the seen ratings set, not over the all matrix (as all the zeros or other value actually represent unknown/unobserved cases). Thus, the natural choice is to use sparse matrix for your input data.