Matrix Factorisation Improvement
In my view, you should better go for user-movie matrix, to create a sparse matrix that can be more meaningful. By doing that, if you already know polarity of choices i.e. positive, negative or neutral, you may also have some intensity score of like or dislike like in case of rating. Now, if you want to just classify them as likes, neutral, and dislike, you may assign 1,0 and -1 respectively. This method can have more meaning when you have many movies and many users. Further this can help you establish user-user, movie-user and movie-movie relationship as well. These relationship can potentially help you understand and explore more about recommendation engines.

     m1 m2 m3 m4
u1 -1   0  -1  1
u2  1   1  -1  0
u3  0   0   1  0
u4  -1  1   1   0


If you want to add rating instead of just polarity, just add numbers scaled 0-1. I personally worked on them to write hybrid recommendation engines which have both collaborative filtering and context based filtering models embedded. I would suggest you to work more on relationships and try to explore more about next steps like possibility of like or dislike bias, association of movie info/ actors-directors influence and similarity in movies etc. I hope this explanation helps.