Leveraging Long and Short-term Information in Content-aware Movie Recommendation
Wei Zhao Haixia Chai Benyou Wang Jianbo Ye Min Yang Zhou Zhao Xiaojun Chen
cs.IR










  SIAT, Chinese Academy of Sciences

wei.zhao@siat.ac.cn
  


  Tencent

haixiachai,wabywang@tencent.com



  Pennsylvania State University

jxy198@ist.psu.edu


Min Yang is the corresponding author.

  SIAT, Chinese Academy of Sciences

min.yang@siat.ac.cn



  Zhejiang University

zhaozhou@zju.edu.cn



  Shenzhen University

xjchen@szu.edu.cn





Movie recommendation systems provide users with ranked lists of movies based on individual's preferences and constraints. 
Two types of models are commonly used to generate ranking results: long-term models and session-based models. While long-term models represent the interactions between users and movies that are supposed to change slowly across time, session-based models encode the information of users' interests and changing dynamics of movies' attributes in short terms. 
In this paper, we propose an LSIC model, leveraging Long and Short-term Information in Content-aware movie recommendation using adversarial training. 
In the adversarial process, we train a generator as an agent of reinforcement learning which recommends the next movie to a user sequentially. We also train a discriminator which attempts to distinguish the generated list of movies from the real records. The poster information of movies is integrated to further improve the performance of movie recommendation, which is specifically essential when few ratings are available. The experiments demonstrate that the proposed model has robust superiority over competitors and sets the state-of-the-art. We will release the source code of this work after publication. 










<ccs2012>
 <concept>
  <concept_id>10010520.10010553.10010554</concept_id>
  <concept_desc>Recommender system Movie Recommendation</concept_desc>
  <concept_significance>100</concept_significance>
 </concept>
</ccs2012>  

[100]Recommender system Movie Recommendation







Leveraging Long and Short-term Information in Content-aware Movie
Recommendation
    Xiaojun Chen
    December 30, 2023
================================================================================






§ INTRODUCTION

With the sheer volume of online information, much attention has been given to data-driven recommender systems.  Those systems automatically guide users to discover products or services respecting their personal interests from a large pool of possible options. 
Numerous recommendation techniques have been developed. Three main categories of them are: collaborative filtering methods, content-based methods and hybrid methods  <cit.>. In this paper, we aim to develop a method producing a ranked list of n movies to a user at a given moment (top-n movie recommendation) by exploiting both historical user-movie interactions and the content information of movies.

Matrix factorization (MF) <cit.> is one of the most successful techniques in the practice of recommendation due to its simplicity, attractive accuracy and scalability. It has been used in a broad range of applications such as recommending movies, books, web pages, relevant research and services. The matrix factorization technique is usually effective because it discovers the latent features underpinning the multiplicative interactions between users and movies. Specifically, it models the user preference matrix approximately as a product of two lower-rank latent feature matrices representing user profiles and movie profiles respectively. 


Despite the appeal of matrix factorization, this technique does not explicitly consider the temporal variability of data <cit.>. Firstly, the popularity of an movie may change over time. For example, movie popularity booms or fades, which can be triggered by external events such as the appearance of an actor in a new movie. Secondly, users may change their interests and baseline ratings over time. For instance, a user who tended to rate an average movie as “4 stars”, may now rate such a movie as “3 stars”. Recently, recurrent neural network (RNN) <cit.> has gained significant attention by considering such temporal dynamics for both users and movies and achieved high recommendation quality <cit.>. The basic idea of these RNN-based methods is to formulate the recommendation as a sequence prediction problem. They take the latest observations as input, update the internal states and make predictions based on the newly updated states. As shown in <cit.>, such prediction based on short-term dependencies is likely to improve the recommendation diversity.

More recent work <cit.> reveals that matrix factorization based and RNN-based recommendation approaches have good performances for the reasons that are complementary to each other. Specifically, the matrix factorization recommendation approaches make movie predictions based on users' long-term interests which change very slowly with respect to time. On the contrary, the RNN recommendation approaches predict which movie will the user consume next, respecting the dynamics of users' behaviors and movies' attributes in the short term. It therefore motivates us to devise a joint approach that takes advantage of both matrix factorization and RNN, exploiting both long-term and short-term associations among users and movies.



Furthermore, most existing recommender systems  take into account only the users' past behaviors when making recommendation.  compared with tens of thousands of movies in the corpus, the historical rating set is too sparse to learn a well-performed model.  It is desirable to exploit the content information of movies for recommendation.  For example,  movie
posters reveal a great amount of information
to understand movies and users, as demonstrated in <cit.>. 
Such a poster is usually the first contact that a user has with a movie, and plays 
an essential role in the  user's decision to watch it or not. 
When a user is watching the movie presented in cold, blue and
mysterious visual effects, he/she may be interested in receiving
recommendations for movies with similar styles, rather
than others that are with the same actors or subject <cit.>. 
These visual features of  movies are usually captured by the corresponding posters.









In this paper, we propose a novel LSIC model, which leverages Long and Short-term Information in Content-aware movie recommendation using adversarial training. 
The LSTC model employs an adversarial framework to combine the MF and RNN based models for the top-n movie recommendation, taking the best of each to improve the final recommendation performance. In the adversarial process, we simultaneously train two models: a generative model G and a discriminative model D. In particular, the generator G takes the user u_i and time t as input, and predicts the recommendation list for user i at time t based on the historical user-movie interactions. We implement the discriminator D via a siamese network that incorporates long-term and session-based ranking model in a pair-wise scenario. The two point-wise networks of siamese network share the same set of parameters. The generator G and the discriminator D are optimized with a minimax two-player game. The discriminator D tries to distinguish the real high-rated movies in the training data from the  recommendation list generated by the generator G, while the training procedure of generator G is to maximize the probability of D making a mistake. Thus, this adversarial process can eventually adjust G to generate plausible and high-quality recommendation list. 
In addition, we integrate poster information of movies to further improve the performance of movie recommendation, which is specifically essential when few ratings are available. 


We summarize our main contributions as follows:


  * To the best of our knowledge, we are the first to use GAN framework to leverage the MF and RNN approaches for top-n recommendation. This joint model adaptively adjusts how the contributions of the long-term and short-term information of users and movies are mixed together. 

  * We propose hard and soft mixture mechanisms to integrate MF and RNN. We use the hard mechanism to calculate the mixing score straightforwardly and explore several soft mechanisms to learn the temporal dynamics with the help of the long-term profiles.

  * Our model uses reinforcement learning to optimize the generator G for generating highly rewarded recommendation list. Thus, it effectively bypasses the non-differentiable task metric issue by directly performing policy gradient update. 

  * We automatically crawl the posters of the given movies, and explore the potential of integrating poster information to improve the accuracy of movie recommendation. The release of the collected posters would push forward the research of integrating content information in movie recommender systems. 

  * To verify the effectiveness of our model, we conduct extensive experiments on two widely used real-life datasets: Netflix Prize Contest data and Movielens data. The experimental results demonstrate that our model consistently outperforms the state-of-the-art methods. 


The rest of the paper is organized as follows. In Section 2, we review the related work on recommender systems. Section 3 presents the proposed adversarial learning framework for movie recommendation in details. In Section 4, we describe the experimental data, implementation details, evaluation metrics and baseline methods. The experimental results and analysis are provided in Section 5. Section 6 concludes this paper.



Our model naturally incorporates the framework of generative adversarial network (GAN) by ...... 

The  MF component tries to build a global profiles for both users and movies, while the temporal RNN component models the session-based information for users' short-term interests and movies' changing profile over time. This joint model could adaptively adjust the mixture between the long-term and short-term profiles in the data-driven training. 

Movie recommendation focuses on how to infer the users' preference over large amount of items via past user-item interactions or user/item profiles. Previous CF related works (like some recommendation contests) usually optimized for minimizing root mean squared error (RMSE) to fit the possible user-item pairs. However, the RMSE-orient algorithms do not necessarily perform as expected in top-N recommendation task for each users <cit.>. Namely, we can not provide accurate top-N items for a user even we have gotten a good rating prediction over given items.

Rather than RMSE-orient algorithms, Rank-orient algorithms is alternate approach  for Top-N Recommendation Tasks, which tries to predict all unlabeled user-item pairs by collaborative filtering and to optimize the rank-based metric that is more sensitive to the high-position items.
However, the current Rank-orient CF algorithms is just filling the time-unaware user-item rating matrix.ranked the candidate movies in a time-unaware manner. Firstly, it treats all rating in individual time without distinction, although it is indeed a temporal behavior and the profiles of user (and even item) are evolving along with user-item interactions or changing contexts. Secondly, the train/test data splitting is arbitrary, for example, it even uses a future user-item rating records to infer the previous ones. Generally speaking, time-unaware approaches indeed  simplify this task and ignore the unbalanced train-/test dataset biases . But the real-time task is obviously  temporal.

In order to bridge this gap, we propose a dual structure to jointly train the Matrix factorization and RNN-based temporal model. Both two models are learning a representation of user/item in a siamese  structure <cit.> please add some citations, WEI . The  MF sub-component tries to build a global profiles for both users and items, while the Temporal RNN sub-component models the session-based information for users' short-term interests and items' changing profile over time. This joint model could adaptively adjust the mixture between the long-term and short-term profiles in the data-driven training.

We summarize our main contributions as follows:


  * We firstly propose a long-term and session-based hybrid to provide users with a ranking or recommendation list instead of concerning on the accuracy of rating prediction<cit.> given the past behavioral trajectories.
To the best our knowledge, it is the first try to incorporate the temporal information in the top-N recommendation. In which global static profiles and short-term interests are jointly taken into consideration in an end-to-end way. This framework can be easily extended with any complexed models and extern features, whenever


  * We provide a unified explanation of thinking in ranking inference and pairwise ranking model with adversarial network so that learn each other iteratively, inspired by <cit.> in IR community. We treat generator G as ranking inference network to fool the discriminator to generate(predicts) the well-rated and highly rewarded recommendation list, and treat discriminator D as Siamese Network in pairwise scenario to distinguish real high-rated items on training data from ranking list generated(predicted) by G.
We provide a unified thinking between the inference and its instant adversarial feedback. A extern discriminator D is proposed to fine-tune our predicting models (denoted as Generator G). 


  * We attempt to deal with cold-start issue using long-term and session-based hybrid as in the <cit.> and develop this approach into GANs framework so that further tackle this problem.


  * To verify the effectiveness of our model, we use the well-known dataset from the Netflix Prize Contest and Movielens to evaluate our model. The experimental results show that our model consistently outperforms previous methods. Also, we conduct our experiment with real item recommendation dataset, called QZone, which have millions of posted short-videos.


it should be clarified about the difference between our models with RRN and irgan





§ RELATED WORK

Recommender system is an active research field.
The authors of <cit.>
describe most of the existing techniques for recommender systems.
In this section, we briefly review the following major approaches
for recommender systems that are mostly related to our work. 



  
Matrix factorization for recommendation
Modeling the long-term interests of users, the matrix factorization method and
its variants have grown to become dominant in the literature <cit.>.
In the standard matrix factorization, the recommendation task can
be formulated as inferring missing values of a partially observed
user-item matrix <cit.>.
The Matrix Factorization techniques are effective because they are designed to discover the latent
features underlying the interactions between users and items. 
 srebro2005maximum suggested the Maximum Margin Matrix
Factorization (MMMF), which used low-norm instead of low-rank factorizations.
 mnih2008probabilistic presented the Probabilistic
Matrix Factorization (PMF) model that characterized the user preference
matrix as a product of two lower-rank user and item matrices. The
PMF model was especially effective at making better predictions for
users with few ratings.  he2016fast proposed a new
 MF method which considers the implicit feedback for on-line recommendation.
In <cit.>, the weights of the missing data were assigned based on the popularity of items. 

To exploit the content of items and solve the sparse issue in recommender systems, <cit.> presented the model for movie recommendation using additional visual features (e.g.. posters and still frames) to better understand movies, and further improved the performance of movie recommendation.  




  
Recurrent neural network for recommendation

These traditional MF methods for recommendation systems are based on
the assumption that the user interests and movie attributes are near static,
which is however not consistent with reality.  koren2010collaborative
discussed the effect of temporal dynamics in recommender systems
and proposed a temporal extension of the SVD++ (called TimeSVD++)
to explicitly model the temporal bias in data. However, the features
used in TimeSVD++ were hand-crafted and computationally expensive to obtain. Recently,
there have been increasing interests in employing recurrent neural
network to model temporal dynamic in recommendation systems. For example,  hidasi2015session applied recurrent
neural network (i.e. GRU) to session-based recommender systems. This work
treats the first item a user clicked as the initial input of GRU.
Each follow-up click of the user would then trigger a recommendation
depending on all of the previous clicks.  wu2016recurrent
proposed a recurrent neural network to perform the time heterogeneous
feedback recommendation.  wu2017recurrent used LSTM autoregressive model for the user and movie dynamics and employed matrix factorization to model
the stationary components that encode fixed properties. Different from their work, 
we use GAN framework to leverage the MF and RNN approaches for top-n recommendation, 
aiming to generate plausible and high-quality recommendation lists.  
To address the cold start problem in recommendation,  <cit.>  presented a visual and textural recurrent neural network (VT-RNN), which simultaneously learned the sequential latent vectors of user’s interest and captured the content-based representations that contributed
to address the cold start.




  
Generative adversarial network for recommendation

In parallel, previous work has demonstrated the effectiveness of generative
adversarial network (GAN) <cit.> in various
tasks such as image generation <cit.>,
image captioning <cit.>, and sequence generation<cit.>.
The most related work to ours is <cit.>, which proposed
a novel IRGAN mechanism to iteratively optimize a generative retrieval
component and a discriminative retrieval component. IRGAN
reported impressive results on the tasks of web search, item recommendation,
and question answering. Our approach differs from theirs
in several aspects. First, we combine the MF approach and the RNN
approach with GAN, exploiting the performance contributions of both approaches. 
Second, IRGAN does not attempt to estimate the future behavior since the experimental data is split randomly in their setting. In fact, they use future trajectories to infer the historical records, which seems not useful in real-life applications. 
Third, we incorporate poster information of movies to deal with the cold-start issue and boost the recommendation performance. 

Moreover, we found ranking performance can be skewed by few popular items and splitting dataset randomly would be not serious to bias accuracy metrics as proposed in<cit.>, but it is impractical in the real-world. To further improve the performance, we provide Siamese Network that independently learn a representation for each user and candidate item and then maximum the distance between the two estimated representations with a margin constraint in pair-wise scenario.







§ OUR MODEL




Suppose there is a sparse user-movie rating matrix R that consists of U users and M movies. Each entry r_ij,t denotes the rating of user i on movie j at time step t. The rating is represented by numerical values from 1 to 5, where the higher value indicates the stronger preference. 

Instead of predicting the rating of a specific user-movie pair as is done in <cit.>, the proposed LSIC model aims to provide users with ranked lists of movies (top-n recommendation) <cit.>. 

In this section, we elaborate each component of LSIC model for content-aware movie recommendation.  
The main notations of this work are summarized in Table <ref> for clarity. 
The LSTC model employs an adversarial framework to combine the MF and RNN based models for the top-n movie recommendation. The overview of our proposed architecture and its data-flow are illustrated in Figure <ref>.  In the adversarial process, we simultaneously train two models: a generative model G and a discriminative model D. 







 §.§ Matrix Factorization (MF)
 
The MF framework <cit.> models the long-term states (global information) for both users  (𝐞^u) and movies (𝐞^m). In its standard setting, the recommendation task can be formulated as inferring missing values of a partially observed user-movie rating matrix R. 

The formulation of MF is given by:

    _𝐞^u,𝐞^m∑_i,j I_ij(r_ij-ρ((𝐞^u_i)^T 𝐞^m_j))^2
    +λ^u𝐞^u_F^2 +λ^m𝐞^m_F^2

where 𝐞^u_i and 𝐞^m_j represent the user and movie latent factors in the shared d-dimension space respectively. r_ij denotes the user i's rating on movie j. I_ij is an indicator function and equals 1 if r_ij>0, and 0 otherwise. λ^u and λ^m are regularization coefficients. The ρ(·) is a logistic scoring function that bounds the range of outputs.

In most recommender systems, matrix factorization techniques <cit.> recommend movies based on estimated ratings. Even though the predicted ratings can be used to rank the movies, it is known that it does not provide the best prediction for the top-n recommendation. Because minimizing the objective function – the squared errors – does not perfectly align with the goal to optimize the ranking order. In this paper, we apply MF for ranking prediction (top-n recommendation) directly, similar to <cit.>.  



 §.§ Recurrent Neural Network (RNN)
 
The RNN based recommender system focuses on modeling session-based trajectories instead of global (long-term) information <cit.>. It predicts future behaviors and provides users with a ranking list given the users' past history. The main purpose of using RNN is to capture time-varying state for both users and movies. 
Particularly, we use LSTM cell as the basic RNN unit. Each LSTM unit at time t consists of a memory cell c_t, an input gate i_t, a forget gate f_t, and an output gate o_t. These gates are computed from previous hidden state 𝐡_t-1 and the current input 𝐱_t:

    [f_t,i_t,o_t]    = sigmoid(W[𝐡_t-1, 𝐱_t])


The memory cell c_t is updated by partially forgetting the existing memory and adding a new memory content 𝐥_t:


    𝐥_t   =tanh(V[𝐡_t-1, 𝐱_t])
    𝐜_t   =f_t⊙𝐜_t-1+i_t⊙𝐥_t



Once the memory content of the LSTM unit is updated, the hidden state at time step t is given  by:

    𝐡_t   =o_t⊙tanh(𝐜_t)

For simplicity of notation, the update of the hidden states of LSTM at time step t is denoted as 
𝐡_t= LSTM(𝐡_t-1, 𝐱_t).

Here, we use 𝐳^u_i,t∈ℝ^U and 𝐳^m_j,t∈ℝ^M to represent the rating vector of user i and movie j given time t respectively. Both 𝐳^u_i,t and 𝐳^m_j,t serve as the input to the LSTM layer at time t to infer the new states of the user and the movie:



    𝐡^u_i,t   =  LSTM(𝐡^u_i,t-1,  𝐳^u_i,t)
    𝐡^m_j,t   =  LSTM(𝐡^m_j,t-1, 𝐳^m_j,t)


Here,  𝐡^u_i,t and 𝐡^m_j,t denote hidden states for user i and movie j at time step t respectively. 

In this work, we explore the potential of integrating posters of movies to boost the performance of movie recommendation. Inspired by the recent advances of CNNs in computer vision,  the poster is mapped to the same space of the movie  by using a CNN. More concretely, we encode each image into a FC-2k feature vector with Resnet-101 (101 layers), resulting in a 2048-dimensional vector representation. The poster P_j of movie j is only inputted once, at t=0, to inform the movie LSTM about the poster content: 


    𝐳^m_j,0=CNN(P_j).









 §.§ RNN and MF Hybrid

The session-based model deals with temporal dynamics of the user and movie states, we further incorporate the long-term preference of users and the fixed properties of movies. To exploit their advantages together, similar to <cit.>, we define the rating prediction function as:

    r_ij,t   = g(𝐞^u_i, 𝐞^m_j, 𝐡^u_i,t, 𝐡^m_j,t)

where g(.) is a score function, 𝐞^u_i and 𝐞^m_j denote the global latent factors of user i and movie j learned by Eq.  (<ref>); 𝐡^u_i,t and 𝐡^m_j,t denote the hidden states at time step t of two RNNs learned by Eq. (<ref>) and Eq. (<ref>) respectively.
In this work, we study four strategies to calculate the score function g, integrating MF and RNN. The details are described below. 
















 





  
LSIC-V1
This is a hard mechanism, using a simple way to calculate the mixing score from MF and RNN with the following formulation:


    r_ij,t   =g(𝐞^u_i, 𝐞^m_j, 𝐡^u_i,t, 𝐡^m_j,t)=1/1+ exp(-𝐬)
    𝐬   =𝐞^u_i·𝐞^m_j + 𝐡^u_i,t·𝐡^m_j,t +b^u_i+b^m_j


where b^u_i and b^m_j are the biases of user i and movie j; 𝐡^u_i,t and 𝐡^m_j,t are computed by Eq. (<ref>) and Eq. (<ref>). 

In fact, LSIC-V1 does not exploit the global factors in learning the temporal dynamics. In this paper, we also design a soft mixture mechanism and provide three strategies to account for the global factors 𝐞^u_i and 𝐞^m_j in learning 𝐡^u_i,t and 𝐡^m_j,t, as described below (i.e., LSIC-V2, LSIC-V3 and LSIC-V4). 



  
LSIC-V2
We use  the latent factors of user i (𝐞^u_i) and movie j  (𝐞^m_j) pre-trained by MF model  to initialize the hidden states of the LSTM cells 𝐡^u_i,0 and 𝐡^m_j,0 respectively, as depicted in Figure <ref>(b). 



  
LSIC-V3
As shown in Figure <ref>(c), we extend  LSIC-V2 by treating 𝐞^u_i (for user i) and 𝐞^m_j (for movie j)  as the static context vectors, and feed them as an extra input into the computation of the temporal hidden states of users and movies by LSTM. At each time step, the context information assists the inference of the hidden states of LSTM model. 



  
LSIC-V4
This method use an attention mechanism to compute a weight for each hidden state by exploiting the global factors. The mixing scores at time t can be reformulated by:


    r_ij,t   =g(𝐞^u_i, 𝐞^m_j, 𝐡^u_i,t-1, 𝐡^m_j,t-1, 𝐜^u_i,t, 𝐜^m_j,t)=1/1+ exp(-𝐬)
    𝐬   =𝐞^u_i·𝐞^m_j + 𝐡^u_i,t·𝐡^m_j,t +b_i+b_j


where 𝐜^u_i,t and 𝐜^m_j,t are the context vectors at time step t for user i and movie j; 𝐡^u_i,t and 𝐡^m_j,t are the hidden states of LSTMs at time step t, computed by 



    𝐡^u_i,t   =  LSTM(𝐡^u_i,t-1, 𝐳^u_i,t, 𝐜^u_i,t)
    𝐡^m_j,t   =  LSTM(𝐡^m_j,t-1, 𝐳^m_j,t, 𝐜^m_j,t)



The context vectors 𝐜^u_i,t and 𝐜^m_j,t act as extra input in the computation of the hidden states in LSTMs to make sure that every time step of the LSTMs can get full information of the context (long-term information). 
The context vectors 𝐜^u_i,t and 𝐜^m_j,t are the dynamic representations of the relevant long-term information for user  i and movie j at time t, calculated by 



    𝐜^u_i,t=∑_k=1^Uα^i_k,t𝐞^u_k ;  𝐜^m_j,t=∑_p=1^Mβ^j_p,t𝐞^m_p

where U and M are the number of users and movies. The attention weights α^i_k,t and β^j_p,t for user i and movie j  at time step t are computed by 



    α^i_k,t=exp(σ(𝐡^u_i,t-1, 𝐞^u_k))/∑_k'=1^U exp(σ(𝐡^u_i,t-1, 𝐞^u_k'))
    β^j_p,t=exp(σ(𝐡^m_j,t-1, 𝐞^m_p))/∑_p'=1^M exp(σ(𝐡^m_j,t-1, 𝐞^m_p'))


where  σ is a feed-forward neural network to produce a real-valued score. The attention weights α^i_t and β^j_t  together determine which user and movie factors should be selected to generate r_ij,t. 






 §.§ Generative Adversarial Network (GAN) for Recommendation
 

We introduce the architecture of long-term and session-based ranking model in the pair-wise scenario. It learns a scoring function s(u_i,m_j|t;θ) for user i on movie j given time t. During the inference, we treat the trained model as a point-wise scoring function to score user i on all movies given time t, inspired by <cit.> in IR community. For this reason, we provides a unified explanation of thinking in ranking inference and pairwise ranking model with adversarial network shown as Figure 1.

Generative adversarial network (GAN)<cit.> consist of a generator G and a discriminator D that compete in a  minimax game with two players: The discriminator tries to distinguish real high-rated movies on training data from ranking or recommendation list predicted by G, and the generator tries to fool the discriminator to generate(predict) well-ranked recommendation list. Concretely, D and G play the following game on V(D,G):


    min_Gmax_Dv(D,G)=   𝔼_x ∼ P_true(x)[ log D(x)] + 
       𝔼_z∼ P_z(z)[ log(1-D(G(z)))]

Here, x is the input data from training set, z is the noise variable sampled from normal distribution. 

We propose an  adversarial framework to iteratively optimize two models: the generative model G predicting recommendation list given historical user-movie interactions, and the discriminative model D predicting the relevance of the generated list. Like the standard generative adversarial networks (GANs) <cit.>, our model also optimizes the two models with a minimax two-player game.  D tries to distinguish the real high-rated movies in the training data from the recommendation list generated by G, while G maximizes the probability of D making a mistake. Hopefully, this adversarial process can eventually adjust G to generate plausible and high-quality recommendation list. We further elaborate the generator and discriminator below.







  §.§.§ Discriminate Model
 
As depicted in Figure 1 (right side), we implement the discriminator D via a Siamese Network that incorporates long and session-based ranking models in a pair-wise scenario. The discriminator D has two symmetrical point-wise networks that share parameters and are updated by minimizing a pair-wise loss. 

The objective of discriminator D is to maximize the probability of correctly distinguishing the ground truth movies from generated recommendation movies. For G fixed, we can obtain the optimal parameters for the discriminator D with the following formulation.




    θ^*= _θ∑_i ∈𝒰(𝔼_m_+,m_-∼ p_true[log D_θ(u_i,m_-,m_+|t)]+ 
    𝔼_m_+∼ p_true,m_g,t∼ G_ϕ(m_g,t|u_i,t)[log(1-D_θ(u_i,m_g,t,m_+|t)])

where 𝒰 denotes the user set, u_i denotes user i, m_+ is a positive (high-rating) movie, m_- is a negative movie randomly chosen from the entire negative (low-rating) movie space, θ and ϕ  are parameters of D and G, and m_g,t is the generated movie by  G given time t. Here, we adopt hinge loss as our training objective since  it performs better than other training objectives. Hinge loss is widely adopted in various learning to rank scenario, which aims to penalize the examples that violate the margin constraint: 


    D(u_i,m_-, m_+|t)=max{0,ϵ - 
    g(𝐞^u_i, 𝐞^m_m_+, 𝐡^u_i,t, 𝐡^m_m_+,t)
    
    +g(𝐞^u_i, 𝐞^m_m_-, 𝐡^u_i,t, 𝐡^m_m_-,t)}

where ϵ is the hyper-parameter determining the margin of hinge loss, and we compress the outputs to the range of (0,1).





  §.§.§ Generative Model
 




Similar to conditional GANs proposed in <cit.>, our generator G takes in the auxiliary information (user u_i and time t) as input, and generates the ranking list for user i. Specifically, when D is optimized and fixed after computing Eq. <ref>, the generator G can be optimized by minimizing the following formulation:


    ϕ^* = _ϕ∑_m ∈ℳ(𝔼_m_g,t∼ G_ϕ(m_g,t|u_i,t)[ log(1-D(u_i,m_g,t,m_+|t)])

Here, ℳ denotes the movie set. As in <cit.>, instead of minimizing log(1-D(u_i,m_g,t,m_+|t)), we train G to maximize log(D(u_i,m_g,t,m_+|t)).  



  §.§.§ Policy Gradient

Since the sampling of recommendation list by generator G is discrete, it cannot be directly optimized by gradient descent as in the standard GAN formulation. Therefore, we use policy gradient based reinforcement learning algorithm <cit.> to optimize the generator G  so as to generate highly rewarded recommendation list. Concretely, we have the following derivations:


    ∇_ϕ J^G(u_i)   =∇_ϕ𝔼_m_g,t∼ G_ϕ(m_g,t|u_i,t)[log D(u_i,m_g,t,m_+|t)] 
       =∑_m ∈ℳ∇_ϕG_ϕ(m|u_i,t)log D(u_i,m,m_+|t)
       =∑_m ∈ℳ G_ϕ(m|u_i,t)∇_ϕlog G_ϕ(m|u_i,t)log D(u_i,m,m_+|t)
       =𝔼_m_g,t∼ G_ϕ(m|u_i,t)[∇_ϕ logG_ϕ(m_g,t|u_i,t)log D(u_i,m_g,t,m_+|t)]
       ≈1/K∑_k=1^K∇_ϕ logG_ϕ(m_k|u_i,t)log D(u_i,m_k,m_+|t)


where K is number of movies sampled by the current version of generator and  m_k is the k-th sampled  item. With reinforcement learning terminology, we treat the term  logD(u_i,m_k,m_+|t) as the reward at time step t, and take an action m_k at each time step. To accelerate the convergence, the rewards within a batch are normalized with a  Gaussian distribution to make the significant differences. 










The overall procedure is summarized in Algorithm 1. During the training stage, the discriminator and the generator are trained alternatively in a adversarial manner via Eq.(<ref>) and Eq.(<ref>), respectively.



§ EXPERIMENTAL SETUP




 §.§ Datasets



In order to evaluate the effectiveness of our model, we conduct experiments on two widely-used real-life datasets: Movielens100K  and Netflix (called “Netflix-Full”). To evaluate the robustness of
our model, we also conduct experiments on a 3-month Netflix (called “Netflix-3M”) dataset, which is a small version of Netflix-Full and has different training and testing period.
For each dataset, we split the whole data into several training and testing intervals based on time, as is done in <cit.>, to simulate the actual situation of predicting future behaviors of users given the data that occurred strictly before current time step. Then, each testing interval is randomly divided into a validation set and a testing set. We removed the users and movies that do not appear in training set from the validation and test sets.  The detailed statistics are presented in Table <ref>[“Density” shows the average number of 5-ratings for the user per day. “Sparsity” shows the filling-rate of user-movie rating matrix as used in <cit.>].  Following <cit.>, we treat “5-star” in Netflix, “4-start” and “5-start” for  Movielens100K as positive feedback and all others as unknown (negative) feedback.









































 §.§ Implementation Details





  
Matrix Factorization. We use matrix factorization with 5 and 16 factor numbers for Movielens and Netflix respectively <cit.>. The parameters are randomly initialized by a uniform distribution ranged in [-0.05, 0.05].  We take gradient-clipping to suppress the gradient to the range of [-0.2,0.2]. L2 regularization (with λ = 0.05)  is used to the weights and biases of user and movie factors. 



  
Recurrent Neural Network.  We use a single-layer LSTM with 10 hidden neurons,  15-dimensional input embeddings, and 4-dimensional dynamic states where each state contains 7-days  users/movies behavioral trajectories. That is, we take one month as the length of a session. The parameters are initialized with the same way as in MF. L2 regularization  (with λ = 0.05) is used to the weights and biases of the LSTM layer to avoid over-fitting.



  
Generative Adversarial Nets. We pre-train G and D on the training data with a pair-wise scenario, and  use  SGD algorithm with learning rate 1 × 10^-4 to optimize its parameters. The number of sampled movies is set to 64 (i.e., K=64). In addition, we use matrix factorization model to generate 100 candidate movies, and then re-rank these movies with LSTM.  In all experiments, we conduct mini-batch training with batch size 128.




 §.§ Evaluation Metrics

To quantitatively evaluate our method, we adopt the rank-based evaluation metrics to measure the performance of top-n recommendation <cit.> , including Precision@N, Normalised Discounted Cumulative Gain (NDCG@N), Mean Average Precision (MAP) and Mean Reciprocal Ranking (MRR).



 §.§ Comparison to Baselines

In the experiments, we evaluate and compare our models with several state-of-the-art methods.












































































































        







































  
Bayesian Personalised Ranking (BPR) Given a positive movie, BPR uniformly samples negative movies to resolve the imbalance issue and provides a basic baseline for top-N recommendation  <cit.>.



  
Pairwise Ranking Factorization Machine (PRFM)  This is one of the state-of-the-art movie recommendation algorithms, which applies Factorization Machine model to
microblog ranking on basis of pairwise classification <cit.>.  We use the same settings as in <cit.> in our experiments. 



  
LambdaFM It is a strong baseline for recommendation, which directly optimizes the rank biased metrics <cit.>.  We run the LambdaFM model with the publicly available code[https://github.com/fajieyuan/LambdaFM], and use default settings for all hyperparameters. 



  
Recurrent Recommender Networks (RRN)  This model supplements matrix factorization with recurrent neural network model via a hard mixture mechanism <cit.>. We use the same setting as in <cit.>. 



  
IRGAN This model trains the generator and discriminator alternatively with MF in an adversarial process  <cit.>. 

We run the IRGAN model with the publicly available code[https://github.com/geek-ai/irgan], and use default settings for all hyperparameters. 



§ EXPERIMENTAL RESULTS

In this section, we compare our model with baseline methods quantitatively and qualitatively. 



 §.§ Quantitative Evaluation


We first evaluate the performance of top-n recommendation. The experimental results are summarized in Tables <ref> ,<ref> and <ref>. Our model substantially and consistently outperforms the baseline methods by a noticeable margin on all the experimental datasets. In particular, we have explored several versions of our model with different mixture mechanisms. As one anticipates,  LSIC-V4 achieves the best results across all evaluation metrics and all datasets. For example, on MovieLens dataset, LSIC-V4  improves  7.45% on percision@5 and 6.87% on NDCG@5  over the baseline methods. The main strength of our model comes from its capability of prioritizing both long-term and short-term information in content-aware movie recommendation. In addition, our mixture mechanisms (hard and soft) also seem quite effective to integrate MF and RNN.


To better understand the adversarial training process,  we visualize the learning curves of LSIC-V4 as shown in Figure 3. Due to the limited space, we only report the Pecision@5 and NDCG@5 scores  as in <cit.>. The other metrics exhibit a similar trend. As shown in Figure 3, after about 50 epoches, both Precision@5 and NDCG@5 converge and the winner player is the generator which is used to generate recommendation list for our final top-n movie recommendation. 
The performance of generator G becomes better with the effective feedback (reward) from discriminator D. 
On the other hand, once we have a set of high-quality recommendation movies, the performance of D deteriorates gradually in the training procedure and makes mistakes for predictions.
In our experiments, we use the generator G with best performance to predict test data.




 §.§ Ablation Study


In order to analyze the effectiveness of different components of our model for  top-n movie recommendation, in this section, we report the ablation test of LSIC-V4 by discarding poster information (w/o poster)  and replacing the reinforcement learning with Gumbel-Softmax <cit.> (w/o RL), respectively.  Gumbel-Softmax is an alternative method to address the non-differentiation problem so that G can be trained straightforwardly.

Due to the limited space, we only illustrate the experimental results for Netflix-3M dataset that is widely used in movie recommendation (see Table <ref>).  Generally, both factors contribute, and reinforcement learning contributes most. This is within our expectation since discarding reinforcement learning will lead the adversarial learning inefficient. With Gumbel-Softmax, G does not benefit from the reward of D, so that we do not know which movies sampled by G are good and should be reproduced. Not surprisingly, poster information also contributes to movie recommendation. 





 §.§ Case Study

In this section, we will further show the advantages of our models through some quintessential examples.


In Table 7, we provide the recommendation lists generated by three state-of-the-art baseline methods (i.e., IRGAN, RNN, LambdaFM) as well as the proposed  LSIC-V4 model for two users who are randomly selected from the Netflix-3M dataset. Our model can rank the positive movies in higher positions than other methods. For example, the ranking of the movie “9 souls” for user “8003” has increased from 5-th position (by LambdaFM) to 1st position (by LSIC-V4). Meanwhile some emerging movies such as “Session 9” and “The Last Valley” that are truly attractive to the user “1382” have been recommended by our models, whereas they are ignored by baseline methods. In fact, we can include all positive movies in the top-10 list for user “1382” and in  top-3 list for user “8003”. Our model benefits from the fact that both dynamic and static knowledge are incorporated into the model with adversarial training.   


We conduct some case studies in Netflix-Full dataset as shown in the Table <ref>. MF based approach (in the 3rd column)  provides a basic ranking list with the global user/movie profile. Since the incorporation with hard-mixture session-based model in RRN (in the 4rd column), some potential positive movies have been recalled in a higher position (movie named "9 souls" from 9 to 2) due to the complementarity between the global and session-term user/movie profiles. In the same time, some emerging movies (like "9 souls" and "Stuart Saves His Family" for user 1382) have been mined by the extra joint with the session information, which is ignored by a single MF model but is really attractive to the current user. Rather than this hard-mixture joint models, a soft-version one  with a flexible data-driven mechanism achieves a better performance for ranking (with more accurate recall in the 5th column). Finally, GAN mechanism boosts our models in a adversarial training, especially recalling  the positive movies in all the top 5 positions for user 1382 .














 §.§ Cold Start

If we mentioned "cold start", the removal of little-rating users might be argued, we should take care of the related statements
Figure [?] illustrates that long-term and session-based hybrid provides some improvements for both users and movies that have few ratings in the training data as claimed in <cit.>, and then we develop this approach into GANs framework so that gains significant improvements. As we expected, new users and movies doesn't have history in the past, but recently have few behavioral trajectories. However, the long-term model such as (PMF) focuses on long-history preference that is difficult to track the shifts of recent preference for both user and movie. 

Figure :  the difference over users between Gan and NonGAN
Wei prepare the saved models of joint model and its GAN version in Netflix Three mouth , and I will draw the picture







 §.§ Re-rank Effect

From our experiments, we observe that it could be time-consuming for RNN to infer all movies. In addition, users may be interested in a few movies that are subject to a long-tailed distribution. Motivated by these observations, we provide a re-ranking strategy as used in <cit.>. Specifically, we first generate N candidate movies by MF, and then re-rank these candidate movies with our model. In this way, the inference time can been greatly reduced. Figure <ref> illustrates the performance curves over the number of candidate movies (i.e. N) generated by MF. We only report the Pecision@5 and NDCG@5 results due to the limited space, the other metrics  exhibit a similar trend.  As shown in Figure <ref>, when the number of candidate movies is small, i.e., N≤ 100 for Netflix-3M dataset, the Percision@5 raises gradually with the number of candidate movies increases. Nevertheless, the performance drops rapidly with the increasing candidate movies when N≥ 100. It suggests that the candidates in a long-tail side generated by MF is inaccurate and these candidate movies deteriorate the overall performance of the re-rank strategy. 


































































 §.§ Session Period Sensitivity

The above experimental results have shown that the session-based (short-term) information indeed improves the performance of top-N recommendation. We conduct an experiment on Netflix-3M to investigate how the session period influences the final recommendation performance. As shown in Figure <ref>, the bars in purple color (below) describe the basic performance of MF component while the red ones (above) represent the extra improvement by the LSIC-V4. The RNN plays an insignificant role in the very early sessions since it lacks enough historical interaction data. For later period of sessions, RNN component tends to be more effective, and our joint model achieves a clear improvement over the model with only MF component.  











 


 
 

§ CONCLUSION

 In this paper, we proposed a novel adversarial process for top-n recommendation.
Our model incorporated both matrix factorization and recurrent neural network to exploit the benefits of the long-term and short-term knowledge. We also integrated poster information to further improve the performance of movie recommendation.  Experiments on two real-life datasets showed the performance superiority of our model. 
 






ACM-Reference-Format

