DBRec: Dual-Bridging Recommendation via Discovering Latent Groups
Jingwei Ma Jiahui Wen Mingyang Zhong Liangchen Liu Chaojie Li Weitong Chen Yin Yang Honghui Tu Xue Li



  











  The University of Queensland
jingwei.ma@uq.edu.au


Corresponding author

  National University of Defense Technology
wen_jiahui@outlook.com



  Central Queensland University; Chongqing Meiqi Industry Co.
m.zhong@cqu.edu.au



 The University of Queensland
l.liu9@uq.edu.au
 


  Aliexpress, Alibaba group
  Hang Zhou
  Zhe Jiang
  311121
Cjlee.cqu@163.com



  The University of Queensland
w.chen9@uq.edu.au


Hamad Bin Khalifa University
yyang@hbku.edu.qa


National University of Defense Technology
tuhkjet@foxmail.com


The University of Queensland
xueli@itee.uq.edu.au









In recommender systems, the user-item interaction data is usually sparse and not sufficient for learning comprehensive user/item representations for recommendation. To address this problem, we propose a novel dual-bridging recommendation model (DBRec). DBRec performs latent user/item group discovery simultaneously with collaborative filtering, and interacts group information with users/items for bridging similar users/items. Therefore, a user's preference over an unobserved item, in DBRec, can be bridged by the users within the same group who have rated the item, or the user-rated items that share the same group with the unobserved item. In addition, we propose to jointly learn user-user group (item-item group) hierarchies, so that we can effectively discover latent groups and learn compact user/item representations. We jointly integrate collaborative filtering, latent group discovering and hierarchical modelling into a unified framework, so that all the model parameters can be learned toward the optimization of the objective function. We validate the effectiveness of the proposed model with two real datasets, and demonstrate its advantage over the state-of-the-art recommendation models with extensive experiments.







<ccs2012>
 <concept>
  <concept_id>10010520.10010553.10010562</concept_id>
  <concept_desc>Computer systems organization Embedded systems</concept_desc>
  <concept_significance>500</concept_significance>
 </concept>
 <concept>
  <concept_id>10010520.10010575.10010755</concept_id>
  <concept_desc>Computer systems organization Redundancy</concept_desc>
  <concept_significance>300</concept_significance>
 </concept>
 <concept>
  <concept_id>10010520.10010553.10010554</concept_id>
  <concept_desc>Computer systems organization Robotics</concept_desc>
  <concept_significance>100</concept_significance>
 </concept>
 <concept>
  <concept_id>10003033.10003083.10003095</concept_id>
  <concept_desc>Networks Network reliability</concept_desc>
  <concept_significance>100</concept_significance>
 </concept>
</ccs2012>


[500]Computer systems organization Embedded systems
[300]Computer systems organization Redundancy
Computer systems organization Robotics
[100]Networks Network reliability










DBRec: Dual-Bridging Recommendation via Discovering Latent Groups
    Xue Li
    December 30, 2023
=================================================================





§ INTRODUCTION

Due to the prevalence of the Internet, recommender systems have attracted remarkable attentions in industrial and research communities. Recommender systems mainly aim to alleviate the problem of information overload for online services, and explore a tremendous amount of information and recommend interested items for each user. Collaborative Filtering (CF) is one of the most popular recommendation techniques, which leverages users' historical behaviors to collaboratively infer user preferences over items. Among the CF-based methods, Matrix Factorization (MF) is efficient in capturing user preferences and providing superior recommendation performance. The basic idea behinds matrix factorization is to embed each user/item into a low-dimensional vector, and model a user-item rating with the interaction (i.e. inner product) between the corresponding user/item vector. 

Many previous works have developed MF-based methods for modeling user-item interactions. For example, neural matrix factorization <cit.> combines generalize matrix factorization and multiple-layer perceptron for learning the interaction function. The multiple-layer of nonlinear transformations is proved to be efficient in learning user-item interactions, and yields state-of-the-art recommendation performance. ConvNCF <cit.> utilizes outer product to transform each user-item embedding pair into a two-dimensional interaction map, and employs convolution and pooling layers for modeling high-order user-item correlations. <cit.> involves context items (e.g. items rated by users) for modeling user-item interactions, hence user preferences over items are comprised of user-item and item-item similarities. Eventhough the aforementioned methods are shown to be effective, they suffer from the problem of data sparseness, as they are mainly based on user-item interactions, and users usually give few ratings to the items in real scenarios.

To address the data sparseness problem, existing research propose to incorporate additional information for boosting recommendation performance. Some of them <cit.> leverage social networks for bridging similar users, and propagating user preferences along the social links. Some others resort to attribute data of different modalities to mitigate data sparseness. Typical attribute data include user profile <cit.>, descriptive texts <cit.>, images <cit.>, etc. However, these recommender models are tightly coupled with one information source or another, and the requirement of extra information sources limits their scalability. 

In this work, we propose a novel method for boosting recommendation without the requirement of auxiliary data sources. To this end, we discover latent user/item groups and model user-item interactions collaboratively. The discovered latent groups are then interacted with users/items to boost recommendation performance. The rationale underlying DBRec is that, the estimation of a user's rating over an unobserved item, in the proposed model, can be bridged by the users within the same group who have interacted with the item, or the user-rated items that share the same group with the unobserved item. The in-group users and items are abstracted with high-level group representations, and the interactions of user-item group and item-user group result in a dual-bridging architecture. In this light, the dual-bridging mechanism can mitigate data sparseness through bridging similar users/items. Also, we utilize deep neural networks for modeling user-user group (item-item group) hierarchies, so that users/items are forced to be close to their groups for learning compact representations. 

Notice that, in <cit.>, it exploits group information to boost social recommendation. However, the group information is explicitly required, hence those methods cannot be scaled to the scenarios where the group information is not explicitly available. 

The contributions of this work can be concluded as follows, 



  * We propose to seamlessly combine collaborative filtering and latent group discovery for recommendation. The discovered group information can dual-bridge similar users and items for boosting recommendation performance. 



  * We propose to jointly model user-user group (item-item group) hierarchies for learning compact user/item representations. 


  * We validate the effectiveness of the proposed model on two real datasets, and demonstrate its advantage over the state-of-the-art recommendation models with comparison experiments and comprehensive analysis. 






§ RELATED WORK

In this section, we review the representative state-of-the-art CF-based recommender models. 


 §.§ Neural Recommendation

In recently years, deep learning techniques have been widely applied to recommender community. The majority of works employ multiple-layer neural networks to model deep user-item interactions. For example, NeuMF <cit.> combines generalized matrix factorization and multiple-layer perceptron for modeling user-item similarities. ConvMF <cit.> proposes to use outer product to transform embeddings of each user-item pair into a two-dimensional interaction map, and then employs convolutional and pooling layer to model high-order interrelations among embedding dimensions. Cheng et al.cheng2018delf leverage item contexts (i.e. historical items rated by users) to compensate the interaction function. Ebesu et al. ebesu2018collaborative propose an attention to aggregate neighboring users, and jointly exploit the neighborhoods with user-item interactions to derive the recommendation. However, one major shortcoming of those models is that, they are mainly based on the interaction data, and suffer from data sparseness. 



 §.§ Recommendation with Side Information

To alleviate data sparseness and boost recommendation performance, existing works incorporate side information of different modalities for recommendation. The auxiliary information reflects item characteristics, and can bridge similar items (users) for alleviating data sparseness. In most cases, they employ deep neural networks to extract high-level representations for the extra information, and populate them into CF-based methods for recommendation. The extra data sources that are commonly exploited include user profile <cit.>, texts <cit.>, images <cit.> and demands <cit.>. However, the aforementioned methods are usually coupled with one type of information source or another, and lack the scalability when they are deployed for recommendation in different scenarios. Notice that group information has been exploited for group recommendation. For example, in <cit.>, the authors address group recommendation problem, and aggregate representations of group members with an attention mechanism to obtain group representation. However, the group information needs to be specified in the data, and hence those methods are inapplicable in our case. 


In <cit.>, the authors combine latent factor model and neighborhood-based structure, and propose an attention mechanism to find similar users based on the specific user and item. However, they simply aggregate similar users for estimating the rating scores, therefore the neighboring users are not sufficiently leveraged for bridging unobserved user-item pairs. Chen et al. <cit.> aggregate social friends with an attention mechanism given the specific items, and interact the aggregated neighborhoods with the item to derive the recommendation. However, the interactions between the neighborhoods and the target items exert too strict regularization between the users and their social friends.



 §.§ Community Detection

The proposed latent group discovery is related to community detection <cit.> that identifies groups of densely connected nodes in a graph. However, those methods <cit.> explicitly require proximity information among the nodes for discovering latent communities, and they are inapplicable in the case when the link information is not available. 

By contrast, we propose to discover latent user/item groups and model user-item interactions collaboratively, the proposed latent group discovery and collaborative filtering can be mutually beneficial to each other. On one hand, users/items with similar rating activities are more likely to have similar representations in collaborative filtering, which facilitates the user/item clustering process. On the other hand, user/item groups can bridge similar users/items and boost recommendation performance. 



§ THE PROPOSED MODEL



 §.§ Preliminaries

In a recommendation problem, we have a set of m user U = {u_1,⋯,u_m}, a set of n items V={v_1,⋯,v_n} and a rating matrix 𝐑∈{0,1}^m× n, where each element r_ij∈{0,1} equals 1 if user u_i has an observed interaction (e.g. clicked, viewed, etc.) with item v_j and 0 otherwise. The task of item recommendation is to estimate the rating score r̂_ij for unobserved user-item interactions, and recommend for each user the items with high estimated rating scores. In this paper, we also propose to discover latent user groups G_u = {1,...,k} and item groups G_v = {1,...,k} for bridging similar users and items, where k is the pre-specified number of groups for users and items. 

In this paper, two-dimension matrices are denoted with bold uppercase symbols, single-dimension vectors are denoted bold lowercase symbols, and scalars are represented with lowercase symbols. The symbols used in this paper are summarized in Table.<ref>, other intermediate symbols are described in the corresponding sections. 




 §.§ Basic model

Inspired by previous works <cit.>, we employ multiple neural networks as the basic model for capturing deep user-item interactions. Without loss of generality, given the embedding vectors 𝐮_i∈ℝ^d, 𝐯_j∈ℝ^d for user u_i and item v_j respectively, where d is the number of dimensions of user and item embedding spaces. The corresponding rating score of u_i over v_j can be estimated as:

    𝐳_0    = [𝐮_i;𝐯_j;𝐮_i∘𝐯_j]
    𝐳_ij   = ϕ_L(...ϕ_1(𝐳_0) ...)
    ϕ_l(𝐳_l-1)    = σ_l(𝐖_l𝐳_l-1+𝐛_l),l∈[1,L]
    r̂_ij   = 1/1+exp(-𝐰_uv^T𝐳_ij)

where [;] and ∘ are the concatenation and element-wise multiply operation respectively. L is the number of hidden layers in the neural network, and 𝐖_l, 𝐛_l, σ_l are the weight matrix, bias vector and non-linear activation function of the l-th layer, respectively. 𝐰_uv is the parameter vector for transforming high-level user-item interaction vector 𝐳_ij into a logit, which is then used for rating score estimation. 



 §.§ Latent group discovery


In many online services, some users (items) have the same interests (characteristics) and constitute a group. Users/items in a group share the same group representation, and it can bridge similar users/items and mitigate data sparseness. However, in most cases, the group information is not explicitly available. In this paper, we propose to discover latent user/item groups and model user-item interactions collaboratively. Specifically, we propose to learn user and item group embeddings, and introduce embedding matrix 𝐆_u∈ℝ^k× d_g and 𝐆_v∈ℝ^k× d_g for user and item group respectively, where k is the number of user/item group as specified previously, and it is much smaller than m and n. d_g is the dimension of user/item group embeddings, and it is assumed to be smaller than the user/item dimension d as group embeddings represent users/items at a more general level of granularity. 

For learning user group embeddings, as an example, each user embedding is input into a non-linear transformation to obtain a vector that has the dimension of k, which is equal to the number of user groups. The value of each dimension of the vector can be viewed as the weight that the user belongs to the corresponding group, and the weighted sum of the user group embedding matrix is regarded as the soft latent group representation of the user. The user embedding is then reconstructed from the soft latent group representation. Therefore, user group embeddings are learned by transforming user embeddings into their reconstructions with least possible of information loss, and preserving most of the information in the k embedded user groups. In addition, users with similar representations are supposed to have similar soft latent group representations, and are clustered automatically into the same user group. The user group learning process is illustrated in Fig.<ref>, and we detail the learning process in the following subsections.






  §.§.§ Soft latent group representation

Each user embedding 𝐮_i is fed into a non-linear transformation to obtain a vector β_i∈ℝ^k, as shown in the following equation. 


    β_i = softmax(𝐖_u𝐮_i+𝐛_𝐮)

where 𝐖_u∈ℝ^k× d is the weighted matrix parameter, and 𝐛_𝐮∈𝐑^k is the bias vector. β_i can be viewed as the activations toward the user group embeddings, and each dimension of β_i is the probability that the user belongs to the related group. The weighted sum of the user group embeddings with respect to β_i is then regarded as the soft latent group representation of user u_i. 


    μ_i = ∑_s=1^kβ_i,s𝐠_s^u

where β_i,s is the s-th dimension of vector β_i, and 𝐠_s^u is the s-th row of embedding matrix 𝐆_u, representing the embedding of s-th user group. μ_i∈ℝ^d_g is the soft latent group representation of user u_i. Therefore, users with similar embeddings are more likely to have similar activations toward the group embeddings, and they are automatically clustered into the same group in an unsupervised manner. 



  §.§.§ Reconstruction user embeddings

We have obtained the soft latent group representations for the users. Now we describe how to calculate the reconstruction of the embedding for each user u_i from his/her soft latent group representations μ_i. We reconstruct user embeddings from their related soft latent group representations μ_i. The underlying reason is that if two users have similar soft latent group representations, then they are more likely within the same user group and the reconstructions of their embeddings are assume to be similar. In this paper, we employ non-linear transformations to reconstruct user embeddings, as shown follows:

    𝐮_i'=sigmoid(𝐖_𝐮'μ_i+𝐛_u')

where 𝐖_𝐮'∈ℝ^d× d_g is the weighted matrix parameter, and 𝐛_u'∈ℝ^d is the bias vector. At this point, the reconstruction of user embeddings is similar to an autoencoder, and obtaining the soft latent group representations can be viewed as the encoder stage of an autoencoder that takes user embeddings as input and maps them to the latent space while preserving group information. Eq.(<ref>) is similar to the decoder stage of an autoencoder that maps the soft latent group representation to the reconstruction 𝐮_i' of the same shape as 𝐮_i, as illustrated in Fig.<ref>. 




  §.§.§ Learning objective

Learning user groups is equivalent to minimizing the reconstruction error of 𝐮_i' from 𝐮_i. We employ the contrastive max-margin objective function that is commonly used in previous work <cit.>. Specifically, for each user embedding 𝐮_i, we randomly sample p users as negative users. The embeddings of the negative samples are represented as {𝐮_i_s|s=1,2,...,p}, and the learning objective is to make the reconstructed embedding 𝐮_i' similar to its origin embedding 𝐮_i while different from those negative embeddings. Therefore, the objective of user group learning ℒ_u is defined as a hinge loss that maximize the cosine similarity between 𝐮_i' and 𝐮_i and simultaneously minimize that between 𝐮_i' and the negative samples:

    ℒ_u = ∑_u_i∑_s=1^p max(0,1-𝐮_i'^T𝐮_i+𝐮_i'^T𝐮_i_s)


Similarly, the objective of item group learning can be formulated as:

    ℒ_v = ∑_v_j∑_t=1^p max(0,1-𝐯_j'^T𝐯_j+𝐯_j'^T𝐯_j_t)

where 𝐯_j' is the reconstructed embedding from 𝐯_j. The calculation of 𝐯_j' is similar to that of 𝐮_i' and we leave out the detail description here to avoid redundancy. {𝐯_j_t|t=1,2...,p} are p negative samples for 𝐯_j' when computing objective ℒ_v. 




 §.§ Hierarchies modeling




The intuition of hierarchy modeling is that users/items should be close to their groups to learn compact representations. To this end, take user-user group hierarchy for example, we introduce a latent variable ϵ_i for each user u_i, and minimize the distance between 𝐮_i+ϵ_i and its group representation μ_i in the embedding space <cit.>. In this paper, we propose to utilize deep neural networks to model the user-user group (item-item group) hierarchies. 

We present the user-user group hierarchy modeling in Fig.<ref>. As illustrated in the figure, we feed the vector 𝐮_i+ϵ_i into a multiple-layer neural network, and obtain a high-level representation 𝐳^u_i that has the same dimensions as the group representations. The similarities between 𝐳^u_i and all the group representations are measured with the inner product operation, and the resulted logits are transformed into a posterior probability vector with a softmax function, as shown in Eq.(<ref>). 


    𝐳_0^u_i   = 𝐮_i+ϵ_i
    𝐳^u_i   = ϕ_L^u(...ϕ_1^u(𝐳_0^u_i)...)
    
    a_i    = argmax_s(β_i,s),   s=1,2,...,k
    
    r̂^u_i   = exp(𝐠_a_i^u^T𝐳^u_i)/∑_s=1^kexp(𝐠_s^u^T𝐳^u_i)

where β_i,s is described in Eq.(<ref>), a_i is the label of the user group that user u_i has the maximum activation, and 𝐠_a_i^u is the corresponding group embedding, hence 𝐠_a_i^u can be viewed as the hard latent group representation of u_i. r̂^u_i is the posterior probability that user u_i belongs to group a_i, and therefore bridging the gap between a user and his/her group is equivalent to maximizing the corresponding posterior probability. In other words, the neural network takes a user and his/her group representations as input, and produces a predicted distribution over the groups. The training objective is defined as the minimization between the predicted distribution and the distribution of actual labels <cit.>, and it is demonstrated to be better than other loss functions <cit.>. The objective function of user-user group hierarchy can be defined as follows:

    ℒ_uu=-∑_u_ilogexp(𝐠_a_i^u^T𝐳^u_i)/∑_s=1^kexp(𝐠_s^u^T𝐳^u_i)

Similarly, the objective function of item-item group hierarchy can defined as:

    ℒ_vv=-∑_v_jlogexp(𝐠_b_j^v^T𝐳^v_j)/∑_t=1^kexp(𝐠_t^v^T𝐳^v_j)

where b_j is group label of item v_j, and it is calculated in the same way as a_i.  𝐠_b_j^v is the corresponding group embedding of v_j. The neural network for modeling item-item group hierarchy has the same architecture as that in Fig.<ref>, and 𝐳^v_j can be viewed as the high-level representation of item v_j extracted from the neural network. 




 §.§ Dual-bridging collaborative filtering




Once we have obtained group representations, we can incorporate them for boosting recommendation performance. The underlying reason is that, the user-item group (item-user group) interactions can bridge similar users/items and alleviate data sparseness. For example, even though a user's interest in an item is unknown, the item can still be ranked higher in the recommendation list, as long as the user has positively interacted with some other items sharing the same group with the unobserved item, or users within the same group have positively rated the unobserved item. Notice that, the collaborative filtering and latent group discovery are iteratively performed, and they mutually benefit each other for better learning representations. First, collaborative filtering learns similar user/item representations based on the user-item interaction data, and this facilitates the clustering process for discovering users/items with similar representations. In turn, the latent group information can bridge similar users (items), and boost recommendation performance. 


The general architecture of the proposed model is presented in Fig.<ref>. As shown in the figure we introduce three neural networks for modeling user-item, user-item group, user group-item interactions respectively. The rating score of a user u_i over an item v_j is predicted as follows,

    𝐳_0^uv   = [𝐮_i;𝐯_j;𝐮_i∘𝐯_j]
    𝐳_ij^uv   = ϕ_L^uv(...ϕ_1^uv(𝐳_0^uv) ...)



    𝐳_0^ug   = [𝐮_i;𝐠_b_j^v;𝐮_i^T𝐌_u𝐠_b_j^v]
    𝐳_ij^ug   = ϕ_L^ug(...ϕ_1^ug(𝐳_0^ug) ...)



    𝐳_0^vg   = [𝐠_a_i^u;𝐯_j; 𝐯_j^T𝐌_v𝐠_a_i^u]
    𝐳_ij^vg   = ϕ_L^vg(...ϕ_1^vg(𝐳_0^vg) ...)





    r̂_ij=1/1+exp[-(𝐰_uv^T𝐳_ij^uv+𝐰_ug^T𝐳_ij^ug+𝐰_vg^T𝐳_ij^vg)]

where 𝐌_u∈ℝ^d× d_g is a transformation matrix for measuring the similarity between 𝐮 and 𝐠_b_j^v in a common latent space, while 𝐌_v∈ℝ^d× d_g is a matrix for measuring the similarity between 𝐯_j and 𝐠_a_i^u. ϕ_{1,⋯,L}^{uv,ug,vg} are the respective neural networks for modeling user-item, user-item group, item-user group interaction, and 𝐰_{uv,ug,vg} are the parameter vectors for transforming high-level representations (i.e. 𝐳_ij^uv,𝐳_ij^ug,𝐳_ij^vg ) of different interaction spaces into the final predictive logit. Therefore, with the dual-bridging architecture, an item can be ranked higher in the recommendation list for a user, as long as the user's preference over the item is properly modeled in any one of the three interaction spaces. The training objective of the proposed dual-bridging collaborative filtering can be defined as follows,

    ℒ_uv = -∑_(u_i,v_j)∈𝒟r_ijlogr̂_ij+(1-r_ij)log(1-r̂_ij)

where 𝒟 is the training set. 






 §.§ Model learning

Due to the nature of implicit feedback and the large number of items in the recommendation task <cit.>, negative sampling <cit.> is employed to approximate ℒ_uv. As for ℒ_uu and ℒ_vv, even though the computation involves the summation over the interactions between a user/item and all the groups (i.e. ∑_k=1^Kexp(𝐠_k^u^T𝐳^u_i)), the number of groups are much fewer than that of users/items, and hence the summation does not incur high computational overhead. In the proposed DBRec model, we jointly define the loss function, so that it can be optimized efficiently by backpropagation. Taking into account the loss of dual-bridging collaborative filtering, hierarchy modeling and group learning, the final objective function of DBRec can be defined as follows, 

    ℒ = ℒ_uv+α(ℒ_uu+ℒ_vv+ℒ_u+ℒ_v)

where α controls the tradeoff between the loss of dual-bridging collaborative filtering and that of hierarchy modeling and group learning. The training of the proposed model can be decomposed into two parts. In the part of collaborative filtering and hierarchy modeling, user and item embeddings are updated to fit user-item interaction data and preserve group hierarchies. While in the part of group learning, group embeddings are updated to discover representative group information.

The model parameters include user/item representations and parameters of neural networks as shown in Fig.<ref>, Fig.<ref> and Fig.<ref>. We update the object function with Adam optimizer, which is a variant of Stochastic Gradient Descent with a dynamically tuned learning rate and updates parameters every step along the gradient direction with the following protocol:

    θ^t←θ^t-1-lr∂ℒ/∂θ

where lr is the learning rate, and θ are the model parameters and ∂ℒ/∂θ are the partial derivatives of the objective function with respect to the model parameters, and they can be automatically computed with typical deep learning libraries.



§ EXPERIMENTS



 §.§ Experimental Settings



  §.§.§ Datasets




To validate the effectiveness of the proposed model, we conduct experiments on three publicly datasets, namely Amazon[2], MovieLens-1M (ML1M)[1] and Gowalla[3]. For the amazon dataset, we select the items of musical instruments for evaluation. To transform the explicit ratings of Amazon and ML1M into implicit feedback, ratings higher than 3 are regarded as positive feedback, and the others are regarded as missing values. As for Gowalla, for each user the check-in locations are viewed as positive items, while the other locations are treated as missing items. Inspired by previous works <cit.>, we filter out users with fewer than 5 positive items and the items with fewer than 2 users. The statistics of the datasets is shown in Table.<ref>.

[1]https://grouplens.org/datasets/movielens/
[2]http://jmcauley.ucsd.edu/data/amazon/ 
[3]http://snap.stanford.edu/data/loc-gowalla.html



  §.§.§ Setup

To evaluate the proposed model, we randomly split the datasets into training set (70%), validation set (10%) and testing set (20%). In the training process, for each positive item, we randomly sample 5 items as the negative samples. In the testing process, as it is time-consuming to rank all the items for every user at each time, hence for each (u_i,v_j) pair in the testing set, we mix the testing item with 99 random items, and rank the testing item along with the 99 items for the related user. We measure the recommendation performance with the commonly used Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG), as shown follows,

    HR   =#hits/#test
    
    NDCG   =1/#test∑_i=1^#test1/log_2(p_i+1)

where #hits is the number of testing item that appears in the the recommendation list of the related user and #test is the total number of (u_i,v_j) pair in the testing set. p_i is the position of the testing item in the recommendation list for the i-th hit. HR measures whether the testing item is in the recommendation list, while NDCG assigns higher score to the testing item with higher position. In this paper, we truncate the ranking list at k∈[1,2,⋯,10] for both metrics.


  §.§.§ Baselines

The baselines employed for performance comparison are list as follows, 


  * NeuMF<cit.>. It combines generalized MF and Multi-Layer Perception (MLP) for modeling user-item latent structures. 


  * ConvMF<cit.>. It uses outer product to transform latent vectors of a user-item pair into two-dimensional map, and applies convolutional and pooling layers to model deep user-item interactions. 


  * CMN <cit.>. It identify similar neighboring users with an attention mechanism based on the specific user-item pair, and jointly exploits the neighborhood state and user-item interactions to derive recommendation. 


  * DELF<cit.>. It proposes an attention mechanism to aggregate an additional embedding for each user/item, and then further introduce a neural network architecture to incorporate dual embeddings for recommendation. 


For fair comparison, baselines in this paper mainly consist of the state-of-the-art recommendation models based on user-item interactions, and the hyper-parameters of these baselines are set according to the original works. Notice that even though CMN exploits neighborhood information for recommendation, the neighboring users are selected mainly based on their rating behaviours. Beside those baselines, we also study various variants of the proposed model, list as follows,


  * DBRec-o, a variant of the proposed model that excludes user and item group learning, and it mainly relies on user-item interactions for learning user preferences over the items. 


  * DBRec-u, a variant of the proposed model that excludes item group learning, and it jointly learns user group and model user-group hierarchy, and interacts user group information with item for boosting recommendation.


  * DBRec-i, a variant of the proposed model that excludes user group learning, and it jointly learns item group and model item-group hierarchy, and interacts item group information with users for boosting recommendation.




  §.§.§ Implementation

We implement DBRec based on Tensorflow, and the sources are made publicly available[3] to facilitate community research. We set the batch size to 256 and the learning rate to 0.0001.The dimensions of latent factor is 128. We employ two hidden layers for the neural networks illustrated in Fig.<ref> and Fig.<ref>, and the hidden units of the respective layer are [64,16] for dual-bridging collaborative filtering, and [64,128] for hierarchy modeling. The model parameters are fine tuned on the validation set. We set the number of user/item group to 5, and the trade-off weight (i.e. α in Eq.(<ref>)) for group learning and hierarchy modeling to 0.01. We analyze the sensitivity of these two hyper-parameters later in this section.

[3]https://github.com/uqjwen/DBRec



  
pre-training The objective function is non-convex, and the learning method can be easily trapped in local optimums. In our work, we pre-train DBRec with NeuMF to obtain initial user and item embeddings in DBRec. As for the group embeddings, we perform clustering on the pre-trained user and item embeddings, and use the cluster centroids to initialized the group embeddings. Since users/items and their corresponding groups do not share the same latent space, we employ a dimensionality reduction method (i.e. t-SNE <cit.>) to reduce the centroids of d-dimension to that of d_g dimensions. 



 §.§ Experiments



  §.§.§ Recommendation comparisons









We compare the proposed model, DBRec, with the state-of-the-art recommendation models in terms of top-K recommendation, and the results are presented in Fig.<ref> and Fig.<ref>. From the figures, we have the following observations. First, DELF, ConvMF and CMN outperform NeuMF across different datasets and metrics, which is consistent with previous works <cit.>. The reason is that NeuMF mainly relies user-item interactions for exploiting user preferences, and it suffers from data sparseness due to the sparsity nature of the rating matrix. Second, the performance differences between CMN and ConvMF vary across different datasets. For example, ConvMF achieves marginally better performance than CMN on ML1M, while CMN outperforms ConvMF on Gowalla. This is because they leverage different aspects of interaction data for boosting performance. Specifically, ConvMF utilizes outer product to exploit dimension-wise correlations for recommendation, while CMN exploits informative neighboring users for bridging each user-item pair. Therefore, the performance differences between these two models are quite data-dependent. Third, the figures show that DELF is superior to CMN, and one possible explanation might be that DELF aggregates for each user/item an additional embedding based on the interacted items/users, and introduce a neural network to incorporate the dual embeddings for recommendation. In addition, DELF can identify the most informative context users/items for collaborative filtering, due to the proposed effective attention mechanism.

Finally, the proposed model outperforms the baselines by a large margin across the datasets for both HR and NDCG. Specifically, in terms of top-10 recommendation, DBRec outperforms the best baseline, DELF, with a relative improvement of 6.71%(HR@10) and 5.19%(NDCG@10) on Amazon, 5.59%(HR@10) and 5.46%(NDCG@10) on ML1M, and 3.84%(HR@10) and 7.11%(NDCG@10) on Gowalla, respectively. The reason behinds the advantage of DBRec over the baselines is that, we collaboratively discover latent group information for collaborative filtering, and the incorporation of group information can capture user preferences (item characteristics) at different levels of granularity and mitigate data sparseness. 





  §.§.§ Efficacy of dual-bridging architecture



In this subsection, we study the performance of various variants of the proposed model, to validate the effectiveness of the proposed dual-bridging architecture. As shown in Table.<ref>, without latent group information, DBRec degrades to the version (DBRec-o) that is mainly based on user-item interactions, and is vulnerable to data density. However, by discovering latent groups and incorporating group information for collaborative filtering, DBRec-i and DBRec-u significantly outperform DBRec-o, demonstrating the benefit of discovering latent groups for bridging similar users/items. 


DBRec combines DBRec-i and DBRec-u, and results in a dual-bridging architecture, which models user preferences and item characteristics at different levels of granularity. General group representations can alleviate the problem of data sparseness by bridging similar users/items, and achieves better recommendation performance than otherwise single-bridging model (e.g. DBRec-i, DBRec-u). 



  §.§.§ Sensitivity analysis



In this subsection, we investigate the robustness of the proposed model with respect to different hyper-parameters. Especially, we study the performance of DBRec under different pre-specified user/item group numbers, and the weight α (Eq.(<ref>)) for group learning and hierarchy modeling. In this experiment, α is varied amongst [0.001,0.01,0.1,1,10], and the group number [5,10,15,20,25]. As illustrated in Fig.<ref>, we present the performance in terms of HR@10 and NDCG@10 across the datasets. One can see from the figures that, DBRec experiences a rapid performance degradation with α=10 and #group=20 on Amazon. However, the recommendation performance is relatively stable under a wide-range choice of the hyper-parameters. As shown in Fig.<ref>(b) and Fig.<ref>(d), Gowalla presents a steady performance improvement with more group number, probably because larger group number is required to preserve group information due to the size of the dataset. HR@10 and NDCG@10 usually show the similar trends across the datasets except for Gowalla, which experiences a sudden drop on NDCG@10 with α=1, but the expected phenomenon is not observed on HR@10 with α=1, as shown in Fig.<ref>(a) and Fig.<ref>(c). The reason for the slump of NDCG on Gowalla is that, the testing items are ranked lower at the bottom of the recommendation list. Overall, this experiment demonstrates that the proposed model is invulnerable to the group number. Even though the users/items are clustered into different granularities, the group information can still boost recommendation performance as long as discriminate information provided by the user/item groups can be preserved in the clustering process.



  §.§.§ Visualization



To examine whether DBRec can discover latent groups, we visualize the learned user representations with t-SNE <cit.>, and plot users in the same group with the same color. As shown in Fig.<ref>, several user/item groups can be clearly identified. This experiment demonstrates that DBRec can effectively discover latent groups, and visually interpret the rationale behind the dual-bridging architecture in DBRec.



§ CONCLUSION

In this paper, we propose a novel dual-bridging recommendation model. The advantage of the proposed model is that it can discover latent group information for bridging similar users/items, and boost recommendation performance. We unified collaborative filtering, group learning and hierarchy modeling into a framework, so that all the sub-components can be jointly optimized and compensate each other for boosting recommendation. Extensive experiments on two real datasets demonstrate the advantage of DBRec in terms of recommendation performance, and robustness in terms of parameters sensitivity.

The proposed model in this paper mainly relies on user-item interaction data, and discover extra latent group information for boosting recommendation performance. In real recommendation scenarios, many attribute data are explicitly available, and integrating heterogeneous modalities of implicit and explicit information for recommendation is an interesting research topic in the future. 



§ ACKNOWLEDGMENTS

This publication was made possible by Opinion Analysis grant 018493 from Australian Research Council Fund and NPRP grant NPRP10-0208-170408 from the Qatar National Research Fund (a member of Qatar Foundation).

ACM-Reference-Format


