Reviewing 170 'self-supervised learning' recommendation algorithms, HKU releases SSL4Rec: the code and database are fully open source!

Recommendation systems are important to address the challenge of information overload as they provide customized recommendations based on users’ personal preferences. In recent years, deep learning technology has greatly promoted the development of recommendation systems and improved insights into user behavior and preferences.

However, traditional supervised learning methods face challenges in practical applications due to data sparsity issues, which limits their ability to effectively learn user performance.

To protect and overcome this problem, self-supervised learning (SSL) technology is applied to students, which uses the inherent structure of the data to generate supervision signals and does not rely entirely on labeled data.

This method uses a recommendation system that can extract meaningful information from unlabeled data and make accurate predictions and recommendations even when data is scarce.

Article address: https://arxiv.org/abs/2404.03354

Open source database : https://github.com/HKUDS/Awesome-SSLRec-Papers

Open source code base: https://github.com/HKUDS/SSLRec

This article reviews self-supervised learning frameworks designed for recommender systems and conducts an in-depth analysis of more than 170 related papers. We explored nine different application scenarios to gain a comprehensive understanding of how SSL can enhance recommendation systems in different scenarios.

For each domain, we discuss different self-supervised learning paradigms in detail, including contrastive learning, generative learning, and adversarial learning, showing how SSL can improve recommendation systems in different situations. performance.

1 Recommended system

The research on recommender system covers various tasks in different scenarios, such as collaborative filtering and sequence recommendation and multi-behavior recommendations, etc. These tasks have different data paradigms and goals. Here, we first provide a general definition without going into specific variations for different recommendation tasks. In the recommendation system, there are two main sets: the user set, denoted as , and the item set, denoted as .

Then, use an interaction matrix to represent the recorded interactions between the user and the item. In this matrix, the entry Ai,j of the matrix is ​​assigned the value 1 if the user ui has interacted with the item vj, otherwise it is 0.

The definition of interaction can be adapted to different contexts and data sets (e.g., watching a movie, clicking on an e-commerce site, or making a purchase).

In addition, in different recommendation tasks, there are different auxiliary observation data, recorded as X. For example, in knowledge graph enhanced recommendation, X contains the knowledge graph containing external item attributes. , these attributes include different entity types and corresponding relationships.

In social recommendation, X includes user-level relationships, such as friendship. Based on the above definition, the recommendation model optimizes a prediction function f(⋅), aiming to accurately estimate the preference score between any user u and item v:

The preference score yu,v represents the possibility of user u interacting with item v.

Based on this score, the recommendation system can recommend uninteracted items to each user by providing a ranked list of items based on the estimated preference score. In the review, we further explore the data form of (A,X) in different recommendation scenarios and the role of self-supervised learning in it.

2 Self-supervised learning in recommendation systems

In the past few years, deep neural networks have performed well in supervised learning, which has been widely used in fields including computer vision, natural language processing and recommendation systems. It is reflected in all fields. However, due to its heavy reliance on labeled data, supervised learning faces challenges in dealing with label sparsity, which is also a common problem in recommender systems.

To address this limitation, self-supervised learning emerged as a promising method, which utilizes the data itself as labels for learning. Self-supervised learning in recommender systems includes three different paradigms: contrastive learning, generative learning, and adversarial learning.

2.1 Contrastive Learning

##Contrastive Learning as a A prominent self-supervised learning method whose main goal is to maximize the consistency between different views enhanced from the data. In contrastive learning for recommender systems, the goal is to minimize the following loss function:

E∗∘ω∗ represents the comparison view creation operation. Different recommendation algorithms based on contrastive learning have different creation processes. The construction of each view consists of a data augmentation process ω∗ (which may involve nodes/edges in the augmented graph) and an embedding encoding process E∗.

The goal of minimization is to obtain a robust encoding function that maximizes the consistency between views. This consistency across views can be achieved through methods such as mutual information maximization or instance discrimination.

2.2 Generative Learning

The goal of generative learning It is about understanding the structure and patterns of data to learn meaningful representations. It optimizes a deep encoder-decoder model that reconstructs missing or corrupted input data.

The encoder creates a latent representation from the input, while the decoder reconstructs the original data from the encoder output. The goal is to minimize the difference between the reconstructed and original data as follows:

Here, ω represents operations such as masking or perturbation. D∘E represents the process of encoding and decoding to reconstruct the output. Recent research has also introduced a decoder-only architecture that efficiently reconstructs data without an encoder-decoder setup. This approach uses a single model (e.g., Transformer) for reconstruction and is typically applied to serialized recommendations based on generative learning. The format of the loss function depends on the data type, such as mean square error for continuous data and cross-entropy loss for categorical data.

2.3 Adversarial Learning

Adversarial learning is a training method that uses the generator G (⋅) generates high-quality output and contains a discriminator Ω(⋅) that determines whether a given sample is real or generated. Unlike generative learning, adversarial learning differs by including a discriminator that uses competitive interactions to improve the generator's ability to produce high-quality output in order to fool the discriminator.

Therefore, the learning goal of adversarial learning can be defined as follows:

Here, the variable x represents the real sample obtained from the underlying data distribution, while represents the one generated by the generator G(⋅) Synthetic samples. During training, both the generator and the discriminator improve their capabilities through competitive interactions. Ultimately, the generator strives to produce high-quality outputs that are beneficial for downstream tasks.

3 Classification System (Taxonomy)

In this section, we propose the application of self-supervised learning in recommendation systems comprehensive classification system. As mentioned before, self-supervised learning paradigms can be divided into three categories: contrastive learning, generative learning, and adversarial learning. Therefore, our classification system is built based on these three categories, providing deeper insights into each category.

3.1 Comparative learning in recommendation systems

##The basic principle of contrastive learning (CL) is to maximize the consistency between different views. Therefore, we propose a view-centric taxonomy consisting of three key components to consider when applying contrastive learning: creating views, pairing views to maximize consistency, and optimizing consistency.

View Creation. Create views that emphasize the various aspects of the data that the model focuses on. It can combine global collaborative information to improve the recommendation system's ability to handle global relationships, or introduce random noise to enhance the robustness of the model.

We consider the enhancement of input data (e.g., graphs, sequences, input features) as view creation at the data level, while the enhancement of latent features during inference is regarded as the feature level View creation. We propose a hierarchical classification system that includes view creation techniques from the basic data level to the neural model level.

• Data level Data-based: In comparative learning-based recommendation systems, diverse views are created by enhancing input data. These enhanced data points are then processed through the model. The output embeddings obtained from different views are finally paired and used for comparative learning. The enhancement methods vary depending on the recommendation scenario. For example, graph data can be enhanced using node/edge dropping, while sequences can be enhanced using masking, cropping, and replacement.
• Feature level Feature-based: In addition to generating views directly from data, some methods also consider enhancing the encoded hidden features in the model forward process. These hidden features can include node embeddings of graph neural network layers or token vectors in Transformers. By applying various enhancement techniques multiple times or introducing random perturbations, the final output of the model can be viewed as different views.
• Model level Model-based: Data-level and feature-level enhancements are non-adaptive because they are non-parametric. So there are also ways to use models to generate different views. These views contain specific information based on the model design. For example, intent-decoupled neural modules can capture user intentions, while hypergraph modules can capture global relationships.

Pair Sampling. The view creation process generates at least two different views for each sample in the data. The core of contrastive learning is to maximize the alignment of certain views (i.e., bring them closer) while pushing other views away.

To do this, the key is to identify the positive sample pairs that should be brought closer, and identify other views that form negative sample pairs. This strategy is called paired sampling, which mainly consists of two paired sampling methods:

• Natural Sampling Natural Sampling: A common method of paired sampling is direct rather than heuristic, which we call natural sampling. Positive sample pairs are formed from different views generated by the same data sample, while negative sample pairs are formed from views of different data samples. In the presence of a central view, such as a global view derived from the entire graph, local-global relationships can also naturally form positive sample pairs. This method is widely used in most contrastive learning recommendation systems.
• Score-based Sampling Score-based Sampling: Another method of paired sampling is score-based sampling. In this approach, a module calculates the scores of sample pairs to determine positive or negative sample pairs. For example, the distance between two views can be used to determine positive and negative sample pairs. Alternatively, clustering can be applied on the view, where positive pairs are within the same cluster and negative pairs are within different clusters. For an anchor view, once a positive sample pair is determined, the remaining views are naturally considered negative views and can be paired with the given view to create negative sample pairs, allowing push-away.

Contrastive Objective. The learning goal in contrastive learning is to maximize the mutual information between pairs of positive samples, which in turn can improve the performance of the learning recommendation model. Since it is not feasible to directly calculate mutual information, a feasible lower bound is usually used as the learning target in contrastive learning. However, there are also explicit goals of bringing positive pairs closer together.

• InfoNCE-based: InfoNCE is a variant of noise contrastive estimation. Its optimization process aims to bring positive sample pairs closer and push away negative sample pairs.
• JS-based: In addition to using InfoNCE to estimate mutual information, you can also use Jensen-Shannon divergence to estimate the lower bound. The derived learning objective is similar to combining InfoNCE with standard binary cross-entropy loss, applied to positive and negative sample pairs.
• Explicit Objective: Both the InfoNCE-based and JS-based objectives aim to maximize the estimated lower bound of mutual information in order to maximize the mutual information itself, which is guaranteed in theory of. In addition, there are explicit objectives, such as minimizing the mean square error or maximizing the cosine similarity within a sample pair, to directly align pairs of positive samples. These goals are called explicit goals.

3.2 Generative Learning in Recommender Systems

In generative self-supervised learning, the main goal is to maximize the likelihood estimate of the real data distribution. This allows the learned, meaningful representations to capture the underlying structure and patterns in the data, which can then be used in downstream tasks. In our classification system, we consider two aspects to distinguish different generative learning-based recommendation methods: generative learning paradigm and generative goal.

Generative Learning Paradigm. In the context of recommendation, self-supervised methods employing generative learning can be classified into three paradigms:

• Masked Autoencoding: In a masked autoencoder, the learning process follows the mask-reconstruction method, where the model reconstructs the complete data from partial observations.
• Variational Autoencoding: Variational Autoencoder is another generation method that maximizes the likelihood estimate and has theoretical guarantees. Typically it involves mapping input data onto latent factors that follow a normal Gaussian distribution. The model then reconstructs the input data based on the sampled latent factors.
• Denoised Diffusion: Denoised diffusion is a generative model that generates new data samples by inverting the noise process. In the forward process, Gaussian noise is added to the original data and, over multiple steps, a series of noisy versions are created. During the reverse process, the model learns to remove noise from the noisy version, gradually restoring the original data.

#Generation Target. In generative learning, which pattern of data is considered as a generated label is another issue that needs to be considered to bring meaningful auxiliary self-supervised signals. In general, the generation goals vary for different methods and in different recommendation scenarios. For example, in sequence recommendation, the generation target can be the items in the sequence, with the purpose of simulating the relationship between items in the sequence. In interactive graph recommendation, the generation targets can be nodes/edges in the graph, aiming to capture high-level topological correlations in the graph.

3.3 Adversarial Learning in Recommended Systems

##In In adversarial learning of recommendation systems, the discriminator plays a crucial role in distinguishing generated false samples from real samples. Similar to generative learning, the classification system we propose covers adversarial learning methods in recommender systems from two perspectives: learning paradigm and discrimination goal:

Adversarial Learning Paradigm (Adversarial Learning Paradigm). In recommender systems, adversarial learning consists of two different paradigms, depending on whether the discriminative loss of the discriminator can be back-propagated to the generator in a differentiable manner.

• Differentiable Adversarial Learning (Differentiable AL): The first method involves objects represented in a continuous space, and the gradient of the discriminator can be naturally backpropagated to the generator optimize. This approach is called differentiable adversarial learning.
• Non-Differentiable Adversarial Learning (Non-Differentiable AL): Another method involves identifying the output of the recommendation system, especially the recommended products. However, since the recommendation results are discrete, backpropagation becomes challenging, forming a non-differentiable case where the gradient of the discriminator cannot be directly propagated to the generator. To solve this problem, reinforcement learning and policy gradient are introduced. In this case, the generator acts as an agent that interacts with the environment by predicting goods based on previous interactions. The discriminator acts as a reward function and provides a reward signal to guide the learning of the generator. The discriminator's reward is defined to emphasize different factors that affect recommendation quality, and is optimized to assign higher rewards to real samples rather than generated samples, guiding the generator to produce high-quality recommendations.

Discrimination Target. Different recommendation algorithms cause the generator to generate different inputs, which are then fed to the discriminator for discrimination. This process aims to enhance the generator's ability to produce high-quality content that is closer to reality. Specific discrimination goals are designed based on specific recommendation tasks.

3.4 Diverse recommendation scenarios

In this review, we An in-depth discussion of the design methods of different self-supervised learning methods from nine different recommendation scenarios. These nine recommendation scenarios are as follows (please read the article for details):

• General Collaborative Filtering - This is the most basic form of recommendation system, which mainly relies on interaction data between users and items to generate personalized recommendations.
• Sequential Recommendation (sequence recommendation) - considers the time series of user interaction with items, with the purpose of predicting the user's next possible interaction item.
• Social Recommendation - Combines user relationship information in social networks to provide more personalized recommendations.
• Knowledge-aware Recommendation - Use structured knowledge such as knowledge graphs to enhance the performance of recommendation systems.
• Cross-domain Recommendation - Apply user preferences learned from one domain to another domain to improve recommendation results.
• Group Recommendation - Providing recommendations for groups with common characteristics or interests, rather than for individual users.
• Bundle Recommendation - Recommend a group of items as a whole, usually for promotions or package services.
• Multi-behavior Recommendation - Consider the user's multiple interactions with items, such as browsing, purchasing, rating, etc.
• Multi-modal Recommendation - Combines multiple modal information of items, such as text, images, sounds, etc., to provide richer recommendations.

This article provides a comprehensive review of the application of self-supervised learning (SSL) in recommendation systems. More than 170 papers were analyzed. We proposed a self-supervised classification system covering nine recommendation scenarios, discussed three SSL paradigms of contrastive learning, generative learning and adversarial learning in detail, and discussed future research directions in the article.

We emphasize the importance of SSL in handling data sparsity and improving recommendation system performance, and point out the integration of large language models into recommendation systems, adaptive dynamic recommendation environments, and Potential research directions such as establishing a theoretical foundation for the SSL paradigm. We hope that this review can provide valuable resources for researchers, inspire new research ideas, and promote the further development of recommendation systems.