Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences: New knowledge-enhanced graph neural network to achieve interpretable recommendations

In recent years, artificial intelligence based on big data and deep learning has demonstrated excellent computing power and learning capabilities. However, deep learning models often contain deeply nested nonlinear structures, making it difficult to determine the specific factors that led to such a decision, lacking interpretability and transparency in decision-making.

At the same time, since explainable recommendations not only improve the transparency, explainability, and credibility of the recommendation system, but also improve user satisfaction, the explainable recommendation task It has attracted more and more attention from researchers [1].

With the development of deep learning methods and language processing technology, when providing personalized recommendations, many methods use natural language generation technology to generate natural text explanations [2-3]. However, due to data sparsity, it is difficult to generate high-quality text explanations and has poor readability.

In addition, because knowledge graphs can contain more facts and connections, some researchers use knowledge graphs for recommendations and enhance the interpretability of recommendations through graph reasoning paths [4-5 ]. However, graph path-based methods require some prerequisites or definitions, such as pre-defined paths or multiple types of associations in the data set. At the same time, the knowledge graph may contain redundant entities, leading to homogeneous recommendation results.

Paper link: https://ieeexplore.ieee.org/abstract/document/9681226

To this end, the author proposes A new knowledge-enhanced graph neural network (KEGNN) is proposed to achieve interpretable recommendation.

KEGNN uses the semantic knowledge in the external knowledge base to learn knowledge from three aspects of users, products and user-product interaction to enhance semantic embedding.

From the perspective of user-product interaction, the user behavior graph is constructed and the user behavior graph is initialized using knowledge-enhanced semantic embedding.

Then a user behavior learning and reasoning model based on graph neural network is proposed. This model transfers the user's preference information and performs multi-hop reasoning on the user behavior graph to comprehensively understand user behavior.

Finally, a hierarchical collaborative filtering layer is designed for recommendation prediction, and the copy mechanism is combined with the GRU generator to generate high-quality, human-readable semantic explanations. The authors have conducted extensive experiments on three real-world datasets. Experimental results show that KGNN outperforms existing methods.

Method

The author proposed a knowledge-enhanced graph neural network to achieve interpretability. The architecture of the proposed method is shown in Figure 1 Show.

It mainly includes four modules: knowledge-enhanced semantic representation learning, user behavior learning and reasoning based on graph neural networks, hierarchical collaborative filtering and text explanation generation.

Module 1: In order to learn the semantic representation of users, products and user-product interactions, the author pooled user and product review documents in chronological order to form three Types of text documents, respectively represented as users, items and user-item interactions.

Further knowledge-enhanced semantic representation learning was performed on the three documents. Figure 2 shows the structure of the knowledge-enhanced semantic representation learning module.

First, in the context representation part, word-level embedding representation and semantic context representation are learned first, and BiLSTM is used to globally capture the top layer of word-level embedding to obtain hierarchical semantic representation. Secondly, in knowledge perception, the author uses the knowledge base to enhance semantic representation learning.

In addition, the author uses one-hot to represent the user/item encoding, and uses fully connected layer mapping to convert the sparse one-hot representation into a dense representation as the inherent representation of the user/item. Finally , using multi-head attention to further fuse the knowledge-aware representation and the inherent representation of the user/item to output a knowledge-enhanced user/item/user-item interaction representation.

Module 2: In order to fully understand user preferences, the author designed a user behavior learning and reasoning module based on graph neural network, including three steps: user behavior graph construction, information dissemination layer and multi-hop reasoning, as shown in Figure 3 .

First, from the user-product interaction relationship, the author constructs a user behavior graph, and uses knowledge-enhanced semantic representation to initialize the node representation and edge representation of the user behavior graph.

Secondly, based on the architecture of graph neural network, the author designed information propagation and information fusion based on the GNN information propagation layer to capture the first-order adjacency network (ego-network) between user behaviors.

Finally, a recursive approach is used to model the high-order connection information of the graph structure using multi-hop reasoning.

Module 3: The author designs a hierarchical neural collaborative filtering algorithm based on the neural collaborative filtering framework, as shown in Figure 4(a), It mainly includes three layers of neural collaborative filtering layers to achieve user-product interaction prediction.

The first neural collaboration layer is a fully connected layer. The user representation and product representation obtained in user behavior graph learning and reasoning are spliced ​​together as input to output the first layer of user-product interaction. Represented as Formula 1:

(Formula 1)

In the second layer, we designed a relationship-aware neural network layer, which fuses the user-item relationship representation and the output of the first layer.

Use two fusion methods, namely Hadamard product (such as formula 2) and fully connected layer nonlinear fusion (such as formula 3), to output relationship-aware user-product interaction respectively. Representation and high-level interactive representation:

(Formula 2)

(Formula 3)

The third layer uses relationship-aware user-item interaction representation and high-level interaction representation as input to achieve rating prediction, as shown in Formula 4:

(Formula 4)

Module 4: Combining the generation model and copy mechanism (generation mode and copy mode), the author designs a novel text explanation generation module to generate high-quality human-readable text Read the explanation.

The right part of Figure 4 demonstrates the details of this module. The recurrent neural network GRU is used as the explanation generator; in addition, a copy mechanism is introduced to extract information from users' original comments, and two modes (generation mode and copy mode) are combined to generate intuitive text explanations (word sequences), which are easy for users to read and understand.

Experiment

Dataset selection

This article uses three data sets of Amazon5 core. They are electronics, home-kitchen and music-equipment. The rating range is [0,5]. For all datasets, the authors randomly selected 80% of the user-item interactions in each dataset as the training set, 10% of the user-item interactions as the test set, and the remaining 10% of the user-item interactions were regarded as the validation set.

Rating prediction performance

In method comparison, the author compares KEGNN with CTR, PMF, NARRE, NRT, GCMC, LightGCN, and RippleNet (Calculate the RMSE and MAE of each method), as shown in the figure.

It can be seen from the results that for all data sets, the author's method KEGNN outperforms all compared methods in both MAE and RMSE.

Explanation of generation quality

It can be seen from the results that the author's method has better performance in accuracy and F1 index It has the best performance, but the recall rate is not as good as CTR or NARRE. Quality assessment results show that the text explanations generated by the authors resemble ground-truth reviews related to processing behaviors and reveal the implicit user intentions behind processing behaviors. The authors further investigate interpretability in a subsequent case analysis.

Case Analysis

The author selects some cases to demonstrate the interpretability of generated explanations. The selected cases All come from the test set. Therefore, the real text is hidden during the explanation generation process. Typical cases of the three data sets are shown in the table above. Ground represents the real comments given by users.

From the case study, we can see that the author's method generates explanations that explain the user's choice and purchase reasons for the rated items. Explainable concepts and aspects are highlighted in bold italics, indicating the underlying intent of user behavior and demonstrating the explainability of recommended results.

Conclusion

This paper proposes an interpretable recommendation method based on knowledge-enhanced graph neural network (KEGNN), which utilizes external knowledge The semantic knowledge in the library is used to enhance representation learning in three aspects: users, products and user-product interactions.

The author constructed a user behavior graph and designed a user behavior learning and reasoning module based on graph neural network to comprehensively understand user behavior.

Finally, the GRU generator and copy mechanism are combined to generate semantic text explanations, and hierarchical neural collaborative filtering is used to achieve accurate recommendations. Please see the paper details for details.

The above is the detailed content of Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences: New knowledge-enhanced graph neural network to achieve interpretable recommendations. For more information, please follow other related articles on the PHP Chinese website!