Exploring Large Language Models for Knowledge Graph Completion

Knowledge graphs play a vital role in numerous artificial intelligence tasks, yet they frequently face the issue of incompleteness. In this study, we explore utilizing Large Language Models (LLM) for knowledge graph completion. We consider triples in knowledge graphs as text sequences and introduce an innovative framework called Knowledge Graph LLM (KG- LLM) to model these triples. Our technique employs entity and relation descriptions of a triple as prompts and utilizes the response for predictions. Experiments on various benchmark knowledge graphs demonstrate that our method attains state-of-the-art performance in tasks such as triple classification and relation prediction. We also find that fine-tuning relatively smaller models (e.g., LLaMA-7B, ChatGLM- 6B) outperforms recent ChatGPT and GPT-4.

Introduction. Large knowledge graphs (KG) like FreeBase (Bollacker et al., 2008), YAGO (Suchanek et al., 2007), and WordNet (Miller, 1995) serve as a powerful foundation for numerous critical AI tasks, including semantic search, recommendation (Zhang et al., 2016), and question answering (Cui et al., 2017). A KG is generally a multi-relational graph with entities as nodes and relations as edges. Each edge is depicted as a triplet (head entity, relation, tail entity) (abbreviated as (h, r, t)), signifying the relationship between two entities, for instance, (Steve Jobs, founded, Apple Inc.). Despite their effectiveness, knowledge graphs remain incomplete. This issue leads to the challenge of knowledge graph completion, which aims to evaluate the plausibility of triples that are not present in a knowledge graph. A significant amount of research has been dedicated to knowledge graph completion. One prevalent method is knowledge graph embedding (Wang et al., 2017).

Discussion / Conclusion. The main reasons why KG-LLM performs well are: 1). LLMs contain more general knowledge compared with smaller pre-trained language models. 2). Instruction tuning fills the gap between the pre-trained weights in LLMs and KG triple descriptions. In this work, we propose a novel KG completion method termed KG-LLM. Our approach attains state-of-the-art performance in KG completion tasks such as triple classification and relation prediction. For future work, we plan to apply our KG-LLM as a knowledge-enhanced language model to other NLP tasks and incorporate the structure information of KGs. Furthermore, we are going to explore more effective prompt engineering and context instructions of LLM. Although our method has demonstrated promising results in KG completion using LLMs, it currently lacks the ability to handle KGs that lack textual names or descriptions for entities and relations. Additionally, we have not yet fully utilized the KG structure information, which has the potential to significantly improve results, particularly in the entity prediction task.