Temporal Knowledge Graph Embedding and Reasoning

Abstract

Knowledge Graphs (KGs) have emerged as an efficient way to organize and represent knowledge by storing the underlying relations between entities. Recently, a large amount of research works have been devoted to KG embeddings, aiming at mapping entities and relations in KGs to low-dimensional continuous vector spaces for fast reasoning. KG embedding models have been widely used for different learning tasks over KGs, e.g., KG completion, multi-hop complex reasoning, and KG alignment. Since most structured knowledge is only valid at a specific time point or within a specific interval, a lot of large-scale KGs attach time information into triple facts to capture the temporal dynamics of knowledge in addition to their multi-relational nature. The recent availability of temporal KGs has created a demand for new KG embedding approaches that can model time-aware quadruple facts.<br /> This thesis aims to delve further into the research of temporal KG representation learning and reasoning. Our motivation is to improve the performance of embedding models on temporal KGs by proposing new temporal KG embedding approaches. In this work, we extend three fundamental learning tasks of static KGs to temporal KGs, i.e., temporal KG completion, multi-hop temporal KG reasoning, and temporal entity alignment. We first propose three novel temporal KG embedding models, namely, ATiSE, TeRo, TGeomE, for temporal KG completion tasks. Specifically, ATiSE models the temporal evolution of entity/relation representations using multi-dimensional additive time series decomposition, TeRo defines the evolution of an entity embedding as a rotation over time in the complex vector space and TGeomE performs 4th-order tensor factorization of a temporal KG using multivector embeddings from a multi-dimensional geometric algebra and considers a new linear temporal regularization. Our proposed temporal KG completion models achieved the state-of-the-art at the time of publishing. To tackle the problem of multi-hop temporal KG reasoning, we generate three temporal query datasets from three common temporal KG benchmarks and propose a vector logic-based temporal query embedding framework, TFLEX. TFLEX is the first query embedding framework that can handle first-order logical operations and temporal logical operations at the same time, and answer both multi-hop entity queries and timestamp queries over TKGs. Lastly, we introduce two new temporal KG embedding models based on graph neural networks, TEA-GNN and TREA, for entity alignment between temporal KGs, and propose three new temporal KG datasets as references for evaluating entity alignment methods. TEA-GNN regards timestamps as attentive properties of links between entities and uses a time-aware graph self-attention mechanism to effectively incorporate time information into graph neural networks. Built on the top of TEA-GNN, TREA has a better inductive learning ability to represent new emerging entities and timestamps, and a higher training efficiency on large-scale temporal KGs. We empirically prove that the proposed TEA models significantly outperform the existing static entity alignment methods and temporal KG completion-oriented temporal KG embedding models. Overall, this thesis tackles different challenges of temporal KG embeddings by introducing new tasks, metrics, datasets and models. Experimental results demonstrate that our proposed methods successfully integrate time information into representation learning models of KGs.