Knowledge Base Embedding By Cooperative Knowledge Distillation

Raphaël Sourty (1) , Jose G. Moreno (1) , Francois-Paul Servant (2) , Lynda Tamine (1)

1 IRIT-IRIS - Recherche d’Information et Synthèse d’Information
2 RENAULT

Knowledge bases are increasingly exploited as gold standard data sources which benefit various knowledge-driven NLP tasks. In this paper, we explore a new research direction to perform knowledge base (KB) representation learning grounded with the recent theoretical framework of knowledge distillation over neural networks. Given a set of KBs, our proposed approach KD-MKB, learns KB embeddings by mutually and jointly distilling knowledge within a dynamic teacher-student setting. Experimental results on two standard datasets show that knowledge distillation between KBs through entity and relation inference is actually observed. We also show that cooperative learning significantly outperforms the two proposed baselines, namely traditional and sequential distillation.

Domaines: Informatique [cs]

Raphaël Sourty, Jose G. Moreno, Francois-Paul Servant, Lynda Tamine. Knowledge Base Embedding By Cooperative Knowledge Distillation. International Conference on Computational Linguistics (COLING 2020), Dec 2020, Barcelone (on line), Spain. pp.5579-5590, ⟨10.18653/v1/2020.coling-main.489⟩. ⟨hal-03172074⟩