before_revision: Knowledge graphs (KGs) have helped neural-symbolic models improve performance on various knowledge-intensive tasks, like question answering and item recommendation. By using attention over the KG, such  models can also "explain" which KG information was most relevant for making a given prediction. In this paper, we question whether these models are really behaving as we expect. We demonstrate that, through a reinforcement learning policy (or even simple heuristics), one can produce deceptively perturbed KGs  which maintain the downstream performance of the original KG while significantly deviating from the original  semantics and structure. Our findings raise doubts about KG-augmented models' ability to leverage KG information and provide plausible explanations.
after_revision: Knowledge graphs (KGs) have helped neural models improve performance on various knowledge-intensive tasks, like question answering and item recommendation. By using attention over the KG, such KG-augmented models can also "explain" which KG information was most relevant for making a given prediction. In this paper, we question whether these models are really behaving as we expect. We show that, through a reinforcement learning policy (or even simple heuristics), one can produce deceptively perturbed KGs , which maintain the downstream performance of the original KG while significantly deviating from the original KG's semantics and structure. Our findings raise doubts about KG-augmented models' ability to reason about KG information and give sensible explanations.
edit_actions: R
edit_actions: neural-symbolic
edit_actions: neural
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: A
edit_actions: KG-augmented
edit_actions: meaning-changed
edit_actions: meaning-changed clarity meaning-changed
edit_actions: R
edit_actions: demonstrate
edit_actions: show
edit_actions: clarity
edit_actions: clarity clarity style
edit_actions: A
edit_actions: ,
edit_actions: fluency
edit_actions: fluency fluency fluency
edit_actions: A
edit_actions: KG's
edit_actions: clarity
edit_actions: meaning-changed clarity clarity
edit_actions: R
edit_actions: leverage
edit_actions: reason about
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: provide plausible
edit_actions: give sensible
edit_actions: clarity
edit_actions: clarity style clarity