before_revision: Symbolic knowledge (e.g., entities, relations, and facts in a knowledge graph) has become an increasingly popular component of neural-symbolic models applied to machine learning tasks, such as question answering and recommender systems. Besides improving downstream performance, these symbolic structures (and their associated attention weights) are often used to help explain the model's predictions and provide " insights " to practitioners . In this paper, we question the faithfulness of such symbolic explanations . We demonstrate that, through a learned strategy (or even simple heuristics), one can produce deceptively perturbed symbolic structures which maintain the downstream performance of the original structure while significantly deviating from the original semantics . In particular, we train a reinforcement learning policy to manipulate relation types or edge connections in a knowledge graph, such that the resulting downstream performance is maximally preserved. Across multiple models and tasks, our approach drastically alters knowledge graphs with little to no drop in performance. These results raise doubts about the faithfulness of explanations provided by learned symbolic structures and the reliability of current neural-symbolic modelsin leveraging symbolic knowledge .
after_revision: Knowledge graphs (KGs) have helped neural-symbolic models improve performance on various knowledge-intensive tasks, like question answering and item recommendation. By using attention over the KG, such models can also " explain " which KG information was most relevant for making a given prediction . In this paper, we question whether these models are really behaving as we expect . We demonstrate that, through a reinforcement learning policy (or even simple heuristics), one can produce deceptively perturbed KGs which maintain the downstream performance of the original KG while significantly deviating from the original semantics and structure. Our findings raise doubts about KG-augmented models' ability to leverage KG information and provide plausible explanations .
edit_actions: R
edit_actions: Symbolic knowledge (e.g., entities, relations, and facts in a knowledge graph) has become an increasingly popular component of
edit_actions: Knowledge graphs (KGs) have helped
edit_actions: clarity
edit_actions: clarity clarity meaning-changed
edit_actions: R
edit_actions: applied to machine learning tasks, such as
edit_actions: improve performance on various knowledge-intensive tasks, like
edit_actions: meaning-changed
edit_actions: meaning-changed coherence meaning-changed
edit_actions: R
edit_actions: recommender systems. Besides improving downstream performance, these symbolic structures (and their associated attention weights) are often used to help explain the model's predictions and provide
edit_actions: item recommendation. By using attention over the KG, such models can also
edit_actions: clarity
edit_actions: meaning-changed clarity clarity
edit_actions: R
edit_actions: insights
edit_actions: explain
edit_actions: clarity
edit_actions: clarity clarity style
edit_actions: R
edit_actions: to practitioners
edit_actions: which KG information was most relevant for making a given prediction
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed
edit_actions: R
edit_actions: the faithfulness of such symbolic explanations
edit_actions: whether these models are really behaving as we expect
edit_actions: clarity
edit_actions: clarity style clarity
edit_actions: R
edit_actions: learned strategy
edit_actions: reinforcement learning policy
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: symbolic structures
edit_actions: KGs
edit_actions: clarity
edit_actions: clarity clarity meaning-changed
edit_actions: R
edit_actions: structure
edit_actions: KG
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed fluency
edit_actions: R
edit_actions: . In particular, we train a reinforcement learning policy to manipulate relation types or edge connections in a knowledge graph, such that the resulting downstream performance is maximally preserved. Across multiple models and tasks, our approach drastically alters knowledge graphs with little to no drop in performance. These results
edit_actions: and structure. Our findings
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: the faithfulness of explanations provided by learned symbolic structures and the reliability of current neural-symbolic modelsin leveraging symbolic knowledge
edit_actions: KG-augmented models' ability to leverage KG information and provide plausible explanations
edit_actions: clarity
edit_actions: clarity clarity meaning-changed