before_revision: Visual Question Answering (VQA ) models tend to rely on the language bias and thus fail to learn the reasoning from visual knowledge , which is however the original intention of VQA . In this paper, we propose a novel cause-effect look at the language bias , where the bias is formulated as the direct  effect of question on answer from the view of causal inference. The effectcan be captured by counterfactual VQA, where the image had not existed in an imagined scenario. Our proposed cause-effect look 1) is general to any baseline VQA architecture , 2) achieves significant improvement on the language-bias sensitive VQA-CP dataset , and 3) fills the theoretical gap in recent language prior based works .
after_revision: Recent VQA models may tend to rely on language bias as a shortcut and thus fail to sufficiently learn the multi-modal knowledge from both vision and language . In this paper, we investigate how to capture and mitigate language bias in VQA. Motivated by causal effects, we proposed a novel counterfactual inference framework, which enables us to capture the language bias as the direct causal effect of questions on answers and reduce the language bias by subtracting the direct language effect from the total causal effect . Experiments demonstrate that our proposed counterfactual inference framework 1) is general to various VQA backbones and fusion strategies , 2) achieves competitive performance on the language-bias sensitive VQA-CP dataset while performs robustly on the balanced VQA v2 dataset .
edit_actions: R
edit_actions: Visual Question Answering (VQA ) models
edit_actions: Recent VQA models may
edit_actions: clarity
edit_actions: clarity style clarity
edit_actions: R
edit_actions: the language bias
edit_actions: language bias as a shortcut
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: learn the reasoning from visual knowledge , which is however the original intention of VQA
edit_actions: sufficiently learn the multi-modal knowledge from both vision and language
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed clarity
edit_actions: R
edit_actions: propose a novel cause-effect look at
edit_actions: investigate how to capture and mitigate language bias in VQA. Motivated by causal effects, we proposed a novel counterfactual inference framework, which enables us to capture
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed
edit_actions: D
edit_actions: , where the bias is formulated
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: A
edit_actions: causal effect of questions on answers and reduce the language bias by subtracting the direct language effect from the total causal
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed
edit_actions: R
edit_actions: of question on answer from the view of causal inference. The effectcan be captured by counterfactual VQA, where the image had not existed in an imagined scenario. Our proposed cause-effect look
edit_actions: . Experiments demonstrate that our proposed counterfactual inference framework
edit_actions: coherence
edit_actions: coherence clarity meaning-changed
edit_actions: R
edit_actions: any baseline VQA architecture
edit_actions: various VQA backbones and fusion strategies
edit_actions: meaning-changed
edit_actions: meaning-changed style meaning-changed
edit_actions: R
edit_actions: significant improvement
edit_actions: competitive performance
edit_actions: clarity
edit_actions: clarity clarity style
edit_actions: R
edit_actions: , and 3) fills the theoretical gap in recent language prior based works
edit_actions: while performs robustly on the balanced VQA v2 dataset
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed