before_revision: Natural Language Processing (NLP) systems learn harmful societal biases that cause them to extend and proliferate inequality widely, as they are deployed in more and more situations. To address and combat this, the NLP community has come to rely on a variety of metrics to identify and quantify bias in black-box models , which are used to monitor model behaviour and to guide efforts at debiasing. Some of these metrics are intrinsic, and are measured in word embedding spaces, and some are extrinsic, which measure the bias present downstream in the tasks that the word embeddings are plugged into. This research examines whether intrinsic metrics (which are easy to measure) correlate well to extrinsic metrics (which reflect real world bias) . We measure both intrinsic and extrinsic bias across hundreds of trained models covering different tasks and experimental conditions and find that there is no reliable correlation between these metrics that holds in more than extremely specific settings . We advise that efforts to debias embedding spaces be always also paired with measurement of downstream model bias, and suggest that that community direct more effort into making downstream measurement simpler and easier .
after_revision: Natural Language Processing (NLP) systems learn harmful societal biases that cause them to widely proliferate inequality as they are deployed in more and more situations. To address and combat this, the NLP community relies on a variety of metrics to identify and quantify bias in black-box models and to guide efforts at debiasing. Some of these metrics are intrinsic, and are measured in word embedding spaces, and some are extrinsic, which measure the bias present downstream in the tasks that the word embeddings are plugged into. This research examines whether easy-to-measure intrinsic metrics correlate well to real world extrinsic metrics . We measure both intrinsic and extrinsic bias across hundreds of trained models covering different tasks and experimental conditions and find that there is no reliable correlation between these metrics that holds in all scenarios across tasks and languages . We advise that efforts to debias embedding spaces be always also paired with measurement of downstream model bias, and suggest that that community increase effort into making downstream measurement more feasible via creation of additional challenge sets and annotated test data. We additionally release code, a new intrinsic metric, and an annotated test set for gender bias for hatespeech .
edit_actions: R
edit_actions: extend and proliferate inequality widely,
edit_actions: widely proliferate inequality
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: has come to rely
edit_actions: relies
edit_actions: clarity
edit_actions: clarity clarity fluency
edit_actions: D
edit_actions: , which are used to monitor model behaviour
edit_actions: coherence
edit_actions: clarity coherence coherence
edit_actions: R
edit_actions: intrinsic metrics (which are easy to measure)
edit_actions: easy-to-measure intrinsic metrics
edit_actions: clarity
edit_actions: clarity style clarity
edit_actions: R
edit_actions: extrinsic metrics (which reflect real world bias)
edit_actions: real world extrinsic metrics
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: more than extremely specific settings
edit_actions: all scenarios across tasks and languages
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: direct more
edit_actions: increase
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: simpler and easier
edit_actions: more feasible via creation of additional challenge sets and annotated test data. We additionally release code, a new intrinsic metric, and an annotated test set for gender bias for hatespeech
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed