before_revision: One key principle for assessing semantic similarity between texts is to measure the degree of semantic overlap of them by considering word-by-word alignment. However, alignment-based approaches are inferior to the generic sentence vectorsin terms of performance. We hypothesize that the reason for the inferiority of alignment-based methods is due to the fact that they do not distinguish word importance and word meaning. To solve this , we propose to separate word importance and word meaning by decomposing word vectors into their norm and direction , then compute the alignment-based similarity with the help of earth mover's distance . We call the method word rotator's distance (WRD) because direction vectors are aligned by rotation on the unit hypersphere. In addition, to incorporate the advance of cutting edge additive sentence encoders, we propose to re-decompose such sentence vectors into word vectors and use them as inputs to WRD. Empirically, the proposed method outperforms current methods considering the word-by-word alignment including word mover's distance with a big difference; moreover, our method outperforms state-of-the-art additive sentence encoders on the most competitive dataset, STS-benchmark .
after_revision: One key principle for assessing textual similarity is measuring the degree of semantic overlap between two texts by considering the word alignment. Such alignment-based approaches are both intuitive and interpretable; however, they are empirically inferior to the simple cosine similarity between general-purpose sentence vectors. To remedy this, we focus on the fact that the norm of word vectors is a good proxy for word importance, and the angle of them is a good proxy for word similarity. Alignment-based approaches do not distinguish the norm and direction, whereas sentence-vector approaches automatically use the norm as the word importance. Accordingly , we propose to decouple word vectors into their norm and direction then computing the alignment-based similarity using earth mover's distance (optimal transport cost), which we refer to as word rotator's distance . Furthermore, we demonstrate how to grow the norm and direction of word vectors (vector converter); this is a new systematic approach derived from the sentence-vector estimation methods, which can significantly improve the performance of the proposed method. On several STS benchmarks, our simple proposed methods outperformed not only alignment-based approaches but also strong baselines .
edit_actions: R
edit_actions: semantic similarity between texts is to measure
edit_actions: textual similarity is measuring
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: of them by considering word-by-word alignment. However,
edit_actions: between two texts by considering the word alignment. Such
edit_actions: clarity
edit_actions: clarity coherence clarity
edit_actions: R
edit_actions: are
edit_actions: are both intuitive and interpretable; however, they are empirically
edit_actions: meaning-changed
edit_actions: meaning-changed style meaning-changed
edit_actions: R
edit_actions: generic sentence vectorsin terms of performance. We hypothesize that the reason for the inferiority of alignment-based methods is due to the fact that they
edit_actions: simple cosine similarity between general-purpose sentence vectors. To remedy this, we focus on the fact that the norm of word vectors is a good proxy for word importance, and the angle of them is a good proxy for word similarity. Alignment-based approaches
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed
edit_actions: R
edit_actions: word importance and word meaning. To solve this
edit_actions: the norm and direction, whereas sentence-vector approaches automatically use the norm as the word importance. Accordingly
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed clarity
edit_actions: R
edit_actions: separate word importance and word meaning by decomposing word
edit_actions: decouple word
edit_actions: clarity
edit_actions: clarity clarity style
edit_actions: R
edit_actions: , then compute
edit_actions: then computing
edit_actions: clarity
edit_actions: clarity fluency clarity
edit_actions: R
edit_actions: with the help of
edit_actions: using
edit_actions: clarity
edit_actions: clarity clarity style
edit_actions: R
edit_actions: . We call the method
edit_actions: (optimal transport cost), which we refer to as
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed clarity
edit_actions: R
edit_actions: (WRD) because direction vectors are aligned by rotation on the unit hypersphere. In addition, to incorporate the advance of cutting edge additive sentence encoders, we propose to re-decompose such sentence vectors into word vectors and use them as inputs to WRD. Empirically, the proposed method outperforms current methods considering the word-by-word alignment including word mover's distance with a big difference; moreover, our method outperforms state-of-the-art additive sentence encoders on the most competitive dataset, STS-benchmark
edit_actions: . Furthermore, we demonstrate how to grow the norm and direction of word vectors (vector converter); this is a new systematic approach derived from the sentence-vector estimation methods, which can significantly improve the performance of the proposed method. On several STS benchmarks, our simple proposed methods outperformed not only alignment-based approaches but also strong baselines
edit_actions: clarity
edit_actions: clarity meaning-changed clarity