before_revision: Word embeddings represent words in a numeric space in such a way that semantic relations between words are encoded as distances and directions in the vector space. Cross-lingual word embeddings map words from one language to the vector space of another language, or words from multiple languages to the same vector space where similar words are aligned.  Cross-lingual embeddings can be used to transfer machine learning models between languages and thereby compensate for insufficient data in less-resourced languages. We use cross-lingual word embeddings to transfer machine learning prediction models for Twitter sentiment between 13 languages. We focus on two transfer mechanisms using the joint numerical space for many languages as implemented in the LASER library : the transfer of trained models, and expansion of training sets with instances from other languages . Our experiments show that the transfer of models between similar languages is sensible, while dataset expansion did not increase the predictive performance .
after_revision: Word embeddings represent words in a numeric space so that semantic relations between words are represented as distances and directions in the vector space. Cross-lingual word embeddings transform vector spaces of different languages so that similar words are aligned. This is done by constructing a mapping between vector spaces of two languages or learning a joint vector space for multiple languages. Cross-lingual embeddings can be used to transfer machine learning models between languages , thereby compensating for insufficient data in less-resourced languages. We use cross-lingual word embeddings to transfer machine learning prediction models for Twitter sentiment between 13 languages. We focus on two transfer mechanisms that recently show superior transfer performance. The first mechanism uses the trained models whose input is the joint numerical space for many languages as implemented in the LASER library . The second mechanism uses large pretrained multilingual BERT language models . Our experiments show that the transfer of models between similar languages is sensible, even with no target language data. The performance of cross-lingual models obtained with the multilingual BERT and LASER library is comparable, and the differences are language-dependent. The transfer with CroSloEngual BERT, pretrained on only three languages, is superior on these and some closely related languages .
edit_actions: R
edit_actions: in such a way
edit_actions: so
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: encoded
edit_actions: represented
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: map words from one language to the vector space of another language, or words from multiple languages to the same vector space where
edit_actions: transform vector spaces of different languages so that
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: A
edit_actions: This is done by constructing a mapping between vector spaces of two languages or learning a joint vector space for multiple languages.
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed
edit_actions: R
edit_actions: and thereby compensate
edit_actions: , thereby compensating
edit_actions: clarity
edit_actions: clarity clarity coherence
edit_actions: R
edit_actions: using the
edit_actions: that recently show superior transfer performance. The first mechanism uses the trained models whose input is the
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed style
edit_actions: R
edit_actions: : the transfer of trained models, and expansion of training sets with instances from other languages
edit_actions: . The second mechanism uses large pretrained multilingual BERT language models
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed clarity
edit_actions: R
edit_actions: while dataset expansion did not increase the predictive performance
edit_actions: even with no target language data. The performance of cross-lingual models obtained with the multilingual BERT and LASER library is comparable, and the differences are language-dependent. The transfer with CroSloEngual BERT, pretrained on only three languages, is superior on these and some closely related languages
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed style