before_revision: An effective method to generate a large number of parallel sentences for training improved neural machine translation (NMT) systems is the use of back-translations of the target-side monolingual data.  Tagging, or using gates, has been used to enable translation models to distinguish between synthetic and natural data. This improves standard back-translation and also enables the use of iterative back-translation on language pairs that underperformed using standard back-translation. This work presents a simplified approach of differentiating between the two data using pretraining and finetuning . The approach - tag-less back-translation - trains the model on the synthetic data and finetunes it on the natural data. Preliminary experiments have shown the approach to continuously outperform the tagging approach on low resource English-Vietnamese neural machine translation . While the need for tagging (noising) the dataset has been removed, the approach outperformed the tagged back-translation approach by an average of 0.4 BLEU  .
after_revision: An effective method to generate a large number of parallel sentences for training improved neural machine translation (NMT) systems is the use of back-translations of the target-side monolingual data. The method was not able to utilize the available huge amount of monolingual data because of the inability of models to differentiate between the authentic and synthetic parallel data. Tagging, or using gates, has been used to enable translation models to distinguish between synthetic and authentic data, improving standard back-translation and also enabling the use of iterative back-translation on language pairs that under-performed using standard back-translation. This work presents pre-training and fine-tuning as a simplified but more effective approach of differentiating between the two data . The approach - tag-less back-translation - trains the model on the synthetic data and fine-tunes it on the authentic data. Experiments have shown the approach to outperform the baseline and standard back-translation by 4.0 and 0.7 BLEU respectively on low resource English-Vietnamese NMT . While the need for tagging (noising) the dataset has been removed, the technique outperformed tagged back-translation by 0.4 BLEU . The approach reached the best scores in less training time than the standard and tagged back-translation approaches .
edit_actions: A
edit_actions: The method was not able to utilize the available huge amount of monolingual data because of the inability of models to differentiate between the authentic and synthetic parallel data.
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed
edit_actions: R
edit_actions: natural data. This improves
edit_actions: authentic data, improving
edit_actions: clarity
edit_actions: coherence clarity clarity
edit_actions: R
edit_actions: enables
edit_actions: enabling
edit_actions: fluency
edit_actions: fluency fluency clarity
edit_actions: R
edit_actions: underperformed
edit_actions: under-performed
edit_actions: fluency
edit_actions: fluency fluency fluency
edit_actions: R
edit_actions: a simplified
edit_actions: pre-training and fine-tuning as a simplified but more effective
edit_actions: coherence
edit_actions: coherence clarity coherence
edit_actions: D
edit_actions: using pretraining and finetuning
edit_actions: clarity
edit_actions: clarity coherence clarity
edit_actions: R
edit_actions: finetunes
edit_actions: fine-tunes
edit_actions: fluency
edit_actions: fluency fluency fluency
edit_actions: R
edit_actions: natural data. Preliminary experiments
edit_actions: authentic data. Experiments
edit_actions: clarity
edit_actions: clarity clarity coherence
edit_actions: R
edit_actions: continuously outperform the tagging approach
edit_actions: outperform the baseline and standard back-translation by 4.0 and 0.7 BLEU respectively
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed
edit_actions: R
edit_actions: neural machine translation
edit_actions: NMT
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: approach outperformed the
edit_actions: technique outperformed
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: R
edit_actions: approach by an average of
edit_actions: by
edit_actions: clarity
edit_actions: clarity clarity clarity
edit_actions: A
edit_actions: . The approach reached the best scores in less training time than the standard and tagged back-translation approaches
edit_actions: meaning-changed
edit_actions: meaning-changed meaning-changed meaning-changed