text: This paper's main thesis is that automatic metrics like BLEU, ROUGE, or METEOR is suitable for task-oriented natural language generation (NLG).  In particular, the paper presents a counterargument to "How NOT To Evaluate Your Dialogue System..."  where Wei et al argue that automatic metrics are not correlated or only weakly correlated with human eval on dialogue generation.  The authors here show that the performance of various NN models as measured by automatic metrics like BLEU and METEOR is correlated with human eval. Overall, this paper presents a useful conclusion: use METEOR for evaluating task oriented NLG.  However, there isn't enough novel contribution in this paper to warrant a publication.  Many of the details unnecessary:  1) various LSTM model descriptions are unhelpful  given the base LSTM model does just as well on the presented tasks  2) Many embedding based eval methods are proposed  but no conclusions are drawn from any of these techniques.