text: Twin Networks: Using the Future as a Regularizer ** PAPER SUMMARY ** The authors propose to regularize RNN for sequence prediction by forcing states of the main forward RNN to match the state of a secondary backward RNN. Both RNNs are trained jointly and only the forward model is used at test time. Experiments on conditional generation (speech recognition, image captioning), and unconditional generation (MNIST pixel RNN, language models) show the effectiveness of the regularizer. ** REVIEW SUMMARY ** The paper reads well, has sufficient reference. The idea is simple and well explained. Positive empirial results support the proposed regularizer. ** DETAILED REVIEW ** Overall, this is a good paper. I have a few suggestions along the text but nothing major. In related work, I would cite co-training approaches. In effect, you have two view of a point in time, its past and its future and you force these two views to agree, see  (Blum and Mitchell, 1998) or Xu, Chang, Dacheng Tao, and Chao Xu. "A survey on multi-view learning." arXiv preprint arXiv:1304.5634 (2013). I would also relate your work to distillation/model compression which tries to get one network to behave like another. On that point, is it important to train the forward and backward network jointly or could the backward network be pre-trained? In section 2, it is not obvious to me that the regularizer (4) would not be ignored in absence of regularization on the output matrix. I mean, the regularizer could push h^b to small norm, compensating with higher norm for the output word embeddings. Could you comment why this would not happen? In Section 4.2, you need to refer to Table 2 in the text. You also need to define the evaluation metrics used. In this section, why are you not reporting the results from the original Show&Tell paper? How does your implementation compare to the original work? On unconditional generation, your hypothesis on uncertainty is interesting and could be tested. You could inject uncertainty in the captioning task for instance, e.g. consider that multiple version of each word e.g. dogA, dogB, docC which are alternatively used instead of dog with predefined substitution rates. Would your regularizer still be helpful there? At which point would it break?