text: The paper proposes an online distillation method, called co-distillation, where the two different models are trained to match the predictions of other model in addition to minimizing its own loss.  The proposed method is applied to two large-scale datasets  and showed to perform better than other baselines such as label smoothing, and the standard ensemble.  The paper is clearly written and was easy to understand.  My major concern is the significance and originality of the proposed method.  As written by the authors, the main contribution of the paper is to apply the codistillation method, which is pretty similar to Zhang et. al (2017), at scale.  But, because from Zhang's method, I don't see any significant difficulty in applying to large-scale problems,  I'm not sure that this can be a significant contribution.  Rather, I think, it would have been better for the authors to apply the proposed methods to a smaller scale problems as well in order to explore more various aspects of the proposed methods including the effects of number of different models.  In this sense, it is also a limitation that the authors showing experiments where only two models are codistillated.  Usually, ensemble becomes stronger as the number of model increases.