text: The author unveils some properties of the resnets, for example, the cosine loss and l2 ratio of the layers.  I think the author should place more focus to study "real" iterative inference with shared parameters rather than analyzing original resnets. In resnet without sharing parameters, it is quite ambiguous to say whether it is doing representation learning or iterative refinement. 1. The cosine loss is not meaningful in the sense that the classification layer is trained on the output of the last residual block and fixed.  Moving the classification layer to early layers will definitely result in accuracy loss.  Even in non-residual network, we can always say that the vector h_{i+1} - h_i is refining h_i towards the negative gradient direction.  The motivation of iterative inference would be to generate a feature that is easier to classify rather than to match the current fixed classifier.  Thus the final classification layer should be retrained for every addition or removal of residual blocks. 2. The l2 ratio. The l2 ratio is small for higher residual layers, I'm not sure how much this phenomenon can prove that resnet is actually doing iterative inference. 3. In section 4.4 it is shown that unrolling the layers can improve the performance of the network.  However, the same can be achieved by adding more unshared layers.  I think the study should focus more on whether shared or unshared is better. 4. Section 4.5 is a bit weak in experiments,  my conclusion is that currently it is still limited by batch normalization and optimization,  the evidence is still not strong enough to show that iterative inference is advantageous / disadvantageous. The the above said, I think the more important thing is how we can benefit from iterative inference interpretation, which is relatively weak in this paper.