text: This manuscript is fairly well-written,  and discusses how the batch normalization step helps to stabilize the scale of the gradients.   Intriguingly, the analysis suggests that using a shallower but wider resnet should provide competitive performance, which is supported by empirical evidence.   This work should help elucidate the structure in the learning, and help to support efforts to improve both learning algorithms and the architecture. Pros: Clean, simple analysis Empirical support suggests that theory captures reasonable effects behind learning Cons: The reasonableness of the assumptions used in the analysis needs a more careful analysis.   In particular, the assumption that all weights are independent is valid only at the first random iteration.   Therefore, the utility of this theory during initialization seems reasonable,  but during learning the theory seems quite tenuous.   I would encourage the authors to discuss their assumptions, and talk about how the math would change as a result of relaxing the assumptions. The empirical support does provide evidence that the theory is reasonable.   However, it is limited to a single dataset.   It would be nice to see that the effect happens more generally.  Second, it is clear that shallow+wide networks may be better than deep+narrow networks,  but it's not clear about how the width is evaluated and supported.  I would encourage the authors to do more extensive experiments and evaluate the architecture further.