text: This paper proposes several client-server neural network gradient update strategies aimed at reducing uplink usage while maintaining prediction performance. The main approaches fall into two categories: structured, where low-rank/sparse updates are learned, and sketched, where full updates are either sub-sampled or compressed before being sent to the central server. Experiments are based on the federated averaging algorithm. The work is valuable, but has room for improvement. The paper is mainly an empirical comparison of several approaches, rather than from theoretically motivated algorithms. This is not a criticism, however, it is difficult to see the reason for including the structured low-rank experiments in the paper (itAs a reader, I found it difficult to understand the actual procedures used. For example, what is the difference between the random mask update and the subsampling update (why are there no random mask experiments after figure 1, even though they performed very well)? How is the structured update "learned"? It would be very helpful to include algorithms. It seems like a good strategy is to subsample, perform Hadamard rotation, then quantise. For quantization, it appears that the HD rotation is essential for CIFAR, but less important for the reddit data. It would be interesting to understand when HD works and why, and perhaps make the paper more focused on this winning strategy, rather than including the low-rank algo. If convenient, could the authors comment on a similarly motivated paper under review at iclr 2018: VARIANCE-BASED GRADIENT COMPRESSION FOR EFFICIENT DISTRIBUTED DEEP LEARNING pros:- good use of intuition to guide algorithm choices - good compression with little loss of accuracy on best strategy - good problem for FA algorithm / well motivated cons:- some experiment choices do not appear well motivated / inclusion is not best choice - explanations of algos / lack of 'algorithms' adds to confusion a useful reference:  Strom, Nikko. "Scalable distributed dnn training using commodity gpu cloud computing." Sixteenth Annual Conference of the International Speech Communication Association. 2015.