text: The paper proposes a technique for quantizing the weights of a neural network, with bit-depth/precision varying on a per-parameter basis.  The main idea is to minimize the number of bits used in the quantization while constraining the loss to remain below a specified upper bound.  This is achieved by formulating an upper bound on the number of bits used via a set of "tolerances";  this upper bound is then minimized while estimating any increase in loss using a first order Taylor approximation. I have a number of questions and concerns about the proposed approach.  First, at a high level, there are many details that aren't clear from the text.  Quantization has some bookkeeping associated with it: In a per-parameter quantization setup it will be necessary to store not just the quantized parameter, but also the number of bits used in the quantization (takes e.g. 4-5 extra bits), and there will be some metadata necessary to encode how the quantized value should be converted back to floating point (e.g., for 8-bit quantization of a layer of weights, usually the min and max are stored).  From Algorithm 1 it appears the quantization assumes parameters in the range [0, 1].  Don't negative values require another bit?  What happens to values larger than 1?  How are even bit depths and associated asymmetries w.r.t. 0 handled  (e.g., three bits can represent -1, 0, and 1, but 4 must choose to either not represent 0 or drop e.g. -1)?  None of these details are clearly discussed in the paper,  and it's not at all clear that the estimates of compression are correct if these bookkeeping matters aren't taken into account properly. Additionally the paper implies that this style of quantization has benefits for compute in addition to memory savings.  This is highly dubious,  since the method will require converting all parameters to a standard bit-depth on the fly (probably back to floating point, since some parameters may have been quantized with bit depth up to 32).  Alternatively custom GEMM/conv routines would be required which are impossible to make efficient for weights with varying bit depths.  So there are likely not runtime compute or memory savings from such an approach. I have a few other specific questions: Are the gradients used to compute \mu computed on the whole dataset or minibatches?  How would this scale to larger datasets?  I am confused by the equality in Equation 8: What happens for values shared by many different quantization bit depths  (e.g., representing 0 presumably requires 1 bit, but may be associated with a much finer tolerance)?  Should "minimization in equation 4" refer to equation 3? In the end, while do like the general idea of utilizing the gradient to identify how sensitive the model might be to quantization of various parameters,  there are significant clarity issues in the paper,  I am a bit uneasy about some of the compression results claimed without clearer description of the bookkeeping,  and I don't believe an approach of this kind has any significant practical relevance for saving runtime memory or compute resources.