text: This paper proposes an empirical measure of the intrinsic dimensionality of a neural network problem.  Taking the full dimensionality to be the total number of parameters of the network model, the authors assess intrinsic dimensionality by randomly projecting the network to a domain with fewer parameters (corresponding to a low-dimensional subspace within the original parameter), and then training the original network while restricting the projections of its parameters to lie within this subspace.  Performance on this subspace is then evaluated relative to that over the full parameter space (the baseline).  As an empirical standard, the authors focus on the subspace dimension that achieves a performance of 90% of the baseline.  The authors then test out their measure of intrinsic dimensionality for fully-connected networks and convolutional networks, for several well-known datasets,  and draw some interesting conclusions. Pros:* This paper continues the recent research trend towards a better characterization of neural networks and their performance.  The authors show a good awareness of the recent literature,  and to the best of my knowledge, their empirical characterization of the number of latent parameters is original.  * The characterization of the number of latent variables is an important one,  and their measure does perform in a way that one would intuitively expect.  For example, as reported by the authors, when training a fully-connected network on the MNIST image dataset, shuffling pixels does not result in a change in their intrinsic dimensionality.  For a convolutional network the observed 3-fold rise in intrinsic dimension is explained by the authors as due to the need to accomplish the classification task while respecting the structural constraints of the convnet. * The proposed measures seem very practical -  training on random projections uses far fewer parameters than in the original space (the baseline), and presumably the cost of determining the intrinsic dimensionality would presumably be only a fraction of the cost of this baseline training. * Except for the occasional typo or grammatical error, the paper is well-written and organized.  The issues are clearly identified, for the most part (but see below...). Cons:* In the main paper, the authors perform experiments and draw conclusions without taking into account the variability of performance across different random projections.  Variance should be taken into account explicitly, in presenting experimental results and in the definition and analysis of the empirical intrinsic dimension itself.  How often does a random projection lead to a high-quality solution, and how often does it not? * The authors are careful to point out that training in restricted subspaces cannot lead to an optimal solution for the full parameter domain unless the subspace intersects the optimal solution region (which in general cannot be guaranteed).  In their experiments (FC networks of varying depths and layer widths for the MNIST dataset), between projected and original solutions achieving 90% of baseline performance, they find an order of magnitude gap in the number of parameters needed.  This calls into question the validity of random projection as an empirical means of categorizing the intrinsic dimensionality of a neural network. * The authors then go on to propose that compression of the network be achieved by random projection to a subspace of dimensionality greater than or equal to the intrinsic dimension.  However, I don't think that they make a convincing case for this approach.  Again, variation is the difficulty:  two different projective subspaces of the same dimensionality can lead to solutions that are extremely different in character or quality.  How then can we be sure that our compressed network can be reconstituted into a solution of reasonable quality, even when its dimensionality greatly exceeds the intrinsic dimension? * The authors argue for a relationship between intrinsic dimensionality and the minimum description length (MDL) of their solution,  in that the intrinsic dimensionality should serve as an upper bound on the MDL.  However they don't formally acknowledge that there is no standard relationship between the number of parameters and the actual number of bits needed to represent the model -  it varies from setting to setting, with some parameters potentially requiring many more bits than others.  And given this uncertain connection, and given the lack of consideration given to variation in the proposed measure of intrinsic dimensionality, it is hard to accept that "there is some rigor behind" their conclusion that LeNet is better than FC networks for classification on MNIST  because its empirical intrinsic dimensionality score is lower. * The experimental validation of their measure of intrinsic dimension could be made more extensive.  In the main paper, they use three image datasets - MNIST, CIFAR-10 and ImageNet.  In the supplemental information, they report intrinsic dimensions for reinforcement learning and other training tasks on four other sets. Overall, I think that this characterization does have the potention to give insights into the performance of neural networks, provided that variation across projections is properly taken into account.  For now, more work is needed.