text: The authors propose a procedure to generate an ensemble of sparse structured models.  To do this, the authors propose to (1) sample models using SG-MCMC with group sparse prior, (2) prune hidden units with small weights, (3) and retrain weights by optimizing each pruned model.  The ensemble is applied to MNIST classification and language modelling on PTB dataset.  I have two major concerns on the paper.  First, the proposed procedure is quite empirically designed.  So, it is difficult to understand why it works well in some problems.  Particularly. the justification on the retraining phase is weak.  It seems more like to use SG-MCMC to *initialize* models which will then be *optimized* to find MAP with the sparse-model constraints.  The second problem is about the baselines in the MNIST experiments.  The FNN-300-100 model without dropout, batch-norm, etc. seems unreasonably weak baseline.  So, the results on Table 1 on this small network is not much informative practically.  Lastly, I also found a significant effort is also desired to improve the writing.  The following reference also needs to be discussed in the context of using SG-MCMC in RNN. - "Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling", Zhe Gan*, Chunyuan Li*, Changyou Chen, Yunchen Pu, Qinliang Su, Lawrence Carin