text: This paper uses an information geometric view on hierarchical models to discuss a bias - variance decomposition in Boltzmann machines,  presenting interesting conclusions,  whereby some more care appears to be needed for making these claims.  The paper arrives at the main conclusion that it is possible to reduce both the bias and the variance in a hierarchical model.  The discussion is not specific to deep learning nor to Boltzmann machines, but actually addresses hierarchical exponential family models.  The methods pertaining hierarchical models are interesting and presented in a clear way.  My concern are the following points: The main theorem presents only a lower bound, meaning that it provides no guarantee that the variance can indeed be reduced.  The paper seems to ignore that a model with hidden variables may be singular,  in which case the Fisher metric is not positive definite and the Cramer Rao bound has no meaning.  This interferes with the claims and derivations made in the paper in the case of models with hidden variables.  The problem seems to lie in the fact that the presented derivations assume that an optimal distribution in the data manifold is given (see Theorem 1 and proof), effectively making this a discussion about a fully observed hierarchical model.  In particular, it is not further specified how to obtain θˆB(s) in page 6 before (13).  Also, in page 5 the paper states that ``it is known that the EM-algorithm can obtain the global optimum of Equation (12) (Amari, 2016, Section 8.1.3)''.  However, what is shown in that reference is only that:  (Theorem 8.2., Amari, 2016) ``The KL-divergence decreases monotonically by repeating the E-step and the M-step. Hence, the algorithm converges to an equilibrium.''  A model with hidden variables can have several global and local optimisers  (see, e.g. https://arxiv.org/abs/1709.05276).  The critical points of the EM algorithm can have a non trivial structure,  as has been observed in the case of non negative rank matrix varieties  (see, e.g., https://arxiv.org/pdf/1312.5634.pdf).  OTHER In page 3, ``S_\beta is e-flat and S_\alpha ... '', should this not be the other way around?  (See also page 5 last paragraph of Section 2.)  Please also indicate the precise location in the provided reference.   All pages up to page 5 are introduction.  Section 2.3. as presented is very vague and does not add much to the discussion.  In page 7, please explain E ψ(θˆ )^2 −ψ(θ∗ )^2=0