text: The paper explores momentum SGD and an adaptive version of momentum SGD which the authors name YF (Yellow Fin). They compare YF to hand tuned momentumSGD and to Adam in several deep learning applications. I found the first part which discusses the theoretical motivation behind YF to be very confusing and misleading: Based on the analysis of 1-dimensional problems, the authors design a framework and an algorithm that  supposedly ensures accelerated convergence. There are two major problems with this approach:-First: Exploring 1-dim functions is indeed a nice way to get some intuition. Yet,  algorithms that work in the 1-dim case do not trivially generalize to high dimensions, and such reasoning might lead to very bad solutions. -Second: Accelerated GD does not benefit over GD in the 1-dim case. And therefore, this is not an appropriate setting to explore acceleration. Concretely, the definition of the generalized condition number $\nu$, and relating it to the standard definition of the condition number $\kappa$, is very misleading. This is since $\kappa =1$ for 1-dim problems, and therefore accelerated GD does not have any benefits over non accelerated GD in this case. However, $\nu$ might be much larger than 1 even in the 1-dim case. Regarding the algorithm itself: there are too many hyper-parameters (which depend on each other) that are tuned (per-dimension). And as I have mentioned, the design of the algorithm is inspired by the analysis of 1-dim quadratic functions. Thus, it is very hard for me to believe that this algorithm works in practice unless very careful fine tuning is employed. The authors mention that their experiments were done without tuning or with very little tuning, which is very mysterious for me. In contrast to the theoretical part, the experiments seems very encouraging. Showing YF to perform very well on several deep learning tasks without (or with very little) tuning. Again, this seems a bit magical or even too good to be truth. I suggest the authors to perform a experiment with say a qaudratic high dimensional function, which is not aligned with the axes in order to illustrate how their method behaves and try to give intuition.