text: The paper proposes an analysis on different adaptive regularization techniques for deep transfer learning.  Specifically it focuses on the use of an L2-SP condition that constraints the new parameters to be close to the ones previously learned when solving a source task.  + The paper is easy to read and well organized + The advantage of the proposed regularization against the more standard L2 regularization is clearly visible from the experiments - The idea per se is not new:  there is a list of shallow learning methods for transfer learning based on the same L2 regularization choice [Cross-Domain Video Concept Detection using Adaptive SVMs, ACM Multimedia 2007] [Learning categories from few examples with multi model knowledge transfer, PAMI 2014] [From n to n+ 1: Multiclass transfer incremental learning, CVPR 2013] I believe this literature should be discussed in the related work section - It is true that the L2-SP-Fisher regularization was designed for life-long learning cases with a fixed task,  however, this solution seems to work quite well in the proposed experimental settings.  From my understanding L2-SP-Fisher can be considered the best competitor of L2-SP  so I think the paper should dedicate more space to the analysis of their difference and similarities both from the theoretical and experimental point of view.  For instance: --  adding the L2-SP-Fisher results in table 2 --  repeating the experiments of figure 2 and figure 3 with L2-SP-Fisher