text: Paper summary: Existing works on multi-task neural networks typically use hand-tuned weights for weighing losses across different tasks. This work proposes a dynamic weight update scheme that updates weights for different task losses during training time by making use of the loss ratios of different tasks. Experiments on two different network indicate that the proposed scheme is better than using hand-tuned weights for multi-task neural networks. Paper Strengths:- The proposed technique seems simple yet effective for multi-task learning. - Experiments on two different network architectures showcasing the generality of the proposed method. Major Weaknesses:- The main weakness of this work is the unclear exposition of the proposed technique. Entire technique is explained in a short section-3.1 with many important details missing. There is no clear basis for the main equations 1 and 2. How does equation-2 follow from equation-1? Where is the expectation coming from? What exactly does ‘F’ refer to? There is dependency of ‘F’ on only one of sides in equations 1 and 2? More importantly, how does the gradient normalization relate to loss weight update? It is very difficult to decipher these details from the short descriptions given in the paper. - Also, several details are missing in toy experiments. What is the task here? What are input and output distributions and what is the relation between input and output? Are they just random noises? If so, is the network learning to overfit to the data as there is no relationship between input and output? Minor Weaknesses:- There are no training time comparisons between the proposed technique and the standard fixed loss learning. - Authors claim that they operate directly on the gradients inside the network. But, as far as I understood, the authors only update loss weights in this paper. Did authors also experiment with gradient normalization in the intermediate CNN layers? - No comparison with state-of-the-art techniques on the experimented tasks and datasets. Clarifications:- See the above mentioned issues with the exposition of the technique. - In the experiments, why are the input images downsampled to 320x320? - What does it mean by ‘unofficial dataset’ (page-4). Any references here? - Why is 'task normalized' test-time loss as good measure for comparison between models in the toy example (Section 4)? The loss ratios depend on initial loss, which is not important for the final performance of the system. Suggestions:- I strongly suggest the authors to clearly explain the proposed technique to get this into a publishable state. - The term ’GradNorm’ seem to be not defined anywhere in the paper. Review Summary:Despite promising results, the proposed technique is quite unclear from the paper. With its poor exposition of the technique, it is difficult to recommend this paper for publication.