text: This paper considers the problem of improving sequence generation by learning better metrics.  Specifically, it focuses on addressing the exposure bias problem, where traditional methods such as SeqGAN uses GAN framework and reinforcement learning.  Different from these work, this paper does not use GAN framework.  Instead, it proposed an expert-based reward function training, which trains the reward function (the discriminator) from data that are generated by randomly modifying parts of the expert trajectories.  Furthermore, it also introduces partial reward function that measures the quality of the subsequences of different lengths in the generated data.  This is similar to the idea of hierarchical RL, which divide the problem into potential subtasks, which could alleviate the difficulty of reinforcement learning from sparse rewards.  The idea of the paper is novel.  However, there are a few points to be clarified. In Section 3.2 and in (4) and (5), the authors explains how the action value Q_{D_i} is modeled and estimated for the partial reward function D_i of length L_{D_i}.  But the authors do not explain how the rewards (or action value functions) of different lengths are aggregated together to update the model using policy gradient.  Is it a simple sum of all of them? It is not clear why the future subsequences that do not contain y_{t+1} are ignored for estimating the action value function Q in (4) and (5).  The authors stated that it is for reducing the computation complexity.  But it is not clear why specifically dropping the sequences that do not contain y_{t+1}.  Please clarify more on this point.