text: This paper presents a new reinforcement learning architecture called Reactor by combining various improvements in deep reinforcement learning algorithms and architectures into a single model. The main contributions of the paper are to achieve a better bias-variance trade-off in policy gradient updates, multi-step off-policy updates withdistributional RL, and prioritized experience replay for transition sequences. The different modules are integrated well and the empirical results are very promising. The experiments (though limited to Atari) are well carried out and the evaluation is performed on both sample efficiency and training time. Pros: 1. Nice integration of several recent improvements in deep RL, along with a few novel tricks to improve training. 2. The empirical results on 57 Atari games are impressive, in terms of final scores as well as real-time training speed. Cons: 1. Reactor is still less sample-efficient than Rainbow, with significantly lower scores after 200M frames. While the reactor trains much faster, it does use more parallel compute, so the comparison with Rainbow on wall clock time is  not entirely fair. Would a distributed version of Rainbow perform better in this respect? 2. Empirical comparisons are restricted to the Atari domain. The conclusions of the paper will be much stronger if results are also shown on other environments like Mujoco/Vizdoom/Deepmind Lab. 3. Since the paper introduces a few new ideas like prioritized sequence replay, it would help if a more detailed analysis was performed on the impact of these individual schemes, even if in a model simpler than the Reactor. For instance, one could investigate the impact of prioritized sequence replay in models like multi-step DQN or recurrent DQN. This will help us understand  the impact of each of these ideas in a more comprehensive fashion.