text: This paper considers the problem of autonomous lane changing for self-driving cars in multi-lane multi-agent slot car setting.  The authors propose a new learning strategy called Q-masking which couples well a defined low level controller with a high level tactical decision making policy. The authors rightly say that one of the skills an autonomous car must have is the ability to change lanes,  however this task is not one of the most difficult for autonomous vehicles to achieve and this ability has already been implemented in real vehicles.   Real vehicles also decouple wayfinding with local vehicle control, similar to the strategy employed here.   To make a stronger case for this research being relevant to the real autonomous driving problem, the authors would need to compare their algorithm to a real algorithm and prove that it is more “data efficient.”   This is a difficult comparison  since the sensing strategies employed by real vehicles – LIDAR, computer vision, recorded, labeled real maps are vastly different from the slot car model proposed by the authors.   In term of impact, this is a theoretical paper looking at optimizing a sandbox problem where the results may be one day applicable to the real autonomous driving case. In this paper the authors investigate “the use and place” of deep reinforcement learning in solving the autonomous lane change problem they propose a framework that uses Q-learning to learn “high level tactical decisions” and introduce “Q-masking” a way of limiting the problem that the agent has to learn to force it to learn in a subspace of the Q-values. The authors claim that “By relying on a controller for low-level decisions we are also able to completely eliminate collisions during training or testing, which makes it a possibility to perform training directly on real systems.”   I am not sure what is meant by this since in this paper the authors never test their algorithm on real systems  and in real systems it is not possible to completely eliminate collisions.   If it were, this would be a much sought breakthrough.  Additionally for their experiment authors use the SUMO top view driving simulator.   This choice makes their algorithm not currently relevant to most autonomous vehicles that use ego-centric sensing.   This paper presents a learning algorithm that can “outperform a greedy baseline in terms of efficiency” and “humans driving the simulator in terms of safety and success” within their top view driving game.   The game can be programmed to have an “n” lane highway, where n could reasonable go up to five to represent larger highways.   The authors limit the problem by specifying that all simulated cars must operate between a preset minimum and maximum and follow a target (random) speed within these limits.   Cars follow a fixed model of behavior, do not collide with each other and cannot switch lanes.   It is unclear if the simulator extends beyond a single straight section of highway, as shown in Figure 1.   The agent is tasked with driving the ego-car down the n-lane highway and stopping at “the exit” in the right hand lane D km from the start position.  The authors use deep Q learning from Mnih et al 2015 to learn their optimal policy.   They use a sparse reward function of +10 for reaching the goal and -10x(lane difference from desired lane) as a penalty for failure.   This simple reward function is possible because the authors do not require the ego car to obey speed limits or avoid collisions.     The authors limit what the car is able to do  – for example it is not allowed to take actions that would get it off the highway.   This makes the high level learning strategy more efficient  because it does not have to explore these possibilities (Q-masking).   The authors claim that this limitation of the simulation is made valid by the ability of the low level controller to incorporate prior knowledge and perfectly limit these actions.   In the real world, however, it is unlikely that any low level controller would be able to do this perfectly. In terms of evaluation, the authors do not compare their result against any other method.   Instead, using only one set of test parameters, the authors compare their algorithm to a “greedy baseline” policy that is specified a “always try to change lanes to the right until the lane is correct” then it tries to go as fast as possible while obeying the speed limit and not colliding with any car in front.    It seems that baseline is additionally constrained vs the ego car due to the speed limit and the collision avoidance criteria and is not a fair comparison.   So given a fixed policy and these constraints it is not surprising that it underperforms the Q-masked Q-learning algorithm.   With respect to the comparison vs. human operators of the car simulation, the human operators were not experts.   They were only given “a few trials” to learn how to operate the controls before the test.   It was reported that the human participants “did not feel comfortable” with the low level controller on,  possibly indicating that the user experience of controlling the car was less than ideal.   With the low level controller off, collisions became possible.   It is possibly not a fair claim to say that human drivers were “less safe” but rather that it was difficult to play the game or control the car with the safety module on.   This could be seen as a game design issue.   It was not clear from this presentation how the human participants were rewarded for their performance.   In more typical HCI experiments the gender distribution and ages ranges of participants are specified as well as how participants were recruited and how the game was motivated, including compensation (reward) are specified.   Overall, this paper presents an overly simplified game simulation with a weak experimental result.