text: # Summary This paper presents a new external-memory-based neural network (Neural Map) for handling partial observability in reinforcement learning.  The proposed memory architecture is spatially-structured so that the agent can read/write from/to specific positions in the memory.  The results on several memory-related tasks in 2D and 3D environments show that the proposed method outperforms existing baselines such as LSTM and MQN/FRMQN.  [Pros] - The overall direction toward more flexible/scalable memory is an important research direction in RL. - The proposed memory architecture is new.  - The paper is well-written. [Cons] - The proposed memory architecture is new but a bit limited to 2D/3D navigation tasks. - Lack of analysis of the learned memory behavior. # Novelty and Significance The proposed idea is novel in general.  Though [Gupta et al.] proposed an ego-centric neural memory in the RL context,  the proposed memory architecture is still new in that read/write operations are flexible enough for the agent to write any information to the memory,  whereas [Gupta et al.] designed the memory specifically for predicting free space.  On the other hand, the proposed method is also specific to navigation tasks in 2D or 3D environment,  which is hard to apply to more general memory-related tasks in non-spatial environments.  But, it is still interesting to see that the ego-centric neural memory works well on challenging tasks in a 3D environment. # Quality The experiment does not show any analysis of the learned memory read/write behavior especially for ego-centric neural map and the 3D environment.  It is hard to understand how the agent utilizes the external memory without such an analysis.  # Clarity The paper is overall clear and easy-to-follow except for the following.  In the introduction section, the paper claims that "the expert must set M to a value that is larger than the time horizon of the currently considered task" when mentioning the limitation of the previous work.  In some sense, however, Neural Map also requires an expert to specify the proper size of the memory based on prior knowledge about the task.