text: This paper describes computationally efficient methods for training adversarially robust deep neural networks for image classification. (These methods may extend to other machine learning models and domains as well, but that's beyond the scope of this paper.) The former standard method for generating adversarially images quickly and using them in training was to do a single gradient step to increase the loss of the true label or decrease the loss of an alternate label. This paper shows that such training methods only lead to robustness against these "weak" adversarial examples, leaving the adversarially-trained models vulnerable to multi-step white-box attacks and black-box attacks (adversarial examples generated to attack alternate models). There are two proposed solutions. The first is to generate additional adversarial examples from other models and use them in training. This seems to yield robustness against black-box attacks from held-out models as well. Of course, it requires that you have a somewhat diverse group of models to choose from. If that's the case, why not directly build an ensemble of all the models? An ensemble of neural networks can still be represented as a neural network, although a more computationally costly one. Thus, while this heuristic appears to be useful with current models against current attacks, I don't know how well it will hold up in the future. The second solution is to add random noise before taking the gradient step. This yields more effective adversarial examples, both for attacking models and for training, because it relies less on the local gradient. This is another simple idea that appears to be effective. However, I would be interested to see a comparison to a 2-step gradient-based attack. R+Step-LL can be viewed as a 2-step attack: a random step followed by a gradient step. What if both steps were gradient steps instead? This interpolates between Step-LL and I-Step-LL, with an intermediate computational cost. It would be very interesting to know if R+Step-LL is more or less effective than 2+Step-LL, and how large the difference is. I like that this paper demonstrates the weakness of previous methods, including extensive experiments and a very nice visualization of the loss landscape in two adversarial dimensions. The proposed heuristics seem effective in practice, but they're somewhat ad hoc and there is no analysis of how these heuristics might or might not be vulnerable to future attacks.