text: The paper proposes an interesting alternative to recent approaches to learning from logged bandit feedback, and validates their contribution in a reasonable experimental comparison. The clarity of writing can be improved (several typos in the manuscript, notation used before defining, missing words, poorly formatted citations, etc.). Implementing the approach using recent f-GANs is an interesting contribution and may spur follow-up work. There are several lingering concerns about the approach (detailed below) that detract from the quality of their contributions. [Major] In Lemma 1, L(z) is used before defining it. Crucially, additional assumptions on L(z) are necessary (e.g. |L(z)| <= 1 for all z. If not, a trivial counter-example is: set L(z) >> 1 for all z and Lemma 1 is violated). It is unclear how crucially this additional assumption is required in practice (their expts with Hamming losses clearly do not satisfy such an assumption). [Minor] Typo: Section 3.2, first equation; the integral equals D_f(...) + 1 (not -1). [Crucial!] Eqn10: Expected some justification on why it is fruitful to *lower-bound* the divergence term, which contributes to an *upper-bound* on the true risk. [Crucial!] Algorithm1: How is the condition of the while loop checked in a tractable manner? [Minor] Typos: Initilization -> Initialization, Varitional -> Variational [Major] Expected an additional "baseline" in the expts -- Supervised but with the neural net policy architecture (NN approaches outperforming Supervised on LYRL dataset was baffling before realizing that Supervised is implemented using a linear CRF). [Major] Is there any guidance for picking the new regularization hyper-parameters (or at least, a sensible range for them)? [Minor] The derived bounds depend on M, an a priori upper bound on the Renyi divergence between the logging policy and any new policy. It's unclear that such a bound  can be tractably guessed (in contrast, prior work uses an upper bound on the importance weight -- which is simply 1/(Min action selection prob. by logging policy) ).