text: The paper presents a method to parametrize unitary matrices in an RNN as a Kronecker product of smaller matrices.  Given N inputs and output, this method allows one to specify a linear transformation with O(log(N)) parameters, and perform a forward and backward pass in O(Nlog(N)) time.   In addition a relaxation is performed allowing each constituent to deviate a bit from unitarity (“soft unitary constraint”). The paper shows nice results on a number of small tasks.  The idea is original to the best of my knowledge and is presented clearly. I especially like the idea of “soft unitary constraint” which can be applied very efficiently in this factorized setup.  I think this is the main contribution of this work. However the paper in its current form has a number of problems: - The authors state that a major shortcoming of previous (efficient) unitary RNN methods is the lack of ability to span the entire space of unitary matrices.  This method presents a family that can span the entire space, but the efficient parts of this family (which give the promised speedup) only span a tiny fraction of it,  as they require only O(log(N)) params to specify an O(N^2) unitary matrix.  Indeed in the experimental section only those members are tested. - Another claim that is made is that complex numbers are key, and again the argument is the need to span the entire space of unitary matrices,  but the same comment still hold - that is not the space this work is really dealing with,  and no experimental evidence is provided that using complex numbers was really needed. - In the experimental section an emphasis is made as to how small the number of recurrent params are,  but at the same time the input/output projections are very large, leaving the reader wondering if the workload simply shifted from the RNN to the projections.  This needs to be addressed. - Another aspect of the previous points is that it’s not clear if stacking KRU layers will work well.  This is important  as stacking LSTMs is a common practice.  Efficient KRU span a restricted subspace whose elements might not compose into structures that are expressive enough.  One way to overcome this potential problem is to add projection matrices between layers that will do some mixing,  but this will blow the number of parameters.  This needs to be explored. - The authors claim that the soft unitary constraint was key for the success of the network,  yet no details are provided as to how this constraint was applied,  and no analysis was made for its significance.