text: The paper consider a method for "weight normalization" of layers of a neural network.   The weight matrix is maintained normalized, which helps accuracy.   However, the simplest way to normalize on a fully connected layer is quadratic (adding squares of weights and taking square root). The paper proposes "FastNorm", which is a way to implicitly maintain the normalized weight matrix using much less computation.   Essentially, a normalization vector is maintained an updated separately. Pros:   Natural method to do weight normalization efficeintly Cons:   A very natural and simple solution that is fairly obvious. Limited experiments