text: The paper discusses the problem of optimizing neural networks with hard threshold and proposes a novel solution to it. The problem is of significance because in many applications one requires deep networks which uses reduced computation and limited energy. The authors frame the problem of optimizing such networks to fit the training data as a convex combinatorial problems. However since the complexity of such a problem is exponential, the authors propose a collection of heuristics/approximations to solve the problem. These include, a heuristic for setting the targets at each layer, using a soft hinge loss, mini-batch training and such. Using these modifications the authors propose an algorithm (Algorithm 2 in appendix) to train such models efficiently. They compare the performance of a bunch of models trained by their algorithm against the ones trained using straight-through-estimator (SSTE) on a couple of datasets, namely, CIFAR-10 and ImageNet. They show superiority of their algorithm over SSTE. I thought the paper is very well written and provides a really nice exposition of the problem of training deep networks with hard thresholds. The authors formulation of the problem as one of combinatorial optimization and proposing Algorithm 1 is also quite interesting. The results are moderately convincing in favor of the proposed approach. Though a disclaimer here is that I'm not 100% sure that SSTE is the state of the art for this problem. Overall i like the originality of the paper and feel that it has a potential of reasonable impact within the research community. There are a few flaws/weaknesses in the paper though, making it somewhat lose. - The authors start of by posing the problem as a clean combinatorial optimization problem and propose Algorithm 1. Realizing the limitations of the proposed algorithm, given the assumptions under which it was conceived in, the authors relax those assumptions in the couple of paragraphs before section 3.1 and pretty much throw away all the nice guarantees, such as checks for feasibility, discussed earlier. - The result of this is another algorithm (I guess the main result of the paper), which is strangely presented in the appendix as opposed to the main text, which has no such guarantees. - There is no theoretical proof that the heuristic for setting the target is a good one, other than a rough intuition - The authors do not discuss at all the impact on generalization ability of the model trained using the proposed approach. The entire discussion revolves around fitting the training set and somehow magically everything seem to generalize and not overfit.