The paper provides a mean-field analysis of infinitely wide neural networks with quantized activations, proposing a relation between the choice of initialization hyper-parameters and the maximal depth by primarily by considering how correlations between two inputs propagate through the network at initialization as well as numerical stability issues. All reviewers agree that it is a good contribution.