Exponentially many local minima for single neurons

Peter Auer, Mark Herbster, Manfred K. Warmuth

Advances in Neural Information Processing Systems 8 (NIPS 1995)

We show that for a single neuron with the logistic function as the transfer function the number of local minima of the error function based on the square loss can grow exponentially in the dimension.