NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:8871
Title:Structured and Deep Similarity Matching via Structured and Deep Hebbian Networks


		
This paper presents a local learning rule for similarity matching in multilayer neural networks. Similarity matching means that the pairwise dot products of input vectors are preserved in output vectors. The authors factorize the global similarity matching cost function and use this to develop local cost functions for each synapse. They generalize this to deep structured networks, and derive a learning algorithm with Hebbian training between layers and anti-Hebbian training within layers. The authors demonstrate that these networks learn potentially useful hierarchical features. The reviewers agreed that this paper provided a novel, elegant algorithmic contribution. There were concerns that the authors did not successfully show that the hierarchical representations learned actually provided any utility, but it was decided in discussion that despite this concern, the paper was interesting enough to accept.