Modeling Neural Populations with Mixture Models

Abstract

A Winner Takes it All Network (WTAn) is a special kind of artificial neural network for classification that has been studied in computational neuroscience because it provides a simple model for a common connectivity pattern in the brain cortex. Interestingly it has been shown that WTAns are analogous to the online K-means algorithm.

One of the known issues with online K-means/WTAns is the uneven membership assignment, where occasionally and depending on initialization one centroid is assigned responsibility over multiple clusters of data points while others are left without any.

By generalizing the online K-means to a Gaussian Mixture parameter inference problem we find a natural solution to this problem without compromising the formulation of the problem using a WTAn. Furthermore using a Poisson Mixture Model we formulate an algorithm that we think is more fitting to represent neural populations and their dynamics.

Finally, we do some simulations and demonstrate the potential of the proposed algorithm. This work is very similar to “Keck, 2012” and “Moraitis T., 2022” specially in terms of the theory behind it, but the proposed algorithm differs from those proposed in these papers. We argue that the framework used here and in the aforementioned papers provide a probabilistic interpretation of the underlying learning process.

Generative fields for each one of the 50 neurons after 30 epochs of training.

Schematic representation of the WTAn and its associated weight matrix.

Want to know more?

🚀 Check out the GitHub repo. You can find a copy of the written report of the project there.

References

2022

SoftHebb: Bayesian inference in unsupervised Hebbian soft winner-take-all networks

Timoleon Moraitis, Dmitry Toichkin, Adrien Journé , and 2 more authors

Neuromorphic Computing and Engineering, Dec 2022

2012

Feedforward Inhibition and Synaptic Scaling – Two Sides of the Same Coin?

Christian Keck, Cristina Savin, and Jörg Lücke

PLOS Computational Biology, Mar 2012

Publisher: Public Library of Science

Abs

Feedforward inhibition and synaptic scaling are important adaptive processes that control the total input a neuron can receive from its afferents. While often studied in isolation, the two have been reported to co-occur in various brain regions. The functional implications of their interactions remain unclear, however. Based on a probabilistic modeling approach, we show here that fast feedforward inhibition and synaptic scaling interact synergistically during unsupervised learning. In technical terms, we model the input to a neural circuit using a normalized mixture model with Poisson noise. We demonstrate analytically and numerically that, in the presence of lateral inhibition introducing competition between different neurons, Hebbian plasticity and synaptic scaling approximate the optimal maximum likelihood solutions for this model. Our results suggest that, beyond its conventional use as a mechanism to remove undesired pattern variations, input normalization can make typical neural interaction and learning rules optimal on the stimulus subspace defined through feedforward inhibition. Furthermore, learning within this subspace is more efficient in practice, as it helps avoid locally optimal solutions. Our results suggest a close connection between feedforward inhibition and synaptic scaling which may have important functional implications for general cortical processing.