Inhibitory Interneuron Literature Review
In 2023, I reviewed the scientific literature on inhibitory interneurons. At the time I had studied HTM theory, which presents a simplified high-level model of them. I wanted to learn the details about inhibitory neurons, and to analyse how they effect neural networks.
Anatomy of Inhibitor Interneurons
Let's start with an overview explaining what interneurons are, their function, and the basic types. The term "interneuron" refers to cortical neurons that are not pyramidal neurons. Interneurons make up about 20% of the cortex. The term "inhibitory" means that they supress the activity of other neurons using the GABA neurotransmitter.
INTERNEURONS OF THE NEOCORTICAL INHIBITORY SYSTEM
Henry Markram, Maria Toledo-Rodriguez, Yun Wang, Anirudh Gupta, Gilad Silberberg and Caizhi Wu (2004)
doi: 10.1038/nrn1519Abstract
Mammals adapt to a rapidly changing world because of the sophisticated cognitive functions that are supported by the neocortex. The neocortex, which forms almost 80% of the human brain, seems to have arisen from repeated duplication of a stereotypical microcircuit template with subtle specializations for different brain regions and species. The quest to unravel the blueprint of this template started more than a century ago and has revealed an immensely intricate design. The largest obstacle is the daunting variety of inhibitory interneurons that are found in the circuit. This review focuses on the organizing principles that govern the diversity of inhibitory interneurons and their circuits.
This review article covers the history of the inhibitory neurons and summarizes the state of the art
Inhibitory Plasticity: Balance, Control, and Codependence
Guillaume Hennequin, Everton J. Agnes, and Tim P. Vogels (2017)
doi: 10.1146/annurev-neuro-072116-031005Abstract
Inhibitory neurons, although relatively few in number, exert powerful control over brain circuits. They stabilize network activity in the face of strong feedback excitation and actively engage in computations. Recent studies reveal the importance of a precise balance of excitation and inhibition in neural circuits, which often requires exquisite fine-tuning of inhibitory connections. We review inhibitory synaptic plasticity and its roles in shaping both feedforward and feedback control. We discuss the necessity of complex, codependent plasticity mechanisms to build nontrivial, functioning networks, and we end by summarizing experimental evidence of such interactions.
This article is not surprising, but the data may be useful nonetheless
Precision of Inhibition: Dendritic Inhibition by Individual GABAergic Synapses on Hippocampal Pyramidal Cells Is Confined in Space and Time
Fiona E. Müllner, Corette J. Wierenga, and Tobias Bonhoeffer (2015)
doi: 10.1016/j.neuron.2015.07.003Abstract
Inhibition plays a fundamental role in controlling neuronal activity in the brain. While perisomatic inhibition has been studied in detail, the majority of inhibitory synapses are found on dendritic shafts and are less well characterized. Here, we combine paired patch-clamp recordings and two-photon Ca2+ imaging to quantify inhibition exerted by individual GABAergic contacts on hippocampal pyramidal cell dendrites. We observed that Ca2+ transients from back-propagating action potentials were significantly reduced during simultaneous activation of individual nearby inhibitory contacts. The inhibition of Ca2+ transients depended on the precise spike- timing (time constant < 5 ms) and declined steeply in the proximal and distal direction (length constants 23–28 mm). Notably, Ca2+ amplitudes in spines were inhibited to the same degree as in the shaft. Given the known anatomical distribution of inhibitory synapses, our data suggest that the collective inhibitory input to a pyramidal cell is sufficient to control Ca2+ levels across the entire dendritic arbor with micrometer and millisecond precision.
Algorithmic Analyses
A computational function of inihibition is to create sparse representations
Forming sparse representations by local anti-Hebbian learning
P. Foldiak (1990)
Abstract
How does the brain form a useful representation of its environment? It is shown here that a layer of simple Hebbian units connected by modifiable anti-Hebbian feed-back connections can learn to code a set of patterns in such a way that statistical dependency between the elements of the representation is reduced, while information is preserved. The resulting code is sparse, which is favourable if it is to be used as input to a subsequent supervised associative layer. The operation of the network is demonstrated on two simple problems.
This article presents a comprehensive mathematical analysis of the brain's winner-takes-all algorithm
On the Computational Power of Winner-Take-All
W. Maass (2000)
Abstract
This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-takeall in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity.
This article presents "gain control" as another computational function of inhibition
A New Mechanism for Neuronal Gain Control (or How the Gain in Brains Has Mainly Been Explained)
Nicholas J. Priebe and David Ferster (2002)
doi: 10.1016/S0896-6273(02)00829-2Abstract
One of the more prosaic but necessary features of almost any information processing system is gain control. All such systems must have some way to adjust the relationship between input, which can vary dramatically depending on changes in the environment, and output, which is almost always required to remain within a limited range of amplitudes. While the volume control on a radio or the brightness control on a computer monitor are not the most exciting or highly touted features, imagine such devices without these forms of gain control. Many an engineer can attest to the large effort required to design automatic gain controls in telephones, cameras, and radio transmitters.
Balance and Criticality
There is a category of research about how inhibitory plasticity leads to balanced and critical levels of activity. First, here is a good review article explaining what "criticality" means:
Being critical of criticality in the brain
John M. Beggs and Nicholas Timme (2012)
doi: 10.3389/fphys.2012.00163Abstract
Relatively recent work has reported that networks of neurons can produce avalanches of activity whose sizes follow a power law distribution. This suggests that these networks may be operating near a critical point, poised between a phase where activity rapidly dies out and a phase where activity is amplified over time. The hypothesis that the electrical activity of neural networks in the brain is critical is potentially important, as many simulations suggest that information processing functions would be optimized at the critical point. This hypothesis, however, is still controversial. Here we will explain the concept of criticality and review the substantial objections to the criticality hypothesis raised by skeptics. Points and counter points are presented in dialog form.
Balancing Feed-Forward Excitation and Inhibition via Hebbian Inhibitory Synaptic Plasticity
Yotam Luz, Maoz Shamir (2012)
doi: 10.1371/journal.pcbi.1002334Abstract
It has been suggested that excitatory and inhibitory inputs to cortical cells are balanced, and that this balance is important for the highly irregular firing observed in the cortex. There are two hypotheses as to the origin of this balance. One assumes that it results from a stable solution of the recurrent neuronal dynamics. This model can account for a balance of steady state excitation and inhibition without fine tuning of parameters, but not for transient inputs. The second hypothesis suggests that the feed forward excitatory and inhibitory inputs to a postsynaptic cell are already balanced. This latter hypothesis thus does account for the balance of transient inputs. However, it remains unclear what mechanism underlies the fine tuning required for balancing feed forward excitatory and inhibitory inputs. Here we investigated whether inhibitory synaptic plasticity is responsible for the balance of transient feed forward excitation and inhibition. We address this issue in the framework of a model characterizing the stochastic dynamics of temporally anti-symmetric Hebbian spike timing dependent plasticity of feed forward excitatory and inhibitory synaptic inputs to a single post-synaptic cell. Our analysis shows that inhibitory Hebbian plasticity generates ‘negative feedback’ that balances excitation and inhibition, which contrasts with the ‘positive feedback’ of excitatory Hebbian synaptic plasticity. As a result, this balance may increase the sensitivity of the learning dynamics to the correlation structure of the excitatory inputs.
Discussion
We have studied the computational effect of temporally asymmetric Hebbian plasticity of feed forward inhibition. Hebbian plasticity of inhibition generates negative feedback, in contrast to the positive feedback generated by Hebbian plasticity of excitation. This can be understood by the following intuitive explanation. If the feed forward inhibitory synapse is very strong, then it is less likely that a postsynaptic spike will follow a presynaptic spike. As a result more pre-post spike pairs will fall on the acausal branch of the STDP learning curve than on the causal branch. This, in turn, will depress the strong synapse. On the other hand, if the synapse is weak, then pre and post spike times will be largely uncorrelated and the STDP dynamics will sample uniformly both branches of the STDP curve with equal probability. [...]
Cortical Circuit Dynamics Are Homeostatically Tuned to Criticality In Vivo
Zhengyu Ma, Gina G. Turrigiano, Ralf Wessel, and Keith B. Hengen (2019)
doi: 10.1016/j.neuron.2019.08.031SUMMARY
Homeostatic mechanisms stabilize neuronal activity in vivo, but whether this process gives rise to balanced network dynamics is unknown. Here, we continuously monitored the statistics of network spiking in visual cortical circuits in freely behaving rats for 9 days. Under control conditions in light and dark, networks were robustly organized around criticality, a regime that maximizes information capacity and transmission. When input was perturbed by visual deprivation, network criticality was severely disrupted and subsequently restored to criticality over 48 h. Unexpectedly, the recovery of excitatory dynamics preceded homeostatic plasticity of firing rates by >30 h. We utilized model investigations to manipulate firing rate homeostasis in a cell-typespecific manner at the onset of visual deprivation. Our results suggest that criticality in excitatory networks is established by inhibitory plasticity and architecture. These data establish that criticality is consistent with a homeostatic set point for visual cortical dynamics and suggest a key role for homeostatic regulation of inhibition.
This article presents a simplified neural network and does a lot of math and statistics on it.
Self-Tuned Critical Anti-Hebbian Networks
Marcelo O. Magnasco, Oreste Piro, and Guillermo A. Cecchi (2009)
doi: 10.1103/PhysRevLett.102.258102Abstract
It is widely recognized that balancing excitation and inhibition is important in the nervous system. When such a balance is sought by global strategies, few modes remain poised close to instability, and all other modes are strongly stable. Here we present a simple abstract model in which this balance is sought locally by units following ‘‘anti-Hebbian’’ evolution: all degrees of freedom achieve a close balance of excitation and inhibition and become ‘‘critical’’ in the dynamical sense. At long time scales, a complex ‘‘breakout’’ dynamics ensues in which different modes of the system oscillate between prominence and extinction; the model develops various long-tailed statistical behaviors and may become self-organized critical.