# Adaptive Coding Efficiency in Recurrent
Cortical Circuits via Gain Control

Lyndon R. Duong1, Colin Bredenberg1, David J. Heeger1,2, Eero P. Simoncelli1,3
1Center for Neural Science, New York University, New York, NY
2Department of Psychology, New York University, New York, NY
3Center for Computational Neuroscience, Flatiron Institute, New York, NY
{lyndon.duong, cjb617, david.heeger<EMAIL_ADDRESS>
###### Abstract

Sensory systems across all modalities and species exhibit adaptation to
continuously changing input statistics. Individual neurons have been shown to
modulate their response gains so as to maximize information transmission in
different stimulus contexts. Experimental measurements have revealed
additional, nuanced sensory adaptation effects including changes in response
maxima and minima, tuning curve repulsion from the adapter stimulus, and
stimulus-driven response decorrelation. Existing explanations of these
phenomena rely on changes in inter-neuronal synaptic efficacy, which, while
more flexible, are unlikely to operate as rapidly or reversibly as single
neuron gain modulations. Using published V1 population adaptation data, we
show that propagation of single neuron gain changes in a recurrent network is
sufficient to capture the entire set of observed adaptation effects. We
propose a novel adaptive efficient coding objective with which single neuron
gains are modulated, maximizing the fidelity of the stimulus representation
while minimizing overall activity in the network. From this objective, we
analytically derive a set of gains that optimize the trade-off between
preserving information about the stimulus and conserving metabolic resources.
Our model generalizes well-established concepts of single neuron adaptive gain
control to recurrent populations, and parsimoniously explains experimental
adaptation data.

##  1 Introduction

Some of the earliest neurophysiological recordings showed that repeated or
prolonged stimulus presentation leads to a relative decrease in neural
responses (Adrian and Zotterman, , 1926). Indeed, neurons across different
species, brain areas, and sensory modalities adjust their gains (i.e. input-
output sensitivity) in response to recent stimulus history (Kohn, , 2007;
Weber et al., , 2019, for reviews). Gain control provides a mechanism for
single neurons to rapidly and reversibly adapt to different stimulus contexts
(Abbott et al., , 1997; Brenner et al., , 2000; Fairhall et al., , 2001;
Młynarski and Hermundstad, , 2021; Muller et al., , 1999) while preserving
synaptic weights that serve to represent features that remain consistent
across contexts (Ganguli and Simoncelli, , 2014). From a normative standpoint,
this allows a single neuron to adjust the dynamic range of its responses to
accommodate changes in input statistics (Laughlin, , 1981; Fairhall et al., ,
2001) – a core tenet of theories of efficient sensory coding (Attneave, ,
1954; Barlow, , 1961).

Experimental measurements, however, reveal that adaptation induces additional
complex changes in neural responses, including tuning-dependent reductions in
both response maxima and minima (Movshon and Lennie, , 1979), tuning curve
repulsion (Shen et al., , 2015; Hershenhoren et al., , 2014; Yaron et al., ,
2012), and stimulus-driven decorrelation (Benucci et al., , 2013; Wanner and
Friedrich, , 2020; Gutnisky and Dragoi, , 2008; Muller et al., , 1999).
Although coding efficiency and gain-mediated adaptation is well studied in
single neurons, it appears as though these nuanced empirical observations
require a more complex adaptation mechanism, involving joint coordination
among neurons in the population. Indeed, to explain these phenomena, previous
studies have relied on adaptive changes in feedforward or recurrent synaptic
efficacy (i.e. by changing the entire network’s set of synaptic weights;
Wainwright et al., , 2001; Westrick et al., , 2016; Rast and Drugowitsch, ,
2020; Młynarski and Hermundstad, , 2021). However, this requires synaptic
weights to continuously remap under different statistical contexts, which may
change significantly and transiently at short time scales.

Here, we hypothesize that adaptation effects reported in neural population
recording data can be explained by combining normative theory with a
mechanistic recurrent population model that includes single neuron gain
modulation. The primary contributions of our study are as follows:

  1. 1.

We introduce an analytically tractable recurrent neural network (RNN)
architecture for adaptive gain control, in which single neurons adjust their
gains in response to novel stimulus statistics. The model respects
experimental evidence that cortical anatomy is dominated by recurrence
(Douglas and Martin, , 2007), allowing the effects of single neuron gain
changes to propagate through lateral connections.

  2. 2.

We propose a novel adaptive efficient coding objective for adjustment of the
single neuron gains, which optimizes coding fidelity of the stimulus ensemble,
subject to metabolic and homeostatic constraints.

  3. 3.

Through numerical simulations, we compare model predictions to experimental
measurements of cat V1 neurons responding to a sequence of gratings drawn from
an ensemble with either uniform or biased orientation probability (Benucci et
al., , 2013). We show that adaptive adjustment of neural gains, with no
changes in synaptic strengths, parsimoniously captures the full set of
adaptation phenomena observed in the data.

Figure 1: Recurrent adaptation model. A) A population of recurrently-connected
orientation-tuned cells receives external feedforward drive (purple arrows)
from a presented oriented grating stimulus, randomly sampled from a set of
possible orientations. The width of the arrow denotes the strength of the
drive, and indicates that the center neuron is tuned towards the horizontal-
oriented stimuli. The feedforward drive of each neuron is multiplicatively
modulated by its a scalar gain (orange dials). Lateral recurrent input between
neurons is denoted by green arrows. Recurrent connectivity is all-to-all, with
synaptic strengths determined by the distance between neurons’ preferred
feedforward orientation. Output responses (red) of each neuron are a function
of both feedforward drive and recurrent drive. B) Response tuning curves for
orientation-tuned units to stimuli presented with uniform probability (left
column), or biased probability (right column). Middle row shows recordings of
neurons in visual area V1 of cats, aggregated over 11 sessions. Bottom row
shows model responses. Shaded regions are standard error of the mean (SEM).

##  2 Related Work

#### Models of statistical adaptation in neural populations.

While evidence for adaptive efficient coding via gain modulation in single
neurons is relatively well understood (Fairhall et al., , 2001; Nagel and
Doupe, , 2006; Młynarski and Hermundstad, , 2021), the question of whether
neural population adaptation can be explained by efficient coding and gain
modulation remains underexplored. Normative models of population adaptation
have generally relied on synaptic plasticity (i.e. between-neuron synaptic
weight adjustments) as the mechanism mediating adaptation (Młynarski and
Hermundstad, , 2021; Rast and Drugowitsch, , 2020; Pehlevan and Chklovskii, ,
2015; Lipshutz et al., , 2023; Wainwright et al., , 2001; Westrick et al., ,
2016). For example, Westrick et al.,  (2016) argue that empirical observations
of V1 neural populations (Benucci et al., , 2013) can be explained by adapting
normalization weights (parameterized by all-to-all synaptic connections) to
different stimulus statistical contexts. The major downside of this approach
is that changes in synaptic weights require $\mathcal{O}(N^{2})$ adaptation
parameters, for a population of size $N$. Here, we examine the effects of
classical single-neuron adaptive gain modulation on responses of a
recurrently-connected population, and demonstrate that these are sufficient to
explain adaptation phenomena, while requiring only $\mathcal{O}(N)$ adaptation
parameters. Holding the synaptic weights fixed prevents overfitting, and
allows the network to remain stable across input contexts. Network stability
is also relevant for contemporary machine learning applications that rely on
adaptive adjustments to changing input statistics (e.g. Hu et al., , 2022;
Ballé et al., , 2020; Mohan et al., , 2021).

The adaptation model most similar to ours, developed by Gutierrez and Denève,
(2019), proposes an adaptive recurrent spiking neural network whose dynamics
are derived from an efficient coding objective. Our model is complementary to
this, but is simpler and more tractable, providing an analytic solution for
population steady-state responses that facilitates comparisons to experimental
data. Finally, recent work (published while this manuscript was being written)
uses gain control as a normative population adaptation mechanism, but with the
central goal of statistically whitening neural responses, while ignoring the
means of responses (i.e. redundancy reduction via decorrelation and variance
equalization; Duong et al., , 2023). Here, we demonstrate that our model
captures adaptive effects involving mean responses as well as population
response redundancy reduction, but that its steady-state responses are not
whitened. We show that these deviations from whitening are similar to those
seen in the neural recordings analyzed here.

#### Recurrent circuitry in sensory cortex.

It is well known that recurrent excitation dominates cortical circuits
(Douglas and Martin, , 2007). In early sensory areas, a series of optogenetic
inactivation experiments showed that recurrent excitation in cortex serves to
progressively amplify thalamic inputs (Reinhold et al., , 2015; Lien and
Scanziani, , 2013). In the context of sensory adaptation, King et al.,  (2016)
performed silencing experiments in mice to show that the majority of
adaptation effects seen in V1 arise from local activity-dependent processes,
rather than being inherited from depressed thalamic responses upstream.
Similarly, in monkey V1 neurophysiological recordings, Westerberg et al.,
(2019) used current source density analyses to show that stimulus-driven
adaptation is primarily due to recurrent intracortical effects rather than
feedforward effects. We leverage these functional observations, along with
anatomical measurements of intracortical synaptic connectivity (Ko et al., ,
2011; Lee et al., , 2016; Rossi et al., , 2020) to inform the recurrent
architecture used in our study.

##  3 An Analytically Tractable RNN with Gain Modulation

###  3.1 Notation

We denote matrices with capital boldface letters (e.g. ${\mathbf{W}}$),
vectors as lowercase boldface letters (e.g. ${\mathbf{r}}$), and scalar
quantities as non-boldface letters (e.g. $N,\alpha$). The
$\operatorname{diag(\cdot)}$ operator forms a diagonal matrix by embedding the
elements of a $K$-dimensional vector onto the main diagonal of a $K\times K$
matrix whose off-diagonal elements are zero. $\circ$ is the Hadamard (i.e.
element-wise) product. $\mathbb{S}_{+}^{N}$ is the space of $N\times N$
symmetric positive definite matrices.

###  3.2 Adaptive gain modulation in a population without recurrence

We first consider the steady-state response of $N$ neurons,
${\mathbf{r}}_{\text{f}}\in\mathbb{R}^{N}$, receiving sensory stimulus inputs
${\mathbf{s}}\in\mathbb{R}^{M}$, with feedforward drive, ${\bf
f(s)}=[f_{1}({\mathbf{s}}),f_{2}({\mathbf{s}}),\dots,f_{N}({\mathbf{s}})]^{\top}$,
which are each multiplicatively scaled by gains,
${\mathbf{g}}=[g_{1},g_{2},\dots,g_{N}]^{\top}$:

${\bf
r_{\text{f}}}({\mathbf{s}},{\mathbf{g}})={\mathbf{g}}\circ{\mathbf{f}}({\mathbf{s}}).$
(1)

The gain vector ${\mathbf{g}}$ has the effect of adjusting the amplitudes of
responses ${\bf f(s)}$, and therefore the dynamic range of each neuron. As we
demonstrate in Section 6, these simple multiplicative gain scalings are
incapable of shifting the peaks of tuning curves, as seen in physiological
data (Movshon and Lennie, , 1979; Muller et al., , 1999; Saul and Cynader, ,
1989). Previous approaches modeling neural population adaptation in cortex
modify the structure of ${\bf f(s)}$ in response to changes in input
statistics (e.g. Wainwright et al., , 2001; Westrick et al., , 2016). Here, we
propose a fundamentally different approach, requiring no changes in synaptic
weighting between neurons.

###  3.3 Gain modulation in a recurrent neural population

We show that by incorporating single neuron gain modulation into a recurrent
network, adaptive effects in each neuron propagate laterally to affect other
cells in the population. Consider a model of $N$ recurrently connected neurons
with fixed feedforward and recurrent weights (Fig. 1A), presumed to have been
learned over timescales much longer than the adaptive timescales examined in
this study. We assume that the population of neural responses
${\mathbf{r}}\in\mathbb{R}^{N}$, driven by input stimuli
${\mathbf{s}}\in\mathbb{R}^{M}$ presented with probability $p({\mathbf{s}})$,
are governed by linear dynamics:

$\frac{d{\mathbf{r}}({\mathbf{s}},{\mathbf{g}})}{dt}=-{\mathbf{r}}+{\mathbf{g}}\circ{\mathbf{f}}({\mathbf{s}})+{\mathbf{W}}{\mathbf{r}},$
(2)

where ${\mathbf{W}}\in\mathbb{R}^{N\times N}$ is a matrix of recurrent
synaptic connection weights; and neuronal gains,
${\mathbf{g}}\in\mathbb{R}^{N}$, are adaptively optimized to a given
$p({\mathbf{s}})$. Both the feedforward functions $f_{i}({\mathbf{s}})$ and
recurrent weights ${\mathbf{W}}$ are assumed to be fixed despite varying
stimulus contexts (i.e. non-adaptive). For notational convenience, we omit
explicit time-dependence of the responses and stimuli (i.e.
${\mathbf{r}}({\mathbf{s}},{\mathbf{g}},t),{\mathbf{s}}(t)$).

Empirical studies typically consider neural activity at steady-state before
and after adapting to changes in stimulus statistics (Clifford et al., ,
2007). We therefore analyze the responses of our network at steady-state,
${\bf r_{\ast}(s,g)}$, to facilitate comparison with data. The network
dynamics of Equation 2 are linear in ${\mathbf{r}}$, and computing its steady-
state is analytically tractable. Setting Eq. 2 to zero and isolating
${\mathbf{r}}$ (with the mild assumptions on invertibility; see Appendix A),
yields the steady-state solution,

${{\mathbf{r}}_{\ast}}({\mathbf{s}},{\mathbf{g}})=\left[{\mathbf{I}}-{\mathbf{W}}\right]^{-1}\left({\mathbf{g}}\circ{\mathbf{f}}({\mathbf{s}})\right).$
(3)

We can interpret these equilibrium responses as a modification of the gain-
modulated feedforward drive, ${\mathbf{g}}\circ{\mathbf{f}}({\mathbf{s}})$,
which is propagated to other cells in the network via recurrent interactions,
$\left[{\mathbf{I}}-{\mathbf{W}}\right]^{-1}$. When ${\mathbf{W}}$ is the
zeros matrix (i.e. no recurrence), Equation 3 reduces to Equation 1, and
adjusting neuronal gains simply rescales the feedforward responses without
affecting the shape of response curves. The presence of the recurrent weight
matrix ${\mathbf{W}}$ allows changes in neuronal gains to alter the effective
tuning of other neurons in the network without changes to any synaptic
weights.

###  3.4 Structure of recurrent connectivity matrix ${\mathbf{W}}$

Importantly, in our recurrent network, there are no explicit excitatory and
inhibitory neurons – the recurrent activity term (last term in Eq. 2)
represents the net lateral input to a neuron (i.e. the combination of both
excitatory and inhibitory inputs). In addition, model simulations in this
study use a ${\mathbf{W}}$ that is translation invariant (i.e. convolutional)
in preferred orientation space, with strong net recurrent excitation near the
preferred orientation of the cell, and relatively weak net excitation far
away. This structure is motivated by functional and anatomical measurements in
V1, indicating that orientation-tuned cells receive excitatory and inhibitory
presynaptic inputs from cells tuned to every orientation, with
disproportionate excitatory bias from similarly-tuned neurons (Lee et al., ,
2016; Rossi et al., , 2020; Rubin et al., , 2015). We elaborate on specific
choices of ${\mathbf{W}}$ in Appendix A.

##  4 A Novel Objective for Adaptive Efficient Coding via Gain Modulation

Theories of efficient coding postulate that sensory neurons optimally encode
the statistics of the natural environment (Barlow, , 1961; Laughlin, , 1981),
subject to constraints on finite metabolic resources (e.g. energy expenditure
from firing spikes; Ganguli and Simoncelli, , 2014; Olshausen and Field, ,
1996). However, sensory input statistics vary with context, and the means by
which a neural population might confer an adaptive and dynamic efficient code
remains an open question (Gutierrez and Denève, , 2019; Młynarski and
Hermundstad, , 2021; Barlow and Foldiak, , 1989; Duong et al., , 2023). How
should our network (Equation 3) adaptively modulate its gains, ${\mathbf{g}}$,
according to the statistics of a novel stimulus ensemble? We assume an initial
stimulus ensemble, with probability density $p_{0}({\mathbf{s}})$ (Fig. 1B),
with a corresponding set of optimal gains, ${\mathbf{g}}_{0}$, toward which
adaptive gains are homeostatically driven; and an optimal linear decoder,
${\mathbf{D}}\in\mathbb{R}^{N\times M}$. ${\mathbf{D}}$ is fixed and set to
the pseudoinverse of ${\mathbf{r}}_{\ast}({\mathbf{g}},{\mathbf{s}})$ under
the initial stimulus ensemble (see Appendix C).

Given a novel stimulus ensemble with probability density $p({\mathbf{s}})$, we
propose an adaptive efficient coding objective that neurons minimize by
adjusting their gains,

$\mathcal{L}({\mathbf{g}},p({\mathbf{s}}))=\mathbb{E}_{{\mathbf{s}}\sim
p({\mathbf{s}})}\left\\{\|{\mathbf{s}}-{\mathbf{D}}^{\top}{\mathbf{r}}_{\ast}({\mathbf{s}},{\mathbf{g}})\|_{2}^{2}+\alpha\|{\mathbf{r}}_{\ast}({\mathbf{s}},{\mathbf{g}})\parallel_{2}^{2}\right\\}+\gamma\parallel{\mathbf{g}}-{\mathbf{g}}_{0}\parallel^{2}_{2},$
(4)

where $\alpha$ and $\gamma$ are scalar hyperparameters. Intuitively, as the
stimulus ensemble changes $p_{0}({\mathbf{s}})\rightarrow p({\mathbf{s}})$,
the gains ${\mathbf{g}}$ are adaptively adjusted to maximize the fidelity of
the representation (first term), while minimizing overall activity in the
network (second term), and minimally deviating from the initial gain state
(third term). The gain homeostasis term serves to prevent catastrophic
forgetting in the network under different stimulus contexts (Kirkpatrick et
al., , 2017): minimizing the gains’ deviation from their optimal state under
$p_{0}({\mathbf{s}})$ allows the system to stably maintain reasonable
performance on previously presented data and prevents the system from
radically reorganizing itself on a fast time scale. In Appendix B, we show
that adapting to $p({\mathbf{s}})$ with gain homeostasis allows the network to
maintain improved stimulus representation error under the
$p_{0}({\mathbf{s}})$ ensemble relative to a network optimized without gain
homeostasis. We also perform ablations to show that the three terms in the
objective are jointly necessary to produce the adaptation effects observed in
data.

###  4.1 Objective optimization

The objective given in Equation 4 is bi-convex in ${\mathbf{g}}$ and
${\mathbf{D}}$, and we can analytically solve for either variable
independently or in alternation (i.e., coordinate descent via alternating
least squares). See Appendix C for the complete derivation. We initialize the
network under the uniform stimulus density $p_{0}({\mathbf{s}})$ to obtain a
homeostatic gain target, ${\mathbf{g}}_{0}$, and a fixed decoder,
${\mathbf{D}}$.

##  5 V1 Neural Population Adaptation Data Reanalysis

In the following section, we compare our simulated adaptation model responses
to reanalyzed neural population recordings from cat primary visual cortex
(data obtained with permission from Benucci et al., , 2013). Here, we provide
an overview of our data analysis procedure which we also apply to our
simulated model responses. Some of our analysis plots are new and are not in
the original study111Additionally, our plots are derived from steady-state
fitted response curves, whereas the original publication used temporal
information.. For details on the recordings and preprocessing, we refer the
reader to the original paper.

In the experiment, oriented stimuli were briefly presented randomly in rapid
succession, with presentation probability determined by one of two contextual
distributions: a uniform distribution $p_{0}({\mathbf{s}})$, or a biased
distribution, in which one orientation was presented significantly more
frequently than the others, $p({\mathbf{s}})$ (Figure 1B, top row). Figure 1B
(middle row) shows responses for $N=13$ units, aggregated over 11 recording
sessions. For $N$ units and $K$ distinct stimuli, the authors fit orientation
tuning curves to neural responses to produce matrices of orientation tuning
curves, ${\mathbf{R}}\in\mathbb{R}^{N\times K}$ for each of the uniform and
biased stimulus ensembles.

We normalize each unit’s response curves under both contexts according to its
minimum and maximum response during the $p_{0}({\mathbf{s}})$ context, such
that all responses lie in the interval [0, 1] for $p_{0}({\mathbf{s}})$. That
is, zero is the minimum stimulus-evoked response under the uniform ensemble,
and one is the maximum. For responses to the biased ensemble,
$p({\mathbf{s}})$, a minimum response less than 0 indicates that the evoked
response after adaptation has decreased relative to the uniform ensemble;
similarly, a maximum response less than 1 indicates the response maximum after
adaptation has decreased relative to that of the uniform ensemble (Figure 1B).

We compute response means, $\bm{\mu}\in\mathbb{R}^{N}$, and signal (as opposed
to noise) covariance matrices, ${\bm{\Sigma}}\in\mathbb{S}_{+}^{N}$,

$\bm{\mu}=\mathbb{E}[{\mathbf{R}}],\qquad{\bm{\Sigma}}=\mathbb{E}[{\mathbf{R}}{\mathbf{R}}^{\top}]-\bm{\mu}\bm{\mu}^{\top},$
(5)

where the expectation is over $p_{0}({\mathbf{s}})$ or $p({\mathbf{s}})$. To
facilitate comparisons between response covariances under the uniform and
biased stimulus ensembles, we scale response covariance matrices by the
variances of the neurons under the uniform stimulus probability condition,
${\bm{\sigma}}_{0}^{2}\in\mathbb{R}_{+}^{N}$,

$\hat{\bm{\Sigma}}=\operatorname{diag}\left({\bm{\sigma}}_{0}\right)^{-1}{\bm{\Sigma}}\operatorname{diag}\left({\bm{\sigma}}_{0}\right)^{-1}.$
(6)

##  6 Numerical Simulations and Comparisons to Neural Data

We compare numerical simulations of our normative adaptation model with
reanalyzed cat V1 population recording data (Benucci et al., , 2013).

###  6.1 Model and simulation parameters

For all simulation results and figures in this study, we consider a network
comprised of $N=255$ recurrently connected neurons, with $K=M=511$ orientation
stimuli as inputs. The neuronal gains, ${\mathbf{g}}$, adapt to changes in
stimulus ensemble statistics ($p_{0}({\mathbf{s}})\rightarrow
p({\mathbf{s}})$), while the feedforward synaptic weights,
${\mathbf{f}}({\mathbf{s}})$, and recurrent synaptic weights, ${\mathbf{W}}$,
remain fixed. We set the homeostatic target gains, ${\mathbf{g}}_{0}$, to the
optimal values of ${\mathbf{g}}$ under the uniform probability stimulus
ensemble, $p_{0}({\mathbf{s}})$. Feedforward orientation-tuning functions,
${\mathbf{f}}({\mathbf{s}})$, are evenly distributed in the stimulus domain,
and are broadly-tuned Gaussians with full-width half-max (FWHM) of 30$\circ$
(Benucci et al., , 2013). The recurrent weight matrix, ${\mathbf{W}}$, is a
Gaussian with 10$\circ$ FWHM, summed with a weaker, broad, untuned excitatory
component (see Appendix A).

To determine appropriate values of $\alpha$ and $\gamma$ in the objective
(Equation 4), we performed a grid-search hyperparameter sweep, minimizing the
deviation between model and experimentally-measured tuning curves for the
biased stimulus ensemble. The figures here all use model responses from a
simulation using $\alpha=$ 1E-3, $\gamma=$ 1E-2. We find that qualitative
effects are insensitive to small changes in these parameters. The key finding
from this parameter sweep is that the gain homeostasis penalty weight must be
sufficiently greater than the activity penalty weight (i.e. $\gamma>\alpha$).
After initializing the network gains to the statistics of
$p_{0}({\mathbf{s}})$, we adapt the gains to $p({\mathbf{s}})$ by optimizing
Equation 4, then compare our model predicted responses to cat V1 population
recordings Figure 1B.

Figure 2: Adaptive response equalization. Each dot is the average response of
a neuron. A)  Response averages under the uniform stimulus ensemble condition.
B)  Without adaptation, response averages under the biased stimulus ensemble
show substantial deviation from equalization (which corresponds to the dashed
black line). C)  After adaptation, response averages to the biased ensemble
are nearly equalized. Shaded regions are SEM.

###  6.2 Adaptive gain modulation predicts response equalization

Population response equalization is an adaptive mechanism first proposed in
the psychophysics literature (Anstis et al., , 1998). The authors argued that
adaptation should serve as a “graphic equalizer” in response to alterations in
environmental statistics. Others have have described equalization as a
mechanism that centers a population response by subtracting the responses to
the prevailing stimulus ensemble (Clifford et al., , 2000), to rescale
responses such that the average of a measured signal remains constant (Ullman
and Schechtman, , 1982). Figure 2 shows how our model recapitulates mean
firing rates across all stimuli under the uniform and biased ensembles without
adaptation, along with adaptive population response equalization under the
biased ensemble. Figure 2A shows how the average response of each pre-adapted
neuron under the uniform ensemble is equal. By contrast, Figure 2B
demonstrates that our model predicts how the pre-adaptation tuning curves
under the biased stimulus ensemble would produce a substantial deviation from
equalization. Finally, adaptively optimizing neuron gains via Equation 4
predicts the compensatory response equalization under the biased stimulus
ensemble observed in data (panel C).

Figure 3: Recurrent network model with adaptive gain modulation (red) captures
the full set of post-adaptation first-order response changes observed in data
(black points), while a network without recurrence (blue) does not. A) Ratios
of after/before adaptation response maxima. B) Ratios of response amplitudes
($\|\max-\min\|$ response) after/before adaptation. C) Changes in average
minimum evoked response. D) Shifts in tuning away from the adapter. Shaded
regions are SEM. Dashed lines indicate predictions for a non-adaptive model.

###  6.3 Adaptive gain modulation predicts nuanced changes in first-order
statistics of responses

Figure 3 summarizes adaptive changes in neural responses by comparing tuning
curve responses under the biased stimulus ensemble compared to responses under
the uniform stimulus ensemble (i.e. right vs. left columns of Fig. 1B). Our
gain-modulating efficient coding model can capture this entire array of
observed adaptation effects.

#### Changes in response maxima, amplitudes, and minima.

Stimulus-dependent response reductions are a ubiquitous finding in adaptation
experiments (Weber et al., , 2019). Figure 3A,B show changes in response
maxima, and response amplitudes (peak-to-trough height) following adaptation
to the biased stimulus ensemble. Ratios less than 1 indicate a reduction in
maxima or amplitudes following adaptation. Under the biased stimulus ensemble,
the model optimizes its gains according to the objective (Eq. 4) to
preferentially reduce activity near the over-presented adapter stimulus. By
optimizing gains to the adaptive efficient coding objective, our linear model
undershoots the magnitude of change around the adapter (Fig. 3A,B red curve
near 0$\circ$), but captures the overall effect of adaptive amplitude and
maxima reduction.

Figure 3C shows that adaptation induces a tuning-dependent, global reduction
in minimum stimulus-evoked response across the population. These minima
typically occur at the anti-preferred orientation for each neuron (Fig. 1B).
Previous work has attributed this to an untuned reduction in thalamic inputs,
or a drop in base firing (Benucci et al., , 2013; Westrick et al., , 2016).
Our model proposes a different mechanism: gain reductions in neurons tuned for
the adapter propagate laterally through the network, and result in
commensurate reductions in the broad/untuned recurrent excitation to other
neurons in the population. This ultimately leads to a reduction in minimum
evoked response across the entire population (Fig. 3C); importantly, the model
also captures the qualitative shape of the change. Our mechanistic prediction
that this effect arises due to recurrent contributions is in concordance with
the broad literature on recurrent cortical circuitry, its role in
amplification (Reinhold et al., , 2015), and in sensory adaptation
(Hershenhoren et al., , 2014; King et al., , 2016).

#### Shifts in tuning preference.

Tuning curve shifts following adaptation have been reported across many visual
and auditory adaptation studies (Clifford et al., , 2007; Whitmire and
Stanley, , 2016, for reviews). Figure 3D quantifies changes in neuron
preferred orientation (i.e. the orientation at which response maximum occurs)
after adapting to the biased stimulus ensemble. The sinusoidal shape of the
curve indicates that adapted tuning curves are repelled from the over-
presented adapter stimulus. This rearrangement of tuning curve density is
consistent with efficient coding studies that argue that a sensory neural
population should optimally allocate its finite resources toward encoding
information about the current stimulus ensemble (Gutnisky and Dragoi, , 2008;
Ganguli and Simoncelli, , 2014). Here, we show that these effects can
mechanistically be explained by optimizing neuronal gains to maintain a high
fidelity representation of the stimulus under the biased ensemble.

#### Objective and network ablations.

In Appendix B, we assess the importance of each term of the adaptation
objective (Eq. 4) by ablating them from the objective and re-simulating the
network adapting to the biased stimulus ensemble. We show that each of the
three terms is jointly necessary to capture the adaptation effects shown here.
In terms of network architectural ablations, the blue curves in Figure 3
demonstrate how removing recurrence (i.e. ${\mathbf{W}}=0$; Equation 1)
impacts adaptive changes in neural responses. While this single stage
feedforward model can reproduce reductions in response maxima Fig. 3A, it is
incapable of producing the appropriate change in response amplitudes (Fig.
3B), and completely fails at producing adaptive reductions in minimum
response, or shifts in tuning preference (Fig. 3C,D). Intuitively, this is
because the gains in this reduced model serve to set the amplitude of the
output, and cannot alter the qualitative shape of the tuning curve without
propagating through the recurrent circuitry. The structure of ${\mathbf{W}}$
used in our model is informed by functional and anatomical studies in cortical
circuits (Lee et al., , 2016; Rossi et al., , 2020), comprising strong net
excitation from similarly-tuned neurons and untuned weak net excitation from
dissimilarly-tuned neurons. In Appendix A, we study the impact of
${\mathbf{W}}$’s structure on model adapted responses. The structure of
${\mathbf{W}}$ can be quite flexible while still producing the effects shown
here, so long as recurrent input includes weak net excitation from
dissimilarly-tuned neurons.

###  6.4 Adaptive gain modulation predicts homeostasis in second-order
statistics of responses

The principle of redundancy reduction is core to the efficient coding
hypothesis (Barlow, , 1961), and evidence supporting adaptive redundancy
reduction has been reported across multiple brain regions and modalities
(Atick and Redlich, , 1992; Wanner and Friedrich, , 2020; Muller et al., ,
1999). In the task modeled in our study, over-presenting the adapter stimulus
can be viewed as increasing redundancy in the stimulus ensemble (Figure 1B,
top). This manifests as a “hot spot” in the center of $\hat{\Sigma}$ if the
neural responses were to remain unadapted to $p({\mathbf{s}})$ (Fig. 4A,
middle column). However, when the model adapts its gains according to the
objective (Eq. 4), the covariance near the adapter stimulus is reduced, and
the predicted signal covariance is well matched to data (Fig. 4A, right
column, Fig. 4B).

A signal covariance matrix devoid of redundancy would be one that is
statistically white (i.e. the identity matrix). However, under both the
uniform and biased stimulus ensemble conditions (Figure 4A, top left and
right), we note that the experimentally observed signal covariance matrix is
not statistically white222In their study, Benucci et al.,  (2013, Fig. 3)
replaced negative entries of $\hat{\Sigma}$ with zeros.. Thus, previous
normative approaches to population adaptation that explicitly whiten neural
responses may not be suitable models for this data (e.g. Pehlevan and
Chklovskii, , 2015; Młynarski and Hermundstad, , 2021). By contrast, our
adaptation objective, which emphasizes stimulus signal fidelity subject to
metabolic and homeostatic constraints predicts an adapted signal covariance
matrix whose deviations from the identity matrix are similar to those observed
in data. Notably, this effect naturally emerges from our model without
additional parameter-tuning.

Figure 4: Population response redundancy reduction and signal covariance
homeostasis. A) Scaled response covariance matrices, $\hat{\bm{\Sigma}}$ (Eq.
6), for V1 data (top row) and model simulations (bottom row), for unadapted
tuning curves and uniform stimulus ensemble (left column), unadapted tuning
curves and biased stimulus ensemble (middle column), and adapted tuning curves
and biased stimulus ensemble (right column). B) Three example horizontal
slices of the data (dashed) and model (solid) $\hat{\bm{\Sigma}}$ from A, at
0, -45, and -90 degrees orientation (colors).

##  7 Discussion

#### Study limitations.

The network considered here is a rate model whose tractable linear dynamics
allow us to examine adaptation responses at steady-state. Response dynamics
during adaptation are rich (Patterson et al., , 2013; Quiroga et al., , 2016;
Dragoi et al., , 2000), and are relatively understudied. Developing our model
and objective into a biologically plausible online network with explicit
excitatory and inhibitory neurons, while adapting gains according to only
local signals (Duong et al., , 2023; Gutierrez and Denève, , 2019) is an
interesting direction worth pursuing. Furthermore, because we model trial-
averaged experimental data in this study, our model does not account for
stochasticity in neural responses. Thus, our model cannot explain adaptive
changes in trial-to-trial variability (Gutnisky and Dragoi, , 2008). Finally,
there exist adaptive changes to simultaneously-presented stimuli, usually
explained via divisive normalization (Yiltiz et al., , 2020; Aschner et al., ,
2018; Solomon and Kohn, , 2014), which is not included in our model (see
Appendix D). One possible way to bridge this gap would be to combine our
normative approach with recently-proposed recurrent models of normalization
(Heeger and Mackey, , 2018; Heeger and Zemlianova, , 2020).

#### Alternative network architectures.

There are alternative, equivalent formulations of our model that may give rise
to the same steady-state responses as Eq. 3, which we illustrate in Appendix
D. Firstly, our model is equivalent to a two-stage feedforward network with
gain modulation preceding the inputs of the second stage. Since orientation
tuning arises in V1, these two stages could be two different layers within V1;
the core mechanism of our framework can thus be related to studies describing
adaptive gain changes being inherited from one group of neurons to the next
(Kohn and Movshon, , 2003; Dhruv and Carandini, , 2014; Stocker and
Simoncelli, , 2009). Secondly, gain modulation in our model, which serves to
multiplicatively scale input drive, ${\mathbf{f}}({\mathbf{s}})$, can
equivalently be interpreted as multiplicatively attenuating the recurrent
drive of the network. In this sense, our model resembles that of Heeger and
Zemlianova,  (2020), in which divisive normalization is mediated by gating
recurrent amplification.

#### Experimental predictions.

We propose that rapid neural population adaptation in cortex can be mediated
by single neuron adaptive gain modulation. Validating this hypothesis would
require careful experimental measurements of neurons during adaptation. First,
our framework predicts that between-neuron synaptic connectivity (i.e.
${\mathbf{W}}$) remains stable through adaptation. Second, our normative
objective suggests that gain homeostasis plays a central role in population
adaptation (see Appendix B). Evidence for stimulus-dependent gain control such
as this can possibly be found by measuring neuron membrane conductance during
adaptation, mediated by changes in slow hyperpolarizing Ca2+\- and Na+-induced
K+ currents (Sanchez-Vives et al., , 2000). Lastly, while there has been
considerable progress in mapping the circuits involved in sensory adaptation
(Wanner and Friedrich, , 2020), determining the exact structure of functional
recurrent connectivity remains an open problem. Indeed, we show how different
(but not all) forms of ${\mathbf{W}}$ can give rise to the same qualitative
results shown here (Appendix A). Performing adaptation experiments with richer
sets of stimulus ensembles, $p({\mathbf{s}})$, can provide better constraints
for solving this functional inverse problem.

###  7.1 Conclusion

We demonstrate that adaptation effects observed in cortex – changes in
response maxima and minima, tuning curve repulsion, and stimulus-dependent
response decorrelation – can be explained as arising from the recurrent
propagation of single neuron gain adjustments aimed at coding efficiency. This
adaptation mechanism is general, and can be applied to modalities other than
vision. For example, studies of neural adaptation in auditory cortex have
shown that adaptive responses such tuning curve shifts cannot be explained by
feedforward mechanisms, and likely arise from adaptive changes to
intracortical recurrent interactions (Hershenhoren et al., , 2014; Lohse et
al., , 2020). Previous population adaptation models rely on changes in all-to-
all synaptic weights to explain these phenomena (e.g. Westrick et al., ,
2016), but our results suggest that single neuron gain modulations may provide
a more plausible mechanism which uses $\mathcal{O}(N)$ instead of
$\mathcal{O}(N^{2})$ adaptive parameters. Adaptation in cortex happens on the
order of hundreds of milliseconds, and is just as quickly reversible (Muller
et al., , 1999); a network whose synaptic weights were constantly remapping
would be undesirable due to a lack of stability, while a mechanism such as
adaptive single neuron gain modulation can be local, fast, and reversible
(Ferguson and Cardin, , 2020). Taken together, our study offers a simple
mechanistic explanation for observed adaptation effects at the level of a
neural population, and expands upon well-established concepts of adaptive
coding efficiency with single neuron gain control.

## Acknowledgments and Disclosure of Funding

We thank Matteo Carandini for providing us with V1 neural recording data. We
also thank Teddy Yerxa, Pierre-Étienne Fiquet, Stefano Martiniani, Shivang
Rawat, Gabrielle Gutierrez, Ann Hermundstad, and Wiktor Młynarski for their
feedback on earlier versions of this work.

## References

  * Abbott et al., (1997) Abbott, L. F., Varela, J. A., Sen, K., and Nelson, S. B. (1997).  Synaptic Depression and Cortical Gain Control.  Science, 275(5297):221–224.
  * Adrian and Zotterman, (1926) Adrian, E. D. and Zotterman, Y. (1926).  The impulses produced by sensory nerve-endings.  The Journal of Physiology, 61(2):151–171.
  * Anstis et al., (1998) Anstis, S., Verstraten, F. A., and Mather, G. (1998).  The motion aftereffect.  Trends in cognitive sciences, 2(3):111–117.
  * Aschner et al., (2018) Aschner, A., Solomon, S. G., Landy, M. S., Heeger, D. J., and Kohn, A. (2018).  Temporal Contingencies Determine Whether Adaptation Strengthens or Weakens Normalization.  Journal of Neuroscience, 38(47):10129–10142.
  * Atick and Redlich, (1992) Atick, J. J. and Redlich, A. N. (1992).  What does the retina know about natural scenes?  Neural Computation, 4:196–210.
  * Attneave, (1954) Attneave, F. (1954).  Some informational aspects of visual perception.  Psychological Review, 61(3):183–193.
  * Ballé et al., (2020) Ballé, J., Chou, P. A., Minnen, D., Singh, S., Johnston, N., Agustsson, E., Hwang, S. J., and Toderici, G. (2020).  Nonlinear transform coding.  IEEE Journal of Selected Topics in Signal Processing, 15(2):339–353.
  * Barlow, (1961) Barlow, H. B. (1961).  Possible Principles Underlying the Transformations of Sensory Messages.  In Sensory Communication, pages 216–234. The MIT Press.
  * Barlow and Foldiak, (1989) Barlow, H. B. and Foldiak, P. (1989).  Adaptation and decorrelation in the cortex.  In The Computing Neuron, pages 54–72. Addison-Wesley.
  * Benucci et al., (2013) Benucci, A., Saleem, A. B., and Carandini, M. (2013).  Adaptation maintains population homeostasis in primary visual cortex.  Nature Neuroscience, 16(6):724–729.
  * Brenner et al., (2000) Brenner, N., Bialek, W., and de Ruyter van Steveninck, R. (2000).  Adaptive Rescaling Maximizes Information Transmission.  Neuron, 26(3):695–702.
  * Carandini and Ringach, (1997) Carandini, M. and Ringach, D. L. (1997).  Predictions of a recurrent model of orientation selectivity.  Vision Research, 37(21):3061–3071.
  * Clifford et al., (2000) Clifford, C. W., Wenderoth, P., and Spehar, B. (2000).  A functional angle on some after-effects in cortical vision.  Proceedings of the Royal Society of London. Series B: Biological Sciences, 267(1454):1705–1710.
  * Clifford et al., (2007) Clifford, C. W. G., Webster, M. A., Stanley, G. B., Stocker, A. A., Kohn, A., Sharpee, T. O., and Schwartz, O. (2007).  Visual adaptation: Neural, psychological and computational aspects.  Vision Research, 47(25):3125–3131.
  * Dayan and Abbott, (2005) Dayan, P. and Abbott, L. F. (2005).  Theoretical neuroscience: computational and mathematical modeling of neural systems.  MIT press.
  * Dhruv and Carandini, (2014) Dhruv, N. and Carandini, M. (2014).  Cascaded Effects of Spatial Adaptation in the Early Visual System.  Neuron, 81(3):529–535.
  * Douglas and Martin, (2007) Douglas, R. J. and Martin, K. A. (2007).  Recurrent neuronal circuits in the neocortex.  Current Biology, 17(13):R496–R500.
  * Dragoi et al., (2000) Dragoi, V., Sharma, J., and Sur, M. (2000).  Adaptation-induced plasticity of orientation tuning in adult visual cortex.  Neuron, 28(1):287–298.
  * Duong et al., (2023) Duong, L. R., Lipshutz, D., Heeger, D. J., Chklovskii, D. B., and Simoncelli, E. P. (2023).  Statistical whitening of neural populations with gain-modulating interneurons.  International Conference on Machine Learning.
  * Fairhall et al., (2001) Fairhall, A. L., Lewen, G. D., and Bialek, W. (2001).  Efficiency and ambiguity in an adaptive neural code.  Nature, 412:787–792.
  * Ferguson and Cardin, (2020) Ferguson, K. A. and Cardin, J. A. (2020).  Mechanisms underlying gain modulation in the cortex.  Nature Reviews Neuroscience, 21(2):80–92.
  * Ganguli and Simoncelli, (2014) Ganguli, D. and Simoncelli, E. P. (2014).  Efficient Sensory Encoding and Bayesian Inference with Heterogeneous Neural Populations.  Neural Computation, 26(10):2103–2134.  Publisher: MIT Press.
  * Gutierrez and Denève, (2019) Gutierrez, G. J. and Denève, S. (2019).  Population adaptation in efficient balanced networks.  eLife, 8.
  * Gutnisky and Dragoi, (2008) Gutnisky, D. A. and Dragoi, V. (2008).  Adaptive coding of visual information in neural populations.  Nature, 452(7184):220–224.
  * Heeger and Mackey, (2018) Heeger, D. J. and Mackey, W. E. (2018).  ORGaNICs: A Theory of Working Memory in Brains and Machines.  arXiv:1803.06288 [cs, q-bio].
  * Heeger and Zemlianova, (2020) Heeger, D. J. and Zemlianova, K. O. (2020).  A recurrent circuit implements normalization, simulating the dynamics of V1 activity.  Proceedings of the National Academy of Sciences, 117(36):22494–22505.
  * Hershenhoren et al., (2014) Hershenhoren, I., Taaseh, N., Antunes, F. M., and Nelken, I. (2014).  Intracellular Correlates of Stimulus-Specific Adaptation.  Journal of Neuroscience, 34(9):3303–3319.
  * Hu et al., (2022) Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022).  Lora: Low-rank adaptation of large language models.  International Conference on Learning Representations.
  * King et al., (2016) King, J. L., Lowe, M. P., Stover, K. R., Wong, A. A., and Crowder, N. A. (2016).  Adaptive Processes in Thalamus and Cortex Revealed by Silencing of Primary Visual Cortex during Contrast Adaptation.  Current Biology, 26(10):1295–1300.
  * Kirkpatrick et al., (2017) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017).  Overcoming catastrophic forgetting in neural networks.  Proceedings of the national academy of sciences, 114(13):3521–3526.
  * Ko et al., (2011) Ko, H., Hofer, S. B., Pichler, B., Buchanan, K. A., Sjöström, P. J., and Mrsic-Flogel, T. D. (2011).  Functional specificity of local synaptic connections in neocortical networks.  Nature, 473(7345):87–91.
  * Kohn, (2007) Kohn, A. (2007).  Visual Adaptation: Physiology, Mechanisms, and Functional Benefits.  Journal of Neurophysiology, 97(5):3155–3164.
  * Kohn and Movshon, (2003) Kohn, A. and Movshon, J. A. (2003).  Neuronal Adaptation to Visual Motion in Area MT of the Macaque.  Neuron, 39(4):681–691.
  * Laughlin, (1981) Laughlin, S. (1981).  A Simple Coding Procedure Enhances a Neuron’s Information Capacity.  Zeitschrift fur Naturforschung. C, Journal of biosciences, pages 910–2.
  * Lee et al., (2016) Lee, W.-C. A., Bonin, V., Reed, M., Graham, B. J., Hood, G., Glattfelder, K., and Reid, R. C. (2016).  Anatomy and function of an excitatory network in the visual cortex.  Nature, 532(7599):370–374.
  * Lien and Scanziani, (2013) Lien, A. D. and Scanziani, M. (2013).  Tuned thalamic excitation is amplified by visual cortical circuits.  Nature neuroscience, 16(9):1315–1323.
  * Lipshutz et al., (2023) Lipshutz, D., Pehlevan, C., and Chklovskii, D. B. (2023).  Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation.  International Conference on Learning Representations.
  * Lohse et al., (2020) Lohse, M., Bajo, V. M., King, A. J., and Willmore, B. D. (2020).  Neural circuits underlying auditory contrast gain control and their perceptual implications.  Nature Communications, 11(1):324.
  * Mohan et al., (2021) Mohan, S., Vincent, J. L., Manzorro, R., Crozier, P., Fernandez-Granda, C., and Simoncelli, E. (2021).  Adaptive denoising via gaintuning.  Advances in neural information processing systems, 34:23727–23740.
  * Movshon and Lennie, (1979) Movshon, J. A. and Lennie, P. (1979).  Pattern-selective adaptation in visual cortical neurones.  Nature, 278(5707):850–852.
  * Muller et al., (1999) Muller, J. R., Metha, A. B., Krauskopf, J., and Lennie, P. (1999).  Rapid adaptation in visual cortex to the structure of images.  Science, 285(5432):1405–1408.
  * Młynarski and Hermundstad, (2021) Młynarski, W. F. and Hermundstad, A. M. (2021).  Efficient and adaptive sensory codes.  Nature Neuroscience, 24(7):998–1009.
  * Nagel and Doupe, (2006) Nagel, K. I. and Doupe, A. J. (2006).  Temporal Processing and Adaptation in the Songbird Auditory Forebrain.  Neuron, 51(6):845–859.
  * Olshausen and Field, (1996) Olshausen, B. A. and Field, D. J. (1996).  Emergence of simple-cell receptive field properties by learning a sparse.  Nature, 381(June):607–609.
  * Patterson et al., (2013) Patterson, C. A., Wissig, S. C., and Kohn, A. (2013).  Distinct Effects of Brief and Prolonged Adaptation on Orientation Tuning in Primary Visual Cortex.  Journal of Neuroscience, 33(2):532–543.
  * Pehlevan and Chklovskii, (2015) Pehlevan, C. and Chklovskii, D. B. (2015).  A normative theory of adaptive dimensionality reduction in neural networks.  Advances in Neural Information Processing Systems, 28.
  * Quiroga et al., (2016) Quiroga, M., Morris, A., and Krekelberg, B. (2016).  Adaptation without Plasticity.  Cell Reports, 17(1):58–68.
  * Rast and Drugowitsch, (2020) Rast, L. and Drugowitsch, J. (2020).  Adaptation Properties Allow Identification of Optimized Neural Codes.  In Advances in Neural Information Processing Systems.
  * Reinhold et al., (2015) Reinhold, K., Lien, A. D., and Scanziani, M. (2015).  Distinct recurrent versus afferent dynamics in cortical visual processing.  Nature Neuroscience, 18(12):1789–1797.
  * Rossi et al., (2020) Rossi, L. F., Harris, K. D., and Carandini, M. (2020).  Spatial connectivity matches direction selectivity in visual cortex.  Nature, 588(7839):648–652.
  * Rubin et al., (2015) Rubin, D., Van Hooser, S., and Miller, K. (2015).  The Stabilized Supralinear Network: A Unifying Circuit Motif Underlying Multi-Input Integration in Sensory Cortex.  Neuron, 85(2):402–417.
  * Sanchez-Vives et al., (2000) Sanchez-Vives, M. V., Nowak, L. G., and McCormick, D. A. (2000).  Membrane mechanisms underlying contrast adaptation in cat area 17in vivo.  Journal of Neuroscience, 20(11):4267–4285.
  * Saul and Cynader, (1989) Saul, A. B. and Cynader, M. (1989).  Adaptation in single units in visual cortex: the tuning of aftereffects in the spatial domain.  Visual neuroscience, 2(6):593–607.
  * Shen et al., (2015) Shen, L., Zhao, L., and Hong, B. (2015).  Frequency-specific adaptation and its underlying circuit model in the auditory midbrain.  Frontiers in Neural Circuits, 9.
  * Solomon and Kohn, (2014) Solomon, S. G. and Kohn, A. (2014).  Moving sensory adaptation beyond suppressive effects in single neurons.  Current Biology, 24(20):R1012–R1022.
  * Stocker and Simoncelli, (2009) Stocker, A. A. and Simoncelli, E. P. (2009).  Visual motion aftereffects arise from a cascade of two isomorphic adaptation mechanisms.  Journal of Vision, 9(9):9–9.
  * Teich and Qian, (2010) Teich, A. F. and Qian, N. (2010).  V1 orientation plasticity is explained by broadly tuned feedforward inputs and intracortical sharpening.  Visual neuroscience, 27(1-2):57–73.
  * Ullman and Schechtman, (1982) Ullman, S. and Schechtman, G. (1982).  Adaptation and gain normalization.  Proceedings of the Royal Society of London. Series B. Biological Sciences, 216(1204):299–313.
  * Wainwright et al., (2001) Wainwright, M. J., Schwartz, O., and Simoncelli, E. P. (2001).  Natural Image Statistics and Divisive Normalization: Modeling Nonlinearities and Adaptation in Cortical Neurons.  In Statistical Theories of the Brain, page 22. MIT Press.
  * Wanner and Friedrich, (2020) Wanner, A. A. and Friedrich, R. W. (2020).  Whitening of odor representations by the wiring diagram of the olfactory bulb.  Nature Neuroscience.
  * Weber et al., (2019) Weber, A. I., Krishnamurthy, K., and Fairhall, A. L. (2019).  Coding Principles in Adaptation.  Annual Review of Vision Science, 5:427–449.
  * Westerberg et al., (2019) Westerberg, J. A., Cox, M. A., Dougherty, K., and Maier, A. (2019).  V1 microcircuit dynamics: altered signal propagation suggests intracortical origins for adaptation in response to visual repetition.  Journal of Neurophysiology, 121(5):1938–1952.
  * Westrick et al., (2016) Westrick, Z. M., Heeger, D. J., and Landy, M. S. (2016).  Pattern Adaptation and Normalization Reweighting.  Journal of Neuroscience, 36(38):9805–9816.
  * Whitmire and Stanley, (2016) Whitmire, C. and Stanley, G. (2016).  Rapid Sensory Adaptation Redux: A Circuit Perspective.  Neuron, 92(2):298–315.
  * Yaron et al., (2012) Yaron, A., Hershenhoren, I., and Nelken, I. (2012).  Sensitivity to Complex Statistical Regularities in Rat Auditory Cortex.  Neuron, 76(3):603–615.
  * Yiltiz et al., (2020) Yiltiz, H., Heeger, D. J., and Landy, M. S. (2020).  Contingent adaptation in masking and surround suppression.  Vision Research, 166:72–80.

##  Appendix A Details on model recurrent connectivity matrix

###  A.1 Initializing the recurrent connectivity matrix ${\mathbf{W}}$

We restrict ${{\mathbf{W}}}\in\mathbb{R}^{N\times N}$ to the space of
circularly symmetric (i.e. convolutional) positive definite matrices. In our
model, the recurrent weight kernel forming the convolutional matrix is (net)
positive everywhere, with higher probability between similarly tuned
excitatory neurons than between dissimilarly tuned neurons (Ko et al., , 2011;
Lee et al., , 2016). The kernel we use is a Gaussian (10$\circ$ FWHM) summed
with a uniform density. To prevent recurrence from diverging (Eq. 3), the
operator norm of ${\mathbf{W}}$ (i.e. the max eigenvalue) must be less than 1.
We fixed $\|{\mathbf{W}}\|_{\text{Op}}=0.8$ for all recurrent weight matrices
in this study.

###  A.2 Recurrent synaptic connectivity influences adaptation effects

The structure of the recurrent weight matrix ${\mathbf{W}}$ greatly impacts
the adaptive changes in neural responses. Specifically, we find that, with our
recurrent model and objective (Eq. 3 and Eq. 4), weak net excitatory inputs
from dissimilarly tuned neurons is needed to capture the observed effects in
data. This recurrent composition is line with broad, untuned excitatatory
signal amplification contributing to overall background activity in cortical
circuits (Reinhold et al., , 2015). For reference, we reproduce a subset of
the post-adaptation tuning curves from the main text here in Fig. A.1A.

Figure A.1B shows response curves from an example model using a convolutional
${\mathbf{W}}$ derived from the Mexican hat weight kernel, which is an
excitatory Gaussian (10$\circ$ FWHM) minus a wider Gaussian (60$\circ$ FWHM)
(Carandini and Ringach, , 1997; Teich and Qian, , 2010; Quiroga et al., ,
2016). We re-scaled the weight matrix to have operator norm of 0.8. The
responses in Fig. A.1B here are from a model minimizing $\ell_{2}$ error
between the data and model adapted responses after a hyper-parameter sweep
($\alpha=$ 3E-4, $\gamma=$ 2E-3). In contrast to the recurrent weight matrix
we use in the main text (described above), the Mexican hat kernel has
recurrent net excitation from neurons with similar tuning, and net inhibition
from neurons with dissimilar tuning. A model with this recurrent weight kernel
is unable to capture the effects of response maxima, minima, and amplitudes,
and produces tuning curve attraction rather than repulsion from the adapter
Fig. A.2A-D. Taken together, this suggests that broad, untuned weak recurrent
excitation is necessary for our model to capture the wide array of post-
adaptation effects found in this dataset.

Figure A.1:  Data (dashed) vs model (solid) with different forms of
${\mathbf{W}}$. Dashed lines (identical in left and right panels) are observed
post-adaptation response curves for a subset of the population. Left Model
from main text, using a convolutional ${\mathbf{W}}$ with recurrent net
excitation from similarly tuned neurons, and broad/untuned net excitation from
dissimilarly tuned neurons. Right Simulated post-adaptation responses from a
model with ${\mathbf{W}}$ comprising recurrent net excitation from similarly
tuned neurons, and net inhibition from dissimilarly tuned neurons.  Figure
A.2: All panels are the same as Fig. 3 in the main text, but blue is now a
model with ${\mathbf{W}}$ set to a convolutional matrix with a Mexican hat
kernel. Notably, this model cannot reproduce the adaptive maxima, minima, and
amplitude effects observed in data. Furthermore, the tuning curves are no
longer repelled from the adapter (panel D), but are instead attracted toward
it.

##  Appendix B Objective ablation

Here, we assess the contribution of each term of the objective (Equation 4) to
explain the adaptation effects found in data.

###  B.1 Gain homeostasis confers representation stability with adaptation

Fig. B.1 shows the impact of removing the gain homeostasis term from Eq 4.
Gain homeostasis prevents the network from drastically re-configuring its
representation after adaptation, and allows the network to maintain a stable
representation of the stimulus ensemble (compare orange to green).

Figure B.1: Gain homeostasis induces stability across statistical contexts.
Histograms are bootstrap samples (1000 repeats) of the average stimulus
reconstruction error under the uniform stimulus ensemble without adaptation,
after adaptation with gain homeostasis, and after adaptation without gain
homeostasis.

###  B.2 Contributions of each term to adaptation

Figure B.2: Each term in the objective (Equation 4) is necessary to account
for the full array of adaptation effects observed in data.

We assess the importance of the three terms in the objective (Eq. 4) and show
that they are all jointly necessary to produce the effects shown in main text.
Figure B.2 shows the adapted model responses using Eq. 4 to adapt in red.
Without the gain homeostasis term (i.e. $\gamma=0$, green), the gains
radically change after adapting to the biased stimulus ensemble. This produces
higher responses in neurons tuned for orientations along the flank (far from
the adapter at zero degrees), and completely fails to reproduce any of the
adaptation effects observed in data. Without the activity penalty (i.e.
$\alpha=0$, blue), the model’s responses are equivalent to one with no
adaptation. Finally, without the $\ell_{2}$ reconstruction penalty (first term
of objective; purple), the maxima and minima undershoot what is observed in
data; however, the model does reasonably well at capturing the shifts in
tuning preference and response amplitude. The reconstruction term of the
objective encourages the network to maintain a high fidelity representation of
the stimulus after adaptation. This ablation finding suggests that the shifts
in tuning preference observed in many previous studies (Clifford et al., ,
2007) may arise from adaptive sensory information-preserving properties of the
system. Taken together, each component of the objective works in concert to
yield the adaptation response phenomena seen in th data.

##  Appendix C Analytic solution to the adaptation objective

From Eq. 3, the steady state response of a network is given by

$\displaystyle{\mathbf{r}}_{*}({\bf s},{\bf g})$ $\displaystyle=[{\bf
I}-{{\mathbf{W}}}]^{-1}\left({\bf g}\circ{\bf f}({\bf s})\right)$
$\displaystyle{\mathbf{r}}_{*}({\bf s},{\bf g})$ $\displaystyle={\bf
M}\left({\bf g}\circ{\bf f}({\bf s})\right)$ (C.1)

where ${{\mathbf{W}}}\prec{\bf I}$, and ${\bf M}:=[{\bf
I}-{{\mathbf{W}}}]^{-1}$ is a matrix capturing the effect of leak and lateral
recurrence in the network. The feedforward and recurrent weights of our model
are assumed to be fixed through adaptation. We can isolate ${\bf g}$ using the
identity $\operatorname{diag}\left(\bf a\right){\bf
b}=\operatorname{diag}\left(\bf b\right){\bf a}$, for two vectors $\bf a$ and
$\bf b$, to get:

$\displaystyle{\bf M}\left({{\bf g}}\circ{\bf f}({\bf s})\right)$
$\displaystyle={\bf M}\operatorname{diag}\left({\bf f}({\bf s})\right){\bf g}$
$\displaystyle\equiv{\bf H}({\bf s}){\bf g},$

where we define ${\bf H}:\mathbb{R}^{N}\mapsto\mathbb{R}^{N\times N}$ as a
linear operator that maps s to a matrix using M and ${\bf f}({\bf s})$.

In the main text, the loss functional (Eq. 4) omitted dependence on the
decoder ${\bf D}$ for clarity. The full objective is

$\displaystyle\mathcal{L}(p({\bf s}),{\bf g},{\bf D})$
$\displaystyle=\mathbb{E}_{{\bf s}\sim p({\bf
s})}\left[\underbrace{\parallel{\bf s}-{\bf D}^{\top}{\bf H}({\bf s}){\bf
g}\parallel_{2}^{2}}_{\text{reconstruction}}+\alpha\underbrace{\parallel{\bf
H}({\bf s}){\bf
g}\parallel_{2}^{2}}_{\text{activation}}\right]+\gamma\underbrace{\parallel{\bf
g}-{\bf g}_{0}\parallel^{2}_{2}}_{\text{homeostatic
gain}}+\delta\underbrace{\parallel{\bf
D}\parallel_{\text{F}}^{2}}_{\text{decoder}},$ (C.2)

where $\delta$ is a hyperparameter controlling the decoder weights
${\mathbf{D}}$. The results in the main text have $\delta$ set to zero, and
our findings do not qualitatively change with small deviations away fro
$\delta=0$. This objective is bi-convex in ${\bf D}$ and ${\bf g}$ (i.e.
convex when one of the two variables is held fixed). Indeed, with ${\bf g}$
fixed, the loss simply becomes an $\ell_{2}$-regularized least-squares problem
in ${\bf D}$. One can show that the linear decoder regularization term,
$\delta$, is equivalent to assuming noisy outputs with additive isotropic
Gaussian noise. We solve for each optimization variable in alternation until
they reach convergence. As our stimulus ensemble comprises a discrete set of
$K$ stimuli, we can write our objective explicitly as a weighted summation,

$\mathcal{L}({\bf g})=\frac{1}{2}\sum_{k=1}^{K}p({\bf
s}_{k})\left\\{\parallel{\bf s}_{k}-{\bf D}^{\top}{\bf H}({\bf s}_{k}){\bf
g}_{s}\parallel_{2}^{2}+\alpha\parallel{\bf H}({\bf s}_{k}){\bf
g}\parallel_{2}^{2}\right\\}+\gamma\parallel{\bf g}-{\bf
g}_{0}\parallel^{2}_{2}.$ (C.3)

Computing $\nabla\mathcal{L}_{\bf g}=0$, and isolating for ${\bf g}$ yields a
linear system of equations,

$\left[\sum_{k}p({\bf s}_{k})\left\\{{\bf H}({\bf s}_{k})^{\top}\left({\bf
DD}^{\top}+\alpha{\bf I}\right){\bf H}({\bf s}_{k})\right\\}+\gamma{\bf
I}\right]{\bf g}=\left[\sum_{k}p({\bf s}_{k})\left\\{{\bf H}({\bf
s}_{k})^{\top}{\bf D}\hat{\bf s}_{k}\right\\}\right]+\gamma{\bf g}_{0}.$ (C.4)

This is in the form of ${\bf Ag}={\bf b}$ and can therefore be solved exactly
(e.g. using numpy.linalg.solve()). A similar derivation can be done for the
optimal ${\bf D}$. To initialize ${\bf g}_{0}$ and ${\bf D}$, we alternated
optimization between ${\bf D}$ and ${\bf g}$ (using the control context
stimulus ensemble) until convergence using co-ordinate descent.

##  Appendix D Alternative models

###  D.1 Equivalent circuit with adaptive recurrent gain

Here, we explore an alternative network parameterization which has identical
steady-state behavior as the network we have selected for our model:
consequently, adaptation under our training procedure will have identical
behavior at a network level for both parameterizations. The dynamics of our
network, replicated from Equation 2 for clarity, are given by

$\displaystyle\frac{d{\mathbf{r}}({\mathbf{s}},{\mathbf{g}})}{dt}$
$\displaystyle=-{\mathbf{r}}+{{\mathbf{W}}}{\mathbf{r}}+{\bf g}\circ{\bf
f}({\bf s}),$ $\displaystyle=[-{\bf I+W}]{\mathbf{r}}+{\bf g}\circ{\bf f}({\bf
s}),$ (D.1)

where we denote both the leak term,$-{\bf Ir}$, and the recurrent weight term,
${{\mathbf{W}}}{\mathbf{r}}$, as the recurrent drive. As an alternative to
this parameterization, consider instead multiplicatively scaling each neuron’s
recurrent drive with ${\bf g}\in\mathbb{R}^{N}_{+}$,

$\frac{d{\mathbf{r}}({\mathbf{s}},{\mathbf{g}})}{dt}={\bf g}^{-1}\circ[-{\bf
I+W}]{\mathbf{r}}+{\bf f}({\bf s}),$ (D.2)

where ${\bf g}^{-1}=[1/g_{1},1/g_{2},...,1/g_{N}]^{\top}$. Intuitively,
Equation D.2 states that an increase in each neuron’s $g_{i}$ attenuates its
recurrent drive. Gain changes in this model adjust the overall sensitivity to
recurrent (including self-recurrence, i.e. leak) drive. To solve for this new
network’s steady-state, ${\mathbf{r}}_{\ast}({\bf s},{\bf g})$, we set
Equation D.2 to zero and solve,

$\displaystyle{{\bf g}^{-1}}\circ[{\bf I-W}]\left({\mathbf{r}}_{\ast}({\bf
s},{\bf g})\right)$ $\displaystyle={\bf f}({\bf s})$ (D.3)
$\displaystyle{\mathbf{r}}_{\ast}({\bf s},{\bf g})$
$\displaystyle=\left[\operatorname{diag}\left({\bf g}^{-1}\right)[{\bf
I-W}]\right]^{-1}{\bf f}({\bf s})$ (D.4)
$\displaystyle{\mathbf{r}}_{\ast}({\bf s},{\bf g})$ $\displaystyle=[{\bf
I-W}]^{-1}\left({{\bf g}}\circ{\bf f}({\bf s})\right),$

which is the same as our steady-state equation given by Equation 3 in the main
text. Therefore, our original formulation of multiplicatively scaling the
network’s feedforward drive is mathematically equivalent to inversely scaling
(attenuating) its recurrent drive. This means that upscaling gain on
feedforward inputs is equivalent to downscaling net inhibition on each neuron.

###  D.2 Equivalent circuit with two-layer feedforward architecture

The overall action of the network at steady-state (Equation 3) is to mix the
gain-modulated feedforward responses, ${\bf g\circ f(s)}$, by a linear
transformation dependent on the recurrent circuitry
$[{\mathbf{I}}-{\mathbf{W}}]^{-1}$. The steady-state response of this
recurrent network is equivalent to a two-layer feedforward network with gain
modulation after the first layer. This means that viewing the steady-state
responses alone, as is frequently done in neurophysiological adaptation
experiments (Weber et al., , 2019), it is impossible to tell whether the
system was exclusively feedforward or recurrent. This is well-known (Dayan and
Abbott, , 2005) and is a fundamental result in signal processing theory of
linear feedback systems. Our model can be interpreted as a cascade of
transformations, propagating adaptive response changes downstream (Kohn and
Movshon, , 2003; Dhruv and Carandini, , 2014).

###  D.3 Relationship to divisive normalization

Our model is linear and not divisive. Writing the steady-state for the $i$th
neuron explicitly (omitting ${\bf g}$ to reduce clutter) yields

$\displaystyle r_{i}$ $\displaystyle=f_{i}({\bf s})+\sum_{j=1}^{N}w_{j}r_{j}$
$\displaystyle r_{i}$ $\displaystyle=f_{i}({\bf s})+\sum_{j\neq
i}w_{j}r_{j}+w_{i}r_{i}$ $\displaystyle(1-w_{i})r_{i}$
$\displaystyle=f_{i}({\bf s})+\sum_{j\neq i}w_{j}r_{j}$ $\displaystyle r_{i}$
$\displaystyle=\frac{1}{1-w_{i}}\left[\sum_{j}f_{j}({\bf s})+\sum_{j\neq
i}w_{j}r_{j}\right].$

Thus, there is no nonlinear interaction between $r_{i}$ and other neurons in
the population.