Deep Predictive Learning in Neocortex and Pulvinar
Randall C. O’Reilly, Jacob L. Russin, Maryam Zolfaghar, and John Rohrlich
Department of Psychology, Computer Science, and Center for Neuroscience
University of California Davis
1544 Newton Ct
Davis, CA 95618
<EMAIL_ADDRESS>
August 27, 2024

We thank Dean Wyatte, Tom Hazy, Seth Herd, Kai Krueger, Tim Curran, David
Sheinberg, Lew Harvey, Jessica Mollick, Will Chapman, Helene Devillez, and the
rest of the CCN Lab for many helpful comments and suggestions. Supported by:
ONR grants ONR N00014-19-1-2684 / N00014-18-1-2116, N00014-14-1-0670 /
N00014-16-1-2128, N00014-18-C-2067, N00014-13-1-0067, D00014-12-C-0638. This
work utilized the Janus supercomputer, which is supported by the National
Science Foundation (award number CNS-0821794) and the University of Colorado
Boulder. The Janus supercomputer is a joint effort of the University of
Colorado Boulder, the University of Colorado Denver and the National Center
for Atmospheric Research. All data and materials will be available at
https://github.com/ccnlab/deep-obj-cat upon publication.

###### Abstract

How do humans learn from raw sensory experience? Throughout life, but most
obviously in infancy, we learn without explicit instruction. We propose a
detailed biological mechanism for the widely-embraced idea that learning is
driven by the differences between predictions and actual outcomes (i.e.,
_predictive error-driven learning_). Specifically, numerous weak projections
into the pulvinar nucleus of the thalamus generate top-down predictions, and
sparse, focal _driver_ inputs from lower areas supply the actual outcome,
originating in layer 5 intrinsic bursting (5IB) neurons. Thus, the outcome
representation is only briefly activated, roughly every 100 ms (i.e., 10 Hz,
_alpha_), resulting in a _temporal difference error signal_ , which drives
local synaptic changes throughout the neocortex. This results in a
biologically-plausible form of error backpropagation learning. We implemented
these mechanisms in a large-scale model of the visual system, and found that
the simulated inferotemporal (IT) pathway learns to systematically categorize
3D objects according to invariant shape properties, based solely on predictive
learning from raw visual inputs. These categories match human judgments on the
same stimuli, and are consistent with neural representations in IT cortex in
primates.

The fundamental epistemological conundrum of how knowledge emerges from raw
experience has challenged philosophers and scientists for centuries. Although
there have been significant advances in cognitive and computational models of
learning (Ashby Maddox, 2011; Watanabe Sasaki, 2015; LeCun ., 2015) and in our
understanding of the detailed biochemical basis of synaptic plasticity
(Lüscher Malenka, 2012; Shouval ., 2002; Cooper Bear, 2012; Urakubo ., 2008),
there is still no widely-accepted answer to this puzzle that is clearly
supported by known biological mechanisms and also produces effective learning
at the computational and cognitive levels. The idea that we learn via an
active _predictive_ process was advanced by Helmholtz in his _recognition by
synthesis_ proposal (von Helmholtz, 1867), and has been widely embraced in a
range of different frameworks (JL. Elman, 1990; J. Elman ., 1996; Mumford,
1992; Kawato ., 1993; Dayan ., 1995; Rao Ballard, 1999; Friston, 2005; Hawkins
Blakeslee, 2004; George Hawkins, 2009; Clark, 2013; Summerfield de Lange,
2014; de Lange ., 2018).

Here, we propose a detailed biological mechanism for a specific form of
_predictive error-driven learning_ based on distinctive patterns of
connectivity between the neocortex and the higher-order nuclei of the thalamus
(i.e., the pulvinar) (SM. Sherman Guillery, 2006; Usrey Sherman, 2018). We
hypothesize that learning is driven by the difference between top-down
predictions, generated by numerous weak projections into the thalamic relay
cells (TRCs) in the pulvinar, and the actual outcomes supplied by sparse,
strong _driver_ inputs from lower areas. Because these driver inputs originate
in layer 5 intrinsic bursting (5IB) neurons, the outcome is only briefly
activated, roughly every 100 ms (i.e., 10 Hz, _alpha_). Thus, the prediction
error is a _temporal difference_ in activation states over the pulvinar, from
an earlier prediction to a subsequent burst of outcome. This temporal
difference can drive local synaptic changes throughout the neocortex,
supporting a biologically-plausible form of error backpropagation that
improves the predictions over time (O’Reilly, 1996; Ackley ., 1985; Hinton
McClelland, 1988; Bengio ., 2017; Whittington Bogacz, 2019; Lillicrap .,
2020). The temporal-difference form of error-driven learning contrasts with
prevalent alternative hypotheses that require a separate population of neurons
to compute a prediction error _explicitly_ and transmit it directly through
neural firing (Rao Ballard, 1999; Kawato ., 1993; Friston, 2005, 2010; Ouden
., 2012; Lotter ., 2016).

In the following, our primary objective is to describe the hypothesized
biologically based mechanism for predictive error-driven learning, contrast it
with other existing proposals regarding the functions of this thalamocortical
circuitry and other ways that the brain might support predictive learning, and
evaluate it relative to a wide range of existing anatomical and
electrophysiological data. We provide a number of specific empirical
predictions that follow from this functional view of the thalamocortical
circuit, which could potentially be tested by current neuroscientific methods.
Thus, this work proposes a clear functional interpretation of this distinctive
thalamocortical circuitry that contrasts with existing ideas in testable ways.

A second major objective is to implement this predictive error-driven learning
mechanism in a large-scale computational model that faithfully captures its
essential biological features, to test whether the proposed learning mechanism
can drive the formation of cognitively useful representations. In particular,
we ask a critical question for any predictive-learning model: can it develop
high-level, abstract representations while learning from nothing but
predicting low-level visual inputs. Most visual object recognition models that
provide a reasonable fit to neurophysiological data rely on large human-
labeled datasets to explicitly train abstract category information via error-
backpropagation (Cadieu ., 2014; Khaligh-Razavi Kriegeskorte, 2014;
Rajalingham ., 2018). Thus, it is perhaps not too surprising that the higher
layers of these models, which are closer to these category output labels,
exhibited a greater degree of categorical organization.

Through large-scale simulations based on the known structure of the visual
system, we found that our biologically based predictive learning mechanism
developed high-level, abstract representations that significantly diverge from
the similarity structure present in the lower layers of the network, and
systematically categorize 3D objects according to invariant shape properties.
Furthermore, we found in an experiment using the same stimuli that these
categories match human similarity judgments, and that they are also
qualitatively consistent with neural representations in inferotemporal (IT)
cortex in primates (Cadieu ., 2014). In addition, we show that comparison
predictive backpropagation models lacking these biological features (Lotter .,
2016) did not learn object categories that go beyond the visual input
structure. Thus, there may be some important features of the biologically
based model that enable this ability to learn higher-level structure beyond
that of the raw inputs.

It is important to emphasize that our objectives for these simulations are
_not_ to produce a better machine-learning (ML) algorithm _per se_ , but
rather to test whether our biologically based model can capture some of the
known high-level, cognitive phenomena that the mammalian brain learns. Thus,
we explicitly dissuade readers from the inevitable desire to evaluate the
importance of our model based on differences in narrow, performance-based ML
metrics. As discussed later, there are various engineering-level issues
regarding the biologically based model’s computational cost and performance,
that currently limit its ability to compete with simpler, much larger-scale
backpropagation models, but we do not think these are relevant to the
evaluation of the scientific questions of relevance here. In short, this model
is an instantiation of a scientific theory, and it should be evaluated on its
ability to explain a wide range of data across multiple levels of analysis,
just as every other scientific theory is evaluated.

The remainder of the paper is organized as follows. First, we provide a
concise overview of the biologically based predictive error-driven learning
framework, including the most relevant neural data. Then, we present a small-
scale implementation of the model that learns a probabilistic grammar, to
illustrate the basic computational mechanisms of the theory. This is followed
by the large-scale model of the visual system, which learns by predicting over
brief movies of 3D objects rotating and translating in space. We evaluate this
model and compare it to two other predictive learning models that directly use
error-backpropagation, based on current deep convolutional neural network
(DCNN) mechanisms. Then, we circle back to discuss the relevant biological
data in greater detail, along with testable predictions that can differentiate
this account from other existing ideas. Finally, we conclude with a discussion
of related models and outstanding issues.

##  1 Predictive Error-driven Learning in the Neocortex and Pulvinar

Figure 1: Summary figure from Sherman & Guillery (2006) showing the strong
feedforward _driver_ projection emanating from layer 5IB cells in lower layers
(e.g., V1), and the much more numerous feedback “modulatory” projection from
layer 6CT (corticothalamic) cells. We interpret these same connections as
providing a prediction (6CT) vs. outcome (5IB) activity pattern over the
pulvinar. Figure 2: Corticothalamic information flow under our predictive
learning hypothesis, shown as a sequence of movie frames (Retina),
illustrating the three key steps taking place within a single 125 ms time
window, broken out separately across the three panels: a) prior context is
updated in the V2 CT layer; b) which is then used to generate a prediction
over the pulvinar (V2 P); c) against which the outcome, driven by bottom-up
5IB bursting, represents the prediction error _as a temporal difference
between the prediction and outcome states over the pulvinar_. Changes in
synaptic weights (learning) in all superficial (S) and CT layers are driven
from the _local_ temporal difference experienced by each neuron, using a form
of the contrastive hebbian learning (CHL) term as shown, where the ‘$+$’
superscripts indicate outcome activations, and ‘$-$’ superscripts indicate
prediction. CHL approximates the backpropagated prediction error gradient
experienced by each neuron (O’Reilly, 1996), reflecting both direct pulvinar
error signals, and indirect corticocortical error signals as well. In
specific: a) CT context updating occurs via 5IB bursting (not shown) in higher
layer (V2) during prior alpha (100 ms) cycle — this context is maintained in
the CT layer and used to generate predictions. b) The prediction over pulvinar
is generated via numerous top-down CT projections. This prediction state also
projects up to S and CT layers, and from S to all other S layers via extensive
bidirectional connectivity, so their activation state reflects this prediction
as well. c) The subsequent outcome drives pulvinar activity bottom-up via V1
5IB bursting, and is likewise projected to S and CT layers, ensuring that the
relevant temporal difference error signal is available locally in cortex. The
difference in activation values across these two time points, in S and CT
layers throughout the network, drives learning to reduce prediction errors.
Note that the single most important property of the 5IB bursting is that these
driver cells are _not_ active during the prediction phase — the bursting
itself may also be useful in the driving property, but that is a secondary
consideration to the critical feature of having a time when the prediction
alone can be projected onto the pulvinar.

Figure 1 shows the thalamocortical circuits characterized by SM. Sherman
Guillery (2006) (see also SM. Sherman Guillery, 2013; Usrey Sherman, 2018),
which have two distinct projections converging on the principal thalamic relay
cells (TRCs) of the _pulvinar_ , the primary thalamic nucleus that is
interconnected with higher-level posterior cortical visual areas (Shipp, 2003;
Arcaro ., 2015; Halassa Kastner, 2017). One projection consists of numerous,
weaker connections originating in deep layer VI of the neocortex (the 6CT
corticothalamic projecting cells), which we hypothesize generate a top-down
prediction on the pulvinar. The other is a sparse, focal (Rockland, 1998,
1996) and strong _driver_ pathway that originates from lower-level layer 5
intrinsic bursting cells (5IB), which we hypothesize provide the outcome.
These 5IB neurons fire discrete bursts with intrinsic dynamics having a period
of roughly 100 ms between bursts (Connors ., 1982; Silva ., 1991; Larkum .,
1999; Franceschetti ., 1995; Saalmann ., 2012), which is thought to drive the
widely-studied _alpha_ frequency of $\sim$ 10 Hz that originates in cortical
deep layers and has important effects on a wide range of perceptual and
attentional tasks (Buffalo ., 2011; VanRullen Koch, 2003; K. Mathewson .,
2009; Jensen ., 2012; Clayton ., 2018). Critically, unlike many other such
bursting phenomena, this 5IB bursting occurs in awake animals (Luczak ., 2009,
2013; Sakata Harris, 2009, 2012), consistent with presence of alpha in awake
behaving states.

The existing literature generally characterizes the 6CT projection as
_modulatory_ (SM. Sherman Guillery, 2013; Usrey Sherman, 2018), but a number
of electrophysiological recordings from awake, behaving animals clearly show
sustained, continuous patterns of neural firing in pulvinar TRC neurons, which
is not consistent with the idea that they are only being driven by their
phasic bursting 5IB inputs (Bender, 1982; Petersen ., 1985; Bender Youakim,
2001; Robinson, 1993; Saalmann ., 2012; Komura ., 2013; Zhou ., 2016). Indeed,
these recordings show that pulvinar neural firing generally resembles that of
the visual areas with which they interconnect, in terms of neural receptive
field properties, tuning curves, etc. This is important because our predictive
learning framework requires that these 6CT top-down projections be capable of
directly driving TRC activity. Specifically, in contrast to the standard view,
the core idea behind our theory is that the top-down 6CT projections drive a
_predicted_ activity pattern across the extent of the pulvinar, which precedes
the subsequent _outcome_ activation state driven by the strong 5IB inputs.

Figure 2 illustrates the temporal evolution of activity states according to
our predictive learning theory, which is somewhat challenging to convey
because the critical signals driving learning unfold _over time_ (Kachergis .,
2014; O’Reilly ., 2014, 2017). We hypothesize that synaptic plasticity
throughout the cortex is sensitive to the resulting _temporal differences_
that emerge initially in the pulvinar. Thus, unlike other models (as we
discuss in depth later) the prediction error here is not captured directly in
the firing of a special population of error-coding neurons, but rather remains
as a temporal difference error signal.

The figure shows a single 125 ms time window of a 100 ms alpha cycle for the
purposes of illustration (the actual timing is likely to be more dynamic as
discussed next). The activity state in pulvinar TRC neurons, representing a
prediction, as driven by the top-down 6CT projections, should develop during
the first $\sim$ 75 ms, when the 5IB neurons are paused between bursting. Then
the final $\sim$ 25 ms largely reflects the strong 5IB bottom-up ground-truth
driver inputs when they burst. Thus, the prediction error signal is reflected
in the temporal difference of these activation states as they develop over
time. In other words, our hypothesis is that the pulvinar is directly
representing either the top-down prediction or the bottom-up outcome at any
given time, and the temporal difference between these states implicitly
encodes a prediction error. While the deep 6CT layer is involved in generating
a top-down prediction over the pulvinar, the superficial layer neurons
continuously represent the current state, simultaneously incorporating bottom-
up and top-down constraints via their own connections with other areas. To
ensure that the prediction is not directly influenced by this current state
representation (i.e., “peeking at the right answer”), it is important that the
6CT neurons encode temporally delayed information, consistent with available
data (Harris Shepherd, 2015; Sakata Harris, 2009; Thomson, 2010).

The actual biological system is likely to be much more dynamic than the
simplistic cartoon with rigid 100 ms timing as shown in Figure 2, based on a
set of neural mechanisms that can work together to enable it to more flexibly
entrain the predictive learning cycle to the environment. These mechanisms
would also tend to increase activity and learning associated with unexpected
outcomes relative to expected ones, consistent with the observed _expectation
suppression_ phenomena (Summerfield ., 2008; Todorovic ., 2011; Meyer Olson,
2011; Bastos ., 2012).

Specifically, various underlying mechanisms result in neural _adaptation_ ,
which is generally thought to increase neural activity and learning associated
with novel inputs relative to recently familiar ones (Müller ., 1999; Abbott
., 1997; Brette Gerstner, 2005; Grill-Spector ., 2006; Hennig, 2013). In the
case where outcomes are consistent with prior predictions (i.e., the
predictions are accurate), the same population of neurons across pulvinar and
cortex should be active over time, whereas unpredicted outcomes will generally
activate new subsets of neurons in superficial cortical layers representing
the current state. Thus, due to adaptation, there should be a phasic increase
in activity in these superficial neurons at the onset of unpredicted stimuli
relative to predicted ones. Furthermore, the 5IB neurons downstream of these
superficial neurons may be particularly responsive to these phasic activity
increases, causing their bursting to coincide preferentially with unexpected
outcomes, thereby driving the phase resetting of the alpha cycle to such
events. Thus, during a sequence of predicted states, the pulvinar may
experience relatively weaker or even absent 5IB driving inputs, until an
unpredicted stimulus arises. At this point, error-driven learning would be
more strongly engaged as a function of the phasic release from adaptation and
5IB burst activation. We discuss these dynamics more later in the context of
the comparison with explicit error coding models.

We also hypothesize that 5IB bursting preferentially drives the synaptic
plasticity processes to take place at that time, due the strong driving nature
of the outputs from these neurons. In computational terms originating with the
_Boltzmann Machine_ (Ackley ., 1985; Hinton Salakhutdinov, 2006), this anchors
the target or _plus_ phase to be at this point of 5IB bursting. Furthermore,
this means that the predictive nature of the prior minus phase naturally
emerges just by virtue of it being the state prior to 5IB bursting: the
learning rule automatically causes that prior state to better anticipate the
subsequent state. Thus, even if no prediction was initially generated,
learning over multiple iterations will work to create one, to the extent that
a reliable prediction can be generated based on internal states and
environmental inputs. Likewise, assuming relevant activity traces naturally
persist over timescales longer than the alpha cycle, this predictive learning
process can take advantage of any such remaining traces to learn across these
longer timescales, even though it is operating at the faster alpha scale.

In short, learning always happens whenever something unexpected occurs, at any
point, and drives the development of predictions immediately prior, to the
extent such predictions are possible to generate. In the typical lab
experiment where phasic stimuli are presented without any predictable temporal
sequence (which is uncharacteristic of the natural world), there may often be
no significant prediction prior to stimulus onset, and we would expect such
stimuli to reliably drive 5IB bursting, which is consistent with available
electrophysiological data (Bender, 1982; Petersen ., 1985; Bender Youakim,
2001; Robinson, 1993; Luczak ., 2009, 2013; Komura ., 2013; Zhou ., 2016).
Thus, unlike Figure 2, such situations would _start_ with a 5IB-triggered plus
phase, without a significant minus phase prior to that.

As may be evident by this point, we are mainly focused on _prediction_ in the
sense of the humorous quote: “prediction is very difficult, especially about
the future” (attributable to Danish author Robert Storm Petersen), whereas
this term is potentially confusingly used in a much broader sense in most
Bayesian-inspired predictive coding frameworks (Rao Ballard, 1999; Friston,
2005; de Lange ., 2018). These frameworks use “prediction” to encompass
everything from genetic biases to the results of learning in the feedforward
synaptic pathways to top-down filling-in or biasing of the current stimulus
properties, and fairly rarely use it in the “about the future” sense. We think
these different phenomena are each associated with different neural mechanisms
at different time scales (O’Reilly ., 2016, 2013, 2012), and thus prefer to
treat them separately, while also recognizing that they can clearly interact
as well.

Thus, our use of the term _prediction_ here refers specifically to
_anticipatory_ neural firing that predicts subsequent stimuli. We use the term
_postdiction_ to refer to the operation of this predictive mechanism after a
stimulus has been initially processed (to consolidate and more deeply encode,
as in an auto-encoder model), and distinguish both from _top-down excitatory
biasing_ , which directly influences the online superficial layer neural
representations of the current stimulus (Desimone Duncan, 1995; Reynolds .,
1999; EK. Miller Cohen, 2001; O’Reilly ., 2013). Finally, many discussions of
prediction error in the literature include late, frontally-associated
processes such as those associated with the P300 ERP component (Holroyd Coles,
2002). We specifically exclude these from the scope of the mechanisms
described here, which are anticipatory, fast, and low-level, as is appropriate
for the posterior cortical sensory processing areas that interconnect with the
pulvinar.

###  1.1 Computational Properties of Predictive Learning in the
Thalamocortical Circuits

We next elaborate the connections between the computational properties
required for predictive learning, and the properties of the circuits
interconnecting cortex and the pulvinar, which appear to be notably well
suited for their hypothesized role in predictive learning. We begin with a
relatively established interpretation of superficial layer processing, to
contextualize subsequent points about the special functions required of the
deep layers and the thalamus.

  * •

The superficial cortical layers continuously represent the current state: The
superficial layer pyramidal neurons are densely and bidirectionally
interconnected with other cortical areas, and update quickly to new stimulus
inputs, with continuous, relatively rapid firing (i.e., up to about 100 Hz for
preferred stimuli). These neurons integrate higher-level top-down information
with bottom-up sensory information to resolve ambiguities, focus attention,
fill in missing information, and generally enhance the consistency and quality
of the online representations (Rumelhart McClelland, 1982; Hopfield, 1984;
Desimone Duncan, 1995; Reynolds ., 1999; EK. Miller Cohen, 2001; O’Reilly .,
2013, 2012, 2016). As noted above, we distinguish this form of top-down
processing, which is often most evident during the period _after_ stimulus
onset (Lee Mumford, 2003), from the specifically predictive, anticipatory
sort.

  * •

Predictions must be insulated against receiving current state information (it
isn’t prediction if you already know what happens): Given that the superficial
layers are continuously updating and representing the current state, some kind
of separate neural system insulated from this current state information must
be used to generate predictions, otherwise the prediction system can just
“cheat” and directly report the current state. It may seem counter-intuitive,
but making the prediction task _harder_ is actually beneficial, because that
pushes the learning to capture deeper, more systematic regularities about how
the environment evolves over time. In other words, like any kind of cheating,
the cheater itself is cheated because of the reduced pressure to learn, and
learning is the real goal.

  * •

Predictions take time and space to generate: Non-trivial predictions likely
require the integration of multiple converging inputs from a range of higher-
level cortical areas, each encoding different dimensions of relevance (e.g.,
location, motion, color, texture, shape, etc). Thus, sufficient time and space
(i.e., neural substrates with relevant connectivity) must be available to
integrate these signals into a coherent predicted state, and per the above
point, these substrates must be separated from the influence of current state
information. This fits with the properties of the layer 6CT neurons and their
deep layer inputs, which we hypothesize are insulated from superficial-layer
firing by virtue of being driven locally by the 5IB bursting within their own
cortical microcolumn, such that the inter-bursting pause period provides a
time window when these deep layers can integrate and generate the prediction.

Biologically, this is consistent with the delayed responses of 6CT neurons
(Harris Shepherd, 2015; Sakata Harris, 2009; Thomson, 2010). Computationally,
these neurons function much like the simple recurrent network (SRN) context
layer updating (JL. Elman, 1990; Jordan, 1989) which reflects the prior
trial’s state, as discussed in detail in the Appendix. The overall duration of
the alpha cycle may represent a reasonable compromise between the prediction
integration time and the need to keep up with predictions tracking changes in
the world. Notably, films are typically shown at just over 2 times the alpha
frequency (24 Hz), suggesting a Nyquist sampling relative to the underlying
alpha processing.

  * •

The predicted state must be directly aligned with the outcome state it
predicts: A prediction error is a difference between two states, so these
prediction and outcome states must be directly comparable such that their
difference meaningfully represents the actual prediction error, and not some
other kind of irrelevant encoding differences. In other words, the prediction
and the outcome must be represented in the same “language”, so that the
“words” from the prediction can be directly compared against those of the
outcome — if the prediction was in Japanese and the outcome in English, it
would be hard to tell whether the prediction was correct or not! Thus, a
common neural substrate with two different input pathways is required, one
reflecting the prediction and the other the outcome, so that both converge
onto the same representational system within this common neural substrate.
This fits well with the two pathways converging into the pulvinar: the 6CT
top-down prediction-generation pathway and the lower-level 5IB driving inputs.

  * •

The outcome signal should be as _veridical_ as possible (i.e., directly
reflecting the bottom-up outcome), and should arise from lower areas in the
hierarchy relative to the corresponding predictive 6CT inputs: Given that the
outcome is the driver of learning, if it were to be corrupted or inaccurate,
then everything that is learned would then be suspect. To the extent that
delusional thinking is present in all people (some moreso than others perhaps)
this principle must be violated at some level, but for the lowest levels of
the perceptual system at least, it is important that strongly grounded,
accurate training signals drive learning. The bottom-up, sparse, strongly
driving nature of the 5IB projections to the pulvinar can directly convey such
veridical outcome signals, and ensure that they dominate the activation of
their TRC targets. Based on indirect available data, it is likely that each
pulvinar TRC neuron receives only roughly 1-6 driver inputs (SM. Sherman
Guillery, 2006, 2011), such that these sparse inputs directly convey the
signal from lower layers, without much further mixing or integration (which
could distort the nature of the signal). Furthermore, these inputs are likely
not plastic (Usrey Sherman, 2018), again consistent with a need for unaltered,
veridical signals. Lastly, the TRC neurons are distinctive in having no
significant lateral interconnectivity (SM. Sherman Guillery, 2006), enabling
them to faithfully represent their inputs. These properties led Mumford (1991)
to characterize the pulvinar as a _blackboard_ , and we further suggest the
metaphor of a _projection screen_ upon which the predictions are projected.

  * •

The prediction error must drive learning to reduce subsequent prediction
errors: Obviously, this is the goal of prediction error learning in the first
place, and given that the cortex is what generates predictions, it must be
capable of learning based on prediction error signals represented over the
pulvinar. Computationally, the critical problem here is _credit assignment_ :
how do the error signals direct learning in the proper direction for each
individual neuron, to reduce the overall prediction error? The error
backpropagation procedure solves this problem (Rumelhart ., 1986), but
requires biologically implausible retrograde signaling across the entire
network of neural communication (Crick, 1989), to propagate the error
proportionally back along the same channels that drive forward activation.
Bidirectional connections, which are ubiquitous in the cortex (Markov, Ercsey-
Ravasz ., 2014; Felleman Van Essen, 1991) and computationally beneficial for
other reasons as noted earlier, can eliminate that problem by “implicitly”
propagating error signals via standard neural communication mechanisms along
both directions of connectivity (O’Reilly, 1996).

This solution to the credit assignment problem relies on a _temporal
difference_ error signal, as originally developed for the _Boltzmann machine_
(Ackley ., 1985). The bidirectional neural communication at one point in time
is encoding and sharing the prediction among the entire network of neurons.
Then, this same network of connections is reused at another point in time to
encode and communicate the outcome. Mathematically, the difference in
activation state across these two points in time, locally at each individual
neuron, provides an accurate estimate of the error backpropagation gradient
(O’Reilly, 1996). In effect, this temporal difference tells each neuron which
direction it needs to change its activation state to reduce the overall error.
The reuse of the very same network of connections across both points in time
ensures the overall alignment of the two activation states, as noted above,
such that this temporal difference precisely represents the error signal.
While various other schemes for error-driven learning in biologically-
plausible networks have been proposed (e.g., Bengio ., 2017; Whittington
Bogacz, 2019; Lillicrap ., 2020), the temporal-difference framework with
bidirectional connectivity provides a particularly good fit with the natural
temporal ordering of predictive learning (prediction then outcome) and the
extensive bidirectional connectivity of the thalamocortical circuits (Shipp,
2003).

  * •

Temporal differences in activation state across the alpha cycle, between
prediction and outcome states, must drive synaptic plasticity: The final step
needed to connect all of the elements above is that neurons actually modify
their synaptic strengths in proportion to the temporal-difference error
signal. We have recently provided a fully explicit mechanism for this form of
learning (O’Reilly ., 2012), based on a biologically-detailed model of spike
timing dependent plasticity (STDP) (Urakubo ., 2008). We showed that when
activated by realistic Poisson spike trains, this STDP model produces a
nonmonotonic learning curve similar to that of the BCM model (Bienenstock .,
1982), which results from competing calcium-driven postsynaptic plasticity
pathways (Shouval ., 2002; Cooper Bear, 2012). As in the BCM framework, we
hypothesized that the threshold crossover point in this nonmonotonic curve
moves dynamically — if this happens on the alpha timescale (Lim ., 2015), then
it can reflect the prediction phase of activity, producing a net error-driven
learning rule based on a subsequent calcium signal reflecting the outcome
state. The resulting learning mechanism naturally supports a combination of
both BCM-style hebbian learning and error-driven learning, where the BCM
component acts as a kind of regularizer or bias, similar to weight decay
(O’Reilly Munakata, 2000; O’Reilly ., 2012).

Thus, remarkably, the pulvinar and associated thalamocortical circuitry
appears to provide _precisely_ the necessary ingredients to support predictive
error-driven learning, according to the above analysis. Interestingly,
although SM. Sherman Guillery (2006) did not propose a predictive learning
mechanism as just described, they did speculate about a potential role for
this circuit in motor forward-model learning and the predictive remapping
phenomenon (SM. Sherman Guillery, 2011; Usrey Sherman, 2018). In addition,
Pennartz . (2019) also suggested that the pulvinar may be involved in
predictive learning, but within the explicit error-coding framework and not
involving the detailed aspects of the above-described circuitry.

It bears emphasizing the synergy between the various considerations above for
the benefits of the pause in 5IB firing between bursts. First, this pause is
critical for creating the time window when the predictive network is
representing and communicating the prediction state, without influence from
the outcome state. Further, it creates the temporal difference in activation
state in the pulvinar between prediction and outcome, which is needed for
driving error-driven learning. Thus, for both the 6CT and pulvinar layers, the
periodic pausing of 5IB neurons is essential for creating the predictive
learning dynamic. Interestingly, by these principles, the lack of such burst /
pause dynamics in the driver inputs to first-order sensory thalamus areas such
as the LGN and MGN (SM. Sherman Guillery, 2006) means that these areas should
_not_ be directly capable of error-driven predictive learning. This is
consistent with a number of models and theoretical proposals suggesting that
primary sensory areas may learn predominantly through hebbian-style self-
organizing mechanisms (KD. Miller, 1994; Bednar, 2012). Nevertheless, primary
sensory areas do receive “collateral” error signals from the pulvinar (Shipp,
2003), which could provide some useful indirect error-driven learning signals.

Note that this form of temporal-difference learning signal is distinct from
the widely-used TD (temporal-difference) model in reinforcement learning
(Sutton Barto, 1998), which is scalar, and applies to reward expectations, not
sensory predictions (although see Gardner ., 2018 and Dayan, 1993 for
potential connections between these two forms of prediction error). Finally,
as we discuss later, this proposed predictive role for the pulvinar is
compatible with the more widely-discussed role it may play in attention
(LaBerge Buchsbaum, 1990; Bender Youakim, 2001; Snow ., 2009; Saalmann
Kastner, 2011; Zhou ., 2016; Fiebelkorn Kastner, 2019). Indeed, we think these
two functions are synergistic (i.e., you predict what you attend, and vice-
versa; Richter de Lange, 2019), and have initial computational results
consistent with this idea.

##  2 Predictive Learning of Temporal Structure in a Probabilistic Grammar

Figure 3: Finite state automaton (FSA) grammar used in implicit sequential
learning experiments (Reber, 1967) and in early simple recurrent networks
(SRNs) (Cleeremans & McClelland, 1991). It generates a sequence of letters
according to the link transitioned between state nodes, where each outgoing
link to another node has a 50% probability of being selected. Each letter
(except for the B=begin and E=end) appears at 2 different points in the
grammar, making them locally ambiguous. This combination of randomness and
ambiguity makes it challenging for a learning system to infer the true
underlying structure of the grammar. Figure 4: Predictive learning model
applied to the FSA grammar shown in previous figure. The first three panels
(a, b, c) show the prediction state (end of the _minus_ phase, e.g., the first
75 ms of an alpha cycle) of the trained model on the first three steps of the
sequence ‘BTX’ (plus phases also occurred, but are not shown). The last panel
(d) shows the plus phase after the third step. The _Input_ layer provides the
5IB drivers for the corresponding _HiddenP_ pulvinar layer, so the plus phase
is always based on the specific randomly-selected path taken. The _Targets_
layer is purely for display, showing the two valid possible labels that could
have been predicted. To track learning, the model’s prediction is scored as
accurate if either or both targets are activated. Computationally, the model
is similar to an SRN, where the CT layer that drives the prediction over the
pulvinar encodes the activation state from the previous time step (alpha
cycle), due to the phasic bursting of the 5IB neurons that drive CT updating.
Note how the CT layer in b) reflects the Hidden activation state in a), and
likewise for c) reflecting b). This is evident because we’re using one-to-one
connectivity between Hidden and HiddenCT layers (which works well in general,
along with full lateral connectivity within the CT layer). Thus, even though
the correct answer is always present on the Input layer for each step, the CT
layer is nevertheless attempting to predict this Input based on the
information from the prior time step. a) In the first step, the B label is
unambiguous and easily predicted (based on prior E context). b) In the 2nd
step, the network correctly guesses that the T label will come next, but there
is a faint activation of the other P alternative, which is also activated
sometimes based on prior learning history and associated minor weight tweaks.
c) In the 3rd step, both S and X are equally predicted. d) In the _plus_
phase, only the Input pattern (‘X’ on this trial) drives HiddenP activations,
and the projections from pulvinar back to the cortex convey both the minus-
phase prediction and plus-phase actual input. You can see one HiddenCT neuron,
just above the arrow, visibly changes its activation as a result (and all
neurons experience smaller changes), and learning in all these cortical
(Hidden) layer neurons is a function of their local temporal difference
between minus and plus phases.

To illustrate and test the predictive learning abilities of this biologically
based model, we first ran a classical test of sequence learning (Reber, 1967;
Cleeremans McClelland, 1991) that has been explored using simple recurrent
networks (SRNs) (JL. Elman, 1990; Jordan, 1989). The biologically based model
was implemented using the _Leabra_ algorithm, which is a comprehensive
framework that uses conductance-based point neuron equations, inhibitory
competition, bidirectional connectivity, and the biologically plausible
temporal difference learning mechanism described above (O’Reilly ., 2016,
2012; O’Reilly Munakata, 2000; O’Reilly, 1998, 1996). Leabra serves as a model
of the bidirectionally connected processing in the cortical superficial
layers, and has been used to simulate a large number of different cognitive
neuroscience phenomena. It is described in the Appendix, which also provides a
detailed mapping between the SRN and our biological model.

As shown in Figure 3, sequences were generated according to a finite state
automaton (FSA) grammar, as used in implicit sequence learning experiments by
Reber (1967). Each node has a 50% random branching to two different other
nodes, and the labels generated by node transitions are locally ambiguous
(except for the B=begin and E=end states). Thus, integration over time and
across many iterations are required to infer the systematic underlying
grammar. It is a reasonably challenging task for SRNs and people to learn and
provides an important validation of the power of these predictive learning
mechanisms. Given the random branching, accurately predicting the specific
path taken is impossible, but we can score the model’s output as correct if it
activates either or both of the possible branches for each state.

The model (Figure 4) required around 20 epochs of 25 sequences through the
grammar to learn it to the point of making no prediction errors for 5 epochs
in a row (which guarantees that it had completely learned the task). This
model is available in the standard _emergent_ distribution, at
https://github.com/emer/leabra/tree/master/examples/deep_fsa. A few steps
through a sequence are shown in the figure, illustrating how the CT context
layer, which drives the P pulvinar layer prediction, represents the
information present on the _previous_ alpha cycle time step. Thus, the network
is attempting to predict the current Input state, which then drives the
pulvinar plus phase at the end of each alpha cycle, as shown in the last
panel. On each trial, the difference between plus and minus phases locally
over each cortical neuron drives its synaptic weight changes, which accumulate
over trials to allow accurate prediction of the sequences, to the extent
possible given their probabilistic nature.

##  3 Predictive Learning of Object Categories in IT Cortex

Figure 5: a) The _What-Where-Integration, WWI_ deep predictive learning model.
The dorsal _Where_ pathway learns first, using easily-abstracted _spatial
blobs_ , to predict object location based on prior motion, visual motion, and
saccade efferent copy signals. This drives strong top-down inputs to lower
areas with accurate spatial predictions, leaving the _residual_ error
concentrated on _What_ and _What * Where_ integration. The V3 and DP (dorsal
prelunate) constitute the _What * Where_ integration pathway, binding features
and locations. V4, TEO, and TE are the _What_ pathway, learning abstracted
object category representations, which also drive strong top-down inputs to
lower areas. Suffixes: _s_ = superficial, _d_ = deep, _p_ = pulvinar. c)
Example sequence of 8 alpha cycles that the model learned to predict, with the
reconstruction of each image based on the V1 gabor filters (_V1h recon_), and
model-generated prediction (correlation $r$ prediction error shown). The low
resolution and reconstruction distortion impair visual assessment, but $r$
values are well above the $r$’s for each V1 state compared to the previous
time step (mean = .38, min of .16 on frame 4 — see Appendix for more
analysis). Eye icons indicate when a saccade occurred.

Now we describe a large-scale, systems-neuroscience implementation of the
proposed thalamocortical predictive error-driven learning framework, in a
model of visual predictive learning (Figure 5). Our second major objective,
and a critical question for predictive learning, is determining whether the
model can develop high-level, abstract ways of representing the raw sensory
inputs, while learning from nothing but predicting these low-level visual
inputs. We showed the model brief movies of 156 3D object exemplars drawn from
20 different basic-level categories (e.g., car, stapler, table lamp, traffic
cone, etc.) selected for their overall shape diversity from the CU3D-100
dataset (O’Reilly ., 2013). The objects moved and rotated in 3D space over 8
movie frames, where each frame was sampled at the alpha frequency (Figure 5b).
There were also saccadic eye movements every other frame, introducing an
additional, realistic, predictive-learning challenge. An efferent copy signal
enabled full prediction of the effects of the eye movement, and allows the
model to capture the signature predictive remapping phenomenon (Duhamel .,
1992; Cavanagh ., 2010; Neupane ., 2017). The _only_ learning signal available
to the model was the prediction error generated by the temporal difference
between what it predicted to see in the V1 input in the next frame and what
was actually seen.

As described in detail in the Appendix, our model was constructed to capture
critical features of the visual system, including the major division between a
dorsal _Where_ and ventral _What_ pathway (Ungerleider Mishkin, 1982), and the
overall hierarchical organization of these pathways derived from detailed
connectivity analyses (Rockland Pandya, 1979; Felleman Van Essen, 1991;
Markov, Vezoli ., 2014; Markov, Ercsey-Ravasz ., 2014). In addition to these
biological constraints, we conducted extensive exploration of the connectivity
and architecture space, and found a remarkable convergence between what worked
functionally and the known properties of these pathways (O’Reilly ., 2017).
For example, the feedforward pathway has projections from lower-level
superficial layers to superficial layers of higher levels, while feedback
originated in both the superficial and deep and projected back to both
(Rockland Pandya, 1979; Felleman Van Essen, 1991). Also, consistent with the
core features of the pulvinar pathways discussed above, deep layer predictive
(6CT) inputs originated in higher levels, while driver (5IB) inputs originated
in lower levels. For simplicity we organized the model layers in terms of
these driver inputs, whereas the topographic organization of pulvinar in the
brain is organized more according to the 6CT projection loops (Shipp, 2003).

Another important set of parameters are the strength of deep-layer recurrent
projections, which influence the timescale of temporal integration, producing
a simple biologically based version of _slow feature analysis_ (Wiskott
Sejnowski, 2002; Foldiak, 1991). We followed the biological data suggesting
that recurrence increases progressively up the visual hierarchy (Chaudhuri .,
2015). It was essential that the _Where_ pathway learn first, consistent with
extant data (Bourne Rosa, 2006; Kiorpes ., 2012), including early pathways
interconnecting LIP and pulvinar (Bridge ., 2016), and a rare asymmetric
pathway, from V1 to LIP (Markov, Ercsey-Ravasz ., 2014), providing a direct
short-cut for high-level spatial representations in LIP. Results from various
informative model architecture and parameter manipulations are discussed below
after the primary results from the standard intact model. Learning curves and
other model details are shown in the Appendix. We have also implemented a full
_de-novo_ replication of the model in a new modeling framework, which also
replicated the results shown here (see Appendix for more details).
Furthermore, much of the model was originally developed in the context of a
set of object-like patterns generated systematically from a set of simple line
features, indicating the general applicability of this architecture.

Figure 6: Emergence of abstract category structure over the hierarchy of
layers, comparing similarity structure in each layer vs that present in V1
(black line) or in TE (red line). Both cases, which are roughly symmetric,
clearly show that IT layers (TEO, TE) progressively differentiate from raw
input similarity structure present in V1, and, critically, that the model has
learned structure beyond that present in the input. This is is the simplest,
most objective summary statistic showing this progressive emergence of
structure, while subsequent figures provide a more concrete sense of what
kinds of representations actually developed.

To directly address the question of whether the hierarchical structure of the
network supports the development of abstract, higher-level representations
that go beyond the information present in the visual inputs, we applied a
second-order similarity measure across the object-level similarity matrices
computed at each layer in the network (Figure 6). This shows the extent to
which the similarity matrix across objects in one layer is itself similar to
the object similarity matrix in another layer, in terms of a correlation
measure across these similarity matrices. Critically, this measure does not
depend on any kind of subjective interpretation of the learned representations
— it just tells us whether whatever similarity structure was learned differs
across the layers. Starting from either V1 compared to all higher layers, or
the highest TE layer compared to all lower layers, we found a consistent
pattern of progressive emergence of the object categorization structure in the
upper IT pathway (TEO, TE).

This analysis confirms that indeed the IT category structure is significantly
different from that present at the level of the V1 primary visual input. Thus
the model, despite being trained only to generate accurate visual input-level
predictions, has learned to represent these objects in an abstract way that
goes beyond the raw input-level information. We further verified that at the
highest IT levels in the model, a consistent, spatially-invariant
representation is present across different views of the same object (e.g., the
average correlation across frames within an object was .901).

Figure 7: a) Category similarity structure that emerged in the highest layer,
TE, of the biologically based predictive learning model, showing
_dissimilarity_ (1-correlation) of the TE representation for each 3D object
against every other 3D object (156 total objects). Blue cells have high
similarity. Model has learned block-diagonal clusters or categories of high-
similarity groupings, contrasted against dissimilar off-diagonal other
categories. Clustering maximized average _within – between_ dissimilarity (see
Appendix), and clearly corresponded to the shown shape-based categories, with
exemplars from each category shown. Also, all items from the same basic-level
object categories (N=20) are reliably subsumed within learned categories. b)
Human similarity ratings for the same 3D objects, presented with the V1
reconstruction (see Fig 1b) to capture coarse perception in the model,
aggregated by 20 basic-level categories (156 x 156 matrix was too large to
sample densely experimentally). Each cell is 1 - proportion of time given
object pair was rated more similar than another pair (see Appendix). The human
matrix shares the same centroid categorical structure as the model (confirmed
by permutation testing and agglomorative cluster analysis, see Appendix),
indicating that human raters used the same shape-based category structure. c)
One object from each of the 20 basic level categories, organized into the
shape-based categories. The Vertical, Box and Horizontal categories are fairly
self-evident and the model was most consistent in distinguishing those, along
with subsets of the Pyramid (layer-cake, traffic-cone, sailbot) and Round
(donut, doorknob) categories, while banana, trex, chair, and handgun were more
variable.

To better understand the nature of these learned representations, Figure 7
shows a representational similarity analysis (RSA) on the activity patterns at
each layer in the model, which reveals the explicit categorical structure of
the learned representations (Kriegeskorte ., 2008; Cadieu ., 2014). As shown
in Figure 7a, we found that the highest IT layer (TE) produced a systematic
organization of the 156 3D objects into 5 categories. In our admittedly
subjective judgment, these categories seemed to correspond to the overall
shape of the objects, as shown by the object exemplars in the figure (pyramid-
shaped, vertically-elongated, round, boxy / square, and horizontally-
elongated). Furthermore, the basic-level categories were subsumed within these
broader shape-level categories, so the model appears to be sensitive to the
coherence of these basic-level categories as well, but apparently their shapes
were not sufficiently distinct between categories to drive differentiated TE-
level representations for each such basic-level category.

Given that the model only learns from a passive visual experience of the
objects, it has no access to any of the richer interactive multi-modal
information that people and animals would have. Furthermore, as evident in
Figure 5b, the relatively low resolution of the V1 layers (required to make
the model tractable computationally) means that complex visual details are not
reliably encoded (and even so, are not generally reliable across object
exemplars), such that the overall object shape is the most salient and
sensible basis for categorization for this model.

Although these object shape categories appeared sensible to us, we ran a
simple experiment to test whether a sample of 30 human participants would use
the same category structure in evaluating the pairwise similarity of these
objects. Figure 7b shows the results, confirming that indeed this same
organization of the objects emerged in their similarity judgments. These
judgments were based on the V1 reconstruction as shown in Figure 5b to capture
the model’s coarse-grained perception; see Appendix for methods and further
analysis.

Figure 8: Comparison of progression from V4 to IT in macaque monkey visual
cortex (top row, from Cadieu et al., 2014) versus same progression in model
(replotted using comparable color scale). Although the underlying categories
are different, and the monkeys have a much richer multi-modal experience of
the world to reinforce categories such as foods and faces, the model
nevertheless shows a similar qualitative progression of stronger categorical
structure in IT, where the block-diagonal highly similar representations are
more consistent across categories, and the off-diagonal differences are
stronger and more consistent as well (i.e., categories are also more clearly
differentiated). Note that the critical difference in our model versus those
compared in Cadieu et al. 2014 and related papers is that they explicitly
trained their models on category labels, whereas our model is _entirely self-
organizing_ and has no external categorical training signal.

The progressive emergence of increasingly abstract category structure across
visual areas, evident in Figure 6, has been investigated in recent comparisons
between monkey electrophysiological recordings and deep convolutional neural
networks (DCNNs), which provide a reasonably good fit the the overall
progressive pattern of increasingly categorical organization (Cadieu ., 2014).
However, these DCNNs were trained on large datasets of human-labeled object
categories, and it is perhaps not too surprising that the higher layers closer
to these category output labels exhibited a greater degree of categorical
organization. In contrast, because the only source of learning in our model
comes from prediction errors over the V1 input layers, the graded emergence of
an object hierarchy here reflects a truly self-organizing learning process.

Figure 8 compares the similarity structures in layers V4 and IT in macaque
monkeys (Cadieu ., 2014) with those in corresponding layers in our model. In
both the monkeys and our model, the higher IT layer builds upon and clarifies
the noisier structure that is emerging in the earlier V4 layer, showing that
our model replicates the essential qualitative hierarchical progression in the
brain. As noted, we would not expect our model to exactly replicate the
detailed object-specific similarity structure found in macaques, due to the
impoverished nature of our model’s experience, so this comparison remains
qualitative in terms of the respective differences between V4 and IT in each
model, rather than a direct comparison of the similarity structure between
corresponding layers in the model and the macaque. In the future, when we can
scale up our model and tune the attentional processing dynamics necessary to
deal with cluttered visual scenes, we will be able to train our model on the
same images presented to the macaques, and can provide this more direct
comparison.

Finally, we did not use analyses based on decoding techniques, because with
high-dimensional distributed neural representations, it is generally possible
to decode many different features that are not otherwise compactly and
directly represented (Fusi ., 2016). In preliminary work using decoding in the
context of the simpler feature-based input patterns, we indeed found that
decoding was not a very sensitive measure of the differentiation of
representations across layers, which is so clearly evident in Figure 6. Thus,
as advocates of the RSA approach have argued, measuring similarity structure
evident in the activity patterns over a given layer generally provides a
clearer picture of what that layer is explicitly encoding (Kriegeskorte .,
2008).

In summary, the model learned an abstract category organization that reflects
the overall visual shapes of the objects as judged by human participants, in a
way that is invariant to the differences in motion, rotation, and scaling that
are present in the V1 visual inputs. We are not aware of any other model that
has accomplished this signature computation of the ventral _What_ pathway in a
purely self-organizing manner operating on realistic 3D visual objects,
without any explicit supervised category labels. Furthermore, our model does
this using a learning algorithm directly based on detailed properties of the
underlying biological circuits in this pathway, providing a coherent overall
account.

###  3.1 Backpropagation Comparison Models

Figure 9: Similarity of similarity structure across layers for the comparison
backprop models, comparing each layer to the first layer. a) Backpropagation
(Bp) model with the same What / Where structure as the biological model.
Unlike the biologically based model (Figure 6) the higher IT layers (TE, TEO)
do not diverge significantly from the similarity structure present in V1,
indicating that the model has not developed abstractions beyond the structure
present in the visual input. Layer V3 is most directly influenced by spatial
prediction errors, so it differs from both in strongly encoding position
information. b) PredNet model, which has 6 layers. Layers 2-6 diverge from
layer 1, but there is no progressive change in the higher layers as we see in
our model moving from V4 to TEO. The divergence in correlation starting at
layer 2 is likely due to the fact that higher layers only encode errors, not
stimulus-driven positive representations of the input. Aside from this large
distinction (which is inconsistent with the similarity in neural coding seen
in actual V1 and V2 recordings), there is no evidence of a cumulative
development of abstraction in higher layers. Figure 10: a) Best-fitting
category similarity for TE layer of the backpropagation (Bp) model with the
same What / Where structure as the biological model. Only two broad categories
are evident, and the lower _max_ distance (0.3 vs. 1.5 in biological model)
means that the patterns are much less differentiated overall. b) Best-fitting
similarity structure for the PredNet model, in the highest of its layers
(layer 6), which is more differentiated than Bp (max = 0.75) but also less
cleanly similar within categories (i.e., less solidly blue along the block
diagonal), and overall follows a broad category structure similar to V1. c)
The best fitting V1 structure, which has 2 broad categories and banana is in a
third category by itself. The lack of dark blue on the block diagonal
indicates that these categories are relatively weak, and every item is fairly
dissimilar from every other. d) The Bp TE similarity values from panel a shown
in the same ordering as V1 from panel c, demonstrating how the similarity
structure has not diverged very much, consistent with the results shown in
Figure 9 — the within – between contrast differences are 0.0838 for panel a
and 0.0513 for d — see Appendix for details.

To help discern some of the factors that contribute to the categorical
learning in our model, and provide a comparison with more widely-used error
backpropagation models, we tested a backpropagation-based (Bp) version of the
same _What_ vs. _Where_ architecture as our biologically based predictive
error model, and we also tested a standard _PredNet_ model (Lotter ., 2016)
with extensive hyperparameter optimization (see Appendix). Due to the
constraints of backpropagation, we had to eliminate any bidirectional
connectivity loops in the Bp version, but we were able to retain a form of
predictive learning by configuring the V1p pulvinar layer as the final target
output layer, with the target being the next visual input relative to the
current V1 inputs.

Figure 9 shows the same second-order similarity analysis as Figure 6, to
determine the extent to which these comparison networks also developed more
abstract representations in the higher layers that diverge from the similarity
structure present in the lowest layers. According to this simple objective
analysis, they did not — the higher layers showed no significant, progressive
divergence in their similarity structure. The PredNet model did show a larger
difference between the first layer and the rest of the layers, due to the
subsequent layers encoding errors while the first layer has a positive
representation of the image, but there was no progressive difference beyond
that up into the higher layers.

Next, we examined the RSA matrices for the highest (TE) layer in the
comparison models, also in comparison with the same for the V1 layer (Figure
10). This shows that the TE layer in the Bp model formed a simple binary
category structure overall, which is similar to the RSA for the V1 input
layer. It is also important to emphasize that the scales on these figures are
different (as shown in their headers), such that these comparison models had
much less differentiated representations overall. Similar results were found
in the PredNet model. Because existing work with these models has typically
relied on additional supervised learning and decoder-based analyses (which are
essentially equivalent to an additional layer of supervised learning), these
RSA-based analyses provide an important, more sensitive way of determining
what they learn purely through predictive learning.

These results show that the additional biologically derived properties in our
model are playing a critical role in the development of abstract categorical
representations that go beyond the raw visual inputs. These properties
include: excitatory bidirectional connections, inhibitory competition, and an
additional Hebbian form of learning that serves as a regularizer (similar to
weight decay) on top of predictive error-driven learning (O’Reilly, 1998;
O’Reilly Munakata, 2000). Each of these properties could promote the formation
of categorical representations. Bidirectional connections enable top-down
signals to consistently shape lower-level representations, creating
significant attractor dynamics that cause the entire network to settle into
discrete categorical attractor states. Another indication of the importance of
bidirectional connections is that a greedy layer-wise pretraining scheme,
consistent with a putative developmental cascade of learning from the sensory
periphery on up (Shrager Johnson, 1996; Bengio ., 2013; Valpola, 2014; Hinton
Salakhutdinov, 2006), did not work in our model. Instead, we found it
essential that higher layers, with their ability to form more abstract,
invariant representations, interact and shape learning in lower layers right
from the beginning.

Furthermore, the recurrent connections within the TEO and TE layers likely
play an important role by biasing the temporal dynamics toward longer
persistence (Chaudhuri ., 2015). By contrast, backpropagation networks
typically lack these kinds of attractor dynamics, and this could contribute
significantly to their relative lack of categorical learning. Hebbian learning
drives the formation of representations that encode the principal components
of activity correlations over time, which can help more categorical
representations coalesce (and results below already indicate its importance).
Inhibition, especially in combination with Hebbian learning, drives
representations to specialize on more specific subsets of the space.

Ongoing work is attempting to determine which of these is essential in this
case (perhaps all of them) by systematically introducing some of these
properties into the backpropagation model, though this is difficult because
full bidirectional recurrent activity propagation, which is essential for
conveying error signals top-down in the biological network, is incompatible
with the standard efficient form of error backpropagation, and requires
significantly more computationally intensive and unstable forms of fully
recurrent backpropagation (Williams Zipser, 1992; Pineda, 1987). Furthermore,
Hebbian learning requires dynamic inhibitory competition which is difficult to
incorporate within the backpropagation framework.

###  3.2 Architecture and Parameter Manipulations

Figure 11: Effects of various manipulations on the extent to which TE
representations differentiate from V1. For all plots, _Intact_ is the same
result shown in Figure 6 from the intact model for ease of comparison (panel a
is missing V3 and DP dorsal pathway layers). All of the following
manipulations significantly impair the development of abstract TE categorical
representations (i.e., TE is more similar to V1 and the other layers). a)
Dorsal _Where_ pathway lesions, including lateral inferior parietal sulcus
(LIP), V3, and dorsal prelunate (DP). This pathway is essential for regressing
out location-based prediction errors, so that the residual errors concentrate
feature-encoding errors that train the _What_ pathway. b) Allowing the deep
layers full access to current-time information, thus effectively eliminating
the prediction demand and turning the network into an auto-encoder, which
significantly impairs representation development, and supports the importance
of the challenge of predictive learning for developing deeper, more abstract
representations. c) Reducing the strength of Hebbian learning by 20% (from 2.5
to 2), demonstrating the essential role played by this form of learning on
shaping categorical representations. Eliminating Hebbian learning entirely
(not shown) prevented the model from learning anything at all, as it also
plays a critical regularization and shaping role on learning.

Figure 11 shows just a few of the large number of parameter manipulations that
have been conducted to develop and test the final architecture. For example,
we hypothesized that separating the overall prediction problem between a
spatial _Where_ vs. non-spatial _What_ pathway (Ungerleider Mishkin, 1982;
Goodale Milner, 1992), would strongly benefit the formation of more abstract,
categorical object representations in the _What_ pathway. Specifically, the
_Where_ pathway can learn relatively quickly to predict the overall spatial
trajectory of the object (and anticipate the effects of saccades), and thus
effectively regress out that component of the overall prediction error,
leaving the residual error concentrated in object feature information, which
can train the ventral _What_ pathway to develop abstract visual categories.
Figure 11a shows that, indeed, when the _Where_ pathway is lesioned, the
formation of abstract categorical representations in the intact _What_ pathway
is significantly impaired. We also hypothesized that full predictive learning
(about the future), as compared to just encoding and decoding the current
state (i.e., an auto-encoder, which is much easier computationally), is also
critical for the formation of abstract categorical representations —
prediction is a “desirable difficulty” (Bjork, 1994). Figure 11b shows that
this was the case. Finally, consistent with our hypothesis that Hebbian
learning provides an important bias on learning, Figure 11c shows the
impairment associated with reducing this learning bias.

###  3.3 Predictive Behavior

Figure 12: Predictive Remapping. top: Original remapping data in LIP from
Duhamel et al (1992). A) shows stimulus (star) response within receptive field
(dashed circle) relative to fixation dot (upper right of fixation). B) Just
prior to monkey making a saccade to new fixation (moving left), stimulus is
turned on in receptive field location that _will be_ upper right of the new
fixation point, and the LIP neuron responds to that stimulus in advance of the
saccade completing. The neuron does not respond to the stimulus in that
location if it is not about to make a saccade that puts it within its
receptive field (not shown). This is predictive remapping. C) response to the
old stimulus location goes away as saccade is initiated. bottom: Data from our
model, from individual units in LIPd, V2d, and V2s, showing that the LIP deep
neurons respond to the saccade first, activating in the new location and
deactivating in the old, and this LIP activation goes top-down to V3 and V2 to
drive updating there, generally at a longer latency and with less activation
especially in the superficial layers. When the new stimulus appears at the
point of fixation (after a 50 ms saccade here), the _primed_ V2s units get
fully activated by the incoming stimulus. But the deep neurons are insulated
from this superficial input until the plus phase, when the cascade of 5IB
firing drives activation of the actual stimulus location into the pulvinar,
which then reflects up into all the other layers.

A signature example of predictive behavior at the neural level in the brain is
the _predictive remapping_ of visual space in anticipation of a saccadic eye
movements (Duhamel ., 1992; Colby ., 1997; Gottlieb ., 1998; Nakamura Colby,
2002; Marino Mazer, 2016) (Figure 12a). Here, parietal neurons start to fire
at the _future_ receptive field location where a currently-visible stimulus
will appear after a planned saccade is actually executed. Remapping has also
been shown for border ownership neurons in V2 (O’Herron von der Heydt, 2013)
and in area V4 (Neupane ., 2016, 2020). These are examples, we believe, of a
predictive process operating throughout the neocortex to predict what will be
experienced next. A major consequence of this predictive process is the
perception of a stable, coherent visual world despite constant saccades and
other sources of visual change.

Figure 12b shows that our model exhibits this predictive remapping phenomenon.
Specifically, LIP, which is most directly interconnected with the saccade
efferent copy signals, is the first to predict the new location, and it then
drives top-down activation of lower layers. This top-down dynamic is
consistent with the account of predictive remapping given by Wurtz (2008) and
Cavanagh . (2010), who argue that the key remapping takes place at the high
levels of the dorsal stream, which then drive top-down activation of the
predicted location in lower areas, instead of the alternative where lower-
levels remap themselves based on saccade-related signals. The lower-level
visual layers are simply too large and distributed to be able to remap across
the relevant degrees of visual angle — the extensive lateral connectivity
needed to communicate across these areas would be prohibitive.

##  4 Neural Data and Predictions

Having tested the computational and functional learning properties of this
biologically based predictive learning mechanism, we now return to consider
some of the most important neural data of relevance to our hypotheses, beyond
that summarized in the introduction, including contrasts with a widely-
discussed alternative framework for predictive coding, and some of the
extensive data on alpha frequency effects, followed by a discussion of
predictions that would clearly test the validity of this framework.

###  4.1 Additional Neuroscience Data

We begin with data relevant to the basic neural-level properties of the
framework. First, a central element of the proposed model is the alpha cycle
bursting, and subsequent inter-burst pauses, in the 5IB neurons. Direct
electrophysiological recording of deep layer neurons shows periodic alpha-
scale bursting for continuous tones _in awake animals_ (Luczak ., 2009, 2013;
Sakata Harris, 2009, 2012). _In vitro_ , a variety of potential mechanisms
behind the generation and synchronization of the 5IB bursts driving this alpha
cycle have been identified (Connors ., 1982; Silva ., 1991; Franceschetti .,
1995). Furthermore, the pulvinar has been shown to drive alpha-frequency
synchronization of cortical activity across areas in the alpha band in awake
behaving animals (Saalmann ., 2012). We review the larger alpha frequency
literature in more detail below, but it is critical to emphasize that this
alpha bursting dynamic is actually found in awake, behaving animals, because
so many other bursting and up / down state phenomena have recently been shown
to only occur in anesthetized brains, including bursting in the thalamic TRC
neurons.

In contrast to the 5IB bursting, the 6CT neurons exhibit regular spiking
behavior, (Thomson, 2010; Thomson Lamy, 2007), providing consistent activation
to the pulvinar. Also, they do not have axonal branches that project to other
cortical areas — the subpopulation that projects to the pulvinar only project
there and not to other cortical areas (Petrof ., 2012), whereas there are
other layer 6 neurons that do project to other cortical areas. This distinct
connectivity is consistent with a specific role of this neuron type in
generating predictions in the pulvinar. The 6CT synaptic inputs on pulvinar
TRCs have metabatropic glutamate receptors (mGluR) that have longer time-scale
temporal dynamics consistent with the alpha period (100 ms) and even longer
(SM. Sherman, 2014), and the 6CT neurons themselves also have temporally-
delayed responding (Harris Shepherd, 2015; Sakata Harris, 2009; Thomson,
2010). Furthermore, they have significantly more plasticity-inducing NMDA
receptors compared to the 5IB projections (Usrey Sherman, 2018). These
properties are consistent with the 6CT inputs driving a longer-integrated
prediction signal that is subject to learning, whereas the 5IB are likely non-
plastic and their effects are tightly localized in time.

The 5IB inputs often have distinctive _glomeruli_ structures at their synapses
onto pulvinar neurons, which contain a complete feedforward inhibition circuit
involving a local inhibitory interneuron, in addition to the direct strong
excitatory driver input (Wilson ., 1984). Computationally, this can provide a
balanced level of excitatory and inhibitory drive so as to not overly excite
the receiving neuron, while still dominating its firing behavior.

Although there are well-documented and widely-discussed burst vs. tonic firing
modes in pulvinar neurons (SM. Sherman Guillery, 2006), there is not much
evidence of these playing a clear role in the awake, behaving state, and as
noted earlier the growing electrophysiological evidence shows a remarkable
correspondence between cortical and pulvinar response properties across
multiple different pulvinar areas in this awake state. Nevertheless, there may
be important dynamics arising from these firing modes that are more subtle or
emerge in particular types of state transitions that may have yet to be
identified.

###  4.2 Contrast with Explict Error (EE) Frameworks

Figure 13: Comparison between: a) The proposed thalamocortical temporal-
difference predictive learning model (from Figure 2), versus b) The Bayesian-
style explicit error (EE) coding model (Rao & Ballard, 1999; Friston, 2010,
Bastos et al., 2012). The EE model holds that superficial (S, lamina 2/3)
error-coding neurons receive the prediction via a net inhibitory top-down
projection from higher-level deep layer (D) neurons, and an excitatory bottom-
up projection representing the outcome, such that their activation represents
the difference. To encode both signs of the error (omissions, false alarms)
with positive-only spike rates, two separate populations of EE neurons would
be required, or a more complicated deviation from tonic firing level scheme.
Unambiguous evidence of such EE coding neurons has not been found (Walsh et
al, 2020). In contrast, error signals in our proposed framework remain as a
temporal difference between the two states of prediction vs. outcome, _which
enables all connectivity between cortical areas to be excitatory and always
represent a positive encoding of either the prediction or outcome_. In
contrast, under EE, after one error subtraction at the lowest level, only
error signals are hypothesized to flow forward to higher layers, meaning that
the representations at higher layers are about increasingly higher-order
_errors_ , not positive encodings of the environmental state at increasing
levels of abstraction. These are indicated by ? because they are difficult to
picture intuitively, and they are inconsistent with extensive available data
showing similar positive representations of the external world at all levels
in the visual hierarchy. Although some frameworks make claims about temporal
dynamics, these are not strongly constrained by the basic computational
framework, so that also remains a question.

To further clarify the nature of the present theory, and introduce a body of
relevant data, we contrast it with the widely-discussed explicit error (_EE_)
framework for predictive coding (Rao Ballard, 1999; Kawato ., 1993; Friston,
2005, 2010; Ouden ., 2012; Bastos ., 2012; Lotter ., 2016) (Figure 13). The
hypothesized locus for computing errors in this framework is in the
superficial layers of the neocortex, which are suggested to directly compute
the difference between bottom-up inputs from lower layers and top-down inputs
from higher areas. Despite many attempts to identify such explicit error-
coding neurons in the cortex, no substantial body of unambiguous evidence has
been discovered (Kok de Lange, 2015; Kok ., 2012; Summerfield Egner, 2009; Lee
Mumford, 2003; Walsh ., 2020). Furthermore, due to the positive-only firing
rate nature of neural coding, two separate populations would be required to
convey both signs of prediction error signals, or it would have to be encoded
as a variation from tonic firing levels, which are generally low in the
neocortex.

By contrast, the use of temporal-difference error signals enables all
connections between cortical layers to be excitatory and each layer can
represent the positive encoding of either the prediction or outcome state, at
different levels of abstraction. These properties are overwhelmingly supported
by extensive electrophysiological data about the hierarchical organization of
representations, e.g., in the visual object recognition pathway (Kobatake
Tanaka, 1994; VanRullen Thorpe, 2002; Cadieu ., 2014), and are consistent with
the widely-supported biased competition model for excitatory top-down
attentional effects (Desimone Duncan, 1995; Reynolds ., 1999; EK. Miller
Cohen, 2001; O’Reilly ., 2013).

The EE approach requires net inhibitory top-down predictions, and it sends
error signals forward, not positive representations of the actual state at a
given level of abstraction. Thus a literal interpretation (and at least one
existing implementation; Lotter ., 2016) has only error signals represented at
all levels above the lowest level, which is inconsistent with the positive
encoding of stimuli at various levels of abstraction across the visual
hierarchy. For example, although Issa . (2018) observed an error-signal-like
increase in activation for atypical faces in some pIT neurons, these neurons
overall had a positive stimulus encoding, with only a relatively small, later,
error-like modulation.

Furthermore, as discussed below, anticipatory predictions typically closely
resemble the subsequent stimulus-driven activity, suggesting a positive, not
inhibitory, effect (Duhamel ., 1992; Lee Mumford, 2003; Cavanagh ., 2010;
Walsh ., 2020). However, there are various different ways of reformulating the
neural implementation of EE that can avoid some of these issues (Spratling,
2008; Bastos ., 2012), but perhaps this flexibility renders the framework
difficult to falsify (Kogo Trengove, 2015). In any case, an extensive
treatment of the issues with EE is beyond the scope of this paper and has
already been aptly covered by Walsh . (2020) — our goal here is to highlight
some of the core differences as a way to clarify the framework by way of
contrast, and in relation to available data.

First, there are many examples of anticipatory predictive neural firing in the
brain. Of perhaps greatest relevance, Barczak . (2018) recently showed that
the auditory pulvinar in monkeys exhibits predictive firing using a carefully
controlled auditory sequence that had no first-order acoustic differences from
a background noise signal. The pulvinar predictive activation preceded that of
A1, suggesting a strong predictive role for pulvinar. Unfortunately, the deep
layers of higher auditory areas that should contribute to the formation of the
pulvinar prediction were not recorded in this study, so their role in
generating the prediction could not be determined.

Nevertheless, there is extensive additional evidence for top-down anticipatory
activation of predicted stimuli, with activity patterns closely resembling the
subsequent stimulus-driven ones (Walsh ., 2020). For example, the widely
replicated predictive remapping effect, simulated in our model (Figure 12) is
of this nature (Duhamel ., 1992; Wurtz, 2008; Cavanagh ., 2010). The fact that
these anticipatory activations are of a positive nature, consistent with the
stimulus-driven activations, is inconsistent with the expected behavior of EE
neurons, which should be inhibited by the top-down prediction, while not
receiving any bottom-up stimulus.

However, the neural response to the actual predicted stimulus itself is
typically suppressed relative to unexpected stimuli, i.e., _expectation
suppression_ (Summerfield ., 2008; Todorovic ., 2011; Meyer Olson, 2011;
Bastos ., 2012). This phenomenon is widely cited as evidence in favor of the
EE predictive coding framework, consistent with an inhibitory effect of the
expectation. Nevertheless, despite various conflicting results and many
complications of interpretation, multiple comprehensive reviews conclude that
it is difficult to distinguish expectation suppression from the neural
adaptation effects that underlie the well-documented _repetition suppression_
effect (Walsh ., 2020; Vinken Vogels, 2017; Kok de Lange, 2015; Kok ., 2012;
Summerfield Egner, 2009; Lee Mumford, 2003). Furthermore, detailed single-
neuron level recordings are the least likely to show these effects — instead,
they are most evident in aggregate signals such as the BOLD response in fMRI,
suggesting that they may more strongly reflect population-level differences in
activity, rather than individual explicit error coding neurons.

As noted earlier, accurately predicted outcomes in our framework would result
in a continued adaptation of the neural response carrying over from the
prediction to the outcome state, whereas unexpected outcomes would be
associated with two distinct patterns of activity over a given area: first the
prediction and then the outcome. Thus, the unexpected outcome state would not
be subject to the prior neural adaptation effects, and furthermore the time-
integrated aggregate activity over these two patterns would be greater
compared to the single activity state associated with an accurately predicted
outcome. Thus, our model explains expectation suppression without invoking EE
neurons, meaning that considerably more detailed and replicable experimental
paradigms using single-neuron resolution techniques are needed to distinguish
EE from our framework.

###  4.3 Alpha Frequency Effects

The alpha frequency bursting of 5IB neurons acting as drivers into the
pulvinar naturally entrains the predictive learning process in our model to
this fundamental rhythm, which has long been recognized as an important
signature of posterior cortical function (Berger, 1929; Walter, 1953; Nunn
Osselton, 1974; Varela ., 1981; VanRullen Koch, 2003). A number of different
functional associations with alpha have been established, and this literature
is large and growing rapidly. Thus, we refer the reader to recent reviews
(Jensen ., 2015; VanRullen, 2016; Clayton ., 2018; Foster Awh, 2019) while
highlighting the data most relevant to our specific framework here, organized
according to a set of key points.

  * •

_Alpha is specifically associated with deep neocortical layers and the
pulvinar, and with feedback pathways in the cortex._ This has been established
using direct laminar-specific electrophysiological single-neuron and local
field potential (LFP) recordings (Luczak ., 2013; Buffalo ., 2011; Maier .,
2010, 2011; Spaak ., 2012; Xing ., 2012), and feedforward vs. feedback
manipulations (von Stein ., 2000; van Kerkoerle ., 2014; Bastos ., 2015;
Jensen ., 2015; Michalareas ., 2016). These data are consistent with the 5IB
alpha bursting and the major role of cortical deep layers in driving top-down
corticocortical projections (in addition to the 6CT pathway which is specific
to the pulvinar). By contrast, these same papers show that superficial
cortical layers are associated with gamma frequency (40 Hz) dynamics. However,
the next point raises some important interpretational difficulties.

  * •

_Increases in cortical activity levels, e.g., due to attention, produce a
corresponding decrease in alpha power, while decreased activity increases
alpha power_ (Worden ., 2000; Kelly ., 2006; Klimesch ., 2007; Fries ., 2008;
Jensen Mazaheri, 2010; Foster Awh, 2019). This pattern is not exactly what you
might expect if alpha was a signature of predictive learning. However, given
that these same pulvinar and thalamocortical pathways are also widely regarded
as important for attention (LaBerge Buchsbaum, 1990; Bender Youakim, 2001;
Snow ., 2009; Saalmann Kastner, 2011; Zhou ., 2016; Fiebelkorn Kastner, 2019),
this pattern presents a challenge for many theorists. However, it is possible
to explain this pattern as arising directly from the desynchronizing effects
of cortical activity on alpha power. Specifically, neural spiking is
associated with broadband noise, due to the highly random, Poisson nature of
spike firing, which can desynchronize the entrainment of lower-frequency
oscillations including alpha (Waldert ., 2013; Ray Maunsell, 2011; Privman .,
2013; Solomon ., 2017). In other words, because cortical activity is
inherently noisy, it tends to interfere with the coherent activity across
populations of neurons needed to produce a strong alpha frequency power
signal. This explanation is directly supported by studies manipulating and
measuring cortical activity (Zhou ., 2016; Fries ., 2008), and is consistent
with alpha power changes being a _result_ of attentional modulation, but not
their cause (Antonov ., 2020). Thus, while attention and predictive learning
can both affect overall activity levels in cortex, and thus drive changes in
alpha power, alpha power itself is not a transparent measure of the underlying
mechanisms supporting these functions, which may help to explain some
contradictory patterns of results (Foster Awh, 2019; Gundlach ., 2020; Keitel
., 2019).

  * •

_Alpha phase effects provide a more direct measure of thalamocortical function
than alpha power, and have been more consistently related to perception,
attention, and prediction_ (Nunn Osselton, 1974; Varela ., 1981; VanRullen
Koch, 2003; Busch ., 2009; KE. Mathewson ., 2010; Palva Palva, 2011; Jaegle
Ro, 2013; Neupane ., 2017; Solís-Vivanco ., 2018). For example, weak, near-
threshold stimuli are more reliably detected and processed when presented in
the trough of the individual’s ongoing alpha cycle. Of greatest relevance to
the present paper are studies showing effects of prediction on alpha phase
(Samaha ., 2015; Mayer ., 2016; MT. Sherman ., 2016). For example, Mayer .
(2016) showed that prestimulus alpha phase directly correlated with the
predictability of the upcoming stimulus, and the pattern of this prestimulus
activation was indistinguishable from the subsequent stimulus activation
pattern. This is consistent with our model, and less consistent with the EE
framework, as discussed previously. Neupane . (2017) found strong alpha
coherence effects in LFP recordings distributed across V4, associated with the
predictive remapping of receptive fields (Duhamel ., 1992).

  * •

_Discrete, salient, or oscillatory stimuli entrain the alpha cycle in the
brain_ (Spaak ., 2014; KE. Mathewson ., 2012). Furthermore, the massive
literature on _event related potentials_ (ERPs) may represent a significant
contribution from alpha-level entrainment (Makeig ., 2002; Gruber ., 2005;
Klimesch, 2011). These entrainment effects are consistent with the 5IB
entrainment mechanisms in our framework, as described earlier, and entrainment
is functionally important for aligning predictive learning with relevant
salient or unexpected outcomes.

  * •

_The pulvinar contributes to synchronizing alpha phase relationships across
different brain areas_ (Saalmann ., 2012; Fiebelkorn ., 2018). This is
consistent with the broad, convergent pattern of projections into the pulvinar
from many different cortical areas, and the corresponding broad projections
back out to these same areas (Shipp, 2003; Arcaro ., 2015). Functionally, this
convergence and synchronization is important for integrating the contributions
from these different areas at the same time, to generate predictions over the
pulvinar.

  * •

_The theta cycle, comprised of a pair of alpha cycles, organizes saccades, and
attentional, motor, and mnemonic processes_ (Fiebelkorn Kastner, 2019). The
theta rhythm is dominant in the medial temporal lobe and hippocampus, and has
been extensively studied there (Kahana ., 2001; Buzsáki, 2005). Furthermore,
there is a sharp peak of saccade fixation durations at 200 ms, which suggests
that two alpha cycles are typically required for complete processing of a
given fixation. On the first cycle, the predictions from before the eye moved
may be fairly vague depending on factors such as the size of the saccade and
familiarity with the environment. But after the first alpha cycle of a
fixation, a subsequent _postdiction_ phase can provide an important additional
learning opportunity, to consolidate and more deeply encode the current
fixation (computationally equivalent to an auto-encoder). Also, a mix of
smaller saccades (including microsaccades) and larger saccades enables a range
of more and less predictable outcomes on the first alpha cycle after the
saccade, and matches human behavior (Martinez-Conde ., 2013, 2004).

Putting all of these points together, a particularly effective way of testing
the predictions of our framework would be measuring alpha phase changes
emerging in the prestimulus period as a function of predictive learning in
predictable sequential stimulus streams. In addition, it would also be
important to examine theta and alpha-cycle dynamics in relation to predictive
learning in the context of attention, motor control, and memory processes, to
better understand the larger systems-level temporal organization of learning
and processing in the brain (Fiebelkorn Kastner, 2019).

###  4.4 Predictions for Predictive Learning

In this section, we enumerate a set of direct, testable predictions from our
framework. Before doing so, there are several important considerations for any
experimental test of the theory. First, the nature of what is to be learned
must be matched to the pulvinar area in question. For example, learning a new
variation of basic physics in movies at the alpha time scale (e.g., altering
properties such as gravity, inertia, or elasticity), would be appropriate for
the lower level visual pathways. At higher visual levels (e.g., IT cortex), it
might be possible to use simple sequences of different objects, although it is
not clear to what extent the hippocampus or prefrontal cortex might also
contribute in this case (Gavornik Bear, 2014; Fiser ., 2016). To distinguish
pulvinar learning effects from pervasive motor learning supported by other
brain areas, it would be most effective to directly measure activity in the
pulvinar and / or associated perceptual neocortical areas, instead of
involving overt behavioral performance.

Much of the learning in posterior sensory cortex should take place early in
development, requiring very early developmental interventions or genetic
knockouts that are expressed from the start (which can also have other
interpretational issues if not highly selective). In our models, the bulk of
the basic sensory predictive learning happens very quickly, because the basic
first-level regularities are quite strong and relatively easily learned. While
there are longer-term changes in the higher-level pathways in our models, more
fine-grained measurements would likely be required to see these changes. Once
this learning has taken place, the remaining contributions of the
thalamocortical circuit are likely more strongly weighted toward its role in
attention, as we discuss below. Finally, directly lesioning or inactivating
the pulvinar is not likely to be very informative, because existing work has
shown dramatic effects on cortical activity (Zhou ., 2016; Purushothaman .,
2012), and also any effects could be attributed to the attentional
contributions of the pulvinar.

With these considerations in mind, here are a set of strong predictions from
our model that should be testable using existing techniques. Failure to obtain
the predicted result, while adhering to all the relevant constraints, would
constitute a falsification of our model.

  * •

_Blocking 5IB bursting mechanisms early in developmental learning should
disrupt learning_. It should be possible to selectively knock out or modify
the channels that cause this specific population of neurons to burst fire, and
doing so should have a significant effect on learning in associated
neocortical and pulvinar areas, given the critical role that this burst firing
plays on the predictive learning process as elaborated above.

  * •

_Blocking synaptic plasticity in pulvinar (specifically the 6CT inputs) very
early in developmental learning should impair learning_. While most of the
learning overall should occur in the neocortex as a result of the temporal
difference error signal broadcast by the pulvinar (which should remain
generally intact), learning in the 6CT projections is important, especially
right at the start, to map the emerging neocortical representations into the
space defined by the 5IB projections.

  * •

_Temporal differences on an alpha cycle timescale actually drive synaptic
plasticity in an error-driven learning manner, in neocortical pyramidal
neurons and in 6CT inputs to pulvinar_. That is, if a pre / post pair of
neurons across a synapse is more active in the prediction than the subsequent
outcome, the synapse should experience LTD (long term depression), and vice-
versa if the activity pattern is reversed (long term potentiation, LTP, for
more activity in outcome than prediction). Furthermore, if activity is
essentially stable across both prediction and outcome phases, then weights
should not change (modulo a small level of Hebbian learning; O’Reilly
Munakata, 2000; O’Reilly ., 2012). This should be directly testable using
current experimental methods, and is perhaps the single most important
empirical test of this entire framework, and it also underlies many other
current approaches to error-driven learning in the brain (Bengio ., 2017;
Whittington Bogacz, 2019; Lillicrap ., 2020). One general consideration is the
extent to which an awake _in vivo_ preparation would be required to capture
all the neuromodulatory and other factors present when this learning normally
takes place. Some suggestive evidence in such a preparation is generally
consistent with a sensitivity to relatively short-term temporal dynamics (Lim
., 2015), although these results lacked the direct measurement of individual
neural activity across a synapse.

##  5 Discussion

We have hypothesized a novel computational function for the distinctive
features of thalamocortical circuits (SM. Sherman Guillery, 2006; Usrey
Sherman, 2018), as supporting a specific form of prediction-error driven
learning, where predictions arise from the numerous top-down layer 6CT
projections into the pulvinar, and the strong, sparse, focal driving 5IB
inputs supply the bottom-up sensory-driven outcome. The phasic bursting nature
of the 5IB inputs results in a natural temporal-difference error signal of
prediction followed by outcome, consistent with extensive neural recording
data. This temporal dynamic is also essential for enabling predictions to be
generated without contamination from current sensory inputs, and predicts a
characteristic alpha frequency prediction cycle based on the 10hz bursting
cycle of the 5IB inputs, consistent with the pervasive influence of alpha on
perception and neural dynamics (Jensen ., 2015; VanRullen, 2016; Clayton .,
2018; Foster Awh, 2019). In short, the hypothesized predictive learning
function fits remarkably well with a number of well-established properties of
these thalamocortical circuits, and we also provided a set of additional
predictions that could be tested to further evaluate this theory, especially
in contrast to the widely-discussed alternative of explicit error coding
neurons, which have not been unambiguously supported across a range of
empirical studies (Walsh ., 2020).

Furthermore, we implemented this theory in a large scale model of the visual
system, and demonstrated that learning based strictly on predicting what will
be seen next is, in conjunction with a number of critical biologically
motivated network properties and mechanisms, capable of generating abstract,
invariant categorical representations of the overall shapes of objects. The
nature of these shape representations closely matches human shape similarity
judgments on the same objects. Thus, predictive learning has the potential to
go beyond the surface structure of its inputs, and develop systematic,
abstract encodings of the environment. We found that comparison models based
on standard error backpropagation learning did not learn a categorical
structure that went beyond the surface similarity present in the visual input
layers, and future work is focused on narrowing down the specific mechanisms
required to drive this learning.

In addition to the predictive learning functions of the deep / thalamic
layers, these same circuits are also likely critical for supporting powerful
top-down attentional mechanisms that have a net multiplicative effect on
superficial-layer activations (Bortone ., 2014; Olsen ., 2012; Bortone .,
2014; Olsen ., 2012). The importance of the pulvinar for attentional
processing has been widely documented (LaBerge Buchsbaum, 1990; Bender
Youakim, 2001; Saalmann ., 2012, e.g.,), and there is likely an additional
important role of the thalamic reticular nucleus (TRN), which can contribute a
surround-inhibition contrast-enhancing effect on top of the incoming
attentional signal from the cortex (Crick, 1984; Pinault, 2004; Wimmer .,
2015; Jaramillo ., 2019). In other work in progress, we have shown that the
deep / thalamic circuits in our model produce attentional effects consistent
with the abstract Reynolds Heeger (2009) model, while the contributions of the
deep layer networks to this function are broadly consistent with the folded-
feedback model (Grossberg, 1999). These attentional modulation signals cause
the bidirectional constraint satisfaction process in the superficial network
to focus on task-relevant information while down-regulating responses to
irrelevant information — in the real world, there are typically too many
objects to track at any given time, so predictive learning must be directed
toward the most important objects (Pylyshyn, 1989; Cavanagh ., 2010; Richter
de Lange, 2019).

There is also data suggesting that the pulvinar is important for supporting
_confidence_ judgments, driven by relative ambiguity in a random dot motion
categorization task (Komura ., 2013). Critically for the present framework,
this confidence modulation only emerged in the period after the first 100 ms
of processing, and manifested as a positive correlation with confidence (i.e.,
more unambiguous stimuli resulted in higher firing rates). We can interpret
this as reflecting an ongoing generative _postdiction_ of the stimulus signal,
with stronger firing associated with more unambiguous top-down activation
based on the current internal representation. Note that this directionality is
the opposite of explicit error-coding neurons, which would presumably increase
with increasing error / ambiguity in the prediction. Interestingly,
inactivation of these pulvinar neurons resulted in a substantial (200%)
increase in opt-out choices on the most ambiguous stimuli, suggesting a level
of metacognitive awareness of the pulvinar signal (or at least a direct effect
of pulvinar on relevant metacognitive processes). Predictive accuracy would be
an ideal source of metacognitive confidence signals across a wide range of
domains, suggesting another important contribution of pulvinar even after
initial learning. Jaramillo . (2019) present a comprehensive model of
attentional, decision-making, and working memory contributions of the
pulvinar, including this confidence data, which is generally compatible with
our framework, although it does not address any learning phenomena.

Considerable further work remains to be done to more precisely characterize
the essential properties of our biologically motivated model necessary to
produce this abstract form of learning, and to further explore the full scope
of predictive learning across different domains. We strongly suspect that
extensive cross-modal predictive learning in real-world environments,
including between sensory and motor systems, is a significant factor in infant
development and could greatly multiply the opportunities for the formation of
higher-order abstract representations that more compactly and systematically
capture the structure of the world (Yu Smith, 2012). Future versions of these
models could thus potentially provide novel insights into the fundamental
question of how deep an understanding a pre-verbal human, or a non-verbal
primate, can develop (Spelke ., 1992; J. Elman ., 1996), based on predictive
learning mechanisms. This would then represent the foundation upon which
language and cultural learning builds, to shape the full extent of human
intelligence.

##  6 Appendix

All of the materials described here, including the experimental study, the
computational models, and the code to perform the representational similarity
analysis, are all available on our github account at:
https://github.com/ccnlab/deep-obj-cat and the new version of the emergent
simulation environment is at: https://github.com/emer/leabra which contains
extensive documentation and examples that can be run in Python or the Go
language. The best place to start in understanding computationally how the
predictive learning model works is with the FSA model described in the main
text, which is available at:
https://github.com/emer/leabra/tree/master/examples/deep_fsa. For the large
and complex WWI model, the most complete understanding can only be had by
directly examining the code, as there are a number of details that are not
efficiently captured in this Appendix text.

##  7 Representational Similarity Analysis Methods

The different representations being compared here are:

Leabra:

The DeepLeabra (biological model) TE layer representations (specifically TEs =
superficial – results are very similar for deep as well).

Bp:

The TEs layer representations from the backpropagation version of biological
model, including What, Where and What * Where integration layers, trained with
the V1p and V1hp (low and high resolution pulvinar) layers as final output
layers, using the time $t$ target pattern from the $t-1$ input (i.e., as a
predictive network).

V1:

The gabor-filtered representation of the visual input to both of the above
models, which was identical across them.

PredNet:

Highest layer (6th Layer) of the PredNet architecture.

Expt:

Similarity matrix constructed from human pairwise similarity judgments (see
Behavioral Experiment Methods).

Centroid | Bp
---|---
1. pyramid • banana • layercake • trafficcone • sailboat • trex 2. vertical • person • guitar • tablelamp 3. round • doorknob • donut |  3. round cont’d • handgun • chair 4. box • slrcamera • elephant • piano • fish 5. horiz • car • heavycannon • stapler • motorcycle |  1. cat1 • banana • layercake • trafficcone • sailboat • trex • person • guitar • tablelamp • doorknob • donut |  1. cat1 cont’d • handgun • chair • slrcamera • elephant • piano • fish • car 2. cat2 • heavycannon • stapler • motorcycle
V1 | PredNet
1. cat1 • trafficcone • sailboat • person • guitar • tablelamp • chair 2. cat2 • layercake • trex • doorknob • donut |  2. cat2 cont’d • handgun • slrcamera • elephant • piano • fish • car • heavycannon • stapler • motorcycle 3. cat3 • banana |  1. cat1 • trafficcone • sailboat • person • guitar • tablelamp • layercake 2. cat2 • trex • donut • banana • handgun |  2. cat2 cont’d • slrcamera • elephant • fish • car • heavycannon • stapler • motorcycle 3. cat3 • chair • doorknob • piano
Figure 14: Shape categories used for similarity matrix plots in main paper.
Centroid shape categories are near-best for both the Leabra model and the Expt
results, and fit our visual intuitions about overall shape. Bp are reliably
optimal for Bp model from all starting points. V1 are reliably optimal for V1
inputs, and also were close to the best for the Bp and PredNet layer 6
representations. PredNet are best stable solution for PredNet layer 6.

An optimal category cluster can be defined as one that has high within-cluster
similarity and low between-cluster similarity. This can be operationalized by
the contrast distance metric, based on a 1-correlation (dissimilarity)
measure, as the difference between the average within-cluster similarity and
the average between-cluster similarity:

$cd=\langle 1-r_{in}\rangle-\langle 1-r_{out}\rangle$ (1)

With distance-like 1-correlation values, this contrast distance should be
minimized (it is typically negative), or equivalently the contrast on raw
correlation values can be maximized (it is typically a positive number – just
the sign flip of distance value). We refer to the positive numbers and
maximization here as that is more intuitive.

Starting with an initial set of clusters, a permutation-based hill-climbing
strategy was used to determine a local minimum in this measure: each item was
tested in each of the other possible categories, and if that configuration
reduced the overall average contrast distance metric across all items, then it
was adopted and the process iterated until no such permutation improved the
metric. This algorithm can only decrease the number of clusters (by moving all
items out of a given cluster), so different numbers of initial clusters can be
used to search the overall space.

Figure 14 shows the resulting categories. The Bp model converged on the same
cluster state from all starting configurations tested, varying from 5 to 2
initial categories. This is the cluster set shown in Figure 10 of the main
paper, and has an average contrast distance (acd) of 0.0838 (this is
relatively low because the patterns were overall quite similar). Likewise, the
V1 patterns (which were the same across Leabra and Bp models) reliably
converged on the same pattern (shown in Figure 10), with acd = 0.2448.

For the PredNet layer 6 representations, starting from the V1 categories gave
the best results of any other set (acd = 0.1967), and a few permutations
resulted in a reliable solution that was arrived at from all other 3 category
starting points tested, shown in Figure 14 (acd = 0.2820). This indicates that
PredNet did not go much beyond the structure present in the input, even though
it did not use the V1 gabor filtering used in the Leabra and Bp models (i.e.,
this V1-level encoding well-captures the structure of the visual inputs in
general). The PredNet pixel and layer 1 representations both converged on
essentially a single monolithic category with very low acd (0.0018, 0.0013).

For the Leabra TE representations, we found a set of centroid shape categories
that are near-best when considering both the Leabra model and the results from
the human behavioral experiment (Expt). Starting from these categories, the
permutation analysis converged on reducing the size of the vertical and round
categories to one item each, over a sequence of 5 steps. This is consistent
with the observation from Figure 7 that there are three broader categories
within which the 5 finer-grained categories are embedded (i.e., vertical and
pyramid are overall similar to each other, as are round and box).
Nevertheless, our initial visual intuition about the broad shape categories,
along with a bias against having single-item categories, reinforced the use of
the finer-grained centroid selection. The average contrast difference of our
centroid selection is 0.5071, while the maximal result from the permutation
was 0.5526, which is a relatively small proportional difference.

Furthermore, once we had collected the human experimental data (Expt), it was
clear that it strongly coincided with our original shape intuitions, and with
the finer-grained 5 category centroid structure. Starting from the centroid
categories, the maximal permutation made only 3 changes, moving trex (T-rex)
and handgun into the horizontal category, and chair into the pyramid, going
from a distance score of 0.3083 to 0.3225, which is a relatively small
improvement. However, using the maximal Expt clusters directly on the Leabra
model gives a lower acd measure of 0.3745 (compared to 0.5071 for centroid),
so the centroid categories represent a good middle-ground between experiment
and the model, and this strong shared similarity structure with near-optimal
cluster structures confirms that the model and people are encoding largely the
same information.

In contrast, if we organize the experiment similarity matrix using the Bp
categories, it produces a very poor average contrast distance measure of
0.0643 (compared to 0.3083 for the centroid categories), strongly suggesting
that people’s shape representations are not compatible with that simple
structure.

Figure 15: Agglomerative clustering on the Leabra and Expt representations,
with the centroid categories color coded. The most reliable information from
this is the leaf-level groupings, as the rest of the structure is
indeterminante and history dependent in reducing higher-dimensional structure
down to a 2D plot. Both cluster plots show a strong tendency to group leaf
items together in the same centroid categories, with a few exceptions in each
case. Also, the Leabra plot nicely captures the broader 3-category structure
evident in the similarity matrix plots, within which the 5 finer-grained
centroid categories are organized. Overall, this provides further confirmation
that the model and the human subjects are organizing the shapes in largely the
same way.

Another approach to determining clusters from similarity matrices,
agglomerative clustering, starts with all items as singletons, and iteratively
combines the closest two into a new cluster. The results for the Leabra and
Expt similarity matrices are shown in Figure 15, which has also color-coded
the items in terms of their category status according to the centroid
structure. Due to a strong history dependency in the clustering process, and
the indeterminacy of reducing a high-dimensional similarity structure down to
two dimensions, structure beyond the leaf level is not very reliable (ties are
also broken by a random number generator), but nevertheless you can clearly
see that in both cases items from the same cluster are almost always together
as leaves in the plots. This then provides additional converging support for
the idea that the model is learning the same kind of shape categories as
people have.

For the network layer RSA computations, activation vectors were accumulated
separately for each 3D object item, and within that separately for each frame
index of the movie. To be able to monitor similarity metrics as the model
trained, we used a running-average integration of neural activity across
trials to accumulate the patterns. Specifically, the current activation
pattern across each layer was recorded and averaged unit-by-unit with a time
constant of $\tau=10$. Critically, by integrating separately for each frame,
this running-average computation did not introduce any bias for temporally-
adjacent frames to be more similar. Nevertheless, when we computed the frame-
to-frame similarities for TE, they were quite high (.901 correlation on
average across all objects).

##  8 Behavioral Experiment Methods

Figure 16: Example stimulus from the behavioral experiment, using the V1
reconstruction of the actual input images presented to the model, to better
capture the coarse-grained perception of the model. Subjects were requested to
choose which of the two pairs, Left or Right, was most similar in terms of
overall shape.

The behavioral experiment was conducted on Amazon.com’s MTurk web platform
under University of Colorado IRB approval (19-0176), using 30 participants
each categorizing up to 800 image pairs as shown in Figure 16, using the
standard simple image categorization framework with a lightly customized
script. Objects were drawn from the 156 3D object set, but data was aggregated
in terms of the 20 basic-level categories (car, stapler, etc) because we could
not sample all 156 x 156 object pairs. Thus, the resulting data was aggregated
for each category pair in terms of the proportion of times when that pair was
selected when presented.

The individual images were produced by reconstructing from the V1 transform
that the computational model used in its high resolution V1 input layer, to
give human participants as similar of an experience as possible to how the
model “saw” the objects, and to reduce the influence of existing semantic
knowledge which was entirely missing in our model (Figure 16).

##  9 Biological Model Methods

This section provides more information about the DeepLeabra What-Where
Integration (WWI) model. The purpose of this information is to give more
detailed insight into the model’s function beyond the level provided in the
main text, but with a model of this complexity, the only way to really
understand it is to explore the model itself. It is available for download at:
https://github.com/ccnlab/deep-obj-cat/sims/cemer. We now have a full
replication of this model in our new, much more transparent simulation
framework, available at https://github.com/ccnlab/deep-obj-cat/sims/wwi3d —
this is more readable and recommended. Furthermore, the best way to understand
this model is to understand the framework in which it is implemented, which is
explained in great detail, with many running simulations explaining specific
elements of functionality, at http://CompCogNeuro.org

###  9.1 Layer Sizes and Structure

|  | Units | Pools |
---|---|---|---|---
Area | Name | X | Y | X | Y | Receiving Projections
V1 | V1s | 4 | 5 | 8 | 8 |
| V1p | 4 | 5 | 8 | 8 | V1s V2d V3d V4d TEOd
V1h | V1hs | 4 | 5 | 16 | 16 |
| V1hp | 4 | 5 | 16 | 16 | V1s V2d V3d V4d TEOd
Eyes | EyePos | 21 | 21 |  |  |
| SaccadePlan | 11 | 11 |  |  |
| Saccade | 11 | 11 |  |  |
Obj | ObjVel | 11 | 11 |  |  |
V2 | V2s | 10 | 10 | 8 | 8 | V1s LIPs V3s V4s TEOd V1p V1hp
| V2d | 10 | 10 | 8 | 8 |  V2s V1p V1hp LIPd LIPp V3d V4d V3s TEOs
LIP | MtPos | 1 | 1 | 8 | 8 | V1s
| LIPs | 4 | 4 | 8 | 8 | MtPos ObjVel SaccadePlan EyePos LIPp
| LIPd | 4 | 4 | 8 | 8 |  LIPs LIPp ObjVel Saccade EyePos
| LIPp | 1 | 1 | 8 | 8 |  MtPos V1s LIPd
V3 | V3s | 10 | 10 | 4 | 4 | V2s V4s TEOs DPs LIPs V1p V1hp DPp TEOd
| V3d | 10 | 10 | 4 | 4 |  V3s V1p V1hp DPp LIPd DPd V4d V4s DPs TEOs
| V3p | 10 | 10 | 4 | 4 |  V3s V2d DPd TEOd
DP | DPs | 10 | 10 |  |  | V2s V3s TEOs V1p V1hp V3p TEOp
| DPd | 10 | 10 |  |  |  DPs V1p V1hp DPp TEOd
| DPp | 10 | 10 |  |  |  DPs V2d V3d DPd TEOd
V4 | V4s | 10 | 10 | 4 | 4 | V2s TEOs V1p V1hp
| V4d | 10 | 10 | 4 | 4 |  V4s V1p V1hp V4p TEOd TEOs
| V4p | 10 | 10 | 4 | 4 |  V4s V2d V3d V4d TEOd
TEO | TEOs | 10 | 10 | 4 | 4 | V4s V1p V1hp TEs
| TEOd | 10 | 10 | 4 | 4 |  TEOs TEOd V1p V1hp V4p TEOp TEp TEd
| TEOp | 10 | 10 | 4 | 4 |  TEOs V3d V4d TEOd TEd
TE | TEs | 10 | 10 | 4 | 4 | TEOs V1p V1hp
| TEd | 10 | 10 | 4 | 4 |  TEs TEd V1p V1hp V4p TEOp TEp TEOd
| TEp | 10 | 10 | 4 | 4 |  TEs V3d V4d TEOd
Table 1: Layer sizes, showing numbers of units in one pool (or entire layer if
Pool is missing), and the number of Pools of such units, along X,Y axes. Each
area has three associated layers: s = superficial layer, d = deep layer
(context updated by 51B neurons in same area, shown in bold), p = pulvinar
layer (driven by 5IB neurons from associated area, shown in bold).

Figure 5 in the main text shows the general configuration of the model, and
Table 1 shows the specific sizes of each of the layers, and where they receive
inputs from.

All the activation and general learning parameters in the model are at their
standard Leabra defaults.

###  9.2 Projections

Figure 17: Principles of connectivity in DeepLeabra. a) Markov et al (2014)
data showing density of retrograde labeling from a given injection in a
middle-level area (d): most feedforward projections originate from superficial
layers of lower areas (a,b,c) and deep layers predominantly contribute to
feedback (and more strongly for longer-range feedback). b) Summary diagram
showing most feedforward connections originating in superficial layers of
lower area, and terminating in layer 4 of higher area, while feedback
connections can originate in either superficial or deep layers, and in both
cases terminate in both superficial and deep layers of the lower area (adapted
from Felleman & Van Essen, 1991). c) Anatomical hierarchy as determined by
percentage of superficial layer source labeling (SLN) by Markov et al (2014) —
the hierarchical levels are well matched for our model, but we functionally
divide the dorsal pathway (shown in green background) into the two separable
components of a Where and a What * Where integration pathway. d) Superficial
and deep-layer connectivity in the model. Note the repeating motif between
hierarchically-adjacent areas, with bidirectional connectivity between
superficial layers, and feedback into deep layers from both higher-level
superficial and deep layers, according to canonical pattern shown in panels a
and b. Special patterns of connectivity from TEO to V3 and V2, involving
crossed super-to-deep and deep-to-super pathways, provide top-down support for
predictions based on high-level object representations. e) Connectivity for
deep layers and pulvinar in the model, which generally mirror the
corticocortical pathways (in d). Each pulvinar layer (p) receives 5IB driving
inputs from the labeled layer (e.g., V1p receives 5IB drivers from V1). In
reality these neurons are more distributed throughout the pulvinar, but it is
computationally convenient to organize them together as shown. Deep layers (d)
provide predictive input into pulvinar, and pulvinar projections send error
signals (via temporal differences between predictions and actual state) to
both deep and superficial layers of given areas (only d shown). Most areas
send deep-layer prediction inputs into the main V1p prediction layer, and
receive reciprocal error signals therefrom. The strongest constraint we found
was that pulvinar outputs (colored green) must generally project only to
higher areas, not to lower areas, with the exceptions of DPp $\rightarrow$ V3
and LIPp $\rightarrow$ V2. V2p was omitted because it is largely redundant
with V1p in this simple model.

The general principles and patterns of connectivity are shown in Figure 17
(and Figures 1 and 2 in the main text). As noted in the main text, the
connectivity and overall structure obeys the established principles identified
in neocortical anatomy (Rockland Pandya, 1979; Felleman Van Essen, 1991;
Markov, Vezoli ., 2014; Markov, Ercsey-Ravasz ., 2014).

Detailing each of the specific parameters associated with the different
projections shown in Table 1 would take too much space — those interested in
this level of detail should download the model from the link shown above.
There are topographic projections between many of the lower-level
retinotopically-mapped layers, consistent with our earlier vision models
(O’Reilly ., 2013). For example the 8x8 unit groups in V2 are reduced down to
the 4x4 groups in V3 via a 4x4 unit-group topographic projection, where
neighboring units have half-overlapping receptive fields (i.e., the field
moves over 2 unit groups in V2 for every 1 unit group in V3), and the full
space is uniformly tiled by using a wrap-around effect at the edges. Similar
patterns of connectivity are used in standard deep convolutional neural
networks. However, we do not share weights across units as in a true
convolutional network.

The projections from ObjVel (object velocity) and SaccadePlan layers to LIPs,
LIPd were initialized with a topographic sigmoidal pattern that moved as a
function of the position of the unit group, by a factor of .5, while the
projections from EyePos were initialized with a gaussian pattern. These
patterns multiplied uniformly distributed random weights in the .25 to .75
range, with the lowest values in the topographic pattern having a multiplier
of .6, while the highest had a multiplier of 1 (i.e., a fairly subtle effect).
This produced faster convergence of the LIP layer when doing Where pathway
pre-training compared to purely random initial weights, consistent with Pouget
Sejnowski (1997) and related work on parietal gain field basis function
representations.

In addition to exploring different patterns of overall connectivity, we also
explored differences in the relative strengths of receiving projections, which
can be set with a wt_scale.rel parameter in the simulator. All feedforward
pathways have a default strength of 1. For the feedback projections, which are
typically weaker (consistent with the biology), we explored a discrete range
of strengths, typically .5, .2, .1, and .05. The strongest top-down
projections were into V2s from LIP and V3, while most others were .2 or .1.
Likewise projections from the pulvinar were weaker, typically .1. These
differences in strength sometimes had large effects on performance during the
initial bootstrapping of the overall model structure, but in the final model
they are typically not very consequential for any individual projection.

###  9.3 Training Parameters

Figure 18: a) Predictive learning curve for DeepLeabra, showing the
correlation between prediction and actual over the two different V1 layers.
Initial learning is quite rapid, followed by a slower but progressive learning
process that reflects development of the IT representations (e.g.,
manipulations that interfere with those areas selectively impair this part of
the learning curve). Overall prediction accuracy remains far from perfect, as
shown in Figure 5 in main text, and significantly worse than the
backpropagtion-based models. This is a typical finding from Leabra models
which are significantly more constrained as a result of bidirectional
attractor dynamics, Hebbian learning, and inhibitory competition – i.e., the
very things that are likely important for forming abstract catgorical
representations. b) Similarity matrix over TEs layer at 200 epochs, which has
less contrast and definition (particularly evident in the off-block-diagonal
differences) compared to the 1,000 epoch result (c also shown in Figure 7 in
main text).

Training typically consisted of 512 alpha trials per epoch (51.2 seconds of
real time equivalent), for 1,000 such epochs. Each trial was generated from a
virtual reality environment in the emergent simulator, that rendered first-
person views with moving eye position onto the object tumbling through space
with fixed motion and rotation parameters over the sequence of 8 frames (see
Figure 5 in main text for representative example). Each frame was rendered at
256 x 256 resolution, and processed through our standard V1 gabor filters
which are described in detail in O’Reilly . (2013).

Because the start of each sequence of 8 frames is unpredictable, we turned off
learning for that trial, which improves learning overall. We have recently
developed an automatic such mechanism based on the running-average (and
running variance) of the prediction error, where we turn off learning whenever
the current prediction error z-normalized by these running average values is
below 1.5 standard deviations, which works well, and will be incorporated into
future models. Biologically, this could correspond to a connection between
pulvinar and neuromodulatory areas that could regulate the effective learning
rate in this way.

Figure 18a shows the learning trajectory of the model, indicating that it
learns quite rapidly. This rapid initial learning is likely facilitated by the
extensive use of shortcut connections convering from all over the simulated
visual system onto the V1 pulvinar layers, and direct projections back from
these pulvinar layers. Thus, error signals are directly communicated and can
drive learning quickly and efficiently. However, there are also extensive
indirect, bidirectional connections among the superficial layers, which can
drive indirect error backpropagation learning as well.

###  9.4 Model Algorithms

The biologically-based model was implemented using the Leabra framework, which
is described in detail in previous publications (O’Reilly ., 2016, 2012;
O’Reilly Munakata, 2000; O’Reilly, 1998, 1996), and summarized here. The
online textbook at https://CompCogNeuro.org provides the most comprehensive
description of the framework, while https://github.com/emer/leabra has a
summary of all the equations (and the code itself). There are two main
implementations of Leabra, one in the C++ emergent software, and a new one