# Large Language Models Converge on Brain-Like Word Representations

Jiaang Li1,†, Antonia Karamolegkou1,†, Yova Kementchedjhieva1,
Mostafa Abdou3, Sune Lehmann2,4 & Anders Søgaard1,2,∗

1 Dpt. of Computer Science, University of Copenhagen, Lyngbyvej 2, Copenhagen
Ø, DK-2300, Denmark
2 Pioneer Centre for Artificial Intelligence, University of Copenhagen, Øster
Voldgade 3, Copenhagen K, DK-1350, Denmark
3 Princeton University, Princeton Neuroscience Institute, Rm. 234, US-08544,
New Jersey
4 Technical University of Denmark, Anker Engelundsvej 1, Kgs Lyngby, DK-2800,
Denmark
† These authors contributed equally to this work.
∗ To whom correspondence should be addressed<EMAIL_ADDRESS>
## Abstract

One of the greatest puzzles of all time is how understanding arises from
neural mechanics. Our brains are networks of billions of biological neurons
transmitting chemical and electrical signals along their connections. Large
language models are networks of millions or billions of digital neurons,
implementing functions that read the output of other functions in complex
networks. The failure to see how meaning would arise from such mechanics has
led many cognitive scientists and philosophers to various forms of dualism –
and many artificial intelligence researchers to dismiss large language models
as stochastic parrots or jpeg-like compressions of text corpora. We show that
human-like representations arise in large language models. Specifically, the
larger neural language models get, the more their representations are
structurally similar to neural response measurements from brain imaging.

On a daily basis, we depend heavily on interactions with dozens, if not
hundreds, of other intelligent beings, some human, some artificial (your car,
Alexa, ChatGPT, etc.). In this work, we address two deep, profound mysteries
in one go: whether there is understanding taking place inside large language
models (LLMs) such as Alexa TM or ChatGPT, and how understanding takes place
in our brains. Our main research question is: Are LLMs as intelligent as they
seem, or are they merely ‘stochastic parrots’?111The term ‘stochastic parrot’
was coined by Emily Bender and colleagues [1] to suggest that an LLM ‘is not
grounded in communicative intent, any model of the world, or any model of the
reader’s state of mind.’ They continue: ‘It can’t have been, because the
training data never included sharing thoughts with a listener, nor does the
machine have the ability to do that.’ There is growing evidence LLMs do have
(reasonably good) models of the world [2, 3, 4]. We show that, moreover, these
models are increasingly brain-like. The (even more interesting) flipside of
this question, however, is: Are humans as intelligent as they seem, or are
they merely (something very similar to) LLMs?

Figure 1: Experimental flow and main results. We run experiments with three
families of LLMs (comparing LLMs of different sizes within families), three
fMRI datasets, and three state-of-the-art projection algorithms, and results
are the same across all 27 combinations: LLMs converge toward human-like
representations, enabling precision@10 (P@10) retrieval rates of up to 25%,
i.e., half of all concepts can be decoded from the fMRI signals. The datasets,
our Gaussian smoothing technique, and the projection methods are described in
Methods. Right side: Convergence results for three families of LLMs across
three datasets, using Procrustes Analysis. The task here is: Given a neural
response, which word (in a vocabulary of 1,291 words, was read at the time the
response was recorded? Chance (baseline) P@10 is $<0.01$ (the dotted red
lines). Convergence is consistent, and some retrieval rates are remarkably
high, decoding almost half of the words correctly to a neighborhood of 10 word
forms. More results are presented in the Supplementary Materials.

There are many ways to approach this question. Artificial intelligence
researchers evaluate LLMs by measuring their performance on benchmark data and
protocols [5, 4]. Doing so, they aim to infer what LLMs have learned, from how
they behave. The methodology is behaviorist and has obvious limitations. We
instead suggest exploring the inside of LLMs and our brains – or, to be
precise, their representational geometries. Our experimental flow is
represented in Figure 1; see Methods for details.

As Leibniz remarked three centuries ago, if you could blow the brain up to the
size of a mill and walk about inside, you would not find understanding. This
argument is referred to as Leibniz’s Mill. Nothing in how individual axons and
dendrites pass messages back and forth seems to explain how we come to have
thoughts about the world, how we come to understand. In the same way, nothing
in how numbers are passed around in neural large language models, explains how
semantics would arise from these activities. Several researchers have claimed,
relying on arguments remarkably similar to Leibniz’, that LLMs are nothing but
‘stochastic parrots’, ‘jpegs of the Internet’, or ‘simple pattern matchers’.
We think that such metaphors are misleading and fail to explain many
properties of LLMs. One such property is presented here – the structural
similarity between LLM representations and neural response measurements – and
we think that this property has serious implications for Leibniz’s Mill and
its more recent reincarnations [6, 7].

What we find is remarkable structural similarity between how words are
represented in LLMs, and the neural response measurements of humans reading
the same words. The LLM representations of a vocabulary form a geometry in a
$d$-dimensional vector space; and the neural response measurements from one or
more participants reading these words in a brain scanner, form in a similar
way a geometry in a $d^{\prime}$-dimensional space. We can compute the
structural similarity (degree of isomorphism) between these two geometries by
multi-way ridge regression, Representational Similarity Analysis [8], and
Procrustes Analysis [9] (if $d=d^{\prime}$), for example. We present
experiments for three families of LLMs (as well as static word embeddings),
all three of these evaluation methods, across three different datasets. Across
the board, we see high degrees of isomorphism, enabling decoding or retrieval
performance of up to<EMAIL_ADDRESS>(with random performance being
P@10$<$0.01). Word-level brain decoding thus seems much more feasible than
previously expected.

Figure 2: t-SNE plot of fMRI and LLM representations using OPT-30b (large,
uncased) over select target words from dataset 10. We evaluate the retrieval
performance of our alignments using precision@$k$, which measures the ratio of
word tokens $w_{i}$, e.g., for $k=5$, ’Potter2183’, whose fMRI representations
are projected into the LLM space such that the LLM representation for the word
type $w$, e.g., Potter, is among the $5$-nearest neighbors of $w_{i}$. In this
case, the neural activity associated with ’never313’ is not read as writing
directly – but still with words as the top-5 guess. That said, the words
Potter and would are decoded correctly by our alignment (top-1 guess or P@1).

Results: Convergence to Word-Level fMRI Measurements. Our main results are
presented in Figure 6 and in the Supplementary Material and concern the
convergence of three families of LLMs on representations that are remarkably
similar to those seen in neural response measurements. These results are
consistent across three large-scale fMRI datasets and three mapping methods.
The projections map the fMRI vectors to LLM representations, enabling direct
comparison of geometries; see Figure 2. We use precision@$k$ (P@$k$) to
quantify the alignment precision of linear projections; see Methods for
details. In brief, the P@$k$ score is the fraction of words for which the
LLM’s representation is a $k$-nearest neighbor of the fMRI encoding. Word-
level decoding thus amounts to simple nearest neighbor retrieval in the
projected space. The plots show average and maximum performance across
participants.

The scores are plotted by model size, showing the convergence toward brain-
like representations as LLMs increase in size. The best scores indicate that
LLMs up to 1.5B parameters can achieve alignments such that a bit more than 1
in 5 words are decoded correctly,222The reason we count P@5 or P@10 as correct
decoding is that a neighborhood of 5-10 words will tend to consist of
inflections of the same lemma or synonymous words [11] . P@1 would amount to
guessing the lemma, the exact inflection, and the correct spelling variant.
and a bit more than 2 in 5 almost correctly (within neighborhoods of 20-30
word forms); see Supplementary Materials for more analysis. The results are
obtained with limited supervision for learning the mapping. In fact, we only
rely on 950 data points to induce this linear projection, a small number given
the high dimensionality of the derived word representations; see Methods for
details. It is likely that somewhat better mappings can be induced with more
supervision – a question we briefly explore in the Supplementary Materials.

Figure 3: Alignment precision results across layers. The alignment with fMRI
improves with model depth, for BERT and Procrustes Analysis; see Supplementary
Materials for similar results for other LLMs and projection methods.

Results: Where are LLMs Most Brain-Like? We also consider at what layers the
different language models align best with the representations extracted from
the fMRI data. The results presented in Figure 3 and the Supplementary
Materials are unambiguous and show that deeper representations align better
with neural response measurements. This holds across all architectures and
model sizes.

Interestingly, the alignment improvements at deeper layers do not wear off to
reach a plateau. Our results, in fact, suggest that better alignment results
can be achieved by training even deeper models. This may also explain the
strong correlation between depth and generalization often observed in the
literature [12].

Discussion. Our results have direct philosophical and practical implications.
As for the philosophical impact, there is currently a very lively debate about
whether large language models exhibit ‘understanding’ [3, 4]. Half of the AI
community (51%) – according to a recent survey [4] – are willing to attribute
non-trivial understanding to large language models (LLMs). The other half of
the community (49%) argue that the illusion of understanding is the result of
an Eliza effect. The research question, as formulated by 4, is:

> ”do these systems (or will their near-term successors) actually, even in the
> absence of physical experience, create something like the rich concept-based
> mental models that are central to human understanding, and, if so, does
> scaling these models create even better concepts?”

Our results suggest the answer to both questions are affirmative. Our finding,
by the way, is also corroborated by previous results suggesting that language
models for different languages converge on similar knowledge representations
[13]. This may be a side-effect of inducing ‘something like the rich concept-
based mental models that are central to human understanding’.

This provides a new point of departure for studying how semantics arises in
biological systems. LLMs are trained to predict the next stimulus and
implicitly induce word models. Philosophers argue whether structural
similarities (isomorphisms, homomorphisms, etc.) between representations and
what is represented, is sufficient for content [14]. One observation that goes
all the way back to Carnap’s Aufbau [15] is: Structural similarities are
generally trivial to obtain, but if the relations (distances in the vector
space) serve a purpose, structural similarities can ground content. Structural
similarity is evidently sufficient to solve semantic problems, such as
bilingual dictionary induction [16] or multi-modal alignment [17]. The fact
that fMRI vectors exhibit structural similarities to LLMs (and by
transitivity, across languages and to computer vision models), is suggestive
of such similarities playing a role in grounding.

## Methods

### Data description and pre-processing

#### fMRI datasets

fMRI is a non-invasive neural response measurement technique that records on a
spatial resolution in the region of 1 to 6 millimetres, higher than any other
technique. fMRI records activity (blood flow) in the entire network of brain
areas engaged when subjects undertake particular tasks. On the downside, fMRI
is somewhat susceptible to influences of non-neural changes, and the temporal
response is poor relative to the electrical signals that define neuronal
communication. To compensate for low temporal resolution, we introduce
Gaussian smoothing below.

fMRI measurements of human neural responses during reading or listening are
somewhat difficult to obtain. It is expensive, requires a large number of
participants, and requires the participants to lie still during reading for
extended periods of time. We list three datasets below that we use in our
experiments. We see small differences across the three datasets (like we see
small differences across language model families and alignment methods), but
the general trend remains the same.

The datasets are: Harry Potter Dataset 18 (8 subjects), Pereira Dataset 19 (16
subjects), and Natural Stories Audio Dataset 20 (19 subjects). All three
datasets are publicly available.

#### Gaussian smoothing

Gaussian smoothing has been used before to study speech-aligned fMRI data [21,
22]. In cases where fMRI data is not collected at the granularity of
individual words, we can use Gaussian smoothing to generate word-level fMRI
information. For instance, to obtain the fMRI vector for a specific word like
”Harry” at a given time point t (Harryt), we can extract the fMRI vectors for
a certain timeframe T around t, such as $t\pm T$ seconds. We then apply
Gaussian smoothing to this set of vectors, resulting in a final vector that
represents the fMRI information for the word ”Harryt”.

This approach has potential benefits for fMRI analysis in various
applications, such as studies of language processing and cognitive
neuroscience. By generating word-level fMRI information using Gaussian
smoothing, we can potentially extend scope from sequence-level to word-level,
and improve the interpretability and accuracy of the results obtained from
fMRI analyses. Extracting word-level signals differentiates our work from much
other work, but was also shown to be crucial in recent work on brain decoding
[23].

### Models

#### Non-auto-regressive models

Non-auto-regressive models are a type of machine learning models that take in
an entire input sequence of text and generate a single output vector
representation for the entire sequence. Training such models, we mask a
fraction of the words in the input text. The language model is then expected
to predict the masked words based on the other words in the text. We use the
BERT family of language models [24] as an example of a non-auto-regressive
model family.

#### Auto-regressive models

Auto-regressive models generate output sequences by predicting each element in
the sequence based on the previously generated elements. In other words, the
output is generated one element at a time, with the model conditioned on the
previous output elements. These language models are used to generate text, but
typically provide slightly worse similarity estimates. We use two auto-
regressive language model families: GPT2 [25] and OPT [26].

### Comparison and projection methods

#### Representational Similarity Analysis

Relational Similarity Analysis (RSA) is a multivariate analysis technique
commonly used in cognitive neuroscience and computational linguistics to
compare the similarity between two sets of representations. RSA can be used to
measure the similarity between the neural activity patterns observed in the
fMRI data and the representations learned by LLMs. RSA operates by first
representing the neural activity and language model features as vectors in a
high-dimensional space. The similarity between these vectors is then
quantified using a rank-based correlation metric [27]. Specifically, we use
the second-order Spearman’s correlation coefficient as a way of measuring
structural similarity or degree of isomorphism.

#### Ridge regression

Ridge regression is a widely used method in statistics and machine learning to
address the issue of multicollinearity, which can arise when there are highly
correlated predictor variables in a linear regression model. We use ridge
regression for every target dimension in our data, modeling the relationship
between the brain signals and the individual dimensions in the language model
representations. Specifically, ridge regression adds a penalty term to the
standard least squares regression objective function, which shrinks the
magnitude of the regression coefficients towards zero, effectively reducing
the impact of less informative predictors. This regularization technique helps
prevent overfitting and can improve the generalization performance of the
model See more details in Supplementary Materials.

#### Procrustes Analysis

We use Procrustes Analysis, a form of statistical shape analysis, to align
brain fMRI representations with those of language models, using a bimodal
dictionary. Procrustes Analysis is a method for matching corresponding points
in two shapes and finding the transformation (translation, rotation, and
scaling) that best aligns them. Specifically, we seek to find the orthogonal
matrix $\Omega$ that optimally maps the brain fMRI matrix $A$ representing
brain responses to the words onto the language model matrix $B$, i.e. the
language model representations of the words, using the $\min_{R}|R-M|_{F}$
problem subject to $R^{T}R=I$, which can be solved using singular value
decomposition with $R=UV^{T}$. To evaluate the effectiveness of the alignment,
we induce it from a small set of point pairs and test it on held-out data. We
stress that successful alignment is contingent on the source and target spaces
having the same dimensionality. If a mismatch occurs, we employ principle
component analysis to reduce the dimensionality of the larger space. To
evaluate the alignment, we induce it from a small set of point pairs and test
it on held-out data. It is essential that the source and target spaces have
the same dimensionality for successful alignment. In cases where a mismatch
occurs, we employ principle component analysis to reduce the dimensionality of
the larger space.

## References

  * [1] Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S.  On the dangers of stochastic parrots: Can language models be too big?  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’21, 610–623 (Association for Computing Machinery, New York, NY, USA, 2021).  URL https://doi.org/10.1145/3442188.3445922.
  * [2] Garneau, N. _et al._ Analogy training multilingual encoders.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 35, 12884–12892 (2021).  URL https://ojs.aaai.org/index.php/AAAI/article/view/17524.
  * [3] Søgaard, A.  Understanding models understanding language.  _Synthese_ 200, 1–16 (2022).
  * [4] Mitchell, M. & Krakauer, D. C.  The debate over understanding in ai’s large language models.  _Proceedings of the National Academy of Sciences_ 120, e2215907120 (2023).  URL https://www.pnas.org/doi/abs/10.1073/pnas.2215907120.  https://www.pnas.org/doi/pdf/10.1073/pnas.2215907120.
  * [5] Lewkowycz, A. _et al._ Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.  Tech. Rep. (2022).
  * [6] Block, N.  Troubles with functionalism.  _Minnesota Studies in the Philosophy of Science_ 9, 261–325 (1978).
  * [7] Dennett, D. C.  Toward a cognitive theory of consciousness.  _Minnesota Studies in the Philosophy of Science_ 9 (1978).
  * [8] Kriegeskorte, N., Mur, M. & Bandettini, P.  Representational similarity analysis - connecting the branches of systems neuroscience.  _Front. Syst. Neurosci._ 2, 4 (2008).
  * [9] Kementchedjhieva, Y., Ruder, S., Cotterell, R. & Søgaard, A.  Generalizing Procrustes analysis for better bilingual dictionary induction.  In _Proceedings of the 22nd Conference on Computational Natural Language Learning_ , 211–220 (Association for Computational Linguistics, Brussels, Belgium, 2018).  URL https://aclanthology.org/K18-1021.
  * [10] Wehbe, L., Vaswani, A., Knight, K. & Mitchell, T.  Aligning context-based statistical models of language with brain activity during reading.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 233–243 (Association for Computational Linguistics, Doha, Qatar, 2014).  URL https://aclanthology.org/D14-1030.
  * [11] Kementchedjhieva, Y., Hartmann, M. & Søgaard, A.  Lost in evaluation: Misleading benchmarks for bilingual dictionary induction.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , 3336–3341 (Association for Computational Linguistics, Hong Kong, China, 2019).  URL https://aclanthology.org/D19-1328.
  * [12] Goodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S. & Shet, V.  Multi-digit number recognition from street view imagery using deep convolutional neural networks (2014).  URL https://arxiv.org/pdf/1312.6082.pdf.
  * [13] Vulić, I., Ruder, S. & Søgaard, A.  Are all good word vector spaces isomorphic?  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 3178–3192 (Association for Computational Linguistics, Online, 2020).  URL https://aclanthology.org/2020.emnlp-main.257.
  * [14] Shea, N.  Content and its vehicles in connectionist systems.  _Mind and Language_ 22, 246–269 (2007).
  * [15] Carnap, R.  _The Logical Structure of the World_ (Berkeley: University of California Press, 1967).
  * [16] Søgaard, A., Vulić, I., Ruder, S. & Faruqui, M.  _Cross-Lingual Word Embeddings_.  Synthesis Lectures on Human Language Technologies (Morgan & Claypool Publishers, United States, 2019), 2 edn.
  * [17] Li, J., Kementchedjhieva, Y. & Søgaard, A.  Implications of the convergence of language and vision model geometries (2023).  2302.06555.
  * [18] Wehbe, L. _et al._ Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses.  _PLoS One_ 9, e112575 (2014).
  * [19] Pereira, F. _et al._ Toward a universal decoder of linguistic meaning from brain activation.  _Nature Communications_ 9 (2018).
  * [20] Zhang, Y., Han, K., Worth, R. & Liu, Z.  Connecting concepts in the brain by mapping cortical representations of semantic relations.  _bioRxiv_ (2020).  URL https://www.biorxiv.org/content/early/2020/03/26/649939.  %****␣main_scales.bbl␣Line␣175␣****https://www.biorxiv.org/content/early/2020/03/26/649939.full.pdf.
  * [21] Bingel, J., Barrett, M. & Søgaard, A.  Extracting token-level signals of syntactic processing from fMRI - with an application to PoS induction.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 747–755 (Association for Computational Linguistics, Berlin, Germany, 2016).  URL https://aclanthology.org/P16-1071.
  * [22] Brodoehl, S., Gaser, C., Dahnke, R., Witte, O. W. & Klingner, C. M.  Surface-based analysis increases the specificity of cortical activation patterns and connectivity results.  _Scientific Reports_ 10 (2020).
  * [23] Tang, J., LeBel, A., Jain, S. & Huth, A. G.  Semantic reconstruction of continuous language from non-invasive brain recordings.  _bioRxiv_ (2022).  URL https://www.biorxiv.org/content/early/2022/09/29/2022.09.29.509744.  https://www.biorxiv.org/content/early/2022/09/29/2022.09.29.509744.full.pdf.
  * [24] Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 4171–4186 (Association for Computational Linguistics, Minneapolis, Minnesota, 2019).  URL https://aclanthology.org/N19-1423.
  * [25] Radford, A. _et al._ Language models are unsupervised multitask learners (2019).
  * [26] Zhang, S. _et al._ Opt: Open pre-trained transformer language models (2022).  2205.01068.
  * [27] Lepori, M. & McCoy, R. T.  Picking BERT’s brain: Probing for linguistic dependencies in contextualized embeddings using representational similarity analysis.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , 3637–3651 (International Committee on Computational Linguistics, Barcelona, Spain (Online), 2020).  URL https://aclanthology.org/2020.coling-main.325.
  * [28] Radovanović, M., Nanopoulos, A. & Ivanović, M.  Hubs in space: Popular nearest neighbors in high-dimensional data.  _J. Mach. Learn. Res._ 11, 2487–2531 (2010).
  * [29] Conneau, A., Lample, G., Ranzato, M., Denoyer, L. & Jégou, H.  Word Translation Without Parallel Data.  In _Proceedings of ICLR 2018_ (2018).  URL http://arxiv.org/abs/1710.04087.  1710.04087.
  * [30] Zhu, Y. _et al._ Aligning books and movies: Towards story-like visual explanations by watching movies and reading books.  _2015 IEEE International Conference on Computer Vision (ICCV)_ 19–27 (2015).
  * [31] Trinh, T. H. & Le, Q. V.  A simple method for commonsense reasoning.  _ArXiv_ abs/1806.02847 (2018).
  * [32] Gao, L. _et al._ The pile: An 800gb dataset of diverse text for language modeling.  _CoRR_ abs/2101.00027 (2021).  URL https://arxiv.org/abs/2101.00027.  2101.00027.
  * [33] Baumgartner, J., Zannettou, S., Keegan, B., Squire, M. & Blackburn, J.  The pushshift reddit dataset.  _CoRR_ abs/2001.08435 (2020).  URL https://arxiv.org/abs/2001.08435.  2001.08435.
  * [34] Paszke, A. _et al._ Pytorch: An imperative style, high-performance deep learning library.  In _Advances in Neural Information Processing Systems 32_ , 8024–8035 (Curran Associates, Inc., 2019).  URL http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
  * [35] Wolf, T. _et al._ Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , 38–45 (Association for Computational Linguistics, Online, 2020).  URL https://aclanthology.org/2020.emnlp-demos.6.

Author Contributions. The first and last authors designed the study and the
model. The first and second authors performed the analyses and implemented the
model. All authors analyzed the results and wrote the paper.
Competing Interests. The authors declare that they have no competing financial
interests.
Additional Information. Supplementary Information is available for this paper.
Code will be available at https://github.com/coastalcph/brainlm.

## Supplementary Materials

### Supplementary Information

##### Ridge Regression

The models are trained to predict the signal of word $w$ in layer $l$, denoted
as $y_{l^{w}}$, using the vector of fMRI voxels for that word, $x^{w}$. For
each subject and layer $l$, we employ cross-validation to estimate the
predictiveness of the fMRI representation of the word in each dimension $i$.
In each fold, the fMRI data matrix, denoted as
$X=x_{w^{1}},x_{w^{2}},...,x_{w^{n}}$, and the semantic vector matrix, denoted
as $Z=z_{w^{1}},z_{w^{2}},...,z_{w^{n}}$, are split into corresponding
training and validation matrices which are individually normalized to have a
mean of 0 and a standard deviation of 1 for each dimension across words,
ending with training matrices $X^{R}$ and $Z^{R,l}$, as well as validation
matrices $X^{V}$ and $Z^{V,l}$. Using the training fold, we estimate a model
$\theta^{i,l}$ as follows:

$\arg\min_{\theta^{i,l}}||z^{R,i}-X^{R}\theta^{i,l}||^{2}_{2}+\lambda^{i}||\theta^{i,l}||^{2}_{2}$

To identify the best $\lambda^{i}$ for each dimension $i$ that minimizes the
nested cross-validation error, we employ a ten-fold nested cross-validation.
Subsequently, we estimate $\theta^{i,l}$ using $\lambda^{i}$ on the entire
training fold. Thus, the predictions for each dimension in the validation fold
are obtained as $p^{l}=X^{V}\theta^{i,l}$.

##### Cross-domain Similarity Local Scaling (CSLS)

Nearest neighbors are naturally asymmetric, which means if y is a K-NN of x,
it does not follow that x is also a K-NN of y. In high-dimensional spaces
[28], the nearest neighbor rule can lead to a phenomenon called hubness, where
some vectors (hubs) are nearest neighbors of many other points, while others
(anti-hubs) are not nearest neighbors of any point. This is detrimental to
matching pairs based on the nearest neighbor rule. To address this issue,
Conneau et al. propose a bi-partite neighborhood graph [29], in which each
word of a given dictionary is connected to its $K$ nearest neighbors in the
other language. The neighborhood of a mapped source word embedding $Wx_{s}$,
denoted as $N_{T}(Wx_{s})$, is represented on the bipartite graph. It consists
of $K$ elements, all of which are words from the target language. Similarly,
the neighborhood of a target word $t$, denoted as $N_{S}(y_{t})$. The mean
similarity between a source embedding $x_{s}$ and its corresponding target
neighborhood is considered as

$r_{T}(Wx_{s})=\frac{1}{K}\sum_{y_{t}\in N_{T}(Wx_{s})}cos(Wx_{s},y_{t}),$

where cos(.,.) means cosine similarity. They use $r_{S}(y_{t})$ to represent
the mean similarity of a target word $y_{t}$ to its neighborhood. The
definition of Cross-domain Similarity Local Scaling (CSLS) between mapped
source words and target words is

$CSLS(Wx_{s},y_{t})=2cos(Wx_{s},y_{t})\\\ -r_{T}(Wx_{s})-r_{S}(y_{t}).$

LMs | Hidden Layers | Hidden Size | Attention Heads | Total # of Params | Datasets
---|---|---|---|---|---
BERT$\\_{\text{TINY}}$ | 2 | 128 | 2 | 4.4M |  BooksCorpus[30] ; English Wikipedia [24]
BERT$\\_{\text{MINI}}$ | 4 | 256 | 4 | 11.3M
BERT$\\_{\text{SMALL}}$ | 4 | 512 | 8 | 29.1M
BERT$\\_{\text{MEDIUM}}$ | 8 | 512 | 8 | 41.7M
BERT$\\_{\text{BASE}}$ | 12 | 768 | 12 | 110.1M
BERT$\\_{\text{LARGE}}$ | 24 | 1,024 | 16 | 336M
GPT2$\\_{\text{BASE}}$ | 12 | 768 | 12 | 117M |  WebText [25]
GPT2$\\_{\text{Medium}}$ | 24 | 1,024 | 16 | 345M
GPT2$\\_{\text{LARGE}}$ | 36 | 1,280 | 20 | 762M
GPT2$\\_{\text{XL}}$ | 48 | 1,600 | 25 | 1,542M |
|  |  |  |  |  BooksCorpus; CC-Stories[31] ; CCNewsV2[26] ; The Pile[32] ; Pushshift.io Reddit dataset[33]
OPT$\\_{\text{125M}}$ | 12 | 768 | 12 | 125M
OPT$\\_{\text{1.3B}}$ | 24 | 1,024 | 32 | 1.3B
OPT$\\_{\text{6.7B}}$ | 32 | 4,096 | 32 | 6.7B
OPT$\\_{\text{30B}}$ | 48 | 7,168 | 56 | 30B

Table 1: The 14 language models used in our experiments. Table 3 lists the links of LMs. Datasets | U.W. | P@1 | P@5 | P@10 | P@30 | P@50 | P@100
---|---|---|---|---|---|---|---
Harry Potter | 1291 | 0.07 | 0.36 | 0.71 | 2.14 | 3.56 | 7.12
Pereira | 2561 | 0.04 | 0.20 | 0.39 | 1.17 | 1.95 | 3.90
Natural Stories (story one) | 381 | 0.26 | 1.31 | 2.62 | 7.87 | 13.12 | 26.25
Table 2:  The random retrieval baseline precision-at-$k$ (P@$k$) with
$k\in\\{1,5,10,30,50,100\\}$ of three datasets. The baselines are calculated
by the U.W. in stimulus content, respectively. U.W. = the number of unique
words.

##### Implementation

Our implementation is based on PyTorch v.1.13.1[34] and Transformer v4.25.1
[35] for Python 3.9.13 and builds on code from the repositories in Table 3.

LMs | Links
---|---
BERT$\\_{\textsc{TINY}}$ | https://huggingface.co/google/bert_uncased_L-2_H-128_A-2
BERT$\\_{\textsc{MINI}}$ | https://huggingface.co/google/bert_uncased_L-4_H-256_A-4
BERT$\\_{\textsc{SMALL}}$ | https://huggingface.co/google/bert_uncased_L-4_H-512_A-8
BERT$\\_{\textsc{MEDIUM}}$ | https://huggingface.co/google/bert_uncased_L-8_H-512_A-8
BERT$\\_{\textsc{BASE}}$ | https://huggingface.co/bert-base-uncased
BERT$\\_{\textsc{LARGE}}$ | https://huggingface.co/bert-large-uncased
GPT2$\\_{\textsc{BASE}}$ | https://huggingface.co/gpt2
GPT2$\\_{\textsc{MEDIUM}}$ | https://huggingface.co/gpt2-medium
GPT2$\\_{\textsc{LARGE}}$ | https://huggingface.co/gpt2-large
GPT2$\\_{\textsc{XL}}$ | https://huggingface.co/gpt2-xl
OPT$\\_{\textsc{125M}}$ | https://huggingface.co/facebook/opt-125m
OPT$\\_{\textsc{1.3B}}$ | https://huggingface.co/facebook/opt-1.3b
OPT$\\_{\textsc{6.7B}}$ | https://huggingface.co/facebook/opt-6.7b
OPT$\\_{\textsc{30B}}$ | https://huggingface.co/facebook/opt-30b

Table 3:  Links of 14 Transformer-based language models used in our
experiments.

### Supplementary Figures

Figure 4: Convergence results for three families of LLMs using Relational
Similarity Analysis. See Figure 6 and 8 for the results with Procrustes
Analysis and Ridge Regression.

Figure 5: Alignment precision results across layers. The alignment with fMRI
improves with model depth, for BERT$\\_$LARGE, GPT2$\\_$XL and OPT$\\_$30B
with Relational Similarity Analysis.

Figure 6: Convergence results for three families of LLMs across three
datasets, using Procrustes Analysis with PCA. See the random retrieval
precision baseline in Table 2. Convergence is consistent, and some retrieval
rates are remarkably high, decoding almost half of the words correctly to a
neighborhood of 10 word forms.

Figure 7: Convergence results for three families of LLMs across three
datasets, using Procrustes Analysis with Gaussian random projection. See the
random retrieval precision baseline in Table 2.

Figure 8: Convergence results for three families of LLMs using Ridge
Regression. See the random retrieval precision baseline in Table 2, and Figure
6 and 7 for results with Procrustes Analysis.

Figure 9: Alignment precision results across layers. The alignment with fMRI
(Harry Potter dataset) improves with model depth for LLMs and Procrustes
Analysis with Gaussian Random Projection.

Figure 10: Alignment precision results across layers. The alignment with fMRI
(Harry Potter dataset) improves with model depth for LLMs and Ridge Regression
with Gaussian Random Projection.