# Compositional learning of functions in humans and machines

Yanli Zhou1,3
<EMAIL_ADDRESS>

1Center for Data Science
New York University &Brenden M. Lake1,2
<EMAIL_ADDRESS>

2Department of Psychology
New York University &Adina Williams3
<EMAIL_ADDRESS>

3Meta AI

###### Abstract

The ability to learn and compose functions is foundational to efficient
learning and reasoning in humans, enabling flexible generalizations such as
creating new dishes from known cooking processes. Beyond sequential chaining
of functions, existing linguistics literature indicates that humans can grasp
more complex compositions with interacting functions, where output production
depends on context changes induced by different function orderings. Extending
the investigation into the visual domain, we developed a function learning
paradigm to explore the capacity of humans and neural network models in
learning and reasoning with compositional functions under varied interaction
conditions. Following brief training on individual functions, human
participants were assessed on composing two learned functions, in ways
covering four main interaction types, including instances in which the
application of the first function creates or removes the context for applying
the second function. Our findings indicate that humans can make zero-shot
generalizations on novel visual function compositions across interaction
conditions, demonstrating sensitivity to contextual changes. A comparison with
a neural network model on the same task reveals that, through the meta-
learning for compositionality (MLC) approach, a standard sequence-to-sequence
Transformer can mimic human generalization patterns in composing functions.

Keywords: function composition; meta-learning; compositional generalization;
order of operations

## Introduction

Humans are efficient and flexible learners, with the ability to infer an
underlying function from exposure to just a few input-output examples. Humans
are also adept in composing functions together in sequence: we can flexibly
combine previously learned functions in new ways. For example, someone who
knows how to chop vegetables and knows how to fry food can learn how to make
fries, by chaining their chopping skills with their frying skills, even if
they have never previously attempted that dish. Moreover, function composition
skills emerge early in life, with children beginning to learn how to compose
visual functions without explicit training at as early as 3.5 years of age (?,
?). Mastery of these skills over time becomes a cornerstone for comprehending
complex symbolic systems, fostering abstract thinking and aiding the
acquisition of conceptual knowledge (?, ?, ?, ?).

Figure 1: Function interactions influence function compositions. The chopping
operation provides context for frying while the puréeing operation renders
frying inapplicable.

Beyond the sequential application of functions, humans can also track
contextual changes that occur during function composition as a result of the
order of operations. In the previous example, if the potatoes have been
puréed, one would quickly realize that the frying operation can no longer take
place, as the puréeing function has transformed the potatoes into a state
unsuitable for the second function to apply. Linguists have formalized
different context-shifting phenomena into four types of function ordering (?,
?): feeding describes scenarios in which the application of the first function
creates the context for the second one to apply, as illustrated in the example
when chopping feeds the function frying (Fig.1, top); counter-feeding
describes the reverse order of application, with the context-creating function
applied too late. For example, trying—and failing—to fry whole potatoes before
chopping them. Bleeding occurs when the context of the second function is
removed by the first function’s application, akin to the example where
puréeing potatoes makes the unsuitable for later frying (Fig.1, bottom);
counter-bleeding is its reverse, where the context-removal happens after the
first function has successfully been applied (i.e., frying does not prevent
one from later puréeing). Together, we see that function composition emerges
as a challenging learning program, requiring not only generalization to novel
instances on the individual function level, but also to compositions of two
functions with sensitivity to different interactions induced by various
function orders.

Past research has investigated whether models can capture human-like
compositional learning and generalization. For instance, ? (?) proposed the
SCAN task to assess the compositional skills of neural network models. The
authors, as well as in many subsequent investigations (?, ?, ?, ?), found that
neural networks continue to struggle with systematic generalizations despite
recent AI advances. ? (?) introduced a more explicit test of function
composition, and found that only a small portion of tested recurrent neural
networks converged to compositional solutions in computing the outputs of
composite lookup tables. Previous studies have also evaluated model
performance alongside empirical investigations, including explorations of how
humans learn structured visual concepts by reasoning about how parts compose
(?, ?, ?). More recently, ? (?) showed that humans excel at composing
functions in instruction learning, and demonstrated that a standard neural
network-based model can be optimized to exhibit human-like behavior through a
meta-learning procedure that encourages compositionality. Despite varying
levels of modeling success reported in previous work on compositional function
learning, none of them systematically studied the learning of functions and
their interactions in humans and machines.

Towards this end, we proposed a learning paradigm suitable for testing both
humans and models on their compositional function learning skills, with a
special focus on the variety of function interactions. Extending the
experimental framework from ? (?), we investigated whether participants could
learn visual functions as transformations of cartoon cars moving in and out of
factory units with minimal input-output exposure. For each prompted input car,
participants were evaluated on zero-shot function composition in feeding,
counter-feeding, bleeding, and counter-bleeding conditions. Our experiment
demonstrated that humans efficiently generalize from newly learned single
functions to their compositions, achieving consistently high levels of
accuracy across different orderings of visual functions, contrary to previous
linguistic theories on human learning biases in function interactions. Our
results suggested that humans are sensitive to contextual changes during
function composition, generating different outputs based on the order of
operations.

Following ? (?), we trained a neural network through meta-learning for
compositionality (MLC) in order to learn functions and their compositional
interactions. When comparing model performance directly to behavioral data on
the same learning task, we found that a standard Transformer (?, ?), without
any explicit tooling for symbolic reasoning, can be trained to perform our
function composition task at near-human accuracy. Additionally, we
demonstrated by fine-tuning on behaviorally-informed data distributions (?, ?,
?), model generations further captured nuanced error patterns observed in
human-generated function outputs. Extending previous work on compositional
function learning, our study presented empirical data on context and order-
sensitive visual function composition. Side-by-side evaluations of human and
machine performance illustrate that generic neural network models can learn-
to-learn function compositions through behaviorally-guided meta-learning.

## Behavioral experiment

Figure 2: Stimuli and experimental procedure. (A) Car stimuli are structured
as trees: a car part like the window is a child node of the car body, and it
has two nodes denoting its type and color. All functions are defined as tree
edits that add, remove or modify nodes of the car tree. (B) Participants learn
3 functions $A$, $B$ and $C$ satisfying a set of 4 interaction relations.
Arrows denote the order of function application (see concrete examples in
Fig.4). (C) During the training period, participants saw 9 cars moving through
each factory unit representing a different function. If a car is a valid input
to the function, it will come out of the unit with changes reflecting the
underlying function. Cars remain unchanged if they are invalid inputs. (D)
Participants were first asked to generate the correct output based on each
prompted input car and factory unit. (E) For each of the interaction types,
represented by the relevant factory units and the order of their application,
participants were asked to generate the correct output car for 8 different
input cars. Figure 3: Behavioral results. (A) Generation accuracy by
interaction type. Performance did not differ significantly across different
conditions. We also did not observe higher performance in F/CBL trials over
CF/BL trials (maximum utilization bias), nor F/BL over CF/CBL trials
(transparency bias). (B) Proportion of each error type over all incorrect
generations by interaction type.

We retrofitted the task design introduced by ? (?) with an extended space of
functions and features suitable for testing both humans and models in various
interaction conditions. Specifically, participants took part in an online
experiment titled “car assembly game” in which they played the role of
production line workers in a car factory. The task was to assemble cars based
on the knowledge of the different factory units installed on each assembly
line.

#### Stimuli generation.

The main class of stimuli consisted of programmatically generated cartoon cars
represented as tree-like objects (Fig.2A). Each car is a tree with its root
node being the car body. The car body can have up to 3 different child nodes
denoting the window, tires or lights respectively. Each car part also has two
child nodes representing its type and color. There are 3 possible types for
each car part, reflecting different part shapes, and 7 possible, colorblind
safe colors, including a “no color” option. This defines a space of 375,000
distinct car objects.

All functions are defined as tree operations on the car objects. For example,
a function describing the action “adding an oval shaped window to a car
without any existing window” can be written as

if  _not HAS(car.body, window)_ then

ADDCHILD(car.body, CarNode(window, style=1, color=None)).

end if

Other possible edits to the car are remove by deleting a specific child node
of the car body, paint by changing the color of a child node, and editpart by
changing the type of a child node. We limit the scope of the functions
considered in this experiment to target only one child node of the car body at
a time. The condition of all function applications are limited to targeting
the same part of the car in which the function transformation takes place. The
resulting functional space consisted of 1,081 distinct functions.

#### Task procedure.

The experiment was split up into a training stage and a testing stage. During
the training stage, participants were familiarized with 3 individual functions
each represented by a different factory unit ($A$, $B$ or $C$). The functions
were chosen to satisfy a set of pairwise interaction relations (Fig.2B) based
on the order of application. The pair $AB$ always represents a feeding
relationship, with its reverse $BA$ representing a counter-feeding
relationship; the pair $CB$ always satisfies a bleeding relation, with $BC$
representing its reverse (see example trials in Fig.4). Eight possible
function triplets were pre-generated, and each participant was randomly
assigned one of the sets.

Participants observed 9 unique cars moving through each factory unit on a
conveyor belt (Fig.2C). If a given car was a valid input to the function, it
emerged from the unit with one part transformed; otherwise, the car remained
unchanged. After the presentation of exemplars for each factory unit, a short
test followed to determine whether participants inferred the expected
underlying single function. Specifically, we asked them to assemble the
correct output car for each of 3 prompted input cars using the game interface
(Fig.2D). All example input-output pairs remained on screen throughout this
period, and participants were informed of the correctness of their
generations. For each input car, participants were given up to 3 opportunities
to configure the correct output.

Finally, participants were asked to generate the correct output for each
prompted input car based on two different factory units shown in consecutive
orders (Fig.2E). No feedback was provided during this phase. For each of the
interaction pairs $AB$, $BA$, $CB$ and $BC$, participants completed a block of
8 output generations sequentially. The order of each block was randomized for
each participant, and the prompted input cars were also presented in a
randomized order. Throughout the testing stage, participants had access to all
previously seen examples of each relevant function.

#### Participants.

We recruited participants via Amazon Mechanical Turk for the online
experiment. To recruit high quality participants, we first conducted a pre-
experiment survey consisting of 10 simple attention-checking questions about
the car assembly game setup. Of the 196 workers who responded to the survey,
we invited those who scored over 90% ($n=117$) to participate in the main
experiment. Participants completed the main task within an hour and 30
minutes, and were compensated $14.00, plus up to $2.00 of performance-based
bonus ($2.00 $\times$ overall accuracy) upon the completion of the experiment.

### Behavioral results

Figure 4: Examples of human and model output for each interaction type.
Individual functions $A$, $B$ and $C$ are shown on the left. Prompted input
cars (top rows) are shown alongside different output generations by humans,
the base MLC model, and the fine-tuned MLC. The number in the top-left corner
reflects the count (or percentage for model samples) of generation for each
car. Correct outputs are marked by $*$; erroneous generations are underlined
with colors corresponding to the error types (green: function mismatch;
purple: input copying; grey: feature mismatch).

The first analysis examined how people performed on few-shot learning of
individual, isolated functions. Specifically, during the testing stage,
participants generated the expected responses for most of the prompted inputs
to individual (non-compositional) functions ($M=95.4\%$, $SEM=0.010$). The
three input cars for each single function were analyzed based on three groups:
familiar, novel, and identity. Cars in the familiar group consisted of example
inputs seen during the training stage, which checked for attention lapses and
familiarized participants with the generation interface. The average accuracy
was $95.2\%$ ($SEM=0.012$) for this group. The novel group contained novel
cars that are valid inputs to the considered function, and the average
accuracy was $95.7\%$ ($SEM=0.012$) for this group. Finally, identity items
were invalid inputs accepted by the functions; therefore, the expected output
generation for each input in this group should be an identical copy.
Participants were able to identify conditions in which the input is invalid
with an average accuracy of $95.4\%$ ($SEM=0.013$) for this group. Together,
high accuracy on all three groups suggests sufficient learning of the single
functions on which participants were trained.

The second analysis examined how people reasoned about function composition
under interaction. During test, there were four consecutive blocks, each
representing one of the feeding (F), counter-feeding (CF), bleeding (BL), and
counter-bleeding (CBL) interaction conditions (Fig. 3A). Participants achieved
an average accuracy of $86.8\%$ ($SEM=0.019$) across all trials and
conditions. Additionally, we found that participants performed comparably well
across interaction conditions, with an average accuracy of $85.8\%$
($SEM=0.028$) for the feeding trials, $86.3\%$ ($SEM=0.027$) for counter-
feeding, $86.3\%$ ($SEM=0.024$) for bleeding and $88.8\%$ ($SEM=0.020$) for
counter-bleeding. Using test conditions as fixed effects while accounting for
the random effect of participant ID, a mixed effects logistic model on
generation correctness was performed. The analysis did not reveal a
significant effect for each of the interaction conditions
($\beta_{BL}=-1.609,SEM=1.111,p=0.148$;
$\beta_{CBL}=-1.904,SEM=1.063,p=0.073$; $\beta_{CF}=1.109,SEM=1.429,p=0.438$).

We also evaluated whether biases suggested in past linguistic studies are
observed in the current study in visual function learning. ? (?) proposed the
maximum utilization bias, whereby feeding and counter-bleeding are easier to
learn because both functions apply. In comparison, bleeding and counter-
feeding are predicted to be harder because one function cannot apply due to a
lack of context. A paired samples t-test was performed on the average accuracy
for a participant for feeding/counter-bleeding trials versus for
bleeding/counter-feeding trials. The results did not suggest an active maximum
utilization bias ($t(116)=0.488$, $p=0.627$). ? (?) also discussed a
transparency bias that favors instead the more transparent interactions of
feeding and bleeding, over the more opaque interactions concerning functions
that do not apply when it seems like it should (counter-feeding) or apply when
it seems like it should not (counter-bleeding). Again, we did not find a
significant difference in generation performance between feeding/bleeding
trials and counter-feeding/counter-bleeding trials ($t(116)=-0.951$,
$p=0.343$). Overall, our results did not find any participant preferences for
a particular set of function interactions in accordance with either linguistic
theories.

#### Error analysis

For each composition of two functions, we computed the key transformation
expected from input to output. For instance, composing both functions $A$ and
$B$ results in the addition of a green oval-shaped window (Fig. 4). The
majority of human-generated output cars were correct, and reflected the key
transformation expected in all interaction conditions. The majority of
incorrect generations fall into three main error types observed in the
behavioral data (Fig.3B). The most frequent type of error, which we term
function mismatch, describes the scenario in which a participant fails to
reflect the key transformation in their generation, by either applying only
one of the functions to the input or by reversing the order of function
application. For instance, participants sometimes only applied the first
function to the input, and ignored the second function in the feeding
condition, leaving the newly attached window unpainted (left-most column,
green underlines in Fig. 4). There were also cases where participants only
applied the second function, as seen in the bleeding examples in Fig. 4 of
cars that had a green window rather than no window. In the counter-feeding
example of Fig. 4, some participants erroneously applied function $A$ before
$B$, resulting in a green window in their generations. Another type of error
occurs when the participant simply makes an identical copy of the input (see
examples in Fig. 4, purple underlines). In the current task setup, input-
copying is sometimes indistinguishable from function mismatch errors. For
example, in the counter-feeding examples in Fig. 4 that are purple-underlined,
it is unclear whether participants intended to explicitly copy the input as a
strategy, or whether they tried applying only the first function, ending up
with an identical copy due to input invalidity. Finally, in the feature
mismatch scenario, participants correctly reflected the key transformation but
made a mistake in a part of the car irrelevant to the functions in Fig. 4,
grey underlines).

## Composing functions with meta-learning

To develop a computational account of human behavior in the function
composition task, we trained a neural network for learning compositional
functions following the meta-learning for compositionality approach (MLC) set
forth by ? (?). Similar to the motivation in ? (?), instead of modeling the
learning process, we aim to model human inductive biases by forming a
generative process over systems of functions (equivalent to a Bayesian prior)
and using samples from this process as meta-learning episodes for the neural
network. We then further enrich the network with additional behavioral nuance
by fine-tuning the model on behaviorally-guided data distributions (?, ?, ?).

Figure 5: Model schematics and training details.

#### Model description.

A schematic of the MLC model structure is in Fig. 5A. The MLC model uses a
standard sequence-to-sequence Transformer optimized to generate the correct
output in response to a novel query based a set of support examples (?, ?). To
process the study examples and the queries, an encoder network takes as input
a sequence of strings each representing the input car, the function handle and
the corresponding output car for each support example (e.g.,
$\text{car}_{in}\rightarrow fA\rightarrow\text{car}_{out}$), and only the
input car and function handle(s) for each query (e.g.,
$\text{car}_{in}\rightarrow fAfB\rightarrow$). The tree structure of each car
object is flattened into a string representation to form each
$\text{car}_{in}$ and $\text{car}_{out}$ (see an example in Fig.2A), before
concatenation with function handles indicating the participating functions and
the “$\rightarrow$” separator in between. Each query is duplicated and paired
with each of the $m$ study examples per episode forming $m$ input sequences,
which are processed by the shared encoder into $m$ separate latent embeddings.
The set of latent embeddings are then each combined with an index embedding
indicating its original place in the support set, before being processed
together by a decoder network that generates the output sequence
$\text{car}_{out}$. The encoder and the decoder are both 2-layer Transformers
with 8 attention heads in each layer; additional training hyper-parameters are
set to default values used in ? (?).

#### Base training.

To simulates the level of adult skills on composing functions when coming into
the experiment, we train the MLC model via meta-learning (?, ?), encouraging
the model to develop task-general compositional skills through a series of
episodes that each constitutes a different function composition task. The
objective of each task is to generate the correct output of a novel
composition of two functions, given the input and a limited set of support
input-output examples for each function. Each episode consisted of (1) support
input-output pairs of two functions $A$ and $B$; (2) A set of query inputs for
the single functions $A$ and $B$, as well as the compositions $AB$ and $BA$
(see Fig.5B). This setup is comparable to the behavioral experiment, where
participants were only trained on two functions and tested on their
compositions.

To form the episodes used in base training, we first divided all possible
1,081 car-edit functions into a training set and a held-out validation set
(with no individual functions appearing in both). We also withheld all
functions used in the behavioral experiment from the training. To generate
each training episode, we randomly sampled two functions (arbitrarily labeled
$A$ and $B$) from the training distribution that either satisfy a
feeding/counter-feeding relationship or a bleeding/counter-bleeding
relationship. For each function, we sampled 6 valid support input-output
examples, and 4 invalid examples (input and output are identical). To train
the model on generalizing to both new input of learned single functions, as
well as novel function compositions and interactions, we queried the model
with (1) novel inputs for each single function (e.g.,
$\text{car}_{in}\rightarrow fA\rightarrow$), of which 2 are valid and 2 are
invalid inputs; (2) inputs to function compositions (e.g.,
$\text{car}_{in}\rightarrow fAfB\rightarrow$), two for $fAfB$, and two for
$fBfA$. All query items were randomly sampled from all possible inputs to the
relevant function(s) given validity constraint. We generated 50,000 training
episodes in total, half of which contained feeding/counter-feeding function
pairs and the other contained bleeding/counter-bleeding functions. With a
batch size of 5 episodes, the model parameters were optimized via minimizing
the average cross-entropy loss per output token over 50 epochs. Over the
course of training, the model must learn to represent single functions based
on the support examples, and to compose two functions according to the order
of function application, all without explicit compositional supervision.

#### Behaviorally-informed fine-tuning.

To capture additional behavioral nuance, we fine-tuned the MLC model on two
data distributions reflective of the human experience. The first data
distribution $H$ was formed using raw human data. We used human responses on 4
out of the 8 total function triplets assigned to participants in the
behavioral experiment for additional training, and reserved the rest for
evaluation. Each episode in $H$ was structured identically to the base
training episodes, except that support input-output pairs and queried items
were replaced with the ones used in the behavioral experiment. Since
participants were trained on 3 single functions, each participant’s data was
used to form two separate episodes, one for feeding/counter-feeding trials,
and the other for bleeding/counter-bleeding trials. As a result, the $H$
distribution contained 110 episodes, each with 18 support examples (9 for each
function), 6 single function queries (3 for each function) and 16 composition
queries (8 for each function composition). Additionally, we created a
synthetic data distribution $S$ containing 2,000 episodes that is human
behaviorally-guided. Specifically, we modified the generative process for
creating the base training distributions such that noise was injected into the
output sequences to the queries by flipping the function order with a small
probability $p_{flip}$. This simple manipulation creates similar error
patterns in output generations like the ones underlined in green and purple
shown in Fig. 4. Together, the model parameters were updated to predict the
raw human generations in $H$ and noisy outputs in $S$ during the fine-tuning
stage, instead of the ground-truth sequence.

### Modeling results

Using the reserved validation function set, we created a set of 3,000 base
validation episodes. At the conclusion of base training, the MLC model
generated the correct output sequence for each query at an average of $97.9\%$
accuracy across query types and different random initializations
($SEM=0.037$). When queried on single functions, the model was able to make
near-perfect generalizations ($M=98.1\%$, $SEM=0.004$). When divided into
interaction types, validation accuracy on function compositions remained
consistently high ($M_{F}=97.6\%$, $SEM_{F}=0.007$; $M_{CF}=98.0\%$,
$SEM_{CF}=0.004$; $M_{BL}=97.3\%$, $SEM_{BL}=0.004$; $M_{CBL}=97.1\%$,
$SEM_{CBL}=0.005$).

To directly compare the generalization behavior of the model with humans, we
evaluated generations from MLC models on the set of experimental trials using
the 4 held-out function triplets. Model accuracy results are summarized in
Fig. 6. We observed slightly lower overall accuracy from the MLC model with
only base training. However, our primary modeling goal here is not to train a
model that surpasses humans on these trials but to provide a comprehensive
account of human compositional generalization behavior. Reviewing the examples
of model samples shown in Fig. 4, we observe that the base MLC model can
produce the correct output most of the time, but the samples do not reflect
the variety of human-like error patterns. With additional behavioral guidance
by fine-tuning on the $H$ and $S$ distributions, we observed an improved
account of human behavior in terms of both accuracy and average log-likelihood
for held-out human data (Fig. 6). When fine-tuned with only human-generated
data ($H$), with a minimal number of training episodes compared to the base
training set, the MLC model already demonstrated an improved account of held-
out data in terms of log-likelihood. When MLC model was provided with
additional guidance from both $H$ and $S$, it performed the best on held-out
data and exhibited an overall improvement in log-likelihood over the base MLC
model. Qualitatively, examples generated by the fine-tuned MLC model show a
more diverse set of error patterns mirroring human data Fig. 4, demonstrating
the effectiveness of the additional behavioral enrichment.

Figure 6: Modeling results. (Column 1) Model goodness of fit for predicting
human generated examples. For each model version, the overall average log-
likelihood per human generated output car is in the first column. (Column 2-6)
Human vs. model accuracy by interaction type on held-out experiment trials.
Bolding indicates best performance.

## Discussion

To study zero-shot visual function composition in both humans and machines, we
designed a learning paradigm to examine generation behavior under different
interaction conditions. Specifically, through training participants on small
sets of input-output examples, we observed highly accurate generalizations to
novel function inputs. During the test phase, we presented participants with
queries involving the four logical patterns that result from combining pairs
of interacting functions. Contrary to previous work in a different domain
which finds different accuracy across interaction types, our results indicate
consistently high levels of generation accuracy across all conditions. This
suggests humans can adeptly handle contextual shifts when composing functions.
While we do not observe overall mean differences between different interaction
types, we observed that humans make non-random, structural mistakes when
processing two functions consecutively: partial application of functions or a
reversal of function order constitute the main families of errors.

In a side-by-side evaluation of human and machine performance, we found that a
standard sequence-to-sequence Transformer can be trained to compose novel
function pairs via meta-learning on streams of function composition tasks. In
particular, our MLC model was capable of learning both single functions from
limited examples and generalizing to unseen function composites by producing
output sequences to promoted inputs at a near-human accuracy. Additionally, we
used error patterns uncovered in behavioral data analysis to form the basis of
comparison with model error patterns, and further revealed how distant models
are from humans in understanding function interactions. To decrease the gap in
human-model generalization behavior, we conducted an additional experiment by
fine-tuning the model on behaviorally-informed data distributions, and the
results indicated that the model can be further improved to provide better
accounts of human behavior within the existing framework.

Although we found relatively uniform levels of performance across conditions
in both behavioral and computational analyses, the various learning
trajectories of the 4 function interaction types during base model training
might shed light on how different processes are mastered over the course of
development (Fig.5C). While validation performance ultimately converged to
high levels at the end of training, feeding was clearly the hardest for the
model to master, followed by bleeding and counter-bleeding, with counter-
feeding learned most early and easily. This result was expected as feeding
always requires the model to apply two function transformations when the input
is valid, while in other cases only one function application takes place at
times. However, it is interesting to note that counter-feeding, or the
application of only the second function in a novel composite, is also
demonstrated to be an infant behavior before the onset of function composition
skills (?, ?). Extending the current study to evaluate populations at various
developmental stages might help elucidate if certain function interaction
types are indeed harder to grasp.

As a future step, sequences currently used by our model can be readily
translated into other representational formats such as raw images, which are
what human participants observed, or tree structures, which do not enforce
arbitrary ordering of car parts and features. More direct comparisons between
human and model behavior can offer additional insight on how function
interactions are learned and processed, and may further inform how to build
computational systems with more compositional, human-like forms of learning.

## Acknowledgements

This work was supported by NSF Award 1922658 NRT-HDR: FUTURE Foundations,
Translation, and Responsibility for Data Science. Yanli Zhou was supported by
the Meta AI Mentorship Program. We thank Solim LeGris and Cindy Luo for
helpful discussions of this manuscript.

## References

  * Bastings, Baroni, Weston, Cho, KielaBastings et al. Bastings, J., Baroni, M., Weston, J., Cho, K., Kiela, D.  (2018).  Jump to better conclusions: SCAN both left and right.  _EMNLP 2018 - 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Proceedings of the 1st Workshop_ , 47–55.
  * Curry FeysCurry Feys Curry, H. B., Feys, R.  (1958).  _Combinatory logic_ (Vol. 1).  North-Holland Amsterdam.
  * Hospedales, Antoniou, Micaelli, StorkeyHospedales et al. Hospedales, T., Antoniou, A., Micaelli, P., Storkey, A.  (2022).  Meta Learning in Neural Networks: A Survey.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_.
  * KiparskyKiparsky Kiparsky, P.  (1968).  Linguistic universals and linguistic change.  In E. Bach R. Harms (Eds.), _Universals in linguistic theory_ (pp. 170–202).  Holt, Rinehart, and Winston.
  * KiparskyKiparsky Kiparsky, P.  (1971).  Historical linguistics.  In W. O. Dingwall (Ed.), _A survey of linguistic science_ (pp. 576–642).  College Park: University of Maryland Linguistics Program.
  * Lake BaroniLake Baroni Lake, B. M., Baroni, M.  (2018).  Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks.  In _International Conference on Machine Learning (ICML)_ (pp. 2873–2882).
  * Lake BaroniLake Baroni Lake, B. M., Baroni, M.  (2023).  Human-like systematic generalization through a meta-learning neural network.  _Nature_ , _623_(7985), 115–121.  doi: 10.1038/s41586-023-06668-3
  * Liška, Kruszewski, BaroniLiška et al. Liška, A., Kruszewski, G., Baroni, M.  (2018).  _Memorize or generalize? searching for a compositional rnn in a haystack._
  * McCoy GriffithsMcCoy Griffiths McCoy, R. T., Griffiths, T. L.  (2023).  _Modeling rapid language learning by distilling bayesian priors into artificial neural networks._
  * Overlan, Jacobs, PiantadosiOverlan et al. Overlan, M. C., Jacobs, R. A., Piantadosi, S. T.  (2017).  Learning abstract visual concepts via probabilistic program induction in a Language of Thought.  _Cognition_ , _168_ , 320–334.
  * Piantadosi AslinPiantadosi Aslin Piantadosi, S. T., Aslin, R.  (2016).  Compositional reasoning in early childhood.  _PLoS ONE_ , _11_ , 1–12.
  * Piantadosi, Palmeri, AslinPiantadosi et al. Piantadosi, S. T., Palmeri, H., Aslin, R.  (2018).  Limits on Composition of Conceptual Operations in 9-Month-Olds.  _Infancy_ , _23_ , 310–324.
  * Piantadosi, Tenenbaum, GoodmanPiantadosi et al. Piantadosi, S. T., Tenenbaum, J. B., Goodman, N. D.  (2012).  Bootstrapping in a language of thought: A formal model of numerical concept learning.  _Cognition_ , _123_ , 199–217.
  * Ruis, Andreas, Baroni, Bouchacourt, LakeRuis et al. Ruis, L., Andreas, J., Baroni, M., Bouchacourt, D., Lake, B. M.  (2020).  A Benchmark for Systematic Generalization in Grounded Language Understanding.  In _Advances in Neural Information Processing Systems 33._
  * SchönfinkelSchönfinkel Schönfinkel, M.  (1967).  On the building blocks of mathematical logic.  _From Frege to Gödel_ , 355–366.
  * Valvoda, Saphra, Rawski, Williams, CotterellValvoda et al. Valvoda, J., Saphra, N., Rawski, J., Williams, A., Cotterell, R.  (2022, October).  Benchmarking compositionality with formal languages.  In N. Calzolari et al. (Eds.), _Proceedings of the 29th international conference on computational linguistics_ (pp. 6007–6018).  Gyeongju, Republic of Korea: International Committee on Computational Linguistics.  Retrieved from https://aclanthology.org/2022.coling-1.525
  * Vaswani et al.Vaswani et al. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … Polosukhin, I.  (2017).  Attention is all you need.  _Advances in neural information processing systems_ , _30_.
  * Zhou, Feinman, LakeZhou et al. Zhou, Y., Feinman, R., Lake, B. M.  (2024).  Compositional diversity in visual concept learning.  _Cognition_ , _244_ , 105711.  doi: https://doi.org/10.1016/j.cognition.2023.105711


The transport coefficients in the expressions of the turbulent fluxes are
determined by the statistical properties and dynamical behavior of the
turbulence. We performed a multiple-scale renormalized perturbation expansion
analysis on the MHD turbulence. We systematically obtained the expressions for
the turbulent fluxes in the mean-field equations, including the analytical
expressions of the transport coefficients. On the basis of these analytical
expressions, the coefficients are represented by appropriate one-point
turbulent statistical quantities such as the turbulent energy, kinetic,
current, and cross helicities. By simultaneously considering and solving the
transport equations of the statistical quantities, as well as the mean-field
equations, a self-consistent system of model equations was constructed. In
this framework, the turbulent transport coefficients in the mean-field
equations are determined through the transport equations of the turbulent
statistical quantities. These transport equations include the production,
dissipation, and transport rates of the turbulent statistical quantities,
which are subject to the mean-field inhomogeneities. As such, this turbulence
model can self-consistently determine the evolutions of the mean- and
fluctuation-fields. In this sense, the present approach provides a new
framework that can treat global structure formation in extremely strong
turbulence far beyond the previous heuristic turbulence modeling approach
(Yokoi, 2018c).

Acknowledgments

The author would like to thank Pat Diamond for inviting him to write this
article on the topic of the cross-helicity effects. This work was supported by
the Japan Society of the Promotion of Science (JSPS) Grants-in-Aid for
Scientific Research JP18H01212. Part of this work was done during his stay at
Nordic Institute for Theoretical Physics (NORDITA) in May and June 2022. The
author also would like to thank the Isaac Newton Institute for Mathematical
Sciences, Cambridge, for support and hospitality during the programme DYT2
where work on this paper was undertaken. This work was supported by EPSRC
grant no EP/R014604/1.

## Declarations

The author has no conflict of interest.

## References

  * Balbus & Hawley (1998) Balbus, S. A. and Hawley, J. F. “Instability, turbulence, and enhanced transport in accretion disks,” Rev. Mod. Phys. 70, 1–53 (1998).
  * Brandenburg (2016) Brandenburg, A. “Stellar mixing length theory with entropy rain,” Astrophys. J. 832, 6 (2016).
  * Browning (2008) Browning, M. K. “Simulation of dynamo action in fully convective stars,” Astrophys. J. 676, 1262–1280 (2008). doi:10.1086/527432
  * Dobrowolny, Mangeney & Veltri, P (1980a) Dobrowolny, M., Mangeney, A. and Veltri, P. “Fully developed anisotropic hydromagnetic turbulence in interplanetary space,” Phys. Rev. Lett. 45, 144 (1980a).
  * Dobrowolny, Mangeney & Veltri (1980b) Dobrowolny, M., Mangeney, A., and Veltri, P. “Properties of magnetohydrodynamic turbulence in the solar wind,” Astron. Astrophys. 83 (1980b), 26.
  * Donati, Morin, Petit, et al. (2008) Donati, J.-F., Morin, J., Petit, P., Delfosse, X., Forveille, T., Aurières, F., Cabanac, R., Dintrans, B., Fares, R., Gastine, T., Jardine, M. M., Lignières, F., Paletou, F., Ramires Velez, J. C., and Thèado, S. “Large-scale magnetic topologies of early M dwarfs,” Mon. Not. R. Astron. Soc. 390, 545–560 (2008). doi:10.1111/j.1365-2966.2008.13799.x
  * Duarte, Wicht, Browning, et al. (2016) Duarte, L. D. V., Wicht, J, Browning, M. K. and Gastine, T. “Helicity inversion in spherical convection as a means for equatorward dynamo wave propagation,” Mon. Not. Roy. Astron. Soc. 456, 1708–1722 (2016).
  * Fujita (1997) Fujita, T. “Effects of negative magnetic shear in fusion plasma confinement (1) A review of experimental studies,” J. Plasma Fusion Res. 73, 549, (1997).
  * Fujita, Oikawa, Suzuki, et al. (2001) Fujita, T., Oikawa, T., Suzuki, T., Ide, S., Sakamoto, Y., Koide, Y. Hatate, T., Naito, O., Isayama, A., Hayashi, N., Shirai, H. “Plasma equilibrium and confinement in a tokamak with nearly zero central current density in JT-60U,” Phys. Rev. Lett. 87, 245001 (2001).
  * Green, Vlaykov, Mellado, et al. (2020) Green, G., Vlaykov, D. G., Mellado, J. P., and Wilczek, M. “Resolved energy budget of superstructures in Reyleigh–Bénard convection,” J. Fluid Mech. 887, A21 (2020). doi:10.1017/jfm.2019.1008
  * Hamba (2006) Hamba, F. “Euclidean invariance and weak-equilibrium condition for the algebraic Reynolds stress model,” J. Fluid Mech. 569, 399–408 (2006). doi:10.1017/S0022112006003041
  * Hamba & Sato (2008) Hamba, F. and Sato, H. “Turbulent transport coefficients and residual energy in mean-field dynamo theory,” Phys. Plasmas 15, 022302 (2008). doi:10.1063/1.2839767
  * Hanasoge, Duvall & Sreenivasan (2012) Hanasoge, S. M., Duvall, T. L., and Sreenivasan, K. R. 2012. “Anomalously weak solar convection,” PNAS, 109, 11928 (2012).
  * Hanasoge, Gizon & Sreenivasan (2016) Hanasoge,S., Gizon, L., and Sreenivasan, K. R. “Seismic sounding of convection in the Sun,” Annu. Rev. Fluid Mech. 48, 191–217 (2016). doi:10.1146/annurev-fluid-122414-034534
  * Heinonen, Diamond, Katz, et al. (2023) Heinonen,R. A., Diamond, P. H., Katz, M. F. D., and Ronimo, G. E. “Generation of momentum transport in weakly turbulent $\beta$-plane magnetohydrodynamics,” Phys. Rev. E 107, 025202 (2023). doi:10.1103/PhysRevE.107.025202
  * Higashimori, Yokoi & Hoshino (2013) Higashimori, K., Yokoi, N., and Hoshino, M. “Explosive turbulent magnetic reconnection,” Phys. Rev. Lett. 110, 255001 (2013). doi:10.1103/PhysRevLett.110.255001
  * Hossain, Gray, Pontius Jr., et al. (1995) Hossain, M. Gray, P.C., Pontius Jr. D.H., Matthaeus, W. H. and Oughton, S. “Phenomenology for the decay of energy-containing eddies in homogeneous MHD turbulence,” Phys. Fluids 7 2886 (1995).
  * Krause & Rädler (1980) Krause, F. and Rädler, K.-H. Mean-Field Magnetohydrodynamics and Dynamo Theory (1980, Pergamon Press, Oxford).
  * Loureiro, Uzdensky, Schekochihin, et al. (2009) Loureiro, N. F., Uzdensky, D. A., Schekochihin, A. A., and Cowley, T. A. “Turbulent magnetic reconnection in two dimensions,” Mon. Not. Roy. Astron. Soc. 399, L146–L150 (2009).
  * Matthaeus, Oughton, Pontius Jr. et al. (1994) Matthaeus, W. H., Oughton, S., Pontius Jr. D. H., and Zhou, Y. “Evolution of energy containing turbulent eddies in the solar wind,” J. Geophys. Res. 99, 19267 (1994).
  * Mizerski, Yokoi & Brandenburg (2023) Mizerski, K. A., Yokoi, N., and Brandenburg, A. “Cross-helicity effect on $\alpha$-type dynamo in non-equilibrium turbulence,” submitted to J. Plasma Phys. pp. 1-24 (2023). doi:10.5281/zenodo.7683615
  * Moffatt (1978) Moffatt, K. Magnetic Field Generation in Electrically Conducting Fluids (1978, Cambridge Univ. Press, Cambridge).
  * Moffatt & Dormy (2019) Moffatt, K. and Dormy, E. Self-Exciting Fluid Dynamos (2019, Cambridge Univ. Press, Cambridge).
  * Morin, Donati, Petit, et al. (2008) Morin, J., Donati, J.-F., Petit, P., Delfosse, X., Forveille, T., Albert, L., Auriére, M., Cabanac, R., Dintrans, B., Fares, R., Gastine, T., Jardine, M. M., Lingnières, Paletou, F., Ramirez Velez, J. C., and Thèado, S. “Large-scale magnetic topologies of mid M dwarfs,” Mon. Not. R. Astron. Soc. 390, 567–581 (2008). doi:10.1111/j.1365-2966.2008.13809.x
  * Nowak, Kowal & Falceta-Goncalves (2022) Nowak, N., Kowal, G., and Falceta-Goncalves, D. A. “Generation and effects of electromotive force in turbulent stochastic reconnection,” Phys. Plasmas 28, 062310 (2022).
  * Parker (1955) Parker, E. N. “Hydromagnetic dynamo models,” Astrophys. J. 121, 293–314 (1955).
  * Parker (1957) Parker, E. N. “Sweet’s mechanism for merging magnetic fields in conducting fluids,” J. Fluid Mech. 62, 509–520 (1957).
  * Pipin & Yokoi (2018) Pipin, V. and Yokoi, N. “Generation of a large-scale magnetic field in a convective full-sphere cross-helicity dynamo,” Astrophys. J. 859, 18-1–11, (2018). https://doi.org/10.3847/1538-4357/aabae6
  * Pouquet & Yokoi (2022) Pouquet, A. and Yokoi, N. “Helical fluid and (Hall)-MHD turbulence: a brief review,” Phil. Trans. R. Soc. A380, 20210087 (2022). doi.org/10.1098/rsta.2021.0087
  * Pouquet, Frisch & Léorat (1976) Pouquet, A., Frisch, U., and Léorat, J. “Strong MHD helical turbulence and the nonlinear dynamo effect,” J. Fluid Mech. 77, 321–354 (1997).
  * Proxauf (2021) Proxauf, B. “Observations of large-scale solar flows,” Dissertation, Georg-August Universität Göttingen arXiv: 2106.07251 (2021). http://doi.org/10.53846/goediss-8502
  * Rahbarnia, Brown, Clark, et al. (2012) Rahbarnia, K., Brown, B. P., Clark, M. M., Kaplan, E. J., Norberg, M. D., Rasmus, A. M., Taylor, N. Z., Forest, C. B., Jenko, F., and Limone, A. “Direct observation of the turbulent emf and transport of magnetic field in a liquid sodium experiment,” Astrophys. J. 759, 80-1–6 (2012). doi:10.1088/0004-637X/759/2/80
  * Ranjan & Davidson (2023) Ranjan, A. and Davidson, P. “The spatial segregation of kinetic helicity in geodynamo simulations,” Chap. 16 in Helicities in Geophysics, Astrophysics, and Beyond, K. Kuzanyan, N. Yokoi, M. Georgoulis, and R. Stepanov (Eds.) AGU Books (2023, Wiley, New York).
  * Rast (1998) Rast, M. P. “Compressible plume dynamics and stability,” J. Fluid Mech. 369, 125–149 (1998).
  * Rogachevskii & Kleeorin (2001) Rogachevskii, I. and Kleeorin, N. “Nonlinear turbulent magnetic diffusion and mean-field dynamo,” Phys. Rev. E 64, 056307 (2001).
  * Schumacher & Sreenivasan (2020) Schumacher, J. and Sreenivasan, K. R. “Colloquium: Unusual dynamics of convection in the Sun,” Rev. Mod. Phys. 92, 041001 (2020). doi:10.1103/RevModPhys.92.041001
  * Schüssler & Ferriz-Mas (2003) Schüssler, M. and Ferriz-Mas, A. “Magnetic flux tubes and the dynamo problem,” in Advances in Nonlinear Dynamos A. Ferriz-Mas and M, Nunez (Eds.) pp. 123-146 (2003, Taylor and Francis, London).
  * Shakura & Sunyaev (1973) Shakura, N. I. and Sunyaev, R. A. “Black holes in binary systems: Observational appearance,” Astron. Astrophys. 24, 337-355 (1973).
  * Stribling & Matthaeus (1991) Stribling, T. and Matthaeus, W. H. “Relaxation processes in a low-order three-dimensional magnetohydrodynamics model,” Phys. Fluids B 3, 1848 (1991).
  * Sur & Brandenburg (2009) Sur, S. and Brandenburg, A. “The role of the Yoshizawa effect in the Archontis dynamo,” Mon. Not. Roy. Astron. Soc. 399, 273–280 (2009).
  * Sweet (1958) Sweet, P. A. “The neutral point theory of solar flares,” in IAU Symposium No. 6, Electromagnetic Phenomena in Cosmical Physics, pp. 123–134 (1958).
  * Thiffeault (2001) Thiffeault, J.-L. “Covariant time derivatives for dynamical systems,” J. Phys. A: Math. Gen. 34, 5875–5885 (2001). doi:10.1088/0305-4470/34/29/309
  * Ting, Montgomery & Matthaeus (1986) Ting, A.C. and Montgomery, D., and Matthaeus, W. H. “Turbulent relaxation processes in magnetohydrodynamics," Phys. Fluids 29, 3261 (1986).
  * Tobias (2021) Tobias, S. M. “The turbulent dynamo,” J. Fluid Mech. 912, P1 (2021) doi:10.1017/jfm.2020.1055
  * Vasil, Julien & Featherstone (2021) Vasil, G. M., Julien, K., and Featherstone, A. “Rotation suppresses giant-scale solar convection,” PNAS, 118, 31, e2022518118-1–9 (2021). doi,org/10.1073/pnas.2022518118
  * Widmer, Büchner & Yokoi (2016a) Widmer, F., Büchner, J., and Yokoi, N. “Sub-grid-scale description of turbulent magnetic reconnection in magnetohydrodynamics,” Phys. Plasmas 23, 04231 (2016). doi.org/10.1063/1.4947211
  * Widmer, Büchner & Yokoi (2016b) Widmer, F., Büchner, J., and Yokoi, N. “Characterizing plasmoid reconnection by turbulence dynamics,” Phys. Plasmas 23, 092304 (2016). doi.org/10.1063/1.4962694
  * Widmer, Büchner & Yokoi (2019) Widmer, F., Büchner, J., and Yokoi, N. “Analysis of fast turbulent reconnection with self-consistent determination of turbulence timescale,” Phys. Plasmas 26, 102112 (2019). doi: 10.1063/1.5109020
  * Yokoi (2013) Yokoi, N. “Cross helicity and related dynamo,” Geophys. Astrophys. Fluid Dyn., 107, 114–184 (2013). doi.org/10.1080/03091929.2012.754022
  * Yokoi (2018a) Yokoi, N. “Electromotive force in strongly compressible magnetohydrodynamic turbulence,” J. Plasma Phys. 84, 735840501-1-26 (2018a). doi:10.1017/S0022377818000727
  * Yokoi (2018b) Yokoi, N. “Mass and internal-energy transports in strongly compressible magnetohydrodynamic turbulence,” J. Plasma Phys. 84, 775840603-1-30 (2018b). doi:10.1017/S0022377818001228
  * Yokoi (2018c) Yokoi, N. “Turbulent dynamos beyond the heuristic modeling: Helicities and density variance,” AIP Conf. Proc. 1993, 020010-1-12 (2018). doi.org/10.1063/1.5048720
  * Yokoi (2020) Yokoi, N. “Turbulence, Transport and Reconnection,” pp. 177–265, Chap. 6 in D. MacTaggart, A. Hillier (Eds.), Topics in Magnetohydrodynamic Topology, Reconnection and Stability Theory, CISM International Centre for Mechanical Sciences 591, (2020, Springer, Cham) doi.org/10.1007/978-3-030-16343-3-6
  * Yokoi (2023) Yokoi, N. “Transport in helical fluid turbulence,” Chap. 3 in Helicities in Geophysics, Astrophysics, and Beyond, K. Kuzanyan, N. Yokoi, M. Georgoulis, and R. Stepanov (Eds.) AGU Books (2023, Wiley, New York).
  * Yokoi & Balarac (2011) Yokoi, N. and Balarac, G. “Cross-helicity effects and turbulent transport in magnetohydrodynamic flow,” Journal of Physics: Conference Series 318, 072039-1-10 (2011). doi:10.1088/1742-6596/318/7/072039
  * Yokoi & Brandenburg (2016) Yokoi, N. and Brandenburg, A. “Large-scale flow generation by inhomogeneous helicity,” Phys. Rev. E 93, 033125-1–14 (2016). doi:10.1103/PhysRevE.93.033125
  * Yokoi & Hoshino (2011) Yokoi, N. and Hoshino, M. “Flow–turbulence interaction in magnetic reconnection,” Phys. Plasmas 18, 111208 (2011). doi:10.1063/1.3641968
  * Yokoi & Yoshizawa (1993) Yokoi, N. and Yoshizawa, A. “Statistical analysis of the effects of helicity in inhomogeneous turbulence,” Phys. Fluids A 5, 464-477 (1993).
  * Yokoi, Higashimori & Hoshino (2013) Yokoi, N., Higashimori, K., and Hoshino, M. “Transport enhancement and suppression in turbulent magnetic reconnection: A self-consistent turbulence model,” Phys. Plasmas 20, 122310 (2013). doi.org/10.1063/1.4851976
  * Yokoi, Masada & Takiwaki (2022) Yokoi, N., Masada, Y., and Takiwaki, T. “Modelling stellar convective transport with plumes – I. Non-equilibrium turbulence effect in double-averaging formulation,” Mon. Not. Roy. Astron. Soc. 516, 2718–2735 (2022). doi.org/10.1093/mnras/stac1181
  * Yokoi, Rubinstein, Yoshizawa, et al. (2008) Yokoi, N., Rubinstein, R., Yoshizawa, A., and Hamba, F. “A turbulence model for magnetohydrodynamic plasmas,” J. Turb. 824, 9, 37-1–25 (2008). doi:10.1080/14685240802433057
  * Yokoi, Schmitt, Pipin, et al. (2016) Yokoi, N., Schmitt, D., Pipin, V., and Hamba, F. “A new simple dynamo model for stellar activity cycle,” Astrophys. J. 824, 67 (2016). doi:10.3847/0004-637X/824/2/67
  * Yoshizawa (1984) Yoshizawa, A. “Statistical analysis of the deviation of the Reynolds stress from its eddy-viscosity representation, Phys. Fluids, 27, 1377–1387 (1984).
  * Yoshizawa (1990) Yoshizawa, A. “Self-consistent turbulent dynamo modeling of reversed field pinches and planetary magnetic fields,” Phys. Fluids B, 2, 1589–1600 (1990).
  * Yoshizawa, Yokoi, Itoh, et al. (1999) Yoshizawa, A., Yokoi, N., Itoh, S.-I., and Itoh, K. “Magnetohydrodynamic mechanisms of electric-field transport suppression and plasma-rotation generation, with special reference to tokamak’s reversed-shear confinement,” Phys. Plasmas, 6, 3194–3206 (1999).