above to get a broad overview of our benchmarked models on a range of task
difficulties. 22footnotetext: $h$ is chosen to match the number of parameters
of the reservoir-based methods.

###  6.9 Results

We report the final weighted average data efficiency (WADE) scores on our
benchmark in Table 6.5 and the accuracy results for comparison in Table 6.6.
We include the best elementary reservoir cellular automaton (ReCA) and the
echo state network (ESN) with the same internal state size for each task.

Task ID - Name | Reservoir |  Human subject
---|---|---
ReCA | ESN
1 - Periodic | 0.78 ${\scriptscriptstyle\pm 0.05}$ | 0.74 ${\scriptscriptstyle\pm 0.05}$ | 1.00
2 - Incremental periodic | 0.57 ${\scriptscriptstyle\pm 0.02}$ | 0.72 ${\scriptscriptstyle\pm 0.02}$ | 1.00
3 - Symbol counting | 0.06 ${\scriptscriptstyle\pm 0.01}$ | 0.04 ${\scriptscriptstyle\pm 0.01}$ | 0.11
4 - Pattern counting | 0.12 ${\scriptscriptstyle\pm 0.04}$ | 0.14 ${\scriptscriptstyle\pm 0.03}$ | 0.09
5 - Basic question answering (QA) | 0.31 ${\scriptscriptstyle\pm 0.08}$ | 0.26 ${\scriptscriptstyle\pm 0.03}$ | 0.15
6 - Harder QA | 0.35 ${\scriptscriptstyle\pm 0.11}$ | 0.26 ${\scriptscriptstyle\pm 0.03}$ | 0.12
7 - QA with world def. | 0.32 ${\scriptscriptstyle\pm 0.10}$ | 0.27 ${\scriptscriptstyle\pm 0.04}$ | —
8 - QA with world def. & counting | 0.09 ${\scriptscriptstyle\pm 0.03}$ | 0.14 ${\scriptscriptstyle\pm 0.06}$ | —
9 - Adjective QA | 0.04 ${\scriptscriptstyle\pm 0.03}$ | 0.04 ${\scriptscriptstyle\pm 0.04}$ | —
10 - Adjective QA & counting | 0.04 ${\scriptscriptstyle\pm 0.02}$ | 0.06 ${\scriptscriptstyle\pm 0.02}$ | —
| Fully supervised
| RNN | LSTM | Transformer
1 | 0.28 ${\scriptscriptstyle\pm 0.07}$ | 0.31 ${\scriptscriptstyle\pm 0.04}$ | 0.31 ${\scriptscriptstyle\pm 0.06}$
2 | 0.32 ${\scriptscriptstyle\pm 0.11}$ | 0.49 ${\scriptscriptstyle\pm 0.15}$ | 0.42 ${\scriptscriptstyle\pm 0.16}$
3 | 0.04 ${\scriptscriptstyle\pm 0.02}$ | 0.03 ${\scriptscriptstyle\pm 0.01}$ | 0.05 ${\scriptscriptstyle\pm 0.02}$
4 | 0.10 ${\scriptscriptstyle\pm 0.05}$ | 0.08 ${\scriptscriptstyle\pm 0.03}$ | 0.13 ${\scriptscriptstyle\pm 0.04}$
5 | 0.17 ${\scriptscriptstyle\pm 0.08}$ | 0.16 ${\scriptscriptstyle\pm 0.07}$ | 0.20 ${\scriptscriptstyle\pm 0.06}$
6 | 0.16 ${\scriptscriptstyle\pm 0.07}$ | 0.12 ${\scriptscriptstyle\pm 0.06}$ | 0.24 ${\scriptscriptstyle\pm 0.05}$
7 | 0.17 ${\scriptscriptstyle\pm 0.07}$ | 0.11 ${\scriptscriptstyle\pm 0.06}$ | 0.24 ${\scriptscriptstyle\pm 0.05}$
8 | 0.10 ${\scriptscriptstyle\pm 0.05}$ | 0.05 ${\scriptscriptstyle\pm 0.02}$ | 0.02 ${\scriptscriptstyle\pm 0.01}$
9 | 0.05 ${\scriptscriptstyle\pm 0.03}$ | 0.03 ${\scriptscriptstyle\pm 0.02}$ | 0.02 ${\scriptscriptstyle\pm 0.01}$
10 | 0.05 ${\scriptscriptstyle\pm 0.02}$ | 0.03 ${\scriptscriptstyle\pm 0.02}$ | 0.01 ${\scriptscriptstyle\pm 0.01}$
Table 6.5: Comparison of WADE scores (also shown in parentheses in the legend
in figure 6.5, higher is better) of the best cellular automaton rules [[,
ReCA, ]]yilmazReservoirComputingUsing2014 for each task against an echo-state
network [[, ESN,]]jaegerEchoStateApproach2001, a RNN, a LSTM [124], and a
Transformer [315] with the same number of parameters. The dashed line “—”
indicates that the task was too difficult to complete from memory alone for
the human subject. The dots indicate that the task name is the same as in the
first line. Accuracy scores are also reported in Table 6.6.

Figure 6.5 shows the learning curves for the best reservoir cellular
automaton, as well as the echo-state network, the recurrent neural network
(RNN), LSTM, and the Transformer for all ten tasks. Interestingly, the
reservoir-based models are consistently more efficient learners than the fully
supervised methods, reaching better accuracy in much fewer training steps. For
example, for tasks 5, 6, and 7, the LSTM needs ten times more steps to
approach the accuracy that the reservoir CA model reached in less than 1000
training steps.

Echo-state networks (ESN) and reservoir cellular automata (ReCA) appear to
learn at similar rates, with no clear advantage for one or the other, as each
model outperforms the others in five tasks out of ten. On tasks 2, 4, 8, and
10, the ESN is visibly faster than the ReCA (see curves Figure 6.5), which
explains the ESN’s higher WADE scores even though the ReCA reaches higher
accuracy values after several more training steps.

Task ID - Name | Reservoir
---|---
ReCA | ESN
1 - Periodic | 0.99 ${\scriptscriptstyle\pm 0.01}$ | 1.00 ${\scriptscriptstyle\pm 0.01}$
2 - Incremental periodic | 0.88 ${\scriptscriptstyle\pm 0.01}$ | 0.87 ${\scriptscriptstyle\pm 0.03}$
3 - Symbol counting | 0.80 ${\scriptscriptstyle\pm 0.02}$ | 0.30 ${\scriptscriptstyle\pm 0.02}$
4 - Pattern counting | 0.56 ${\scriptscriptstyle\pm 0.01}$ | 0.59 ${\scriptscriptstyle\pm 0.02}$
5 - Basic QA | 0.81 ${\scriptscriptstyle\pm 0.03}$ | 0.73 ${\scriptscriptstyle\pm 0.03}$
6 - Harder QA | 0.72 ${\scriptscriptstyle\pm 0.04}$ | 0.57 ${\scriptscriptstyle\pm 0.03}$
7 - QA with world def. | 0.66 ${\scriptscriptstyle\pm 0.03}$ | 0.60 ${\scriptscriptstyle\pm 0.03}$
8 - QA with world def. & counting | 0.59 ${\scriptscriptstyle\pm 0.03}$ | 0.56 ${\scriptscriptstyle\pm 0.03}$
9 - Adjective QA | 0.62 ${\scriptscriptstyle\pm 0.04}$ | 0.49 ${\scriptscriptstyle\pm 0.03}$
10 - Adjective QA & counting | 0.54 ${\scriptscriptstyle\pm 0.02}$ | 0.46 ${\scriptscriptstyle\pm 0.03}$
| Fully supervised
RNN | LSTM | Transformer
1 | 0.79 ${\scriptscriptstyle\pm 0.08}$ | 0.68 ${\scriptscriptstyle\pm 0.04}$ | 0.69 ${\scriptscriptstyle\pm 0.04}$
2 | 0.87 ${\scriptscriptstyle\pm 0.00}$ | 0.89 ${\scriptscriptstyle\pm 0.02}$ | 0.88 ${\scriptscriptstyle\pm 0.00}$
3 | 0.33 ${\scriptscriptstyle\pm 0.03}$ | 0.36 ${\scriptscriptstyle\pm 0.03}$ | 0.97 ${\scriptscriptstyle\pm 0.01}$
4 | 0.61 ${\scriptscriptstyle\pm 0.05}$ | 0.61 ${\scriptscriptstyle\pm 0.02}$ | 0.54 ${\scriptscriptstyle\pm 0.03}$
5 | 0.51 ${\scriptscriptstyle\pm 0.02}$ | 0.75 ${\scriptscriptstyle\pm 0.05}$ | 1.00 ${\scriptscriptstyle\pm 0.00}$
6 | 0.51 ${\scriptscriptstyle\pm 0.03}$ | 0.68 ${\scriptscriptstyle\pm 0.07}$ | 1.00 ${\scriptscriptstyle\pm 0.00}$
7 | 0.51 ${\scriptscriptstyle\pm 0.04}$ | 0.64 ${\scriptscriptstyle\pm 0.07}$ | 1.00 ${\scriptscriptstyle\pm 0.00}$
8 | 0.47 ${\scriptscriptstyle\pm 0.05}$ | 0.50 ${\scriptscriptstyle\pm 0.06}$ | 0.98 ${\scriptscriptstyle\pm 0.01}$
9 | 0.43 ${\scriptscriptstyle\pm 0.04}$ | 0.44 ${\scriptscriptstyle\pm 0.03}$ | 0.87 ${\scriptscriptstyle\pm 0.11}$
10 | 0.47 ${\scriptscriptstyle\pm 0.04}$ | 0.49 ${\scriptscriptstyle\pm 0.03}$ | 0.57 ${\scriptscriptstyle\pm 0.04}$
Table 6.6: Comparison of accuracy scores (higher is better) of the ReCA, ESN,
RNN, LSTM, and Transformer models with a similar number of parameters. In
contrast to the results of table 6.5, the fully-supervised models are more
performant when we measure the accuracy, as shown here, but do not necessarily
do as well when we measure the speed of learning, as shown in table 6.5.

Even if they lack slightly in accuracy, as seen for example on the curves of
task 4, the two reservoir-based methods consistently outperform the fully
supervised sequential methods (RNNs and LSTMs) in terms of learning speeds.
This better learning efficiency could be explained by the internal state
structure introduced by the CA rules and the special form of the ESN random
matrix which favors memory retention whereas usual recurrent network
initialization does not — we initialize the weights of the fully trained
models from a uniform distribution $\mathcal{U}(-\sqrt{h^{-1}},\sqrt{h^{-1}})$
in our experiments, where $h$ is the hidden size.

The human subject scores are estimated based on the authors’ performance on
obfuscated versions of the tasks — all input tokens are mapped to random
symbols. These scores may appear surprisingly low in some of these
experiments. Although the task examples presented in Section 6.3.2 seem easy
to understand. We rely heavily on our prior knowledge about the symbols to
understand the patterns — the words in tasks 5–10 or the numbers in tasks 3
and 4. These symbols are randomly remapped in our human experiments, and more
examples, as well as a good memory, are needed to understand the tasks and
learn the mapping itself, hence the slower learning.

Figure 6.5: Average learning curves and WADE scores (shown in parentheses in
the legend) for each task in the benchmark. The dark blue curves represent the
best elementary cellular automata (CA) rule, and the green curves represent
the echo-state networks (ESN). Note that the x-axis is logarithmic, showing
ten times more training steps for the RNN and LSTM. Shaded areas represent one
standard deviation around the average over the 100 different experiments.

It is interesting to note that RNNs also seem to perform better than LSTMs and
Transformers at the beginning of the training in all tasks but the first two.
Even though the Transformer is nowadays generally considered a superior model,
vanilla RNNs may still remain competitive in the very low data and computation
regimes. This is especially pronounced for tasks 8, 9, and 10, where the RNN
test accuracy curve starts increasing significantly 1000 steps before the
LSTM. After several epochs, the Transformer still outperforms other
alternatives in terms of test accuracy on most tasks.

We note that we implicitly assume a fixed cost per training iteration in our
experiments and that each training example is seen once. Without these
requirements, one could achieve higher WADE results at the cost of additional
computations and memory usage by using _replay_ -inspired methods [122, 256,
95, 251] that retrain the models with past stored inputs. The results in Table
6.5 used each input sequence once.

###  6.10 Reservoir cellular automata (ReCA) results

We present some detailed results from our experiments with reservoir cellular
automata. First, general statistics about ECA performance on the five first
tasks (tasks with ID 1 to 5).

Figure 6.6: Average test accuracies across rules, hyperparameters, and runs
for each task. The best solution for each task is shown in red.

Fig. 6.6 shows a summary of the accuracy scores for each task. Score
distributions vary a lot from task to task because of the different output
spaces (binary for Periodic, Inc-per, and QA and multi-modal for the others).

Next, we give a more in-depth analysis of the best performing ECAs on the
first two tasks.

####  6.10.1 Binary sequences

The binary sequence task should be the simplest to solve. Since it requires
some memory, we expect some CA rules with very chaotic behavior — like ECA
rule 30 — to fail at it. We observed that the best rules are consistent with
the periodic and ordered types.

Figure 6.7: Top performing rules and hyperparameters for the binary periodic
sequence task. All of the best rules seem to make use of a lateral
translation, which is an effective memory mechanism. Rule 2, 130 and 194 only
differ by 1 transition in their rule function.

The four best rules displayed in figure 6.7 all implement a similar mechanism,
using translations of cells from one column to the next at every step. This
translation is a memory mechanism, because it shifts any cell that was turned
on in reaction to input before it could be overwritten by a subsequent input
that would be mapped to the same grid position.

| Top 10 rules
---|---
Rule | 38 | 24 | 231 | 152 | 66
Average accuracy | 0.969 | 0.97 | 0.97 | 0.97 | 0.97
| 189 | 194 | 52 | 130 | 2
| 0.97 | 0.97 | 0.97 | 0.971 | 0.971
Table 6.7: Top 10 rules for the periodic binary sequence task (ordered left to
right) — best hyper-parameter combination. Many rules get close to the best
possible accuracy.

####  6.10.2 Symbol counting

The easy version of this task yielded the most surprising results: some CA
rules unexpectedly reach very good scores. These top rules are reliably found
to have the best accuracy scores, even when using powerful decoders. The best
performing rule of all the possible ECAs is rule 37.

Figure 6.8: The two best performing rules for the easy symbol counting task
(.82 average accuracy).

As shown in Figure 6.8, the two best rules seem to employ a similar “computing
mechanism” to solve the task. They seem to grow tree-shaped structures that
sometimes become wider. One hypothesis for the success of these two rules is
that they encode information about the number of symbols observed in the width
of these tree structures. RC is useful for this kind of analysis because it
has some level of interpretability. The weights of the output decoding layer
can inform us of which of the components of the CAs internal state are the
most important for the task being solved. However, even with this tool, it can
still be hard to disentangle the effect of each component and their
interactions.

| Top 10 rules
---|---
Rule | 28 | 92 | 220 | 70 | 198
Average accuracy | 0.571 | 0.572 | 0.573 | 0.574 | 0.582
| 156 | 78 | 123 | 91 | 37
| 0.594 | 0.598 | 0.643 | 0.721 | 0.807
Table 6.8: Top 10 rules for the easy symbol counting task (ordered left to
right) — best hyper-parameter combination. Notice the three best rules are
largely better than most of the others.

###  6.11 Conclusions

Learning speed and data efficiency are essential components of any learning
system. Our learning speed metric can offer a novel perspective on several
machine learning models. Instead of focusing on pure performance, measuring
and comparing the data efficiency of different models will hopefully lead to
better systems for online and continual learning.

Even with the right metric, it is difficult to thoroughly evaluate the ability
of a model to learn efficiently. Our benchmark evaluates a range of problem
difficulties which would be challenging to construct by combining or
manipulating existing datasets. However, the tasks remain easy to understand
and use. Since the tasks are language-based, they can be further mixed or
chained to create continuous learning problems and may also be easily extended
in a follow-up work.

We study lesser-known machine learning models based on evolving states of
complex systems that can learn through self-organization. A complex dynamical
system comprises many interacting agents and evolves over time according to a
fixed update rule. These agents can be hidden neurons of a RNN or nodes in a
graph. Such systems often exhibit emergent global dynamics resulting from the
actions of its parts rather than the decisions of a central controller. These
dynamics lead to surprisingly complex behavior, which can be random, chaotic,
or may lead to unbounded growth of complexity [29]. Due to these properties,
reservoir computing systems may be a promising alternative that addresses the
shortcomings of standard supervised models.

Surprisingly, such models achieve remarkable learning efficiency compared to
the more standard sequentially supervised models trained with stochastic
gradient-based learning. Complex systems-based models consistently outperform
sequential supervised methods and even achieve better learning efficiency than
humans on some tasks. They demonstrate more efficient learning on our
benchmark at a fraction of the computational and data cost of the conventional
models. We believe that more advanced models of this type could lead to more
robust and data-efficient machine learning in the future, especially in low-
data applications or problems where supervision is limited. Complex systems
are underexplored and seem worth further investigating for building the next
generation of learning algorithms.

##  Chapter 7 Conclusion

In this chapter, we summarize our contributions in the thesis.

###  7.1 Contributions

In this thesis, we have developed some tools to better understand and make use
of the computations that occur within complex systems. We summarize our
contributions in the following.

  * •

In Chapter 2, we gave an overview of the cellular automaton (CA) model and the
specific challenges it poses. We review the deep connection between CAs and
recurrent neural networks (RNNs) and how this can be used to apply CAs in
reservoir computing (RC). Next, in Chapter 3, we place our work within the
vast body of literature on complex systems, complexity, emergence, and
learning.

  * •

In Chapter 4, we define a novel metric of complexity for complex systems. The
metric is inspired by principles of compression and algorithmic complexity. It
uses the ability of small neural networks to learn a model of the local
structures in a system, and the evolution of the predictive power of that
model as the system evolves over time. We evaluated the quality of this metric
by measuring its correlation with human perception of complexity. We find a
higher correlation with human annotations than alternative methods on a
dataset of CAs rules labeled as complex or not. We use the metric to discover
new CAs rules with surprisingly complex behavior semi-automatically.

  * •

In Chapter 5, we built on the work of Chapter 4 to work on the specific
challenges posed by working with large-scale systems such as cellular
automata. In these systems, qualitatively different behavior may emerge at
various scales. To allow dealing with these large systems and apply complexity
measures and classification methods on a range of scales, we developed three
coarse-graining methods based on simple statistical analysis, clustering
algorithms and autoencoder neural networks that can reduce the size of a
system while retaining useful information.

  * •

In Chapter 6, we tackled the issue of using complex systems for general-
purpose task solving, using the RC paradigm. We also developed a metric for
the speed of learning of machine learning systems. Using that metric, we
looked at the data efficiency of some well-known algorithms rather than their
task-based performance only. Surprisingly, RC models using random frozen RNNs
or CAs are significantly more efficient than other alternatives on a range of
tasks. We evaluated this on some standard language datasets and introduced our
own dataset of progressively more complex tasks. Efficiency is a crucial
property in low data and compute settings, and our work showed that some
overlooked methods may actually be competitive in these situations.

## References

  * [1] Harold Abelson, Don Allen, Daniel Coore, Chris Hanson, George Homsy, Thomas F. Knight Jr, Radhika Nagpal, Erik Rauch, Gerald Jay Sussman and Ron Weiss  “Amorphous Computing”  In _Communications of the ACM_ 43.6 ACM New York, NY, USA, 2000, pp. 74–82
  * [2] Alyssa Adams, Hector Zenil, Paul C.. Davies and Sara Imari Walker  “Formal Definitions of Unbounded Evolution and Innovation Reveal Universal Mechanisms for Open-Ended Evolution in Dynamical Systems”  In _Scientific Reports_ 7.1, 2017, pp. 997  DOI: 10.1038/s41598-017-00810-8
  * [3] Peter M. Allen and Mark Strathern  “Evolution, Emergence, and Learning in Complex Systems”  In _Emergence_ 5.4 Taylor & Francis, 2003, pp. 8–33
  * [4] M.. Alomar, V. Canals, V. Martínez-Moll and J.. Rosselló  “Low-Cost Hardware Implementation of Reservoir Computers”  In _2014 24th International Workshop on Power and Timing Modeling, Optimization and Simulation (PATMOS)_ , 2014, pp. 1–5  DOI: 10.1109/PATMOS.2014.6951899
  * [5] P.. Anderson  “More Is Different”  In _Science_ 177.4047 American Association for the Advancement of Science, 1972, pp. 393–396  DOI: 10.1126/science.177.4047.393
  * [6] David Andre, Forrest H. Bennett and John R. Koza  “Discovery by Genetic Programming of a Cellular Automata Rule That Is Better than Any Known Rule for the Majority Classification Problem”  In _Proceedings of the 1st Annual Conference on Genetic Programming_ Cambridge, MA, USA: MIT Press, 1996, pp. 3–11
  * [7] Piotr Antonik  “Application of FPGA to Real-Time Machine Learning: Hardware Reservoir Computers and Software Image Processing”, Springer Theses  Cham: Springer International Publishing, 2018  DOI: 10.1007/978-3-319-91053-6
  * [8] Piotr Antonik, Nicolas Marsal, Daniel Brunner and Damien Rontani  “Human Action Recognition with a Large-Scale Brain-Inspired Photonic Computer”  In _Nature Machine Intelligence_ 1.11 Nature Publishing Group, 2019, pp. 530–537  DOI: 10.1038/s42256-019-0110-8
  * [9] Piotr Antonik, Anteo Smerieri, Francois Duport, Marc Haelterman and Serge Massar  “FPGA Implementation of Reservoir Computing with Online Learning”, 2015  URL: http://hdl.handle.net/2013/
  * [10] Luis Antunes, Lance Fortnow, Dieter Melkebeek and N.. Vinodchandran  “Computational Depth: Concept and Applications”  In _Theoretical Computer Science_ 354.3, Foundations of Computation Theory (FCT 2003), 2006, pp. 391–404  DOI: 10.1016/j.tcs.2005.11.033
  * [11] Luís Antunes and Lance Fortnow  “Sophistication Revisited”  In _Theory of Computing Systems_ 45.1, 2009, pp. 150–161  DOI: 10.1007/s00224-007-9095-5
  * [12] Hideki Arata, Yoshiaki Takai, Nami K. Takai and Tsuyoshi Yamamoto  “Free-Form Shape Modeling by 3D Cellular Automata”  In _Proceedings Shape Modeling International’99. International Conference on Shape Modeling and Applications_ IEEE, 1999, pp. 242–247
  * [13] Neil Babson and Christof Teuscher  “Reservoir Computing with Complex Cellular Automata”  In _Complex Systems_ 28.4, 2019, pp. 433–455  DOI: 10.25088/ComplexSystems.28.4.433
  * [14] Thomas Bäck and Hans-Paul Schwefel  “An Overview of Evolutionary Algorithms for Parameter Optimization”  In _Evolutionary computation_ 1.1 mit Press, 1993, pp. 1–23
  * [15] R.J. Bagley and J.D. Farmer  “Spontaneous Emergence of a Metabolism”  In _Artificial Life II_ X, 1991, pp. 93–140
  * [16] Pierre Baldi and Kurt Hornik  “Neural Networks and Principal Component Analysis: Learning from Examples without Local Minima”  In _Neural Networks_ 2.1, 1989, pp. 53–58  DOI: 10.1016/0893-6080(89)90014-2
  * [17] Wolfgang Banzhaf, Frank D. Francone, Robert E. Keller and Peter Nordin  “Genetic Programming: An Introduction: On the Automatic Evolution of Computer Programs and Its Applications”  San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1998
  * [18] Shawn Beaulieu, Lapo Frati, Thomas Miconi, Joel Lehman, Kenneth O. Stanley, Jeff Clune and Nick Cheney  “Learning to Continually Learn”, 2020  arXiv: http://arxiv.org/abs/2002.09571
  * [19] Ralph Beckers, Owen E. Holland and Jean-Louis Deneubourg  “Fom Local Actions to Global Tasks: Stigmergy and Collective Robotics”  In _Prerational Intelligence: Adaptive Behavior and Intelligent Systems Without Symbols and Logic, Volume 1, Volume 2 Prerational Intelligence: Interdisciplinary Perspectives on the Behavior of Natural and Artificial Systems, Volume 3_ , Studies in Cognitive Systems  Dordrecht: Springer Netherlands, 2000, pp. 1008–1022  DOI: 10.1007/978-94-010-0870-9_63
  * [20] Mark A. Bedau, Nicholas Gigliotti, Tobias Janssen, Alec Kosik, Ananthan Nambiar and Norman Packard  “Open-Ended Technological Innovation”  In _Artificial Life_ 25.1, 2019, pp. 33–49  DOI: 10.1162/artl_a_00279
  * [21] Mark A. Bedau, John S. McCaskill, Norman H. Packard and Steen Rasmussen  “A Less Abstract Artificial Chemistry”  In _Artificial Life VII: Proceedings of the Seventh International Conference on Artificial Life_ MIT Press, 2000, pp. 49–53  URL: https://ieeexplore.ieee.org/document/6278502
  * [22] Charles H. Bennett  “Dissipation, Information, Computational Complexity and the Definition of Organization”  In _Emerging Syntheses In Science_ CRC Press, 1988
  * [23] Charles H. Bennett  “Logical Depth and Physical Complexity”  In _The Universal Turing Machine A Half-Century Survey_ 2 Vienna: Springer Vienna, 1995, pp. 207–235  DOI: 10.1007/978-3-7091-6597-3_8
  * [24] Elwyn R. Berlekamp, John Horton Conway and Richard K. Guy  “Winning Ways for Your Mathematical Plays”  Natick, Mass: A.K. Peters, 2001
  * [25] Nils Bertschinger and Thomas Natschläger  “Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks”  In _Neural Computation_ 16.7, 2004, pp. 1413–1436  DOI: 10.1162/089976604323057443
  * [26] Eleonora Bilotta and Pietro Pantano  “ARTIFICIAL MICRO-WORLDS PART II: CELLULAR AUTOMATA GROWTH DYNAMICS”  In _International Journal of Bifurcation and Chaos_ 21.03, 2011, pp. 619–645  DOI: 10.1142/S0218127411028672
  * [27] Sonja E.. Boas, Yi Jiang, Roeland M.. Merks, Sotiris A. Prokopiou and Elisabeth G. Rens  “Cellular Potts Model: Applications to Vasculogenesis and Angiogenesis”  In _Probabilistic Cellular Automata: Theory, Applications and Future Perspectives_ , Emergence, Complexity and Computation  Cham: Springer International Publishing, 2018, pp. 279–310  DOI: 10.1007/978-3-319-65558-1_18
  * [28] N. Boccara, J. Nasser and M. Roger  “Particlelike Structures and Their Interactions in Spatiotemporal Patterns Generated by One-Dimensional Deterministic Cellular-Automaton Rules”  In _Physical Review A_ 44.2, 1991, pp. 866–875  DOI: 10.1103/PhysRevA.44.866
  * [29] Nino Boccara  “Modeling Complex Systems”  Springer Science & Business Media, 2010  GOOGLEBOOKS: boUorPmcbKMC
  * [30] Lashon B. Booker  “Perspectives on Adaptation in Natural and Artificial Systems (Proceedings Volume in the Santa Fe Institute Studies in the Sciences of comPlexity.)”  New York, NY, USA: Oxford University Press, Inc., 2004
  * [31] Lashon B. Booker, David E. Goldberg and John H. Holland  “Classifier Systems and Genetic Algorithms”  In _Artificial intelligence_ 40.1-3 Elsevier, 1989, pp. 235–282
  * [32] Léon Bottou, Frank E. Curtis and Jorge Nocedal  “Optimization Methods for Large-Scale Machine Learning”  In _Siam Review_ 60.2 SIAM, 2018, pp. 223–311
  * [33] S. Boyd and L. Chua  “Fading Memory and the Problem of Approximating Nonlinear Operators with Volterra Series”  In _IEEE Transactions on Circuits and Systems_ 32.11, 1985, pp. 1150–1161  DOI: 10.1109/TCS.1985.1085649
  * [34] Jacob Buckman, Danijar Hafner, George Tucker, Eugene Brevdo and Honglak Lee  “Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion”  In _Advances in neural information processing systems_ 31, 2018
  * [35] Marius Buliga and Louis H. Kauffman  “Chemlambda, Universality and Self-Multiplication”  In _Artificial Life 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems_ , 2014, pp. 490–497  DOI: 10.7551/978-0-262-32621-6-ch079
  * [36] Dean V. Buonomano and Michael M. Merzenich  “Temporal Information Transformed into a Spatial Code by a Neural Network with Realistic Properties”  In _Science_ 267.5200, 1995, pp. 1028–1030  DOI: 10.1126/science.7863330
  * [37] William Joseph Butera  “Programming a Paintable Computer”, 2002
  * [38] Nicholas Carlini  “Digital Logic Gates on Conway’s Game of Life - Part 1”, 2020  URL: https://nicholas.carlini.com/writing/2020/digital-logic-game-of-life.html
  * [39] H.. Ceccatto and B.. Huberman  “The Complexity of Hierarchical Systems”  In _Physica Scripta_ 37.1, 1988, pp. 145  DOI: 10.1088/0031-8949/37/1/021
  * [40] Gregory J Chaitin  “Algorithmic Information Theory”  In _IBM journal of research and development_ 21.4, 1977, pp. 350–359
  * [41] Gregory J. Chaitin  “On the Length of Programs for Computing Finite Binary Sequences: Statistical Considerations”  In _J. ACM_ 16.1, 1969, pp. 145–159  DOI: 10.1145/321495.321506
  * [42] Gregory J. Chaitin  “Toward a Mathematical Definition of "Life"”  In _Information, Randomness & Incompleteness_ Volume 8.Volume 8, World Scientific Series in Computer Science  WORLD SCIENTIFIC, 1987, pp. 86–104  DOI: 10.1142/9789814434058_0011
  * [43] Gregory J. Chaitin  “Information, Randomness & Incompleteness: Papers on Algorithmic Information Theory”  World Scientific, 1990
  * [44] Bert Wang-Chak Chan  “Lenia - Biology of Artificial Life”, 2019  arXiv: http://arxiv.org/abs/1812.05433
  * [45] Hao-Hsuan Chang, Hao Song, Yang Yi, Jianzhong Zhang, Haibo He and Lingjia Liu  “Distributive Dynamic Spectrum Access Through Deep Reinforcement Learning: A Reservoir Computing-Based Approach”  In _IEEE Internet of Things Journal_ 6.2, 2019, pp. 1938–1948  DOI: 10.1109/JIOT.2018.2872441
  * [46] Alastair Channon  “Improving and Still Passing the ALife Test: Component-normalised Activity Statistics Classify Evolution in Geb as Unbounded”  In _Proceedings of Artificial Life VIII, Sydney, RK Standish, MA Bedau, and HA Abbass,(eds.), MIT Press: Cambridge, MA_ , 2003, pp. 173–181
  * [47] Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen and Yoshua Bengio  “Babyai: A Platform to Study the Sample Efficiency of Grounded Language Learning”, 2018  arXiv:1810.08272
  * [48] Kyunghyun Cho, Bart Merriënboer, Dzmitry Bahdanau and Yoshua Bengio  “On the Properties of Neural Machine Translation: Encoder–Decoder Approaches”  In _Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation_ , 2014, pp. 103–111
  * [49] Hugo Cisneros  “Open-Ended Creation of Hybrid Creatures with Neural Cellular Automata”, 2021  Hugo Cisneros’ blog  URL: https://hugocisneros.com/blog/open-ended-creation-of-hybrid-creatures-with-neural-cellular-automata/
  * [50] Hugo Cisneros, Tomas Mikolov and Josef Sivic  “Benchmarking Learning Efficiency in Deep Reservoir Computing”  In _Proceedings of The 1st Conference on Lifelong Learning Agents_ 199 PMLR, 2022, pp. 532–547  URL: https://proceedings.mlr.press/v199/cisneros22a.html
  * [51] Hugo Cisneros, Josef Sivic and Tomas Mikolov  “Evolving Structures in Complex Systems”  In _2019 IEEE Symposium Series on Computational Intelligence (SSCI)_ Xiamen, China: IEEE, 2019, pp. 230–237  DOI: 10.1109/SSCI44817.2019.9002840
  * [52] Hugo Cisneros, Josef Sivic and Tomas Mikolov  “Visualizing Computation in Large-Scale Cellular Automata”  In _Artificial Life Conference Proceedings_ 32 MIT Press, 2020, pp. 239–247  DOI: 10.1162/isal_a_00277
  * [53] Matthew Cook  “Universality in Elementary Cellular Automata”  In _Complex Systems_ , 2004, pp. 40
  * [54] Daniel Coore  “Botanical Computing: A Developmental Approach to Generating Interconnect Topologies on an Amorphous Computer”, 1999
  * [55] J.. Crutchfield and M. Mitchell  “The Evolution of Emergent Computation.”  In _Proceedings of the National Academy of Sciences_ 92.23, 1995, pp. 10742–10746  DOI: 10.1073/pnas.92.23.10742
  * [56] James P. Crutchfield  “The Calculi of Emergence: Computation, Dynamics and Induction”  In _Physica D: Nonlinear Phenomena_ 75.1, 1994, pp. 11–54  DOI: 10.1016/0167-2789(94)90273-9
  * [57] James P. Crutchfield  “Between Order and Chaos”  In _Nature Physics_ 8.1 Nature Publishing Group, 2012, pp. 17–24  DOI: 10.1038/nphys2190
  * [58] James P. Crutchfield and James E. Hanson  “Turbulent Pattern Bases for Cellular Automata”  In _Physica D: Nonlinear Phenomena_ 69.3-4, 1993, pp. 279–301  DOI: 10.1016/0167-2789(93)90092-F
  * [59] James P. Crutchfield and Cosma Rohilla Shalizi  “Thermodynamic Depth of Causal States: Objective Complexity via Minimal Representations”  In _Physical review E_ 59.1 APS, 1999, pp. 275
  * [60] James P. Crutchfield and Karl Young  “Inferring Statistical Complexity”  In _Physical Review Letters_ 63.2, 1989, pp. 105–108  DOI: 10.1103/PhysRevLett.63.105
  * [61] Antoine Cully and Yiannis Demiris  “Quality and Diversity Optimization: A Unifying Modular Framework”  In _IEEE Transactions on Evolutionary Computation_ 22.2 IEEE, 2017, pp. 245–259
  * [62] Haskell B. Curry  “Combinatory Logic”  Amsterdam: North-Holland Pub. Co., 1958
  * [63] R.. Daley  “Minimal-Program Complexity of Sequences with Restricted Resources”  In _Information and Control_ 23.4, 1973, pp. 301–312  DOI: 10.1016/S0019-9958(73)80001-4
  * [64] Robert Daley  “On the Inference of Optimal Descriptions”  In _Theoretical Computer Science_ 4.3, 1977, pp. 301–319  DOI: 10.1016/0304-3975(77)90015-9
  * [65] Rajarshi Das, Melanie Mitchell and James P. Crutchfield  “A Genetic Algorithm Discovers Particle-Based Computation in Cellular Automata”  In _Parallel Problem Solving from Nature — PPSN III_ , Lecture Notes in Computer Science  Berlin, Heidelberg: Springer, 1994, pp. 344–353  DOI: 10.1007/3-540-58484-6_278
  * [66] Tom De Wolf and Tom Holvoet  “Emergence Versus Self-Organisation: Different Concepts but Promising When Combined”  In _Engineering Self-Organising Systems_ , Lecture Notes in Computer Science  Berlin, Heidelberg: Springer, 2005, pp. 1–15  DOI: 10.1007/11494676_1
  * [67] J. Haan  “How Emergence Arises”  In _Ecological Complexity_ 3.4, Complexity and Ecological Economics, 2006, pp. 293–301  DOI: 10.1016/j.ecocom.2007.02.003
  * [68] D.. Dennett  “Darwin’s Dangerous Idea: Evolution and the Meanings of Life”, A Touchstone Book  New York: Simon & Schuster, 1996
  * [69] David Deutsch  “Quantum Theory, the Church–Turing Principle and the Universal Quantum Computer”  In _Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences_ 400.1818 The Royal Society London, 1985, pp. 97–117
  * [70] Dung Di Caprio, Janusz Stafiej, Giorgio Luciano and Laurent Arurault  “3D Cellular Automata Simulations of Intra and Intergranular Corrosion”  In _Corrosion Science_ 112 Elsevier, 2016, pp. 438–450
  * [71] Peter Dittrich, Jens Ziegler and Wolfgang Banzhaf  “Artificial Chemistries—A Review”  In _Artificial Life_ 7.3, 2001, pp. 225–275  DOI: 10.1162/106454601753238636
  * [72] P. Dominey, M. Arbib and J.. Joseph  “A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and Sequences”  In _Journal of Cognitive Neuroscience_ 7.3, 1995, pp. 311–336  DOI: 10.1162/jocn.1995.7.3.311
  * [73] Peter F. Dominey  “Complex Sensory-Motor Sequence Learning Based on Recurrent State Representation and Reinforcement Learning”  In _Biological Cybernetics_ 73.3, 1995, pp. 265–274  DOI: 10.1007/BF00201428
  * [74] Colin Donahue, Cory Merkel, Qutaiba Saleh, Levs Dolgovs, Yu Kee Ooi, Dhireesha Kudithipudi and Bryant Wysocki  “Design and Analysis of Neuromemristive Echo State Networks with Limited-Precision Synapses”  In _2015 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA)_ , 2015, pp. 1–6  DOI: 10.1109/CISDA.2015.7208623
  * [75] Stéphane Doncieux, Alban Laflaquière and Alexandre Coninx  “Novelty Search: A Theoretical Perspective”  In _Proceedings of the Genetic and Evolutionary Computation Conference_ Prague Czech Republic: ACM, 2019, pp. 99–106  DOI: 10.1145/3321707.3321752
  * [76] Jonathan Dong, Mushegh Rafayelyan, Florent Krzakala and Sylvain Gigan  “Optical Reservoir Computing Using Multiple Light Scattering for Chaotic Systems Prediction”  In _IEEE Journal of Selected Topics in Quantum Electronics_ 26.1 Institute of Electrical and Electronics Engineers, 2020, pp. 1–12  DOI: 10.1109/JSTQE.2019.2936281
  * [77] Freeman J. Dyson  “Origins of Life”  Cambridge [England] ; New York: Cambridge University Press, 1999
  * [78] A.A. Efros and T.K. Leung  “Texture Synthesis by Non-Parametric Sampling”  In _Proceedings of the Seventh IEEE International Conference on Computer Vision_ Kerkyra, Greece: IEEE, 1999, pp. 1033–1038 vol.2  DOI: 10.1109/ICCV.1999.790383
  * [79] Manfred Eigen and Peter Schuster  “The Hypercycle”  Berlin, Heidelberg: Springer, 1979  DOI: 10.1007/978-3-642-67247-7
  * [80] Jeffrey L. Elman  “Finding Structure in Time”  In _Cognitive Science_ 14.2, 1990, pp. 179–211  DOI: 10.1207/s15516709cog1402_1
  * [81] Kari Eloranta and Esa Nummelin  “The Kink of Cellular Automaton Rule 18 Performs a Random Walk”  In _Journal of Statistical Physics_ 69.5-6, 1992, pp. 1131–1136  DOI: 10.1007/BF01058766
  * [82] S.. Evans, G.. Saulnier and Stephen F. Bush  “A New Universal Two Part Code for Estimation of String Kolmogorov Complexity and Algorithmic Minimum Sufficient Statistic”  In _DIMACS Workshop on Complexity and Inference_ , 2003
  * [83] M. Feder, N. Merhav and M. Gutman  “Universal Prediction of Individual Sequences”  In _IEEE Transactions on Information Theory_ 38.4, 1992, pp. 1258–1270  DOI: 10.1109/18.144706
  * [84] David P Feldman and James P Crutchfield  “Measures of Statistical Complexity: Why?”  In _Physics Letters A_ 238.4, 1998, pp. 244–252  DOI: 10.1016/S0375-9601(97)00855-4
  * [85] Christoph Flamm, Alexander Ullrich, Heinz Ekker, Martin Mann, Daniel Högerl, Markus Rohrschneider, Sebastian Sauer, Gerik Scheuermann, Konstantin Klemm, Ivo L Hofacker and Peter F Stadler  “Evolution of Metabolic Networks: A Computational Frame-Work”  In _Journal of Systems Chemistry_ 1.1, 2010, pp. 4  DOI: 10.1186/1759-2208-1-4
  * [86] Robert W. Floyd and L.. Steinberg  “An Adaptive Algorithm for Spatial Gray Scale”  In _Journal of the Society for Information Display_ 17, 1976, pp. 75–77
  * [87] Lawrence J. Fogel, Alvin J. Owens and Michael J. Walsh  “Artificial Intelligence through Simulated Evolution”, Artificial Intelligence through Simulated Evolution  Oxford, England: John Wiley & Sons, 1966, pp. xii170
  * [88] W Fontana and L W Buss  “What Would Be Conserved If "the Tape Were Played Twice"?”  In _Proceedings of the National Academy of Sciences_ 91.2 Proceedings of the National Academy of Sciences, 1994, pp. 757–761  DOI: 10.1073/pnas.91.2.757
  * [89] Peter Gács  “Reliable Computation with Cellular Automata”  In _Journal of Computer and System Sciences_ 32.1 Elsevier, 1986, pp. 15–78
  * [90] Peter Gács  “Reliable Cellular Automata with Self-Organization”  In _Journal of Statistical Physics_ 103.1, 2001, pp. 45–267  DOI: 10.1023/A:1004823720305
  * [91] Ch-A. Gandin and Michel Rappaz  “A 3D Cellular Automaton Algorithm for the Prediction of Dendritic Grain Growth”  In _Acta Materialia_ 45.5 Elsevier, 1997, pp. 2187–2195
  * [92] Martin Gardner  “Mathematical Games”  In _Scientific American_ 223.4, 1970, pp. 120–123  DOI: 10.1038/scientificamerican1070-120
  * [93] Max Garzon and Fernanda Botelho  “Real Computation with Cellular Automata”  In _Cellular Automata and Cooperative Systems_ Springer, 1993, pp. 191–202
  * [94] Murray Gell-Mann  “Simplicity and Complexity in the Description of Nature”  In _Engineering and Science_ 51.3 California Institute of Technology, 1988, pp. 2–9
  * [95] Alexander Gepperth and Cem Karaoguz  “A Bio-Inspired Incremental Learning Architecture for Applied Perceptual Problems”  In _Cognitive Computation_ 8.5, 2016, pp. 924–934  DOI: 10.1007/s12559-016-9389-5
  * [96] William Gilpin  “Cellular Automata as Convolutional Neural Networks”, 2018  arXiv: http://arxiv.org/abs/1809.02942
  * [97] Tom Eivind Glover, Pedro Lind, Anis Yazidi, Evgeny Osipov and Stefano Nichele  “The Dynamical Landscape of Reservoir Computing with Elementary Cellular Automata”  In _ALIFE 2021: The 2021 Conference on Artificial Life_ MIT Press, 2021
  * [98] David E. Goldberg  “Simple Genetic Algorithms and the Minimal Deceptive Problem”  In _Genetic Algorithms and Simulated Annealing_ Los Altos, CA: Pitman, 1987, pp. 74–88
  * [99] Jeffrey Goldstein  “Emergence in Complex Systems”  In _The sage handbook of complexity and management_ Sage Thousand Oaks, CA, 2011, pp. 65–78
  * [100] Jorge Gomes, Pedro Mariano and Anders Lyhne Christensen  “Devising Effective Novelty Search Algorithms: A Comprehensive Empirical Study”  In _Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation_ , GECCO ’15  New York, NY, USA: Association for Computing Machinery, 2015, pp. 943–950  DOI: 10.1145/2739480.2754736
  * [101] Jorge Gomes, Paulo Urbano and Anders Lyhne Christensen  “Evolution of Swarm Robotics Systems with Novelty Search”  In _Swarm Intelligence_ 7.2, 2013, pp. 115–144  DOI: 10.1007/s11721-013-0081-z
  * [102] Ian J. Goodfellow, Mehdi Mirza, Xia Da, Aaron C. Courville and Yoshua Bengio  “An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks”  In _2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings_ , 2014  URL: http://arxiv.org/abs/1312.6211
  * [103] R.Wm. Gosper  “Exploiting Regularities in Large Cellular Spaces”  In _Physica D: Nonlinear Phenomena_ 10.1-2, 1984, pp. 75–80  DOI: 10.1016/0167-2789(84)90251-3
  * [104] Peter Grassberger  “Toward a Quantitative Theory of Self-Generated Complexity”  In _International Journal of Theoretical Physics_ 25.9, 1986, pp. 907–938  DOI: 10.1007/BF00668821
  * [105] Peter Grassberger  “Problems in Quantifying Self-Generated Complexity”  In _Helvetica Physica Acta_ 62 Birkhäuser, 1989, pp. 489–511  DOI: 10.5169/SEALS-116045
  * [106] Peter Grassberger  “Randomness, Information, and Complexity”  In _Proceedings of the 5th Mexican School on Statistical Physics_ , 1989  arXiv: http://arxiv.org/abs/1208.3459
  * [107] Alex Graves, Greg Wayne and Ivo Danihelka  “Neural Turing Machines”, 2014  arXiv:1410.5401
  * [108] Peter Grunwald  “The Minimum Description Length Principle”, Adaptive Computation and Machine Learning  Cambridge, Mass: MIT Press, 2007
  * [109] Howard Gutowitz  “Cellular Automata and the Sciences of Complexity (Part I): A Review of Some Outstanding Problems in the Theory of Cellular Automata”  In _Complexity_ 1.5, 1995, pp. 16–22  DOI: 10.1002/cplx.6130010505
  * [110] Howard A. Gutowitz  “Transients, Cycles, and Complexity in Cellular Automata”  In _Physical Review A_ 44.12 APS, 1991, pp. R7881
  * [111] Howard A. Gutowitz and Christopher G. Langton  “Methods for Designing Cellular Automata with” Interesting” Behavior”  In _CNLS News Letter_ , 1988
  * [112] Katrin Hammerschmidt, Caroline J. Rose, Benjamin Kerr and Paul B. Rainey  “Life Cycles, Fitness Decoupling and the Evolution of Multicellularity”  In _Nature_ 515.7525, 2014, pp. 75–79  DOI: 10.1038/nature13884
  * [113] R.. Hamming  “Error Detecting and Error Correcting Codes”  In _Bell System Technical Journal_ 29.2, 1950, pp. 147–160  DOI: 10.1002/j.1538-7305.1950.tb00463.x
  * [114] James E. Hanson and James P. Crutchfield  “The Attractor-Basin Portrait of a Cellular Automaton”  In _Journal of Statistical Physics_ 66.5-6, 1992, pp. 1415–1462  DOI: 10.1007/BF01054429
  * [115] James E. Hanson and James P. Crutchfield  “Computational Mechanics of Cellular Automata: An Example”  In _Physica D: Nonlinear Phenomena_ 103.1, Lattice Dynamics, 1997, pp. 169–189  DOI: 10.1016/S0167-2789(96)00259-X
  * [116] Masateru Harao and Shoichi Noguchi  “A Consideration on Cellular Automata with Errors”  In _Comittee on Automaton, IECE Japan, AL_ 73, 1973, pp. 15
  * [117] Masateru Harao and Shoichi Noguchi  “Fault Tolerant Cellular Automata”  In _Journal of Computer and System Sciences_ 11.2, 1975, pp. 171–185  DOI: 10.1016/S0022-0000(75)80066-3
  * [118] Brosl Hasslacher  “Discrete Fluids”  In _Los Alamos Science_ , 1987, pp. 175–217
  * [119] Helmut Hauser, Auke J. Ijspeert, Rudolf M. Füchslin, Rolf Pfeifer and Wolfgang Maass  “Towards a Theoretical Foundation for Morphological Computation with Compliant Bodies”  In _Biological Cybernetics_ 105.5, 2011, pp. 355–370  DOI: 10.1007/s00422-012-0471-0
  * [120] David Herel, Dominika Zogatova, Matej Kripner and Tomas Mikolov  “Emergence of Novelty in Evolutionary Algorithms”  MIT Press, 2022  DOI: 10.1162/isal_a_00501
  * [121] Geoffrey E. Hinton  “Connectionist Learning Procedures”  In _Artificial Intelligence_ 40.1-3, 1989, pp. 185–234  DOI: 10.1016/0004-3702(89)90049-0
  * [122] Geoffrey E. Hinton and David C. Plaut  “Using Fast Weights to Deblur Old Memories”  In _Proceedings of the Ninth Annual Conference of the Cognitive Science Society_ , 1987, pp. 177–186
  * [123] Geoffrey E. Hinton and Terrence J. Sejnowski  “Unsupervised Learning: Foundations of Neural Computation”  MIT Press, 1999  GOOGLEBOOKS: yj04Y0lje4cC
  * [124] Sepp Hochreiter and Jürgen Schmidhuber  “Long Short-Term Memory”  In _Neural Computation_ 9.8, 1997, pp. 1735–1780  DOI: 10.1162/neco.1997.9.8.1735
  * [125] John H. Holland  “Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence”, Complex Adaptive Systems  Cambridge, Mass: MIT Press, 1992
  * [126] John H. Holland  “Emergence: From Chaos to Order”  Oxford University Press, 2000  GOOGLEBOOKS: VjKtpujRGuAC
  * [127] John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman  “Introduction to Automata Theory, Languages, and Computation”  Boston: Pearson/Addison Wesley, 2007
  * [128] Barbora Hudcová and Tomas Mikolov  “Classification of Complex Systems Based on Transients”  MIT Press, 2020, pp. 367–375  DOI: 10.1162/isal_a_00260
  * [129] Barbora Hudcová and Tomas Mikolov  “Computational Hierarchy of Elementary Cellular Automata”  MIT Press, 2021  DOI: 10.1162/isal_a_00447
  * [130] Barbora Hudcová and Tomas Mikolov  “Classification of Discrete Dynamical Systems Based on Transients”  In _Artificial Life_ 27.3–4, 2022, pp. 220–245  DOI: 10.1162/artl_a_00342
  * [131] Tim J. Hutton  “Evolvable Self-Reproducing Cells in a Two-Dimensional Artificial Chemistry”  In _Artificial Life_ 13.1, 2007, pp. 11–30  DOI: 10.1162/artl.2007.13.1.11
  * [132] “Igblan - Life Universal Computer”  URL: http://www.igblan.free-online.co.uk/igblan/ca/
  * [133] Takashi Ikegami, Yasuhiro Hashimoto and Mizuki Oka  “Open-Ended Evolution and a Mechanism of Novelties in Web Services”  In _Artificial Life_ 25.2, 2019, pp. 168–177  DOI: 10.1162/artl_a_00287
  * [134] Andrew Ilachinski  “Cellular Automata: A Discrete Universe”  World Scientific Publishing Company, 2001  GOOGLEBOOKS: BPY7DQAAQBAJ
  * [135] Helgi I. Ingólfsson, Cesar A. Lopez, Jaakko J. Uusitalo, Djurre H. Jong, Srinivasa M. Gopal, Xavier Periole and Siewert J. Marrink  “The Power of Coarse Graining in Biomolecular Simulations”  In _Wiley Interdisciplinary Reviews: Computational Molecular Science_ 4.3, 2014, pp. 225–248  DOI: 10.1002/wcms.1169
  * [136] Akio Ishiguro, Akinobu Fujii and Peter Eggenberger Hotz  “Neuromodulated Control of Bipedal Locomotion Using a Polymorphic CPG Circuit”  In _Adaptive Behavior_ 11.1 SAGE Publications Ltd STM, 2003, pp. 7–17  DOI: 10.1177/10597123030111001
  * [137] Navot Israeli and Nigel Goldenfeld  “On Computational Irreducibility and the Predictability of Complex Physical Systems”  In _Physical Review Letters_ 92.7, 2004, pp. 074105  DOI: 10.1103/PhysRevLett.92.074105
  * [138] Navot Israeli and Nigel Goldenfeld  “Coarse-Graining of Cellular Automata, Emergence, and the Predictability of Complex Systems”  In _Physical Review E_ 73.2, 2006, pp. 026203  DOI: 10.1103/PhysRevE.73.026203
  * [139] Herbert Jaeger  “The “Echo State” Approach to Analysing and Training Recurrent Neural Networks-with an Erratum Note”  In _Bonn, Germany: German National Research Center for Information Technology GMD Technical Report_ 148.34 Bonn, 2001, pp. 13
  * [140] Herbert Jaeger  “Adaptive Nonlinear System Identification with Echo State Networks”  In _Proceedings of the 15th International Conference on Neural Information Processing Systems_ , NIPS’02  Cambridge, MA, USA: MIT Press, 2002, pp. 609–616
  * [141] Herbert Jaeger  “Long Short-Term Memory in Echo State Networks: Details of a Simulation Study”, 2012
  * [142] Azarakhsh Jalalvand, Glenn Van Wallendael and Rik Van De Walle  “Real-Time Reservoir Computing Network-Based Systems for Detection Tasks on Visual Contents”  In _2015 7th International Conference on Computational Intelligence, Communication Systems and Networks_ , 2015, pp. 146–151  DOI: 10.1109/CICSyN.2015.35
  * [143] N.. Jennings  “Agent-Based Computing: Promise and Perils”, 1999, pp. 1429–1436  URL: https://eprints.soton.ac.uk/252172/
  * [144] Neil F. Johnson  “Simply Complexity: A Clear Guide to Complexity Theory”  Oxford: Oneworld, 2009
  * [145] Michael I. Jordan  “Serial Order: A Parallel Distributed Processing Approach”  In _Advances in Psychology_ 121 Elsevier, 1997, pp. 471–495  DOI: 10.1016/S0166-4115(97)80111-2
  * [146] Armand Joulin and Tomas Mikolov  “Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets”  In _Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1_ , NIPS’15  Cambridge, MA, USA: MIT Press, 2015, pp. 190–198
  * [147] Satoshi Kanazawa  “General Intelligence as a Domain-Specific Adaptation.”  In _Psychological review_ 111.2 American Psychological Association, 2004, pp. 512
  * [148] Kazutaka Kanno and Atsushi Uchida  “Photonic Reinforcement Learning Based on Optoelectronic Reservoir Computing”  In _Scientific Reports_ 12.1 Nature Publishing Group, 2022, pp. 3720  DOI: 10.1038/s41598-022-07404-z
  * [149] Jarkko J. Kari  “Basic Concepts of Cellular Automata”  In _Handbook of Natural Computing_ Berlin, Heidelberg: Springer, 2012, pp. 3–24  DOI: 10.1007/978-3-540-92910-9_1
  * [150] Stuart Kauffman  “At Home in the Universe: The Search for Laws of Self-organization and Complexity”  Oxford University Press, 1995
  * [151] David Kelly, Mark Dillingham, Andrew Hudson and Karoline Wiesner  “A New Method for Inferring Hidden Markov Models from Noisy Time Sequences”  In _PloS One_ 7.1, 2012, pp. e29703  DOI: 10.1371/journal.pone.0029703
  * [152] Diederik P. Kingma and Jimmy Ba  “Adam: A Method for Stochastic Optimization”  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , 2015  URL: http://arxiv.org/abs/1412.6980
  * [153] Kevin G. Kirby  “Context Dynamics in Neural Sequential Learning”  In _Proc. Florida AI Research Symposium (FLAIRS) 1991_ , 1991, pp. 66–70
  * [154] Steijn Kistemaker and Shimon Whiteson  “Critical Factors in the Performance of Novelty Search”  In _Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation_ , GECCO ’11  New York, NY, USA: Association for Computing Machinery, 2011, pp. 965–972  DOI: 10.1145/2001576.2001708
  * [155] Ken-ichi Kitayama, Masaya Notomi, Makoto Naruse, Koji Inoue, Satoshi Kawakami and Atsushi Uchida  “Novel Frontier of Photonics for Data Processing—Photonic Accelerator”  In _APL Photonics_ 4.9 American Institute of Physics, 2019, pp. 090901  DOI: 10.1063/1.5108912
  * [156] Denis Kleyko, E. Frady and Friedrich T. Sommer  “Cellular Automata Can Reduce Memory Requirements of Collective-State Computing”, 2020  arXiv: http://arxiv.org/abs/2010.03585
  * [157] Sebastian Kmiecik, Dominik Gront, Michal Kolinski, Lukasz Wieteska, Aleksandra Elzbieta Dawid and Andrzej Kolinski  “Coarse-Grained Protein Models and Their Applications”  In _Chemical Reviews_ 116.14, 2016, pp. 7898–7936  DOI: 10.1021/acs.chemrev.6b00163
  * [158] Ker-I Ko  “On the Notion of Infinite Pseudorandom Sequences”  In _Theoretical Computer Science_ 48, 1986, pp. 9–33  DOI: 10.1016/0304-3975(86)90081-2
  * [159] A.. Kolmogorov  “Three Approaches to the Quantitative Definition of Information”  In _International Journal of Computer Mathematics_ 2.1-4, 1968, pp. 157–168  DOI: 10.1080/00207166808803030
  * [160] Moshe Koppel  “Structure”  In _The Universal Turing Machine: A Half-Century Survey_ 2 Oxford University Press, 1988, pp. 403–419
  * [161] Moshe Koppel and Henri Atlan  “An Almost Machine-Independent Theory of Program-Length Complexity, Sophistication, and Induction”  In _Information Sciences_ 56.1, 1991, pp. 23–33  DOI: 10.1016/0020-0255(91)90021-L
  * [162] Taras Kowaliw  “Measures of Complexity for Artificial Embryogeny”  In _Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation - GECCO ’08_ Atlanta, GA, USA: ACM Press, 2008, pp. 843  DOI: 10.1145/1389095.1389259
  * [163] J Koza  “Evolution of Subsumption Using Genetic Programming”  In _Proc. First European Conference on Artificial Life Towards a Practice of Autonomous Systems, 1992_ MIT Press, 1992  URL: https://cir.nii.ac.jp/crid/1570854175390434048
  * [164] John R. Koza  “Genetic Programming as a Means for Programming Computers by Natural Selection”  In _Statistics and Computing_ 4.2, 1994, pp. 87–112  DOI: 10.1007/BF00175355
  * [165] Mark A. Kramer  “Nonlinear Principal Component Analysis Using Autoassociative Neural Networks”  In _AIChE Journal_ 37.2, 1991, pp. 233–243  DOI: 10.1002/aic.690370209
  * [166] Alex Krizhevsky and Geoffrey E. Hinton  “Learning Multiple Layers of Features from Tiny Images”, 2009
  * [167] Germán Kruszewski and Tomas Mikolov  “Emergence of Self-Reproducing Metabolisms as Recursive Algorithms in an Artificial Chemistry”  In _Artificial Life_ 27.3–4, 2022, pp. 277–299  DOI: 10.1162/artl_a_00355
  * [168] William B. Langdon and Riccardo Poli  “Foundations of Genetic Programming”  Berlin, Heidelberg: Springer, 2002  DOI: 10.1007/978-3-662-04726-2
  * [169] Christopher G. Langton  “Self-Reproduction in Cellular Automata”  In _Physica D: Nonlinear Phenomena_ 10.1, 1984, pp. 135–144  DOI: 10.1016/0167-2789(84)90256-2
  * [170] Christopher G. Langton  “Studying Artificial Life with Cellular Automata”  In _Physica D: Nonlinear Phenomena_ 22.1, Proceedings of the Fifth Annual International Conference, 1986, pp. 120–149  DOI: 10.1016/0167-2789(86)90237-X
  * [171] “Artificial Life: The Proceedings of an Interdisciplinary Workshop on the Synthesis and Simulation of Living Systems, Held September, 1987, in Los Alamos, New Mexico”, Santa Fe Institute Studies in the Sciences of Complexity v. 6  Redwood City, Calif: Addison-Wesley Pub. Co., Advanced Book Program, 1989
  * [172] Christopher G. Langton  “Computation at the Edge of Chaos: Phase Transitions and Emergent Computation”  In _Physica D: Nonlinear Phenomena_ 42.1-3, 1990, pp. 12–37  DOI: 10.1016/0167-2789(90)90064-V
  * [173] “Editor’s Introduction”  In _Artificial Life_ 1, 1993, pp. v–viii  DOI: 10.1162/artl.1993.1.1_2.v
  * [174] Peter A. Lawrence  “The Making of a Fly: The Genetics of Animal Design”  Oxford [England] ; Cambridge, Mass., USA: Blackwell Science, 1992
  * [175] Robert Legenstein and Wolfgang Maass  “Edge of Chaos and Prediction of Computational Performance for Neural Circuit Models”  In _Neural Networks_ 20.3, Echo State Networks and Liquid State Machines, 2007, pp. 323–334  DOI: 10.1016/j.neunet.2007.04.017
  * [176] Joel Lehman and Kenneth O. Stanley  “Efficiently Evolving Programs through the Search for Novelty”  In _Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation_ , GECCO ’10  New York, NY, USA: Association for Computing Machinery, 2010, pp. 837–844  DOI: 10.1145/1830483.1830638
  * [177] Joel Lehman and Kenneth O. Stanley  “Abandoning Objectives: Evolution Through the Search for Novelty Alone”  In _Evolutionary Computation_ 19.2, 2011, pp. 189–223  DOI: 10.1162/EVCO_a_00025
  * [178] Joel Lehman and Kenneth O. Stanley  “Evolving a Diversity of Virtual Creatures through Novelty Search and Local Competition”  In _Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation_ , 2011, pp. 211–218
  * [179] A. Lempel and J. Ziv  “On the Complexity of Finite Sequences”  In _IEEE Transactions on Information Theory_ 22.1, 1976, pp. 75–81  DOI: 10.1109/TIT.1976.1055501
  * [180] Michael Levitt and Arieh Warshel  “Computer Simulation of Protein Folding”  In _Nature_ 253.5494, 1975, pp. 694–698  DOI: 10.1038/253694a0
  * [181] Wentian Li and Norman H. Packard  “The Structure of the Elementary Cellular Automata Rule Space”  In _Complex Systems_ 4.3, 1990, pp. 17
  * [182] Wentian Li, Norman H. Packard and Christopher G. Langton  “Transition Phenomena in Cellular Automata Rule Space”  In _Physica D: Nonlinear Phenomena_ 45.1, 1990, pp. 77–94  DOI: 10.1016/0167-2789(90)90175-O
  * [183] Antonios Liapis, Georgios N. Yannakakis and Julian Togelius  “Constrained Novelty Search: A Study on Game Content Generation”  In _Evolutionary Computation_ 23.1, 2015, pp. 101–129  DOI: 10.1162/EVCO_a_00123
  * [184] Kristian Lindgren and Mats G.  “Complexity Measures and Cellular Automata”, 1988, pp. 32
  * [185] Seth Lloyd and Heinz Pagels  “Complexity as Thermodynamic Depth”  In _Annals of Physics_ 188, 1988, pp. 186–213  DOI: 10.1016/0003-4916(88)90094-2
  * [186] Dirk M. Lorenz, Alice Jeng and Michael W. Deem  “The Emergence of Modularity in Biological Systems”  In _Physics of life reviews_ 8.2, 2011, pp. 129–160  DOI: 10.1016/j.plrev.2011.02.003
  * [187] Mantas Lukoševičius and Herbert Jaeger  “Reservoir Computing Approaches to Recurrent Neural Network Training”  In _Computer Science Review_ 3.3 Elsevier, 2009, pp. 127–149
  * [188] Brian J. LuValle  “The Effects of Boundary Conditions on Cellular Automata”  In _Complex Systems_ 28.1, 2019, pp. 97–124  DOI: 10.25088/ComplexSystems.28.1.97
  * [189] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng and Christopher Potts  “Learning Word Vectors for Sentiment Analysis”  In _The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA_ The Association for Computer Linguistics, 2011, pp. 142–150  URL: https://aclanthology.org/P11-1015/
  * [190] Wolfgang Maass, Thomas Natschläger and Henry Markram  “Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations”  In _Neural Computation_ 14.11, 2002, pp. 2531–2560  DOI: 10.1162/089976602760407955
  * [191] Wolfgang Maass, Thomas Natschläger and Henry Markram  “Fading Memory and Kernel Properties of Generic Cortical Microcircuit Models”  In _Journal of Physiology, Paris_ 98.4-6, 2004, pp. 315–330  DOI: 10.1016/j.jphysparis.2005.09.020
  * [192] Matthew V Mahoney  “Fast Text Compression with Neural Networks”  In _FLAIRS_ , 2000, pp. 5
  * [193] C.. Maley  “Four Steps toward Open-Ended Evolution”  In _Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation - Volume 2_ , GECCO’99  San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1999, pp. 1336–1343
  * [194] Mrwan Margem and Osman Gedik  “Reservoir Computing Based on Cellular Automata (ReCA) in Sequence Learning”  In _Journal of cellular automata_ 14, 2018, pp. 153–170
  * [195] Mrwan Margem and Ozgür Yilmaz  “An Experimental Study on Cellular Automata Reservoir in Pathological Sequence Learning Tasks”, 2017
  * [196] Romain Martinenghi, Sergei Rybalko, Maxime Jacquot, Yanne K. Chembo and Laurent Larger  “Photonic Nonlinear Transient Computing with Multiple-Delay Wavelength Dynamics”  In _Physical Review Letters_ 108.24 American Physical Society, 2012, pp. 244101  DOI: 10.1103/PhysRevLett.108.244101
  * [197] Genaro J. Martinez, Andrew Adamatzky and Harold V. McIntosh  “Complete Characterization of Structure of Rule 54”  In _Complex Systems_ 23.3, 2014  URL: https://uwe-repository.worktribe.com/output/810658
  * [198] A Mason, A Nicoll and K Stratford  “Synaptic Transmission between Individual Pyramidal Neurons of the Rat Visual Cortex in Vitro”  In _The Journal of Neuroscience_ 11.1, 1991, pp. 72–84  DOI: 10.1523/JNEUROSCI.11-01-00072.1991
  * [199] Nathan McDonald  “Reservoir Computing & Extreme Learning Machines Using Pairs of Cellular Automata Rules”  In _2017 International Joint Conference on Neural Networks (IJCNN)_ , 2017, pp. 2429–2436  DOI: 10.1109/IJCNN.2017.7966151
  * [200] Barry McMullin and Francisco J. Varela  “Rediscovering Computational Autopoiesis”  In _Fourth European Conference on Artificial Life_ , 1997, pp. 38–47
  * [201] Cory Merkel, Qutaiba Saleh, Colin Donahue and Dhireesha Kudithipudi  “Memristive Reservoir Computing Architecture for Epileptic Seizure Detection”  In _Procedia Computer Science_ 41, 5th Annual International Conference on Biologically Inspired Cognitive Architectures, 2014 BICA, 2014, pp. 249–254  DOI: 10.1016/j.procs.2014.11.110
  * [202] Tomas Mikolov, Armand Joulin and Marco Baroni  “A Roadmap Towards Machine Intelligence”  In _Computational Linguistics and Intelligent Text Processing_ , Lecture Notes in Computer Science  Cham: Springer International Publishing, 2018, pp. 29–61  DOI: 10.1007/978-3-319-75477-2_2
  * [203] Geoffrey F. Miller, Peter M. Todd and Shailesh U. Hegde  “Designing Neural Networks Using Genetic Algorithms.”  In _ICGA_ 89, 1989, pp. 379–384
  * [204] Marvin Minsky and Seymour A. Papert  “Perceptrons: An Introduction to Computational Geometry”  Cambridge MA: The MIT Press, 1972  DOI: 10.7551/mitpress/11301.001.0001
  * [205] Melanie Mitchell  “Computation in Cellular Automata: A Selected Review”  In _Non-Standard Computation_ Weinheim, FRG: Wiley-VCH Verlag GmbH & Co. KGaA, 2005, pp. 95–140  DOI: 10.1002/3527602968.ch4
  * [206] Melanie Mitchell, James P Crutchfield and Rajarshi Das  “Evolving Cellular Automata with Genetic Algorithms: A Review of Recent Work”  In _Proceedings of the First International Conference on Evolutionary Computation and Its Applications_ , 1996, pp. 14  URL: http://csc.ucdavis.edu/~evca/Papers/evca-review.pdf
  * [207] Melanie Mitchell, James P. Crutchfield and Peter T. Hraber  “Evolving Cellular Automata to Perform Computations: Mechanisms and Impediments”  In _Physica D: Nonlinear Phenomena_ 75.1, 1994, pp. 361–391  DOI: 10.1016/0167-2789(94)90293-3
  * [208] Melanie Mitchell, Peter Hraber and James P. Crutchfield  “Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform Computations”, 1993  arXiv: http://arxiv.org/abs/adap-org/9303003
  * [209] Alejandro Morán, Christiam F. Frasser and Josep L. Rosselló  “Reservoir Computing Hardware with Cellular Automata”, 2018  arXiv: http://arxiv.org/abs/1806.04932
  * [210] Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson and Michael Levin  “Growing Neural Cellular Automata”  In _Distill_ 5.2, 2020, pp. e23  DOI: 10.23915/distill.00023
  * [211] Pieter Moree and Herman J.. Riele  “The Hexagonal versus the Square Lattice”  In _Mathematics of Computation_ 73.245 American Mathematical Society, 2004, pp. 451–473  JSTOR: https://www.jstor.org/stable/4099880
  * [212] Matthew Andres Moreno and Charles Ofria  “Toward Open-Ended Fraternal Transitions in Individuality”  In _Artificial Life_ 25.2, 2019, pp. 117–133  DOI: 10.1162/artl_a_00284
  * [213] Francisco Mota, Scott Aaronson, Luís Antunes and André Souto  “Sophistication as Randomness Deficiency”  In _Descriptional Complexity of Formal Systems_ 8031 Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 172–181  DOI: 10.1007/978-3-642-39310-5_17
  * [214] Jean-Baptiste Mouret and Stéphane Doncieux  “Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study”  In _Evolutionary Computation_ 20.1 Massachusetts Institute of Technology Press (MIT Press), 2012, pp. 91–133  DOI: 10.1162/EVCO_a_00048
  * [215] “Organic Computing: A Paradigm Shift for Complex Systems”, Autonomic Systems  Basel: Birkhäuser, 2011
  * [216] Radhika Nagpal  “Programmable Self-Assembly: Constructing Global Shape Using Biologically-Inspired Local Interactions and Origami Mathematics”, 2001
  * [217] Radhika Nagpal  “Programmable Self-Assembly Using Biologically-Inspired Multiagent Control”  In _Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 1_ , AAMAS ’02  New York, NY, USA: Association for Computing Machinery, 2002, pp. 418–425  DOI: 10.1145/544741.544839
  * [218] Radhika Nagpal  “Programmable Pattern-Formation and Scale-Independence”  In _Unifying Themes in Complex Systems IV_ Berlin, Heidelberg: Springer, 2008, pp. 275–282  DOI: 10.1007/978-3-540-73849-7_31
  * [219] Ryosho Nakane, Gouhei Tanaka and Akira Hirose  “Reservoir Computing With Spin Waves Excited in a Garnet Film”  In _IEEE Access_ 6, 2018, pp. 4462–4469  DOI: 10.1109/ACCESS.2018.2794584
  * [220] Chrystopher L. Nehaniv  “Evolution in Asynchronous Cellular Automata”  In _Proceedings of the Eighth International Conference on Artificial Life_ , 2003, pp. 65–73
  * [221] J. Neumann  “Probabilistic Logics and the Synthesis of Reliable Organisms From Unreliable Components”  In _Automata Studies. (AM-34)_ Princeton University Press, 1956, pp. 43–98  DOI: 10.1515/9781400882618-003
  * [222] Andrew Y. Ng, Michael I. Jordan and Yair Weiss  “On Spectral Clustering: Analysis and an Algorithm”  In _Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic_ , NIPS’01  Cambridge, MA, USA: MIT Press, 2001, pp. 849–856
  * [223] Cuong V. Nguyen, Yingzhen Li, Thang D. Bui and Richard E. Turner  “Variational Continual Learning”  In _CoRR_ abs/1710.10628, 2017  arXiv: http://arxiv.org/abs/1710.10628
  * [224] Stefano Nichele and Magnus S. Gundersen  “Reservoir Computing Using Non-Uniform Binary Cellular Automata”, 2017  arXiv: http://arxiv.org/abs/1702.03812
  * [225] Stefano Nichele and Andreas Molund  “Deep Learning with Cellular Automaton-Based Reservoir Computing”  In _0891-2513_ Complex Systems Publications Inc, 2017  DOI: http://dx.doi.org/10.25088/ComplexSystems.26.4.319
  * [226] Stefano Nichele and Andreas Molund  “Deep Reservoir Computing Using Cellular Automata”, 2017  arXiv: http://arxiv.org/abs/1703.02806
  * [227] Eyvind Niklasson, Alexander Mordvintsev, Ettore Randazzo and Michael Levin  “Self-Organising Textures”  In _Distill_ 6.2, 2021, pp. e00027–003  DOI: 10.23915/distill.00027.003
  * [228] Hidenosuke Nishio and Youichi Kobuchi  “Fault Tolerant Cellular Spaces”  In _Journal of Computer and System Sciences_ 11.2, 1975, pp. 150–170  DOI: 10.1016/S0022-0000(75)80065-1
  * [229] Charles Ofria and Claus O. Wilke  “Avida: A Software Platform for Research in Computational Evolutionary Biology”  In _Artificial Life_ 10.2, 2004, pp. 191–229  DOI: 10.1162/106454604773563612
  * [230] B. Ostrovsky, G. Crooks, M.. Smith and Y. Bar-Yam  “Cellular Automata for Polymer Simulation with Application to Polymer Melts and Polymer Collapse Including Implications for Protein Folding”  In _Parallel Computing_ 27.5, Cellular Automata: From Modeling to Applications, 2001, pp. 613–641  DOI: 10.1016/S0167-8191(00)00081-8
  * [231] H. Ottavi and O. Parodi  “Simulation of the Ising Model by Cellular Automata”  In _Europhysics Letters (EPL)_ 8.8 IOP Publishing, 1989, pp. 741–746  DOI: 10.1209/0295-5075/8/8/006
  * [232] Norman Packard, Mark A. Bedau, Alastair Channon, Takashi Ikegami, Steen Rasmussen, Kenneth O. Stanley and Tim Taylor  “An Overview of Open-Ended Evolution: Editorial Introduction to the Open-Ended Evolution II Special Issue”  In _Artificial Life_ 25.2, 2019, pp. 93–103  DOI: 10.1162/artl_a_00291
  * [233] Norman H Packard and Stephen Wolfram  “Two-Dimensional Cellular Automata”  In _Journal of Statistical Physics_ 38.5/6, 1985, pp. 46
  * [234] Norman H. Packard  “Adaptation toward the Edge of Chaos”  In _Dynamic patterns in complex systems_ 212 World Scientific, 1988, pp. 293–301
  * [235] Edward W. Packel and Henryk Woźniakowski  “Recent Developments in Information-Based Complexity”  In _Bulletin of the American Mathematical Society_ 17.1, 1987, pp. 9–36  DOI: 10.1090/S0273-0979-1987-15511-X
  * [236] Peng-Zhi Pan, Xia-Ting Feng and John A. Hudson  “Study of Failure and Scale Effects in Rocks under Uniaxial Compression Using 3D Cellular Automata”  In _International Journal of Rock Mechanics and Mining Sciences_ 46.4 Elsevier, 2009, pp. 674–685
  * [237] Y. Paquot, F. Duport, A. Smerieri, J. Dambre, B. Schrauwen, M. Haelterman and S. Massar  “Optoelectronic Reservoir Computing”  In _Scientific Reports_ 2.1 Nature Publishing Group, 2012, pp. 287  DOI: 10.1038/srep00287
  * [238] German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan and Stefan Wermter  “Continual Lifelong Learning with Neural Networks: A Review”  In _Neural Networks_ 113, 2019, pp. 54–71  DOI: 10.1016/j.neunet.2019.01.012.
  * [239] Razvan Pascanu and Herbert Jaeger  “A Neurodynamical Model for Working Memory”  In _Neural Networks_ 24.2, 2011, pp. 199–207  DOI: 10.1016/j.neunet.2010.10.003
  * [240] Deepak Pathak, Chris Lu, Trevor Darrell, Phillip Isola and Alexei A Efros  “Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity”  In _NeurIPS_ , 2019, pp. 13
  * [241] Howard H. Pattee and Hiroki Sayama  “Evolved Open-Endedness, Not Open-Ended Evolution”  In _Artificial Life_ 25.1, 2019, pp. 4–8  DOI: 10.1162/artl_a_00276
  * [242] Umberto Pesavento  “An Implementation of von Neumann’s Self-Reproducing Machine”  In _Artificial Life_ 2.4, 1995, pp. 337–354  DOI: 10.1162/artl.1995.2.4.337
  * [243] Rolf Pfeifer, Josh Bongard and Simon Grand  “How the Body Shapes the Way We Think: A New View of Intelligence”  Cambridge, Mass: MIT Press, 2007
  * [244] Marcus Pivato  “Spectral Domain Boundaries in Cellular Automata”, 2007  arXiv: http://arxiv.org/abs/math/0507091
  * [245] Riccardo Poli and Brian Logan  “On the Relations between Search and Evolutionary Algorithms”  In _COGNITIVE SCIENCE RESEARCH PAPERS-UNIVERSITY OF BIRMINGHAM CSRP_ UNIVERSITY OF BIRMINGHAM, 1996
  * [246] Davit A. Potoyan, Alexey Savelyev and Garegin A. Papoian  “Recent Successes in Coarse-Grained Modeling of DNA: Coarse-grained Modeling of DNA”  In _Wiley Interdisciplinary Reviews: Computational Molecular Science_ 3.1, 2013, pp. 69–83  DOI: 10.1002/wcms.1114
  * [247] Justin K. Pugh, Lisa B. Soros and Kenneth O. Stanley  “Quality Diversity: A New Frontier for Evolutionary Computation”  In _Frontiers in Robotics and AI_ 3 Frontiers, 2016  DOI: 10.3389/frobt.2016.00040
  * [248] Mushegh Rafayelyan, Jonathan Dong, Yongqi Tan, Florent Krzakala and Sylvain Gigan  “Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction”  In _Physical Review X_ 10.4 American Physical Society, 2020, pp. 041037  DOI: 10.1103/PhysRevX.10.041037
  * [249] Ettore Randazzo, Alexander Mordvintsev, Eyvind Niklasson, Michael Levin and Sam Greydanus  “Self-Classifying MNIST Digits”  In _Distill_ 5.8, 2020, pp. e00027.002  DOI: 10.23915/distill.00027.002
  * [250] Thomas S. Ray  “An Approach to the Synthesis of Life”  In _Artificial Life II_ C. G. Langton et al. (Eds.) Addison-Wesley Publishing Co, 1991, pp. 371–408
  * [251] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl and Christoph H. Lampert  “Icarl: Incremental Classifier and Representation Learning”  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 2001–2010
  * [252] Alfréd Rényi  “On Measures of Entropy and Information”  In _Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics_ 4.1 University of California Press, 1961, pp. 547–562  URL: https://projecteuclid.org/ebooks/berkeley-symposium-on-mathematical-statistics-and-probability/Proceedings-of-the-Fourth-Berkeley-Symposium-on-Mathematical-Statistics-and/chapter/On-Measures-of-Entropy-and-Information/bsmsp/1200512181
  * [253] Fred C. Richards, Thomas P. Meyer and Norman H. Packard  “Extracting Cellular Automaton Rules Directly from Experimental Data”  In _Physica D: Nonlinear Phenomena_ 45.1, 1990, pp. 189–202  DOI: 10.1016/0167-2789(90)90182-O
  * [254] Kyle Richardson, Hai Hu, Lawrence Moss and Ashish Sabharwal  “Probing Natural Language Inference Models through Semantic Fragments”  In _Proceedings of the AAAI Conference on Artificial Intelligence_ 34.05, 2020, pp. 8713–8721
  * [255] Sebastian Risi, Charles E. Hughes and Kenneth O. Stanley  “Evolving Plastic Neural Networks with Novelty Search”  In _Adaptive Behavior_ 18.6 Sage Publications Sage UK: London, England, 2010, pp. 470–491
  * [256] Anthony Robins  “Catastrophic Forgetting in Neural Networks: The Role of Rehearsal Mechanisms”  In _Proceedings 1993 The First New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems_ IEEE, 1993, pp. 65–68
  * [257] David E. Rumelhart, Geoffrey E. Hinton and Ronald J. Williams  “Learning Internal Representations by Error Propagation”, 1985
  * [258] Adam Rupe and James P. Crutchfield  “Local Causal States and Discrete Coherent Structures”  In _Chaos: An Interdisciplinary Journal of Nonlinear Science_ 28.7, 2018, pp. 075312  DOI: 10.1063/1.5021130
  * [259] Vladimir Ryabov and Dmitry Nerukh  “Computational Mechanics of Molecular Systems: Quantifying High-Dimensional Dynamics by Distribution of Poincaré Recurrence Times”  In _Chaos: An Interdisciplinary Journal of Nonlinear Science_ 21.3 American Institute of Physics, 2011, pp. 037113  DOI: 10.1063/1.3608125
  * [260] Chris Salzberg and Hiroki Sayama  “Complex Genetic Evolution of Artificial Self-Replicators in Cellular Automata”  In _Complex._ 10.2, 2004, pp. 33–39  DOI: 10.1002/cplx.20060
  * [261] M. San Miguel, J.. Johnson, J. Kertesz, K. Kaski, A. Díaz-Guilera, R.. MacKay, V. Loreto, P. Érdi and D. Helbing  “Challenges in Complex Systems Science”  In _The European Physical Journal Special Topics_ 214.1, 2012, pp. 245–271  DOI: 10.1140/epjst/e2012-01694-y
  * [262] Emmanuel Sapin, Olivier Bailleux and Chabrier Jean-Jacques  “Research of a Cellular Automaton Simulating Logic Gates by Evolutionary Algorithms”  In _Genetic Programming_ 2610 Berlin, Heidelberg: Springer Berlin Heidelberg, 2003, pp. 414–423  DOI: 10.1007/3-540-36599-0_39
  * [263] Hiroki Sayama  “A New Structurally Dissolvable Self-Reproducing Loop Evolving in a Simple Cellular Automata Space”  In _Artificial Life_ 5.4, 1999, pp. 343–365  DOI: 10.1162/106454699568818
  * [264] Hiroki Sayama  “Seeking Open-Ended Evolution in Swarm Chemistry”  In _2011 IEEE Symposium on Artificial Life (ALIFE)_ Paris, France: IEEE, 2011, pp. 186–193  DOI: 10.1109/ALIFE.2011.5954667
  * [265] Hiroki Sayama  “Cardinality Leap for Open-Ended Evolution: Theoretical Consideration and Demonstration by Hash Chemistry”  In _Artificial Life_ 25.2, 2019, pp. 104–116  DOI: 10.1162/artl_a_00283
  * [266] Jürgen Schmidhuber  “The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions”  In _Computational Learning Theory_ , Lecture Notes in Computer Science  Berlin, Heidelberg: Springer, 2002, pp. 216–228  DOI: 10.1007/3-540-45435-7_15
  * [267] Jürgen Schmidhuber and S. Heil  “Sequential Neural Text Compression”  In _IEEE Transactions on Neural Networks_ 7.1, 1996, pp. 142–146  DOI: 10.1109/72.478398
  * [268] Jürgen Schmidhuber, Daan Wierstra, Matteo Gagliolo and Faustino Gomez  “Training Recurrent Networks by Evolino”  In _Neural Computation_ 19.3, 2007, pp. 757–779  DOI: 10.1162/neco.2007.19.3.757
  * [269] Lambert R.. Schomaker  “Neural Network Models of Temporal Pattern Generation.”, 1990, pp. 27
  * [270] Lambert R.. Schomaker  “Simulation and Recognition of Handwriting Movements”, 1991
  * [271] Lambert R.. Schomaker  “A Neural Oscillator-Network Model of Temporal Pattern Generation”  In _Human Movement Science_ 11.1, 1992, pp. 181–192  DOI: 10.1016/0167-9457(92)90059-K
  * [272] M. Schönfinkel  “Über die Bausteine der mathematischen Logik [About the building blocks of mathematical logic]”  In _Mathematische Annalen_ 92.3, 1924, pp. 305–316  DOI: 10.1007/BF01448013
  * [273] Jimmy Secretan, Nicholas Beato, David B. D’Ambrosio, Adelein Rodriguez, Adam Campbell, Jeremiah T. Folsom-Kovarik and Kenneth O. Stanley  “Picbreeder: A Case Study in Collaborative Evolutionary Exploration of Design Space”  In _Evolutionary Computation_ 19.3, 2011, pp. 373–403  DOI: 10.1162/EVCO_a_00030
  * [274] Cosma Rohilla Shalizi and James P. Crutchfield  “Computational Mechanics: Pattern and Prediction, Structure and Simplicity”  In _Journal of Statistical Physics_ 104.3, 2001, pp. 817–879  DOI: 10.1023/A:1010388907793
  * [275] Cosma Rohilla Shalizi, Robert Haslinger, Jean-Baptiste Rouquier, Kristina Lisa Klinkner and Cristopher Moore  “Automatic Filters for the Detection of Coherent Structure in Spatiotemporal Systems”  In _Physical Review E_ 73.3, 2006  DOI: 10.1103/PhysRevE.73.036104
  * [276] Cosma Rohilla Shalizi, Kristina Lisa Shalizi and Robert Haslinger  “Quantifying Self-Organization with Optimal Predictors”  In _Physical Review Letters_ 93.11, 2004, pp. 118701  DOI: 10.1103/PhysRevLett.93.118701
  * [277] Claude E. Shannon and Warren Weaver  “The Mathematical Theory of Communication”  Urbana: University of Illinois Press, 1975
  * [278] Robert Shaw  “The Dripping Faucet as a Model Chaotic System”, The Science Frontier Express Series  Santa Cruz, CA: Aerial Press, 1984
  * [279] Hava T. Siegelmann and Eduardo D. Sontag  “On the Computational Power of Neural Nets”  In _Proceedings of the Fifth Annual Workshop on Computational Learning Theory_ , 1992, pp. 440–449
  * [280] Herbert A. Simon  “The Architecture of Complexity”  In _Proceedings of the American Philosophical Society_ 106.6 American Philosophical Society, 1962, pp. 467–482  JSTOR: https://www.jstor.org/stable/985254
  * [281] Karl Sims  “Evolving Virtual Creatures”  In _SIGGRAPH_ , 1994  DOI: 10.1145/192161.192167
  * [282] Moshe Sipper  “Co-Evolving Non-Uniform Cellular Automata to Perform Computations”  In _Physica D: Nonlinear Phenomena_ 92.3, 1996, pp. 193–208  DOI: 10.1016/0167-2789(95)00286-3
  * [283] Alvy Ray Smith  “Simple Computation-Universal Cellular Spaces”  In _Journal of the ACM_ 18.3, 1971, pp. 339–353  DOI: 10.1145/321650.321652
  * [284] John Maynard Smith and Eors Szathmary  “The Origins of Life: From the Birth of Life to the Origin of Language”  OUP Oxford, 2000
  * [285] Harold Soh and Yiannis Demiris  “Iterative Temporal Learning and Prediction with the Sparse Online Echo State Gaussian Process”  In _The 2012 International Joint Conference on Neural Networks (IJCNN)_ , 2012, pp. 1–8  DOI: 10.1109/IJCNN.2012.6252504
  * [286] Fernando Soler-Toscano, Hector Zenil, Jean-Paul Delahaye and Nicolas Gauvrit  “Calculating Kolmogorov Complexity from the Output Frequency Distributions of Small Turing Machines”  In _PLOS ONE_ 9.5, 2014, pp. e96223  DOI: 10.1371/journal.pone.0096223
  * [287] R.J. Solomonoff  “A Formal Theory of Inductive Inference. Part I”  In _Information and Control_ 7.1, 1964, pp. 1–22  DOI: 10.1016/S0019-9958(64)90223-2
  * [288] R.J. Solomonoff  “A Formal Theory of Inductive Inference. Part II”  In _Information and Control_ 7.2, 1964, pp. 224–254  DOI: 10.1016/S0019-9958(64)90131-7
  * [289] Ray J. Solomonoff  “A Preliminary Report on a General Theory of Inductive Inference”  Zator Company Cambridge, MA, 1960
  * [290] Andrea Soltoggio, John A Bullinaria, Claudio Mattiussi, Peter Durr and Dario Floreano  “Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios”, 2008, pp. 8
  * [291] Lisa B. Soros and Kenneth O. Stanley  “Identifying Necessary Conditions for Open-Ended Evolution through the Artificial Life World of Chromaria”  In _Artificial Life 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems_ The MIT Press, 2014, pp. 793–800  DOI: 10.7551/978-0-262-32621-6-ch128
  * [292] Lisa B. Soros, Kenneth O. Stanley and Joel Lehman  “Open-Endedness: The Last Grand Challenge You’ve Never Heard Of”, 2017  O’Reilly Media  URL: https://www.oreilly.com/radar/open-endedness-the-last-grand-challenge-youve-never-heard-of/
  * [293] Lee Spector, Jon Klein and Mark Feinstein  “Division Blocks and the Open-ended Evolution of Development, Form, and Behavior”  In _Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation_ , GECCO ’07  New York, NY, USA: ACM, 2007, pp. 316–323  DOI: 10.1145/1276958.1277019
  * [294] Rupesh Kumar Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino J. Gomez and Jürgen Schmidhuber  “Compete to Compute”  In _Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a Meeting Held December 5-8, 2013, Lake Tahoe, Nevada, United States_ , 2013, pp. 2310–2318  URL: https://proceedings.neurips.cc/paper/2013/hash/8f1d43620bc6bb580df6e80b0dc05c48-Abstract.html
  * [295] Kenneth O. Stanley  “Why Open-Endedness Matters”  In _Artificial Life_ 25.3, 2019, pp. 232–235  DOI: 10.1162/artl_a_00294
  * [296] Kenneth O. Stanley and Joel Lehman  “Why Greatness Cannot Be Planned: The Myth of the Objective”  Cham, Switzerland: Springer International Publishing, 2015
  * [297] Kenneth O. Stanley and Risto Miikkulainen  “Evolving Neural Networks through Augmenting Topologies”  In _Evolutionary Computation_ 10.2, 2002, pp. 99–127  DOI: 10.1162/106365602320169811
  * [298] Shyam Sudhakaran, Djordje Grbic, Siyan Li, Adam Katona, Elias Najarro, Claire Glanois and Sebastian Risi  “Growing 3D Artefacts and Functional Machines with Neural Cellular Automata”, 2021  arXiv: http://arxiv.org/abs/2103.08737
  * [299] Chihiro Sugano, Kazutaka Kanno and Atsushi Uchida  “Reservoir Computing Using Multiple Lasers With Feedback on a Photonic Integrated Circuit”  In _IEEE Journal of Selected Topics in Quantum Electronics_ 26.1, 2020, pp. 1–9  DOI: 10.1109/JSTQE.2019.2929179
  * [300] Gouhei Tanaka, Toshiyuki Yamane, Jean Benoit Héroux, Ryosho Nakane, Naoki Kanazawa, Seiji Takeda, Hidetoshi Numata, Daiju Nakano and Akira Hirose  “Recent Advances in Physical Reservoir Computing: A Review”  In _Neural Networks_ 115, 2019, pp. 100–123  DOI: 10.1016/j.neunet.2019.03.005
  * [301] Matthew E. Taylor and Peter Stone  “Cross-Domain Transfer for Reinforcement Learning”  In _Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvallis, Oregon, USA, June 20-24, 2007_ 227, ACM International Conference Proceeding Series  ACM, 2007, pp. 879–886  DOI: 10.1145/1273496.1273607
  * [302] Matthew E. Taylor, Peter Stone and Yaxin Liu  “Transfer Learning via Inter-Task Mappings for Temporal Difference Learning”  In _J. Mach. Learn. Res._ 8, 2007, pp. 2125–2167  URL: http://dl.acm.org/citation.cfm?id=1314569
  * [303] Tim Taylor  “Chapter 1 - Creativity in Evolution: Individuals, Interactions, and Environments”  In _Creative Evolutionary Systems_ , The Morgan Kaufmann Series in Artificial Intelligence  San Francisco: Morgan Kaufmann, 2002, pp. 79–108  DOI: 10.1016/B978-155860673-9/50037-9
  * [304] Tim Taylor  “Requirements for Open-Ended Evolution in Natural and Artificial Systems”, 2015  DOI: 10.48550/arXiv.1507.07403
  * [305] Tim Taylor et al.  “Open-Ended Evolution: Perspectives from the OEE Workshop in York”  In _Artificial Life_ 22.3, 2016, pp. 408–423  DOI: 10.1162/ARTL_a_00210
  * [306] Jacob Torrejon et al.  “Neuromorphic Computing with Nanoscale Spintronic Oscillators”  In _Nature_ 547.7664 Nature Publishing Group, 2017, pp. 428–431  DOI: 10.1038/nature23011
  * [307] J.. Traub, G.. Wasilkowski and H. Woźniakowski  “Information, Uncertainty, Complexity”  Reading, Mass: Addison-Wesley Pub. Co., Advanced Book Program/World Science Division, 1983
  * [308] Ph Tsalides, P.. Hicks and T.. York  “Three-Dimensional Cellular Automata and VLSI Applications”  In _IEE Proceedings E (Computers and Digital Techniques)_ 136.6 IET Digital Library, 1989, pp. 490–495  DOI: 10.1049/ip-e.1989.0067
  * [309] G.. Tsertsvadze  “Stochastic Automata and the Problem of Constructing Reliable Automata from Unreliable Elements. i(Reliable Finite Automata Design from Unreliable Elements, Using Stochastic Automaton Model)”  In _Automation and Remote Control_ 25, 1964, pp. 198–210
  * [310] J.. Tyson  “The Belousov-Zhabotinskii Reaction”  Springer Science & Business Media, 2013  GOOGLEBOOKS: hIPqCAAAQBAJ
  * [311] Kristof Vandoorne, Joni Dambre, David Verstraeten, Benjamin Schrauwen and Peter Bienstman  “Parallel Reservoir Computing Using Optical Amplifiers”  In _IEEE Transactions on Neural Networks_ 22.9, 2011, pp. 1469–1481  DOI: 10.1109/TNN.2011.2161771
  * [312] Kristof Vandoorne, Wouter Dierckx, Benjamin Schrauwen, David Verstraeten, Roel Baets, Peter Bienstman and Jan Van Campenhout  “Toward Optical Signal Processing Using Photonic Reservoir Computing”  In _Optics Express_ 16.15 Optica Publishing Group, 2008, pp. 11182–11192  DOI: 10.1364/OE.16.011182
  * [313] Kristof Vandoorne, Pauline Mechet, Thomas Van Vaerenbergh, Martin Fiers, Geert Morthier, David Verstraeten, Benjamin Schrauwen, Joni Dambre and Peter Bienstman  “Experimental Demonstration of Reservoir Computing on a Silicon Photonics Chip”  In _Nature Communications_ 5.1 Nature Publishing Group, 2014, pp. 3541  DOI: 10.1038/ncomms4541
  * [314] Francisco G. Varela, Humberto R. Maturana and Ricardo Uribe  “Autopoiesis: The Organization of Living Systems, Its Characterization and a Model”  In _Facets of Systems Science_ Springer, 1991, pp. 559–569
  * [315] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin  “Attention Is All You Need”, 2017  arXiv: http://arxiv.org/abs/1706.03762
  * [316] D. Verstraeten, B. Schrauwen, D. Stroobandt and J. Van Campenhout  “Isolated Word Recognition with the Liquid State Machine: A Case Study”  In _Information Processing Letters_ 95.6, Applications of Spiking Neural Networks, 2005, pp. 521–528  DOI: 10.1016/j.ipl.2005.05.019
  * [317] David Verstraeten, Benjamin Schrauwen and Dirk Stroobandt  “Reservoir Computing with Stochastic Bitstream Neurons”  In _Proceedings of the 16th Annual ProRISC Workshop_ , 2005, pp. 454–459  URL: http://hdl.handle.net/1854/LU-336133
  * [318] Gérard Y. Vichniac  “Simulating Physics with Cellular Automata”  In _Physica D: Nonlinear Phenomena_ 10.1, 1984, pp. 96–116  DOI: 10.1016/0167-2789(84)90253-7
  * [319] Milan Vispoel, Aisling J. Daly and Jan M. Baetens  “Progress, Gaps and Obstacles in the Classification of Cellular Automata”  In _Physica D: Nonlinear Phenomena_ 432, 2022, pp. 133074  DOI: 10.1016/j.physd.2021.133074
  * [320] P.M. Vitanyi  “Meaningful Information”  In _IEEE Transactions on Information Theory_ 52.10, 2006, pp. 4617–4626  DOI: 10.1109/TIT.2006.881729
  * [321] John Von Neumann and Arthur W. Burks  “Theory of Self-Reproducing Automata”  In _IEEE Transactions on Neural Networks_ 5.1, 1966, pp. 3–14  URL: https://cba.mit.edu/events/03.11.ASE/docs/VonNeumann.pdf
  * [322] Yaqing Wang, Quanming Yao, James T. Kwok and Lionel M. Ni  “Generalizing from a Few Examples: A Survey on Few-Shot Learning”  In _ACM computing surveys (csur)_ 53.3 ACM New York, NY, USA, 2020, pp. 1–34
  * [323] Ron Weiss, George Homsy and Radhika Nagpal  “Programming Biological Cells”  In _Proceedings of the 8th International Conference on Architectural Support for Programming Languages and Operating Systems_ , 1998, pp. 1
  * [324] Ron Weiss, George E. Homsy and Thomas F. Knight  “Toward in Vivo Digital Circuits”  In _Evolution as Computation_ Berlin, Heidelberg: Springer Berlin Heidelberg, 2002, pp. 275–295  DOI: 10.1007/978-3-642-55606-7_14
  * [325] Jason Weston, Antoine Bordes, Sumit Chopra and Tomas Mikolov  “Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks”  In _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_ , 2016  URL: http://arxiv.org/abs/1502.05698
  * [326] Jérôme Williame, Artur Difini Accioly, Damien Rontani, Marc Sciamanna and Joo-Von Kim  “Chaotic Dynamics in a Macrospin Spin-Torque Nano-Oscillator with Delayed Feedback”  In _Applied Physics Letters_ 114.23 American Institute of Physics, 2019, pp. 232405  DOI: 10.1063/1.5095630
  * [327] Shmuel Winograd and Jack D. Cowan  “Reliable Computation in the Presence of Noise”  Mit Press Cambridge, Mass., 1963
  * [328] Carl Woese  “The Universal Ancestor”  In _Proceedings of the National Academy of Sciences_ 95.12 Proceedings of the National Academy of Sciences, 1998, pp. 6854–6859  DOI: 10.1073/pnas.95.12.6854
  * [329] S.. Wolf, D.. Awschalom, R.. Buhrman, J.. Daughton, S. Molnár, M.. Roukes, A.. Chtchelkanova and D.. Treger  “Spintronics: A Spin-Based Electronics Vision for the Future”  In _Science_ 294.5546 American Association for the Advancement of Science, 2001, pp. 1488–1495  DOI: 10.1126/science.1065389
  * [330] Stephen Wolfram  “Statistical Mechanics of Cellular Automata”  In _Reviews of Modern Physics_ 55.3, 1983, pp. 601–644  DOI: 10.1103/RevModPhys.55.601
  * [331] Stephen Wolfram  “Universality and Complexity in Cellular Automata”  In _Physica D: Nonlinear Phenomena_ 10.1, 1984, pp. 1–35  DOI: 10.1016/0167-2789(84)90245-8
  * [332] Stephen Wolfram  “A New Kind of Science”  Champaign, IL: Wolfram Media, 2002  URL: https://www.wolframscience.com/nks/
  * [333] L. Wolpert  “Positional Information and the Spatial Pattern of Cellular Differentiation”  In _Journal of Theoretical Biology_ 25.1, 1969, pp. 1–47  DOI: 10.1016/S0022-5193(69)80016-0
  * [334] Brian G. Woolley and Kenneth O. Stanley  “On the Deleterious Effects of a Priori Objectives on Evolution and Representation”  In _Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation_ , GECCO ’11  New York, NY, USA: Association for Computing Machinery, 2011, pp. 957–964  DOI: 10.1145/2001576.2001707
  * [335] Andrew Wuensche  “Classifying Cellular Automata Automatically: Finding Gliders, Filtering, and Relating Space-Time Patterns, Attractor Basins, and the Z Parameter”  In _Complexity_ 4.3, 1999, pp. 47–66  DOI: 10.1002/(SICI)1099-0526(199901/02)4:3<47::AID-CPLX9>3.0.CO;2-V
  * [336] Andrew Wuensche  “Exploring Discrete Dynamics”  Luniver Press, 2011
  * [337] Andrew Wuensche and Mike Lesser  “The Global Dynamics of Cellular Automata: An Atlas of Basin of Attraction Fields of One-Dimensional Cellular Automata”, Santa Fe Institute Studies in the Sciences of Complexity. Reference Volume v. 1  Reading, Mass: Addison-Wesley, 1992
  * [338] N.. Wulff and J.. Hertz  “Learning Cellular Automaton Dynamics with Neural Networks”  In _Advances in Neural Information Processing Systems_ , 1993, pp. 631–638
  * [339] F. Wyffels and B. Schrauwen  “A Comparative Study of Reservoir Computing Strategies for Monthly Time Series Prediction”  In _Neurocomputing_ 73.10, Subspace Learning / Selected Papers from the European Symposium on Time Series Prediction, 2010, pp. 1958–1964  DOI: 10.1016/j.neucom.2010.01.016
  * [340] Larry Yaeger  “Computational Genetics, Physiology, Metabolism, Neural Systems, Learning, Vision, and Behavior or Poly World: Life in a New Context”  In _Santa Fe Institute Studies in the Sciences of Complexity_ 17 Addison-Wesley Publishing Co, 1994, pp. 263–263  URL: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.57.2435
  * [341] Tadashi Yamazaki and Shigeru Tanaka  “The Cerebellum as a Liquid State Machine”  In _Neural Networks: The Official Journal of the International Neural Network Society_ 20.3, 2007, pp. 290–297  DOI: 10.1016/j.neunet.2007.04.004
  * [342] Xiao Yang, Wanlong Chen and Frank Z. Wang  “Investigations of the Staircase Memristor Model and Applications of Memristor-Based Local Connections”  In _Analog Integrated Circuits and Signal Processing_ 87.2, 2016, pp. 263–273  DOI: 10.1007/s10470-016-0715-3
  * [343] Denis Yarats, Amy Zhang, Ilya Kostrikov, Brandon Amos, Joelle Pineau and Rob Fergus  “Improving Sample Efficiency in Model-Free Reinforcement Learning from Images”, 2019  arXiv:1910.01741
  * [344] Izzet B. Yildiz, Herbert Jaeger and Stefan J. Kiebel  “Re-Visiting the Echo State Property”  In _Neural Networks_ 35, 2012, pp. 1–9  DOI: 10.1016/j.neunet.2012.07.005
  * [345] Ozgür Yilmaz  “Reservoir Computing Using Cellular Automata”, 2014  arXiv: http://arxiv.org/abs/1410.0162
  * [346] Milan Zeleny  “Self-Organization of Living Systems: A Formal Model of Autopoiesis”  In _International journal of general system_ 4.1 Taylor & Francis, 1977, pp. 13–28
  * [347] Hector Zenil  “Compression-Based Investigation of the Dynamical Properties of Cellular Automata and Other Systems”  In _Complex Systems_ 19.1, 2010  URL: http://www.complex-systems.com/abstracts/v19_i01_a01.html
  * [348] Hector Zenil  “What Is Nature-Like Computation? A Behavioural Approach and a Notion of Programmability”  In _Philosophy & Technology_ 27.3, 2014, pp. 399–421  DOI: 10.1007/s13347-012-0095-2
  * [349] Hector Zenil, Jean-Paul Delahaye and Cédric Gaucherel  “Image Characterization and Classification by Physical Complexity”  In _Complexity_ 17.3, 2012, pp. 26–42  DOI: 10.1002/cplx.20388
  * [350] Hector Zenil, Fernando Soler-Toscano, Jean-Paul Delahaye and Nicolas Gauvrit  “Two-Dimensional Kolmogorov Complexity and an Empirical Validation of the Coding Theorem Method by Compressibility”  In _PeerJ Computer Science_ 1, 2015, pp. e23  DOI: 10.7717/peerj-cs.23
  * [351] Hector Zenil and Elena Villarreal-Zapata  “Asymptotic Behavior and Ratios of Complexity in Cellular Automata”  In _International Journal of Bifurcation and Chaos_ 23.09, 2013, pp. 1350159  DOI: 10.1142/S0218127413501599

koppelAlmostMachineindependentTheory1991koppelAlmostMachineindependentTheory1991a

  *[CA]: cellular automaton
  *[CAs]: cellular automaton
  *[RBN]: random boolean network
  *[RNNs]: recurrent neural network
  *[RC]: reservoir computing
  *[ECA]: elementary cellular automaton
  *[ECAs]: elementary cellular automaton
  *[CNN]: convolutional neural network
  *[CNNs]: convolutional neural network
  *[RNN]: recurrent neural network
  *[ESN]: echo-state network
  *[NS]: novelty search
  *[OEE]: open-ended evolution
  *[AC]: artificial chemistry
  *[NCA]: neural cellular automaton
  *[NCAs]: neural cellular automaton
  *[WADE]: Weighted Average Data Efficiency