# Making BREAD: Biomimetic strategies for Artificial Intelligence Now and in
the Future

Jeffrey L. Krichmar  William Severa  Sandia National Laboratories, Data-Driven
and Neural Computing, Albuquerque, NM, 87123, USA  Salar M. Khan  George Mason
University, Schar School, Arlington VA, 22201 USA  James L. Olds

###### Abstract

The Artificial Intelligence (AI) revolution foretold of during the 1960s is
well underway in the second decade of the 21st century. Its period of
phenomenal growth likely lies ahead. AI-operated machines and technologies
will extend the reach of Homo sapiens far beyond the biological constraints
imposed by evolution: outwards further into deep space as well as inwards into
the nano-world of DNA sequences and relevant medical applications. And yet, we
believe, there are crucial lessons that biology can offer that will enable a
prosperous future for AI. For machines in general, and for AI’s especially,
operating over extended periods or in extreme environments will require energy
usage orders of magnitudes more efficient than exists today. In many
operational environments, energy sources will be constrained. The AI’s design
and function may be dependent upon the type of energy source, as well as its
availability and accessibility. Any plans for AI devices operating in a
challenging environment must begin with the question of how they are powered,
where fuel is located, how energy is stored and made available to the machine,
and how long the machine can operate on specific energy units. While one of
the key advantages of AI use is to reduce the dimensionality of a complex
problem for which the required computation is thermodynamically expensive and
therefore energy-constrained, the fact remains that some energy is required
for functionality. Hence, the materials and technologies that provide the
needed energy represent a critical challenge towards future use-scenarios of
AI and should be integrated into their design. Here we make four
recommendations for stakeholders and especially decision makers to facilitate
a successful trajectory for this technology. First, that scientific societies
and governments coordinate Biomimetic Research for Energy-efficient, AI
Designs (BREAD)— a multinational initiative and a funding strategy for
investments in the future integrated design of energetics into AI. Second,
that biomimetic energetic solutions be central to design consideration for
future AI. Third, that a pre-competitive space be organized between
stakeholder partners and fourth, that a trainee pipeline be established to
ensure the human capital required for success in this area.

###### keywords:

AI, Energy, Edge Computing

## Artificial Intelligence’s Energy Requirements

The last few years have seen an explosion of Artificial Intelligence (AI) and
Machine Learning (ML) breakthroughs. What were once AI solutions to small, toy
problems have now become human level complex problem-solving. These solutions
have moved out of research labs and into commercial applications. However,
most AI and ML algorithms for these complex problems are implemented in large
data centers housing power hungry clusters of computers and Graphical
Processing Units (GPUs). In contrast, natural, biological intelligence is
power efficient and self-sufficient. In this article, we argue for Biomimetic
Research for Energy-efficient, AI Designs (BREAD) as AI moves toward edge
computing in remote environments far away from conventional energy sources,
and as energy consumption becomes increasingly expensive.

With the growth of the Internet, data traffic (traffic to and from data
centers) is escalating exponentially, crossing a zettabyte (1.1 ZB) in 2017
[1]. Figure 1 shows this trend. Currently, data centers consume an assessed
200 terawatt hours (TWh) each year equivalent to 1 percent of global
electricity demand [2]. A 2017 International Energy Agency (IEA) report notes
that even with the ongoing explosion of Internet traffic in data centers,
efficiency gains will result in increased electricity demand only by 3 percent
[3].

Figure 1: Energy demand will increase: The graph shows that the total ICT
electricity demand is rising in the 2020s. Data centers will likely constitute
a bulk of this demand. (Chart adapted from [1])

### Current Solutions to AI’s Energy Requirements

The rise of highly efficient data factories, known as hyperscale facilities,
use an organized uniform computing architecture that scales up to hundreds of
thousands of servers (see Figure 2). While these hyperscale centers can be
optimized for high computing efficiency, there are limits to growth due to a
variety of constraints that also affect other electrical grid consumers.
However, the shift to hyperscale facilities is a current trend, and if 80
percent of servers in US conventional data centers were moved over to
hyperscale facilities, energy consumption would drop by 25 percent, according
to Lawrence Berkeley National Laboratory report, 2016.

On average, one server in a hyperscale center can replace $3.75$ servers in a
conventional center. Hyperscale centers have a lower Power Usage Efficiency
(PUE) than conventional data centers; conventional data centers typically have
a PUE of $2.0$, while for hyperscale centers PUE is approximately $1.2$ [2,
3]. One way the hyperscale centers have cut down their PUE is through
efficiencies in cooling. By locating in cooler areas, the data centers can
ingest the cool air outside with positive results. Another solution is
employing warm water cooling loops, a solution tuned for temperate and warm
climates. An innovative solution to address the energy constraints of AI
systems is to employ an AI-powered cloud-based control recommendation system.
For example, Google employs a cloud-based AI to collect information about the
data center cooling system from thousands of physical sensors prior to feeding
this information into deep neural networks. The networks then compute
predictions for how different combinations of possible activities will affect
future energy consumption [4].

Although hyperscale centers and smart cooling strategies can lower energy
consumption, these solutions do not address applications where AI is operating
at the edge or when AI is deployed in extreme conditions far away from
convenient power supplies. We believe that this is where future AI systems are
headed.

Figure 2: As smaller and inefficient data centers shut down, efficient
hyperscale data centers are anticipated to make half of the data-center
electricity demand by 2020. Image adapted from [3].

Our view is that there is a pressing need to address the energy issue as it
applies to the future of AI and ML. While there is a growing research effort
towards developing efficient machine learning methods for embedded and
neuromorphic processors [5, 6, 7, 8, 9], we recognize that these methods do
not address the full needs of future applications, despite offering compelling
first steps. Generally, current methods modify existing techniques rather than
develop de novo algorithms. We emphasize how biological evolution has
addressed this problem, and survey biologically inspired algorithms and
computers. Furthermore, we discuss how the potential of future energy sources
and computing architectures meet these power requirements. Finally, we
recommend a set of policies to implement a coherent US government-wide
investment strategy to ensure that future AI and ML are energetically
practicable.

## Current State

### Edge computing in Remote and Hostile Environments

Trends in many human-built systems point to directions where sensing,
processing, and actuation is situated on distributed platforms. The emerging
Internet-of-Things (IoT) are cyber technologies [10], hardware and software,
that interact with physical components in environments which may or may not be
populated by humans. IoT devices are often thought of as the “edge” of a
large, sophisticated cloud processing infrastructure. Processing data at the
“edge,” reduces system latency by removing the delays in the aggregation tiers
of the information technology infrastructure [11, 12, 13]. In addition to
minimizing latency, edge processing increases system security and mitigates
privacy concerns when processing data in the cloud. Finally, in cases where
the data path between the edge and user is very long, such computation can, by
feature extraction, reduce the dimensionality and hence expense sending
information. However, edge processing may be far away from power sources and
may need to operate without intervention over long time periods.

Space and deep ocean exploration, which will most likely require AI and ML
solutions, represent environments that are inherently hostile to the circuitry
that sub-serves current AI and ML technologies. Gamma-ray bursts, solar
weather, micrometeorite impacts are all exemplars of how space environments
from Low Earth Orbit (LEO) through Deep Space are challenging. Planetary
missions such as NASA’s Curiosity Mars rover have revealed additional
challenges from weather (such as sandstorms) that have put missions at risk.
Radioisotope thermal generation power was added to the Curiosity Mars rover
design to combat the vulnerabilities to solar energy systems on previous
missions. While Curiosity’s computational systems do not constitute true AI,
their power demands are of a similar order of magnitude. Limitations on the
availability of radioisotopes [14] combined with safety concerns [15, 16]
during the launch phase will constrain future deep space missions that might
use nuclear power.

As with Space, in deep ocean environments, power constraints are also a
current challenge. Current Autonomous Underwater Vehicles (AUVs) have limited
capabilities due to limitations on energy storage and the availability of fuel
sources. Furthermore, the extremely high pressures of deep-sea environments
offer their challenges, not only to energy supply for AI but also to the mass
and construction of protection containers for the electronics. Ocean glider
AUV’s use buoyancy engines with fins to convert force in the vertical
direction to horizontal motion [17, 18, 19, 20, 21]. While very slow, such
UAV’s are far less energy-constrained than other current technologies.
However, the power generated by such engines is not currently suitable for
powering AI systems. Batteries are used for such functions and must be
recharged at the ocean surface using photo-voltaic cells. There are proposals
to use nuclear fission power generation to enable deep-sea battery recharging
stations for military UAV’s, though these remain at the development stage and
have similar safety considerations to those mentioned above for space [22].

## Existence proof, human brains as efficient energy consumers

The original goal of AI, which is still an important goal, was to extract
principles from human intelligence. On the one hand, these principles would
allow for a better understanding of intelligence and what makes us human. On
the other hand, we could use those principles to build _intelligent_
artifacts, such as robots, computers, and machines. In both cases, the
implication is to use human intelligence as a use case, which derives from the
function of the brain. We believe that there are also important energy
efficiency principles that can be extracted from neurobiology and applied to
AI. Therefore, nervous system can provide much inspiration for the
construction of low power intelligent systems.

The human nervous system is under tight metabolic constraints. These
constraints include the essential role of glucose as fuel under conditions of
non-starvation, the continuous demand for approximately 20 percent of the
human body’s total energy utilization, and the lack of any effective energy-
reserve among others [23]. And yet, as is well known, the brain operates on a
mere 20 watts of power, approximately the same power required for the interior
light of a typical household refrigerator. While being severely metabolically
constrained is at one level a disadvantage, evolution has optimized brains in
ways that lead to incredibly efficient representations of important
environmental features that stand distinct from those employed in current
digital computers.

The brain utilizes many means to reduce functional metabolic energy
utilization. Indeed, one can observe at every level of the nervous system
strategies to maintain high performance and information transfer, while
minimizing energy expenditure. These range from ion channel distributions, to
coding methods, to wiring diagrams (connectomes). Many of these methods could
inspire new methodologies for constructing power efficient artificial
intelligent systems.

At the neuronal coding level, the brain uses several strategies to reduce
neural activity without sacrificing performance. Neural activity, (i.e., the
generation of an action potential, the return to resting state, and synaptic
processing) is very costly from an energetics standpoint, and this can
determine the optimal number of spikes to encode either an engram or the
neural representation of a new stimulus [24, 25]. Such sparse coding
strategies appear to be ubiquitous throughout the brain [26, 27, 28].

Furthermore, dimensionality reduction methods from machine learning can
explain many neural representations, especially in sensory systems [29].
Because brains face strict constraints on metabolic cost [24] and also
consequent to the widespread existence of anatomical bottlenecks [30], which
often force the information stored in a large number of neurons to be
compressed into an order of magnitude smaller population of downstream
neurons, (for example, storing information from 100 million photoreceptors in
1 million optic nerve fibers), reducing the number of variables required to
represent a particular stimulus space features prominently in efficient
current coding theories of brain function [31, 32, 33]. Such views posit that
the brain performs dimensionality reduction by maximizing mutual information
between the high-dimensional input and the low-dimensional output of neuronal
populations.

The healthy brain must respond quickly to stimuli and changes in the
environment. However, this implies an increase in neural activity, which would
be energetically costly. Evidence suggests that the brain utilizes strategies
to maintain a constant rate of activity. For example, the nervous system can
respond quickly to perturbations by shifting the specific timing rather than
increasing the absolute number of spikes [34]. Moreover, the balance of
excitation and inhibition can further maintain a steady rate of neural
activity while still being responsive [35, 36]. In these ways, the overall
energy utilization of the human brain is relatively constant, while the local
rate of energy consumption varies widely and is dependent upon functional
neuronal activity and the balance between excitatory and inhibitory neurons
[37]

At a macroscopic scale, the brain saves energy by minimizing the wiring
between neurons and brain regions (i.e., number of axons), yet still
communicates information at a high-level of performance [38]. Unlike current
electronic chips, the brain packs its wiring into a three-dimensional space,
which not only reduces the overall volume but also can reduce the energy cost.
Energy is further conserved by maintaining high local connectivity with sparse
distal connectivity. White matter, which are myelinated axons that transmit
information over long distances in the nervous system, make up about half the
human brain but use less energy than gray matter (neuronal somata and
dendrites) because of the scarcity of N+ and K+ ion channels along these axons
[39]. These myelinated axons speed up signal propagation and reduce the volume
of matter in the brain. However, information transfer between neurons and
brain areas is still preserved by the overall architecture, which essentially
is a small world network [40, 41]. That is, even though the probability of any
two distal cortical neurons being connected is extremely low, any two neurons
are only a few connections away from each other. The nervous system also
optimizes energy consumption at the cellular and sub-cellular levels. Glucose
utilization reflects both the Na+-K+ ATPase used to repolarize axonal ionic
gradients and the energy costs of neurotransmitter re-uptake. These together
represent the majority of the brain’s energy budget and must be efficient for
performance [42].

The brain’s efficient power consumption may have a basis in thermodynamics and
information theory. It has been suggested that any self-organizing system that
is at equilibrium with its environment must minimize its free energy [43]. In
other words, the system must adapt or evolve to resist a natural tendency
toward disorder in an ever-changing environment. Top-down signals from
downstream areas (e.g., frontal cortex or parietal cortex) can realize
predictive coding [44, 45, 35]. Predicting outcomes reduces surprises, which
leads to more energy efficient processing [43]. In this way organisms minimize
the long-term average of surprise, which is the inverse of entropy, by
predicting future outcomes. In essence, they minimize the expenditures
required to deal with unanticipated events. The idea of minimizing free energy
has close ties to many existing brain theories, such as the Bayesian brain,
predictive coding, cell assemblies, and Infomax, as well as an evolutionary-
inspired theory called Neural Darwinism or neuronal group selection [43].

The brain represents an important existence proof that extraordinarily
efficient natural intelligence can compute in very complex ways within harsh,
dynamic environments. Beyond an existence proof, brains provide an opportunity
for reverse-engineering in the context of machine learning methods and
neuromorphic computing.

## Future AI

### Edge computing in Remote and CMOS-hostile environments

Unless human beings can be "radiation-hardened," robotic space probes will
continue to dominate exploration and exploitation of space in domains ranging
from low earth orbit to interstellar exploration. All of these domains are
subject to a variety of hazards which are potentially hazardous to CMOS-based
AI. These include collisions with high energy photons (such as gamma rays),
micrometeorites, planetary weather, and anthropogenic attacks.

In many of these space domains, AI will be the preferred computational
modality because of latency issues related to long-distance communication with
Earth-based controllers. Operating in such domains will have the additional
challenge of energetic constraints because readily available solar power may
not always be available in domains such as the moon, solar system planets with
weather and deep space (including interstellar). The primary alternative
energy source for such domains is nuclear (both fission and fusion-based).
Such power sources are in contrast to the current nuclear technologies used
for missions such as the Mars Curiosity Rover. While break-even fusion power
has yet to be demonstrated on Earth, the abundance of fusion fuels in the
solar system makes such power sources attractive. In all these cases, the
nuclear technology must have a similar resiliency to that of the AI in terms
of hazards, and it will be optimal to consider such requirements holistically
at the design stage.

Other such remote domains for AI exist. These include machines that must
operate autonomously during the Antarctic winter, deep sea AI and
military/law-enforcement applications. In each of these cases, the exact
design criteria will be different, yet all will face unique power constraints
that are in addition to the constraints to the AI itself.

### Neuromorphic Computing and other Bio-Inspired Engineering Approaches

A key component in pursuing brain- and neural- inspired computing, coding, and
algorithms lies in the currently shifting landscape of computing
architectures. Moore’s law, which has dictated the development of ever-smaller
transistors since the 1960s, has become more and more difficult to follow,
leading many to claim its demise [46]. This has inspired renewed interest in
heterogeneous and non-Von Neumann computing platforms [47, 48], which take
inspiration from the efficiency of the brain’s architecture. Neuromorphic
architectures can offer order-of-magnitude improvement in performance-per-Watt
compared to traditional CPUs and GPUs [49, 50, 51]. This enables, for example,
IBM’s TrueNorth chip to power convolutional neural networks for embedded
gesture recognition at less than one Watt [52].

Neuromorphic architectures refer to a wide variety of computing hardware
platforms [53], from sensing [54, 55] to processing [49, 50], analog [56] to
digital [49, 57]. However, in most cases the defining characteristics take
inspiration from the brain:

  1. 1.

Massively parallel, simple integrating processing units (neurons)

  2. 2.

Sparse and dynamic low-precision communication via ‘spikes’.

  3. 3.

Event-driven, asynchronous operation.

This event-driven, distributed, processor-in-memory approach provides robust,
low-power processing compatible with many neural-inspired machine learning and
artificial intelligence applications [58]. Hence, size, weight and power
(SWaP) constrained environments, such as edge and IoT devices, can leverage
increased effective remote computation capabilities and provide real-time,
low-latency intelligent and adaptive behavior. Moreover, the often noisy
nature of learned artificial intelligence systems (some incorporate noise by
design [59]) may lead to more robust computation in extreme environments such
as space.

Figure 3: The image shows that in the design of bio-mimetic circuitry, either
the existing hardware will be slightly adjusted to copy the behavior of brain
parts, or new computing architecture will be designed so that it completely
emulate the high energy-efficient biotic neural structures (Image adapted from
[60]).

Power consumption in large-scale computing systems is also forcing the
development of more efficient computing platforms [61]. In data centers,
heterogeneous (non-spiking) architectures are improving performance and
latency, exemplified by a $30$–$80$x improvement on deep learning tasks [62,
63], and new neuromorphic architectures, such as Intel’s Loihi, are themselves
heterogeneous which improves communication between neural and conventional
cores [64]. Emerging neural computing platforms may benefit traditional large-
scale computation both indirectly (e.g., system health [65], failure
prediction [66]) and directly (e.g., meshing, surrogate models [67]), and
recent work indicates that neuromorphic processors may be useful for direct
computation due their high-communication, highly-parallel nature [68, 69, 70,
71].

Neural inspiration has also impacted data collection in the form of spiking
neuromorphic sensors which generally follow the same three characteristics as
neuromorphic architectures. The two most common categories are silicon
cochleas [72, 73] and retina-inspired event-driven cameras [74, 75], though
neuromorphic olfaction is also under active research and development [76].
Neuromorphic sensors can often be thought of as a method for high-speed
preprocessing, fundamentally changing the sample space. For example, for
imagery this allows far low-bandwidth, high-sampling, and high-dynamic range
imagery [77, 75]. These benefits, in turn, have enabled low-latency, low-power
applications such as gesture recognition [52, 78], robotic control [79, 80],
and movement determination [81, 82]. Spiking sensors are innately compatible
with spiking neuromorphic processors, and combining neuromorphic sensors with
a neuromorphic processor can avoid the costly conversion between binary data
formats and spikes.

Computational requirements of artificial intelligence algorithms limit their
remote applications today. Consequently, most current consumer or commercial
machine learning technologies are reliant on connections to remote data
centers. However, as neuromorphic technologies transition from research
platforms to everyday products, learning systems can and will proliferate in
capability and scope. Combined with the expected growth of edge and IoT
devices, we can expect persistence and pervasive learning devices. These
learning devices, extensions of current trends in smart devices (e.g., digital
assistants, smart home control, wearables), will be enhanced with personalized
online learning and enabled with adaptive, intelligent and context-dependent
perception and behaviors. Ultra-low energy neuromorphic chips will power
remote computation at the milliwatt scale. In the industrial, medical and
security spaces, the same technologies will provide low-powered sensors
capable of extended deployment in a variety of extreme environments. Privacy,
ethical, and environmental concerns will need to be balanced against
convenience, productivity, and safety.

There is ample precedent for biologically-inspired approaches to engineering
design that may lead to energy-efficient AI and edge computing. Both designing
algorithms to mimic the brain’s behavior, and building new computer hardware
that mimic neural dynamics can lead to energy efficiency (see Figure 3)[60].
Furthermore, energy efficiency can be inspired by observing nature’s non-
neural solutions. For example, the wing of an airplane takes inspiration from
the wings of flying animals (birds, bats, and insects). The shape of a modern
naval submarine has evolved from early boat-like designs prevalent during the
First and Second World Wars towards a more streamlined whale-like shape. Even
DNA-based computation–by itself incredibly energetically efficient–takes
inspiration from the conserved transcription information transfer mechanism of
Earth’s biosphere.

From a computational perspective, it is not only the brain which offers
precedent. The adaptive immune system, with its sophisticated "learning and
memory" through selection also represents a low-energy approach to artificial
intelligence that may eventually have applications to AI-enhanced cyber-
security applications [83, 84, 85, 86, 87, 88]. This selectionist approach,
which was inspired by the immune system, led to an influential brain theory
where the synaptic selection took place during neural development and through
experiential synaptic plasticity [89, 90]. Such a Darwinist approach can lead
to efficient neural network structures.

As edge computing and mobile sensing devices become ubiquitous, efficient
mobility, whether on land, air, or water will become increasingly important.
Biological organisms have evolved to leverage their environment, and this
morphological computation can lead to efficient movement and information
processing [91]. For example, bipedal walking is somewhat of a controlled
fall, where energy is conserved by allowing gravity to take over after the
swing phase of a step. This strategy has been adopted in passive walker robots
that utilize orders of magnitude less energy than conventional walking robots
[92, 93]. Birds of prey and long-range migrating birds take advantage of
thermal plumes to reduce energy usage during flight [94, 95, 96]. Gliders have
mimicked this strategy in their flight control systems [97, 98, 99]. Similar
to many fish and marine mammals, oceanographic submersible gliders can harvest
energy from the heat flow of thermal gradients [17, 18, 19, 20, 21]. These
submersible gliders can operate across thousands of kilometers over months to
years. Some fish species and flying insects alter their environment (i.e., the
water or air vortices) to create additional thrust [100, 101]. Social insects
and bacterial colonies have inspired highly distributed robots or computing
systems [102, 103]. In these cases, each agent has very low power computation
requirements, and no single agent is a point of failure. However, the
interactions between these agents can lead to complex problem-solving. Taken
together, future AI systems that take inspiration from biology and other
energy harvesting approaches will have a distinct advantage for long-term
operation in harsh or remote environments.

Figure 4: A coherent strategy for ensuring that AI’s of the future are not
starved for power. The four recommendations presented in this Perspective work
together to ensure government support for innovation, research, integrated
design, and a human capital pipeline.

## Policy Recommendations

AI is on a trajectory to fundamentally change society in much the same way
that the industrial revolution did. Even without the development of General
Artificial Intelligence, the trend is towards human-machine partnerships that
collectively will have the ability to substantially extend the reach of humans
in multiple domains (e.g., space, cyber, deep sea, nano). However, as with
many things, there is no free lunch: AI will require energy inputs that we
believe must be accounted for at all stages of the AI design process. We
believe that such design solutions should leverage the solutions that biology,
especially the human brain, has evolved to be energy efficient without
sacrificing functionality. These solutions are critical components to what we
call _intelligence._

### Recommendation 1: A Multinational initiative to make BREAD

To accomplish the above, we advocate that under the auspices of the
Association for Computing Machinery (ACM), American Psychological Association
(APA), Institute of Electrical and Electronics Engineers (IEEE), Society for
Neuroscience, and multilateral governmental agreements, a global technological
innovation initiative be launched, called Biomimetic Research for Energy-
efficient, AI Designs (BREAD), to coordinate and catalyze research public and
private investments into holistic AI design. Energy sustainability would be
central in BREAD, but the initiative would encompass all aspects of AI from
hardware to sensors.

Within the United States, we propose that President’s Science Advisor should
name the chair of a BREAD-inspired inter-agency coordinating sub-committee
under the auspices of the National Science and Technology Council. In the US,
BREAD would be truly cross-cutting and should receive substantial and explicit
funding in relevant agency budgets including, DOE, NSF, DOD, and NIH. Given
the stakes, an annual expenditure of approximately $3B over five years would
be appropriate. The United States portion of BREAD should aim to catalyze
advances in bridging knowledge gleaned from the life sciences to AI energetics
at US research institutions, the national labs and in coordination with
industry. We additionally recommend that DOE stand-up a new BREAD office with
overall government-wide budget authority analogous to NSF with STEM education.

### Recommendation 2: Integrate biomimetic energetic solutions into future AI
designs

It is clear that AI will continue to be energetically constrained into the
future. Even as this technological revolution continues to play out with ever
more efficient strategies, thermodynamic considerations alone make the
energetic considerations important. The source and availability of power to
AI’s will be critical to their function under all conditions, but particularly
those at the Edge such as IoT, Space or other environments that are inherently
hostile to CMOS (or future successor chip technologies). Further, because of
the absolute requirement for adequate energy provision, future AI development
will require an integrated design process where energy supply is not an add-on
or assumed. Such integration of design already is prevalent in the design of
new commercial aircraft and many medical devices; however, is has been
hitherto remarkably absent in AI design. Indeed many instances of current AI
(such as deep learning) approach the problem by purposeful situating of data
centers close to sources of abundant and cheap electric power (such as hydro
plants). We believe that a more fruitful approach should be to leverage the
solutions evolved by biology (nervous system, metabolism, morphology, etc.)
and embed such design thinking into future AI design. Thus, while deep
learning AI’s might continue to use the data center model for current and
future applications that are similar to those used today by Google, Facebook,
IBM, and other large corporations, AI operating in the context of Edge or
otherwise constrained environments would use an architecture that integrate
power provision and management based on biomimetic models.

### Recommendation 3: A pre-competitive international research lab modeled on
IMEC

As with CMOS prototype design and fabrication, the initial costs of
integrating biomimetic solutions into AI are likely to be front-loaded. Hence,
we recommend that stakeholder industrial partners come together with
governments to build out a pre-competitive research laboratory modeled on IMEC
in Leuven Belgium. IMEC, a public-private partnership, allows CMOS design
firms across the globe to prototype new chips in a state-of-the-art
environment that preserves intellectual property. In the context of BREAD, an
IMEC model would serve a similar function for numerous companies active in the
AI industrial space prior to production. Such a public-private partnership
could catalyze the technological innovations necessary for success.

### Recommendation 4: A Trainee Pipeline

We believe that it is critical to establish a pipeline of scientists who can
integrate knowledge from biology, computer science, neuroscience, and
engineering to build new human capital expertise at the intersection of AI and
power management issues. To accomplish that, aligned with BREAD, research
institutions should consider new graduate offerings at this nexus. In the
United State, doctoral support for such students should be provided by the
Engineering Directorate of the NSF and through the NSF’s prestigious Graduate
Research Fellowship Program. At the same time, there should be support for
post-doctoral trainees in this area, both at the National Labs, but also
through individual fellowships from either the Engineering Directorate of NSF,
the Department of Energy or the Department of Defense.

## Conclusion

In conclusion, we see the future development of AI as requiring new strategies
for embedding energy demands of the machine into the overall design strategy.
From our standpoint, this must include biomimetic solutions. As indicated
above, there is much precedence for this type of engineering in other high
aspects of technology, especially those that must operate in challenging
environments. Now such engineering must be applied to future AI design so that
the technological trajectory of this paradigm-changing technology is secure.

## References

  * [1] Andrae, S. A. & Edler, T.  On global electricity usage of communication technology: Trends to 2030.  _Challenges_ 6, DOI: 10.3390/challe6010117 (2015).
  * [2] Jones, N.  The information factories.  _Nature_ 561, 163–166 (2018).
  * [3] Digitalization: A new era in energy?  Report, International Energy Agency (2017).
  * [4] Shehabi, A. _et al._ United states data center energy usage report.  Tech. Rep., LBNL (2016).
  * [5] Howard, A. G. _et al._ Mobilenets: Efficient convolutional neural networks for mobile vision applications.  _arXiv preprint arXiv:1704.04861_ (2017).
  * [6] Severa, W., Vineyard, C. M., Dellana, R., Verzi, S. J. & Aimone, J. B.  Whetstone: A method for training deep artificial neural networks for binary communication.  _arXiv preprint arXiv:1810.11521_ (2018).
  * [7] Hunsberger, E. & Eliasmith, C.  Training spiking deep networks for neuromorphic hardware.  _arXiv preprint arXiv:1611.05141_ (2016).
  * [8] Esser, S. K., Appuswamy, R., Merolla, P., Arthur, J. V. & Modha, D. S.  Backpropagation for energy-efficient neuromorphic computing.  In _Advances in Neural Information Processing Systems_ , 1117–1125 (2015).
  * [9] Rastegari, M., Ordonez, V., Redmon, J. & Farhadi, A.  Xnor-net: Imagenet classification using binary convolutional neural networks.  In _European Conference on Computer Vision_ , 525–542 (Springer, 2016).
  * [10] Atzori, L., Iera, A. & Morabito, G.  The internet of things: A survey.  _Computer Networks_ 54, 2787–2805, DOI: 10.1016/j.comnet.2010.05.010 (2010).
  * [11] Mao, Y. Y., Zhang, J. & Letaief, K. B.  Dynamic computation offloading for mobile-edge computing with energy harvesting devices.  _IEEE Journal on Selected Areas in Communications_ 34, 3590–3605, DOI: 10.1109/Jsac.2016.2611964 (2016).
  * [12] Hu, Y. C., Patel, M., Sabella, D., Sprecher, N. & Young, V.  Mobile edge computing-a key technology towards 5g.  _ETSI (European Telecommunications Standards Institute)_ ISBN No. 979-10-92620-08-5 (2015).
  * [13] Shi, W. S., Cao, J., Zhang, Q., Li, Y. H. Z. & Xu, L. Y.  Edge computing: Vision and challenges.  _Ieee Internet of Things Journal_ 3, 637–646, DOI: 10.1109/Jiot.2016.2579198 (2016).
  * [14] Aebersold, P. C.  Production and availability of radioisotopes.  _The Journal of clinical investigation_ 28, 1247–1254 (1949).
  * [15] Availability of radioisotopes.  _Chemical & Engineering News Archive_ 26, 2260, DOI: 10.1021/cen-v026n031.p2260 (1948).  https://doi.org/10.1021/cen-v026n031.p2260.
  * [16] Al Kattar, Z., El Balaa, H., Nassredine, M. & Haydar, P. M.  Radiation safety issues relevant to radioisotope production medical cyclotron.  In _Advances in Biomedical Engineering (ICABME), 2015 International Conference on_ , 178–181 (IEEE, 2015).
  * [17] Rudnick, D. L., Davis, R. E. & Sherman, J. T.  Spray underwater glider operations.  _Journal of Atmospheric and Oceanic Technology_ 33, 1113–1122, DOI: 10.1175/Jtech-D-15-0252.1 (2016).
  * [18] Rudnick, D. L., Sherman, J. T. & Wu, A. P.  Depth-average velocity from spray underwater gliders.  _Journal of Atmospheric and Oceanic Technology_ 35, 1665–1673, DOI: 10.1175/Jtech-D-17-0200.1 (2018).
  * [19] Rudnick, D. L.  Ocean research enabled by underwater gliders.  _Annual Review of Marine Science_ 8, 519–541, DOI: 10.1146/annurev-marine-122414-033913 (2016).
  * [20] Schofield, O. _et al._ Slocum gliders: Robust and ready.  _Journal of Field Robotics_ 24, 473–485, DOI: 10.1002/rob.20200 (2007).
  * [21] Webb, D. C., Simonetti, P. J. & Jones, C. P.  Slocum: An underwater glider propelled by environmental energy.  _Ieee Journal of Oceanic Engineering_ 26, 447–452, DOI: Doi10.1109/48.972077 (2001).
  * [22] Hambling, D.  Why russia is sending robotic submarines to the arctic.  _BBC Future_ (2017).
  * [23] Sokoloff, L.  The metabolism of the central nervous system in vivo.  _Handbook of Physiology, Section I, Neurophysiology_ 3, 1843–1864 (1960).
  * [24] Lennie, P.  The cost of cortical computation.  _Current biology_ 13, 493–497 (2003).
  * [25] Levy, W. B. & Baxter, R. A.  Energy efficient neural codes.  _Neural Comput_ 8, 531–43 (1996).
  * [26] Beyeler, M., Rounds, E., Carlson, K., Dutt, N. & Krichmar, J. L.  Sparse coding and dimensionality reduction in cortex.  _bioRxiv_ 149880 (2017).
  * [27] Olshausen, B. A. & Field, D. J.  Sparse coding with an overcomplete basis set: A strategy employed by v1?  _Vision research_ 37, 3311–3325 (1997).
  * [28] Olshausen, B. A. & Field, D. J.  Sparse coding of sensory inputs.  _Current opinion in neurobiology_ 14, 481–487 (2004).
  * [29] Beyeler, M., Dutt, N. & Krichmar, J. L.  3d visual response properties of mstd emerge from an efficient, sparse population code.  _Journal of Neuroscience_ 36, 8399–8415 (2016).
  * [30] Ganguli, S. & Sompolinsky, H.  Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis.  _Annual review of neuroscience_ 35, 485–508 (2012).
  * [31] Atick, J. J.  Could information theory provide an ecological theory of sensory processing?  _Network: Computation in Neural Systems_ 22, 4–44 (2011).
  * [32] Barlow, H.  Redundancy reduction revisited.  _Network: computation in neural systems_ 12, 241–253 (2001).
  * [33] Linsker, R.  Perceptual neural organization: Some approaches based on network models and information theory.  _Annual review of Neuroscience_ 13, 257–281 (1990).
  * [34] Malyshev, A., Tchumatchenko, T., Volgushev, S. & Volgushev, M.  Energy-efficient encoding by shifting spikes in neocortical neurons.  _European Journal of Neuroscience_ 38, 3181–3188 (2013).
  * [35] Sengupta, B., Laughlin, S. B. & Niven, J. E.  Balanced excitatory and inhibitory synaptic currents promote efficient coding and metabolic efficiency.  _PLoS computational biology_ 9, e1003263 (2013).
  * [36] Yu, L., Shen, Z., Wang, C. & Yu, Y.  Efficient coding and energy efficiency are promoted by balanced excitatory and inhibitory synaptic currents in neuronal network.  _Frontiers in cellular neuroscience_ 12, 123 (2018).
  * [37] Olds, J., Frey, K. & Agranoff, B.  Sequential double-label deoxyglucose autoradiography for determining cerebral metabolic change: Origins of variability within a single brain.  _Neuroprotocols_ 12–24 (1994).
  * [38] Laughlin, S. B. & Sejnowski, T. J.  Communication in neuronal networks.  _Science_ 301, 1870–1874 (2003).
  * [39] Harris, J. J. & Attwell, D.  The energetics of cns white matter.  _Journal of Neuroscience_ 32, 356–371 (2012).
  * [40] Sporns, O.  Small-world connectivity, motif composition, and complexity of fractal neuronal connections.  _Biosystems_ 85, 55–64 (2006).
  * [41] Sporns, O. & Zwi, J. D.  The small world of the cerebral cortex.  _Neuroinformatics_ 2, 145–162 (2004).
  * [42] Engl, E. & Attwell, D.  Non-signalling energy use in the brain.  _The Journal of physiology_ 593, 3417–3429 (2015).
  * [43] Friston, K.  The free-energy principle: a unified brain theory?  _Nature Reviews Neuroscience_ 11, 127 (2010).
  * [44] Sengupta, B., Stemmler, M. B. & Friston, K. J.  Information and efficiency in the nervous system—a synthesis.  _PLoS computational biology_ 9, e1003157 (2013).
  * [45] Clark, A.  Whatever next? predictive brains, situated agents, and the future of cognitive science.  _Behavioral and brain sciences_ 36, 181–204 (2013).
  * [46] Waldrop, M. M.  The chips are down for moore’s law.  _Nature News_ 530, 144 (2016).
  * [47] Chung, E. S., Milder, P. A., Hoe, J. C. & Mai, K.  Single-chip heterogeneous computing: Does the future include custom logic, fpgas, and gpgpus?  In _Microarchitecture (MICRO), 2010 43rd Annual IEEE/ACM International Symposium on_ , 225–236 (IEEE, 2010).
  * [48] Shalf, J. M. & Leland, R.  Computing beyond moore’s law.  _Computer_ 48, 14–23 (2015).
  * [49] Merolla, P. A. _et al._ A million spiking-neuron integrated circuit with a scalable communication network and interface.  _Science_ 345, 668–673 (2014).
  * [50] Indiveri, G. _et al._ Neuromorphic silicon neuron circuits.  _Frontiers in neuroscience_ 5, 73 (2011).
  * [51] Hasler, J. & Marr, H. B.  Finding a roadmap to achieve large neuromorphic hardware systems.  _Frontiers in neuroscience_ 7, 118 (2013).
  * [52] Amir, A. _et al._ A low power, fully event-based gesture recognition system.  In _CVPR_ , 7388–7397 (2017).
  * [53] Schuman, C. D. _et al._ A survey of neuromorphic computing and neural networks in hardware.  _arXiv preprint arXiv:1705.06963_ (2017).
  * [54] Posch, C., Serrano-Gotarredona, T., Linares-Barranco, B. & Delbruck, T.  Retinomorphic event-based vision sensors: bioinspired cameras with spiking output.  _Proceedings of the IEEE_ 102, 1470–1484 (2014).
  * [55] Liu, S.-C. & Delbruck, T.  Neuromorphic sensory systems.  _Current opinion in neurobiology_ 20, 288–295 (2010).
  * [56] Fieres, J., Schemmel, J. & Meier, K.  Realizing biological spiking network models in a configurable wafer-scale hardware system.  In _Neural Networks, 2008. IJCNN 2008.(IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on_ , 969–976 (IEEE, 2008).
  * [57] Furber, S. B. _et al._ Overview of the spinnaker system architecture.  _IEEE Transactions on Computers_ 62, 2454–2467 (2013).
  * [58] Neftci, E.  Data and power efficient intelligence with neuromorphic learning machines.  _iScience_ (2018).
  * [59] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R.  Dropout: a simple way to prevent neural networks from overfitting.  _The Journal of Machine Learning Research_ 15, 1929–1958 (2014).
  * [60] Calimera, A., Macii, E. & Poncino, M.  The human brain project and neuromorphic computing.  _Funct Neurol_ 28, 191–6, DOI: 10.11138/FNeur/2013.28.3.191 (2013).
  * [61] Hemmert, S.  Green hpc: From nice to necessity.  _Computing in Science & Engineering_ 12, 8–10 (2010).
  * [62] Putnam, A. _et al._ A reconfigurable fabric for accelerating large-scale datacenter services.  _ACM SIGARCH Computer Architecture News_ 42, 13–24 (2014).
  * [63] Jouppi, N. P. _et al._ In-datacenter performance analysis of a tensor processing unit.  In _Computer Architecture (ISCA), 2017 ACM/IEEE 44th Annual International Symposium on_ , 1–12 (IEEE, 2017).
  * [64] Davies, M. _et al._ Loihi: A neuromorphic manycore processor with on-chip learning.  _IEEE Micro_ 38, 82–99 (2018).
  * [65] Das, A., Mueller, F., Siegel, C. & Vishnu, A.  Desh: deep learning for system health prediction of lead times to failure in hpc.  In _Proceedings of the 27th International Symposium on High-Performance Parallel and Distributed Computing_ , 40–51 (ACM, 2018).
  * [66] Bouguerra, M. S., Gainaru, A. & Cappello, F.  Failure prediction: what to do with unpredicted failures.  In _28th IEEE international parallel and distributed processing symposium_ , vol. 2 (2013).
  * [67] Melo, A., Cóstola, D., Lamberts, R. & Hensen, J.  Development of surrogate models using artificial neural network for building shell energy labelling.  _Energy Policy_ 69, 457–466 (2014).
  * [68] Aimone, J. B., Parekh, O. & Severa, W.  Neural computing for scientific computing applications: more than just machine learning.  In _NCS_ (2017).
  * [69] Severa, W., Lehoucq, R., Parekh, O. & Aimone, J. B.  Spiking neural algorithms for markov process random walk.  In _International Joint Conference on Neural Networks 2018_ (IEEE, 2018).
  * [70] Jonke, Z., Habenschuss, S. & Maass, W.  Solving constraint satisfaction problems with networks of spiking neurons.  _Frontiers in neuroscience_ 10, 118 (2016).
  * [71] Lagorce, X. & Benosman, R.  Stick: spike time interval computational kernel, a framework for general purpose computation using neurons, precise timing, delays, and synchrony.  _Neural computation_ 27, 2261–2317 (2015).
  * [72] Chan, V., Liu, S.-C. & van Schaik, A.  Aer ear: A matched silicon cochlea pair with address event representation interface.  _IEEE Transactions on Circuits and Systems I: Regular Papers_ 54, 48–59 (2007).
  * [73] Watts, L., Kerns, D. A., Lyon, R. F. & Mead, C. A.  Improved implementation of the silicon cochlea.  _IEEE Journal of Solid-state circuits_ 27, 692–700 (1992).
  * [74] Delbruck, T., Villanueva, V. & Longinotti, L.  Integration of dynamic vision sensor with inertial measurement unit for electronically stabilized event-based vision.  In _Circuits and Systems (ISCAS), 2014 IEEE International Symposium on_ , 2636–2639 (IEEE, 2014).
  * [75] Delbrück, T., Linares-Barranco, B., Culurciello, E. & Posch, C.  Activity-driven, event-based vision sensors.  In _Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on_ , 2426–2429 (IEEE, 2010).
  * [76] Vanarse, A., Osseiran, A. & Rassau, A.  A review of current neuromorphic approaches for vision, auditory, and olfactory sensors.  _Frontiers in neuroscience_ 10, 115 (2016).
  * [77] Posch, C., Benosman, R. & Etienne-Cummings, R.  Giving machines humanlike eyes.  _IEEE Spectrum_ 52, 44–49 (2015).
  * [78] Ahn, E. Y., Lee, J. H., Mullen, T. & Yen, J.  Dynamic vision sensor camera based bare hand gesture recognition.  In _Computational Intelligence for Multimedia, Signal and Vision Processing (CIMSIVP), 2011 IEEE Symposium on_ , 52–59 (IEEE, 2011).
  * [79] Delbruck, T. & Lang, M.  Robotic goalie with 3 ms reaction time at 4% cpu load using event-based dynamic vision sensor.  _Frontiers in neuroscience_ 7, 223 (2013).
  * [80] Conradt, J., Berner, R., Cook, M. & Delbruck, T.  An embedded aer dynamic vision sensor for low-latency pole balancing.  In _Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on_ , 780–785 (IEEE, 2009).
  * [81] Haessig, G., Cassidy, A., Alvarez, R., Benosman, R. & Orchard, G.  Spiking optical flow for event-based sensors using ibm’s truenorth neurosynaptic system.  _IEEE transactions on biomedical circuits and systems_ 1–11 (2018).
  * [82] Drazen, D., Lichtsteiner, P., Häfliger, P., Delbrück, T. & Jensen, A.  Toward real-time particle tracking using an event-based dynamic vision sensor.  _Experiments in Fluids_ 51, 1465 (2011).
  * [83] Forbes, N.  _Imitation of life: how biology is inspiring computing_ (Mit Press Cambridge, 2004).
  * [84] Forrest, S. & Beauchemin, C.  Computer immunology.  _Immunological reviews_ 216, 176–197 (2007).
  * [85] Somayaji, A., Hofmeyr, S. & Forrest, S.  Principles of a computer immune system.  In _Proceedings of the 1997 workshop on New security paradigms_ , 75–82 (ACM, 1998).
  * [86] Forrest, S.  Genetic algorithms: principles of natural selection applied to computation.  _Science_ 261, 872–878 (1993).
  * [87] Rice, J. & Martin, N.  Using biological models to improve innovation systems: the case of computer anti-viral software.  _European Journal of Innovation Management_ 10, 201–214 (2007).
  * [88] Keller, K. L.  Leveraging biologically inspired models for cyber–physical systems analysis.  _IEEE Systems Journal_ 1–11 (2017).
  * [89] Edelman, G. M.  _Neural Darwinism: The Theory of Neuronal Group Selection_ (Basic Books, 1987).
  * [90] Edelman, G. M.  Neural darwinism: selection and reentrant signaling in higher brain function.  _Neuron_ 10, 115–25 (1993).
  * [91] Pfeifer, R., Iida, F. & Lungarella, M.  Cognition from the bottom up: on biological inspiration, body morphology, and soft materials.  _Trends in Cognitive Sciences_ 18, 404–413, DOI: 10.1016/j.tics.2014.04.004.
  * [92] Bhounsule, P. A. _et al._ Low-bandwidth reflex-based control for lower power walking: 65 km on a single battery charge.  _The International Journal of Robotics Research_ 33, 1305–1321, DOI: 10.1177/0278364914527485 (2014).
  * [93] Collins, S., Ruina, A., Tedrake, R. & Wisse, M.  Efficient bipedal robots based on passive-dynamic walkers.  _Science_ 307, 1082–5, DOI: 10.1126/science.1107799 (2005).
  * [94] Weimerskirch, H., Bishop, C., Jeanniard-du Dot, T., Prudor, A. & Sachs, G.  Frigate birds track atmospheric conditions over months-long transoceanic flights.  _Science_ 353, 74 (2016).
  * [95] Zsuzsa, A., Mate, N., Severin, L. & Tamas, V.  Thermal soaring flight of birds and unmanned aerial vehicles.  _Bioinspiration and Biomimetics_ 5, 045003 (2010).
  * [96] Bousquet, G. D., Triantafyllou, M. S. & Slotine, J. E.  Optimal dynamic soaring consists of successive shallow arcs.  _J R Soc Interface_ 14, DOI: 10.1098/rsif.2017.0496 (2017).
  * [97] Allen, M. & Lin, V.  _Guidance and Control of an Autonomous Soaring Vehicle with Flight Test Results_.  Aerospace Sciences Meetings (American Institute of Aeronautics and Astronautics, 2007).
  * [98] Edwards, D.  _Implementation Details and Flight Test Results of an Autonomous Soaring Controller_.  Guidance, Navigation, and Control and Co-located Conferences (American Institute of Aeronautics and Astronautics, 2008).
  * [99] Reddy, G., Wong-Ng, J., Celani, A., Sejnowski, T. J. & Vergassola, M.  Glider soaring via reinforcement learning in the field.  _Nature_ DOI: 10.1038/s41586-018-0533-0 (2018).
  * [100] Sane, S. P.  The aerodynamics of insect flight.  _The Journal of Experimental Biology_ 206, 4191 (2003).
  * [101] Triantafyllou, M. S. _et al._ Vorticity control in fish-like propulsion and maneuvering.  _Integr Comp Biol_ 42, 1026–31, DOI: 10.1093/icb/42.5.1026 (2002).
  * [102] Rubenstein, M., Cornejo, A. & Nagpal, R.  Robotics. programmable self-assembly in a thousand-robot swarm.  _Science_ 345, 795–9, DOI: 10.1126/science.1254295 (2014).
  * [103] Werfel, J., Petersen, K. & Nagpal, R.  Designing collective behavior in a termite-inspired robot construction team.  _Science_ 343, 754–8, DOI: 10.1126/science.1245842 (2014).

## Acknowledgements

This work was supported by USAF grant FA9550-18-1-0301 to JO.

Sandia National Laboratories is a multi-mission laboratory managed and
operated by National Technology and Engineering Solutions of Sandia, LLC, a
wholly owned subsidiary of Honeywell International, Inc., for the U.S.
Department of Energy’s National Nuclear Security Administration under contract
DE-NA0003525. This paper describes technical results and analysis. Any
subjective views or opinions that might be expressed in the paper do not
necessarily represent the views of the U.S. Department of Energy or the United
States Government.

## Author contributions statement

JO was PI on the USAF grant which supported this research. JK, WS, SK and JO
wrote the manuscript.