# Lost in Algorithms

Andrew N. Sloss  SeattleWashington StateUSA<EMAIL_ADDRESS>
###### Abstract.

Algorithms are becoming more capable, and with that comes hic sunt dracones
(here be dragons). The term symbolizes areas beyond our known maps. We use
this term since we are stepping into an exciting, potentially dangerous, and
unknown area with algorithms. Our curiosity to understand the natural world
drives our search for new methods. For this reason, it is crucial to explore
this subject.

In this document, we look for future algorithms and styles. Each era in
computing has had an algorithm focus. Examples include periods when military
range prediction was necessary, weather prediction, and, more recently,
machine learning. The 1940s saw the starting point when electronic machines
replaced humans (Bhattacharya, 2022). Procedures became too complex to be
handled by people. This time and other historical periods have accelerated
additional specific algorithm development and the associated hardware
architectures. The question for this paper is, what next? As we explore this
question, we will introduce a set of practical terms to help classify the
various algorithms and hopefully provide a straightforward method of
understanding. At the highest level, algorithms are recipes for solving
problems. Problems range from the small & simple to the large & complex.

This project covers the behavior of algorithms and the hardware styles to
execute those algorithms. In other words, we will cover what the algorithms do
rather than how they do it. The world of algorithms is a large and complicated
subject. To assist in the process, we will separate the world into three main
areas: computer science, artificial intelligence, and finally quantum
computing.

The project’s objective is to overlay the information obtained, in conjunction
with the state of hardware today, to see if we can determine the likely
directions for future algorithms’. Even though we slightly cover non-classical
computing in this paper, our primary focus is on classical computing (i.e.,
digital computers). It is worth noting that non-classical quantum computing
requires classical computers to operate; they are not mutually exclusive.

algorithms, theory, mathematics, software, hardware

††ccs: Mathematics of computing††ccs: Theory of computation Models of
computation††copyright: none

##  1\. Introduction

Figure 1. Algorithm-Environment

Algorithms are critical for industry, research, and ideas. And have an
increasing influence on society. We can say every part of our lives involves
some form of an algorithm. It is a ubiquitous tool for problem-solving. To run
an algorithm, we have many machines: mechanical, biological, analog, digital,
quantum, and even human-social. We use them to optimize (e.g., repetition),
explore (e.g., search), and even predict (e.g., models). The subject attracts
some of the brightest and most innovative people wanting to discover better
solutions. These people are always at the cutting edge, striving to do the
next complicated task.

Figure 1 shows how we will approach the subject. An algorithm or recipe has
inputs and outputs and lives inside an environment.

The inputs-outputs come as part of the executing environment, e.g., a digital
computer. We will look at algorithms from a behavioral viewpoint. In other
words, what do the algorithms do?, and less on how they do it?. It is
essential to understand the distinctions; we are not explaining the algorithms
themselves but how they affect or take from the environment. Because of the
vast nature of the field, the focus is mainly on digital computing to help
prune the domain.

We have hopefully explained our approach, and the next stage is to show the
relationship between hardware and software. The Venn diagram shown in Figure 2
attempts to establish the relationship between hardware, software, and
algorithms. We are separating the worlds, so it is easier to see the
connections. Each world has its styles of thought and process. For instance,
algorithms + software gives us applications, and hardware + software +
algorithms give us an end product. A complete solution is about all three
worlds aligning together.

What came first, hardware or algorithms? This is a chicken or egg question.
Algorithms are probably first, with the role of hardware being initially human
and algorithms being mathematical equations. Historically, hardware has had to
catch up with algorithms. This catch-up is why sizable effort is applied to
optimize a new algorithm for existing hardware. With one exception, hardware
jumps in capability every so often, forcing algorithms to catch up, e.g.,
quantum hardware. Hardware provides the enabling canvass for the exploration
and optimization of algorithms.

Figure 2. Relationship between Hardware-Software-Algorithms

An algorithm describes a method to utilize software and hardware for problem-
solving. There is usually an objective, but not always. A problem belongs to a
problem domain. A problem domain is a search space. For example, if a company
has a problem with obtaining electronic components, the problem domain would
probably include suppliers, the ordering process, and the manufacturing
department. The algorithm should search those areas to determine a solution.
The environment consists of the execution machine, data input, and a place for
the final output decision. The execution machine is any computation system.
Data input comes from the environment as filtered or noisy real-world data.
Lastly, output decisions can be anything from classification to an action that
makes an environmental change

As computer scientists, we have historically relied on famous texts. For
instance, Robert Sedgewick et al. book on Algorithms (Sedgewick and Wayne,
2011), or Donald Knuth book series on The Art of Computer Programming (Knuth,
1997) to provide libraries of known solutions. The knowledge includes such
algorithms as recursive tree structure walking or the shortest path between
two nodes on a graph. These libraries were the result of decades of
experimentation. If we fast forward to today, we see statistical and
probabilistic algorithms becoming ever more popular. These algorithms are less
concerned with precision (i.e., absolutes) and more concerned with good enough
(i.e., levels-of-certainty). This trend does not mean the older algorithms are
any less important, but currently, they are not at the cutting edge.

Mathematics allows us to express complex processes or prove correctness. It is
probably our most significant accomplishment. The language of mathematics
underpins the world of algorithms, and it is how we formally describe recipes.
We will start this journey by looking first at the general objectives, i.e.,
what should algorithms explore?.

###  1.1. General objectives

Figure 3. Ideal algorithm

Algorithms optimize, summarize and discover the world around us. These
pursuits are related to us humans, either to augment or to move beyond our
capabilities. We separate objectives into three distinct areas:

  * •

What we know

Using historical knowledge and skills

  * •

What we think we want to know

Using investigative methods of exploration

  * •

What we can’t imagine

Going beyond human perception and capability

In the beginning, we described an algorithm as a recipe. The recipe is a
proven sequence of operations that find an answer or execute computation.
Historically, algorithms were optimizations of what we knew —replicating human
procedures. The early hardware had significant constraints, limiting what
algorithms existed. These early solutions, even though primitive by today’s
standards, could operate 24 hours a day, seven days a week (provided the
thermionic valves did not burn out). The hardware was a digital replacement.
As time has progressed, improvements in computation have allowed us to shift
from what we know to what we think we want to know. In other words, the
hardware allows us to explore, i.e., find new knowledge. Exploring involves
some form of guessing and, by implication, takes time. Guessing allows for
mistakes. Lastly, this brings us to the third objective what we can’t imagine,
these are the algorithms at the edge of a discovery that cannot necessarily be
100% explained or show a reasoned causal path. Their forms and structures are
still in flux.

Certainty requires some form of confirmation. We used the word proven, in an
earlier paragraph, as an aside. A mathematical proof is used for validation
and differs from an algorithm. In that, what is computable and what can be
proven may be different. We have an arsenal of automated mechanisms to help
verify algorithms. These include formal methods and the modern trend to
explore complexity hierarchies with various forms of computation. It is worth
mentioning that Kurt Gödel, in 1933, presented the infamous Incompleteness
theorem, showing that not all algorithms can be proven (Kennedy, 2022).
Another critical example is David Hilbert’s’s Halting problem (Turing, 1936),
where Alan Turing proved that it is impossible to determine when an algorithm
will stop (or not) given an arbitrary program and inputs.

As the objectives become more abstract, uncertainty increases. We can divide
uncertainty into two ideas. There is uncertainty due to the complexity of
nature, and there is uncertainty due to our lack of knowledge. Both play a
critical role as we explore our environment. The first idea is called
ontological uncertainty (e.g., associated with biology and quantum mechanics),
and the second idea is called epistemological uncertainty (e.g., we do not
know the precise number of people who are left-handed?) (Palmer, 2022).

What we know, what we think we want to know, and what we can’t imagine are the
three high-level objectives; we next look at the aspirations. What should we
consider as ideal attributes for a good algorithm?

###  1.2. Ideals

There are many ways to think about algorithms. We can look at demands that an
algorithm has to satisfy (e.g., best voice compression algorithm or highest
security level for buying online). The approach we have decided to adopt is to
look at the attributes we want algorithms to have, the set of ideals. Platonic
idealism is the contemplation of ideal forms. Or, more realistically, a subset
of ideal forms. An ideal involves attempting to find an algorithm without
sacrificing other essential attributes. These attributes include being
efficient with time (the temporal dimension) and using appropriate resources
(the spatial dimension). Resources include physical storage, communication,
and computation.

We should make it clear this is not about what is possible with today’s
technology or even in the future but what we want algorithms to achieve, i.e.,
our expectations. The following list is our first attempt:

  1. I-1.

Perfect solution, is an obvious first ideal. A good outcome is to have several
solutions, each providing a different path and varying levels of precision &
accuracy. Human biases, such as symmetry, are removed from the outcome unless
there is a requirement for a human-biased result, i.e., a decision is made
between impartial (ethical) or partial (practical) solutions (Mozi, BCE).
Finally, the algorithm maps directly onto available hardware per the spatial
dimension.

  2. I-2.

Confidence through consistency, we want consistency; an algorithm creates
confidence by providing reliable results.

  3. I-3.

Self-selection of the objective, one of the essential activities humans
undertake is determining the purpose. For an ideal algorithm, we want the goal
or sub-goals set by the algorithm or offered as a set of options, i.e.,
negotiation.

  4. I-4.

Automatic problem decomposition, we want the algorithm to break down a problem
into testable modules. The breakdown occurs automatically. This process is
essential if we want to handle more significant issues, i.e., more extensive
problems.

  5. I-5.

Replication when required, specific problems lend themselves towards parallel
processing. For these classes of problems, we want the algorithm to self-
replicate. The replication allows solutions to scale automatically; some
problems require scale. As much as possible, the algorithms should also work
out how to scale linearly for a solution to be ideal.

  6. I-6.

Handling the known and unknown, we want an algorithm to handle problems that
are either known (with related solutions) or entirely unknown (where
exploration occurs). An unknown answer, once found, transfers to the known. It
learns.

  7. I-7.

No preconditions on input data, from an ideal perspective, we want to remove
the format strictness imposed on the input data. Data acts as an interface for
algorithm negotiations. Analyzing the data means the data format is deducible,
removing the requirement of a fixed interface. Note Machine Learning has
different criteria that are more to do with the quality of the input data.

  8. I-8.

Self-aware, ideally, we want an algorithm to be aware of the implications of a
decision. This implication is especially true regarding safety-critical
problems where a decision could have dire consequences and, more generally,
the emotional or moral side of a decision. It weighs the effect of the
outcome.

  9. I-9.

Secure and private, we want an algorithm to handle data so that it is secure,
and if human information is concerned, it provides privacy.

  10. I-10.

Being adaptive, an ideal algorithm can change as the environment changes and
continuously learns from new knowledge. Knowledge comes from experimenting
with the environment and subsequently improves the algorithm.

  11. I-11.

Causal chain, where applicable, we want the causal chain that produced the
result. We want to understand why.

  12. I-12.

Total knowledge, an ideal algorithm has all the necessary historical knowledge
for a particular area. The algorithm does not follow information blindly but
has all the knowledge about a specific subject. New knowledge can be
identified as an emergent property if an unknown pattern appears. The ideal
algorithm becomes an encyclopedia on a particular subject. Maybe different
weights are placed on the knowledge that is correct or contradictory. Maturity
means the topic under study is wholly understood and has well-defined
boundaries.

  13. I-13.

Explainable, we want results explained in human understandable terms. As the
problem becomes more complicated, so do the answers. We want the algorithms to
explain the answer and, potentially, the context.

  14. I-14.

Continuous learning and improvement, as alluded to in previous ideals, we want
the algorithm to continue to learn and to continue attempts to improve the
techniques to produce a better, faster, less resource-draining solution.

  15. I-15.

Cooperative or competitive, the ideal algorithm works in a multi-agent
environment, where agents are assistants for or detractors against a zero-sum
scenario. (Poundstone, 1992)—the ideal looks for an alliance with other
agents. If an alliance is not possible, it goes out on its own to solve the
problem (Nash equilibrium (Osborne and Rubinstein, 1994)). In other words, we
want an algorithm to have Games Theory skills.

  16. I-16.

Law-abiding, we need an algorithm to be a law-abiding citizen. It works within
the confines of legal law (not scientific laws). This confinement is essential
for algorithms involved in safety-critical or financial activities.

  17. I-17.

Nice and forgiving, is more of a human constraint. We want algorithms to take
the most friendly society approach in a multi-agent environment; if harm
occurs to the algorithm, then a counter-reaction could be implemented. Within
reason, an algorithm can forget any malicious act. This reaction is essential
when only partial information is available (Poundstone, 1992). We can argue
whether this is a constraint or an ideal, but we want algorithms to have some
human-like tendencies, e.g., compassion over revenge).

As pointed out, these are first-pass ideals. We are certain ideals are missing
or need modification. Even though algorithms are likely to be unique, ideal
characteristics define the algorithms’ boundaries (or extremes). The limits
provide the universal goals for an algorithm.

We now move the journey to the problem domains. At this point, we have
discussed the importance of algorithms and what the high-level ideals should
be. Hopefully, these ideals indicate why we are potentially entering unknown
territory.

###  1.3. Computation

Problems make an algorithm attractive, from the challenge of chasing a
solution to the actual application. A solution is a map of a complex world,
making it understandable. The map comes from a boundless library of ideas,
e.g., The Library of Babel concept (Borges et al., 2000). Each room in the
infinite library includes varying truths and falsities.

As mentioned, problem domains determine the search space of possibilities,
ranging from simple to complex and small to large. A simple mathematical
problem domain tends to have a simple solution; likewise, a chaotic problem
domain leans towards a complicated answer. There is an underlying belief and a
hope that a problem domain is reducible (Mitchell, 2009). For example, Sir
Isaac Newton created the Laws of Motion (Newton, 1687) that reduces the
complexity of movement to a set of rules. By contrast, we have failed to
reduce gravitation, electromagnetism, weak nuclear, and strong nuclear forces
to an agreed-upon single solution, i.e., a Grand Unified Theory (GUT). We have
controversial ideas, such as String Theory, but no provable solutions (Kaku,
2021). There is a possibility and hope a future algorithm will eventually
solve this problem.

Algorithms rely on reducibility. The likelihood of finding a reducible form is
dependent on the complexity level. How we reduce a problem also depends on the
spatial and temporal constraints. A good example of an external constraint is
the timing required for a successful commercial product. Missing the timing
window means a potential loss of revenue.

Figure 4. Searching, ordering, and compression

A solution to a problem domain involves a combination, and sequence, of
searching, ordering, and compression, see Figure 4. The searching function
involves looking for a solution within the problem domain. Searching is about
discovering knowledge. The sorting function organizes the problem domain in a
logical order, i.e., information. Finally, the compression function converts
the elements to a new form, i.e., meta-data. These three functions find
solutions to problems.

There are three embarrassing worlds. We have gone over the goal of reducing
complexity and the functions to map complexity to some form of reasoning; now,
we look at the types of problem domains. We start with the first,
embarrassingly parallel problems; these are problems that map perfectly onto
parallel solutions. These problem types do exist but are relatively rare.
Performance is proportional to the available parallel machines, i.e., the more
parallel machines available, the faster the processing. These improvements
remain accurate, while the serial sections are minimal (i.e., taking into
account Amdahl’s law, i.e., parallel performance becomes limited by the serial
parts (Amdahl, 1967)).

The second embarrassing style is sequential data, i.e., embarrassingly
sequential. Embarrassingly sequential data follows a strict structure that can
be mechanically optimized. We can design efficient computation to work best on
embarrassingly sequential data. These problem spaces map efficiently onto
software-hardware systems. They are less chaotic and random.

As we move to real-world situations, the domain types require complicated
synchronizations (a mixture of parallel and serial components) and deal with
unstructured data. The problems end up being embarrassingly unhelpful.
Bringing the focus back to hardware efficiency, the prior (i.e.,
embarrassingly parallel and sequential) leans towards specialization. The
latter (i.e., embarrassingly unhelpful) leans towards general-purpose
machines. General-purpose machines move towards the Principle of Universality.
As-in is capable of handling all problems. General-purpose machines are better
for embarrassingly unhelpful problems since they reduce complexity using less
specialized operations. The embarrassing aspects of data drive computation
design.

Figure 5. Process function

What is computation? It is the act of running an algorithmic recipe on a
machine. A computation process takes input data and outputs some form of
result; see Figure 5. A process can be serially sequenced or run in parallel.
Optional auxiliary feedback is taken from the output and placed as input,
giving an algorithm the ability to adapt. This characteristic allows for the
creation of complex hardware architectures. Conditional control mechanisms
(i.e., if-then-else) determine the order and flow of the computation either
between processes or within the process.

Figure 6. Computation machines

Figure 6 shows a subset of machines, with combinational logic as a foundation
for the other levels. In this diagram, we place Probabilistic Turing Machine
at the top of the computation stack. This decision will hopefully become self-
apparent as we journey further into algorithms.

Energy is a requirement for computation and, as such, has direct influence on
the choice of algorithms. Carrying out any calculation requires a form of
energy imbalance (following the Laws of Thermodynamics). To achieve energy
imbalance in classical computing, we either supply energy directly or harvest
the energy from the environment. Once energy is supplied, execution results in
the production of heat. Maybe someday we can recycle that heat for further
computation. With future machines, reversibility may become an essential
requirement to reduce energy and to allow more capable algorithms, i.e.,
reversible computing (Bennett, 2002).

###  1.4. Input-output relationship

What are the general relationships between the input and output data? Below
are some connections from simple to complex. Finding any relationship from the
data, even a good-enough one, can be highly complicated.

  * •

Linear relationship, shows a straightforward relationship between input data
and result, example equation $y=2x$.

  * •

Exponential relationship, shows a growth factor between input and output,
example equation, $y=e^{x}$.

  * •

Nonlinear relationship, difficult to determine the relationship, a more
complex pattern is emerging, example equation $x^{2}+y^{2}=42$

  * •

Chaotic relationship, at first sight appears to have no relationship, due to
complexity, example equation $x_{t+1}=kx_{t}(1-x_{t})$

  * •

Random/stochastic relationship, the relationship is truly random, example
equation $y=rand(x)+42$

###  1.5. Exploitative vs exploratory

Algorithms handle the known or unknown, namely exploitative or exploratory.
The first relies on knowledge and provides hopefully a known outcome, for
example, following an applied mathematics equation to determine whether a beam
is in tension or compression. The second type, exploratory algorithms,
explores a problem domain when the exact answer, and maybe even the
environment, is unknown or changing —, for example, an algorithm learning to
fly on a different planet. The alien world will have unknown gravitational or
magnetic challenges. The former concept leans towards precision and accuracy,
whereas the latter is comfortable with good-enough results, i.e., compromise
or palliative.

Exploitative | Exploratory
---|---
Specialized | General purpose
Narrow | Broad
Focused | Unfocused
Known, well understood | Unknown, and less understood
Best result | Good enough
Turing complete/incomplete | Turing complete
Table 1. Exploitation vs Exploratory

What comes first, exploitative or exploratory? This is a cart before the horse
question. Whether human or machine, exploratory takes place before
exploitation. It is part of the learning process because we first have to
understand before we can look for a solution. As we move into the future, we
will rely more on algorithms to explore and find new solutions. And hence,
hardware will need to increase support for more experimental methods. An
example of increasing exploratory support is the trend towards efficient
hardware for training learning systems. Efficiencies in both speeds of
training and power consumption.

Newer algorithms can take advantage of both techniques, i.e., explore first
and then optimize or exploit until further exploration is required. This shift
is a form of simulated annealing. Annealing is the method of toughening metals
using different cooling rates; simulation annealing is the algorithm
equivalent. Forward simulated annealing starts by first exploring global
points (e.g. random jumps) and then shifts to local points (e.g. simple
movements) as the perceived solution becomes more visible.

Hardware-software: the software can play directly with exploitative and
exploratory algorithms. By contrast, hardware is dedicated to the exploitative
side, i.e., getting the most out of a known algorithm. Hardware is static and
fixed. And the software provides the ability to adapt and re-configure
dynamically. In the digital context, the software is the nearest we have to
adaptive biological systems. Exploitation allows for hardware-software
optimizations.

###  1.6. Where do algorithms come from?

Are algorithms entirely invented, or are they driven by the problems? This is
a time-old question with deep philosophical arguments from many sides. If we
believe algorithms are invented, anticipating the future could be difficult.
For this reason, we have chosen the view that problems define algorithms. To
make this even easier, we will state that problems fall under the following
four categories: i. Physics, ii. Evolution, iii. Biology, and iv. Nature.
Where algorithmic ideas develop from one or a combination of categories.

  1. i.

Physics gives us the exploration of thermodynamics, quantum mechanics, and the
fabric of the universe. Problems from a planetary scale to the sub-atomic

  2. ii.

Evolution, gives us exciting ways to create future options. Allows algorithms
to explore their problem domains, i.e., survival of the fittest, natural
selection, crossover, and mutation

  3. iii.

Biology, introduces complex parallel networks, e.g., cell interactions and
neuron communications

  4. iv.

Nature, provides us with big system problems, e.g., climate change

As previously pointed out, mathematics is the language to describe or express
algorithms. It is not necessarily a vital source of inspiration. We believe
that the discovery and understanding of the natural world create algorithms.
By observing the natural world, we can play with predicting future directions
and possibilities.

###  1.7. Measuring computational complexity

Many subjects are concerned with complexity. For example, safety-critical
systems are susceptible to increases in complexity. Computer science has an
area of research labeled Computation Complexity Theory dedicated to the
subject. The theory translates complexity into time to solve. It is worth
mentioning that time taken and energy is closely connected. Our tentative goal
is always to remain within an energy boundary. Problems break down into
different time relationships as shown in the now infamous Euler diagram (see
Figure 7). Why should we care? Because there exist problems that are
impossible to solve efficiently or are just unsolvable. Providing more
engineering time or effort will not culminate in a faster solution in these
cases.

  * •

Polynomial (P) time, represents computational problems that are solvable in
deterministic polynomial time. These problems are relatively straightforward
to solve on a Turing machine (see Section 1.3). Low in complexity. The input
length determines the time required for an algorithm to produce a solution.

  * •

Nondeterministic Polynomial  (NP) time, are solvable problems but in
nondeterministic polynomial-time. The algorithm can be proven correct using a
deterministic Turing machine. Still, the search for the solution uses a
nondeterministic Turing machine. The search involves some form of best guess.

  * •

Nondeterministic Polynomial-complete (NP-complete) time, similar to NP
problem, the verification can occur in quick polynomial time but solution
requires a brute-force algorithm. Brute force means there is no efficient path
to a solution. These problems are the most complicated to solve in the NP set.
Also, each problem is reducible in polynomial time. The reducibility allows
for simulation.

  * •

Nondeterministic Polynomial-Hard (NP-hard) time, covers the truly difficult
problems, the hardest NP problems and continues outside the NP scope. It also
includes potential problems that may not have an answer.

Figure 7. A Euler diagram on complexity - Computer Science

Suppose we look at complexity through an exploitative and exploratory lens. We
see that P covers the exploitative algorithms, and generally, NP covers the
exploratory side, i.e., experimentation.

###  1.8. Measuring probabilistic complexity

Continuing our journey, it becomes apparent that probability is becoming
increasingly important. For this reason, we should try to understand
probabilistic complexity, where good enough, averages, and certainty play a
more critical role. Probability is fundamental for artificial intelligence,
probabilistic computers (Genkina, 2022), and quantum computing, as described
later.

These are problems only solvable by a probabilistic Turing machine (Nakata and
Murao, 2014). This Turing machine operates with a probability distribution,
which means that a distribution governs the transitions. This setup gives a
probabilistic Turing machine-specific unique characteristics.

The characteristics are nondeterministic behavior, transition availability by
a probability distribution, and stochastic (random) results. The stochastic
effects require repeatability before a level of certainty can be established.
The behavior means that the same input and algorithm may produce different run
times. There is also a potential that the machine will fail to halt. Or the
inputs are accepted or rejected on the same machine with varying execution
runs. This variability is why an average of execution runs is required.

Figure 8. Probabilistic complexity (Wikipedia)

Figure 8 shows an extension to the traditional Euler diagram with
probabilistic complexity (Nakata and Murao, 2014). Rather than covering all
the different options, permit us to focus on three especially relevant
examples:

  * •

Bounded-error Probabilistic Polynomial (BPP) time, an algorithm that is a
member of BPP runs on a classical computer to make arbitrary decisions that
run in polynomial time. The probability of an answer being wrong is at most
$\frac{1}{3}$, whether the answer is heads or tails (i.e., a coin flip).

  * •

Pounded-error quantum Polynomial (BQP) time,

an algorithm that is a member of BQP can decide if there exists a quantum
algorithm running on a quantum computer with high probability and guaranteed
to run in polynomial time. As with BPP, the algorithm’s run will correctly
solve the decision problem with a probability of at least $\frac{2}{3}$.

  * •

Probabilistic Turing machine in Polynomial time (PP) time, is simply a
algorithm were the probability of error is less than $\frac{1}{2}$ for all
instances.

Finally, probability complexity is linked directly to the probability of being
wrong. Errors are part of the process and, as such, need to be handled or
mitigated.

Note that Figure 8 captures the relative topology, but not the algorithmic
class size. How important each member will be in comparison is still to be
determined.

Now that we have described some parts of measuring complexity, we can move on
to the objective.

###  1.9. Algorithm objective

An algorithm has to have some form of direction. The direction can take one of
three forms single objective, multi-objective, and objective-less. A single
objective means only one search goal, e.g., performance or power. Single
search goals tend to be simpler problems to solve but not always. A multi-
objective is more complicated with multiple search goals to satisfy, e.g.,
performance, power, and area. Multi-objective searches look for a compromise
between the solutions. These compromises live on what is called the Pareto
optimal, see Figure 9. Finally, objective-less relies on gaining new
experiences in an environment rather than moving towards any particular goal
(Stanley and Lehman, 2015b). The idea is that by gaining experience, a better
understanding of the problem domain occurs, thus allowing for significantly
better solutions.

Figure 9. Multi-objective Pareto Curve

For most of this century, the focus has been on a single objective, but
problems have changed in the last fifty years, and multi-objective problems
are more typical. Figure 9 shows two objectives for an electric car. These
objectives are the best acceleration on the x-axis and the lowest power
consumption on the y-axis. p1 represents the fastest option (e.g., fast tires,
higher voltage, and performance electric motors), whereas p2 represents the
lowest power consumption (e.g., aerodynamic tires, voltage limited, and
balanced electric motors). The edge between the two extremes is the Pareto
front, a point on the front (e.g., p3) is Pareto optimal. The best solution
must be a compromise between acceleration times and power consumption. There
are no perfect solutions, just compromises of opposing objectives.

As a counter-intuitive idea, objective-less is an exciting alternative. The
algorithm is placed in an environment and then left to discover. Progress
occurs when a new experience is discovered and recorded. An example of this
exploration style is Novelty Search (see Section 2.2.8). Potentially these
algorithms can find new knowledge. The objective-based algorithms look for
something known, and the solution is biased by implication. Whereas objective-
less algorithms learn by experience, removing implicit bias.

As well as the objectives, there are constraints (or limitations). These
constraints impose restrictions on any possible solution. For example, the
conditions may include limited resource availability, specific time windows,
or simply restrictions on power consumption. A general mathematical goal is to
provide constraint satisfaction, where each object in the potential solution
must satisfy the constraints.

###  1.10. Environment and data

There are many properties concerned with problem-domain landscapes. The first
is the environment the algorithms operate within; see Figure 10. For example,
one environment could have the properties of solitary, relaxed, and completely
observable. A domain can make the problem space more or less difficult to
cover. The landscape diagram shows some of the variations. The variations act
as a filter to determine which class of algorithms is more likely to be
successful and rule out other ones that are unlikely. It seems common sense
that an algorithm should be chosen by first assessing the environment.

Figure 10. Landscape (Russell, 2019)

Data comes in many different forms. From simple unimodal data with an obvious
solution to the more complex data that includes noise or deception (Luke,
2013), see Figure 11. The format of the input data determines the algorithm.

Figure 11. Problem style (Luke, 2013)

By increasing the dimensions of the input data, we can extract hidden
information (Daniel J. Gauthier and Barbosa, 2021). More advanced algorithms
use this technique to handle more complicated problems. The hope is to remove
some of the noise in the data. The opposite approach can also be valid;
reducing dimensions simplifies the input data.

###  1.11. Determinism, repeatability, and randomness

Algorithms have different characteristics, these include determinism,
repeatability, and randomness. Taking each characteristic in turn. Determinism
is when an algorithm given a set of inputs provides a result in the same
amount of time. Consistency is essential for applications that are time
constrained. These applications come under the term Real-Time. Where the
algorithm flow consistently takes the same amount of time to complete. Real-
Time is an arbitrary measurement since the definition of time can cover a wide
range.

Repeatability is the concept that given the same input, the same output
occurs. Many problems require this type of characteristic. It is fundamental
to most mathematical equations. It is more aligned with perfection and means
that the problem space is wholly understood.

Randomness is the opposite of repeatability, as the results can differ or even
not occur. Randomness allows some degree of uncertainty to provide variation
in the answers, i.e., flexibility in discovery. It goes against mathematical
perfection because it allows for greater exploration of complex spaces.
Algorithmic complexity can appear random because the patterns are so difficult
to comprehend.

Random numbers in computers are called pseudo-random numbers. This label is
because they follow some form of artificial distribution. A pseudo random
number generator (PRNG) creates these numbers. As an important example,
pseudo-random numbers can simulate noise. The simulated noise can help
transition a high-dimensional problem into a more accessible lower-dimensional
problem (Palmer, 2022). Achieving this transition occurs by replacing some of
the state variables with guesses. This transition makes an otherwise
impossible situation searchable (e.g., weather prediction).

###  1.12. Prediction, causality, and counterfactual

Prediction, causality, and counterfactual are at the cutting edge of what
algorithms are capable of achieving. Prediction is probably one of the most
exciting areas for modern algorithms—the ability to predict a future with some
degree of certainty. Science as a discipline has had its challenges with
prediction. Prediction is a difficult subject; it includes everything from
software to determine the next actions for a self-driving car to consistent
economic forecasting on a specific stock. Probably the best-known of all the
algorithmic predictions is weather prediction. Weather prediction is highly
accurate for the next three hours but becomes less certain as we increase the
time.

Casualty is the ability to show cause-and-effect (Pearl and Mackenzie, 2018).
The reason a pencil moved was that a person pushed the pencil. In many ways,
humans want more than just an answer from an algorithm; they want to
understand why. It is problematic for algorithms because it requires more
understanding of how actions are connected and chained together.

Finally, there are counterfactuals—a combination of prediction and causality.
Counterfactual is an alternative history where a decision not to do something
affects a prognosis. This action can play with the future. Again an exciting
area to play with from an algorithm point of view (Pearl and Mackenzie, 2018).

###  1.13. Creativity and diversity

Creativity and diversity are terms primarily associated with humans rather
than algorithms. Creativity is some form of inspirational jump that allows
complex problems to be solved. There is an ongoing debate whether an algorithm
can be creative (Foley, 2022)? And if so, how do we measure creativity? If an
algorithm paints a scenery, is it being creative? These are difficult
questions, but when it comes to algorithms, this is the new frontier.

Figure 12. Creativity (Boden, 2003)

What is creativity? Margaret Boden, a Research Professor of Cognitive Science,
broke down creativity into three useful mechanisms (Boden, 2003), namely
exploration \- playing within the rules, combination \- applying one set of
rules to another domain, and transformation \- rewriting the rules by removing
a critical constraint. See Figure 12. For algorithms, exploration creativity
is risk-averse and limited, and at the other end of the scale, the
transformation has the highest risk with potential novelty.

On the same lines of creativity, we have diversity. Diversity brings about
novelty or new solutions by offering variation in the algorithms. A diverse
set of algorithms can search multiple directions in parallel.

###  1.14. Byzantine style algorithms

Distributed systems, safety-critical systems, and financial systems need to
have some resilience. Resilience is an attempt to avoid wrong decisions. Wrong
decisions occur due to system errors or an intentional lousy agent. This area
is called the Byzantine General Problem, description outlined below:

  1. B-1.

Lieutenant generals need to come to a decision.

  2. B-2.

Unfortunately, there are potential traitors.

  3. B-3.

How do the loyal generals decide on a correct decision?

Redundancy is essential in several areas. For example, hardware or software
has the potential to exhibit problems. It is necessary if the ramifications of
such a situation occur to be able to mitigate those problems. This avoidance
is crucial in safety-critical systems where failure results in harm or
significant financial loss. The solution is to have redundancy and not rely on
one or two agents for a decision.

###  1.15. No free lunch theorem

When we described the Ideals in Section 1.1, we were skirting around the
concept of a free lunch. This idealism is a reverse play on the No Free Lunch
(NFL) theorem. David Wolpert and William McReady formalized the theorem, and
it states that all optimization algorithms perform equally well when averaged
over all possible problems (Wolpert and Macready, 1997). The theorem means no
algorithms stand out as being better or worse than any other. Solving
different problems involves specific knowledge of that problem area.

A modern algorithm has to deal with a knowledge question. The question is
whether an algorithm starts from a clean slate (i.e., no knowledge —Tabula
rasa) or some known captured experience (i.e., known knowledge —Socratic
recollection). We need to decide how an algorithm starts; by learning with no
expertise or giving the algorithm, a jump starts with knowledge.

###  1.16. Network thinking

In algorithms, it is crucial to mention networks. Networks play an essential
role in modeling and analyzing complex systems. Networks are everywhere, from
the interconnection between neurons in the brain to aircraft flight patterns
between airports. Electrical engineering uses networks to design circuitry.
Probably one of the most famous technology networks is The internet which
allows us to communicate efficiently. We commonly describe networks in terms
of nodes and links. There are at least two helpful methods of describing
networks:

  * •

Small-world networks are networks where all nodes are closely connected, i.e.,
requiring only a small number of jumps. Most nodes are not neighbors but are
closely linked. For example, the aircraft flight patterns we already
mentioned. Another example is Karinthy’s 1929 concept of six degrees of
separation (Karinthy, 1929; Palmer, 2022), where just six links connect
everyone.

  * •

A scale-free network, follows a Power law distribution. The Power law states
that a change in one quantity will cause a proportional change in another. All
changes are relative—for example, a social network.

Preferential attachment is a process where a quantity distributes across
several nodes. Where each node already has value. The nodes with more value
gain even more, and the nodes with less value gain less. This value transfer
is necessary when algorithms model wealth distribution or contribution-rewards
in an organization.

Lastly, network thinking is all about modern graph theory. Graph theory covers
graph knowledge models to graph databases (e.g., temporal-spacial databases).
It is an important area for algorithms, and it is constantly expanding.

##  2\. Category

In this section, we will attempt to describe algorithms as categories or
classes; it is a complicated process. Again more from the behavioral
viewpoint. We will divide the world into three main categories computer
science, artificial intelligence, and quantum computing. We cover what we
think are the more interesting behavioral classes, but this is by no means
exhaustive.

###  2.1. Computer science, Cs

Computer science has been creating and formulating algorithms for the past 50
years. This length of time means there is an abundance of algorithms. In this
subsection, we collect the various essential concepts. As an academic subject,
computer science is still relatively young as a discipline, but it acts as a
universal provider. As-in it provides a service to all other fields. Note we
included the first two descriptions as fundamental concepts rather than
algorithms.

####  2.1.1. Cs, Propositional logic

Proposition logic is a language. It is used by algorithms to provide Boolean
answers, i.e., True or False. Combining logic (OR, AND, Exclusive OR, and NOT
operations) together we can create an algorithm. Digital hardware circuits
derive from propositional logic.

####  2.1.2. Cs, Predicate calculus

Predicate calculus is a language. Algorithms use it to produce correct
statements. This correctness means that all statements are provable and true
within the algorithm. Symbols represent logical statements. For example,
$\forall x\in N:x^{2}\geq x$ translates to for all $x$, where $x$ is a natural
number, it is true that $x^{2}$ is equal or greater than $x$. Thus satisfying
the predicate calculus rules that all statements are sound and true. Another
example, $\exists x\in N:x\geq 42$ translates to there exists an $x$, where
$x$ is a natural number, that $x$ is greater than $42$, again sound and true.

####  2.1.3. Cs, Recursive algorithms

A recursive algorithm calls itself. For example, $f(x)=x-f(x-1)$, where the
function $f$ is on both sides of the equation. Recursive algorithms have
interesting behavioral properties because they can converge or diverge. A
convergent recursive function concludes. A divergent recursive algorithm never
stops. In computer terms, memory resources can be pre-calculated in a
convergent algorithm. Divergence means the opposite; an equation will fail to
conclude, potentially resulting in an out of memory error. We can use proof-
by-induction to determine correctness.

Figure 13. Recursive fractal (Public Domain)

Recursion can be seen in many natural objects, for instance, leaves, trees,
and snowflakes. These occurrences mean there are many examples in nature where
recursion is employed. Figure 13 shows recursion in the form of a fractal.

####  2.1.4. Cs, Divide-and-conquer algorithms

The divide-and-conquer algorithms separate a problem into easier-to-manage
problems. They help handle situations that are either too big in their
entirety or too difficult.

Figure 14. Divide and conquer

These algorithms are best employed when the divided problem is less
complicated than the added communication and distribution overhead. If true, a
parallel architecture can lend itself to this type of problem. And this is
especially true if the sub-problems are all solved deterministically, i.e.,
taking the same time to process. Divide-and-conquer serves as a helpful method
for debugging complex systems. This technique is closely related to the
Scientific Method.

####  2.1.5. Cs, Dynamic programming algorithms

Dynamic programming (or DP), is related to the divide-and-conquer algorithms,
see Section 2.1.4. Unlike the divide-and-conquer, DP uses the result of the
sub-problems to find the optimum solution to the main problem. These
algorithms find the shortest path (e.g., map navigation), optimal search
solutions, and operating system schedulers.

####  2.1.6. Cs, Randomized algorithm

Randomized algorithms use artificially generated randomness to solve complex
problems, e.g., molecular interactions. One of the most famous algorithms in
this class is called the Monte Carlo algorithm. Monte Carlo takes an initial
configuration, let us call it the status quo, and, driven by a probability
distribution, a state is randomly changed (e.g., on a coin, heads go to tails)
(Miller, 2016). A calculation then determines the energy of the new
configuration. With that information, the energy acceptance criteria can
determine whether the new configuration becomes the status quo. These
algorithms model probabilistic real-world systems. And as such require
significant amounts of computational resources, not to mention the time is
taken to set them up. Their primary use is in particle Physics, Biochemistry,
and financial modeling.

####  2.1.7. Cs, Fractional factorial design

Fractional factorial designs are included primarily because of their
behavioral characteristics. They use a concept called sparsity-of-effects
principle (Surhone et al., 2011). This principle brings out important features
from the data. The claim is that only a fraction of the processing extracts
the most interesting data features. This characteristic is an important
behavioral style for many problem domains, i.e., using less work to identify
the most interesting features.

####  2.1.8. Cs, Greedy algorithms

A greedy algorithms discover a solution a little piece at a time. These
algorithms find near-optimal solutions for NP-Hard style problems. The
algorithm takes the direction of the most advantage at each point (i.e., local
optima strategy). This algorithm is relatively easy to implement for
optimization problems.

####  2.1.9. Cs, Brute force algorithm

A brute force algorithm, as the name implies, involves looking at every
possible solution to find the best. In other words, no shortcuts. We use these
algorithms when there are no alternatives. From a behavioral point of view,
running these algorithms requires significant resources. These algorithms best
suit problems with no known better solution and can get away with no time
constraints. We default to these algorithms when there is no other solution.
Replacement of these algorithms occurs when there are economic or
environmental pressures. For example, we can see this with cryptocurrencies as
they change from brute force to be more environmentally friendly methods,
i.e., proof-of-work to proof-of-stake (Hetler, 2022).

####  2.1.10. Cs, Backtracking algorithm

Backtracking algorithms move forward step by step. Constraints control each
step. If a potential solution cannot achieve its objective, the path halts,
and the algorithm backtracks to explore another possible solution path. This
approach is robust at exploring different options for success. This algorithm
can stop early if a solution option reaches a good-enough level of success.

####  2.1.11. Cs, Graph traversal algorithms

Graph traversal is simply the process of visiting nodes in a graph. The visit
can involve either reading or updating the node. There are different methods
on the order of visits, e.g., depth-first and breadth-first. Where depth-first
attempts systematically to visit the farthest nodes, by contrast, breadth-
first attempts to visit all the nearest nodes first. Graphs are popular for
many applications, including graph databases, spatial graph databases, and
spatial-temporal graph databases.

####  2.1.12. Cs, Shortest path algorithm

Where graph traversal techniques are general graph algorithms, shortest-path
algorithms discover the shortest path between nodes in a graph. One of the
more famous algorithms in this category is the Dijkstra algorithm. A popular
variant uses a source node and calculates the shortest path to any other node
in the graph. Common usages include navigating between two points on a map.

####  2.1.13. Cs, Linear programming

The linear programming is a mathematical modeling technique where a linear
function is either maximized or minimized under constraints. The behavioral
outcome makes processes more efficient or economically cost-effective. This
behavior means that any problem that requires more efficiency can take
advantage of linear programming, whether the situation involves improving
energy distribution or mathematical problem solving.

###  2.2. Artificial intelligence, Ai

Artificial intelligence encompasses many algorithms, from object recognition
(i.e., correlation) to the far-out attempts to create artificial life. We can
crudely subdivide the subject into people-types connectionist, evolutionist,
bayesian, analogizer, and symbolist. In the early days, artificial
intelligence covered everything we could not achieve. Today, a broader
definition is used that defines the subject by problems. The general goal is
to tackle evermore difficult challenges where the path is less well known.

As with Computer science, we do not cover all the algorithms in the field, but
we will try to cover some interesting behavioral classes.

####  2.2.1. Ai, Reinforcement learning

Reinforcement learning is an award-style algorithm. The algorithm rewards a
path that gets closer to a solution. It encourages forward progression. The
disadvantage is overlooking a better solution, a single-minded approach;
nevertheless, it is a powerful algorithmic technique. The single-mindedness
makes it potentially dangerous without some form of safeguards.

####  2.2.2. Ai, Evolutionary algorithms

Use a synthetic form of evolution to explore problem domains; if we consider
the world as a two-dimensional graph with data on the x-axis and algorithms on
the y-axis. Neural networks live near the x-axis, and evolutionary algorithms
live near the y-axis. They modify algorithms. Either by playing with the
variables or creating & modifying the potential solutions directly. Similar to
biological evolution, there is, for the most part, a population of potential
solutions, and that population goes through generational changes. These
changes occur through mutation and crossover. Each member of the population,
in a generation, is valued by their fitness towards the potential solution. A
population can either start as an initial random seed (i.e., tabular rasa) or
with a known working solution. We use these algorithms for optimization and
discovery. These algorithms are powerful, especially when the problem domain
is too big or the solution is beyond human knowledge (Simon, 2013).

####  2.2.3. Ai, Correlation

Correlation allows pattern recognition with a level of certainty. For example,
we are 87% sure that the orange is behind the pineapple. Neural nets provide
certainty of recognition. Convolution neural networks and deep learning rely
on this technique to solve problems. Hyperparameters configure the network,
which is a complicated process. The network learns a response using training
data. The quality of the training data determines the effectiveness of the
correlation. This technique is good at image and speech recognition.

####  2.2.4. Ai, Gradient descent

Gradient descent is about finding the best descent path down a steep hill. The
technique reduces the cost and loss of setting up a neural network model by
looking for the fastest descent. In other words, it is about finding the
optimal minimum and avoiding the local minimum of a differentiable function.
As the algorithm proceeds, it sets up the various parameters of a neural net
model. Both Machine learning and, more specifically, deep learning use this
technique.

####  2.2.5. Ai, Ensemble learning

Ensemble learning uses multiple learning algorithms to obtain better
predictive performance. Like predicting the weather, numerous futures are
provided, from the extremes to the most likely. In other words, the ensemble
method uses multiple learning algorithms to obtain better predictive
performance than could be obtained from a single algorithm. An ensemble
learning system consists of a set of different models: with diversity in
structure (and hyperparameters). Its outward behavior is to produce several
solutions in the hope of finding a better solution.

####  2.2.6. Ai, Random forest

Random forest is a type of ensemble learning. Random forests apply to
classification (the act of classifying due to shared properties), regression
(finding cause and effect between variables), and tasks that involve building
multiple decision trees (decisions represented by a tree leading to a
consequence). The random forests method generally outperforms traditional
decision trees, but their accuracy can be lower than other methods.

####  2.2.7. Ai, Continuous learning

Continuous learning has a long history with traditional evolutionary
algorithms. An algorithm is left to learn in an environment with resources.
The algorithm continuously adapts. Experiments have shown these algorithms
exhibit the most significant learning jumps at the beginning of the cycle, and
as time progresses, jumps become ever fewer, if not at all. This situation can
alter if changes occur in the environment (e.g., additional resources or
objectives).

####  2.2.8. Ai, Novelty search

Novelty search is a different approach to, say, reinforcement Learning (see
Section 2.2.1), where the rewards are for learning new experiences rather than
moving nearer to a goal (Stanley and Lehman, 2015a). Novelty search has the
behavioral advantage of not requiring an initial objective for learning to
occur. For example, learning to walk occurs by learning how to fall over.
Falling over is not directly linked to the act of walking.

####  2.2.9. Ai, Generative adversarial network, GAN

Generative adversarial network is, in fact, two networks —each one vying to
attain different goals. One side is a creator (i.e., the generative network),
and the other is a critic (i.e., the discriminative network). The creator’s
role is to outsmart the critic, meaning the creator can mimic the dataset.
From a behavioral view, this algorithm can create a fake version (or deep fake
output) of existing material. As of writing this text, this technique is
taking over many traditional learning algorithms.

####  2.2.10. Ai, Supervised learning

This class is a more generalized version of the correlation mentioned in
Section 2.2.3. Supervised learning is a system using input-output pairs. In
other words, known input samples connect to known outputs. The system learns
how to connect the input to the output. The input data is labeled. Currently,
the majority of machine learning is of this form. From a behavioral view, this
algorithm requires human supervision, as the name implies. The quality of
training data is paramount.

####  2.2.11. Ai, Unsupervised learning

Unsupervised learning is the opposite of supervised learning (Section 2.2.10).
The input data is unlabeled, meaning it is not of a particular form. For
example, no pictures labeled cats. This algorithm class attempts to find
connections between the data and output. The unsupervised means no human has
gone along labeling the data. From a behavioral view, this is a desired
attribute but more challenging to control and get right —for example, the risk
of connecting uninteresting features to an outcome.

####  2.2.12. Ai, Self-supervised learning

It is a compromise between supervised and unsupervised learning. Self-
supervised learning learns from unlabeled sample data, similar to unsupervised
learning. What makes it an in-between form is a two-step process. The first
step initializes the network with pseudo-labels. The second step can be to use
either supervised or unsupervised learning on the partially trained model from
step one. From the behavioral view, not sure it shows any difference between
supervised and unsupervised learning.

####  2.2.13. Ai, Bayesian probabilism

Bayesian probabilism is classical logic, as far as the known is concerned. New
variables represent an unknown. Probabilism refers to the amount that remains
unknown. This class of algorithms is best for robotics, particularly
Simultaneous Localization and Mapping (SLAM). These algorithms are good at
mitigating multiple sources of information to establish the best-known
consensus. For example, while a robot tracks its path, it always runs on a
minimal amount of information and makes decisions based on statistical
likelihood.

####  2.2.14. Ai, Knowledge graphs

As the name implies, knowledge graphs use data models as graph structures. The
graphs link information together. The information can be semantic (hold
meaning). Knowledge-engine algorithms use knowledge graphs to provide
question-answer-like services, i.e., expert systems. These algorithms, in
theory, can provide some form of causality mapping since the graphs store the
knowledge relationships.

####  2.2.15. Ai, Iterative deepening A* search

Lastly for artificial intelligence, we decided to include one traditional
artificial intelligence algorithm from the past. Iterative deepening A* search
is a graph traversal search algorithm to find the shortest path between a
start node and a set of goal nodes in a weighted graph. It is a variant of the
iterative deepening search since it is a depth-first search algorithm. We use
these algorithms in simple game-playing —for example, tik-tac-toe or,
potentially, chess.

###  2.3. Quantum computing, Qc

Figure 15. Sum-over-histories (Feynman diagram)

Quantum computing is relatively new in the context of algorithms since
hardware devices are rare and, if not difficult, at least different to
program. When describing the world of quantum computing in a few paragraphs,
it quickly becomes apparent that we could slide into an overly complex
explanation. To avoid some of the complexity and remain relatively helpful, we
decided to explain quantum computing from the perspective of how it differs to
classical digital computing (Chara Yadaf, 2022). Keep in mind quantum
computers require a lot of classical computing to operate.

Quantum computers follow a different set of rules or principles. These rules
come from atomic and subatomic particle physics, i.e., the notoriously
complicated world of quantum mechanics (Bhattacharya, 2022; Halpern, 2017).
Classical digital computing uses transistors to implement bits. Quantum
computers use even smaller atomic-scale elements to represent quantum bits,
also known as qubits. A qubit is the unit of information for quantum
computers. Transistors represent either $0$ or $1$, binary qubits represent
$0$ and $1$ simultaneously by including a continuous phase. Qubits, therefore,
have the unique property of simultaneously being in a combination of all
possible states. This fundamental principle of quantum mechanics is called
superposition. Superposition enables a quantum computer to have non-classical
behavior. This non-classical behavior means we can probe many possibilities at
the same time (Joanna Roberts, 2019) with the potential for saving
considerable energy over classical computing.

Another principle used in quantum computers is entanglement. The basic concept
is that quantum systems can correlate so that a measured state of one can be
used to predict the state of another. This connection enables the construction
of quantum computers to include linkages between sets of qubits. It reinforces
other physical principles, such as causality limiting communications to no
more than the speed of light. Entanglement and superposition are intimately
connected, in subtle ways, even across long distances.

Relative to a classical computer, a contemporary quantum computer has higher
error rates, better data analysis capabilities, and continuous intermediate
states. Compare this with classical computing, which has far lower error
rates, is better for day-to-day processing, and uses discrete states (Chara
Yadaf, 2022). A quantum computer comprises a set of qubits, data transfer
technology, and a set of classical computers. These classical computers
initialize the system, control it, and read the results. Where the qubits
carry out the computation, the transfer technology moves information in and
out of the machine. At the conclusion of the calculation, measurements of
quantum states will return definite classical values at the outputs. Different
runs on the same inputs return a distribution of results whose magnitudes
squared are interpreted as a probability distribution. After completing the
quantum calculation and measurements, classical computers do error correction,
interpolation, and filtering across many runs of the same program. The
information involves quantum (superposing and entangling qubits) and classical
(configuring the initial values and reading out the classical final values).
Modern quantum computers can efficiently collect statistics. These statistics
provide more probable answers to more complex problems instead of definitive
answers to simpler ones.

As with computer science and artificial intelligence, we will explore quantum
algorithms, not from the quantum computing perspective but the algorithm
perspective, i.e., what can they do?. In keeping with the previous
discussions, we provide a subset of algorithms. This technology has enormous
potential but maybe ten or more years away. We believe it is essential to
include this area when exploring future algorithms.

####  2.3.1. Qc, Shor’s algorithm

Shor’s algorithm appears to be currently the most important, or most
practical, in quantum computing. Shor’s algorithm takes an integer $N$ and
returns the prime factors. This is achieved in polynomial time (NP, see
Section 1.7). An alternative to Fourier transform. Shor’s algorithm uses the
quantum Fourier transform to find factors. This behavior has implications for
cryptography. It is an exponential speedup from the ability of a classical
digital computer to break the types of cryptographic codes used for
authentication and signatures (e.g., common methods such as RSA or ECC). This
capability gives quantum computers the concerning potential to break today’s
security algorithms.

Note: In the press and academia, we now hear the term Post-Quantum Encryption
(PQE). PQE is a classical computing response to this capability. The response
is a set of classical algorithms resistant to quantum methods. Many meta
versions can exist because they can be dynamically updated and modified.
Making the approach less reliant on the underlying hardware.

####  2.3.2. Qc, Grover’s algorithm

Grover’s algorithm is probably the next most important quantum algorithm.
Grover is said to be a quantum search algorithm. The algorithm carries out an
unstructured search. And, as such is potentially useful to speed up database
searches. There is a potential that classical algorithms could be equally
competitive. The algorithm finds a unique input with the highest probability
for a particular output value. This in theory can be achieved in
$O({\sqrt{N})}$ time. Where $N$ is defined as the size of the function domain.

####  2.3.3. Qc, Quantum annealing

Quantum annealing (QA) is a metaheuristic. A metaheuristic is a problem-
independent algorithm normally operating at a higher level. Metaheuristics
look at a set of strategies to determine the best approach. Quantum annealing
finds a given objective function’s extreme, either minimum or maximum, over a
set of solutions. In other words, it finds the procedure that finds an
absolute minimum for size, length, cost, or distance from a possibly sizable
set of solutions. Quantum annealing is used mainly for problems where the
search space is discrete with many extremes —limited only by available
resources.

####  2.3.4. Qc, Adiabatic quantum computation (AQC)

Adiabatic quantum computation is reversible (Bennett, 2002). The word
adiabatic means no heat transfer, allowing for reversible computing, see
Section 1.3. Calculations occur as a result of the adiabatic theorem.
Optimization is the first application for these algorithms, but there are
potentially many others. It is an alternative to the circuit model (from
digital computing). This alternative makes it useful for both classical and
quantum computation.

####  2.3.5. Qc, Quantum walks

Lastly, Quantum walks is the quantum equivalent of the classical random walk
algorithm (Venegas-Andraca, 2012). Similar to the other quantum solutions, a
quantum walk operates with different rules. On certain graphs, quantum walks
are faster and, by implication, more energy efficient than the classical
equivalent (Kadian et al., 2021).

##  3\. Hardware options

> ”without hardware, we don’t have algorithms, and without algorithms, there
> is no purpose for the hardware”

Even though we are mindful of analog solutions and the exciting developments
in quantum hardware, we will focus primarily on digital solutions. We are also
aware of Moore’s Law’s limitation, which may affect the future direction of
computation, e.g., neuromorphic computing, DNA computing, analog computing, or
unconventional computing. Maybe over time, there will be a change of emphasis
toward analog, but today, digital systems lead. Digital systems include some
form of traditional Turing machine. Turing machines are either fully
implemented (e.g., a general-purpose processor moving towards the Principle of
Universality) or partially implemented devices (e.g., a specialized
accelerator missing some control elements).

Compute systems fall into two categories based on input data. The data is
either embarrassingly helpful (i.e., sequential or parallel) or embarrassingly
unhelpful. For the former, embarrassingly helpful, we design specialized
hardware. For the latter, embarrassingly unhelpful, we design universal
hardware to accommodate a broader range of problems.

Year | Cause | Effect |  |
---|---|---|---|---
1912 | JK Flip flop | Start of Boolean logic in circuits |  |
1914 | Floating point | Algorithms to handle real-world problems |  |
1936 | Turing machine | Universal computation model |  |
1943 | Finite automata | Original pattern recognition method |  |
1945 | ENIAC | First programmable computer, draft EDVAC report |  |
1946 | Automatic Computing Engine (ACE) | RISC-style computation |  |
1948 | Digital Signal Processing | Analysis and processing of continuous data |  |
1954 | SR, D, T | Adding temporal logic |  |
1955 | Finite State Machine (FSM) | Complex pattern matching |  |
1959 | Metal–Oxide–Semi. Field-Eff. Transistor (MOSFET) | Enabled far more sophisticated algorithms |  |
1961 | Transistor–transistor logic (TTL) | Continuing to enable increased complexity |  |
1961 | Virtual memory | Decoupling from physical constraints |  |
1964 | IBM System/360 | Inflection point in architectures |  |
1965 | Memory Management Unit | Standard control of decoupling |  |
1966 | Single Instruction, Multiple Data (SIMD) | Fast method of handling one dimensional arrays |  |
1967 | Virtualization | Make the underlying hardware virtual |  |
1968 | IBM ACS-360 SMT | Full utilization of the processor |  |
1971 | Intel 4004 | Allow for hard-coded algorithms (no stack) |  |
1972 | Single Instruction, Multiple Threads (SIMT) | SIMD array processing algorithms |  |
1972 | Packed SIMD | Speed-up software CODECs |  |
1973 | Ethernet | Distributed (networked) algorithms |  |
1975 | Dataflow | Execution flows on context |  |
1976 | Harvard cache | Separating data and instruction efficiencies |  |
1976 | RCA’s “Pixie” video chip GPU | Geometry based algorithms |  |
1978 | Ikonas RDS-3000 (claim first GPGPU) | Machine learning & Crytocurrency |  |
1979 | Motorola 68000 (CISC) | Execute sophisticated programming languages |  |
1979 | Berkeley RISC | Change in algorithms to support load/store |  |
1981 | Quantum computing | Probabilistic mathematics i.e., qubits |  |
1983 | Networks (ARPANET) | Adoption of the TCP/IP protocol |  |
1984 | SPMD | Messaging passing |  |
1984 | VLIW | Instruction level parallelism |  |
1985 | Intel 80386 | Mainstream MMU |  |
1990 | Liquid Crystal Display | Ubiquitous flat screen monitors |  |
1994 | Beowulf clusters | Using standard hardware for massive scale |  |
1997 | Samsung DDR memory | Algorithms could be bigger and faster |  |
2006 | MPMD | Games console (Sony PlayStation 3) |  |
2013 | MIMD systems | Ubiquitous parallel compute nodes |  |
2013 | Predicated SIMD | Associative processing |  |
2020 | Unified memory | Heterogeneous compute sharing memory |  |
Table 2. Cause-and-effect table for classical hardware (best guess)

It is challenging to map hardware developments to algorithm advancements, so
we created the best guess using chronological ordering and the effects, see
Table 2. We combined inflection points, and technology jumps to represent
hardware advancements. This list is endlessly complicated, even when
constrained. It is not clear when a technology caused an effect. If we look at
history, what does it show us? It shows at least three emerging patterns:

  1. (i)

Long-term serial speed-ups have been the priority for hardware until
relatively recently. In more modern times, we can see a steady increase in
parallelism at all levels of the computation stack. Endless gains in
parallelism from bit patterns to network scaling. Parallelism gives
performance advantages for problems that can explicitly exploit such designs.
The exploitation allows for more sophisticated algorithms and to scale out.

  2. (ii)

History of hardware has seen a constant struggle for and against the Principle
of Universality. In other words, a continuous fight between computing engines
that can handle everything (Turing-complete) and specialized ones (Turing-
incomplete).

  3. (iii)

Sophistication of hardware has increased exponentially, allowing algorithms to
be more capable and less efficient. Optimization is more complicated, and the
likelihood of hitting an unusual anomaly is more likely.

##  4\. Next algorithms

What next for algorithms? Making any prediction is difficult, but there are
some standard frameworks we can apply. Firstly, we must determine where future
algorithms will likely come from and why. To get this moving, we look at the
existing algorithms; let us call this the $\alpha$ ”alpha” future.

The $\alpha$ future involves taking what we have and improving either the
performance or efficiency of the algorithms. Conversely, we could also take an
older algorithm and run it on modern hardware. Both these concepts are
increments. These improvements should occur naturally and not necessarily
cause a change apart from maybe more modularization and specialization.
Algorithms that fit this category include fast Fourier transforms, geometric
mathematics, and regular expression. These algorithms rely on steady
incremental improvements in both software and hardware.

Next, and remaining with the $\alpha$ future, are the algorithms that
initially start executing on much larger computer systems and eventually
migrate over time to smaller systems. We predict that many of today’s offline
cloud algorithms, requiring specialized computation and vast resources, will
ultimately become online algorithms on mobile devices (assuming some form of
Moore’s law still exists). For example, learning will move from the cloud to
smaller devices in the coming decades. Learning algorithms are offline high-
intensity applications, so processing does not occur in real time. A shift
towards lighter real-time mobile variations will likely happen in the future,
partly due to privacy concerns and partly due to economic ones (end users foot
the bill).

The improvement in the $\alpha$ algorithms allows a jump (not an increment) in
new future algorithms. These new algorithms represent what we call the
$\alpha\prime$ ”alpha prime” future. Many technological advances coincide,
e.g., speech recognition, natural language processing, and gesture
recognition. These technologies require improvements in existing algorithms
and can be combined to help discover the new $\alpha\prime$ algorithms (Davies
et al., 2021; Hutson, 2022). For example, natural language processing will
likely assist in creating future algorithms, i.e., moving away from
programming languages toward negotiation, where we negotiate with existing
ideas to produce a new solution. We need this to happen if we want to explore
more of the natural world. It requires us to hide some development details,
makes problem exploration more abstract, and leverages existing knowledge.

This abstraction means algorithm construction will likely change in that
algorithms will be designed for re-purposing, modification, and interface
negotiation. The modifiable part is to allow for multi-objective goals (see
Section 1.9). Many next-generation algorithms expect insertion into bigger
systems, where discovery and negotiation occur. Traditional methods are too
rigid for the next set of problem domains. This change means algorithm
development is more about setting the multi-objectives and goal-driven
negotiation than gluing pieces of low-level code together. Below are some
interface types that might be involved in an $\alpha\prime$ future:

  * •

No interface \- potentially entirely machine-driven, no human standard
mechanisms are defined. In other words, an automation system works out its
communication language. For example, Facebook proved this possible when two
machine learning systems created their language for communication (Kucera,
2017).

  * •

Static interface \- human, and strict mathematical interface. Restricted to
the lowest level if the compute node or pointers-to-memory provides basic
types.

  * •

Dynamic interface \- again human and strict mathematical interface, similar in
characteristics to static interfaces but not fixed to the image, i.e., late
binding.

  * •

Messaging passing \- human, less mathematical, unstructured, and non-standard,
propriety concept. Powerful since it can handle heterogeneous systems.

  * •

Messaging passing with ontological information \- human, semantic
representation, structured so that the data can be discovered and assessed.
Data is self-described over raw data.

  * •

Evolutionary interfacing, where a method of evolution decides on a dynamic
process of interfacing. The interface changes depending on workloads and a
temporal element.

Lastly, we have the $\beta$ ”beta” future. The $\beta$ future for algorithms
is more about the computing substrate. The substrate may change into an exotic
computing zoo, e.g., quantum computing, DNA computing, optical computing,
biological computing, and unconventional computing. Silicon will remain the
digital workhorse, but the edge of algorithm development may shift. And with
this change come very different approaches to algorithms and datatypes. It is
exciting to look at new algorithms on these new computing options.

As well as the substrate, the types of algorithms in a $\beta$ future are
different—for example, artificial general intelligence. Artificial general
intelligence is currently a theoretical goal to create a universal
intelligence that can solve many problems. A significant part of these
algorithms is driving the decomposition (breaking a problem into smaller sub-
problems) and recombination (taking the sub-problems and putting them back
together to solve the main problem). Where the algorithms are much more
general solutions than specialized ones, this is important as we try to handle
problem domains at a much larger scale.

###  4.1. Major meta-trends

We see potentially three overarching meta-trends occurring in $\alpha$,
$\alpha\prime$ and potentially $\beta$ futures, namely parallelism,
probability, and interaction. Under those headings, we can link other
subjects, such as artificial intelligence, quantum computing, and computer
science.

####  4.1.1. Parallelism

Is there any more parallelism to be extracted? We have taken what we call
algorithmic structural parallelism (e.g., data-level parallelism, instruction-
level parallelism, and thread-level parallelism) to an extreme. Structural
parallelism is when a problem breaks neatly into similar-looking sub-
problems—covering embarrassingly sequential and parallel problem domains. But
there are other forms of parallelism, for example, biological parallelism.
Independent cells work together in parallel to form complex structures.
Analogous to these natural processes are Carl Hewitt’s concept of actors
(Hewitt, 2015) and John Hollands’ view on complexity (Holland, 2014).

An actor is a small computational element comprising compute, memory, evolving
rules, and adaptable communication (i.e., message based). Actors can have
reason-response capabilities. It moves away from mathematics and appears more
like particle physics. Together with evolutionary techniques, actors can solve
complex problems. Actors have to be free-flowing and can connect loosely; the
loosely based connections are the reason for more message-passing style
interfaces for communication. We have not fully utilized these technologies
because (a) the success of classical parallel systems, (b) the lack of
biological level scaling, and (c) advanced interfacing. Actors will likely
play a much more important role in the future as more evolutionary
technologies glue everything together.

####  4.1.2. Probability

Computer science has long predicted the importance of probability. As we
approach limitations in computation, uncertainty will start to dominate. We
believe the subsequent algorithms will have to link directly or indirectly to
probability. Whether quantum or classical computing, they all rely on
statistical approximation over precision and accuracy.

For quantum computing, error correction will be paramount unless we can make
them less noisy (highly unlikely). Error correction will most likely have to
reside in classical computing. Even though quantum computing opens up new
possibilities for algorithms in one direction, it causes problems in another
(error correction). It is worth pointing out that voting systems (a class of
Byzantine algorithms) are likely to become more common. Error correction codes
are suitable for a specific problem, whereas voting systems are helpful for
system-level corrections.

####  4.1.3. Interaction

Interaction is a different take on future algorithms. What does the term
interaction mean in this context? It is to do with the process of creating new
algorithms. This meta-trend brings new algorithms, old algorithms, and humans
together (Davies et al., 2021). In other words, future systems can solve
problems by selecting an old algorithm or creating a new one. This flexibility
is possible by building advanced tools to explore the algorithm space, i.e.,
algorithms exploring algorithms. We are starting to see this with new
programming languages, and libraries, allowing for greater expressiveness.

This greater expressiveness improves productivity by combining new tools with
future user interface technologies (e.g., natural language processing, speech
recognition, gesture recognition, patterning recognition, and goal-oriented
design). We can see a meta-trend toward a more integrated algorithm
exploration framework, moving away from implementation details. Transition is
only made possible due to the advancements in digital hardware.

The interaction is to accommodate all the added complexity, vast data, and
navigation required to find a new algorithm. No longer can algorithm
development occur without such advanced tooling. This becomes especially
interesting if the tooling involved virtual reality (VR), and augmented
reality (AR). Advancing mathematics by guiding human intuition with AI (Davies
et al., 2021) by Davis et al., published in Nature in 2021, highlights that
machine learning, with human assistance, is beginning to tackle hard
mathematical problems. As algorithms become more expressive, they can be re-
applied, with human assistance, to create even more algorithms. Solving
previously intractable problems.

To provide a glimpse of future capabilities, let us look briefly at human
intent as part of a negotiated search for a new algorithm. Human intent is
essential for many technology areas since it is about second-guessing what a
human intends to do. This guessing may become one of algorithm development’s
most potent tools. Allowing a computer system to understand the intent of a
human objective.

###  4.2. Other trends in algorithms

In this section, we discuss some of the other potential trends in algorithms,
some are just a continuation of existing trends, but others are emerging
trends that may become important.

  1. OT-1.

Automation of everything, algorithms continue to automate activities at every
level —, for example, the automation level required to process package
distribution within a warehouse.

  2. OT-2.

Growth of exploration over exploitation, exploitation remains in common
practice, but the use of exploration is rising. We want to understand more
about the natural world. This change will increasingly occur as we shift to
problems beyond human capability.

  3. OT-3.

Mathematics becomes a target. Algorithms can optimize older equations and
formulas (Hutson, 2022). Algorithms can search a much bigger problem domain
than any human in the past (Hutson, 2022). We will see the optimization of
traditional mathematics using modern techniques. A future mathematician will
most likely be very different from the past.

  4. OT-4.

Spiking neurons, basically any algorithms that allow backward information
feed, i.e., from a higher level back to a lower level, improve recognition or
optimization. This biological method can potentially reduce the size of the
required networks and, in many cases, could enhance the quality of results
(connection to neuromorphic computing).

  5. OT-5.

Development through negotiation, already mentioned but worth repeating is the
creation of algorithms that allow humans to define goals. The goal is a
starting point for navigating complex problem domains. Solutions are developed
over time through negotiation (Victor, 2013).

  6. OT-6.

Pressure on better knowledge representation. Knowledge representation is at
the core of all the activities around algorithms. The pressure is due to the
problem domains becoming more complicated. This trend will continue, and we
will likely see an expansion of basic data types.

  7. OT-7.

Pattern recognition through increased dimensionality. With increasing
resources, adding data dimensions will continue as a style of pattern
extraction from complex and noisy data.

  8. OT-8.

Single shot learning algorithms, is the ability to learn with minimum data
sets. We see a continuous trend to reduce the data required to train the next-
generation algorithms.

  9. OT-9.

Sparse data structures will continue to be important. Using these structures
is a desperate attempt to reduce resource requirements and improve
performance. This measure becomes especially important for algorithms that
require enormous data sets.

  10. OT-10.

Prediction, we can say that predictive model will occur in every critical
area, from social decision-making to instruction pre-fetchers. This trend
continues unabated for the foreseeable future.

  11. OT-11.

Physical three-dimensional algorithms, these are algorithms that deal with the
layout and positioning of physical components. Important for 3D transistor
layout, virtual reality systems escaping the real world, and augmented reality
systems that add to the real world.

  12. OT-12.

Map between physical and virtual worlds will increase in importance. This
mapping is required if we want to accelerate the adoption of simulation, i.e.,
transferring environments quickly into virtual representations.

  13. OT-13.

Byzantine algorithms become more critical as society deploys machine learning
models. Multiple model voting increases the likelihood of a correct
prediction; machine learning will move quickly in this direction to avoid
biases.

  14. OT-14.

Generative Adversarial Network (GAN) continues to be more successful and
valuable along with traditional machine learning systems.

  15. OT-15.

Built-in multi-objective capability. These options allow for more of a weather
prediction-type approach to solutions. The variations can range from maximum
optimization to zero optimization or likely to most unlikely; see Figure 9.

##  5\. Auxiliary support

We have created a list of what might be next for algorithms, but what about
auxiliary support?

Figure 16. Meta-Learning

  1. HG-1.

Multi-level randomness. The requirement for much more sophisticated
randomness. Having multiple levels of good randomness becomes imperative. What
we mean by good randomness is having everything from, as near as possible,
true randomness to pseudo-randomness. We require an ability to dial randomness
to a particular level, including some form of stochastic numbers.

  2. HG-2.

Probabilistic operators, extension of operations to include probabilistic
helper functions. We have integer, fixed point, and floating point operations.
We need probabilistic operators.

  3. HG-3.

P-adic datatypes and operators, we add this as a potential auxiliary
extension. p-adic extends the standard number systems adopted by digital
computers (Palmer, 2022; Weitzer, 2019). Based on prime numbers, p-adic allows
for a different style of flexibility from the traditional extension of real
and complex numbers.

  4. HG-4.

Stochastic rounding, is already gaining momentum. It rounds real numbers using
a probability distribution compared to the traditional fixed rounding up or
rounding down to the nearest number (Palmer, 2022; Connolly et al., 2021).
This method is increasing in popularity, especially with the machine learning
community, opening the door to lower precision numbers.

  5. HG-5.

Biological neuron mimicking. If we compare artificial and biological neurons,
the biological neurons have many more links. We predict a change in the base
neuron for future machine learning.

  6. HG-6.

Memory management optimization, as memory systems become more sophisticated
and complicated, we need new methods to help algorithms optimize memory
efficiency and usage. This help may reside in hardware or future tooling.

  7. HG-7.

Agent or chaotic based parallelism, as mentioned previously, structural
parallelism continues at all levels. Still, there is a potential for a
hardware-assisted agent or chaotic, based parallelisms.

  8. HG-8.

Error-correction, is an old subject with a new set of focuses coming from
critical areas such as quantum computing, probabilistic systems, and
traditional digital systems where geometry shrinkage goes to the limits. Any
system that operates at the boundaries of stability where minor errors can
result in significant problems (Brutzman, 2000).

  9. HG-9.

Spatial-temporal datatypes, as we move into more graphing problems (e.g.,
Virtual Reality, Augmented Reality, Digital Twin, and Physical Simulators),
there is a requirement to make spatial-temporal datatypes a first-class
citizen. A universal datatype for physical systems with potential sparse
recursive scaling coordinate systems and velocity coordinates to represent
n-body problems (Palmer, 2022).

##  6\. Conclusion

We have taken a journey through the algorithm world. It has involved crawling
through tunnels, jumping over fences, and running across fields. As with many
subjects, we picked a few tunnels, fences, and fields, realizing this is a
staggeringly small subset of the ideas. The algorithm world is complex,
dynamic, and full of old and exciting new directions. As we see it today,
there is an undertone that probability and statistics have an increasingly
critical role in tackling complex applications. These problem solutions are
less amenable to simple absolutes.

Over time, the application focus has shifted from calculating the ordnance
range for artillery to pattern correlation using machine learning. These
shifts have transitioned the algorithm world in different directions. We see,
through our journey, new transitions towards data-directed outcomes,
adaptability, and meta-learning.

Data-directed outcomes replace rule-based systems. This transition is not new;
we see this as a continuation. Rule-based systems can handle specific problem
domains, but they fail when a none pre-programmed pattern occurs, i.e., they
lack flexibility. Data-directed outcomes can circumvent, within reason, many
of these problems, which are difficult for rule-based systems. For rule-based
systems, the value often is in the exceptions, not the rules, and for data-
directed outcomes, the value comes down to the quality of the training data.
We may see a mixture of the two systems, with rule-based systems ensuring the
other operates within required boundaries.

Adaptability replaces precision. Precision deals with perfection, whereas
adaptability is handling imperfection. For example, we design robots to
perfect specifications. And algorithms rely on those perfections for length,
pressure, and movement. Adaptability in robots has algorithms that constantly
learn and change as the robot matures or the environment changes. In other
words, the new algorithms handle environmental changes, wear, and poor design.

Meta-learning enhances capability. Increasing capability is important; it is a
horizontal activity. See Figure 16. We want to expand algorithm capability, so
they tackle evermore exciting tasks. At the same time, we also want to
accelerate the actual learning and creative process. Meta-learning is the
process of learning to learn, moving away from the specific problems and
focusing on common patterns. These generalizations force algorithms up the
abstract tree. In that, common patterns can transfer to other problem types.
In the coming decades, we predict much more activity around meta-learning and
integrating more abstract approaches. In addition, this could mean more
weather-predicting style algorithms, i.e., ensemble prediction, that provide a
range of solutions with different characteristics. In other words, a group of
solutions with different probabilities of certainty and uncertainty (Palmer,
2022; Stein et al., 2021). We can potentially build these systems using
multiple models based on physics and probability. It allows us to explore the
unlikely so we can create anticipating actions with cost-loss attributes. For
example, moving people out of danger to avoid a unlikely but possible
catastrophic weather system (Palmer, 2022).

In any modern algorithm discussion, it is essential to mention quantum
algorithms. The quantum world is attractive for its potential energy saving
advantages (Bennett, 2002). We are still in an exploratory phase and starting
to learn how to build basic systems and determine possible algorithms. One of
the many concerns about quantum computing is the quick drive to optimization
before the benefits are genuinely discovered, i.e., the race to be valuable.
Also, the problem domains that quantum computing can explore may not be that
exciting, and traditional computation may remain dominant for the majority.
One true unarguable benefit of quantum computing is exploring the quantum
world itself (Adamatzky and Lestocart, 2021). Quantum algorithms’ ultimate
achievement may be to push classical computing in new directions.

Hic sunt dracones (Here be dragons) is the term used at the beginning of this
exploration. The term describes the world beyond the edge of the map. We are
entering an exciting time around algorithms and what they can accomplish, but
concerns about how they can be abused or used for badness come with that
excitement. For example, we have seen algorithms manipulate people on a mass
scale through social media. Or the various fake images and videos depicting
people saying fictional opposites or non-truths. Just the mechanical process
of validation and verification becomes more of a challenge as algorithms
exhibit more extraordinary capabilities.

We want algorithms to be benevolent in our society. We have seen how
algorithms can influence people away from acting in their best interests. For
this reason, we provided a list of ideals at the beginning of our journey.
These ideals are possible areas of further exploration, but they are not
rules. At best, guidelines.

Lastly, Richard Hamming, Albert Einstein, Neil deGrasse Tyson, and many others
pointed out a common mistake: assuming the new is just like the past. The
mistake prevents many from contributing significantly to the next revolution.
For Hamming, this thought came as he observed the transition to digital
filters (Brutzman, 2000).

##  7\. Acknowledgments

The OPEN DASKALOS PROJECT is an open project to help explain complicated
Computer Science concepts. Each paper is open and undergoes a continuous
process of refinement, i.e., a snapshot in thinking. We thank the great
algorithm writers for creating such exceptional solutions. We would also like
to thank the reviewers, Hannah Peeler, for the initial feedback and Paul
Gleichauf for posing such hard questions throughout the editing process.

## OPEN DASKALOS PROJECT series:

Intelligence Primer (2nd Edition), May 2022
by Karl Fezer and Andrew Sloss

Intelligence is a fundamental part of all living things, as well as the
foundation for Artificial Intelligence. In this primer we explore the ideas
associated with intelligence and, by doing so, understand the implications and
constraints and potentially outline the capabilities of future systems.
Artificial Intelligence, in the form of Machine Learning, has already had a
significant impact on our lives.

URL: https://arxiv.org/abs/2008.07324

## References

  * (1)
  * Adamatzky and Lestocart (2021) Andrew Adamatzky and Louis-Jose Lestocart. 2021.  _Thoughts on Unconventional Computing_.  Luniver Press, UK.
  * Amdahl (1967) Gene M. Amdahl. 1967\.  Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities. In _Proceedings of the April 18-20, 1967, Spring Joint Computer Conference_ (Atlantic City, New Jersey) _(AFIPS ’67 (Spring))_. Association for Computing Machinery, New York, NY, USA, 483–485.  https://doi.org/10.1145/1465482.1465560
  * Bennett (2002) Charles H. Bennett. 2002\.  Notes on Landauer’s principle, Reversible Computation and Maxwell’s Demon.  _Studies in History and Philosophy of Modern Physics_ 34 (2002), 501–510.  https://doi.org/10.48550/ARXIV.PHYSICS/0210005
  * Bhattacharya (2022) A. Bhattacharya. 2022\.  _The Man from the Future: The Visionary Life of John Von Neumann_.  W.W. Norton, New York.  https://books.google.com/books?id=flh0zgEACAAJ
  * Boden (2003) Margaret A. Boden. 2003\.  _Creative Mind: Myths and Mechanisms_ (2 ed.).  Routledge, USA.
  * Borges et al. (2000) J.L. Borges, E. Desmazières, A. Hurley, and A. Giral. 2000.  _The Library of Babel_.  David R. Godine, Boston.  https://books.google.com/books?id=dJVdAAAAMAAJ
  * Brutzman (2000) Don Brutzman. 2000\.  The Art of Doing SCIENCE and Engineering: Learning to Learn, Richard W. Hamming (1915-1998).  https://calhoun.nps.edu/handle/10945/62468
  * Chara Yadaf (2022) Chara Yadaf. 2022\.  Difference Between Quantum Computing and Classical Computing.  https://askanydifference.com/difference-between-quantum-computing-and-classical-computing-with-table/.  [Online; accessed 16-October-2022].
  * Connolly et al. (2021) Michael P. Connolly, Nicholas J. Higham, and Theo Mary. 2021\.  Stochastic Rounding and Its Probabilistic Backward Error Analysis.  _SIAM Journal on Scientific Computing_ 43, 1 (2021), A566–A585.  https://doi.org/10.1137/20M1334796 arXiv:https://doi.org/10.1137/20M1334796
  * Daniel J. Gauthier and Barbosa (2021) Aaron Griffith Daniel J. Gauthier, Erik Bollt and Wendson A. S. Barbosa. 2021\.  Next generation reservoir computing.  _Nature Communication_ 12 (2021).  https://doi.org/10.1038/s41467-021-25801-2
  * Davies et al. (2021) Alex Davies, Petar Veličković, Lars Buesing, Sam Blackwell, Daniel Zheng, Nenad Tomašev, Richard Tanburn, Peter Battaglia, Charles Blundell, András Juhász, Marc Lackenby, Geordie Williamson, Demis Hassabis, and Pushmeet Kohli. 2021\.  Advancing mathematics by guiding human intuition with AI.  _Nature_ 600 (2021), 70––74.  https://doi.org/10.1038/s41586-021-04086-x
  * Foley (2022) Joseph Foley. 2022\.  AI-generated art won a fine arts competition – and artists are up in arms.  https://www.creativebloq.com/news/ai-art-wins-competition.
  * Genkina (2022) Dina Genkina. 2022\.  Engineers Push Probabilistic Computers Closer to Reality.  https://spectrum.ieee.org/probablistic-computing
  * Halpern (2017) P. Halpern. 2017\.  _The Quantum Labyrinth: How Richard Feynman and John Wheeler Revolutionized Time and Reality_.  Basic Books, New York.  https://books.google.com/books?id=OHwdDQAAQBAJ
  * Hetler (2022) Amanda Hetler. 2022\.  Proof of work vs. proof of stake: What’s the difference?  https://www.techtarget.com/whatis/feature/Proof-of-work-vs-proof-of-stake-Whats-the-difference
  * Hewitt (2015) Carl Hewitt. 2015\.  Actor Model of Computation: Scalable Robust Information Systems.  arXiv:1008.1459 [cs.PL]
  * Holland (2014) J.H. Holland. 2014\.  _Complexity: A Very Short Introduction_.  Oxford University Press, Oxford.  https://books.google.com/books?id=lmygAwAAQBAJ
  * Hutson (2022) Matthew Hutson. 2022\.  DeepMind AI invents faster algorithms to solve tough maths puzzles.  Accessed: 2022/10/07.
  * Joanna Roberts (2019) Joanna Roberts. 2019\.  Quantum computers will soon outperform classical machines.  https://ec.europa.eu/research-and-innovation/en/horizon-magazine/quantum-computers-will-soon-outperform-classical-machines.  [Online; accessed 16-October-2022].
  * Kadian et al. (2021) Karuna Kadian, Sunita Garhwal, and Ajay Kumar. 2021\.  Quantum walk and its application domains: A systematic review.  _Computer Science Review_ 41 (2021), 100419.  https://doi.org/10.1016/j.cosrev.2021.100419
  * Kaku (2021) M. Kaku. 2021.  _The God Equation: The Quest for a Theory of Everything_.  Knopf Doubleday Publishing Group, New York.  https://books.google.com/books?id=KYXwDwAAQBAJ
  * Karinthy (1929) Frigyes Karinthy. 1929\.  Chain-Links.  https://djjr-courses.wdfiles.com/local--files/soc180%3Akarinthy-chain-links/Karinthy-Chain-Links_1929.pdf
  * Kennedy (2022) Juliette Kennedy. 2022\.  _Gödel’s Incompleteness Theorems_.  Cambridge University Press, Cambridge, UK.  https://doi.org/10.1017/9781108981972
  * Knuth (1997) Donald E. Knuth. 1997\.  _The Art of Computer Programming, Volume 1 (3rd Ed.): Fundamental Algorithms_.  Addison Wesley Longman Publishing Co., Inc., USA.
  * Kucera (2017) Roman Kucera. 2017\.  The truth behind Facebook AI inventing a new language.  Accessed: 2020/05/01.
  * Luke (2013) Sean Luke. 2013\.  _Essentials of Metaheuristics_ (second ed.).  Lulu, Morrisville.  Available for free at http://cs.gmu.edu/$\sim$sean/book/metaheuristics/.
  * Miller (2016) J.H. Miller. 2016\.  _A Crude Look at the Whole: The Science of Complex Systems in Business, Life, and Society_.  Basic Books, New York.  https://books.google.com/books?id=cKy6CgAAQBAJ
  * Mitchell (2009) Melanie Mitchell. 2009\.  _Complexity: A Guided Tour_.  Oxford University Press, Oxford, UK.  http://www.librarything.com/isbn/0195124413
  * Mozi ( BCE) Mozi. 5th Century B.C.E..  Excerpts from Mozi.  , 568–574 pages.
  * Nakata and Murao (2014) Yoshifumi Nakata and Mio Murao. 2014.  Diagonal quantum circuits: Their computational power and applications.  _The European Physical Journal Plus_ 129, 7 (Jul 2014).  https://doi.org/10.1140/epjp/i2014-14152-9
  * Newton (1687) Isaac Newton. 1687\.  Newton’s laws of motion.  https://www.oxfordreference.com/view/10.1093/oi/authority.20110803100232420
  * Osborne and Rubinstein (1994) Martin J. Osborne and Ariel Rubinstein. 1994.  _A Course in Game Theory_.  The MIT Press, Boston.
  * Palmer (2022) Tim Palmer. 2022\.  _The Primacy of Doubt_.  Basic Books, New York.
  * Pearl and Mackenzie (2018) Judea Pearl and Dana Mackenzie. 2018.  _The Book of Why: The New Science of Cause and Effect_ (1st ed.).  Basic Books, Inc., USA.
  * Poundstone (1992) William Poundstone. 1992\.  _Prisoner’s Dilemma: John Von Neumann, Game Theory and the Puzzle of the Bomb_ (1st ed.).  Doubleday, USA.
  * Russell (2019) S. Russell. 2019\.  _Human Compatible: Artificial Intelligence and the Problem of Control_.  Penguin Publishing Group, New York.  https://books.google.com/books?id=M1eFDwAAQBAJ
  * Sedgewick and Wayne (2011) Robert Sedgewick and Kevin Wayne. 2011.  _Algorithms, 4th Edition._ Addison-Wesley, Glenview. I–XII, 1–955 pages.
  * Simon (2013) D. Simon. 2013.  _Evolutionary Optimization Algorithms_.  Wiley, Hoboken, NJ, USA.  https://books.google.com/books?id=gwUwIEPqk30C
  * Stanley and Lehman (2015a) K.O. Stanley and J. Lehman. 2015a.  _Why Greatness Cannot Be Planned: The Myth of the Objective_.  Springer International Publishing, Birkhäuser Verlag AG, Switzerland.  https://books.google.com/books?id=Llb1CAAAQBAJ
  * Stanley and Lehman (2015b) Kenneth O. Stanley and Joel Lehman. 2015b.  _Why Greatness Cannot Be Planned: The Myth of the Objective_.  Springer Publishing Company, Incorporated, New York.
  * Stein et al. (2021) Samuel Stein, Yufei Ding, Nathan Wiebe, Bo Peng, Karol Kowalski, Nathan Baker, James Ang, and Ang Li. 2021\.  EQC : Ensembled Quantum Computing for Variational Quantum Algorithms.  https://doi.org/10.48550/ARXIV.2111.14940
  * Surhone et al. (2011) L.M. Surhone, M.T. Tennoe, and S.F. Henssonow. 2011\.  _Sparsity-Of-Effects Principle_.  Betascript Publishing, Beau Bassin, Mauritius.  https://books.google.com/books?id=IIrlmQEACAAJ
  * Turing (1936) Alan M. Turing. 1936\.  On Computable Numbers, with an Application to the Entscheidungsproblem.  _Proceedings of the London Mathematical Society_ 2, 42 (1936), 230–265.  https://www.cs.virginia.edu/~robins/Turing_Paper_1936.pdf
  * Venegas-Andraca (2012) Salvador Elías Venegas-Andraca. 2012\.  Quantum walks: a comprehensive review.  _Quantum Information Processing_ 11, 5 (jul 2012), 1015–1106.  https://doi.org/10.1007/s11128-012-0432-5
  * Victor (2013) Bret Victor. 2013\.  Bret Victor The Future of Programming.  https://www.youtube.com/watch?v=8pTEmbeENF4.
  * Weitzer (2019) Mario Weitzer. 2019\.  An introduction to $p$-adic systems: A new kind of number systems inspired by the Collatz $3n+1$ conjecture.  https://doi.org/10.48550/ARXIV.1911.09624
  * Wolpert and Macready (1997) David H. Wolpert and William G. Macready. 1997.  No free lunch theorems for optimization.  _IEEE Transactions on Evolutionary Computation_ 1, 1 (1997), 67–82.