# Tumbug: A pictorial, universal knowledge representation method

Mark A. Atkins, Ph.D.
<EMAIL_ADDRESS>
###### Abstract

Since the key to artificial general intelligence (AGI) is commonly believed to
be commonsense reasoning (CSR) or, roughly equivalently, discovery of a
knowledge representation method (KRM) that is particularly suitable for CSR,
the author developed a custom KRM for CSR. This novel KRM called Tumbug was
designed to be pictorial in nature because there exists increasing evidence
that the human brain uses some pictorial type of KRM, and no well-known prior
research in AGI has researched this KRM possibility. Tumbug is somewhat
similar to Roger Schank’s Conceptual Dependency (CD) theory, but Tumbug is
pictorial and uses about 30 components based on fundamental concepts from the
sciences and human life, in contrast to CD theory, which is textual and uses
about 17 components (= 6 Primitive Conceptual Categories + 11 Primitive Acts)
based mainly on human-oriented activities. All the Building Blocks of Tumbug
were found to generalize to only five Basic Building Blocks that exactly
correspond to the three components {O, A, V} of traditional Object-Attribute-
Value representation plus two new components {C, S}, which are Change and
System. Collectively this set of five components, called ”SCOVA,” seems to be
a universal foundation for all knowledge representation.

###### Contents

  1. 1 Introduction
  2. 2 Motivation
  3. 3 Overview and background of Tumbug
  4. 4 Derivation of Tumbug
    1. 4.1 Knowledge representation methods
      1. 4.1.1 The importance of KRMs in AI
      2. 4.1.2 Typical tradeoffs of KRMs
      3. 4.1.3 Clues to the type of KRM the brain uses
      4. 4.1.4 Ranking of KRM methods
      5. 4.1.5 Why dependency on learning is misguided
      6. 4.1.6 Criteria and heuristics that make Tumbug seem particularly promising
    2. 4.2 Applicable systems from the real world
      1. 4.2.1 Physics
      2. 4.2.2 Mathematics
      3. 4.2.3 Other systems and the Big Picture
  5. 5 Object-like Building Blocks of Tumbug (O)
    1. 5.1 Singleton objects
      1. 5.1.1 Object Circles
      2. 5.1.2 State Circles
      3. 5.1.3 Cells
      4. 5.1.4 Pathway Tubes
      5. 5.1.5 Sensor Bars
      6. 5.1.6 Markers
    2. 5.2 Collection objects
      1. 5.2.1 Location Boxes
      2. 5.2.2 Swirly Arrays
      3. 5.2.3 XOR Boxes
  6. 6 Attribute-like Building Blocks of Tumbug (A)
    1. 6.1 Attribute Lines
  7. 7 Value-like Building Blocks of Tumbug (V)
    1. 7.1 Value Bars
      1. 7.1.1 General
      2. 7.1.2 Ranges
      3. 7.1.3 Example: Goals
      4. 7.1.4 Geometrical variations
    2. 7.2 Wildcards (further development anticipated)
    3. 7.3 Range Caps
  8. 8 Change-like Building Blocks of Tumbug (C)
    1. 8.1 Single Time Arrows
    2. 8.2 Motion Arrows
      1. 8.2.1 Motion of objects
      2. 8.2.2 Streams of objects
      3. 8.2.3 Shorthand notation
    3. 8.3 Force Arrows
    4. 8.4 Causation Arrows
      1. 8.4.1 Single causes
      2. 8.4.2 Multiple causes
      3. 8.4.3 Labeled causes
    5. 8.5 Correlation Boxes
  9. 9 System-like Building Blocks of Tumbug (S)
    1. 9.1 State Diagrams
      1. 9.1.1 Overview
      2. 9.1.2 Common pairs of states
      3. 9.1.3 Applications
      4. 9.1.4 The importance of avoidance of assumptions
    2. 9.2 Split Time Arrows (further development anticipated)
    3. 9.3 Data Set Boxes (further development anticipated)
  10. 10 Rules for combining the single Building Blocks
    1. 10.1 Nonquans with Change Arrows
    2. 10.2 Attribute and Values
  11. 11 Convenience Building Blocks of Tumbug
    1. 11.1 Label Strings
    2. 11.2 Attend Rings
    3. 11.3 Motivation Triangle
    4. 11.4 The Modal Verb Icon
    5. 11.5 The Robinson Icon
    6. 11.6 Zoom Boxes
      1. 11.6.1 General
      2. 11.6.2 WS150 example: #52 (worm)
  12. 12 Generalized Building Blocks of Tumbug
    1. 12.1 Nonquantified Objects (= Nonquans)
    2. 12.2 Interchangeably Actualizable Maps (= IAMs)
    3. 12.3 Change Arrows
  13. 13 Heuristics for converting sentences to Tumbug
  14. 14 Relationship to prior work
    1. 14.1 Conceptual Dependency theory
      1. 14.1.1 Building Blocks
      2. 14.1.2 Slots
      3. 14.1.3 Primitive Acts
      4. 14.1.4 Overview of comparison
    2. 14.2 English grammar representation
      1. 14.2.1 Kolln sentence patterns
      2. 14.2.2 Allerton sentence patterns
    3. 14.3 Object-attribute-value (= OAV) triples
    4. 14.4 Vector graphics editors
    5. 14.5 Semantic networks
      1. 14.5.1 Kinship relations
      2. 14.5.2 Specific objects versus generic objects
      3. 14.5.3 Intensions versus extensions
  15. 15 Applications of Tumbug
    1. 15.1 Natural language translation
      1. 15.1.1 Conversion from a natural language to Tumbug
      2. 15.1.2 Discussion of language translation
    2. 15.2 Mathematics: Arithmetic
    3. 15.3 Logic: First Order Predicate Calculus
      1. 15.3.1 Syllogisms with Venn diagrams
      2. 15.3.2 Barbara form (AAA-1)
      3. 15.3.3 Celarent form (EAE-1)
      4. 15.3.4 Darii form (AII-1)
    4. 15.4 Computer language representation
      1. 15.4.1 Sequential code
      2. 15.4.2 Looped code
      3. 15.4.3 Branching code
    5. 15.5 Natural language representation
      1. 15.5.1 Categories of verbs
      2. 15.5.2 Active tense versus passive tense
      3. 15.5.3 Grammatical time aspects
      4. 15.5.4 Modal verbs
      5. 15.5.5 Propositional attitudes
    6. 15.6 Tumbug as software
  16. 16 A few deep implications of Tumbug
    1. 16.1 SCOVA: an extension to OAV triples
    2. 16.2 Mathematical implications
  17. 17 Insights from Tumbug
    1. 17.1 CD theory and Parts of Speech
  18. 18 Pros and cons of Tumbug
    1. 18.1 The main strengths of Tumbug
    2. 18.2 Possible weaknesses of Tumbug
    3. 18.3 Programs writing programs
  19. 19 Discussion of Tumbug
  20. 20 Summary
  21. 21 Future Research Plan
    1. 21.1 The Visualizer Project
    2. 21.2 Roadmap of the next phases
      1. 21.2.1 Phase 2 (non-spatial reasoning)
      2. 21.2.2 Phase 3 (spatial reasoning)
      3. 21.2.3 Phase 4 (learning)
      4. 21.2.4 Phase 5 (visualizer)
  22. 22 References

##  1 Introduction

A major underlying problem with all natural language translation is that there
does not exist any universally accepted knowledge representation method (KRM)
for all natural languages. This problem in turn appears to be based on a more
fundamental problem: No known method exists, whatsoever, of representing
semantics (meaning) of sentences in any natural language.

Figure 1 shows that a universal representation would require exactly two steps
for any language conversion: one step from the source language representation
to the universal representation, then one more step to the target language
representation. Only 1-2 types programs would be needed, namely 1-2 program(s)
that converted between the universal representation and any specific language
representation. This is a diagrammatic way to express an ”interlingual
conceptual base,” as Roger Schank described it (Schank 1972, pp. 553-554).

Figure 1: First alternative for language translation: This strategy uses one
universal representation, and would need only two programs for three
languages.

Figure 2 shows an extreme alternative: only one step would be needed to
convert from the source language representation to the target language
representation, but six total programs would be needed for arbitrary
conversion between three languages. In the first alternative, n languages
would always require only 1-2 types of programs, whereas in the second
alternative, n languages would always require $n^{2}-n$ separate programs. The
first type of program would be concerned with only one type of mapping, the
second type would be concerned with n types of mappings, which suggests the
first type scales better and generalizes better, and is therefore ultimately a
better solution that saves on computer coding effort. Another motivation for
the universal approach is that it would likely lead to insights into the
nature of language in general, which in turn might also lead to insights into
the relationships between language and other fields such as physics, math,
computer programming, and neuroscience.

Figure 2: Second alternative for language translation: This strategy uses no
universal representation, and would need six programs for three languages.

This article presents a new KRM named ”Tumbug” that appears to be universal
for representing semantics, and therefore can represent any given sentence in
any natural language as a single data structure with appropriately associated
values. This structure-with-values combination appears to be invariant for
every language used to write a sentence that has the same meaning, provided
that the corresponding words have exactly the same meaning across the two
languages, and provided that all attributes of the corresponding words across
the two languages are the same. Thus Tumbug may implement the ”universal
representation”–the center circle of the three language circles shown in
Figure 1.

Emphasized clarification: This is a theoretical study, not an empirical study.
Misinterpretation of the nature of this study probably arises from the common
misinterpretation of the term ”artificial intelligence” to mean ”machine
learning,” but machine learning is only one subfield within AI, and the study
in this document is not about the subfield of machine learning.

##  2 Motivation

The overall goal of this research is to advance progress in artificial general
intelligence (AGI). AGI is the ultimate goal of the larger field of AI in
general, although currently only the practical, applied aspects of AI–the
subfield of AI called artificial narrow intelligence (ANI)–is well understood,
already implemented, and making obvious progress. In contrast, the subfield of
AI called AGI is currently languishing and awaiting new ideas that will
vitalize it so that it too can be applied to real-life problems. Since AGI
progress is usually claimed to be based on Common Sense Reasoning (CSR)
progress (e.g., Davis 1990, p. 1; Devlin 1997, p. 167), the most promising
subfield of AGI to tackle appears to be CSR. Within CSR, one well-known test
of machine understanding of written language is the Winograd Schema (WS).
Therefore example sentences from the WS will be used in this study to generate
generalities, insights, and examples into the data structures needed for such
a universal semantic KRM, and these insights will be implemented as a KRM
called ”Tumbug.” This KRM therefore has the potential to advance the field of
AGI. Figure 3 shows the relationship of the main subfields of AI.

Emphasized clarification: The author has never had any involvement with the
Winograd Schema Challenge (WSC), a competition that no longer even exists. The
WS is used here only as a convenient, well-known list of benchmark problems in
CSR, and its presence in this document should not be interpreted to mean that
this research refers to some past, present, or future submission to the WSC by
the author.

Figure 3: The field of AI is divided into ANI and AGI, and in turn the field
of AGI contains CSR. There is currently little or no overlap between ANI and
AGI except in trivial ways such as hierarchies or the massive training of
large language models (LLMs).

A comment about notation: There exist various versions of the Winograd Schema,
each of which has a designation ”WS” or ”WSC” followed by the number of sample
problems in that schema (list). The WS version used in this document is called
”WS150” because it has 150 problems (Davis 2018), though that designation has
not been used elsewhere. Some other named versions of the WS are WSC273
(Kocijan et al. 2022, p. 2), which has 273 problems, and WSC285 (Kocijan et
al. 2022, p. 6), which has 285 problems.

This article’s convention: WS150 problems quoted in this document have the
second word of the word pair omitted, for simplicity, clarity, and
readability. For example, the WS150 sentence ”The city councilmen refused the
demonstrators a permit because they [feared/advocated] violence” would appear
in this document as ”The city councilmen refused the demonstrators a permit
because they feared violence.”

##  3 Overview and background of Tumbug

The author developed Tumbug from 2021-2023 in an attempt to universalize all
the grammars of all the natural languages for the purpose of making foreign
language learning easier. The word ”Tumbug” is a simpler, more readable
version of the acronym ”TUMBVG” (Temporal Universal Model-Based Visual
Grammar). The term ”visual” in the acronym means that diagrams are used in
this KRM instead of text, numbers, logic symbols, or statistics, which makes
Tumbug an extremely unique KRM of a type that no one appears to have tried
yet. The term ”model-based” means that this KRM is intended to represent
arbitrary models of systems, whether a mechanical system of gears and pulleys,
a biological system of organs and bodily fluids, a physical system such as
planets orbiting a star, or any other type of system. The term ”temporal”
means that the models represented also include time, as opposed to static
diagrams drawn on paper. Although Tumbug diagrams are typically drawn on
paper, Motion Arrows and Time Arrows in Tumbug diagrams show where and how the
components are moving relative to each other, which in a good software
implementation would literally appear as moving images, such as via
simulations or animated GIF files. The word ”grammar” originates from Tumbug’s
originally intended use, namely language translation based on sentence
grammatical structure, but Tumbug can be applied to any system whatsoever.

Figure 4 is a taxonomy of the main knowledge-based approaches to CSR, based
mostly on a list by Davis and Marcus (2015, p. 11) and a very similar list
with more details by Huang et al. (2020, p. 93), but organized here by KRM:
mathematics versus algorithm versus images. The highlighted category ”Image-
based” that includes ”Tumbug” is new to this diagram and is the author’s
approach promoted in this article. Note that Tumbug is not only a completely
new approach, but is also an entirely new class of approach for CSR, one that
has never been explored yet. This situation suggests that Image-based CSR may
contain much new ground for exploration.

Figure 4: A summary of knowledge-based approaches to CSR. Tumbug is so unique
that it has its own category.

So far all code-based approaches have been described as ”informal.” Tumbug
might also be considered ”informal” because there is some flexibility in how
the visual structures are pieced together, although not much.

##  4 Derivation of Tumbug

###  4.1 Knowledge representation methods

####  4.1.1 The importance of KRMs in AI

The fact that KRMs are extremely important for AI is known to surprisingly few
researchers in the field. Related to this, very few AI researchers are working
on KRMs.

  * •

Marvin Minsky said, ’Well, I’d like to have lots of people thinking about how
to combine different approaches. It’s not the different approaches themselves:
it’s ’Why aren’t there more people making a machine that uses three different
representations of knowledge and crosses over?’ That’s a very specific kind of
research project, and I see no one doing it. So that’s, to me, that’s the
missing link.” (GBH Archives 1990)

  * •

Herbert L. Dreyfus wrote, ”A second difficulty shows up in connection with
representing the problem in a problem-solving system. It reflects the need or
essential/inessential discrimination. Feigenbaum, in discussing problems
facing artificial intelligence research in the second decade, calls this
problem ’the most important through not the most immediately tractable.’”
(Dreyfus 1979, p. 298)

  * •

Han Reichgelt wrote, ”Knowledge representation is a major concern in
Artificial Intelligence and its importance cannot be overestimated.”
(Reichgelt 1991, p. 1).

  * •

John Sowa wrote, ”A knowledge representation is a fragmentary theory of
intelligent reasoning”. (Sowa 2000, p. 134) This suggests that a KRM itself,
even without being used in a computer program, is already a partial model of
intelligent reasoning.

  * •

Laurence Harris and Michael Jenkin wrote, ”The representation of knowledge is
fundamental to human cognition and machine understanding” (Harris and Jenkin
1997, p. 2).

  * •

Heather Jenkin wrote, ”Fundamental to all human cognition is the
representation of knowledge” (Jenkin 1997, p. 268). It is therefore not
surprising that the field of knowledge representation arose from the field of
AI (Sowa 2000, p. XI; Cercone 1987, p. 3).

  * •

Haugeland wrote: ”In other words, Artificial Intelligence must start by trying
to understand knowledge (and skills and whatever else is acquired) and then,
on that basis, tackle learning.” (Haugeland 1985, p. 11).

There have been cases in the history of science when merely switching to a
different KRM was the key to major insights and rapid progress, if not
immediate solution to a single difficult problem. Some examples are: (1) The
switch from the awkward rules of Roman numbers to Arabic numerals, especially
when augmented by use of the placeholder digit ”0.” (2) Use of Feymann
diagrams to ease visualization of particle interactions in physics (Fischler
and Firschein 1987, p. 69). (3) Standardization of business information in
data base format for consistent, non-redundant storage and ease of access
through query languages. (4) The switch from a 1D list of elements to the
shaped 2D periodic table of the elements. (5) Use of phase portraits to
visualize chaotic systems (Gleick 1987, p. 317). (6) Discovery that the KRM of
genetics was a long chain of paired nucleobases from the set A, C, G, T, where
A pairs only with T, and C pairs only with G, within each link of the chain.

As for some history on the terminology, the term ”knowledge-representation
method” dates back to at least 1974 with Marvin Minsky (Minsky 1974, p. 28).
The abbreviation ”KRM” for this term is still occasionally found on the
Internet (in 2023). More recent terminology includes the term ”knowledge
representation language” (KRL) (e.g., Reichgelt 1991, p. 3; Luger and
Stubblefield 1998, p. 293), but that term is avoided here because ”languages”
are usually assumed to be textual whereas Tumbug is not, which makes the term
somewhat misleading when applied to visual KRMs such as Tumbug.

The concept of knowledge representation is very close to the concept of data
structures in computer science. Technically a data structure is a structured
collection of data and a set of routines to create and manipulate that data,
whereas discussion of knowledge representation methods tends to ignore any
associated routines. Associated routines are generally ignored in this
document, as well.

Emphasized clarification: Tumbug is a knowledge representation method, not an
algorithm or program, therefore the system is inherently incapable of learning
in its current, non-software form. Misinterpretation of the nature of this
study probably arises from the common misinterpretation of the term
”artificial intelligence” to mean ”machine learning,” but machine learning is
only one subfield within AI, and the study in this document is not about the
subfield of machine learning.

####  4.1.2 Typical tradeoffs of KRMs

1\. General properties

Numerous KRMs abound in the sciences, especially in computer science, and even
in games such as chess. In general, KRMs tend to share the following general
properties:

  1. 1.

Each KRM has at least one competing KRM that can represent the same knowledge.

  2. 2.

Any given KRM will be inefficient in some way that one of its competing KRMs
will not.

  3. 3.

When a system is rendered in a KRM that is particularly suitable for a given
type of problem, algorithms become highly simplified in that KRM. In other
words, a good KRM can solve the hardest part of any given problem due to high
efficiency arising from the closeness of the system’s structure with the
system’s description.

  4. 4.

A KRM that attempts to be suitable for solving too many problems at once
becomes impractically large in its notational system. This is an example of a
time-space tradeoff: increased speed comes at the cost of increased
representation size.

  5. 5.

KRMs that use ”exotic isomorphisms” (verbum Hofstadter 1979, p. 159) are
typically so unwieldy that they are impractical for computational systems.
Some examples of exotic isomorphisms are: (1) DNA structure to represent
phenotype structure (Hofstadter 1979, p. 160), (2) cellular automata to
represent numerical computations (Wolfram 2002, p. 640), and (3) an energy
function to ”program” a continuous Hopfield network to perform the appropriate
optimization. Related to exotic isomorphisms is the desirability of a
representation to remain stable when the represented situation changes
slightly (Fischler and Firschein 1987, p. 69), since violation of this
desideratum results in discontinuous mappings. This strongly suggests that
complexity is another key constraint on system efficiency, in addition to the
better known constraints of time, space, and energy. The opposite of ”exotic
isomorphism” is ”prosaic isomorphism” (verbum Hofstadter 1979, p. 159). In
other words, human designers should design only systems that use prosaic
isomorphisms.

More specifically, applied to the question of which type of KRM the brain
uses, these general properties can be interpreted as:

  1. 1.

The brain’s KRM could be any one of multiple possibilities, including a non-
numerical KRM. This is exactly what Tumbug proposes: a non-numerical KRM that
the brain might realistically use.

  2. 2.

A non-numerical KRM will likely be efficient at solving non-numerical problems
but inefficient at solving numerical problems. This is exactly the difference
that has been noticed since the early days of computer science: humans are
fast at spatial reasoning but slow and error-prone at numerical reasoning,
whereas digital computers are slow and error-prone at spatial reasoning but
fast and accurate at numerical reasoning.

  3. 3.

A KRM that is a good candidate for the brain’s KRM might solve the hardest
part of CSR, which might result in the associated algorithms being quite
simple. This is exactly what the author claims: that Tumbug renders WS
problems easily solvable via very simple algorithms. In other words, the
hardest part of the CSR problem might be easily solvable by a suitable KRM.

  4. 4.

A universal computing machine is the wrong approach for AGI. This is exactly
what the author claims: that digital computers will always have their useful
niche of solving large or intricate discretized problems (such as graph
problems, data bases, and number crunching) whereas a non-numerical machine
would likely have its niche in non-numerical problems with noisy data. In
other words, a non-computational/non-computer-like architecture is likely the
solution to AGI problems.

  5. 5.

Exotic isomorphisms should be eschewed as a design heuristic when developing
intelligent machines, otherwise such machines will be too difficult to program
and to understand. This is one reason why the author believes that that first
practical, intelligent machines cannot be based on cellular automata,
chemistry, or quantum computers. In other words, the first intelligent
machines should use prosaic isomorphisms.

2\. Examples of pairs of competing KRMs

In each of the three domain examples of this section, note that there exist
two main, competing KRMs, the ”best” of which depends on the type of
information elicited, and that both KRMs fail to be useful on the last
question, which means there likely exists a third KRM that is applicable and
efficient. There may exist yet more applicable KRMs beyond these three that
are not discussed.

2.1. Chess moves: algebraic notation versus descriptive notation

The earliest successful symbolic KRM for chess moves is called ”descriptive
notation.” For example, the most common first move on the chess board would be
called ”P-K4,” or ”pawn to king 4,” which means that the pawn in front of the
king (this pawn starts on the 2nd row of the chess board) moves to the 4th row
of the chess board. In contrast, the modern, favored KRM for chess moves is
”algebraic notation.” For example, the move that is called ”P-K4” in
descriptive notation would be called ”e4” in algebraic notation, which means
that the pawn in question (a pawn is implied because no piece letter precedes
the pair of characters) moves to the square represented by ”e” (= the 5th
column from the left) and ”4” (= the 4th row from the bottom). See Figure 5
and Figure 6.

Figure 5: Examples of two KRMs for chess: The shown chess move can be
represented as ”e4” in algebraic notation, based on the indices on the left,
or as ”P-K4” in descriptive notation, based on the indices on the right. Which
is ”best” depends on the type of information elicited. Figure 6: The more
general chess move called ”P-R3” can refer to either side of the board
(kingside or queenside) and to either player (White or Black). This diagram
shows all four possibilities. Figure 7: The mentioned hypothetical combined
notation clearly shows that increasing the power of the representation (in
this case by including more information) tends to increase the size of the
representation, in this case in the form of a longer textual description.
Figure 8: The mentioned hypothetical combined notation approaches a detailed
diagram of an arbitrary object’s trajectory.

  1. 1.

Question: In the move by White, KPxQP (or exd5), what is the destination
square of the capturing pawn?
The best KRM to generate the answer: Algebraic notation, since the answer d5
is already provided in that notation, and this answer cannot be determined by
descriptive notation, which can only list all five possibilities, namely exd3,
exd4, exd5, exd6, exd7.
The simple algorithm used: For move Xij, where X = the unit letter (N, B, R,
Q, K, or nil for pawn moves) with a possible capture sign (= ”x”), and where
ij are the last two characters in the string, the destination square is (i,
j).
Answer, and how it was generated: For the move Xij = exd5, ij = d5, which is
the answer.

  2. 2.

Question: How would one describe the moves P-QR3 and P-KR3 in general,
regardless of the player and the side of the board?
The best KRM to generate the answer: Descriptive notation, since the answer is
the intersection of the two move representations, namely P-R3, whereas
algebraic notation can only list all four possibilities, namely a3, h3, …a6,
…h6.
The simple algorithm used: For moves X and Y, the general description of that
move is X$\cap$Y, where the result treats any nils as empty strings that will
not show when concatenated.
Answer, and how it was generated: For the two moves P-QR3 and P-KR3, we
calculate P-QR3 $\cap$ P-KR3 = P-R3, which is the answer. (See Figure 6.)

  3. 3.

Question: In the move P-K4 (or e4), from which square did the pawn originate?
The best KRM to generate the answer: Neither, since neither notation carries
this information. Descriptive notation can only list all two possibilities,
namely K2-K4, K3-K4, and algebraic notation can only list all four
possibilities, namely e2-e4, e3-e4, …e7-e5, …e6-e5. Another, more verbose type
of notation would be needed, such as long algebraic notation (e.g., Eade 1996,
p. 331).
An example of both representations combined into a third, hypothetical
representation: For a given player (White or Black), a combined notation that
would carry all this information could be $S_{1}$ $U_{1}$@$X_{1}$A$S_{2}$
$U_{2}$@$X_{2}$, where $S_{1}$ = side of board (K or Q) of the moving unit,
$U_{1}$ = originating unit (P, N, B, R, Q, or K), $X_{1}$ = originating
square, A = action (- or x, for ”to” or ”takes”, respectively), $S_{2}$ = side
of board of the destination square, $U_{2}$ = the unit on the destination
square (”E” if the destination square is empty), $X_{2}$ = destination square,
and the square coordinates can have any representation desired (such as from
algebraic or descriptive notation). (See Figure 7.) For example, for the move
e4 (or P-K4) of the Ruy Lopez opening, this generalized notation would be
KP@e2-KE@e4, and for the move exd5 (or KPxQP) of the Center Counter Defense,
this generalized notation would be KP@e4xQP@d5.
Answer, and how it was generated: The move e4 would be represented as $S_{1}$
$U_{1}$@$X_{1}$A$S_{2}$ $U_{2}$@$X_{2}$ = KP@e2-KE@e4, and the originating
square is the variable $X_{1}$ in this string, therefore via matching, $X_{1}$
= e2, which is the answer.

2.2. Elements: list versus periodic table

  1. 1.

Question: What is the name of the element that has 18 protons?
The best KRM to generate the answer: A list, since the list equivalent of the
periodic table would be ordered by atomic number, which is the number of
protons in an element, and a list can be easily stored on paper (as a written
list) or on computer (as a data base or a narrow array), and simply traversed
with that structure, either by eye or by computer.
The simple algorithm used: For a given, specified number of protons i, look up
the name of the element at item #i in a list ordered by atomic number.
Answer, and how it was generated: The element at position #18 in the list has
the name argon, which is the answer. See Figure 9.

  2. 2.

Question: Is element #49 (indium) a metal?
The best KRM to generate the answer: The periodic table, since a clearly
defined region occupying about the left 3/4 of the periodic table is metals,
and all other regions are nonmetals.
The simple algorithm used: If the column of the element in question is $\geq$
3 and $\leq$ 12, and if its row is $\geq$ 4 and $\leq$ 7, then the element is
a metal, else it is not a metal.
Answer, and how it was generated: Indium is located at column 13, row 5, and
since column 13 is $\geq$ 12, this location is outside of the metal region,
indium is not a metal. See Figure 10.

  3. 3.

Question: Which superheavy elements, not yet seen, are most likely to be
stable?
The best KRM to generate the answer: Neither, since neither notation carries
enough information to make such a prediction. Not only are several more pieces
of information (attributes of the element) needed that are not carried in
smaller periodic tables, but nuclides (alternative physical forms of each
element) must also be listed, in contrast to the regular periodic table that
shows only the most common nuclide. The additional information needed
includes: (1) the number of neutrons, not just the number of protons, (2)
half-life information for measuring stability. Also, a spatial arrangement of
the number of protons and neutrons is needed for efficiency so that estimates
can be made about regions of high stability by using spatial direction and
clustering information that results, and the range of proton-neutron values of
such nuclides. Ultimately the best way to show the result is by a 3D plot
called a ”stability graph” of proton number, neutron number, and half-life, as
in Figure 11.
An example of both representations combined into a third representation: See
Figure 12. The same information can be carried by both the periodic table and
stability graph, but there are deep visual alterations. For example, the
periodic table’s 2D regions, such as the region of metals that is shown in
blue, disappears because a stability graph ”straightens out” the 2D periodic
table that was folded along the Z (= number of protons) axis into a straight Z
axis, so any 2D region becomes scattered across the 1D Z axis. Also, the half-
life information ($t_{h}$) that was textual in the periodic table becomes
plotted as a point in a stability graph since that value is of the highest
concern in the stability graph. Also, typical periodic tables do not include
more than one nuclide per element, though the figure shown illustrates how a
periodic table would logically include multiple nuclides per element, a scheme
that causes the periodic table to become 3D with nuclides stored along the N
(= number of protons) axis. In this case a generic representation of both KRMs
at once would require a data base (or spreadsheet) + sort and subsort
operation + list folding (see Figure 14) + plot ability. With such a
foundation, an extended periodic table and stability graph could be converted
to one another in either direction. Whether numerical data is represented as a
data in a data base or a plot is only a preference in how it is viewed, not as
an important distinction in KRM. (See Figure 13.)
Answer, and how it was generated: Visually it can be seen that the center of
the cluster called the ”island of stability” is at the coordinates Z = 114
protons and N = 184 neutrons, which corresponds to element 114, or nuclides of
copernicum (= element 112) and flerovium (= element 114).

Figure 9: A list of elements. The elements are listed here in order of atomic
number. This is basically a 1D data structure where each cell contains an
embedded 1D list of attributes. Figure 10: A periodic table. Note that the
metals form a clear-cut, roughly rectangular region (colored in blue here,
which are: Sc through Zn, Y through Cd, Hf through Hg, and Rf through Cn) when
elements are organized into this periodic table. This table is roughly a 2D
data structure where each cell contains an embedded 1D list of attributes.
(Source: Public Domain, PubChem via NIH.) Figure 11: A stability graph.
Knowledgeable prediction of stable heavy elements requires spatial assessment
of nuclides of the basic elements. This is a 3D data structure. (Source:
unknown.) Figure 12: The stability graph has the same information as a
periodic table that includes nuclides and half-life ($t_{\frac{1}{2}}$)
information, but the information has been spatially rearranged. Figure 13: For
numerical data, plots carry the same information as a data base, and vary only
in the way the data is viewed. Figure 14: A list in which attributes form a
pattern can be easily folded into a 2D table, but the resulting table may not
be purely rectangular. The reverse operation of unfolding such a table into a
list is even easier.

2.3. Series notation: infinite series versus infinite product

  1. 1.

Question: Is it easier to take the derivative of an infinite series, as in
Figure 15, or an infinite product, as in Figure 17?
The best KRM to generate the answer: Infinite series.
The simple algorithm used: Take the derivative of each term. At the end, add
all the terms. If this were an infinite product expression, it would instead
be necessary to use the product rule (fg)’ = f’g + fg’, which requires taking
two derivatives, two multiplications, and one addition per factor, as well as
adding all the terms at the end. See Figure 16.
Answer, and how it was generated: The infinite series in Figure 15. See the
steps of differentiation in the lower levels that figure.

  2. 2.

Question: Is it easier to find the roots of an infinite series, as in Figure
18, or an infinite product, as in Figure 17?
The best KRM to generate the answer: Infinite products.
The simple algorithm used: Set each factor in the product to 0, and solve each
such equation for x.
Answer, and how it was generated: The infinite series in Figure 17. Set
1-($x^{2}$) = 0; 1-($x^{2}$/4) = 0; 1-($x^{2}$/9) = 0, etc., or equivalently:
set $x^{2}$ = 1, which has solution x = 1; set $x^{2}$/4 = 1, which has
solution x = 2; set $x^{2}$/9 = 1, which has solution x = 3, etc.

  3. 3.

Question: Which representation is likely to be the easiest for proving that a
number is irrational: infinite series or infinite products?
The best KRM to generate the answer: Neither, considering the history of the
first proofs of irrationality for the numbers $\pi$ and e. Both of those first
proofs were based on continued fractions, which are neither infinite series
nor infinite products.
An example of both representations combined into a third representation:
Infinite sums and infinite products can be artificially generalized by
introducing an auxiliary function ”a” that can be set to a binary operator
such as addition, multiplication, or exponentiation: a(x, y) = (x + y) or (x *
y) or ($x^{y}$). An infinite sequence of simple expressions to be combined can
then be described by using recursion in conjunction with this function ”a” as
follows: $P_{n}$ = a($P_{n-1}$, $t_{n}$), where $P_{n}$ is the current
partially accumulated value to be computed, $P_{n-1}$ is the previous
partially accumulated value computed, and $t_{n}$ is the current term to be
included. For example, the infinite series 1 + (1/2) + (1/4) + (1/8) + … =
1/$2^{0}$ \+ (1/$2^{1}$) + (1/$2^{2}$) + (1/$2^{3}$) + … could be represented
with this new notation as $P_{n}$ = a($P_{n-1}$, (1/2)$t_{n-1}$), where a(x,
y) = (x + y) to designate summation, $t_{0}$ = 1, $P_{0}$ = 1, $t_{i}$ =
(1/2)$t_{i-1}$ = (1/2)1 = 1/2. Therefore $P_{1}$ = a($P_{0}$, (1/2)$t_{0}$) =
1 + (1/2)1 = 1 + 1/2, and $t_{1}$ = (1/2)$t_{0}$ = (1/2)1 = 1/2; $P_{2}$ =
a($P_{1}$, (1/2)$t_{1}$) = (1 + 1/2) + (1/2)(1/2) = 1 + 1/2 + 1/4, and $t_{2}$
= (1/2)$t_{1}$ = (1/2)(1/2) = 1/4; $P_{3}$ = a($P_{2}$, (1/2)$t_{2}$) = (1 +
1/2 + 1/4) + (1/2)(1/4) = 1 + 1/2 + 1/4 + 1/8, and $t_{3}$ = (1/2)$t_{2}$ =
(1/2)(1/4) = 1/8; etc. Some simpler continued fractions can also be
represented with this recursive general notation if a(x, y) is set to k +
1/a(x, y), but more complicated continued fractions may need an additional
auxiliary function. In general, a pair of coupled recursive formulas are
needed, one for the numerator and one for the denominator (Khinchin 1964, p.
4). For a simple example, $\sqrt{2}$ = 1 + (1/(2+1/(2+1/(2+1/…)))) in
continued fraction form, which recursively would use k = 2, or a(x, y) = 2 +
1/a(x, y).
Answer, and how it was generated: Partial fraction representation was
historically the basis of the first proof that $\pi$ is irrational, proven by
Lambert in 1767, and by Legendre in 1794 (Beckmann 1971, pp. 170-171) (Figure
19), as well as the first proof that e is irrational, proven by Euler in 1737
(Maor 1994, p. 192) (Figure 20). Since these two numbers are the most famous
and most commonly occurring natural mathematical constants, history suggests
that partial fractions are the best representation for proving that a given
natural mathematical constant is irrational.

Figure 15: An infinite series (= sum) can be differentiated easily, per term.
Figure 16: An infinite product cannot be differentiated easily, since the
Product Rule requires an indefinite number (k) of products be multiplied by an
indefinite number (k) of sums as k approaches infinity. Figure 17: An infinite
product with an infinite number of roots, all easy to find. Figure 18: An
infinite series with an infinite number of roots, which are difficult to find,
in general. Figure 19: This continued fraction representation for tan(x) and
the accompanying identity were used in the first proof that $\pi$ is an
irrational number. Figure 20: This continued fraction representation for e was
used in the first proof that e is an irrational number.

3\. Generalizations from the specific KRM examples

Further examples beyond the three specific KRM examples would be prohibitively
long for this document, but in general the following observations seem to hold
for all KRMs:

  1. 1.

The best KRM to use for a given domain depends on which information needs to
be made more explicit, though in making that information more explicit, some
other information is usually made less explicit to compensate for the increase
in representation size. This is already a very well-known heuristic in
computer science.

  2. 2.

It is possible to retain both types of information in the KRM as one type
needs to be made more explicit, but the cost will be the increase in
representation size.

  3. 3.

No matter which new KRM is used for each increase in types of information
retained, there will always be some application that needs yet another,
different KRM, until a KRM with maximum complexity is reached for that domain.

  4. 4.

As the representation size becomes large, the KRM approaches a maximum
complexity in a way that suggests some very general form of KRM, such as a
generic data base without specifying any specific tables, records, or fields,
or a generic neural network without specifying any specific number of layers,
number of neurons, or connection pattern.

  5. 5.

Systems with shaped objects in motion reach maximum complexity in the
representation form of simulation type systems (i.e., SCOVA representation).
Examples of such systems include board games, puzzles, folding molecules, and
the piano mover’s problem.

  6. 6.

Systems without shaped objects, motion, aggregation, or rules for directly
combining objects reach maximum complexity in the representation form of data
base type systems (i.e., OAV representation). Examples of such systems include
data bases, data sets, and the periodic table of the elements.

  7. 7.

Systems with aggregation but without shaped objects or motion, but with rules
or heuristics for directly combining objects reach maximum complexity in the
representation form of formulas or grammars (i.e., OAV representation), where
a large number of relatively simple components of limited types combine
according to simple rules or heuristics to form very impressive structures or
descriptions. Examples of such systems include mathematics, music, language,
chemistry, and genetics.

4\. Problem decomposition into KRM + algorithms

The aforementioned generalizations about KRMs can be viewed as a method of
decomposing a difficult problem into two parts: the KRM and the algorithm that
works on pieces that use that KRM. See Figure 21.

Figure 21: A difficult problem can often be solved by decomposing it first
into an appropriate KRM, which then leads to a simple algorithm. The bulk of
the effort in solving the problem therefore goes into development of the KRM.

Recent evidence from modern large language models (LLMs) suggests that
statistical learning works impressively well for giving the illusion of
intelligence operating behind chatbots based on LLMs. Although such systems
have no understanding of what any of the words mean, and cannot do binding or
explicit spatial reasoning, an important lesson learned from their performance
and popularity is that massive statistical training can work extremely well in
providing smooth, real-time, plausible, default assumptions, and can
occasionally even give rise to spontaneously emergent behavior such as
arithmetic abilities (Rorvig 2021, p. 39).

The author believes that the aforementioned behavior from LLMs suggests that
CSR will decompose into six instead of two main components, where the first
pair of components deals with knowledge matching, and the last pair of
components deals with learning. The training time for learned CSR knowledge is
necessarily very lengthy, essentially of the same order of magnitude as for
human CSR learning, and although this is a complication, time is not shown in
the algorithm decomposition diagram of Figure 22. If this proposed
decomposition strategy holds true for CSR then the implication is that Phase 1
alone of this project will be highly incomplete because it address only two
components–KRM and algorithm–not lengthy learning time. This is where the
fusion of explicit and implicit memory merge: this current study uses explicit
memory in the form of connected groups of icons, but true general intelligence
will need to have both forms of memory, including statistical fusion of huge
numbers of Tumbug icon structures in order to immediately default to proper
associations and temporal continuations. This current direction of study must
start somewhere, however, so the issues of training on Tumbug icon structures
are deferred for future work. Hopefully the follow-up work will be greatly
simplified due to the ability of the Tumbug KRM to represent knowledge in a
useful and profound way, which echoes Haugeland’s statement: ”It may even
happen that, once the fundamental structures are worked out, acquisition and
adaptation will be comparatively easy to include” (Haugeland 1985, p. 11). On
the positive side, conventional neural network training per se is mostly
straightforward, although generating quality training data is a challenge.
Also, a certain amount of Tumbug programming can substitute for naive
training, especially with Correlation Boxes, whose function mappings can
resemble the mappings of neural networks.

Figure 22: The author’s prediction: CSR will require three algorithms in its
decomposition, and all will be based on a KRM like Tumbug. The bulk of the
effort in solving the problem therefore goes into development of the KRM.
KRM = knowledge representation method (viz. Tumbug)
KMA = knowledge matching algorithm (probably a special case of the SRA)
SRA = spatial reasoning algorithm
LA = learning algorithm

As most highly experienced people will confirm, especially teachers of
enormous topics such as a given foreign language or chess, there is no
substitute for experience. General intelligence, just like large academic
topics, requires an very large number of training experiences over large spans
of time in order to generalize and predict properly. Some of this burden can
probably be eased by explicit programming of some heuristics, but the amount,
quality, and detail of knowledge learned by the human brain is incredibly
vast, too vast to encode, even when summarized with explicitly coded
heuristics. This implies that there is no shortcut to overcome the need for
vast amounts of training, which usually involves vast amounts of time.

####  4.1.3 Clues to the type of KRM the brain uses

Not many AI researchers have directly conjectured what type of KRM the human
brain might use, but a few researchers have provided clues:

  1. 1.

Per Roger Schank, predicate calculus is insufficient (Schank 1976, p. 172).
Prior to Schank’s CD theory, predicate calculus was the only suggested
canonical form of propositional information (Schank 1976, p. 172), which is
one indication of the paucity of research into KRMs for AGI, a situation that
still persists today (as of 2023). Implication: The brain’s KRM is more
powerful in representational power than predicate logic.

  2. 2.

Roger Schank reasons that a canonical language representational scheme could
not have natural-language words as its elements since natural language is too
ambiguous (Schank 1976, pp. 171-172). In fact, this limitation was Schank’s
motivation for creating CD theory. Implication: The brain’s KRM is more
powerful in representational power than the tree data structure, which is used
in sentence diagramming.

  3. 3.

William A. Woods wrote, ”Viewing the knowledge base of an intelligent agent as
a model of the external world focuses attention on a number of problems that
are not normally addressed in database systems or knowledge-based expert
systems.” (Woods 1986, p. 1323) Implication: The brain’s KRM is more powerful
in representational power than data bases or rule-based expert systems, which
means more powerful than tables or trees, even when outfitted with search and
matching routines, even when outfitted with an uncertainty management system
such as confidence factors, fuzzy logic, or Dempster-Shafer theory (e.g.,
Coppin 2004, p. 483).

  4. 4.

Ben Coppin wrote, ”The human mind uses some form of representation for all
concepts, which enables us to understand such abstract ideas as ”happiness,”
”lateness,” and ”common sense.” . . . Clearly this internal representation has
a lot to do with our ability to think, to understand, and to reason, and it is
no surprise, therefore, that much of Artificial Intelligence research is
concerned with finding suitable representations for problems” (Coppin 2004, p.
466). Implication: The brain’s KRM needs to represent abstract concepts as
well as concrete concepts.

In summary, it is clear the human brain’s KRM is much more complex and
expressive than any of the commonly used KRMs of AI. At the least, the human
brain’s KRM covers any form of symbols. Jerry A. Fodor wrote, ”I suspect that
the representational system with which we think, if that’s the right way to
describe it, is so rich that if you think up any form of symbolism at all, it
probably plays some role in thinking” (Fischler and Firschein 1987, p. 308).
Similarly, John Haugeland wrote, ”AI has discovered that knowledge itself is
extraordinarily complex and difficult to implement–so much so that even the
general structure of a system with common sense is not yet clear” (Haugeland
1985, p. 11).

Side conjecture: The KRM used by the human brain may be at or above the
pictorial level of representation in a list of KRMs ordered by their
representational power that has the most powerful KRMs at the top.

One possibility is that the human brain’s KRM involves images, or some
something like images (= ”imageoids”), which a possibility that has been
suggested by several authors over the years, and even has an entire book
devoted to the topic, viz. Kosslyn et al. 2006:

  1. 1.

Martin Fischler and Oscar Firschein believe the mind uses two, major
representations, namely propositions and images (Fischler and Firschein 1987,
p. 308), though they admit that, when considering images, some representations
are isomorphic to images but would not be considered images per se. Fischler
and Firschein mention some important advantages and disadvantages of using
images as a KRM.

  2. 2.

John Haugeland notes, ”The beauty of images is that (spatial) side effects
take care of themselves. If I imagine myself astride a giraffe, my feet
’automatically’ end up near its belly and my head near the middle of its neck.
If I have a scale model of my living room, and I put the model couch in the
front alcove, it ’automatically’ ends up by the bay window and opposite the
door. I don’t have to arrange for these results deliberately or figure them
out; they just happen, due to the shapes of the images themselves” (Haugeland
1985, p. 229).

  3. 3.

James Hogan wrote, ”Vision and language involve what are perhaps our most
complex and abstract representations of the external and internal realities we
experience. Comprehending how they operate would get close to the core of what
true intelligence is all about” (Hogan 1997, p. 179).

  4. 4.

Kenneth Haase came up with an idea of visual memory that influenced Marvin
Minsky’s idea of K-lines (Minsky 1986, p. 82), which is a theory of memory.
”You want to repair a bicycle. Before you start, smear your hand with red
paint. Then every tool you need to use will end up with red marks on it. When
you’re done, just remember that red means ’good for fixing bicycles.’ Next
time you fix a bicycle, you can save time by taking out all the red-marked
tools in advance.” He then generalized this idea to different colors for
different jobs.

  5. 5.

Rodney Brooks wrote: ”There is certainly no AI vision program which can find
arbitrary chairs in arbitrary images; they can at best find one particular
type of chair in carefully selected images. This characterization, however, is
perhaps the correct AI representation of solving certain problems; e.g., a
person sitting on a chair in a room is hungry and can see a banana hanging
from the ceiling just out of reach. Such problems are never posed to AI
systems by showing them a photo of the scene. A person (even a young child)
can make the right interpretation of the photo and suggest a plan of action.
For AI planning systems, however, the experimenter is required to abstract
away most of the details to form a simple description in terms of atomic
concepts such as PERSON, CHAIR and BANANAS.” (Brooks 1992, p. 143)

  6. 6.

Donald R. Tveter wrote: ”The fact that people do store many memories as
pictures and do at least some of their reasoning about the world using
pictures ought to be one of the most obvious principles of all, and yet it has
been neglected, in part due to the predominance of symbol processing and in
part due to the fact that processing pictures is still underdeveloped and has
not been used in conjunction with representing the real world in programs
where the goal of the program is to reason about the real world.” (Tveter
1998, p. 18)

The fact that imageoids are such an obvious candidate for the brain’s KRM and
yet are still not being researched to any great extent is puzzling. If a
breakthrough in AGI is going to be made, this is likely the subject area in
which it will occur. This is why the Tumbug KRM has been based on imageoids.

####  4.1.4 Ranking of KRM methods

Can the ”best” KRM be determined by its properties alone? Probably, but this
has not been done yet. Only a few authors have mentioned ways to measure the
relative power of KRMs. One is Arthur B. Markman, who suggested three
attributes (Markman 1999, p. 14):

  1. 1.

the duration of the representational states

  2. 2.

the presence of discrete symbols

  3. 3.

the abstractness of representations

The following previously unpublished, additional criteria would likely be
useful, as well:

  1. 1.

analog representation capability (e.g., for representing time/space with
indefinite precision)

  2. 2.

data structure intricacy or nonlinearity (e.g., tree structures are more
intricate than lists)

  3. 3.

ability to represent time (e.g., for representing moving images, or other
changes over time)

Another such author is Han Reichgelt (Reichgelt 1991, p. 5), who divided KRMs
into two practical considerations:

  1. 1.

syntactic aspects - ”the naturalness and the expressiveness” of the KRM

  2. 2.

inferential aspects - ”the power of the underlying inference machinery”

Despite the lack of consensus on this topic, the following hierarchy attempts
to show a few well-known KRMs, and the author’s estimate of their approximate
representation power relative to each other, where #1 is the most powerful KRM
listed here.

  1. 1.

pictures or illustrations

  2. 2.

directed graphs (e.g., semantic nets, knowledge graphs, state diagrams, DFAs,
NFAs)

  3. 3.

directed acyclic graphs

  4. 4.

trees

  5. 5.

binary trees

  6. 6.

jagged arrays

  7. 7.

arrays or adjacency graphs

  8. 8.

Venn diagrams

  9. 9.

lists

  10. 10.

vectors

  11. 11.

scalars

A few other KRMs such as predicate calculus, pushdown automata, CD theory, and
Petri nets are not on this list since their advantages and disadvantages
involve attributes that are distributed across multiple dimensions and are
distributed excessively across even the levels of a 1D list.

The ”pictures” type of KRM mentioned at #1 occurs in Tumbug only inside of a
Location Box of some type, which is a Tumbug Building Block icon for
representing space. Such pictures may be represented verbatim, in which case a
Verbatim Box is used, or they can be abstracted into general regions and
general objects without being rigidly specified, in which case a Descriptive
Box is used. Together these types of Location Boxes allow concrete images and
abstracted images to be represented, or both types mixed together. The fact
that Tumbug uses the most powerful KRM, viz. images, suggests that Tumbug
could be the most powerful KRM developed so far, provided that its Building
Blocks prove to be strong and consistent.

Side conjecture: In theory, it is likely that ultimately no canonical form of
meaning exists, any more than a canonical description of 2D shapes exists,
since any given description can be based on different conceptual foundations
that can be combined in uniquely different ways that produces unmatched forms.
However, in practice there usually does exist a description that is more
intuitive or natural than others, and usually also simpler than others, which
in practice serves as the canonical form.

####  4.1.5 Why dependency on learning is misguided

A common fallacy is that all that is needed to produce AGI is to let a system
learn. Haugeland (1985, p. 11) described this misconception well:

”Learning is the acquisition of knowledge, skills, etc. The issue is typically
conceived as: given a system capable of knowing, how can we make it capable of
acquiring? Or: starting from a static knower, how can we make an adaptable or
educable knower? This tacitly assumes that knowing as such is straightforward
and that acquiring or adapting it is the hard part; but that turns out to be
false. AI has discovered that knowledge itself is extraordinarily complex and
difficult to implement–so much so that even the general structure of a system
with common sense is not yet clear. Accordingly, it’s far from apparent what a
learning system needs to acquire; hence the project of acquiring some can’t
get off the ground.”

Figure 23 shows basic conceptual components of processors (both for IT and
AGI) and their dependencies. Note that learning is dependent upon data
structures or knowledge structures, or in general learning is dependent of
structured data/information/knowledge/wisdom (DIKW), which in turn are based
on KRMs. Therefore there exists no easy way to avoid KRMs because they are
basic to every processor of which humans can perceive.

Figure 23: Conceptual dependencies in processors. Note that KRMs underlie both
types of processors.

####  4.1.6 Criteria and heuristics that make Tumbug seem particularly
promising

1\. The-only-move that-doesn’t-lose Principle

Every current AI design appears to have at least one fatal flaw that prevents
it from every achieving true general intelligence status. An overview is shown
in Figure 24. A question mark in the table means ”unknown.” Merely by virtue
of being new, Tumbug has a particularly promising design because most of its
table cells are still blank because not enough research has been done on
Tumbug to identify any fatal flaws. Although many blank cells seems to be weak
justification of a promising design, Tumbug was specifically designed to
overcome the flaws of every major AI paradigm. Also, by analogy, a row of the
table is like an evaluation of physical paths through a rugged landscape,
where the traveler knows in advance that at least one navigable path exists:
many paths can be tentatively ruled out because they appear to have impasses,
whereas a path where nothing is known has a greater likelihood of being one of
the paths that are navigable. This type of ignorance evaluation logic also
appears in chess, where the proper move can sometimes be chosen only because
all the well-known alternatives are known to lose, therefore the best
candidate for the best move is a move of which little is known.

Figure 24: Some of the most promising current approaches to AGI.

2\. Organization is more important than performance

Some of the deepest insights into the nature of intelligence and its relation
to AI come from well-known authors who emphasize that what is not important
for producing AGI is not speed (therefore faster computers, parallel
computers, and quantum computers will not be the key to AGI) or superficially
impressive results, but rather the organization of the system. AGI will
clearly need a clever type of underlying organization that has not yet been
publicly considered. Below are some quotes to this effect.

Marvin Minsky: ”Hardware is not the limiting factor for building an
intelligent computer. We don’t need supercomputers to do this; the problem is
that we don’t know what’s the software to use with them. A 1 MHz computer
probably is faster than the brain and would do the job provided that it has
the right software.” (Sabbatini 1998)

Jeff Hawkins: ”According to functionalism, being intelligent or having a mind
is purely a property of organization and has nothing inherently to do with
what you’re organized out of. A mind exists in any system whose constituent
parts have the right causal relationship with each other, but those parts can
just as validly be neurons, silicon chips, or anything else.” (Hawkins 2004,
p. 36)

3\. Pei Wang’s tables

Professor Pei Wang, in his course on AGI at Temple University, Philadelphia,
collected and summarized the approaches to AGI that he considers the most
promising, which are listed in Figure 28. His list of criteria is shown in
Figure 27. The architectures are allowed to come from any of three types of
integration, which are listed in Figure 25. The ”integrated” type is the same
concept that the author called ”fusion” type in 2000 (Atkins 2000), which is
the type that the author favors. The author’s Visualizer Project that includes
Tumbug fits two out of three of Wang’s criteria for a promising AGI
architecture:

  1. 1.

The Visualizer Project is not application-specific, therefore Tumbug fits
criteria #1.

  2. 2.

The Visualizer Project is still currently being worked on, therefore Tumbug
fits criteria #2.

  3. 3.

The Visualizer Project is too new to be documented well, therefore Tumbug does
not fit criteria #3.

Wang’s last two criteria are based on the current worldly success of a given
architecture, so if an architecture fails to meet either of these criteria it
does not necessarily mean that the architecture is deficient per se, only that
it is not popular for some reason.

Figure 25: Wang’s three types of combined systems are hybrid, integrated, and
unified.

Regarding Wang’s three categories of combined architectures (viz., hybrid,
integrated, and unified), see Figure 26. Some examples of two different
architectures being combined in these three ways in a single system are:

  * •

rule-based expert system + neural network (NN):

    1. 1.

hybrid: each type of architecture can be called from a program running in the
system, as needed

    2. 2.

integrated: a heuristic search can use the expert system for the search, and
the NN for the heuristics

    3. 3.

unified: the rule-based expert system is implemented with artificial neurons

  * •

digital computer + analog computer:

    1. 1.

hybrid: each type of computer can be called from a program running in the
system, as needed

    2. 2.

integrated: the digital computer performs only algebraic operations, and the
analog computer performs only calculus operations

    3. 3.

unified: each calculation is done in analog, but is iterated digitally to
confirm accuracy

  * •

object-oriented program + data base program:

    1. 1.

hybrid: each type of program can be called from a program running in the
system, as needed

    2. 2.

integrated: the data base holds the initial values of each object, and the
object-oriented program performs simulations with these objects

    3. 3.

unified: an object-oriented data base

Figure 26: A diagrammatic view of Wang’s list of three strategies for
combining existing disparate architectures. Per Wang, the ideal strategy for
AGI is unified, which happens to be the type that Tumbug is. Figure 27: Wang’s
table of criteria of promising AGI approaches. Figure 28: Wang’s table of some
of the most promising AGI approaches.

4\. Kocijan’s criteria

Kocijan et al. (2022) assessed different approaches used so far to solve WSC
problems, and mentioned the following five desiderata. Each desideratum below
is listed with a comment about how Tumbug (or more generally, a visualizer) is
believed to obviate the mentioned pitfall. Page numbers in this list refer to
the aforementioned reference.

  1. 1.

The inference should be carried out automatically, before the question points
out the anomaly (p. 4).
A visualizer conforms to this criterion, but demonstration of this will
require a knowledge matching algorithm, which will not be fully investigated
and documented until the end of Phase 2. Tumbug is quite simple, so an
architecture such as a visualizer using Tumbug does not require ”commonsense
reasoning of some depth and complexity” (p. 4). A visualizer is a novel
implementation of a ”reflexive reasoning” architecture (cf. Shastri and
Ajjanagadde 1990).

  2. 2.

The approach should not be easily solvable using word correlation (p. 16).
As used so far, the Tumbug approach completely ignores the statistics of word
frequency or word correlation, so obviously Tumbug bypasses this pitfall.

  3. 3.

The approach should demonstrate some semblance of CSR, not merely technical
tricks (pp. 7-8).
Because Tumbug uses a visual grammar that the author believes is universal
across all natural languages, and because Tumbug deals with the actions of
real-world objects, at the very least each scenario that Tumbug visualizes
could be run as a video simulation, which would confirm that the correct
corporeal actions on the correct corporeal objects are being used.

  4. 4.

The system should have reasoning capability, not just general pattern mapping
as in neural networks (p. 23).
Tumbug is mostly symbolic/iconic, not a neural network, so it avoids naive
pattern mapping of the neural network type.

  5. 5.

The system should be able to generalize, so that good performance is attained
across any subset of WC, not just one selected subset (p. 26).
If this statement refers to learning, then a visualizer should conform to this
criterion, but research on this will not begin until Phase 4. If this
statement refers to programmable representation of a concept, that a Tumbug
diagram can already do this.

###  4.2 Applicable systems from the real world

####  4.2.1 Physics

Obviously physics is intended to provide explanations for events that happen
in our physical universe, so a KRM that is trying to represent those same
events should do well by basing itself on physics. Less reliable alternatives
would be to cover only topics in a narrow domain, cover only encountered
topics in some convenient corpus of text, or to guess at which foundations
that might span all known possibilities. Physics is the study of space, time,
matter, motion, energy, and any other related phenomena (Baez 1967, p. 3), and
certainly a large percentage of English sentences describe objects (which
consist of matter) and motion, so physics is one of the most obvious
foundations of descriptions of real-world events. Also, many authors (e.g.,
Pezzulo 2007; Berthoz 2000, p.22) have noted that brains of higher animals
seem to simulate real-world events, especially for purposes of prediction,
planning, and CSR, and to simulate is to represent events partly by relying on
physics formulas, therefore physics is probably the best field on which to
found a set of Building Blocks.

The most basic concepts in physics are often listed at the beginnings of
physics textbooks (e.g., Baez 1967, p. 3), and include especially the
following:

  * •

spacetime: a single structure that includes space and time as separate
dimensions

    * –

SUBCOMPONENTS: space, time

  * •

mass-energy: physical objects

    * –

SUBCOMPONENTS: mass (m), energy (E)

    * –

CONSERVATION: approximately yes, for total energy of a system in classical
mechanics

    * –

INTERCHANGABILITY: with energy via the equation $E=mc^{2}$

  * •

momentum: a property of a physical object

    * –

SUBCOMPONENTS: mass (m), velocity (v)

    * –

CONSERVATION: yes, for an isolated system

    * –

INTERCHANGABILITY: with mass and velocity via the equation p=mv; with force
via the equation $F=\Delta p/\Delta t$

  * •

force: deals with the interaction of objects, not so much with single objects

    * –

SUBTYPES of force: contact force, non-contact force

    * –

SUBTYPES of force: strong nuclear, weak nuclear, electromagnetic, gravity (a
pseudo-force)

    * –

CONSERVATION: not in general, but depends on the type of force

  * •

charge: deals more with single objects, not so much multiple objects

    * –

CONSERVATION: yes, for an isolated system

  * •

probability: used in quantum mechanics where waves consist of probabilities,
not energy

    * –

CONSERVATION: no

Since Tumbug is intended to model primarily the human-centric view of the
world, especially that of human intuition, Tumbug currently does not make a
distinction between true forces (viz., electromagnetic force, strong nuclear
force, weak nuclear force) and pseudo-forces (viz., gravitational force).
Tumbug also uses its own version of spacetime, a much simpler concept than
Minkowski space, largely because no quantified interactions (e.g., the
formulas of relativity) between space and time are considered in Tumbug unless
explicitly modeled. Also, Minkowski space is limited to four dimensions, which
Tumbug space is not. Tumbug is able to model those more complex versions of
physical phenomena, but such constraints are not built into Tumbug.

####  4.2.2 Mathematics

Physics is not the sole and ultimate science for modeling the real world,
however, since the laws of physics are in turn described with mathematics;
mathematics is assumed to be the descriptive foundation of physics. A few
mathematical concepts used in physics that are seldom found in basic physics
books but that are concepts that involve useful mathematics for modeling the
real world are:

  * •

combinatorics (from probability)

  * •

randomness (from probability)

  * •

connectivity (from topology)

A few of the more fundamental mathematical concepts needed to represent
natural language concepts have been mentioned and included with Tumbug
concepts in this document, but there may exist many more mathematical
concepts, possibly infinite in number, that may be needed to describe real-
world situations. In particular, more concepts from first-order logic, which
is a subset of math, may well be needed in Tumbug since predicate logic is one
of the oldest and soundest KRMs for AI systems (Cercone and McCalla 1987, p.
3). Already a form of the NOT concept (viz., XOR) and a form of the
existential qualifier (viz., C-A Aggregation Boxes) from predicate logic have
needed to be added to Tumbug in order to represent WS150 sentences.

####  4.2.3 Other systems and the Big Picture

In the big picture, humankind has recently begun to struggle with its ancient
systems of representation such as mathematics and language to cope with the
extreme advancements of science. Several authors have noted that some new type
of mathematics is likely needed to cope with the very complicated concepts
that modern science is uncovering (e.g., Bailey 1996, pp. 9, 26, 28; Devlin
1997, pp. 282-283; Wolfram 2002, p. 627; O’Neill 1981, pp. 48-49). Presumably
the need for new KRMs is correlated with the need for new mathematics, since
mathematics is itself a type of KRM.

Figure 29 shows the most basic fields from which each concept arises, where
”most basic” means ”most universal”: (1) mathematics, (2) logic, (3) physics,
(4) human world, (5) computer science. This justification is based on the
observations that: (1) mathematics is applicable to any universe because it is
based on nonphysical abstractions, (2) physics is universal for this universe,
regardless of whether physical objects exist or not, (3) human perception and
mental organization can detect natural patterns not covered by physics, (4)
computer science is a human-created system of abstraction not covered
naturally by the human mind.

Figure 29: A summary of the originating fields of Tumbug icons, in order of
increasing universality.

See Figure 30 includes the same list with more detail, but with the
corresponding Building Blocks of Tumbug, in all their higher and lower
generalizations. The specific icons that implement these concepts have names
that are capitalized. These icons visually tend to be a type of Box, Arrow,
Circle, Bar, Line, or String. A few of these icons have not yet been used for
any encountered sentences, but are believed to be essential for certain future
extensions and modifications of Tumbug. Since those unused Building Blocks are
untested, those are the Building Blocks most likely to have unforeseen
drawbacks in their current form. All these Building Blocks of Tumbug are
discussed in detail later in this document.

Figure 30: The current Building Blocks of Tumbug and their originating fields.

Relative to the importance of the five overall categories (S, C, O, V, A) that
are clearly defined by SCOVA (see the section on SCOVA), the particular
choices of categories to be considered Building Blocks of Tumbug are
relatively unimportant, other than for historical value in that those icons
gave rise to the more general SCOVA insight and are more frequently used than
other categories with other levels of generality. For example, since a
Physical C Object Circle and a Data C Object Circle can be used
interchangeably with ease, with possibly nobody even noticing the difference
in appearance of their respective icons (viz., a solid line versus a dashed
line), it is not very useful to consider those different Building Blocks. At
the other extreme, a Change Arrow is too general to be of practical value in
translating sentences into Tumbug because, for example, time and cause-and-
effect are very different concepts, so the Change Arrow icon is not considered
a Building Block, only a Basic Building Block of theoretical value for
categorization purposes. In turn, the exact count of Building Blocks in Tumbug
is not particularly important, although an approximate count is useful for
comparing Tumbug to CD theory, or in estimating the amount of time it would
require to code Tumbug, for example.

Regardless of topic, the concepts and relationships between concepts must be
represented somehow, and some system or systems must bear the burden of
performing that task. Ideally there would exist a universal standard
representation system, a system that Roger Schank sought for language. Since
Tumbug largely follows this ideal, and since Tumbug shows signs of exceeding
the representative power of CD theory, Tumbug may be the representation system
needed to replace math in certain areas such as AI.

The goal of Tumbug is to collect all concepts of the real world that humans
would reasonably want to express in language, and then to represent those
concepts with images. Mathematics and its branches are obvious candidates for
sources of concepts, and those fields have been used in AI for decades.
Physics and model-based reasoning (e.g., Luger and Stubblefield 1998, p. 231)
have also been used in AI, though to a lesser extent. The world of human
beings, which includes psychology and sociology, contains concepts (especially
the concept of ”object” and ”motivation”) not covered in the hard sciences of
inanimate objects, so Tumbug needs to include those concepts, also. Several
knowledge representation concepts (especially general aggregations and labels)
do not occur in their general form in the more general fields so computer
science must be included. (For example, math has sets and parentheses, but
does not generalize these notions into a general ”aggregation” concept.) (See
Figure 31.) Because of Tumbug’s eclectic nature and the infinite nature of the
real world, Tumbug is not guaranteed to cover every basic concept in the real
world, but in practice its span is so extreme that the author has found no
lacunas despite extensive perusal of several fields. If a lacuna does exist,
it would probably be found in some of the less frequent concepts of
mathematics, possibly from group theory or topology.

Figure 31: The Tumbug KRM considers this collection of the fundamental
entities from several systems from the real world, all AND-ed together for to
cover the basics of every one of those fields.

##  5 Object-like Building Blocks of Tumbug (O)

The term ”Building Blocks” comes from other authors’ descriptions of Roger
Schank’s CD theory (Schank 1975), evidently not from Schank himself. More
specifically, the term ”building blocks” is used to describe all the
components of CD theory, including the Primitive Acts and the Primitive
Conceptual Categories. The term ”Building Blocks” is used in this document in
the same way as in CD theory: to describe Tumbug’s basic components,
especially since CD theory and Tumbug have roughly the same intent: to form a
foundation on which any action can be described. Tumbug’s Building Blocks were
derived from generalizing many examples of sentence representation, primarily
from the following sources: (1) WS150 problems, (2) CD theory, (3) Kolln
grammatical examples, and (4) the author’s own insights.

Terminology used in this document for CD theory:

Primitive Conceptual Categories = {PP, ACT, PA, AA, LOC, T}
Primitive Acts = {ATRANS, PTRANS, PROPEL, MTRANS, MBUILD, SPEAK, ATTEND, MOVE,
GRASP, INGEST, EXPEL}
building blocks = Primitive Conceptual Categories $\cup$ Primitive Acts

###  5.1 Singleton objects

####  5.1.1 Object Circles

The preliminary letter ”C” is a formality that stands for ”concrete.” Usually
C Object Circles are called simply ”Object Circles.”

1\. C Object Circles

1.1. Physical C Object Circles

1.1.1. General

A Physical C Object Circle is shown in Figure 32.

Figure 32: Tumbug’s icon for a Physical C Object Circle.

Computers ultimately represent everything with numbers. This is because
computers evolved from calculating machines that work with only numbers, and
because the foundation of all science is mathematics, which primarily uses
numbers. (Some exceptions are group theory and topology.) Even images are
represented in a computer by matrix-like file formats such as BMP, JPG, and
GIF, which are ultimately only numbers arranged in an orderly manner in space.
Mathematics, however, is only an abstraction of the real world. In the real
world, the fundamental entities of importance that are perceived and mentally
manipulated by animals are objects, which are different than both numbers and
pictures.

It is probably this last observation that was primarily responsible for the
development of object-oriented programming (OOP), which groups data into an
abstract data structure called an ”object” whose simulated behavior is similar
to that of its corresponding real-world object. Tumbug likewise treats objects
as the fundamental entity of importance. The Tumbug icon used to represent an
object is usually a plain one-lined circle called a ”C Object Circle.”

Any objects where more detailed anatomy (e.g., taste buds on the object’s
interior, or an arm on the object’s exterior) is important to represent can be
drawn as bumps or protrusions on the perimeter of the C Object Circle,
protruding in the appropriate direction, and can have any desired size,
length, or shape, though typically a single truncated or elongated circle or
ellipse is sufficient for the typical level of detail in WS problems. See
Figure 33.

Figure 33: Left: Example of a physical object with interior structure. Right:
Example of an object with exterior structure.

Very commonly an object, especially a person, needs to be represented as
generating output (e.g., speaking, shouting, gesturing) or actively processing
input (e.g., watching, looking, listening). Logically this would involve an
anatomical structure that is partly inside and partly outside the body, so
Tumbug uses a smaller C Object Circle on the border of the entity’s C Object
Circle to represent such an input-output device. The symbol ”$\langle$” can be
placed on this structure to designate it as an input device (sensor),
”$\rangle$” for output, and ”$\langle$ $\rangle$” for either input or output,
as shown in Figure 34.

Figure 34: Left: Object with input sensor. Middle: Object with output
generator. Right: Object with input-output.

Why are Tumbug representations so simplistic and abstracted, instead of being
more detailed and realistic, for example by showing arms and mouths? The
answer is that Tumbug in a future software implementation would eventually
need to view and interpret its own icons, therefore the simpler the icons that
Tumbug must interpret, the better. This is especially true in language
translation, where the Tumbug representation would need to be read after it
had been stored, in order to make the final translation to the target
language.

C Object Circles are allowed to overlap, although such a need tends to be
uncommon. For example, two circular spots of light would logically be
represented as two C Object Circles, and since spots of light can overlap, C
Object Circles should also be allowed to overlap. Similar logic would apply to
clusters of objects, where each cluster could be represented by one C Object
Circle or by one Aggregation Box, and would apply to set boundaries as seen in
Venn diagrams, where each set boundary could be represented by one C Object
Circle or by one Aggregation Box.

This article’s convention: Tumbug object icons are kept as simple as possible,
as often as possible. For example, details such as hands on arms or taste buds
in the mouth are typically never shown unless relevant to the problem, or
unless explicitly stated in the sentence to be represented.

1.1.2. Icon variations

For the purposes of WS problem representation, Tumbug can be easily and
optionally supplied with icons for any encountered concepts whatsoever,
especially for physical objects. A predefined class of intelligent, human-like
entities seems particularly useful for high-level, cross-ideology-set
discussions in philosophy, science, politics, and religion: human, generic
animal, robot, alien, spirit, cryptid, elf. At the lower-level end of the
hierarchy, the set of animals sheep, deer, fox, dog, wolf, cat, mouse,
chicken, bird, duck, shark, fish, minnow, butterfly, worm would be
particularly useful because those are all the animals specifically mentioned
in WS150 problems, the solution of which one Tumbug-based algorithm was
originally designed to provide. The hierarchy of Figure 35 shows animate
object types and their icons is only one example of the endless elaborate
hierarchies that can be created.

Figure 35: Elaborate hierarchies of types of animate objects can be created
with icons for a Tumbug/WS library.

The exact icons used are not critical, though the existence of such a
hierarchy of objects is useful. Such a library is useful because the more
specific the Tumbug icons are, the more readily a Tumbug diagram can be
understood. One downside of any hierarchy, however, is that a hierarchy is
necessarily rigid, therefore any unexpected item (such as a spork) is
automatically excluded from fitting into any existing classes (such as spoon
or fork). In order to keep this presentation simple, the circle with ”X” and
circle with ”O” are not used elsewhere in this document, but in other
documents such a distinction could be quite useful.

Roger Schank referred to objects in CD theory as ”PP,” which stands for
”picture producers,” meaning objects that have a physical appearance that
typically comes to mind upon hearing or seeing the word for that concept. This
term suggests that use of icons would be particularly beneficial to use in
place of Physical C Object Circles in Tumbug whenever possible. Throughout
this document, icons may appear in place of Physical C Object Circles.

As an aside, the icons of Tumbug can be grouped into different types:

  1. 1.

geometrical icons - very simple circles, rectangles, arrows, lines, etc.

  2. 2.

composite geometrical icons - organized groups of geometrical icons

    1. (a)

composite tangible geometrical icons - for articulators, sensory organs, etc.

    2. (b)

composite intangible geometrical icons - for emotions, verb modality, etc.

  3. 3.

realistic icons - for cats, dogs, computers, bottles, etc.

1.2. Data C Object Circles

1.2.1. General

A complication in the real world, especially in the modern world, is that many
of the ”objects” being used are collections of data, not physical objects.
This distinction can be fairly important in the meaning and implications of
sentences, therefore Tumbug modifies the Physical C Object Circle icon so that
the circle has a dotted border (or sometimes a dashed border) when it
represents a label-worthy chunk of data, as shown in Figure 36. As with
physical objects, data objects can have attributes, such as topic, format,
size in number of bytes, truthfulness, source, and so on.

Figure 36: Tumbug’s icon for a Data C Object Circle.

Data C Object Circles can be thought of as envelopes or flash drives that hold
their contents inside the circle. For example, a digital photo is a data
package whose contents could easily be represented verbatim inside a Data C
Object Circle by placing the photo inside a Verbatim Box inside the C Object
Circle.

1.2.2. WS150 example: #35 (subway)

The example in Figure 37 is from a portion of WS150 question #35. A Data C
Object Circle is necessary because data is implied via the words ”broadcast”
and ”announcement.”

”[35] They broadcast an announcement, but a subway came into the station and I
couldn’t hear it. What couldn’t I hear? POSSIBLE ANSWERS: {the announcement,
the subway}”

Figure 37: [35] Tumbug for ”They broadcast an announcement.”

As with physical objects, data objects have direction of travel, although
their transmission pattern can be more flexible since data without a carrier
path they can either be transmitted in a straight line (as with a laser beam
modulated to carry a message) or in a spherical plane (as with a loudspeaker
broadcasting a message). Dashed planes are more difficult to draw than dashed
lines, so a Tumbug diagram uses three representative lines with a single Data
C Object Circle and a single Motion Arrow to help convey the concept of an
emanating plane wave using simpler icons that are already used in simpler
scenarios.

Conceivably a system of icons could be developed for data objects as well as
physical objects. For example, speech (which is particularly common in WS
examples and grammatical examples) could have its own icon, as could vision.

This article’s convention: Although Tumbug diagrams never need to be in color,
a color-coding convention has often been used on C Object Circles in this
document to aid understanding of grammatical examples, as shown in Figure 38
and Figure 39.

Figure 38: This is the Tumbug color coding scheme used in this document for
the different types of grammatical objects, and the underlying logic for this
scheme. Colors are optional in Tumbug, however. Figure 39: This is the Tumbug
color coding scheme used in this document for other types of objects, and the
underlying logic for this scheme. Colors are optional in Tumbug, however.

1.2.3. Data Points

More general Building Block: Data C Object Circle
Simplifications from the more general Building Block: (1) Only 1-2 attributes
used, namely the attribute-value pairs at which it can be plotted, and the
event(s) from which the measurement was taken.

Figure 40 shows a specific Data Point with two attribute-value pairs. Since a
single data point may have arisen from more than one event or observation or
measurement, in general there may be a list of event IDs in the event ID
field.

Figure 40: A Data Point is a small Data C Object Circle that contains only
attribute-value pairs and optionally a reference to the event that produced
it.

2\. C-A Object Circles

The Tumbug icon for a C-A Object Circle is shown in Figure 41.

Figure 41: Tumbug’s icon for a C-A Object Circle.

”C-A” stands for ”concrete-abstract,” which is a pair of concepts at the two
extreme ends of the abstraction spectrum. Both C Object Circles and C
Aggregation Boxes can be abstracted to produce C-A versions of those icons. In
both cases some attribute-values (those protruding above the associated icon)
are forced upon the object by the user, and the other attributes (those
protruding below the associated icon) are the resulting detected attribute-
values. The value of the outputs may or may not have been influenced by a
Correlation Box. For example, an output could be a default value based on the
object’s class, and untouched by a Correlation Box.

Figure 42: If Billy’s weight is fixed at a high value and his height is fixed
at a low value, then he is forced to be fat.

An example of usage of a C-A Object is shown in Figure 42. Other uses of C-A
objects can sometimes be found in the WS150 problems. For example:

”[9] The large ball crashed right through the table because it was made of
steel. What was made of steel? POSSIBLE ANSWERS: {the ball, the table.}”

In this case, the composition of the unknown object is fixed at steel, which
forces the object’s weight and strength to be very high. Since tables tend to
be made of wood, which forces a table’s weight and strength to be only
moderate (or at most, high), the ball described will tend to have more weight
and strength than the table, which allows default reasoning to occur and for
the correct default answer to be chosen.

####  5.1.2 State Circles

Figure 43 shows a State Circle

Figure 43: Tumbug’s icon for a State Circle.

In Tumbug the need to represent states is almost as common as the need to
represent objects. This is because verbs apply to states almost as frequently
as they apply to objects, and the best icon to use is the icon whose concept
most closely matches the sentence meaning, even if the specific words of the
sentence and their grammatical order superficially suggest otherwise. The
State Circle icon used for a state in Tumbug is intended to resemble the Greek
phi symbol ($\phi$), which is used in physics for phase, which is a concept
that is similar to state. The similarity between the C Object Circle icon and
State Circle icon is intentional since it is sometimes desirable to use a
State Circle instead of a C Object Circle, so swapping one for the other
allows the viewer to overlook the minor conceptual difference when
interpreting a Tumbug diagram quickly.

Ordinarily a State Circle is only a part of a state diagram, as described in
the section on State Diagrams.

####  5.1.3 Cells

The meaning of ”Cell” in Tumbug is identical to the meaning of ”cell” in array
terminology: A cell is a discrete location that can hold a single object. The
object may be a number (as in linear algebra), a text string (as in data bases
or spreadsheets), software objects (as in OOP), or any other type of object,
virtual or real. Cells are usually used only in composite structures such as
vectors, arrays, or jagged arrays.

The Cell concept differs slightly from the Tumbug concepts of Verbatim Box and
C Aggregation Box since a Cell may hold only one object whereas those other
structures may hold any number of objects. A Cell is similar to a State
Circle, except the contents of a State Circle must be a state whereas a Cell
can contain an object that is not a state. A Cell could also be accurately
called a ”pigeonhole,” as in the ”pigeonhole principle,” since a cell conforms
to the same constraint of one object per location.

####  5.1.4 Pathway Tubes

Figure 44 shows a Pathway Tube.

Figure 44: Tumbug’s icon for a Pathway Tube.

A Pathway Tube shows a pathway for anything. It can represent a hiking path
used by people, a pipe for carrying water, a track for carrying trains or
monorails, a fiberoptic path used by photons, a data path in the form of a USB
cable between digital computers, a token path used by Petri nets, the arrows
of logic flow direction in a flowchart, a wave guide for electromagnetic
waves, or other. The pointed end of a Pathway Tube shows the direction of
travel of objects or data within the tube/pathway, like a one-way street sign.
The objects moving in a Pathway Tube can be represented by 0D markers, as
shown in Figure 45.

Figure 45: Pathway Tubes normally contain a 0D Marker, like the tokens of
Petri nets.

Pathway Tubes in this document are always shown as straight, but in general
they need not be. Straight Pathway Tubes can be connected end-to-end to
implement changes in direction, if for example a vector graphics editor does
not a simpler means to create such a shape.

####  5.1.5 Sensor Bars

A Sensor Bar is shown in Figure 46.

Figure 46: Tumbug’s icon for a Sensor Bar (left), and Sensor Bar with a signal
output line (right).

Tumbug contains Values Bars to represent values, but these values cannot be
used unless the values can be read somehow. Sensor Bars solve this problem.
Each Value Bar (by current convention) is typically outfitted with a Sensor
Bar that has a signal output line to represent a signal being sent from the
sensor to some other component if and only if that sensor is activated.

Typically a Sensor Bar is overlaid upon a Value Bar so that if the value in
the Value Bar falls within the span of the Sensor Bar then the Sensor Bar
sends an output signal (marked here as ”*”) on its output line, as shown in
Figure 47 and Figure 48.

Figure 47: A Sensor Bar overlaid upon a Value Bar. Figure 48: A Sensor Bar
senses a value in the specified range within a Value Bar and sends a signal to
a State Circle.

In turn, a single activated state within a state diagram may trigger further
activity within that state diagram, or possibly within a program containing an
”if” statement that relies on that state being active in order to choose with
branch of code to execute next.

Value Bars and Sensor Bars are currently proposed for Tumbug for completeness
purposes, but for diagram simplicity it could be assumed that the system
automatically activates whichever applicable state needs to be read by the
system, and does so in the background, without need to illustrate the details
of how that state became activated.

Sensor Bars in this document are shown with a -45-degree hatching pattern,
whereas 2D Markers are shown with a +45-degree hatching pattern.

####  5.1.6 Markers

Technically, a Marker in Tumbug can be defined as a labeled manifold that
marks something of significance in a Tumbug diagram. For example, a non-
intersecting border drawn around an irregular region in a plane would be a 1D
manifold in 2D space that would bear a label mentioning that it is a border,
and typically also what the border surrounds. Markers can be of any dimension,
and can occupy any dimension, and can have any non-intersecting shape. For
example, a wire that traced the edges of a collection of stacked cubes would
be a 1D manifold in 3D space. Markers can also have unbounded ends, such as an
arrowhead at one end of a 1D line segment, or a 3D arrowhead pointing outward
at the border of a 3D planet’s atmosphere.

1\. Geometrical Markers

1.1. 0D Markers

More general Building Block: C Object Circle
Simplifications from the more general Building Block: (1) Only 1-2 attributes
used, namely location, and optionally priority.

The zeroth dimension (0D) is a point, so in Tumbug a 0D Marker is a single
point that is used to highlight some item or location. This 0D Marker icon is
currently represented in Tumbug as a sizable red spot, as shown in Figure 49.

Figure 49: Tumbug’s icon for a 0D Marker.

Some practical uses of 0D Markers are:

  * •

A checkmark of any kind. (Note that this implements Kenneth Haase’s red paint
idea described by Marvin Minsky (Minsky 1986, p. 82).)

  * •

A red dot on a map labeled ”You are here.”

  * •

A radar blip that indicates the position of an aerial object detected by
radar.

  * •

A bouncing dot over lyrics to help singers keep track of the current part of
the song.

  * •

An ”X” on a treasure map that marks the spot where the treasure is buried.

  * •

A spot like the spot from a laser pointer that shows the part of an overhead
slide being discussed by the lecturer.

  * •

Color-coded labels on unmarked cardboard boxes.

  * •

The desired attribute of a query.

  * •

The current state within a state diagram. If the current state is in
transition between two states then the 0D Marker can also represent this
easily by being placed on the Pathway Tube itself.

  * •

The current statement being executed within a flowchart.

  * •

A token within a Petri net.

  * •

A mental marker to remember a possible king position on a chessboard when
visualizing possibilities in a chess endgame.

  * •

A marker like a painted stone on a trail that indicates a place that was
formerly visited, to prevent going in circles if lost on a hike.

  * •

A position affixed to the rim of a rolling circle.

  * •

The conjectured K-Lines of Marvin Minsky (Minsky 1986, p. 82) to mark tools
that were used for a given job.

One use of 0D Markers in Tumbug in this document is to represent instruction
pointers in an executing computer program. See Figure 50. Whereas a digital
computer uses an instruction pointer that points to the instruction currently
being executed in a program, Tumbug uses a small red dot as a visual means to
mark this instruction in the program itself. Then, instead of incrementing an
address in computer memory to step through a program, Tumbug moves a spot
sequentially through a program. Since the program structure is already
represented as a graph-like flowchart, use of a 0D marker in such a flowchart
maintains the higher-level view of the program as completely visual.

Figure 50: One common usage of a 0D Marker is to step sequentially through
program statements.

An important use of 0D Markers is in diagramming queries. For example, a data
base type query might equate to the question ”What color is Bob’s car?”, which
would place a 0D Marker with an ”information goal” attribute atop the value of
the ”color” attribute in the data base record for Bob’s car, then the value of
that attribute would be noted. In this way an entire question reduces to a
single placed 0D Marker. See Figure 51, where the color red here substitutes
for the attribute ”information goal.” Note that a single spot–the 0D Marker–is
difficult to see on a diagram at a glance, therefore a clearer representation
is recommended in practice, which is the representation with the slanted line
with one end on the desired value and the other end marked with ”DK” (= Don’t
Know). (Currently, use of ”?” to represent queries in Tumbug is discouraged
since ”?” has a different meaning as a regular expression wildcard.) Since all
Tumbug lines are normally drawn either horizontally or vertically, a 1D Marker
line that is slanted at about a 45-degree angle stands out at a glance.

Figure 51: A slanted line in Tumbug is much more visible than a tiny 0D marker
when diagramming queries.

In WS150 problems the queries are only slightly more complicated in that they
are always about disambiguation, but 0D Markers can be used there, also. For
example: ”[108] Grace was happy to trade me her sweater for my jacket. She
thinks it looks great on her. What looks great on Grace? POSSIBLE ANSWERS:
{the jacket, the sweater}” In this query a 0D Marker would be placed atop
value of the ”clothing type” attribute for the item of warm clothing that
Grace is wearing after the trade, with the same attribute ”information goal”
attached to the 0D Marker, then the value of that attribute would be noted.
Oftentimes such queries require following an extra link. In this case, Grace
would first need to be located in the Tumbug diagram, then the 0D Marker
placed on her warm clothing.

1.2. 1D Markers

1D Markers are lines used as a visual aid to mark something as lying in a
straight line. In the WS150 problems such lines would be desirable to show
visual interference along a line of sight, for example. The lines could also
be useful to show alignment, or to show an angle from a given point.

Some practical uses of 1D Markers are:

  * •

A straight line that makes clear the alignments of objects or text, whether
through their tops, bottoms, centers, or other.

  * •

A straight line that separates plotted data points into classes.

  * •

A straight line across an opening in a concave geometrical figure, where the
line separates interior from exterior.

  * •

A straight line between extrema in a convex geometrical figure, where the line
shows regions that can be obscured via occlusion from certain angles.

1.3. 2D Markers

2D Markers are merely shaded regions, as in Figure 52. Such regions are
probably most often used for Venn diagrams, such as to show intersection of
two sets as in Figure 53, or to show the conceptual allowable range of
elements of one set. 2D Markers in this document are shown with a +45-degree
hatching pattern, whereas Sensor Bars are shown with a -45-degree hatching
pattern.

Figure 52: Tumbug’s icon for a 2D Marker. Figure 53: 2D Markers are usually
used to show 2D ranges within sets represented by Venn diagrams.

Some practical uses of 2D Markers are:

  * •

A circle around an important region within a picture or within text.

  * •

A ”set cordon”–an irregular contour that shows which parts of a Venn diagram
are being considered.

  * •

A contour that encloses data points that are considered part of a given
classification.

  * •

Boundaries between regions on a geographical map.

  * •

The four borders on a JPG image that is being cropped.

2\. Relationship Markers

See Figure 54. Relationship Markers use dotted lines, which no other Tumbug
convention uses, which is done for uniqueness, clarity, and consistency. The
consistency arises because a 0D Marker can already be considered a
Relationship Marker with a single point, so two points could be considered a
1D Marker, three non-colinear points could be considered a 2D Marker, and so
on.

Relationship Markers are very generic markers that point out any important
relationship that the user would like to flag. Depending on how Relationship
Markers are used, Relationship Markers may be optional or required.

In Venn diagram representations of First Order Predicate Calculus, a
Relationship Marker would almost be required in some cases. For example, the
final inference of the ”Socrates is mortal” example already shows Socrates as
an element of a subset of both men and mortals, but the user may have
overlooked this 2-step removed inference during the derivation of that final
result since each addition of a Venn diagram circle potentially creates
several new inferences simultaneously, so the important final inference should
be explicitly pointed out by a unidirectional Relationship Marker pointing
from Socrates to mortals.

Relationship Markers often have direction. For example, the sentence ”Bill is
Mark’s uncle” must be stated unidirectionally in text in the form ”X is Y’s
uncle” since the converse ”Y is X’s uncle” is not true, therefore one
arrowhead (centralized on the arrow’s stem) is used to point in the needed
direction (per the user’s convention), such as from X to Y to show the
direction of meaning of ”X is a/an of Y.” Some sentences are bidirectional,
however, such as ”Denny and Wanda are siblings,” therefore in that case two
arrowheads pointing in opposite directions are used to show both directions of
unidirectional meaning simultaneously.

Relationship Markers are also nearly required for genitive case, meaning
possessive situations, such as ”Sam’s drawing” [14], ”Ann’s son” [27], and
”Joe’s uncle” [28], all of which are examples from WS150. (The version of
Tumbug described in the 2022 article on Tumbug described such possessive use
of Relationship Markers as ”genitive arrows” (Atkins 2022), but Relationship
Markers are more general than just genitive case, such as in the phrases
”Socrates is mortal” or ”Denny and Wanda are siblings.”)

Relationship Markers are essentially the same node-with-link data structures
used in semantic networks. Since many sentences and phrases can be easily
represented with semantic networks, this suggests that a good use of
Relationship Markers might be as an intermediate stage of conversion from text
to Tumbug.

Figure 54: Relationship Markers have centered arrowheads on dotted lines, in
any of three possible arrangements.

Relationship Markers could be considered a compound type of icon since
Relationship Markers contain 1D lines that are bent into 2D.

###  5.2 Collection objects

####  5.2.1 Location Boxes

”Location Boxes” is a generic term of Tumbug that currently refers to three
types of rectangular icons in Tumbug, each type of which is intended to be a
border in which to hold multiple items inside, to show the locations of those
items inside the box, and to show the location of the Location Box itself, at
least relative to any other icons that may exist outside the box. Figure 55
shows that these three types of Location Boxes exist in a spectrum of severity
of location constraints. These boxes may be any shape that is needed, although
rectangular is the default shape.

  * •

Verbatim Box: every item inside must exist only in the place where it is
shown. There is no flexibility.

  * •

Descriptive Box: every item’s location is subject to constraints that are
specified with the box. There is some flexibility.

  * •

Aggregation Box: any item is allowed to be in any location within the box.
There is complete flexibility inside the box.

  * •

no box: there are no constraints at all on the locations of the items. There
is complete flexibility and there is not even a border.

Figure 55: Tumbug has three types of Location Boxes that span a spectrum of
severity of constraints on the locations of the items inside. Here the
original locations of four Object Circles are shown by the Verbatim Box, this
square arrangement has been scaled to be larger but still maintains the same
shape in the Descriptive Box, and in the Aggregation Box there are no
constraints on location at all except that the Object Circles must be inside
the box. If not even an Aggregation Box exists then no Object Circles have any
location constraints whatsoever.

Although the two extremes of this spectrum, namely Verbatim Boxes and
Aggregation Boxes, are conceptually simple and fully defined, the middle of
this spectrum, namely Descriptive Boxes, have tremendous flexibility, so would
need more research to fully investigate. For example, Figure 55 shows that
four Object Circles in a square arrangement can be scaled to produce an
unmistakably similar arrangement, but scaling is just one geometrical
transformation. Another geometric transformation that would produce an
unmistakably similar arrangement is translation. The result of rotation may
also be considered an unmistakably similar arrangement, depending on a
person’s taste and goals. Figure 56 shows several such geometrical
transformations, and which common transformation groups include those basic
transformations.

Further development anticipated: Since different types of Location Boxes are
allowed to overlap or to be embedded in one another, precedence rules are
likely to be needed, the same issue as scope of name binding in computer
programming. Probably Verbatim Boxes at the strict end of the strictness
spectrum would need to take highest precedence, for example.

Figure 56: There exist several common types of transformation groups, each
with a unique name and with a unique set of more basic transformations
(translation, scaling, rotation, etc.) that are considered part of that group.

This document usually assumes that Descriptive Boxes: (1) retain the same
spatial arrangement of the original items (i.e., allow differences in
scaling), (2) disallow rotation, (3) disallow reflection and any other more
complicated transformations such as projective transformations. Rotations are
disallowed partly because each diagram is usually assumed to contain a gravity
vector that points straight downward on the page, which in turn suggests that
depicted items are usually viewed from the side, and are either resting flat
on something or have an inherent bottom (such as buildings, transportation,
people, animals, plants, and natural geographical features). Ideally all such
assumptions should be stated somewhere and somehow within each Tumbug diagram
or each project, which in turn suggests that it would be useful to attach
codes to Descriptive Boxes, such as placing an ”R” in the corner of a
Descriptive Box to indicate that all objects in that box have the freedoms of
transformation of the ”Rigid transformations” group.

1\. Verbatim Boxes

A Verbatim Box icon is shown in Figure 57.

Figure 57: Tumbug’s icon for a Verbatim Box.

1.1. Concepts that imply space

It should be clear that the concept of space is common in the WS150 problems.
Below, two examples are given for each of the concepts of interior/exterior
and direction. Together these two concepts imply the need to represent spatial
relationships in some parts of some sentences. The Tumbug icon used to
represent places inside of it is a double frame or a triple frame to suggest a
picture frame. Any objects where space (i.e., interior/exterior or direction)
is important to represent must be placed inside of this frame, which can have
any desired size or length-to-width ratio. Some concepts that inherently imply
space are: (1) interior or exterior, (2) direction, (3) position, (4)
alignment.

The Verbatim Box is where bona fide images are contained, therefore this is
where Tumbug has its best prospects of demonstrating that its KRM is more
powerfully expressive than any other existing KRM. ”Bona fide” images in this
context mean images other than predetermined icons or text. For example, a
textual description such as ”two circles of diameter 1 have their centers
aligned along the 64-degree angle line, and their centers are 2.7 diameters
distance from each other” is a bona fide image that potentially spans an
uncountably infinite number of similar images by merely changing two numerical
values. In this way Tumbug can output an uncountably infinite number of
images. Similarly, in applications where the goal is to depict a default scene
to aid the viewer’s comprehension of that scene, any photo whatsoever may be
inserted verbatim into the Verbatim Box. For example, in a Jonny Quest
animated cartoon, stock real jungle scenery (and even background sounds) could
be used as a background wallpaper within the Verbatim Box to make the cartoon
more realistic. Even a video could be placed in the Verbatim Box, such as a
video of rapidly passing city buildings as background for an animated film
where the characters in the foreground are riding aboard a train in the city.

1.1.1. Interior

This example is from a portion of WS150 question #52. A Description Box is
necessary because the general interior of one object (viz., the fish) is
implied by the word ”ate,” but no exact pictorial depiction of the event was
provided.

”[52] The fish ate the worm. It was tasty. What was tasty? POSSIBLE ANSWERS:
{the worm, the fish}”

The verb ”eat” would be defined in object-and-action terminology as something
like ”Subject eats Object = Subject arranges for Object to go inside Subject’s
mouth, whereupon Subject closes Subject’s mouth to entrap Object, then moves
Object toward Subject’s stomach to be digested.” Since any such definition
involves the concept of ”inside,” which implies the concept ”interior,”
direction is inherent in the sentence meaning, therefore some type of
Descriptive Box should be used, as shown in Figure 58.

Figure 58: [52] Tumbug for ”The fish ate the worm.” (with abstract icons)

In Figure 58, the fish has moved toward the stationary worm in time and has
engulfed the worm into an anatomically interior position such as a mouth or
stomach. This interior structure is represented by the smaller C Object Circle
inside of the larger C Object Circle that represents the fish as a whole. The
eaten worm then is represented as an even smaller C Object Circle inside of
the fish’s interior C Object Circle.

In practice, any juxtaposition that involves ”interior” tends to be so clear
that the surrounding Description Box can optionally be removed from the
diagram without confusion. Figure 59 shows a version of the fish diagram that
removes the Location Box and–even more visually effective–uses realistic icons
instead of C Object Circles.

Figure 59: [52] Tumbug for ”The fish ate the worm.” (with realistic icons, and
without emphasizing space)

The fish diagram with realistic icons is more standard for Tumbug, despite the
intermediate positions in time not being shown. Any other 2D temporal
representation, viz. an icon in discrete time or an icon in continuous time,
will not be clear since the separate time axis in 3D space would not be shown,
which would force a moving object to overlay part of itself in the same plane,
which in total will result in a smeared, lengthy composite image. The most
practical option for simulations is to render real-world time the same as
rendering simulation time, so that an object moving in the real world will be
seen as an object moving in the simulation. For applications that require
careful consideration of all points along a trajectory, such as determining
whether several rotating disks in a plane, each of different radius and speed,
and each with a visible spot near its rim, will ever align, rendering time as
a separate dimension might be preferable.

Ideally Tumbug should be implemented in at least 3D space so that the states
of moving 2D objects can be seen easily and collectively across the entire
time span.

1.1.2. Direction

This example is from WS150 question #11. Some type of Location Box is
necessary because the relative direction of an object is specified via the
words ”down” and ”top.”

”[11] Tom threw his schoolbag down to Ray after he reached the top of the
stairs. Who reached the top of the stairs? POSSIBLE ANSWERS: {Tom, Ray}”

If the phrase about the stairs is omitted for simplicity, then the partial
sentence would be diagrammed in Tumbug as in Figure 60.

Figure 60: [11] Tumbug for ”Tom threw the schoolbag down to Ray.”

The ”0” on the Time Arrow represents ”now,” so the fact that the event
completely finished in the time before time 0 means that the event happened in
the past. The direction of the backpack is straight down because the sentence
says ”down” and no mention or implication was made of the throwing having a
horizontal component; if the schoolbag had also been thrown in a horizontal
direction as well as downward, then the trajectory of the schoolbag would be
shown as angled, say toward the right of the diagram, as it approached Ray at
the bottom, who would also be positioned more toward the right.

Further development anticipated: Note that throwing the schoolbag requires a
force, and gravity would be exhibiting a smaller force even without throwing.
So far Tumbug examples have not attempted to show all applicable forces, the
resulting force vector after adding component forces, or even different length
arrows that are proportional to the different force magnitudes. Ideally such
fine details should be worked out eventually, though in the WS150 examples the
need to differentiate between force magnitudes does not ever appear to arise.

The full sentence would be diagrammed in Tumbug as in Figure 61.

Figure 61: [11] Tumbug for ”Tom threw the schoolbag down to Ray after he
reached the top of the stairs.”

Note that a longer Time Arrow was be used in order to show two events, one of
which happened after the other. Each event–(1) the reaching of the top of the
stairs, and (2) the throwing down of the schoolbag to Ray–is enclosed in its
own C Aggregation Box to make the situation conceptually simpler. Since the
person called ”he” was not identified in the sentence, the diagram does not
yet assign a color to the ”he” C Object Circle , or assign the name on that
”he” C Object Circle to ”Tom.” (That is for CSR to resolve at a later stage of
processing.) The word ”after” would be implied by the relative times of the
two events. As before, relative to the current time ”0,” both events happened
in the past. The word ”top” is understood to mean ”top of the diagram” since
the direction ”down” has already been established as the bottom of the page,
and since ”top” is defined as the opposite direction of ”bottom.”

1.1.3. Position

Figure 62 shows an example from a portion of WS150 question #19. Some type of
Location Box is necessary because the relative position of an object is
specified via the word ”above.”

”[19] The sack of potatoes had been placed above the bag of flour, so it had
to be moved first. What had to be moved first? POSSIBLE ANSWERS: {the sack of
potatoes, the bag of flour}”

Figure 62: [19] Tumbug for ”The sack of potatoes had been placed above the bag
of flour.”

1.1.4. Alignment

Figure 63 shows an example from a portion of WS150 question #34. Some type of
Location Box is necessary because the relative alignment of objects is
specified via the word ”between.”

”[34] There is a pillar between me and the stage, and I can’t see it. What
can’t I see? POSSIBLE ANSWERS: {the stage, the pillar}”

Figure 63: [34] Tumbug for ”There is a pillar between me and the stage.”

The dotted line is technically a 1D Marker, used to help the viewer can
discern that the three objects are aligned.

2\. Descriptive Boxes (further development anticipated)

The Tumbug icon for a Descriptive Box is shown in Figure 64.

Figure 64: A Descriptive Box has one less border than a Verbatim Box to
suggest that a Descriptive Box is less constrained than a Verbatim Box.

Whereas a Verbatim Box shows exact, inflexible locations of parts of images, a
Descriptive Box is more flexible and therefore more general in that it uses
links that describe spatial relationships via text instead of rigidly placed
objects in space. See Figure 65.

Descriptive Boxes of any type may be arranged and nested arbitrarily. For
example, to show the upstairs and downstairs of a house, two such Descriptive
Boxes would logically be stacked. Similarly, to show two regions of a picture
where each region could contain only one type of item, but each item could be
in any location within that region, two such boxes would be placed inside of a
third, larger such Descriptive Box. The same observation holds true of
Aggregation Boxes.

However, this observation about nested boxes does not hold for mixtures of
different types of Location Boxes since placing a less constrained Location
Box inside of a more constrained Location Box logically suggests that the
constraints of the less constrained box must then be made more constrained,
which is probably not what a diagrammer would want. For example, in Figure 65,
the painting inside the Verbatim Box could not logically contain regions of
Descriptive Boxes because then those Descriptive Boxes would presumably fall
under the rules of the encompassing Verbatim Box.

Figure 65: A Verbatim Box has every feature in a fixed location, but a
Descriptive Box has only loosely constrained locations. (Painting: ”Café
Terrace at Night” (Vincent Van Gogh, 1888))

3\. Aggregation Boxes

3.1. C Aggregation Boxes

The preliminary letter ”C” is a formality that stands for ”concrete.”
(”Corporeal” would be a good alternative synonym, if so desired.) Usually C
Aggregation Boxes are called simply ”Aggregation Boxes.”

Tumbug has two types of Aggregation Boxes: C Aggregation Boxes and C-A
Aggregation Boxes. Tumbug’s icon for an C Aggregation Box is shown in Figure
66. C-A Aggregation Boxes are described in the different section.

Location Boxes and Aggregation Boxes may be arranged and nested arbitrarily.
For example, to show the upstairs and downstairs of a house, two such boxes
would logically be stacked. Similarly, to show two regions of a picture where
each region could contain only one type of item, but each item could be in any
location within that region, two such boxes would be placed inside of a third,
larger such box.

Figure 66: Tumbug’s icon for a C Aggregation Box.

Note that the border of any type of Aggregation Box need not be rectangular,
though rectangular is the default shape.

Nearly all representation systems have some means of representing aggregation.
Some examples are:

  1. 1.

In English grammar phrases can be aggregated by parentheses, such as ”The
flower fields of Encinitas were gradually either replaced (by condominiums or
a tourist attraction) or left to fall into neglect.”

  2. 2.

In algebra, terms can be aggregated by parentheses. For example: 2(x + y) -
3(x + y).

  3. 3.

In human societies, residential regions are organized into communities, which
are grouped into cities, which are grouped into states or provinces, which are
grouped into countries, and so on.

In Tumbug a (single-bordered) rectangle called a ”C Aggregation Box” is used
to surround the visual objects that are to be aggregated.

If mathematical sets are being represented in Tumbug, an Aggregation Box would
be used since there is a homomorphism between a parenthesized list of numbers
in any order and a box of objects in any location, as in Figure 67. For an
ordered list, however, a Descriptive Box would probably be used since the
relative order (relative places) of the listed objects must be preserved: an
isomorphism. See Figure 68.

Figure 67: A C Aggregation Box would be used to represent a set since element
location is unimportant. Figure 68: A Descriptive Box would be used to
represent a list since element location is important.

An Aggregation Box may also have attribute-value pairs, and often do. Some
attribute-value information that might be useful to include: the count of the
items in the Aggregation Box (e.g., ”count = 3”), attribute-values that all
the items in the Aggregation Box share (e.g., ”shared attribute-value = ’color
= pink’”), the source of the items (e.g., ”source = Madonna Inn gift shop”),
discounted price if a customer buys everything inside (e.g., ”5”), the name of
the set (e.g., ”things that Fred does routinely”). See Figure 69.

Figure 69: Tumbug for ”All the pink souvenirs.”

The ”shared attribute-value” attribute on any type of Aggregation Box is
currently necessary in Tumbug when representing a conjectured set whose
members (and therefore count) are not known.

3.2. C-A Aggregation Boxes

Tumbug’s icon for a C-A Aggregation Box is shown in Figure 70.

Figure 70: Tumbug’s icon for a C-A Aggregation Box.

The C-A Aggregation Box is one of the Building Blocks where Tumbug becomes the
most profound. ”C-A” stands for ”concrete-abstract,” which is a pair of
concepts at the two extreme ends of the abstraction spectrum. A comparison of
the C Aggregation Box and the C-A Aggregation Box follows:

  * •

Both have a rectangular boundary that surrounds a set of arbitrarily selected
objects.

  * •

Both may have a list of detected attribute-values associated with the
represented set.

  * •

Only a C-A Aggregation Box has a second list of forced attribute-values that
define the set.

  * •

A C-A Aggregation Box typically contains an unknown number of objects, so an
ellipsis is shown inside.

A C-A Aggregation Box is a deeply abstracted form of a mathematical function,
where the input is analogous to an independent variable and the resulting
output is analogous to a dependent variable. As in functions, the primary
interest is usually in observing the effect on the output after inputting a
specific value. For example, if the forced input to a C-A Aggregation Box is
the attribute-value pair ”species = human,” the resulting set would contain
all existing humans, and a useful attribute would be the count of all objects
(= all humans) in this set. In this way a C-A Aggregation Box is similar to a
SQL data base query where query word ”WHERE” specifies the attribute-value
pair (which in a data base is the column and row, respectively), the aggregate
function ”COUNT” is a detected (and outputted) property of the set, and the
query word ”FROM” refers to the set in question. Also, as in math functions
and SQL queries, the mapping may not be invertible, such as when the math
function is y = $x^{2}$ for x = -1 and x = +1, or when the SQL query returns
more than one applicable record when queried with ”WHERE surname = Smith”.
Since the resulting matches are typically large and of an unknown count, an
ellipsis is used inside a C-A Aggregation Box.

There also exists an analogy of C-A Aggregation Boxes with first order
predicate calculus (FOPC). For example, the statement of the classic
syllogism, ”Every man is mortal,” which is represented in FOPC as ”($\forall$
x) (MAN(x) → MORTAL(x))”, which could be read as ”For all x, if x is an
element of the set of humans (MAN), then that implies that x has attribute
mortal (MORTAL),” can be represented in Tumbug with the C-A Aggregation Box
shown in Figure 71. Note that the ”count” attribute of this set (one of the
outputs) has unknown value, and its current magnitude would be measured in the
billions. Note also that the forced input that defines this set is that
”species = human” but that the detected output that describes the resulting
set is ”mortality = mortal,” even though that attribute-value pair was not
mentioned in the input.

Figure 71: The classic ”All men are mortal” syllogism.

C-A Aggregation Boxes are needed for mathematical theorems. For example:

”Theorem: In any graph with at least two nodes, there are at least two nodes
of the same degree.”

Tumbug would illustrate this theorem with a C-A Aggregation Box that contains
only one graph of at least two nodes, as shown in Figure 72. The set
membership requirement could be represented as the textual description ”each
graph contains at least two nodes,” and the resulting implied output could be
represented as the textual description ”each graph has at least two nodes of
the same degree.” The upper, forced attributes of the graph could collectively
be considered a large ”if…” condition, and the lower, resulting attributes of
the graph could collectively be considered a large ”…then…” result. In
constructive theorems that show a concrete example, a visual example would go
inside the Aggregation Box, but not all theorems are constructive, so such a
diagram is a luxury. This theorem happens to be a non-constructive theorem.

Figure 72: Tumbug representation of a graph theory theorem: ”In any graph with
at least two nodes, there are at least two nodes of the same degree.”

####  5.2.2 Swirly Arrays

The components of this composite component: Location Box, Cells, optionally
any number of 0D Markers up to the number of cells.

An array, also called a matrix, is a well-known mathematical data structure
that contains cells in a rectangular structure. If the structure contains only
one row or one column then it is called a vector. Information technology has
extended the array concept so that the rows of the array need not be the same
length (or equivalently, that the embedded ”member arrays” need not be the
same length) though the rows are usually shown aligned on their left-hand
sides. Such arrays are called ”jagged arrays” or ”ragged arrays.” Tumbug
extends the concept of a jagged array even further: In Tumbug a ”swirly array”
is defined as a given spatial arrangement of cells where the cells do not need
not to be contiguous or aligned horizontally or vertically. See Figure 73.
(Many mathematicians, high school students, and science fair project creators
have probably reinvented the concept of swirly arrays many times before, but
if so, such documentation is not well-known and not readily available. The
author hereby gives advance credit to any such prior inventors and their
proposed invention names.) Arrays of any kind in Tumbug context could be
called a ”composite Building Block.”

Figure 73: Examples of different types of array arrangements.

Swirly arrays as used here are treated more like annotated diagrams than
mathematical entities. Using obvious methods, two swirly arrays could easily
be added together, and one swirly array could easily be multiplied by a
scalar, but special functions such as taking a determinant of a swirly array,
or multiplying two swirly arrays together would not be possible in general.
However, such structural flexibility in swirly arrays could also be beneficial