# On Mathematical Ways of Knowing

Gizem Karaali
Dept. of Mathematics, Pomona College
610 North College Avenue, Claremont, CA 91711, USA
<EMAIL_ADDRESS>
Humanistic mathematics is a perspective on mathematics that emphasizes the
ways our species creates, interacts with, and lives through it. I summarized
this idea elsewhere (see [13]) by asserting that mathematics is the way our
species makes sense of this world and that it is inherent in our thinking
machinery; our mathematics in turn is dependent on the way we view our
universe and ourselves. Lakoff and Núñez [17] argue carefully and eloquently
for a mathematics inherently based on human cognition.

Cognition is “the mental action or process of acquiring knowledge and
understanding through thought, experience, and the senses” (see Wikipedia). In
this note I attempt to engage with the construct of mathematical cognition
through the lens of humanistic mathematics.

## three questions

Cognition is essentially about mental processes involving knowledge, knowing,
and understanding; mathematical cognition therefore raises questions about
mathematical knowledge, knowing mathematics, and understanding mathematics.
Thus, I first intend to explore broadly three related questions:

1.

What does it mean to know something mathematical?

2.

How do we come to know a mathematical truth?

3.

What does it mean to understand mathematics?

In what follows, I will not pretend to offer a comprehensive treatment of any
of these questions. But in the very least I intend to open up all three
questions in productive ways, so that all readers intrigued by these questions
will find in the following assertions worth agreeing with and arguing against.

## question 1

The first question is a natural extension of traditional epistemological
investigations into mathematics. Philosophers have tinkered with the knowledge
question for centuries, or rather, millennia, and mathematical knowledge has
often been a part of the equation. Knowledge as justified true belief, a core
tenet of epistemology since the Enlightenment, is where I want to start this
note.111 At least since the 1960s with the publication of Edmund Gettier’s “Is
Justified True Belief Knowledge?” [7], this conventional approach to knowledge
has seen many rebuttals and rephrasals, but for this note, I will mostly
ignore this recent body of work. My interest is most in line with the idea of
mathematical knowledge as being justified true belief.

If (mathematical) knowledge is justified true belief (in mathematical
statements), we have multiple avenues to the first question. Or alternatively
we have two related questions to attend to:

1A.

What does it mean that a mathematical assertion is true?

1B.

What does it mean that a belief in a mathematical assertion is justified?

1A is perhaps on the natural playground of mathematicians. Mathematicians seem
to be concerned quite single-mindedly and profoundly in the truth of their
assertions. One can even suggest that mathematics is nothing if not true. That
is, doing math is making true mathematical assertions.222 Truth is surely not
the only target of mathematicians. It is not even the driving force, according
to William Byers, who writes: “Classifying ideas as true or false is just not
the best way of thinking about them. Ideas may be fecund; they may be deep;
they may be subtle; they may be trivial. These are the kinds of attributes we
should ascribe to ideas.” [3]. As a working mathematician, I agree with Byers,
but this does not mean that I don’t also believe that truth is a prerequisite.
Even when mathematicians work with tentative and even patently false
assertions, they have a broader truth in mind, and are not done until
eventually they can reach that truth. In some sense, therefore, I think that
the truth of a mathematical assertion means that it is a part of mathematics,
this edifice we human mathematicians are building together. Philosophers have
tried to clarify what mathematicians might mean when they say that a
mathematical statement is true. Sitting between mathematics and philosophy,
the logician Alfred Tarski [24] proposed a definition of just what truth might
mean in “the deductive sciences”, which presumably include mathematics. Once
again many have commented on Tarski’s definition of truth. I will not go into
that here but there is indeed much more that can be said along these lines if
one is concerned about question 1A.

If we want to address question 1B about justification, we can, like some,
invoke idealized conceptions of mathematical justification involving formal
systems and proof theory. But most mathematicians agree that belief in the
truth of a mathematical statement is justified once there is a proof of the
statement that experts can agree upon. This is quite in tune with Reuben
Hersh’s various definitions of a “proof” in [12], most notably “The ‘proof’ is
a procedure, an argument, a series of claims, that every qualified expert
understands and accepts.” Though some philosophers reading Hersh might
disagree (see for instance [19]), it is indeed the case that when
mathematicians claim that a statement is true, they mean that there is some
consensus among the relevant experts that the statement is true. And an
argument might have been a proof at a given time and place and afterwards,
with contemporary expertise changing sides, it might become invalid.
Similarly, proposed proofs do not become proofs until verified and validated
by experts. Indeed, one might argue that “it is the provision of […] evidence,
not the endorsement of experts, that makes [a display of symbols, words,
diagrams and such] a proof” [19]. However, nothing needs to change in the
display for an argument to remain an alleged proof until experts deem it is
valid, and then and only then is the rest of the mathematical community
comfortable in feeling justified to believe that the mathematical statement in
consideration has finally been proved. What counts as persuasive evidence is
almost always context-dependent. In the case of law this is obvious; even
Wikipedia knows that there are variations on what counts as proof, what counts
as evidence.333See https://en.wikipedia.org/wiki/Burden_of_proof_(law) for a
selection of legal standards of evidence and proof.  Why do we expect
mathematics to be different?

If we see mathematics as something done by humans, the formalist, proof
theory-based understandings of proof and mathematics remain idealized
approximations at best. It is the human (or sometimes, and begrudgingly,
human-assisted) verification that mathematicians look for in a proof.444 Do we
know if the Four-Color Theorem is true? Yes, we do. Or at least most
mathematicians would concede that the human-assisted computer proof (or
alternatively, the computer-assisted human proof) is enough for us to agree
that it is true. There are still those who want more human proofs of the
result, but the truth of the statement does not need further justification.
And this is definitely context-based, both in terms of space-time coordinates
and the cultural makeup of the audience. As Israel Kleiner quotes in [14],
G.F. Simmons wrote “Mathematical rigor is like clothing: in its style it ought
to suit the occasion, and it diminishes comfort and restricts freedom of
movement if it is either too loose or too tight.” This quote captures well how
our understanding of just what should count as proof is dependent on the
fashions of our times. This aligns with Harel’s perspective [10]: “Mathematics
is a human endeavor, not a predetermined reality. As such, it is the community
of the creators of mathematics who makes decisions about the inclusion of new
discoveries in the existing edifice of mathematics.” Among the decisions left
to the human creators of mathematics are the truth of a mathematical statement
and the validity of its proof.

## question 2

The standard modern answer to question 2 is “by a rigorous proof”. Let us
leave aside the historicity and cultural dependency of this response now, and
its vagueness (what is proof? what is rigor?). I already wrote a bit about all
that above. The reader who is not yet convinced may read [14] for more on
rigor and proof.555A tangentially related read which might add some insight to
this conversation is [23] about the role of rigor and proof in mathematics.
But I want to emphasize here the possible distinction between the doer of the
proof and the believer who believes with justification that the proof is valid
and that the related statement is true. Does the believer need to understand
the proof in order to know the related statement is true? How similar and how
different is this from the calculus student saying that they learned calculus
because they passed the final exam? Let us narrow things down a bit more.
Should the successful calculus student be able to state the Fundamental
Theorem of Calculus? Should they be able to prove it? Should they be able to
replicate the argument in their textbook or the one their instructor put on
the board? Should they be able to provide a convincing argument for its truth?
Alternatively, should they be able to use it in a problem that requires the
result? When do we assume a student has learned, or knows the Fundamental
Theorem of Calculus?

Mathematics education researcher Guershon Harel thinks that these kinds of
pedagogical questions are not independent from the philosopher’s concern about
mathematical knowledge. In particular he proposes a definition of mathematics
which originates from his pedagogical research that might help us with our
endeavor here:

> Mathematics consists of two complementary subsets:
> $\bullet$ The first subset is a collection, or structure, of structures
> consisting of particular axioms, definitions, theorems, proofs, problems,
> and solutions. This subset consists of all the institutionalized ways of
> understanding in mathematics throughout history. It is denoted by WoU.
> $\bullet$ The second subset consists of all the ways of thinking, which are
> characteristics of the mental acts whose products comprise the first set. It
> is denoted by WoT. [10].

Here Harel uses “ways of understanding” and “ways of thinking” as technical
terms. According to him “a way of understanding is a particular cognitive
product of a mental act carried out by an individual”, while “a way of
thinking, on the other hand, is a cognitive characteristic of a mental act”.
Here is how he uses them in context:

> “Any statement a teacher (or a classmate) utters or puts on the board will
> be translated by each individual student into a way of understanding that
> depends on her or his experience and background.”
>
> “The range of ways of understanding a fraction makes the area of fractions a
> powerful elementary mathematics topic?one that can offer young students a
> concrete context to construct desirable?indeed, crucial?ways of thinking,
> such as: mathematical concepts can be understood in different ways,
> mathematical concepts should be understood in different ways, and it is
> advantageous to change ways of understanding of a mathematical concept in
> the process of solving problems.”

Consider the mathematician who reads the statement of the Four-Color Theorem
and discussions about its proof and as a result is convinced of the veracity
of the statement and the sufficiency of the proof. This mathematician, in my
opinion, is not that different from the good student who learned of the
Fundamental Theorem of Calculus from their instructor, can use it in various
scenarios, can even perhaps outline a convincing argument about why it might
be a reasonable thing to assume. In each case I’d say the person knows the
concept and construct in question. They believe the truth of a true statement
and are justified in doing so. They put their trust in a relatively
trustworthy source of expertise. But does the student really know (that is,
understand) the Fundamental Theorem of Calculus? Does the mathematician really
know (that is, understand) the Four-Color Theorem? This naturally brings us to
question 3: What does it mean to understand something in mathematics?

## question 3

Harel’s ways of thinking and ways of understanding are related quite visibly
to question 3. For example, Harel offers a handful of ways of understanding
the concept of fractions:

  1. a.

the part-whole interpretation: $m/n$ (where $m$ and $n$ are positive integers)
means “$m$ out of $n$ objects.”

  2. b.

$m/n$ means “the sum $\tfrac{1}{n}+\cdots+\tfrac{1}{n}$, $m$ times”

  3. c.

“the quantity that results from $m$ units being divided into $n$ equal parts”

  4. d.

“the measure of a segment $m$ inches long in terms of a ruler whose unit is
$n$ inches”

  5. e.

“the solution to the equation $nx=m$”

  6. f.

“the ratio $m:n$; namely, $m$ objects for each $n$ objects.”

Similarly, we can develop a list of ways of understanding for the derivative,
in terms of the limit definition; in terms of slopes of tangent lines; in
terms of linear approximations; and so on. And it seems reasonable to assume
that when we say that a student understands fractions, we mean that they have
mastered an indeterminate (but definitely nonzero) number of ways of
understanding the concept. The fluency with which they can move from one
interpretation to the other can help us if we want to further qualify how much
they understand.

Understanding seems to presuppose knowing but is there anything more to it?
More specifically, understanding a piece of mathematics does presuppose
knowing that piece of mathematics; what we want to know is if it involves
anything more.

This is the perfect context for me to bring up my favorite quote from one of
my mathematical heroes. John von Neumann was a polymath, a genius
mathematician who was instrumental in the development of quantum mechanics,
game theory, functional analysis, operator algebras, and computer science, a
great mathematical mind. This distinguished mathematician is known to have
said to a young scholar asking for advice: “Young man, in mathematics you do
not understand things. You just get used to them.”666 For an interesting
discussion of this quote see
https://math.stackexchange.com/questions/11267/what-are-some-interpretations-
of-von-neumanns-quote.

Grant that this is a sharp quote. It hits you hard and shakes you up,
especially if you have at least a passing knowledge of the extent of von
Neumann’s own mathematical contributions. But can you take it seriously? Can
you get anything out of it in the context of mathematical knowledge and
mathematical ways of knowing?

My personal take on this quote is twofold.

One is that of the optimistic student of mathematics. Even if I feel like I am
not understanding something, there is some benefit to pushing forward, if only
a bit more. Doubtless the great mathematician is right; sometimes you simply
have to move on, after accepting the fact as a fact and see where it leads
you. Stubborn patience. Dogged perseverance.

However perhaps von Neumann did not really mean to recommend moving forward
without understanding. Perhaps he was saying something else, that what you
call understanding is not something subtle or sublime. In fact, when we are
learning a new concept, a new theory, don’t we start by making mental
patterns, charting new pathways in our mind, formatting our minds so certain
types of programs run well or smoothly enough? How is this different
cognitively from getting used to brushing our teeth before going to bed,
splashing our faces before leaving the bathroom, eating with the fork in our
right hand? Habit forming is done by doing something over and over again;
aren’t mathematical ways of thinking and ways of understanding reinforced by
repeated practice as well? And is there a genuinely different, a genuinely
distinct sense of “understanding” that goes beyond “getting used to thinking
of the concept in question in a particularly productive manner”?

Imagine a student who learns to think of a complex number first as an ordered
pair, then as a point on the complex plane, and then as a linear
transformation on the complex plane. When can we say the student knows complex
numbers? When do we say the student understands them? I agree with Emily
Grosholz who argues, using complex numbers as a concrete case study, that “the
best way to teach students mathematics is through a repertoire of modes of
representation, which is also the best way to make mathematical discoveries”
[9]. But this also makes things a lot more complicated. If there are multiple
ways of understanding, a la Harel, that need to go into understanding a
construct, when do we really know the construct? When do we understand it?

Bloom’s taxonomy is just one of many ranking frameworks education researchers
use to delineate cognitive tasks and their demands on a learner. In the
original taxonomy of Bloom and colleagues, knowledge is the very lowest level
of cognition needed and includes knowledge of specifics, terminology, specific
facts, ways and means of dealing with specifics, conventions, trends and
sequences, classifications and categories, criteria, methodology, the
universals and abstractions in a field, principles and generalizations,
theories and structures [2]. Understanding is related to the second level,
comprehension. (In the revised version [1], to understand is once again the
second level, and there are several other layers of cognition ranked higher.)
Mathematics education researchers prefer other frameworks to evaluate the
cognitive demand of mathematical tasks; see [22] for a commonly used schema
distinguishing between memorization, procedures without or with connections,
and “doing mathematics”. It is quite interesting that learning mathematics
involves doing mathematics as a subcategory!

Putting taxonomies aside, knowing and understanding a concept might not be
that different from one another after all. There seems to be a psychological
difference for sure; understanding always occurs with knowing but sometimes we
might feel we know something but do not “really understand”. But perhaps von
Neumann was not just being cheeky. Perhaps there is really nothing more than
getting used to knowing something. Perhaps all we need is a brain formatted in
the right way to accept our knowledge as truth and naturally so.777 The
formatting of the brain might sound strange but is not that far from the point
of education. The point of education is to shape students’ minds. What is that
if not brain formatting? Indoctrination might also fit in this view. Some
might take this to discussions of how mathematics education promotes dogmatic
beliefs. Though I am interested in such inquiry, I will not pursue it here.

## mathematical ways of knowing

In the remainder of this note, I want to delineate a construct I will call
“mathematical ways of knowing”. This is in some sense related to what Harel
calls “ways of thinking”. But I believe it is not exactly the same.

I start by the axiomatic definition that mathematics is one of the main
systematic bodies of knowledge that formulates and occasionally aims to
address questions about human perception of the world and the human endeavors
to understand it. The concepts and constructs of number, shape, form, time,
change, and chance are fundamental to our understanding of our world as well
as ourselves. These concepts and constructs are mathematical in nature, or at
least they are naturally amenable to mathematical approaches.

Within this setting, I mean by a mathematical way of knowing a way of
formulating and addressing a question or a set of questions about our world
and ourselves that allows for mathematical inquiry. It might be interesting to
try and put these mathematical ways of knowing in contrast to or in
conversation with a handful of other ways of knowing: scientific / empirical,
faith-based, philosophical. I do no such thing in this note however. Here I
merely put down some ideas as placeholders, as tentative yet suggestive notes
to a self that might or might not be able to come back to revisit them in a
possible future.

## mathematical ways of knowing: rationalism and imagination

The first two mathematical ways of knowing I would like to consider are
rationalism and imagination.

Rationalism in mathematics is the fundamental assumption that we ought to
reach our mathematical truths through reason. Experiment and experience might
provide hints towards a truth, but they are never enough to convince us in the
final count. Even if we “know something in our guts”, we are not convinced we
have mathematical certainty until we can reason our way to that something.

And reason and rational thinking are captured effectively in axiomatic
thinking. The central tenets of axiomatic thinking are captured by Tarski in
the following:

> When we set out to construct a given discipline, we distinguish, first of
> all, a certain small group of expressions of this discipline that seem to us
> to be immediately understandable; the expressions in this group we call
> primitive terms or undefined terms, and we employ them without explaining
> their meanings. At the same time we adopt the principle: not to employ any
> of the other expressions of the discipline under consideration, unless its
> meaning has first been determined with the help of primitive terms and of
> such expressions of the discipline whose meanings have been explained
> previously. [25]

Thus, we begin with some initial assumptions, ideas, fundamental beliefs, core
values, axioms. We take certain things for granted. We try to make these as
self-evident as possible.888 But of course, self-evidence, just like beauty,
is in the eye of the beholder. There is more that can be said here. And from
there we build our argument step by step, using logic as our guide. We define
new constructs in terms of older, already accepted ones, and thus attempt to
build a new world which has a solid foundation.

Mathematical rationality can be found in the various versions of the
ontological argument for the existence of God, as well as in the Declaration
of Independence of the United States of America (see [8] for a convincing
argument about how mathematical rationality is built into the Declaration as
well as many other illustrative examples of the impact of mathematical ways of
knowing on Western thought).

In the history of intellectual thought Rationalism of the European
Enlightenment was met with a backlash movement, Romanticism. Today we can but
do not have to see these two as directly opposing and mutually exclusive
methodologies, each rejecting and invalidating the other. Alternatively, and I
believe more productively, we might choose to accept that they point to two
distinct ways of knowing, and occasionally certain truths will be more
accessible via one way than another.

Mathematical rationalism also has a similar complement, in what I will call
mathematical imagination, or mathematical romanticism, if you will.
Mathematical imagination is the way we select our axioms, the way we fix our
principles. Mathematical imagination is how we determine our target truths.
Human mathematicians do not start with a random formal axiomatic system and
automatically go through all possible provable truths of the theory determined
by it. Instead they engage with the worlds around them, both real and
imaginary, and detect what is interesting, conjecture what might be productive
to pursue, and then set out. Our human mathematics has a freedom to it and
mathematical imagination captures this freedom.

And freedom, broadly construed, may be viewed in these terms as well. Sándor
Szathmári’s utopian, satiric novel, The Voyage to Kazohinia, might just be the
best (fictional) guide to how mathematics is fundamental to a human society.
Susan Siggelakis [21] describes how the protagonist of the novel learns that
in a society without mathematics, “nothing stable exists with which a human
can connect and find meaning in his/her life”. There is only chaos and
violence. As Edward Frenkel says in [6] “where there is no mathematics, there
is no freedom”.

## mathematical ways of knowing: universals and eclecticisms

Two other mathematical ways of knowing beckon us here: Universals and
eclecticisms. The tendency of the mathematician to generalize is well known.
“Mathematics compares the most diverse phenomena and discovers the secret
analogies that unite them,” wrote Jean Baptiste Joseph Fourier. “The art of
doing mathematics is forgetting about the superfluous information,” says
Hendrik Lenstra. Thus the human mathematician tries to generalize, to abstract
from specific examples, to reach universal statements with “any” and “all”
that capture the essence of what is true about a whole slew of eclectic
examples. The human desire to “see the big picture”, the human tendency to
“find patterns” is precisely what I mean by mathematical universalism.

Accompanying and complementing (again productively) this tendency is the
alternative, what I will call mathematical eclecticism. David Hilbert
describes the complementarity of these two tendencies as follows:

> In mathematics, as in any scientific research, we find two tendencies
> present. On the one hand, the tendency toward abstraction seeks to
> crystallize the logical relations inherent in the maze of material that is
> being studied, and to correlate the material in a systematic and orderly
> manner. On the other hand, the tendency toward intuitive understanding
> fosters a more immediate grasp of the objects one studies, a live rapport
> with them, so to speak, which stresses the concrete meaning of their
> relations.

Thus by mathematical eclecticism I mean the search for that one representative
example on the one hand (“the art of doing mathematics consists in finding
that special case which contains all the germs of generality,” wrote Hilbert),
and the excitement of the weirdness of eclectic cases on the other. In fact a
lot of mathematics concerns itself with concrete examples. Paul Halmos wrote,
“the heart of mathematics consists of concrete examples and concrete
problems.” John B. Conway wrote, “mathematics is a collection of examples; a
theorem is a statement about a collection of examples and the purpose of
proving theorems is to classify and explain the examples. . .”

“We think in generalities, but we live in details,” said Alfred North
Whitehead. In fact, I believe we think in both and we live in both. Once
again, these two mathematical ways of knowing complement one another and help
us live our human lives.

## mathematical ways of knowing: certainty and ambiguity

Alan Lightman says it best [18]: “We are idealists and we are realists. We are
dreamers and we are builders. We are experiencers and we are experimenters. We
long for certainties, yet we ourselves are full of the ambiguities of the Mona
Lisa and the I Ching. We ourselves are a part of the yin-yang of the world.”
Certainty and ambiguity find their way into mathematics and mathematical ways
of knowing in a similarly complementary fashion.

Certainty is a part of most people’s perception of mathematics. Mathematics,
for those people, is made up of questions and answers. Answers are certain
once we know them. Indeed mathematics is perhaps the only certain knowledge we
will ever have. I do not wish to minimize this perspective. I admit that I too
have a romantic attachment to the idea that mathematical knowledge has a
quality of certainty that goes farther than any other type of knowledge. And
it took us a long time to get over this perspective as the unique way to
conceive of mathematical truths and mathematical knowledge.

The loss of certainty in mathematics began more than a century ago [15].
Lakatos was also influential in convincing many who cared to listen of the
fallibility of mathematics [16]. And Byers [3] is perhaps the most detailed
expositor of the role of ambiguity in the work of mathematics today.

The complementarity of these two ways of knowing is rich and, at least to me,
inspiring.

## applications of mathematical ways of knowing

Doing math at school or anywhere else is tied deeply in with our views of
ourselves. This has good and bad aspects of course. We can relate our
mathematical experiences to confidence, resilience, and determination as well
as feelings of inadequacy, resistance, and rebelliousness; and as many can
personally attest, any combination of the six can occur together. There is
much emotion in mathematical engagement: hatred, love, anger, fear, anxiety,
surprise, frustration, anticipation. How we handle mathematical challenges
(suffering alone, valiantly standing defeated or undefeated, finding
commiserators and conspirators) tells us about ourselves. Many of those who
continue to do math after school connect with it at an emotional and personal
level. We find aesthetic stimulation and creative joy in mathematical
activity, as well as terrible frustration and occasional bouts of tedium.

But can mathematical ways of knowing allow us to reach self-knowledge and a
sense of identity mathematically? Andres Sanchez recounts how through an
intentional application of set theory to his own personal life, he was able to
discover his true identity and sexuality [20]. Set theory, more generally,
offers us pathways of thinking about belonging and not belonging. A clever
student of mine, when asked to form study groups, chose to name his team “the
identity element” as he wanted to work alone for the project in question.
Mathematical ways of knowing may come in handy when thinking in terms of
borders and boundaries of nations, communities, and cultural groups. They can
help us think of inequalities and intersectionalities in different ways [4].
They can help us with moral dilemmas [27, 28]. Mathematical ways of thinking
and knowing can indeed allow us to view and understand ourselves in new and
insightful ways.999 And in turn other ways of knowing can help us understand
our mathematics better [11].

## parting words – till next time…

We have come a long way since Ptolemy argued that “mathematics alone yields
knowledge and that, furthermore, it is the only path to the good life” [5].
Mathematical knowledge has lost its certainty somewhere along the way, and
primality in the eye of the public a long time ago. Mathematicians eventually
learned to be humbler about the reaches of mathematical ways of knowing. But
mathematics can still yield powerful knowledge, and not just the kind that can
blow up cities and optimize factory production. I urge us to try and open up
to the world once again. If we dig deeper into mathematical ways of knowing
and the contexts where they might apply, mathematics may yet surprise us.

## References

  * [1] Anderson, Lorin W.; Krathwohl, David R., eds. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Allyn and Bacon.
  * [2] Bloom, B. S.; Engelhart, M. D.; Furst, E. J.; Hill, W. H.; Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company.
  * [3] Byers, W. (2007). How Mathematicians Think: using ambiguity, contradiction, and paradox to create mathematics. Princeton: Princeton University Press.
  * [4] Eugenia Cheng, TEDx talk on how abstract mathematics can help us understand the world, available at https://www.ted.com/talks/dr_eugenia_cheng_how_abstract_mathematics_can_help_us_understand_the_world, last accessed on April 15, 2019.
  * [5] Feke, Jacqueline (2018). Ptolemy’s Philosophy: Mathematics as a Way of Life. Princeton University Press.
  * [6] Frenkel, Edward (2013). Love and Math: The Heart of Hidden Reality. Basic Books.
  * [7] Edmund Gettier, “Is Justified True Belief Knowledge?”, Analysis, Volume 23 Issue 6 (1963), pages 121–123.
  * [8] Grabiner, Judith V. (1988). ”The Centrality of Mathematics in the History of Western Thought,” Mathematics Magazine, Volume 61 Issue 4, 220-230.
  * [9] Grosholz, Emily (2013). “Teaching the Complex Numbers: What History and Philosophy of Mathematics Suggest,” Journal of Humanistic Mathematics, Volume 3 Issue 1, 62-73.
  * [10] Harel, Guershon (2008). “What is Mathematics? A Pedagogical Answer to a Philosophical Question”. In Bonnie Gold and Roger Simons (eds.), Proof and Other Dilemmas: Mathematics and Philosophy. Mathematical Association of America. 265-290.
  * [11] Gutiérrez, Rochelle (2012). “Embracing Nepantla: Rethinking ‘Knowledge’ and its Use in Mathematics Teaching”, REDIMAT - Journal of Research in Mathematics Education, Volume 1 Issue 1, 29-56.
  * [12] Hersh, Reuben (2014). Experiencing Mathematics: What Do We Do, When We Do Mathematics? Providence: American Mathematical Society.
  * [13] Karaali, Gizem (2015). “Can Zombies Do Math?” In Mariana Bockarova, Marcel Danesi, Dragana Martinovic and Rafael Nunez (eds.), Mind in Mathematics: Essays on Mathematical Cognition and Mathematical Method, Interdisciplinary Studies on the Nature of Mathematics #3, Lincom. 140?153.
  * [14] Kleiner, Israel (1991). “Rigor and Proof in Mathematics: A Historical Perspective”, Mathematics Magazine, Volume 64 Issue 5, 291-314.
  * [15] Kline, Morris (1980). Mathematics: The Loss of Certainty. Oxford: Oxford University Press.
  * [16] Lakatos, I. (1976). Proofs and Refutations: the logic of mathematical discovery. New York: Cambridge University Press.
  * [17] Lakoff, G. and Núñez, R. E. (2000). Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being. New York: Basic Books.
  * [18] Lightman, Alan (2018). Searching for Stars on an Island in Maine. Penguin Random House.
  * [19] Pollard, Stephen (2014). “Review of Experiencing Mathematics by Reuben Hersh”, Philosophia Mathematica, Volume 22 Issue 2, 271-274.
  * [20] Sanchez, Andres (2018). ”My Sets and Sexuality,” Journal of Humanistic Mathematics, Volume 8 Issue 1, 359-370. https://scholarship.claremont.edu/jhm/vol8/iss1/18/
  * [21] Siggelakis, Susan J. (2019). “S ndor Szathm ri’s Kazohinia: Mathematics and the Platonic Idea”, Journal of Humanistic Mathematics, Volume 9 Issue 1, 3-23. https://scholarship.claremont.edu/jhm/vol9/iss1/3/
  * [22] Smith, M.S., & Stein, M.K. (1998). “Selecting and creating mathematical tasks: From research to practice.” Mathematics teaching in the middle school, Volume 3 Issue 5, 344-350
  * [23] Terence Tao, “There?s more to mathematics than rigour and proofs”, blog pot available at https://terrytao.wordpress.com/career-advice/theres-more-to-mathematics-than-rigour-and-proofs/, last accessed on April 15, 2019.
  * [24] Tarski, Alfred (1933/1956). “The concept of truth in the languages of the deductive sciences” (Polish), Prace Towarzystwa Naukowego Warszawskiego, Wydzial III Nauk Matematyczno-Fizycznych 34, Warsaw; reprinted in Zygmunt 1995, 13-172; expanded English translation in Tarski 1983 [1956], 152-278.
  * [25] Tarski, Alfred (1946). Introduction to Logic and the Methodology of the Deductive Sciences. Oxford University Press.
  * [26] Tarski, A. and Vaught, R., 1956, “Arithmetical extensions of relational systems”, Compositio Mathematica, Volume 13, 81-102.
  * [27] Sarah Voss, “A Workshop to Introduce Concepts of Moral Math”, Journal of Humanistic Mathematics, Volume 2 Issue 2 (July 2012), pages 114–128. Available at: https://scholarship.claremont.edu/jhm/vol2/iss2/10, last accessed on April 15, 2019.
  * [28] Sarah Voss, “Fuzzy Logic in Health Care Settings: Moral Math for Value-Laden Choices”, Journal of Humanistic Mathematics, Volume 6 Issue 2 (July 2016), pages 161–178. Available at: https://scholarship.claremont.edu/jhm/vol6/iss2/12, last accessed on April 15, 2019.


# A Longitudinal Multi-modal Dataset for Dementia Monitoring and Diagnosis

###### Abstract

Dementia is a family of neurogenerative conditions affecting memory and
cognition in an increasing number of individuals in our globally aging
population. Automated analysis of language, speech and paralinguistic
indicators have been gaining popularity as potential indicators of cognitive
decline. Here we propose a novel longitudinal multi-modal dataset collected
from people with mild dementia and age matched controls over a period of
several months in a natural setting. The multi-modal data consists of spoken
conversations, a subset of which are transcribed, as well as typed and written
thoughts and associated extra-linguistic information such as pen strokes and
keystrokes. We describe the dataset in detail and proceed to focus on a task
using the speech modality. The latter involves distinguishing controls from
people with dementia by exploiting the longitudinal nature of the data. Our
experiments showed significant differences in how the speech varied from
session to session in the control and dementia groups.

Index Terms: dementia detection, cognitive decline monitoring, computational
paralinguistics, longitudinal dataset

##  1 Introduction

Motivation Around 50 million people in the world have dementia, a syndrome
involving deterioration in memory and cognitive abilities, with an annual
increase of 10 million [1]. Earlier diagnosis can improve patients’ quality of
life by enabling better planning and medical interventions at the most
effective and appropriate stage [2, 3, 4]–especially if diagnosis occurs
before clinical cognitive symptoms set in [5, 6].

Standard diagnostic biomarkers such as MRI, PET scans and cerebrospinal fluids
are intrusive and expensive, and therefore unsuitable for early diagnosis.
Cognitive tests such as the ADAS-Cog [7], MMSE [8] and ACE-III [9], widely
used in clinical studies, suffer from sensitivity issues [10] and lag behind
biomarkers in their ability to detect dementia onset [11].

Related Work Automatic analysis of patients’ spontaneous speech and language
is a promising non-invasive, inexpensive approach to screening and monitoring
dementia progression [12, 13]. To allow progress in this field a number of
different datasets have been proposed in earlier work.
DementiaBank Pitt Corpus [14]111http://talkbank.org/DementiaBank/: This is the
most widely used speech dataset for dementia detection where 196 older adults
with dementia and 98 matched controls performed the Cookie Theft picture
description task with an interviewer on an annually basis up to seven times.
The full dataset contains 244 speech samples from 98 control healthy subjects
and 255 speech samples from 168 subjects with Alzheimer’s (AD).
The Carolinas Conversations Collection
[15]222http://carolinaconversations.musc.edu/about/: This is a digital
collection of transcribed audio and video recordings of conversations about
health, supported by the National Libraries of Medicine. It contains a
longitudinal set of 400 conversations by 125 people with dementia.
ADReSS [16]: The Alzheimer’s Dementia Recognition through Spontaneous Speech
(ADReSS) dataset contains speech recordings and transcripts of verbal
descriptions elicited through the Cookie Theft picture from the Boston
Diagnostic Aphasia Exam [17]. Unlike DementiaBank, it has 156 speech samples,
each from a unique subject, where 50% of subjects have AD and 50% are healthy
controls.
$\textup{ADReSS}_{o}$ [18]: Following ADReSS [16], the very recent
$\textup{ADReSS}_{o}$ challenge provides a set of recordings of picture
descriptions without manual transcriptions. In addition to the Cookie Theft
picture descriptions, this dataset also includes speech recordings of AD
patients performing a category fluency task at their baseline visit, one
recording per subject, used for predicting cognitive decline over a two year
period.
Carers’ interactions with patients suffering from severe dementia [19]: Video
recordings (2 hrs each) and transcribed text for 14 people with severe
dementia in conversations with carers.
Most existing datasets focus on a single modality (speech, transcribed speech
or writing), or involve tasks that are well known to be subject to learning
effects [20]. Moreover, most of these studies are limited to data snapshots at
a fixed time, and within particular genres or domains via lab-based tasks,
very often the Cookie Theft picture description task. Some datasets permit
analysis of more general language: e.g. the Carolinas dataset contains less
directed conversations between people with dementia and health practitioners,
and has been used to build models for automatic detection of confusion [21].
Learning models for longitudinal monitoring of individuals is not possible
with the current datasets as either the datasets do not contain longitudinal
information or any longitudinal data are either sparse or inconsistently
available for participating individuals.
Contributions 1) We describe the creation of a dataset of rich longitudinal
multi-modal data which contains spoken and written language as well as
associated paralinguistic information. This was obtained from people with
various forms of dementia, mild cognitive impairment (MCI) and healthy
controls on the basis of reminiscence material and in a non-clinical setting.
2) We provide an analysis on the dataset for the different modalities. 3) We
focus on a task based on the speech modality illustrating the usefulness of
the longitudinal setting in distinguishing differences between people with
dementia and controls.

##  2 Dataset creation

Our goal in creating this dataset was to collect longitudinal data including
both spontaneous speech and writing, and to focus on conversations occurring
in a non-clinical setting. There are three important novel aspects in the
dataset design: I) Conversations and written thoughts as well as associated
paralinguistic information are obtained on the basis of _reminiscence_
material, specifically images from past decades on topics of general interest.
Reminiscence is a meaningful and useful activity for people with and without
dementia that can improve cognition, mood, and quality of life [22, 23]. II)
The dataset is designed to contain daily data over three extended periods
(phases) each spanning around 4 weeks or 28 sessions. III) The data is
collected in the participants’ own environment using a custom-built tablet
application.

###  2.1 Data collection process

According to our protocol data is collected in three 4-week phases spreading
over 54 weeks in total, each phase therefore being separated by approximately
14 weeks. In practice due to unforeseen circumstances the period between
phases has been longer than 14 weeks. Participants are paired with a carer and
asked to record daily sessions during each phase, alternating between (a) 15
mins of conversation with their carers, and (b) typed or hand-written
thoughts.

Both spoken and written language are elicited using reminiscence material-
images from the past-created by dementia communication specialists. Images are
presented using a bespoke Android tablet application, which records spoken and
written data and sends it to a secure remote server for storage. The
application is designed to record three modalities: _speech_ , _typed text_ ,
_hand written text_. For the latter two paralinguistic information such as key
strokes and pen strokes, pen pressure and deletions are recorded respectively.
The application was developed, implemented and tested together with a
commercial clinical partner, Clinvivo, in consultation with a focus group from
the Alzheimer’s Society.

At the start of each 4-week phase, participants are given a tablet running the
reminiscence tablet application. Participants are asked to complete a
cognitive assessment test at the start and end of each phase.

A Mini Mental State Examination (MMSE) [8] and an Addenbrooke’s Cognitive
Examination-III (ACE-III) [9] are administered as cognitive impairment
benchmarks by a suitably qualified person. The ACE-III is conducted at the
time of consent and at the beginning of each of the 4-week phases. The MMSE is
conducted at the end of each 4-week phase. While these tests provide only
minimal cognitive data, they allow us to assess cognitive change at the
comparison points needed to analyse the rich linguistic information collected.
Indeed challenges such as ADReSS [16] and $\textup{ADReSS}_{o}$ [18] include a
regression task to predict cognitive scores.

###  2.2 Reminiscence Material & Tablet application

The tablet application shows a participant four images every day, each
representing a topic of general interest from the 50’s, 60’s and 70’s. Each of
the images/topics is accompanied by a set of three questions to help initiate
a conversation or thought process and provide memory ”joggers”. The
participant chooses one topic out of the four each day as well as whether they
would like to record a conversation, type or write their thoughts. The four
images are pseudo-randomly selected from a pool of images available in each
4-week phase. A screenshot of the tablet application after a participant has
chosen an image can be seen in Figure 1.

Figure 1: Screenshot of Tablet application once topic is chosen

The image material and corresponding questions were developed by an
organisation specialising in dementia
communication333https://realcommunicationworks.com/ and have been used with
people in care homes. The 50’s and 60’s material was adapted for use in the
tablet application. The 70’s material was created for the purpose of the
study. The current dataset contains data from phases 1 and 2 of the study,
which covers a set of 67 images/topics. Phase 1 includes 26 topics from the
50’s and Phase 2 includes 41 images from the 60’s and 70’s. By design topics
were meant to be repeated every so often but based on individual’s feedback
phases 2 and 3 were augmented with more images from the 70’s, which also seem
to better fit the memory bump of our participant cohort.

###  2.3 Participant Recruitment

Our target is to recruit a cohort of people living with dementia or MCI (n=20)
and age-matched controls (n=10). These numbers are appropriate for a pilot
study. Previous longitudinal research examining language indicators in people
with Alzheimer’s disease over a year Forbes et al [24, 25] have recruited
similar numbers of participants, with assessments every six months.
Recruitment is still ongoing due to unforeseen circumstances (COVID-19
pandemic). At the current state of development the dataset includes data from
22 people, 14 people with dementia or MCI and 8 age matched controls who have
participated in either phase 1 or 1 and 2.

Eligibility: Participants are aged 65-80 years at the time of consent. They
either have a mild to moderate dementia diagnosis, MCI, or are aged matched
healthy controls. They are resident in their own home or with their family and
are in daily contact with a carer or family member. They need to have lived in
the UK during the 50’s, 60’s or 70’s so they can relate to the reminiscence
material. They need to be able to conduct daily conversations and write their
thoughts using the provided tablet application.

Recruitment: Primary means of recruiting participants has been through the
Join Dementia Research platform (JDR), mailing lists, dementia networks,
dementia cafes and memory clinics. Interested candidate participants are
subsequently screened by qualiﬁed staff (NIHR research nurses) in terms of
providing consent and meeting the study eligibility criteria.

###  2.4 Transcription

A subset of the spoken data (51 sessions from 8 participants) were transcribed
manually by experienced dialogue transcribers, using
PRAAT444https://www.fon.hum.uva.nl/praat/. As well as the words spoken,
transcripts include significant non-verbal events such utterance timings,
pauses, laughter, crying, sighing, yawning, whispering, coughing as well as
disfluencies including mis-speaking and reformulation. The transcription
convention used was developed on the basis of the CHAT protocol [26] and
techniques for transcribers provided in [27]

##  3 Dataset Analysis

Table 1 summarizes the overall statistics of the dataset across all
modalities. So far we have 101:26 hours of audio data (out of an expected
total final c.630 hours).

Modality | Speakers | No Sessions | No Topics | Duration (H)
---|---|---|---|---
Speech | 22 (14,8) | 490 (250,240) | 63 (48,60) | 101:15(50:49,50:36)
Transcripts | 8 (6,2) | 51 (79,5) | 26 (26,5) | 19:04 (18:26,00:38)
Typed/Key | 17 (11,6) | 271(140,131) | 65 (54,58) | -
Hand-written/Pen | 12 (7,5) | 104 (59,45) | 46 (36,34) | -
Overall | 22 (14/8) | 816(408,408) | 66 (63,65) | -

Table 1: Overall statistics of the dataset across all modalities. Numbers in
parenthesis correspond to dementia and control populations, respectively.

There are (so far) 22 subjects and 816 sessions. For the speech modality there
are 22 speakers and 490 sessions. For the Typed/Key modality, there are 17
users and 271 sessions, and for the Hand-written/Pen modality, 12 users and
104 sessions. In each session, speakers addressed 2.5 topics on average
(STD=2.6). In total, they have covered 66/67 topics. In the dementia group,
speakers covered 63/67 topics, whilst in the control group, 65/67 topics. The
mean duration of speech sessions is 12:11 mins (STD=4:31) in the dementia
group and 12:39 mins (STD=4:07) in the control group. Figure 2 shows the
distribution of topics in dementia and control groups for the speech modality.
The majority of sessions fall in the bin $[15,16)$. In total, the overall
duration of speech sessions is 50:49 hours for the dementia and 50:36 hours
for the control groups. For subjects in the dementia group, the mean number of
sessions is 29.1 (STD=11.4), whilst in the control group, 51.1 (STD = 13.1).
In general, subjects in the control group cover more topics than the dementia
ones. However, in terms of speech modality, the overall speech duration of
topics across the two groups is balanced (See Figure 3). Figure 4 summarizes
the population in topics and the corresponding duration by speaker for both
groups in speech modality. The speech dataset is unbalanced in terms of topics
and session duration.

Figure 2: Distribution of topics by duration in dementia and control groups
for the speech modality. Figure 3: Summary of topics duration in two groups.
Figure 4: Summary of topics duration by speaker.

From now onwards we focus particularly on the analysis of the speech and
transcribed data. Part of the speech dataset has been transcribed manually. We
hope to use this for a preliminary linguistic analysis and to prepare for
automatic transcription of the rest of the speech files. In the transcribed
dataset, there are 8 speakers (6 with dementia and 2 controls, from phase 1 of
the study). Most of the transcribed speech sessions are by dementia
participants. In particular 79/84 manually transcribed sessions come from the
dementia group and 5/84 from the control group5.

For the manually transcribed sessions, we analyse characterisations of speech
patterns of people with dementia. To this end, we conduct a frequency-level
analysis across pauses and occurrences of um in the transcribed sessions. To
alleviate the sessions’ length imbalance, we normalise the frequency of
disfluencies by the sessions’ corresponding dictionary length. We encode
pauses in the input sequence by grouping them into three bins: short (under
0.5 seconds); medium (0.5-2 seconds); and long (over 2 seconds).

Figure 5: Mean frequency of pauses of people with dementia.

Figure 5 shows the mean frequency of pauses of people with dementia. Even
though the duration of sessions for the 2nd, 3rd and 5th speakers is similar,
i.e., 5:53, 5:40, and 5:52 minutes, respectively, the 3rd and 5th speakers
make significantly more pauses than the 2nd one.

Figure 6: Mean frequency of Um of people with dementia.

Figure 6 shows the mean frequency of um for the same people with dementia.
Similarly, for some subjects, the use of um is much more frequent than other
subjects.

##  4 Task based on the Speech Modality

We subsequently show how the dataset can be used to identify differences
between people with dementia and controls by exploiting the rich longitudinal
information we have about them. We focus on the speech modality and first
extract a representation of each individual per session from their speech. The
goal is to identify speech variation in speakers $Sp_{i}$ across $T$
consecutive sessions
$S_{p_{i}}=\\{S_{{p_{i}},{1}},S_{{p_{i}},{2}},...,S_{{p_{i}},{T}}\\}$. We are
particularly interested in examining the longitudinal aspect of a subject’s
speech over consecutive sessions and how this differs across the two classes.

##  5 Experiments with the Speech modality

###  5.1 Feature Extraction

To obtain the representation of an individual’s session $S_{i}$ from their
speech we exploit a transfer learning approach, which allows us to utilise
speech representation learned in a self-supervised manner on large-scale
general-domain speech data. We begin by segmenting each recording session
$S_{i}$ into $n$ speech segments $S(i,n)$, using an end-to-end voice activity
detection (VAD) model [28] trained on the DIHARD Speech Diarisation data [29].
For each segment, we then extract embeddings from TRILL (TRIpLet Loss network)
[30], which is a speech encoder pretrained on AudioSet [31] and demonstrated
good performance in a range of speech tasks including AD classification on
DementiaBank. We aggregate the TRILL embeddings $S(i,n)^{T\times D}$ for each
segment, where $T,D\in\mathbb{R}$, over time $T$ using average pooling,
following [30]. Finally, we average the embeddings across all segments of each
session $S(i,n)^{D}$ to obtain session-based representation $S_{i}^{D}$.

###  5.2 Approach

Given a series of consecutive sessions
$[S_{0}^{D},S_{1}^{D},...,S_{T-1}^{D}]$, where $D$ is the dimensionality of
the averaged session embeddings and $T$ is the overall number of sessions for
a particular subject, our goal is to measure how the speech varies across time
within speakers. We hypothesize that this change, measured by a distance
metric $d$, will be indicative of the subject being a case of ‘dementia’ or
‘healthy control’. Therefore, we consider two approaches, where we calculate
$\forall t\in[1,...,T]$: (a) $d_{t}^{f}=d(S_{0}^{D},S_{t}^{D})$ and (b)
$d_{t}^{c}=d(S_{t-1}^{D},S_{t}^{D})$. These measure the distance of a given
session from (a) the first session ( non-consecutive task) and (b) across
consecutive sessions (consecutive task). Without loss of generality, we employ
the cosine distance as our metric $d$. Finally, we construct time series based
on $d_{t}^{f}$ and $d_{t}^{c}$ and consider different statistics (i.e., mean,
median, stdev) as our variables. We investigate the relationship between these
variables – which essentially capture the traces of a subject’s speech
variation across time (sessions) – and our two classes.

###  5.3 Results

We first calculated the mean, median, and stdev distance of consecutive and
non-consecutive sessions for each speaker in both groups. We then averaged the
scores of speakers for the two groups. The results are shown in Table 2. For
the dementia group, we observed that the distance scores are higher for non-
consecutive sessions as compared to consecutive sessions. In particular, the
median is almost double for the non-consecutive task (Wilcoxon rank-sum test
$<.005$), whilst stdev score is the same for both tasks. This may imply that
dementia speakers exhibit changes in linguistic ability between the first
session and subsequent ones. By contrast, for controls the distance scores do
not change substantially in the two tasks, despite variation in topics being
addressed. A comparison between dementia and control groups show that median
scores are considerably higher in the dementia group. Overall results show
that distance scores are higher in the dementia group compared to controls. In
the future, a detailed investigation is needed to clarify if the differences
are due to an increase in speech disfluencies or other factors.

| Non-consecutive | Consecutive
---|---|---
Group | Mean | Median | Stdev | Mean | Median | Stdev
Dementia | 0.16 | 0.15 | 0.03 | 0.02 | 0.08 | 0.03
Control | 0.12 | 0.05 | 0.20 | 0.17 | 0.03 | 0.27

Table 2: Averaged mean, median, and stdev distances across people in two
groups for non-consecutive and consecutive tasks.

##  6 Conclusion

We have presented a novel multi-modal fine-grained longitudinal dataset from
people with dementia and age-matched controls obtained in a natural setting
using a tablet application with reminiscence material. We have explored the
usefulness of the fine-grained longitudinal aspect of the dataset with a novel
task comparing the change in individuals’ speech from session to session
showing significant difference in the change in speech between the control and
dementia groups. This is promising and opens up new directions and
possibilities for exploring this task further and with more elaborate models.

## References

  * [1] W. H. Organization, _Risk reduction of cognitive decline and dementia: WHO guidelines_. Geneva: World Health Organization, 2019.
  * [2] M. Prince, M. Knapp, M. Guerchet, P. McCrone, M. Prina, A. Comas-Herrera, R. Wittenberg, B. Adelaja, B. Hu, D. King, A. Rehill, and D. Salimkumar, “Dementia uk: Update,” 2014, alzheimer’s Society. [Online]. Available: https://www.alzheimers.org.uk/download/downloads/id/2323/dementia_uk_update.pdf
  * [3] All-Party Parliamentary Group on Dementia, “Dementia 2016 report,” 2016, alzheimer’s Society. [Online]. Available: https://www.alzheimers.org.uk/download/downloads/id/3008/appg_on_dementia_2016_report.pdf
  * [4] M. Prince, R. Bryce, and C. Ferri, “World alzheimer report 2011: The benefits of early diagnosis and intervention,” Sep. 2011.
  * [5] M. Mortamais, J. A. Ash, J. Harrison, J. Kaye, J. Kramer, C. Randolph, C. Pose, B. Albala, M. Ropacki, C. W. Ritchie, and K. Ritchie, “Detecting cognitive changes in preclinical Alzheimer’s disease: A review of its feasibility,” _Alzheimer’s & Dementia: The Journal of the Alzheimer’s Association_, vol. 13, no. 4, pp. 468–492, Apr. 2017.
  * [6] K. Ritchie, I. Carrière, C. Berr, H. Amieva, J.-F. Dartigues, M.-L. Ancelin, and C. W. Ritchie, “The clinical picture of Alzheimer’s disease in the decade before diagnosis: clinical and biomarker trajectories,” _The Journal of Clinical Psychiatry_ , vol. 77, no. 3, pp. e305–311, Mar. 2016.
  * [7] W. G. Rosen, R. C. Mohs, and K. L. Davis, “A new rating scale for Alzheimer’s disease,” _American Journal of Psychiatry_ , vol. 141, no. 11, pp. 1356–1364, 1984.
  * [8] M. F. Folstein, S. E. Folstein, and P. R. McHugh, “Mini-mental status. a practical method for grading the cognitive state of patients for the clinician,” _Journal of Psychiatric Research_ , vol. 12, no. 3, pp. 189–198, 1975.
  * [9] P. Noone, “Addenbrooke’s cognitive examination-III,” _Occupational Medicine_ , vol. 65, pp. 418–420, 2015.
  * [10] L. S. Schneider and M. Sano, “Current Alzheimer’s disease clinical trials: methods and placebo outcomes,” _Alzheimers and Dementia_ , vol. 5, pp. 388–397, 2009.
  * [11] C. R. Jack, D. S. Knopman, W. J. Jagust, R. C. Petersen, M. W. Weiner, P. S. Aisen, L. M. Shaw, P. Vemuri, H. J. Wiste, S. D. Weigand, T. G. Lesnick, V. S. Pankratz, M. C. Donohue, and J. Q. Trojanowski, “Tracking pathophysiological processes in Alzheimer’s disease: an updated hypothetical model of dynamic biomarkers,” _The Lancet: Neurology_ , vol. 12, no. 2, pp. 207–216, Feb. 2013.
  * [12] K. C. Fraser, J. A. Meltzer, and F. Rudzicz, “Linguistic features identify Alzheimer’s disease in narrative speech,” _Journal of Alzheimer’s Disease_ , vol. 49, no. 2, pp. 407–422, 2016.
  * [13] A. König, N. Linz, J. Tröger, M. Wolters, J. Alexandersson, and P. Robert, “Fully automatic speech-based analysis of the semantic verbal fluency task,” _Dementia and Geriatric Cognitive Disorders_ , vol. 45, no. 3-4, pp. 198–209, 6 2018.
  * [14] J. T. Becker, F. Boiler, O. L. Lopez, J. Saxton, and K. L. McGonigle, “The natural history of alzheimer’s disease: description of study cohort and accuracy of diagnosis,” _Archives of Neurology_ , vol. 51, no. 6, pp. 585–594, 1994.
  * [15] C. Pope and B. H. Davis, “Finding a balance: The carolinas conversation collection.” _Corpus Linguistics & Linguistic Theory_, vol. 7, no. 1, 2011\.
  * [16] S. Luz, F. Haider, S. de la Fuente, D. Fromm, and B. MacWhinney, “Alzheimer’s dementia recognition through spontaneous speech: The adress challenge,” _Proc. Interspeech 2020_ , pp. 2172–2176, 2020.
  * [17] H. Goodglass, E. Kaplan, and S. Weintraub, _BDAE: The Boston Diagnostic Aphasia Examination_. Lippincott Williams & Wilkins Philadelphia, PA, 2001.
  * [18] S. Luz, F. Haider, S. de la Fuente, D. Fromm, and B. MacWhinney, “Detecting cognitive decline using speech only: The adresso challenge,” _medRxiv_ , 2021\.
  * [19] G. Hansebo and M. Kihlgren, “Carers’ interactions with patients suffering from severe dementia: a difficult balance to facilitate mutual togetherness.” _Journal of Clinical Nursing_ , vol. 11, no. 2, pp. 225–236, 2002.
  * [20] T. E. Goldberg, P. D. Harvey, K. A. Wesnes, P. J. Snyder, and L. S. Schneider, “Practice effects due to serial cognitive assessment: Implications for preclinical Alzheimer’s disease randomized controlled trials,” _Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring_, vol. 1, no. 1, pp. 103–111, Mar. 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S2352872915000068
  * [21] H. Chinaei, L. C. Currie, A. Danks, H. Lin, T. Mehta, and F. Rudzicz, “Identifying and avoiding confusion in dialogue with people with alzheimer’s disease,” _Computational Linguistics_ , vol. 43, no. 2, pp. 377–406, 2017\.
  * [22] M. Pinquart and S. Forstmeier, “Effects of reminiscence interventions on psychosocial outcomes: A meta-analysis,” _Aging & mental health_, vol. 16, no. 5, pp. 541–558, 2012.
  * [23] J. Gonzalez, T. Mayordomo, M. Torres, A. Sales, and J. C. Meléndez, “Reminiscence and dementia: a therapeutic intervention,” _International Psychogeriatrics_ , vol. 27, no. 10, pp. 1731–1737, Oct. 2015.
  * [24] K. Forbes-McKay, M. F. Shanks, and A. Venneri, “Profiling spontaneous speech decline in Alzheimer’s disease: a longitudinal study,” _Acta Neuropsychiatrica_ , 2013.
  * [25] ——, “Charting the decline in spontaneous writing in Alzheimer’s disease: a longitudinal study,” _Acta Neuropsychiatrica_ , 2014.
  * [26] B. MacWhinney, _The CHILDES Project: Tools for analyzing talk. transcription format and programs_. Psychology Press, 2000, vol. 1.
  * [27] P. Garrard, A.-M. Haigh, and C. de Jager, “Techniques for transcribers: assessing and improving consistency in transcripts of spoken language,” _Literary and Linguistic Computing_ , vol. 26, no. 4, pp. 389–405, 05 2011\. [Online]. Available: https://doi.org/10.1093/llc/fqr018
  * [28] M. Lavechin, M.-P. Gill, R. Bousbib, H. Bredin, and L. P. Garcia-Perera, “End-to-end domain-adversarial voice activity detection,” _Proc. Interspeech 2020_ , pp. 3685–3689, 2020.
  * [29] N. Ryant, K. Church, C. Cieri, A. Cristia, J. Du, S. Ganapathy, and M. Liberman, “The second dihard diarization challenge: Dataset, task, and baselines,” _Proc. Interspeech 2019_ , pp. 978–982, 2019.
  * [30] J. Shor, A. Jansen, R. Maor, O. Lang, O. Tuval, F. de Chaumont Quitry, M. Tagliasacchi, I. Shavitt, D. Emanuel, and Y. Haviv, “Towards learning a universal non-semantic representation of speech,” _Proc. Interspeech 2020_ , pp. 140–144, 2020.
  * [31] J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in _2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2017, pp. 776–780.