Alon Vinnikov, Amir Ivry, Aviv Hurvitz, Igor Abramovski, Sharon Koubi, Ilya
Gurvich, Shai Pe`er, Xiong Xiao, Benjamin Martinez Elizalde, Naoyuki Kanda,
Xiaofei Wang, Shalev Shaer, Stav Yagev, Yossi Asher, Sunit Sivasankaran, Yifan
Gong, Min Tang, Huaming Wang, Eyal Krupka

# NOTSOFAR-1 Challenge:
New Datasets, Baseline, and Tasks for Distant Meeting Transcription

###### Abstract

We introduce the first Natural Office Talkers in Settings of Far-field Audio
Recordings (``NOTSOFAR-1'')
Challenge111https://www.chimechallenge.org/current/task2/index. alongside
datasets and baseline system222A GitHub link will be shared by Feb. 1st.. The
challenge focuses on distant speaker diarization and automatic speech
recognition (DASR) in far-field meeting scenarios, with single-channel and
known-geometry multi-channel tracks, and serves as a launch platform for two
new datasets: First, a benchmarking dataset of 315 meetings, averaging 6
minutes each, capturing a broad spectrum of real-world acoustic conditions and
conversational dynamics. It is recorded across 30 conference rooms, featuring
4-8 attendees and a total of 35 unique speakers. Second, a 1000-hour simulated
training dataset, synthesized with enhanced authenticity for real-world
generalization, incorporating 15,000 real acoustic transfer functions. The
tasks focus on single-device DASR, where multi-channel devices always share
the same known geometry. This is aligned with common setups in actual
conference rooms, and avoids technical complexities associated with multi-
device tasks. It also allows for the development of geometry-specific
solutions. The NOTSOFAR-1 Challenge aims to advance research in the field of
distant conversational speech recognition, providing key resources to unlock
the potential of data-driven methods, which we believe are currently
constrained by the absence of comprehensive high-quality training and
benchmarking datasets.

Index Terms: Speech recognition, speaker diarization, speech separation, far-
field, close-talk annotation, NOTSOFAR.

##  1 Introduction

Conversational speech recognition remains a formidable challenge, particularly
in meeting scenarios where distant array devices are commonly employed for
capturing speech. The distance between speakers and the microphones brings
about substantial acoustic complexities including reverberation, variations in
speaker distances and volumes, and background noises. These complexities are
further amplified by the ever-changing positions of speakers and environmental
conditions, altering the acoustic transfer functions (ATFs) [1] between
speakers and microphones. Compounding these acoustic challenges are the
dynamic aspects of conversations, which include speech overlap, interruptions,
rapid changes in speakers, and non-verbal vocalizations [2].

With the emergence of Large Language Models (LLMs), conversational speech
recognition has been infused with unprecedented practical importance. LLMs fed
by speech recognition engines are redefining user-experience with capabilities
ranging from meeting summary, note taking and sentiment analysis, to
personalized and context-aware responses to user queries. The foundation of
these innovative features, particularly in meeting scenarios, lies in distant
speaker diarization and automatic speech recognition (DASR) [3].

The field of conversational speech recognition has seen significant
advancements thanks to numerous datasets such as AMI [4], ICSI [5], Sheffield
Wargames [6], ASpiRE [7], DIRHA [8], VOiCES [9], DiPCo [10], Ego4D [11],
AliMeeting [12], LibriCSS [13], and challenges such as CHiME [14, 15, 16, 17,
18, 19, 20]. However, they suffer from various limitations, particularly with
respect to advancing DASR. The three most-similar corpora to our work,
targeting general meeting scenarios with unsegmented recordings from a single
device, are LibriCSS, AMI, and AliMeetings.

LibriCSS makes a significant step forward by including unsegmented semi-real
meetings and proposing to measure word error rate (WER) directly rather than
signal-based metrics. They also provide matching fully simulated training data
for speech separation. However, since their evaluation data has been obtained
by playing LibriSpeech utterances via loudspeakers, it does not account for
natural conversational dynamics or real-world acoustic complexities, such as
time-varying ATFs. AMI makes an impressive contribution collecting 100 hours
of real meeting recordings with close-talk annotations. However, its capacity
to fully represent real-world complexities is hampered by several limitations.
The recordings, confined to just three rooms and no more than four attendees,
offer limited acoustic variety. Additionally, the development and evaluation
sets consist of only 18 and 16 sessions, respectively. Such a small sample
size introduces uncertainty in predicting real-world performance. For
instance, a severe diarization error in just one session could significantly
impact the overall score. AliMeetings collects a substantial number of real
meeting recordings involving a large number of unique speakers. Yet, its focus
on Mandarin recordings shifts the challenge more towards language and
automatic speech recognition (ASR) aspects, rather than far-field acoustics
and conversational dynamics. Furthermore, their test set consists of only 20
sessions, facing the same small-sample issue as AMI. Notably, neither AMI nor
AliMeetings include a matching simulated training set, which hinders the
application of data-driven front-ends.

Addressing these gaps and several others, our main contributions are as
follows:

  * •

A meeting dataset for benchmarking and training: Consisting of roughly 315
distinct meetings and carefully designed for benchmarking purposes. Meetings
were recorded across a variety of rooms, while capturing a wide spectrum of
real-world scenarios, covering both typical and rare ones, with up to 8
attendees per meeting. Advancing beyond past practices, we emphasize
transcription accuracy with a multi-judge annotation process and avoid the use
of machine-aid to mitigate related biases. We also include detailed metadata
with each meeting for deep-dive analysis, documenting acoustic events and
conversational aspects such as the presence of extra overlap. To the best of
our knowledge, this dataset is the first to offer this kind of annotation.
Furthermore, in our recordings we utilize both single-channel and multi-
channel devices, reflecting the range of typical commercial recording
equipment.

  * •

A matching simulated training dataset: This approximately 1000-hour simulated
training set, intended for speech separation and enhancement, improves the
authenticity of the simulation process to facilitate the application of data-
driven methods. It incorporates 15,000 real ATFs recorded with a geometry
matching the multi-channel devices from the real meeting dataset, clean speech
is carefully filtered for quality, and the mixing process is designed to
exhibit real-world patterns. Additionally, the supervision signals distinguish
between the direct and early reflections, and reverb components of an ATF,
offering more opportunity for exploration.

  * •

An open-source baseline system: To assist participants in getting started, we
provide a fully functional baseline system, complete with inference, training,
data handling, and evaluation code. The system is written entirely in Python
and consists of continuous speech separation (CSS), ASR, and diarization
modules [21, 22].

  * •

NOTSOFAR-1 Challenge: We are launching the NOTSOFAR-1 Challenge to pursue
fundamental questions in the field. It features single-channel and known-
geometry multi-channel tracks, centered on single-device DASR.

The rest of this paper is organized as follows: Sections 2 and 3 discuss the
challenge tracks, metrics, and scientific goals. Section 4 explores the
benchmarking and training datasets and their design considerations in detail.
Section 5 presents the baseline system. In Section 6, we conclude.

##  2 Tracks and Metrics

The challenge features two main tracks centered on the use of a single device:
the “single-channel track” and the “known-geometry multi-channel track”, with
participants able to submit entries to either one or both. The use of a single
device and fixed, known geometry in the multi-channel track are key
distinctions from CHiME-8’s DASR
challenge333https://www.chimechallenge.org/current/task1/index. Participants
receive unsegmented recordings from each meeting and are asked to generate
time-marked, speaker-attributed transcripts. In line with the notations in
[20], each meeting is annotated with a set of segments, denoted as
$r_{i}=\left(\Delta,s,\mathbf{v}\right)$, where
$\Delta=\left(t_{\mathrm{s}},t_{\mathrm{e}}\right)$ are start and end times,
$s\in\mathbb{Z}^{+}$ is the speaker label, and $v\in\Sigma$ is the transcript
for the segment, with vocabulary $\Sigma$. A meeting’s reference is fully
defined by the $N$ reference segments $R=\\{r_{1},…,r_{N}\\}$. The
participants’ task is to estimate this reference through a generated
hypothesis, denoted by $H=\\{h_{1},…,h_{\hat{N}}\\}$. Note that the estimated
speaker labels are not required to be identical to the reference labels.

In each track, systems will be ranked according to two metrics: (1) a speaker-
attributed metric to evaluate the impact of both speaker diarization errors
and word errors, and (2) a speaker-agnostic metric to evaluate word errors.
For the first metric we align with CHiME-8's DASR and choose the time-
constrained minimum-permutation word-error rate (tcpWER) [23]. This is an
enhanced version of the concatenated minimum-permutation word-error rate
(cpWER) [24] that incorporates temporal information to prevent matching words
that are temporally distant, which better reflects temporal accuracy and
speeds up computation compared to cpWER. However, speaker attributed metrics
such as tcpWER are highly sensitive to speaker diarization errors: an
incorrectly attributed segment will be penalized even if the words within it
are accurately recognized. Thus, we include a second speaker-agnostic metric
to isolate word error impact, highlighting systems that excel in this aspect.
Additionally, we aim to promote the development of innovative, practical
systems rather than performance-squeezing approaches that are more brute-force
in nature. We encourage participants to submit their system's runtime and
hardware specifications. Systems deemed practical and efficient will be
featured on a dedicated leaderboard.

##  3 Scientific Goals

We aim to address fundamental questions in the field of distant conversational
speech recognition:

  * •

How much of an advantage does a multi-channel and geometry-specific algorithm
have over a single-channel algorithm?

  * •

How much of an advantage, if any, does a geometry-specific algorithm have over
a geometry-agnostic algorithm?444We work closely with CHiME-8’s DASR challenge
that offers a geometry-agnostic multi-channel track:
www.chimechallenge.org/current/task1/index. Every geometry-agnostic system
submitted to their track will automatically yield a result in our geometry-
specific track on the NOTSOFAR meeting dataset.

  * •

Can the introduced simulated training dataset lead to data-driven front-end
solutions that generalize well to realistic acoustic scenarios?

  * •

How can the various available supervision signals for training be leveraged to
improve algorithms? Namely, the separated speech components within the
simulated training dataset, along with the close-talk recordings,
transcriptions, and speaker labels found in the real meeting dataset.

  * •

Can the data verticals analysis enabled by the metadata reveal potential
avenues for progress?

##  4 Training and Benchmarking Datasets

High-quality and high-quantity datasets are critical resources in advancing
machine learning research. Particularly in far-field speech processing, we
believe the progression of data-driven methods is currently hindered by the
absence of comprehensive training and benchmarking datasets. These datasets
are crucial for unlocking the full potential of such methods and measuring
progress. To address this need, we introduce three key resources: First, a
dataset of natural meeting recordings carefully designed for benchmarking
purposes. Second, to foster innovation and encourage the adoption of data-
driven solutions for speech separation and enhancement, we offer a matched
simulated training dataset that bridges several gaps between training and
testing conditions. Third, to supplement the simulated training set and to
allow training all components of a speech processing pipeline, we allocate a
portion of the meeting dataset for training purposes. Both datasets encompass
multi-channel and single-channel settings.

Table 1: NOTSOFAR Meeting Dataset. Feature |  Training Set |  Development Set |  Evaluation Set (blind)
---|---|---|---
Number of Meetings |  110 meetings |  35 meetings |  170 meetings
Meeting Duration |  6 minutes |  6 minutes |  6 minutes
Number of SC streams |  5 streams |  5 streams |  6 streams
Number of MC devices |  4 devices |  4 devices |  4 devices
Total Duration of SC Audio Recordings |  55 hours (110 meetings $\times$ 6 minutes $\times$ 5 SC streams) |  17.5 hours (35 meetings $\times$ 6 minutes $\times$ 5 SC streams) |  102 hours (170 meetings $\times$ 6 minutes $\times$ 6 SC streams)
Total Duration of MC Audio Recordings |  44 hours (110 meetings $\times$ 6 minutes $\times$ 4 MC devices) |  14 hours (35 meetings $\times$ 6 minutes $\times$ 4 MC devices) |  68 hours (170 meetings $\times$ 6 minutes $\times$ 4 MC devices)
Total Number of Rooms |  20 rooms |  5 rooms |  13 rooms
Number of Participants |  22 Participants |  11 Participants |  13 Participants
Language | English
Table 2: Recording Devices for NOTSOFAR Meeting Dataset. Device |  Position |  Array Geometry |  Single-channel stream |  Multi-channel stream |  Dataset
---|---|---|---|---|---
Yealink SmartVision 60 |  Center of table |  Circular 7 Microphones |  Yes |  Yes |  All sets
Yealink Intelligent Speaker |  Center of table |  Circular 7 Microphones |  Yes |  Yes |  All sets
EPOS Expand Capture 5 |  Center of table |  Circular 7 Microphones |  Yes |  Yes |  All sets
Sennheiser TeamConnect Intelligent Speaker |  Center of table |  Circular 7 Microphones |  Yes |  Yes |  All sets
Logitech MeetUp |  Front of room |  Linear |  Yes |  No |  All sets
Poly Studio |  Front of room |  Linear |  Yes |  No |  Evaluation set
Jabra PanaCast 50 |  Front of room |  Linear |  Yes |  No |  Evaluation set (partial)
Shure WH20 + TASCAM DR-40X Close Talk |  Head mounted microphone |  N/a |  Yes |  No |  All sets (except evaluation set)

###  4.1 NOTSOFAR Recorded Meeting Dataset

This dataset comprises approximately 315 unique meetings, each lasting on
average 6 minutes, featuring authentic, multi-participant English
conversations recorded in about 30 different conference rooms at Microsoft
offices. To increase acoustic diversity, each meeting was captured with
several devices, each positioned differently. This setup, per meeting,
involved around 5 single-channel (SC) devices producing a single internally
processed stream each, and 4 multi-channel (MC) devices producing 7 raw
streams each. In total, this resulted in roughly 170 hours of SC data and 130
hours of MC data across all devices and meetings. We divide the dataset into
training (55 SC hours, 44 MC hours), development (17.5 SC hours, 14 MC hours),
and evaluation sets (102 SC hours, 68 MC hours). The latter, a subset of which
will be used as the fully blind evaluation set in the NOTSOFAR-1 Challenge, is
entirely disjoint from the other sets, with no overlap in speakers or rooms.
Tables 1 and 2 give a compact view of this dataset. Meticulously designed and
collected, the dataset possesses unique characteristics that we believe make
it an excellent benchmark for conversational scenarios.

Capturing the full spectrum of real-world complexities: As many practitioners
observe, to truly evaluate a system’s accuracy, it is essential to test it not
only in typical scenarios but also in rare events that lie at the borders of,
or even beyond, its standard operational envelope. In line with this, the
recordings encompass a wide array of acoustic situations, including speakers
at varying distances and volumes, and different types of transient
interference and noises at low and high levels. Importantly, the dataset
includes acoustic situations that sometimes substantially and frequently
modify the ATF between the active speaker and the device. These range from in-
seat movement, walking, standing or sitting, to speaking near a whiteboard,
and entering or leaving the room. Conversational dynamics include overlapping
speech from multiple speakers, interruptions, rapid speaker change, laughter,
coughing, and fillers. The meetings span 30 different rooms of various
dimensions, layouts, construction materials and acoustic characteristics.
Attendees for each meeting typically vary between 4-8 adults, involving 22
unique speakers in the development and training sets, and a distinct group of
13 speakers in the evaluation set, while ensuring balanced gender
representation.

Metadata for deep-dive analysis: The errors of speech recognition systems in
conversational settings are notoriously hard to investigate. The multitude of
factors impacting the speech pipeline are highly context-dependent and are
often undetectable without specialized tooling or manual tagging by human-in-
the-loop. To facilitate scalable error analysis and uncovering a system’s
blind spots, each meeting is accompanied by metadata, including activities and
acoustic events. Some meetings are purposefully dominated by rare events which
are documented in the metadata. For example, meetings tagged as extra noisy,
or having extra overlapping speech are useful for testing algorithms under
those conditions in an isolated manner. Meetings where speakers are in motion
or are speaking near sound-reflective whiteboards while turning their heads
are useful to test the robustness of an algorithm to changes in the ATF. In
the NOTSOFAR-1 Challenge we intend to share the performance evaluations of
submissions both averaged across all meetings and within each data vertical.

Prioritizing meetings over hours: Speech datasets often highlight the duration
of their recordings in hours, yet the utility of a benchmark is determined by
the distribution it spans and the sample size it provides. Recognizing that
prolonged recordings from the same meeting offer diminishing returns in terms
of yielding new, independent information, our approach focuses on maximizing
the diversity of the dataset by including a larger number of distinct
meetings. Our dataset consists of approximately 315 relatively short meetings,
each lasting roughly six minutes. Since each meeting offers a unique mix of
acoustics, speaker identities, and conversational dynamics, we obtain a highly
diverse sample. This allows us to assume each meeting is approximately an
independent and identically distributed (i.i.d.) element, facilitating the
computation of meaningful confidence intervals, a vital aspect of algorithm
comparison [25]. Our baseline code features a simple method for calculating
confidence intervals, both in absolute terms and relative to the baseline
algorithm. Notably, the narrowness of these confidence intervals is determined
by the number of meetings rather than the total recorded hours, and benefits
from the large volume of meetings in our dataset.

Bias-free transcription process: Accurate transcription is crucial for
training and evaluating speech recognition systems. One concern in this area
is the potential bias toward machine-generated transcriptions. This bias can
occur when the annotators are influenced by the output of automated systems,
leading to transcriptions that are inadvertently biased toward a particular
system’s errors. Consider a machine-aided process where a machine first
transcribes the recordings, and a human conducts a second pass, deciding to
accept some words and correct others. Due to the inherent ambiguities of
speech, even in close-talk recordings, and especially in situations with
heated discussions, overlapping speech, and rapid speaker turns, some
erroneous machine suggestions may seem reasonable and hence may be accepted by
the human annotator. While this process is relatively efficient, it
perpetuates errors of the specific transcription algorithm used. The
implications of this bias are profound: competing algorithms will be penalized
for deviating from the transcription algorithm, potentially concealing the
progress of new algorithms. This is particularly problematic when improvements
are incremental, as is often the case in research. Since even subtler forms of
machine aid can introduce bias, our transcription process for the development
and evaluation sets is based on close-talk recordings and conducted entirely
without human access to machine-generated transcriptions [26].

Multi-judge annotation for accurate segmentation and transcription: To ensure
high-quality ground-truth transcription and segmentation, attendees were
equipped with close-talk microphones. Spectral subtraction between close-talk
channels was used as pre-processing for an automatic segmentation algorithm,
followed by human annotators who reviewed and fine-tuned any segmentation
errors. Throat microphones were used for quality control and tuning of the
algorithms. Professional transcribers carried out the transcription process.
Most meetings were annotated independently by two professionals, with a third
judge resolving non-consensus cases, except for about 50 meetings in the
training set that involved only two annotators. Close-talk recordings of the
development and training sets are also released as part of the challenge to
encourage exploration of creative methods, such as using pseudo labels.
However, these close-talk recordings may still contain a small amount of
speech leakage. Hence, to prevent any impact on ASR performance, we recommend
masking the close-talk data of each participant by their segmented speech.

Distinguishing single-channel from single-microphone: When evaluating speech
recognition systems that process a one-dimensional stream of audio, it is
common to use audio recorded from one of the microphones in an array. However,
in practice, speech recognition systems often deal with audio from existing
conference room devices. These devices are typically equipped with microphone
arrays and employ proprietary on-device algorithms to process the audio,
yielding what we refer to in this paper as a “single-channel” output. This
output is usually the result of processing all the microphones in the array
and using a pipeline that includes echo cancellation, de-reverberation,
beamforming, and noise suppression. This significantly differs from the
single-microphone outcome. To evaluate performance in the single-channel
setting, our recordings include a variety of commercially available single-
channel devices. Although single-microphone setups are of great scientific
interest, particularly when compared to multiple microphones, the NOTSOFAR-1
Challenge's single-channel track focuses on the practical scenario of
processing audio from single-channel devices to bridge the gap to real-world
applications.

A variety of recording devices: Each meeting was recorded by multiple
commercially available conference rooms devices, as listed in Table 2. Each
device produced a single-channel stream and most recorded an additional,
multi-channel stream. The multi-channel streams are recorded in raw format
directly from the microphone arrays without processing and their geometry is
of one central and six surrounding microphones, matching the simulated
training set described in the next section. The purely single-channel devices
have linear microphone-arrays with different numbers of microphones. In the
NOTSOFAR-1 Challenge, although multiple devices were used for recording,
processing during inference is restricted to just one of these devices. The
head-mounted close-talk microphones featured a unidirectional cardioid pickup
pattern that provides some isolation of each speaker from their nearby
speakers, allowing for more faithful transcription even in the presence of
loud overlapped speech. In every conference room, we played chirps (frequency
sweeps) with a loudspeaker and recorded them with each of the multi-channel
devices. These recordings are supplied as well and may be used to compute
reverberation times, e.g., RT60 and T30 [27].

Oriented toward far-field: The primary challenge in far-field speech
transcription lies in creating an algorithm that is robust to the acoustic
complexities introduced by the distance between the microphone and the
speaker, as well as the surrounding environmental conditions. Some aspects of
our dataset are designed for far-field evaluation. Namely, we deliberately
steer away from challenges related to different languages, accents, and
domain-specific jargon. The recordings are exclusively in English, spoken by
individuals who are either native speakers or possess near-native proficiency.
The language employed is predominantly semi-professional. While the meetings
encompass a wide range of topics, care was taken to avoid specialized jargon
or obscure terminology, thereby focusing more on the acoustic challenges
rather than linguistic diversity.

A multitude of topics and interaction styles: Most meetings featured semi-
professional topics, in which participants role-played as professionals
discussing a work-related issue. For example, a cruise ship company planning
an event, administrators planning a city park, or users complaining about IT
problems. Some meetings featured non-work-related topics, such as favorite TV
shows, debating whether to raise kids as vegetarians, or friends sharing
recipes. We expect this role-playing to create a range of conversational
dynamics akin to those found in actual meetings in terms of speech overlap,
turn-taking dynamics, speech disfluencies, interruptions and so on.

###  4.2 NOTSOFAR Simulated Dataset

The simulated training set consists of about 1000 hours simulated with the
same microphone-array geometry of one central and six surrounding microphones,
matching the multi-channel devices in the NOTSOFAR meeting dataset. It
provides separated speech and noise components as supervision signals for
training data-driven speech separation and enhancement methods. It is designed
to foster generalization to realistic settings by identifying and closing
major train-test gaps prevalent in simulated datasets in prior works.

Real-room acoustic transfer functions: Simulated datasets for speech
separation typically rely on the Image Method [28] to generate simulated ATFs,
which are then convolved with clean speech to further simulate in-room
utterances. Despite advancements in simulation techniques, simulated ATFs do
not take into account many elements present in the real-world, such as: the
room’s layout and the unique reflective properties of its surfaces, the
dependency of the reflection coefficients on frequency and angle of incidence,
the reflections from adjacent speakers, the directivity of the speech source,
and the acoustic effects of the specific microphone array casing. The
limitations of the Image Method in this context have been explored in several
studies, e.g., [29, 30, 31]. To address these shortcomings, our dataset
features ATFs recorded in actual conference rooms, in an acoustic setup
arranged to closely replicate authentic meeting environments. We collected a
total of 15,000 real ATFs, measured in various positions and rooms by multiple
devices sharing the same geometry. The ATFs were reconstructed from chirps
emitted by a mouth simulator.

MOS-filtered clean speech: For clean speech we use the Librivox corpus [32],
which contains recordings of varying quality. Many of these recordings are of
excellent speech quality, but others exhibit poor quality in the form of
speech distortion, background noise and reverberation. This quality variance
compromises simulation authenticity, since the ATF convolution operation
assumes true clean speech. Hence, we follow the work of [33] and choose the
upper quartile with respect to the subjective mean-opinion score (MOS) as our
clean speech corpus, amounting to 500 hours of speech.

Augmented clean speech for realistic overlap patterns: We observe that the
Librivox clean speech corpus, comprising recordings of individuals reading
audiobooks, typically consists of rather continuous segments of speech lasting
several seconds, without significant pauses. This makes it harder to achieve a
mixture reflecting realistic overlap patterns which include short
interruptions and rapid speaker turns. We therefore apply a signal processing
algorithm that detects drops in speech power to identify short pauses in
speech and subsequently insert random-length silence breaks at these points.
By augmenting clean speech in this manner, we facilitate a more realistic
simulation of conversational dynamics during the mixing stage.

Mixing up to three speakers: During the mixing process we first convolve the
augmented clean speech utterances with different ATFs selected from the same
room. The resulting individual utterances are roughly 40 seconds long. We then
apply a small random shift before summing utterances from 3 distinct speakers.
This strategy results in mixtures that average 50 seconds in length and
possess two attractive characteristics. First, the mixtures display real-world
patterns such as complex overlaps, and rapid speaker turns featuring up to
three distinct speakers. Qualitatively we observe that randomly sampled
segments of 2-10 seconds, as commonly utilized in speech separation training,
effectively capture patterns that are typical in real meetings. Second, these
segments tend to be more challenging, serving as a form of hard sample mining,
which is beneficial for training data-driven methods. Finally, transient and
stationary noises recorded in real rooms with the matching geometry were added
to the mixture.

Separating direct and early reflections from late reverberations: Previous
studies have highlighted that speech signals captured by a distant microphone
are generally smeared by reverberation, which severely degrades automatic ASR
performance [34]. Motivated by this as well as internal experimentation
findings, we have designed the supervision signals in this training set to
facilitate not only speech separation training but also dereverberation. We
assume a room impulse response (RIR) consists of three parts: a direct-path
response, early reflections, and late reverberations. Since late
reverberations are known to be a major cause of ASR performance degradation,
we divide the recorded ATFs (now viewed as RIRs) into two components: direct
and early reflections, and late reverberations. This division is accomplished
by applying a smoothed-out cutoff at the 50-millisecond mark. The mixed signal
at time $t$, with $i$ indexing speakers, is then represented as follows:

$\displaystyle\mathrm{mixture}(t)$
$\displaystyle=\sum_{i}\left[\mathrm{spk}_{i}^{\textnormal{direct+early}}(t)+\mathrm{spk}_{i}^{\textnormal{reverb}}(t)\right]+\mathrm{noise}(t),$

The individual ground truth (GT) components are provided as supervision
signals in our dataset. The baseline model in the NOTSOFAR-1 Challenge employs
the direct and early reflections component as target labels for its speech
separation module. However, since the sensitivity to late reverberations
depends on the specific ASR system, participants are encouraged to construct
their own target labels, possibly incorporating the full RIR.

Limitations: A notable limitation of the simulated training set is the static
nature of the ATFs, which, despite being recorded in real rooms, remain fixed
throughout each simulated utterance. This fails to mirror common real-world
situations where the ATF varies in time as the speaker or environment moves.
To evaluate an algorithm's robustness against time-varying ATFs, one can
leverage the metadata provided in the NOTSOFAR meeting dataset. Furthermore,
we hope that the training portion of these real recordings, which includes
instances of time-varying ATFs, will contribute significantly to improving an
algorithm’s robustness to such variations.

##  5 Baseline System

The baseline recipe of NOTSOFAR-1, inspired by [21, 22], consists of three
steps: continuous speech separation (CSS), ASR, and speaker diarization. In
this section, we will briefly describe each step.

CSS: The objective of CSS is to receive an input audio stream, consisting of
either 1 or 7 channels, and convert it into a set of overlap-free streams. In
our case, we generate $N=3$ distinct speech streams, which can support up to 3
speakers overlapping at the same time. An effective CSS essentially monitors
the input audio stream, and when overlapping utterances are found, it
distributes them to the different output channels. We follow the conventional
CSS framework, first training a speech separation network with permutation
invariant training (PIT) loss, which takes fixed-length audio segments as
input and outputs $N$ speech masks and one noise mask. The masks are in STFT
(short-time Fourier transform) domain [35]. For the network architecture we
select the conformer model [13, 36].

In the inference phase, the network is applied in a block-wise streaming
fashion to overlapping fixed-length segments. Since the order of the $N$
speech mask outputs may not be consistent across segments, we align every pair
of adjacent segments. To estimate the best order, we consider all permutations
and select the one with the lowest mean squared error (MSE) between the masked
magnitude spectrograms, calculated over the frames shared by the two segments.
After stitching the $N$ speech masks and one noise mask over time, we proceed
to generate each of the N output streams. For the single-channel variant, this
consists of multiplying the input mixture by the target speech mask. For the
multi-channel variant, we rely on mask-based minimum variance distortionless
response (MVDR) beamforming [37]. As part of this scheme, the spatial
covariance matrices (SCMs) of the target and interference signals are
computed, where the interference signal is defined as the sum of all non-
target speakers and the background noise.

ASR: For automatic speech recognition (ASR), we employ Whisper “large-v2”
[38], which supports word-level timestamps. ASR is applied independently to
each audio stream produced by CSS.

Speaker Diarization: The task of the diarization module is to assign a speaker
label to every word transcribed by the ASR module. Speaker labels are unique
identifiers such as `spk0', `spk1', etc., and they are not required to be
identical to the reference speakers. We first apply an offline diarization
method to diarize each of the audio streams produced by CSS. Then, we assign
each word to its speaker label based on its word boundary information and the
diarization output from the source stream.

For offline diarization, we adopted the diarization recipe of the NeMo toolkit
[39]. There are two configurations supported, including “nmesc” which performs
offline clustering using the normalized maximum eigengap-based spectral
clustering (NME-SC) algorithm [40], and “nmesc-msdd” which performs NME-SC
followed by the multi-scale diarization decoder (MSDD) [41].

To assign a speaker label to an ASR word, we look up the active speakers
within its time boundaries in the diarization output from the corresponding
audio stream. In most cases, there is only one active speaker within the
word’s boundaries, and it is assigned to the word. If there is no active
speaker within the word’s boundaries (i.e. diarization didn’t detect speech),
the speaker label of the nearest word in time is assigned. If there are
multiple active speakers within the word’s boundaries (i.e. diarization
detected overlapped speech), the speaker who is active for the longest
duration is assigned.

Note that the sole distinction between the single-channel and multi-channel
variants of our system is in the CSS module which processes either 1 channel
or 7 channels as its input, applying mask multiplication or MVDR respectively.

##  6 Conclusions

The NOTSOFAR-1 Challenge marks a significant stride in the field of DASR. It
provides a 315-meeting benchmarking dataset that captures the nuances and
complexities of real-world meetings. The benchmark includes a large volume of
distinct meetings, enabling reliable performance analysis of algorithms.
Annotations were collected by human listeners, providing ground-truth
annotations with low WER. The challenge also includes a 1000-hour simulated
training data set, crafted to leverage known microphone array geometry, and
mitigate the mismatch between training and testing conditions. These
contributions collectively aim to stimulate further research and innovation,
and to push the boundaries of what is scientifically possible in the field of
DASR.

##  7 Acknowledgements

We express our profound gratitude to Hadas Eilon Carmi, Ross Cutler, Prem
Premkumar, Solomiya Branets, and the IC3-AI team at Microsoft for their
invaluable assistance.

## References

  * [1] J. Chung and D. Blaser, ``Transfer function method of measuring in-duct acoustic properties. i. theory,'' _The Journal of the Acoustical Society of America_ , vol. 68, no. 3, pp. 907–913, 1980.
  * [2] E. Skudrzyk, _The foundations of acoustics: basic mathematics and basic acoustics_. Springer Science & Business Media, 2012.
  * [3] A. Sellen and E. Horvitz, ``The rise of the AI Co-Pilot: Lessons for design from aviation and beyond,'' _arXiv preprint arXiv:2311.14713_ , 2023.
  * [4] W. Kraaij, T. Hain, M. Lincoln, and W. Post, ``The AMI meeting corpus,'' in _Proc. International Conference on Methods and Techniques in Behavioral Research_ , 2005.
  * [5] R. Dhillon, S. Bhagat, H. Carvey, and E. Shriberg, ``Meeting recorder project: Dialog act labeling guide,'' ICSI Technical Report TR-04, Tech. Rep., 2004.
  * [6] C. Fox, Y. Liu, E. Zwyssig, and T. Hain, ``The Sheffield wargames corpus,'' in _Proc. Interspeech_. ISCA, 2013.
  * [7] M. Harper, ``The automatic speech recogition in reverberant environments (ASpIRE) challenge,'' in _Proc. 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU)_. IEEE, 2015, pp. 547–554.
  * [8] L. Cristoforetti, M. Ravanelli, M. Omologo, A. Sosi, A. Abad, M. Hagmüller, P. Maragos _et al._ , ``The DIRHA simulated corpus,'' in _Proc. LREC_ , 2014, pp. 2629–2634.
  * [9] C. Richey, M. A. Barrios, Z. Armstrong, C. Bartels, H. Franco, M. Graciarena, A. Lawson, M. K. Nandwana, A. Stauffer, J. van Hout _et al._ , ``Voices obscured in complex environmental settings (voices) corpus,'' _arXiv preprint arXiv:1804.05053_ , 2018.
  * [10] M. Van Segbroeck, A. Zaid, K. Kutsenko, C. Huerta, T. Nguyen, X. Luo, B. Hoffmeister, J. Trmal, M. Omologo, and R. Maas, ``DiPCo–dinner party corpus,'' _arXiv preprint arXiv:1909.13447_ , 2019.
  * [11] K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu _et al._ , ``Ego4d: Around the world in 3,000 hours of egocentric video,'' in _Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 18 995–19 012.
  * [12] F. Yu, S. Zhang, Y. Fu, L. Xie, S. Zheng, Z. Du, W. Huang, P. Guo, Z. Yan, B. Ma _et al._ , ``M2MeT: The ICASSP 2022 multi-channel multi-party meeting transcription challenge,'' in _Proc. ICASSP_. IEEE, 2022, pp. 6167–6171.
  * [13] Z. Chen, T. Yoshioka, L. Lu, T. Zhou, Z. Meng, Y. Luo, J. Wu, X. Xiao, and J. Li, ``Continuous speech separation: Dataset and analysis,'' in _Proc. ICASSP_. IEEE, 2020, pp. 7284–7288.
  * [14] N. Ma, J. Barker, H. Christensen, and P. Green, ``A hearing-inspired approach for distant-microphone speech recognition in the presence of multiple sources,'' _Computer Speech & Language_, vol. 27, no. 3, pp. 820–836, 2013.
  * [15] E. Vincent, J. Barker, S. Watanabe, J. Le Roux, F. Nesta, and M. Matassoni, ``The second ‘CHiME’ speech separation and recognition challenge: Datasets, tasks and baselines,'' in _Proc. ICASSP_. IEEE, 2013, pp. 126–130.
  * [16] J. Barker, R. Marxer, E. Vincent, and S. Watanabe, ``The third ‘CHiME’ speech separation and recognition challenge: Analysis and outcomes,'' _Computer Speech & Language_, vol. 46, pp. 605–626, 2017.
  * [17] E. Vincent, S. Watanabe, J. Barker, and R. Marxer, ``The 4th CHiME speech separation and recognition challenge,'' _URL: http://spandh. dcs. shef. ac. uk/chime challenge $\\{$Last Accessed on 1 August, 2018$\\}$_, 2016.
  * [18] J. Barker, S. Watanabe, E. Vincent, and J. Trmal, ``The fifth 'CHiME' speech separation and recognition challenge: dataset, task and baselines,'' _arXiv preprint arXiv:1803.10609_ , 2018.
  * [19] S. Watanabe, M. Mandel, J. Barker, E. Vincent, A. Arora, X. Chang, S. Khudanpur, V. Manohar, D. Povey, D. Raj _et al._ , ``CHiME-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,'' _arXiv preprint arXiv:2004.09249_ , 2020.
  * [20] S. Cornell, M. Wiesner, S. Watanabe, D. Raj, X. Chang, P. Garcia, Y. Masuyama, Z.-Q. Wang, S. Squartini, and S. Khudanpur, ``The CHiME-7 DASR challenge: Distant meeting transcription with multiple devices in diverse scenarios,'' _arXiv preprint arXiv:2306.13734_ , 2023.
  * [21] T. Yoshioka, I. Abramovski, C. Aksoylar, Z. Chen, M. David, D. Dimitriadis, Y. Gong, I. Gurvich, X. Huang, Y. Huang _et al._ , ``Advances in online audio-visual meeting transcription,'' in _Proc. 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)_. IEEE, 2019, pp. 276–283.
  * [22] D. Raj, P. Denisov, Z. Chen, H. Erdogan, Z. Huang, M. He, S. Watanabe, J. Du, T. Yoshioka, Y. Luo _et al._ , ``Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis,'' in _Proc. 2021 IEEE spoken language technology workshop (SLT)_. IEEE, 2021, pp. 897–904.
  * [23] T. von Neumann, C. Boeddeker, M. Delcroix, and R. Haeb-Umbach, ``MeetEval: A toolkit for computation of word error rates for meeting transcription systems,'' in _Proc. CHiME-2023 Workshop, Dublin, England_ , 2023.
  * [24] D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz _et al._ , ``The Kaldi speech recognition toolkit,'' in _Proc. IEEE 2011 workshop on automatic speech recognition and understanding_ , no. CONF. IEEE Signal Processing Society, 2011.
  * [25] M. E. Payton, M. H. Greenstone, and N. Schenker, ``Overlapping confidence intervals or standard error intervals: what do they mean in terms of statistical significance?'' _Journal of Insect Science_ , vol. 3, no. 1, p. 34, 2003.
  * [26] M. Levit, Y. Huang, S. Chang, and Y. Gong, ``Don't count on ASR to transcribe for you: Breaking bias with two crowds.'' in _Proc. Interspeech_ , 2017, pp. 3941–3945.
  * [27] A. Matt and D. Stowell, ``Estimating & mitigating the impact of acoustic environments on machine-to-machine signalling,'' in _Proc. 2019 27th European Signal Processing Conference (EUSIPCO)_. IEEE, 2019, pp. 1–5.
  * [28] J. B. Allen and D. A. Berkley, ``Image method for efficiently simulating small-room acoustics,'' _The Journal of the Acoustical Society of America_ , vol. 65, no. 4, pp. 943–950, 1979.
  * [29] L. Savioja and U. P. Svensson, ``Overview of geometrical room acoustic modeling techniques,'' _The Journal of the Acoustical Society of America_ , vol. 138, no. 2, pp. 708–730, 2015.
  * [30] F. Brinkmann, L. Aspöck, D. Ackermann, S. Lepa, M. Vorländer, and S. Weinzierl, ``A round robin on room acoustical simulation and auralization,'' _The Journal of the Acoustical Society of America_ , vol. 145, no. 4, pp. 2746–2760, 2019.
  * [31] E. De Sena, N. Antonello, M. Moonen, and T. Van Waterschoot, ``On the modeling of rectangular geometries in room acoustic simulations,'' _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 23, no. 4, pp. 774–786, 2015.
  * [32] J. Kearns, ``Librivox: Free public domain audiobooks,'' _Reference Reviews_ , vol. 28, no. 1, pp. 7–8, 2014.
  * [33] C. K. Reddy, V. Gopal, R. Cutler, E. Beyrami, R. Cheng, H. Dubey, S. Matusevych, R. Aichner, A. Aazami, S. Braun _et al._ , ``The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,'' _arXiv preprint arXiv:2005.13981_ , 2020.
  * [34] K. Lebart, J.-M. Boucher, and P. N. Denbigh, ``A new method based on spectral subtraction for speech dereverberation,'' _Acta Acustica united with Acustica_ , vol. 87, no. 3, pp. 359–366, 2001.
  * [35] L. Durak and O. Arikan, ``Short-time Fourier transform: two fundamental properties and an optimal implementation,'' _IEEE Transactions on Signal Processing_ , vol. 51, no. 5, pp. 1231–1242, 2003.
  * [36] A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu _et al._ , ``Conformer: Convolution-augmented transformer for speech recognition,'' _arXiv preprint arXiv:2005.08100_ , 2020.
  * [37] E. A. Habets, J. Benesty, S. Gannot, and I. Cohen, ``The MVDR beamformer for speech enhancement,'' in _Speech Processing in Modern Communication: Challenges and Perspectives_. Springer, 2010, pp. 225–254.
  * [38] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, ``Robust speech recognition via large-scale weak supervision,'' 2022.
  * [39] O. Kuchaiev, J. Li, H. Nguyen, O. Hrinchuk, R. Leary, B. Ginsburg, S. Kriman, S. Beliaev, V. Lavrukhin, J. Cook, P. Castonguay, M. Popova, J. Huang, and J. M. Cohen, ``NeMo: a toolkit for building ai applications using neural modules,'' 2019.
  * [40] T. J. Park, K. J. Han, M. Kumar, and S. Narayanan, ``Auto-tuning spectral clustering for speaker diarization using normalized maximum eigengap,'' _IEEE Signal Processing Letters_ , vol. 27, pp. 381–385, 2019.
  * [41] T. J. Park, N. R. Koluguri, J. Balam, and B. Ginsburg, ``Multi-scale speaker diarization with dynamic scale weighting,'' _arXiv preprint arXiv:2203.15974_ , 2022.


# Nonstandard existence proofs
for reaction diffusion equations

Connor Olson  C.O.: Department of Mathematics, University of Wisconsin –
Madison  ,  Marshall Mueller  M.M.: Department of Mathematics, Tufts
University – 503 Boston Avenue, Medford, MA 02155  and  Sigurd B. Angenent
S.B.A.: Department of Mathematics, University of Wisconsin – Madison

(Date: August 28, 2024)

###### Abstract.

We give an existence proof for distribution solutions to a scalar reaction
diffusion equation, with the aim of illustrating both the differences and the
common ingredients of the nonstandard and standard approaches. In particular,
our proof shows how the operation of taking the standard part of a nonstandard
real number can replace several different compactness theorems, such as
Ascoli’s theorem and the Banach–Alaoglu theorem on weak∗-compactness of the
unit ball in the dual of a Banach space.

###### Contents

  1. 1 Introduction
    1. 1.1 Reaction diffusion equations
    2. 1.2 Comments on nonstandard analysis
  2. 2 Distribution solutions
    1. 2.1 The definition of distributions
    2. 2.2 Distributions defined by nonstandard functions on a grid
  3. 3 The Cauchy problem for the heat equation
    1. 3.1 Formulation in terms of distributions
    2. 3.2 The finite difference equation
    3. 3.3 The approximate solution
    4. 3.4 Proof of Theorem 3.3.1
    5. 3.5 Comments on the proof
  4. 4 The Cauchy problem for a Reaction Diffusion Equation
    1. 4.1 Weak Solutions to the Reaction Diffusion Equation
    2. 4.2 Definition of the approximate solution
    3. 4.3 Boundedness of the approximate solution
    4. 4.4 Lipschitz continuity in space of the approximate solution
    5. 4.5 Hölder continuity in time of the approximate solution
    6. 4.6 Definition of the weak solution
    7. 4.7 Proof that $u$ is a weak solution

##  1\. Introduction

###  1.1. Reaction diffusion equations

We consider the Cauchy problem for scalar reaction diffusion equations of the
form

(1a) $\frac{\partial u}{\partial t}=D\frac{\partial^{2}u}{\partial
x^{2}}+f(u(x,t)),\qquad x\in\mathbb{R},t\geq 0$ with prescribed initial
condition (1b) $u(x,0)=u_{0}(x).$

In the setting of reaction diffusion equations the function $u(x,t)$
represents the density at location $x\in\mathbb{R}$ and time $t\geq 0$ of some
substance which diffuses, and simultaneously grows or decays due to chemical
reaction, biological mutation, or some other process. The term
$D\partial^{2}u/\partial x^{2}$ in the PDE (1a) accounts for the change in $u$
due to diffusion, while the nonlinear term $f(u)$ accounts for the reaction
rates. The prototypical example of such a reaction diffusion equation is the
Fisher/KPP equation (see [9], [2]) in which the reaction term is given by
$f(u)=u-u^{2}$.

The Cauchy problem for the reaction diffusion equation (1a) is to find a
function $u:\mathbb{R}\times[0,\infty)\to\mathbb{R}$ that satisfies the
partial differential equation (1a) as well as the initial condition (1b). This
is a classical problem, and the existence of such solutions is well known (see
for example [5] or [12]). As various techniques for constructing solutions are
known, including the use of finite difference approximations to construct
solutions (see [6, Ch.7.2]), our main goal is not to give another existence
proof. Instead, we were inspired by several introductory texts on nonstandard
analysis (notably, Keisler’s undergraduate calculus text [8], Goldblatt’s more
advanced introduction to the hyperreals [4], Nelson’s “radically elementary
approach” to probability [11], as well as Terry Tao’s blog post [14]), and
wanted to see what some standard existence proofs would look like in the
language of nonstandard analysis.

In [8] Keisler presents a proof of Peano’s existence theorem for solutions to
ordinary differential equations

(2) $\frac{dx}{dt}=f(t,x(t)),\quad x(0)=x_{0},$

using nonstandard analysis. One possible standard proof of Peano’s theorem
proceeds by constructing the numerical approximation to the solution by
solving Euler’s method for any small step size $\Delta t>0$, i.e., one defines
numbers $x_{i,\Delta t}$ by setting $x_{0,\Delta t}=x_{0}$, and then
inductively solving

(3) $\frac{x_{i+1,\Delta t}-x_{i,\Delta t}}{\Delta t}=f(i\Delta t,x_{i,\Delta
t}),\quad i=0,1,2,\dots$

The function $x_{\Delta t}:[0,\infty)\to\mathbb{R}$ obtained by linearly
interpolating between the values $x_{\Delta t}(i\Delta t)=x_{i,\Delta t}$ is
Euler’s numerical approximation to the solution of the differential equation
(2). The standard analysis proof of Peano’s existence theorem then uses
Ascoli’s compactness theorem to extract a sequence of step sizes $\Delta
t_{n}\to 0$ such that of the approximate solutions $x_{\Delta t_{n}}(t)$
converge uniformly to some function $\tilde{x}:[0,\infty)\to\mathbb{R}$, and
concludes by showing that the limit $\tilde{x}$ is a solution to the
differential equation (2).

The nonstandard proof in Keisler’s text [8] follows the same outline, but one
notable feature of this proof is that instead of using Ascoli’s theorem, one
“simply” chooses the step size $\Delta t$ to be a positive infinitesimal
number. The approximate solution then takes values in the hyperreals, and
instead of applying a compactness theorem (Ascoli’s in this case), one “takes
the standard part” of the nonstandard approximate solution. The proof is then
completed by showing that the function that is obtained actually satisfies the
differential equation.

The standard and nonstandard proofs have some common ingredients. In both
proofs one must find suitable estimates for the approximate solutions
$x_{i,\Delta t}$, where the estimates should not depend on the step size
$\Delta t$. Namely, the approximate solutions $x_{\Delta t}$ should be
uniformly bounded, and they should be uniformly Lipschitz continuous
($|x_{\Delta t}(t)-x_{\Delta t}(s)|\leq L|t-s|$ for all $t,s\in[0,\infty)$,
$\Delta t>0$). In the standard proof these estimates allow one to use Ascoli’s
theorem; in the nonstandard proof they guarantee that the standard part of the
approximating solution with infinitesimally small $\Delta t$ still defines a
continuous function on the standard reals.

There appear to be two main differences between the standard and nonstandard
proofs. The first, very obviously, is that the nonstandard setting allows one
to speak rigorously of infinitely small numbers, and thereby avoid the need to
consider limits of sequences. The second difference is that the process of
“taking the standard part” of a hyperreal number acts as a replacement for one
compactness theorem or another: in the nonstandard proof of Peano’s theorem
one avoids Ascoli’s theorem by taking standard parts. This too is probably
well-known in some circles (Terry Tao makes the point in his blog post [14]),
but is not as obviously stated in the nonstandard analysis texts we have seen.

In this paper we intend to further illustrate this point by proving an
existence theorem for weak, or distributional solutions of certain partial
differential equations that is analogous to the proof of Peano’s theorem
sketched above (see section 2.1 below for a very short summary of the theory
of distributions.) Thus, to “solve” the reaction diffusion equation (1a) we
choose space and time steps $\Delta x>0$ and $\Delta t>0$, and discretize the
PDE by replacing the second derivative with a second difference quotient, and
the time derivative with a forward difference quotient, resulting in a finite
difference equation

(4) $\frac{u(x,t+\Delta t)-u(x,t)}{\Delta t}=D\,\frac{u(x+\Delta
x,t)-2u(x,t)+u(x-\Delta x,t)}{(\Delta x)^{2}}+f(u(x,t)).$

This kind of discretization is very common in numerical analysis (see [10], or
[13]) For given initial data (but no boundary data) one can use this
difference equation to inductively compute the values of $(x,t)$ for all
$(x,t)$ in a triangular grid (see Figure 1).

Given a solution of the difference equation one can define a generalized
function, or distribution

(5) $\langle U,\varphi\rangle\stackrel{{\scriptstyle\rm
def}}{{=}}\sum_{t}\sum_{x}U(x,t)\varphi(x,t)\,.$

In a standard existence proof of weak solutions to the equation one would now
use a compactness theorem to extract a sequence $(\Delta x_{i},\Delta
t_{i})\to(0,0)$ for which the corresponding distributions $U_{i}$ converge in
the sense of distributions, and then show that the limiting distribution
satisfies the PDE (1a). The compactness theorem that is required in this proof
is the Banach-Alaoglu theorem about weak∗-compactness in duals of Banach
spaces (in our case, $L^{\infty}(\mathbb{R}^{2})$ which is the Banach space
dual of $L^{1}(\mathbb{R}^{2})$).

The nonstandard proof, which we give in this paper, avoids the compactness
theorem (or notions of Lebesgue integration required to define $L^{\infty}$)
by letting $\Delta x$ and $\Delta t$ be infinitesimal positive hyperreals, and
by taking the standard part of the expression on the right in (5). In both the
standard and nonstandard setting this approach works for the linear heat
equation, i.e. in the case where the reaction term $f(u)$ is absent (i.e.
$f(u)\equiv 0$). The nonlinear case is a bit more complicated because there is
no adequate definition of $f(u)$ when $u$ is a distribution rather than a
point-wise defined function. In both the standard and nonstandard proofs we
overcome this by proving that the approximating functions are Hölder
continuous, so that $f(u(x,t))$ can be defined. In the standard proof this
again allows one to use Ascoli-Arzela and extract a convergent subsequence.
However, since the domain $\mathbb{R}^{2}$ is not compact, Ascoli-Arzela
cannot be applied directly, and the standard proof therefore requires one to
apply the compactness theorem on an increasing sequence of compact subsets
$K_{i}\subset\mathbb{R}^{2}$, after which Cantor’s diagonal trick must be
invoked to get a sequence of functions that converges uniformly on every
compact subset of $\mathbb{R}^{2}$. As we show below, these issues do not come
up in the nonstandard proof.

###  1.2. Comments on nonstandard analysis

We will not even try to give an exposition of nonstandard analysis in this
paper, and instead refer the reader to the many texts that have been written
on the subject (e.g., a very incomplete list:[7, 8, 4, 11, 1, 14]).

There are a few different approaches to using the hyperreals. Keisler, in his
undergraduate nonstandard calculus text [8] gives an axiomatic description of
the hyperreals and their relation with the standard reals. In this approach
functions that are defined for standard reals automatically extend to the
hyperreals, according to the _transfer principle._ A different approach that
also begins with an axiomatic description of the hyperreals can be found in
Nelson’s “radically elementary” treatment of probability theory [11].

Our point of view in this paper is that of internal set theory as explained in
Goldblatt’s book [4] (see also Keisler’s “instructor’s guide” [7] to his
calculus text). Goldblatt explains the construction of the hyperreals using
non principal ultra filters (which can be thought of as analogous to the
construction of real numbers as equivalence classes of Cauchy sequences of
rational numbers). He then extends this construction and defines internal
sets, internal functions, etc.

### Acknowledgement

SBA has enjoyed and benefited from conversations about nonstandard analysis
with members of the logic group at Madison, and would in particular like to
thank Terry Millar for his relentless advocacy of matters both infinitesimal
and unlimited.

##  2\. Distribution solutions

###  2.1. The definition of distributions

We recall the definition of a “generalized function,” i.e. of a distribution,
which can be found in many textbooks on Real Analysis, such as Folland’s book
[3].

A real valued function $f$ on $\mathbb{R}^{2}$ is traditionally defined by
specifying its values $f(x,y)$ at each point $(x,y)\in\mathbb{R}^{2}$. In the
Theory of Distributions a generalized function $f$ is defined by specifying
its weighted averages

(6) $\langle f,\varphi\rangle=\int_{\mathbb{R}^{2}}f(x,y)\varphi(x,y)\,dx\,dy$

for all so-called “test functions” $\varphi$. A test function is any function
$\varphi:\mathbb{R}^{2}\to\mathbb{R}$ that is infinitely often differentiable,
and which vanishes outside a sufficiently large ball
$B_{R}=\\{(x,y)\in\mathbb{R}^{2}:x^{2}+y^{2}<R\\}$ whose radius $R$ is allowed
to depend on the particular test function. The set of all test functions,
which is denoted by $C_{c}^{\infty}(\mathbb{R}^{2})$, or sometimes by
$\mathcal{D}(\mathbb{R}^{2})$, is an infinite dimensional vector space. _By
definition,_ a distribution is any linear functional
$T:C_{c}^{\infty}(\mathbb{R}^{2})\to\mathbb{R}$. The most common notation for
the value of a distribution $T$ applied to a test function $\varphi$ is
$\langle T,\varphi\rangle$. For instance, if $f:\mathbb{R}^{2}\to\mathbb{R}$
is a continuous function, then the equation (6) defines $f$ as a distribution.
The canonical example of a distribution that does not correspond to a function
$f$ is the Dirac delta function, which is defined by

$\langle\delta,\varphi\rangle\stackrel{{\scriptstyle\rm
def}}{{=}}\varphi(0,0).$

The full definition of a distribution $T$ includes the requirement that the
value $\langle T,\varphi\rangle$ depend continuously on the test function
$\varphi$. To state this continuity condition precisely one must introduce a
notion of convergence in the space of test functions
$C_{c}^{\infty}(\mathbb{R}^{2})$. We refer the reader to [3] for the details,
and merely observe that a sufficient condition for a linear functional
$\varphi\mapsto\langle T,\varphi\rangle$ to be a distribution is that there
exist a constant $C$ such that

(7) $\left|\langle T,\varphi\rangle\right|\leq
C\iint_{\mathbb{R}^{2}}|\varphi(x,y)|\,dx\,dy$

holds for all test functions $\varphi$. Alternatively, if a constant $C$
exists such that

(8) $\left|\langle T,\varphi\rangle\right|\leq
C\sup_{(x,y)\in\mathbb{R}^{2}}|\varphi(x,y)|$

holds for all $\varphi\in C_{c}^{\infty}(\mathbb{R}^{2})$, then $T$ also
satisfies the definition of a distribution. The conditions (7) and (8) are not
equivalent: either one of these implies that $T$ is a distribution.

###  2.2. Distributions defined by nonstandard functions on a grid

Let $dx,dy$ be two positive infinitesimal hyperreal numbers, and let $N$, $M$
by to positive hyperintegers such that $Ndx$ and $Mdy$ are unlimited. Consider
the rectangular grid

(9) $G=\left\\{(k\,dx,l\,dy)\in{}^{*}\mathbb{R}^{2}\mid
k,l\in{}^{*}\mathbb{N},|k|\leq N,|l|\leq M\right\\}$

From the point of view of nonstandard analysis and internal set theory, $G$ is
a hyperfinite set, and for any internal function $f:G\to{}^{*}\mathbb{R}$
there is an $(x,y)\in G$ for which $f(x,y)$ is maximal.

######  Lemma 2.2.1.

If $g:\mathbb{R}^{2}\to\mathbb{R}$ is a continuous function with compact
support, then

$\int_{\mathbb{R}^{2}}g(x,y)\,dx\,dy\approx\sum_{(x,y)\in G}g(x,y)\,dx\,dy,$

where $x\approx y$ means that $x-y$ is infinitesimal.

###### Proof.

The statement of the lemma is very close to the nonstandard definition of the
Riemann integral of a continuous function, the only difference being that we
are integrating over the unbounded domain $\mathbb{R}^{2}$ rather than a
compact rectangle $[-\ell,\ell]\times[-\ell,\ell]\subset\mathbb{R}^{2}$. Since
the function $g$ has compact support, there is a real $\ell>0$ such that
$g(x,y)=0$ outside the square $[-\ell,\ell]\times[-\ell,\ell]$. By definition
we then have

$\int_{\mathbb{R}^{2}}g(x,y)\,dx\,dy=\int_{-\ell}^{\ell}\int_{-\ell}^{\ell}g(x,y)\,dx\,dy.$

Choose hyperintegers $L,L^{\prime}\in{}^{*}\mathbb{N}$ for which
$Ldx\leq\ell<(L+1)dx$ and $L^{\prime}dy\leq\ell<(L^{\prime}+1)dy$. Then the
nonstandard definition of the Riemann integral implies

$\int_{-\ell}^{\ell}\int_{-\ell}^{\ell}g(x,y)\,dx\,dy\approx\sum_{k=-L}^{L}\sum_{l=-L^{\prime}}^{L^{\prime}}g(kdx,ldy)\,dx\,dy.$

Finally, if $(x,y)\in G$ then $g(x,y)=0$ unless $|x|\leq\ell$ and
$|y|\leq\ell$, so that

$\sum_{k=-L}^{L}\sum_{l=-L^{\prime}}^{L^{\prime}}g(kdx,ldy)\,dx\,dy=\sum_{(x,y)\in
G}g(x,y)\,dx\,dy.$

∎

######  Lemma 2.2.2.

Suppose that $f:G\to{}^{*}\mathbb{R}$ is a hyperreal valued function which is
bounded, in the sense that there exists a limited $C>0$ such that
$|f(x,y)|\leq C$ for all $(x,y)\in G$. Then the expression

(10) $\langle T_{f},\varphi\rangle\stackrel{{\scriptstyle\rm
def}}{{=}}\mathop{\mathrm{St}}\left(\sum_{(x,y)\in
G}f(x,y)\varphi(x,y)\,dx\,dy\right)$

defines a distribution on $\mathbb{R}^{2}$.

If the function $f$ is the nonstandard extension of a (standard) continuous
function $f:\mathbb{R}^{2}\to\mathbb{R}$, then the distribution $T_{f}$
coincides with the distribution defined by (6).

###### Proof.

We first verify that the distribution is well defined. Since $|f(x,y)|\leq C$
for all $(x,y)$ we have

$\left|\sum_{(x,y)\in G}f(x,y)\varphi(x,y)\,dx\,dy\right|\leq C\sum_{(x,y)\in
G}|\varphi(x,y)|\,dx\,dy\approx C\int_{\mathbb{R}^{2}}|\varphi(x,y)|\,dx\,dy.$

Hence the sum in the definition (10) of $\langle T_{f},\varphi\rangle$ is a
limited hyperreal, whose standard part is a well defined real number which
satisfies

$|\langle T_{f},\varphi\rangle|\leq
C\int_{\mathbb{R}^{2}}|\varphi(x,y)|\,dx\,dy.$

Therefore $T_{f}$ is a well defined distribution.

Let $\langle f,\varphi\rangle$ be defined as in (6). Fix $\varphi$. We then
have

$\int_{\mathbb{R}^{2}}f(x,y)\varphi(x,y)\,dx\,dy\approx\sum_{(x,y)\in
G}f(x,y)\varphi(x,y)\,dx\,dy,$

which implies that the distribution defined in (6) coincides with $T_{f}$. ∎

##  3\. The Cauchy problem for the heat equation

In this section we recall the definition of distribution solutions to the
Cauchy problem for the heat equation and show how, by solving the finite
difference approximation to the heat equation on a hyperfinite grid, one can
construct a distribution solution to the Cauchy problem.

###  3.1. Formulation in terms of distributions

We consider the Cauchy problem for the linear heat equation $u_{t}=u_{xx}$
with bounded and continuous initial data $u(x,0)=u_{0}(x)$.

######  Definition 3.1.1.

A distribution $u$ on $\mathbb{R}^{2}$ is a solution to the heat equation
$u_{t}=u_{xx}$ with initial data $u_{0}$ if $u$ satisfies

(11) $u_{t}-u_{xx}=u_{0}(x)\delta(t),\quad x\in\mathbb{R},t\in\mathbb{R}$

in the sense of distributions, and if $u=0$ for $t\leq 0$.

Equality in the sense of distributions in (11) means that both sides of the
equation are to be interpreted as distributions, and that they should yield
the same result when evaluated on any test function $\varphi\in
C_{c}^{\infty}(\Omega)$. To explain this in more detail, recall that $\delta$
is Dirac’s delta function, so that the action of the right hand side in (11)
on a test function is

$\langle u_{0}(x)\delta(t),\varphi\rangle\stackrel{{\scriptstyle\rm
def}}{{=}}\int_{\mathbb{R}}u_{0}(x)\varphi(x,0)\,dx.$

The definition of distributional derivative [3, ch. 9] says that the action of
the left hand side in (11) is given by

$\langle u_{t}-u_{xx},\varphi\rangle=\langle
u,-\varphi_{t}-\varphi_{xx}\rangle.$

If the distribution $u$ is given by a function $u:\mathbb{R}^{2}\to\mathbb{R}$
which vanishes for $t<0$, and is continuous for $t\geq 0$ (so that it has a
simple jump discontinuity at $t=0$) then we get

$\langle
u_{t}-u_{xx},\varphi\rangle=\int_{\mathbb{R}}\int_{0}^{\infty}u(x,t)\bigl{\\{}-\varphi_{t}-\varphi_{xx}\bigr{\\}}\,dt\,dx.$

A piecewise continuous function $u$ therefore satisfies (11) in the sense of
distributions if

(12)
$\int_{\mathbb{R}}u_{0}(x)\varphi(x,0)\,dt+\int_{\mathbb{R}}\int_{0}^{\infty}u(x,t)\bigl{\\{}\varphi_{t}+\varphi_{xx}\bigr{\\}}\,dt\,dx=0$

for all test functions $\varphi\in C_{c}^{\infty}(\mathbb{R}^{2})$. This is
one form of the classical definition of a weak solution to the Cauchy problem.

###  3.2. The finite difference equation

To construct a distribution solution to (11) we introduce a grid with spacing
$dx$ and $dt$, and replace the differential equation by the simplest finite
difference scheme that appears in numerical analysis. If $u$ is the solution
to the differential equation, then we write $U$ for the approximating solution
to the finite difference equation, using the following common notation for
finite differences:

$\displaystyle D_{x}^{+}U(x,t)=\frac{U(x+dx,t)-U(x,t)}{dx},\qquad
D_{x}^{-}U(x,t)=\frac{U(x,t)-U(x-dx,t)}{dx}$ $\displaystyle
D_{t}^{+}U(x,t)=\frac{U(x,t+dt)-U(x,t)}{dt}.$

See for example, [10, chapter 1]. With this notation

$D^{2}_{x}U(x,t)\stackrel{{\scriptstyle\rm
def}}{{=}}D_{x}^{+}D_{x}^{-}U(x,t)=\frac{U(x+dx,t)-2U(x,t)+U(x-dx,t)}{(dx)^{2}}$

The operators $D^{+}_{x}$, $D^{-}_{x}$, and $D_{t}^{+}$ all commute. A finite
difference equation corresponding to the heat equation $u_{t}=u_{xx}$ is then
$D^{+}_{t}U=D^{2}_{x}U$, i.e.

(13)
$\frac{U(x,t+dt)-U(x,t)}{dt}=\frac{U(x+dx,t)-2U(x,t)+U(x-dx,t)}{(dx)^{2}}$

We can solve this algebraic equation for $U(x,t+dt)$, resulting in

(14) $U(x,t+dt)=\alpha U(x-dx,t)+(1-2\alpha)U(x,t)+\alpha U(x+dx,t)$

where

$\alpha\stackrel{{\scriptstyle\rm def}}{{=}}\frac{dt}{(dx)^{2}}.$

###  3.3. The approximate solution

Let $N\in{}^{*}\mathbb{N}$ be an unlimited hyperfinite integer, and assume
that $dx$ and $dt$ are positive infinitesimals. Assume moreover that $N$ is so
large that both $Ndt$ and $Ndx$ are unlimited hyperreals. We then consider the
hyperfinite grid

$G_{C}=\\{(m\,dx,n\,dt)\mid m,n\in{}^{*}\mathbb{N},|m|+n\leq N\\}.$

The initial function $u_{0}:\mathbb{R}\to\mathbb{R}$ extends to an internal
function $u_{0}:{}^{*}\mathbb{R}\to{}^{*}\mathbb{R}$. By assumption there is a
$C\in\mathbb{R}$ such that $|u_{0}(x)|\leq C$ for all $x\in\mathbb{R}$, so
this also holds for all $x\in{}^{*}\mathbb{R}$.

We define $U:G_{C}\to{}^{*}\mathbb{R}$ by requiring

  * •

$U(x,0)=u_{0}(x)$ for all $x$ with $(x,0)\in G_{C}$, i.e. for all $x=kdx$ with
$k\in\\{-N,\dots,+N\\}$.

  * •

$U$ satisfies (4), or, equivalently, (14) at all $(x,t)=(mdx,ndt)\in G_{C}$
with $|m|+n<N$.

Figure 1. The triangular grid $G_{C}$; if $U(x,t)$ is known at all grid points
at the bottom of the triangle, then the finite difference equation (13)
uniquely determines $U(x,t)$ at all other grid points.

######  Theorem 3.3.1.

Let $U:G_{C}\to{}^{*}\mathbb{R}$ be the hyperreal solution of the finite
difference scheme (13) with initial values $U(x,0)=u_{0}(x)$, and suppose that
$\alpha\leq\frac{1}{2}$. Then the expression

(15) $\langle u,\varphi\rangle\stackrel{{\scriptstyle\rm
def}}{{=}}\mathop{\mathrm{St}}\left(\sum
U(x,t)\varphi(x,t)\,dx\,dt\right)\qquad(\varphi\in
C_{c}^{\infty}(\mathbb{R}^{2}))$

defines a distribution on $\mathbb{R}^{2}$ that satisfies (11).

To show that this expression does indeed define a distribution we must show
that the $U(x,t)$ are bounded by a standard real number. This follows from a
discrete version of the maximum principle, which we will again use in section
4, so we state it in slightly greater generality than needed in this section.
The lemma explains why we need the condition
$\alpha=dt/(dx)^{2}\leq\frac{1}{2}$, and is well known in numerical analysis
as a necessary condition for stability of the finite difference scheme.

######  Lemma 3.3.2 (Gronwall type estimate).

Let $W:G_{C}\to{}^{*}\mathbb{R}$ satisfy $W(x,t)\geq 0$ for all $(x,t)\in
G_{C}$, and suppose that for some nonnegative $m\in\mathbb{R}$ one has

$W(x,t+dt)\leq\alpha W(x+dx,t)+(1-2\alpha)W(x,t)+\alpha
W(x-dx,t)+dt\,mW(x,t),$

at all $(x,t)\in G_{C}$. For each $t=n\,dt$ with $0\leq n\leq N$ consider

$w(t)\stackrel{{\scriptstyle\rm def}}{{=}}\max_{x}W(x,t).$

If $0\leq\alpha\leq\frac{1}{2}$ then

$w(t)\leq e^{mt}w(0).$

###### Proof.

(Compare [6, §7.2–Lemma I].) The assumption on $\alpha$ implies that
$\alpha\geq 0$ and $1-2\alpha\geq 0$. Hence for all $x$ with $(x,t+dt)\in
G_{C}$ we have

$\displaystyle W(x,t+dt)$ $\displaystyle=\alpha
W(x+dx,t)+(1-2\alpha)W(x,t)+\alpha W(x-dx,t)+dt\,mW(x,t)$
$\displaystyle\leq\bigl{(}\alpha+(1-2\alpha)+\alpha+m\,dt\bigr{)}w(t)$
$\displaystyle=(1+m\,dt)w(t).$

Taking the maximum over $x$ we see that $w(t+dt)\leq(1+m\,dt)w(t)\leq
e^{m\,dt}w(t)$. By induction we then have for $t=n\,dt$ that
$w(t)\leq(e^{mdt})^{n}w(0)=e^{mt}w(0)$. ∎

###  3.4. Proof of Theorem 3.3.1

The relation (14) which defines $U(x,t)$ implies that
$W(x,t)\stackrel{{\scriptstyle\rm def}}{{=}}|U(x,t)|$ satisfies

$\displaystyle W(x,t+dt)$ $\displaystyle=\bigl{|}\alpha
U(x-dx,t)+(1-2\alpha)U(x,t)+\alpha U(x+dx,t)\bigr{|}$ $\displaystyle\leq\alpha
W(x-dx,t)+(1-2\alpha)W(x,t)+\alpha W(x+dx,t),$

where we have used $\alpha\geq 0$ and $1-2\alpha\geq 0$.

Since the initial condition is bounded by $|U(x,0)|=|u_{0}(x)|\leq M$ the
Gronwall type lemma 3.3.2 implies that $|U(x,t)|\leq M$ for all $(x,t)\in
G_{C}$. According to Lemma 2.2.2 this implies that the expression (15) does
define a distribution $u$ on $\mathbb{R}^{2}$.

We want to prove that $u$ satisfies the heat equation in the sense of
distributions, i.e. we want to show for any test function $\varphi$ that

$\langle u_{t}-u_{xx},\varphi\rangle=\langle
u_{0}(x)\delta(t),\varphi\rangle=\int_{\mathbb{R}}u_{0}(x)\varphi(x,0)\,dx.$

First, we see from the definition of distributional derivative that

$\langle u_{t}-u_{xx},\varphi\rangle=-\langle
u,\varphi_{t}+\varphi_{xx}\rangle.$

We then have from the definition of $u$ that

$-\langle u,\varphi_{t}+\varphi_{xx}\rangle\approx\sum_{(x,t)\in
G_{C}}-U(x,t)(\varphi_{t}(x,t)+\varphi_{xx}(x,t))\,dx\,dt\stackrel{{\scriptstyle\rm
def}}{{=}}T.$

Using Taylor’s formula we replace the partial derivatives of the test function
with its corresponding finite differences, i.e. we write
$\varphi_{t}(x,t)=D_{t}^{+}\varphi(x,t)+\varepsilon_{t}(x,t)$ and
$\varphi_{xx}(x,t)=D_{x}^{2}\varphi(x,t)+\varepsilon_{xx}(x,t)$, where
$\varepsilon_{t},\varepsilon_{xx}:G_{C}\to{}^{*}\mathbb{R}$ are the
infinitesimal error terms in the Taylor expansion. Substituting these gives us

$T=\textstyle\sum_{G_{C}}-U(x,t)\big{(}D_{t}^{+}\varphi(x,t)+D_{x}^{2}\varphi(x,t)+\varepsilon_{t}+\varepsilon_{xx}\big{)}\,dx\,dt.$

We can split this sum into three parts, $T=T_{1}+T_{2}+T_{3}$, with

$\displaystyle T_{1}$
$\displaystyle=\textstyle\sum_{G_{C}}-U(x,t)D_{t}^{+}\varphi(x,t)\,dx\,dt$
$\displaystyle T_{2}$
$\displaystyle=\textstyle\sum_{G_{C}}-U(x,t)D_{x}^{2}\varphi(x,t)\,dx\,dt$
$\displaystyle T_{3}$
$\displaystyle=\textstyle\sum_{G_{C}}-U(x,t)(\varepsilon_{t}+\varepsilon_{xx})\,dx\,dt$

We will first handle the error term, $T_{3}$. Since the test function
$\varphi$ has compact support, there exists a real $\ell>0$ such that
$\varphi=0$ outside the rectangle $\Omega=[-\ell,\ell]\times[-\ell,\ell]$. The
errors in the Taylor expansion therefore also vanish outside of $\Omega$ so
that we can write $T_{3}$ as

$T_{3}=\textstyle\sum_{\Omega\cap
G_{C}}-U(x,t)(\varepsilon_{t}+\varepsilon_{xx})\,dx\,dt$

The key to estimating this sum is that we can estimate all the errors
$\varepsilon_{t}(x,t)$ and $\varepsilon_{xx}(x,t)$ by one fixed infinitesimal
$\varepsilon>0$ that does not depend on $(x,t)$. Indeed, the grid $G_{C}$ is a
hyperfinite internal set, and therefore any internal function such as
$|\varepsilon_{t}|:G_{C}\to{}^{*}\mathbb{R}$ attains its largest value at one
of the $(x,t)\in G_{C}$, say at $(x_{1},t_{1})$. Then
$|\varepsilon_{t}(x,t)|\leq|\varepsilon_{t}(x_{1},t_{1})|$ for all $(x,t)\in
G_{C}$. Similarly, there is an $(x_{2},t_{2})\in G_{C}$ that maximizes
$|\varepsilon_{xx}(x,t)|$. Now define

$\varepsilon_{1}=|\varepsilon_{t}(x_{1},t_{1})|,\qquad\varepsilon_{2}=|\varepsilon_{xx}(x_{2},t_{2})|.$

Then both $\varepsilon_{1}$ and $\varepsilon_{2}$ are positive infinitesimals
for which

$|\varepsilon_{t}(x,t)|\leq\varepsilon_{1},\qquad|\varepsilon_{xx}(x,t)|\leq\varepsilon_{2}$

holds at all grid points $(x,t)\in G_{C}$.

If we let $\varepsilon=\max\\{\varepsilon_{1},\varepsilon_{2}\\}$, then
$|T_{3}|\leq\sum_{G_{C}\cap\Omega}2U(x,t)\varepsilon\,dx\,dt$. By the
construction of $U$, we have $|U(x,t)|\leq M$ for all $(x,t)\in G_{C}$, so we
get

$|T_{3}|\leq\sum_{G_{C}\cap\Omega}2M\varepsilon\,dx\,dt\leq
2M\varepsilon\,dx\,dt\frac{\ell}{2\,dx}\frac{\ell}{\,dt}=M\ell^{2}\varepsilon,$

which is infinitesimal, so $T_{3}$ is infinitesimal.

From the definition we have
$T_{1}=\sum_{G_{C}}-U(x,t)(\varphi(x,t+dt)-\varphi(x,t))\,dx$. Using the
compact support of $\varphi$, we can then rewrite this sum as

$T_{1}=-\sum_{k=-K}^{K}\sum_{l=0}^{L+1}U(k\,dx,l\,dt)\bigl{\\{}\varphi(k\,dx,(l+1)\,dt)-\varphi(k\,dx,l\,dt)\bigr{\\}}\,dx,$

where $K\,dx\approx\ell$ and $L\,dt\approx\ell$.

Applying summation by parts to this sum we then get

$T_{1}=\sum_{k=-K}^{K}\Bigl{\\{}U(k\,dx,0)\varphi(k\,dx,0)+\sum_{l=0}^{L}\varphi\bigl{(}k\,dx,(l+1)\,dt\bigr{)}D_{t}^{+}U(k\,dx,l\,dt)\,dt\Bigr{\\}}\,dx$

Next, for $T_{2}$, we can split the sum into two parts.

(16)
$T_{2}=\sum_{l=0}^{L}\sum_{k=-K-1}^{K+1}-U(k\,dx,l\,dt)D_{x}^{+}\varphi(k\,dx,l\,dt)\,dt\,dx\\\
+\sum_{l=0}^{L}\sum_{k=-K-1}^{K+1}U(k\,dx,l\,dt)D_{x}^{-}\varphi(k\,dx,l\,dt)\,dt\,dx$

Applying summation by parts again to both sums we get

$T_{2}=-\sum_{l=0}^{L}\sum_{k=-K}^{K}\varphi(k\,dx,l\,dt)D_{x}^{2}U(k\,dx,l\,dt)\,dx\,dt.$

Putting the terms $T_{1},T_{2},T_{3}$ all together, we have, because
$T_{3}\approx 0$,

(17) $T_{1}+T_{2}+T_{3}\approx
T_{1}+T_{2}=\sum_{k=-K}^{K}U(k\,dx,0)\varphi(k\,dx,0)\,dx\\\
+\sum_{k=-K}^{K}\sum_{l=0}^{L}\varphi(k\,dx,l\,dt)\bigg{(}D_{t}^{+}U(k\,dx,l\,dt)-D_{x}^{2}U(k\,dx,l\,dt)\bigg{)}\,dx\,dt.$

Since $U$ satisfies the difference equation $D_{t}^{+}U=D^{2}_{x}U$ at all
grid points this reduces to

$T=T_{1}+T_{2}+T_{3}\approx\sum_{k=-K}^{K}U(k\,dx,0)\varphi(k\,dx,0)\,dx\approx\sum_{k=-K}^{K}u_{0}(k\,dx)\varphi(k\,dx,0)\,dx.$

Taking the standard part we get the distribution

$\mathop{\mathrm{St}}(T)=\mathop{\mathrm{St}}\left(\sum_{k=-K}^{K}u_{0}(k\,dx)\varphi(k\,dx,0)\,dx\right)=\int_{-\ell}^{\ell}u_{0}(x)\varphi(x,0)\,dx=\big{\langle}u_{0}(x)\delta(t),\varphi\big{\rangle}.$

This completes the proof that $\langle u_{t}-u_{xx},\varphi\rangle=\langle
u_{0}(x)\delta(t),\varphi\rangle$ for all test functions $\varphi$, and thus
that $u$ is a distributional solution of (11).

###  3.5. Comments on the proof

In our construction of solutions to the linear heat equation we completely
avoided estimating derivatives of the approximate solution $U$. The only
estimate we used was that the approximate solution $U(x,t)$ has the same upper
bound as the given initial function $u_{0}$.

We assumed that the initial function $u_{0}$ is continuous. The one place in
the proof where we needed this assumption was at the end, when we used the
fact that for continuous functions $f:\mathbb{R}\to\mathbb{R}$ one has

$\textstyle\sum_{|k|\leq K}f(k\,dx)\,dx\approx\int_{-\ell}^{\ell}f(x)\,dx$

and applied this to the function $f(x)=u_{0}(x)\varphi(x,0)$.

##  4\. The Cauchy problem for a Reaction Diffusion Equation

We consider the reaction diffusion equation

(1a) $\frac{\partial u}{\partial t}=\frac{\partial^{2}u}{\partial
x^{2}}+f(u(x,t)),\qquad(x\in\mathbb{R},t>0)$

with initial condition (1b). It is known that one cannot expect solutions to
exist for all time $t>0$ without imposing some growth conditions on the
nonlinearity $f(u)$. We will assume that $f$ is a Lipschitz continuous
function, i.e. that for some positive real $K_{1}$ one has

(18) $\forall u,v\in\mathbb{R}:\quad|f(u)-f(v)|\leq K_{1}|u-v|.$

This implies that $f(u)$ grows at most linearly in $u$:

(19) $\forall u\in\mathbb{R}:\quad|f(u)|\leq K_{0}+K_{1}|u|$

where $K_{0}\stackrel{{\scriptstyle\rm def}}{{=}}|f(0)|$.

In contrast to the linear heat equation, (1a) contains the nonlinear term
$f(u)$ which is meaningless if $u$ is an arbitrary distribution. One can
follow the same procedure as in the previous section, i.e. one can replace the
differential equation by a finite difference scheme on the hyperfinite grid
$G_{C}$ and construct an approximating solution $U:G_{C}\to{}^{*}\mathbb{R}$.
After establishing suitable bounds one can then show that by taking standard
parts as in Lemma 2.2.2, both $U(x,t)$ and $f(U(x,t))$ define distributions
$u$ and $F$ on $\mathbb{R}^{2}$. The problem is to give a meaning to the claim
that “$F=f(u)$,” because $u$ is merely a distribution and can therefore not be
substituted in a nonlinear function. In this section we show how to overcome
this problem by adding the assumption that the initial function is Lipschitz
continuous, i.e.

(20) $\forall x,y\in\mathbb{R}:\quad|u_{0}(x)-u_{0}(y)|\leq L|x-y|$

for some real $L>0$, and showing that the standard part of the approximating
solution $U$ is a continuous function on $\mathbb{R}\times[0,\infty)$. The
substitution $f(U(x,t))$ is then well defined and we can verify that the
continuous standard function corresponding to $U$ is a distributional solution
of the reaction diffusion equation (1a).

###  4.1. Weak Solutions to the Reaction Diffusion Equation

Rather than writing the initial value problem in the distributional form
$u_{t}-u_{xx}-f(u)=u_{0}(x)\delta(t)$, we use the integral version (12) of the
definition of weak solution. Thus we define a _weak solution_ to (1a), (1b) to
be a continuous function $u:\mathbb{R}\times[0,\infty)\to\mathbb{R}$ that
satisfies

(21)
$\iint_{\mathbb{R}\times[0,\infty)}\bigl{\\{}u(x,t)(-\varphi_{xx}-\varphi_{t})-f(u(x,t))\varphi\bigr{\\}}\,dx\,dt=\int_{\mathbb{R}}u_{0}(x)\varphi(x,0)\,dx,$

for all test functions $\varphi\in C_{c}^{\infty}(\mathbb{R}^{2})$.

######  Theorem 4.1.1.

If $f$ is Lipschitz continuous as in (18), and if the initial function $u_{0}$
is bounded by

$\forall x\in\mathbb{R}:\quad|u_{0}(x)|\leq M,$

for some positive real $M$, and if $u_{0}$ also is Lipschitz continuous, as in
(20), then the reaction diffusion equation (1a),(1b) has a weak solution.

###  4.2. Definition of the approximate solution

To construct the solution we consider the grid $G_{C}$ as defined in § 3.3
with infinitesimal mesh sizes $dx,dt>0$, and consider the finite difference
scheme

(22) $D^{+}_{t}U(x,t)=D^{2}_{x}U(x,t)+f(U(x,t)).$

Solving for $U(x,t+dt)$ we get

(23) $U(x,t+\,dt)=\alpha U(x+\,dx,t)+(1-2\alpha)U(x,t)+\alpha U(x-\,dx,t)\\\
+dt\,f(U(x,t)),$

where, as before, $\alpha=dt/(dx)^{2}$. We extend the continuous function
$u_{0}$ to an internal function $u_{0}:{}^{*}\mathbb{R}\to{}^{*}\mathbb{R}$,
and specify the initial conditions $U(x,0)=u_{0}(x)$ for $x=m\,dx$,
$m=-N,\dots,+N$. The finite difference equation (23) then determines $U(x,t)$
for all $(x,t)\in G_{C}$.

We now establish a number of _a priori_ estimates for the approximate solution
$U$ that will let us verify that its standard part is well defined and that it
is a weak solution of the initial value problem.

###  4.3. Boundedness of the approximate solution

First we establish a bound for $|U(x,t)|$.

######  Lemma 4.3.1.

For all $(x,t)\in G_{C}$ we have

(24) $|U(x,t)|\leq e^{K_{1}t}M+\frac{K_{0}}{K_{1}}(e^{K_{1}t}-1).$

###### Proof.

Using (19), i.e. $|f(u)|\leq K_{0}+K_{1}|u|$, we get

(25)
$|U(x,t+dt)|\leq\alpha|U(x+dx,t)|+(1-2\alpha)|U(x,t)|+\alpha|U(x-dx,t)|\\\
+dt(K_{0}+K_{1}|U(x,t)|).$

In terms of $M(t)=\max_{x}|U(x,t)|$ this implies

$M(t+dt)\leq M(t)+dt(K_{0}+K_{1}M(t))=(1+K_{1}dt)M(t)+K_{0}dt.$

Setting $t=n\,dt$ we see that this is an inequality of the form $M_{n}\leq
aM_{n-1}+b$ with $M_{n}=M(ndt)$. By induction this implies that

$\displaystyle M(t)$
$\displaystyle=M(ndt)\leq(1+K_{1}dt)^{n}M(0)+\frac{(1+K_{1}dt)^{n}-1}{1+K_{1}dt-1}K_{0}dt$
$\displaystyle\leq e^{K_{1}t}M(0)+\frac{K_{0}}{K_{1}}(e^{K_{1}t}-1).$

Since $M(0)=M$ this proves the (24). ∎

###  4.4. Lipschitz continuity in space of the approximate solution

We now show that $U(x,t)$ is Lipschitz continuous in the space variable.

######  Lemma 4.4.1.

For any two points $(x,t),(x^{\prime},t)\in G_{C}$ we have

(26) $|U(x,t)-U(x^{\prime},t)|\leq Le^{K_{1}t}|x-x^{\prime}|,$

where $L$ is the Lipschitz constant for the initial function $u_{0}$, as in
(20).

###### Proof.

Let

$V(x,t)\stackrel{{\scriptstyle\rm
def}}{{=}}\frac{U(x+\,dx,t)-U(x,t)}{dx}=D_{x}^{+}U(x,t).$

Applying $D_{x}^{+}$ to both sides of the equation (4) for $U$, and using the
definition of $V$ and commutativity of the difference quotient operators, we
find

$D_{t}^{+}V=D_{x}^{2}V+D_{x}^{+}f(U).$

Solving for $V(x,t+dt)$ we find

$V(x,t+dt)=\alpha V(x+dx,t)+(1-2\alpha)V(x,t)+\alpha V(x-dx,t)+D_{x}^{+}f(U)$

Examining $D_{x}^{+}f(U)$, we have

$|D_{x}^{+}f(U)|=\frac{|f(U(x+\,dx,t))-f(U(x,t))|}{dx}\leq\\\
\frac{K_{1}\,|U(x+\,dx,t)-U(x,t)|}{dx}=K_{1}|V(x,t)|,$

so that

$|V(x,t+dt)|\leq\alpha|V(x+dx,t)|+(1-2\alpha)|V(x,t)|+\alpha|V(x-dx,t)|+K_{1}dt|V(x,t)|.$

Using Gronwall’s Inequality on $\max\limits_{x}V$, we get the inequality

$\max_{x}|V(x,t)|\leq e^{K_{1}t}\max_{x}|V(x,0)|.$

The initial condition $u_{0}$ satisfies $|u_{0}(x)-u_{0}(x^{\prime})|\leq
L|x-x^{\prime}|$ for all $x,x^{\prime}\in\mathbb{R}$, and therefore the
extension of $u_{0}$ to the hyperreals satisfies this same inequality.
Therefore $|V(x,0)|\leq L$ for all grid points $(x,0)$, and thus we have
$|V(x,t)|\leq Le^{K_{1}t}$. This implies (26). ∎

###  4.5. Hölder continuity in time of the approximate solution

######  Lemma 4.5.1.

Given any real $\bar{t}>0$ we have for any two grid points
$(x_{0},t_{0}),(x_{0},t_{1})\in G_{C}$ with $0\leq t_{0}\leq t_{1}\leq\bar{t}$

(27) $|U(x_{0},t_{1})-U(x_{0},t_{0})|\leq C\sqrt{t_{1}-t_{0}}$

where $C$ is a constant that only depends on $\bar{t}$, $K_{0}$, $K_{1}$, $L$,
and $M$.

###### Proof.

We begin by oberving that $f(U(x,t))$ is bounded on the time interval we are
considering. Indeed, for $t\leq\bar{t}$ we have shown for all $(x,t)\in G_{C}$
that

$|U(x,t)|\leq A_{0}\stackrel{{\scriptstyle\rm
def}}{{=}}e^{K_{1}\bar{t}}M+\frac{K_{1}}{K_{0}}\left(e^{K_{1}\bar{t}}-1\right),$

while the Lipschitz condition for $f$ implies $|f(U(x,t))|\leq
K_{0}+K_{1}|U|\leq K_{0}+K_{1}A_{0}$. So if we set $A=K_{0}+K_{1}A_{0}$, then
we have

(28) $\forall(x,t)\in G_{C}\text{ with }t\leq\bar{t}:\quad|f(U(x,t))|\leq A.$

Next, we construct a family of upper-barriers for $U$ using parabolas. In
particular, for any real $a,b,c>0$ we consider

$\bar{U}(x,t)\stackrel{{\scriptstyle\rm
def}}{{=}}U(x_{0},t_{0})+a(t-t_{0})+\frac{b}{2}(x-x_{0})^{2}+c.$

For any $b>0$ we will find $a,c>0$ so that $\bar{U}$ is an upper barrier, in
the sense that

(29) $D_{t}^{+}\bar{U}-D^{2}_{x}\bar{U}\geq A+1,$

and

(30) $\bar{U}(x,0)>U(x,0)\text{ for all }(x,0)\in G_{C}.$

A direct computation shows that $D_{t}^{+}\bar{U}-D^{2}_{x}\bar{U}=a-b$, so
for given $b$ we choose $a=b+A+1$ and (29) will hold.

To satisfy (30) we use (26), i.e. that $U(x,t)$ is Lipschitz continuous with
Lipschitz constant $\bar{L}\stackrel{{\scriptstyle\rm
def}}{{=}}e^{K_{1}\bar{t}}L$:

$U(x,t)\leq U(x_{0},t_{0})+\bar{L}|x-x_{0}|\leq
U(x_{0},t_{0})+\frac{2}{b}(x-x_{0})^{2}+\frac{\bar{L}^{2}}{2b}.$

If we choose $c>\bar{L}^{2}/2b$, e.g. $c=\bar{L}^{2}/b$, then our upper
barrier $\bar{U}$ also satisfies (30).

Next, we apply a maximum principle argument to compare $U$ and $\bar{U}$.
Consider $W(x,t)=U(x,t)-\bar{U}(x,t)$. Then we have shown that $W(x,0)<0$ for
all $x$, and $D_{t}^{+}W-D_{x}^{2}W<0$, which implies

$W(x,t+dt)<\alpha W(x-dx,t)+(1-2\alpha)W(x,t)+\alpha W(x+dx,t)$

for all $(x,t)$ for which $(x\pm dx,t)\in G_{C}$. By induction we get
$W(x,t)<0$ for all $(x,t)\in G_{C}$. In particular
$U(x_{0},t)<\bar{U}(x_{0},t)$ for all $t>t_{0}$, i.e. we have shown

$U(x_{0},t_{1})<U(x_{0},t_{0})+(b+A+1)(t_{1}-t_{0})+\frac{\bar{L}^{2}}{b}.$

This upper bound holds for any choice of $b>0$. To get the best upper bound we
minimize the right hand side over all $b>0$. The best bound appears when
$b=\bar{L}/\sqrt{t_{1}-t_{0}}$. After some algebra one then finds

$U(x_{0},t_{1})-U(x_{0},t_{0})<(A+1)(t_{1}-t_{0})+2\bar{L}\sqrt{t_{1}-t_{0}}.$

Finally, using
$t_{1}-t_{0}=\sqrt{t_{1}-t_{0}}\sqrt{t_{1}-t_{0}}\leq\sqrt{\bar{t}}\sqrt{t_{1}-t_{0}}$
we get

$U(x_{0},t_{1})-U(x_{0},t_{0})<\bigl{(}(A+1)\sqrt{\bar{t}}+2\bar{L}\bigr{)}\sqrt{t_{1}-t_{0}}.$

This proves the upper bound in (27). To get the analogous lower bound one
changes the signs of the coefficients $a,b,c$ which will turn $\bar{U}$ into a
lower barrier. After working through the details one finds the appropriate
lower bound. ∎

Now that we have Lipschitz in space and Hölder in time, we also have for any
pair of points $(x,t),(y,s)\in G_{C}$ that

(31) $|U(x,t)-U(y,s)|\leq|U(x,t)-U(y,t)|+|U(y,t)-U(y,s)|\\\ \leq
L|x-y|+C\sqrt{|t-s|}.$

###  4.6. Definition of the weak solution

So far we have been establishing estimates for the solution $U$ to the finite
difference scheme. It is worth pointing out that a standard existence proof
would have required exactly the same estimates. At this point however, the
standard and nonstandard proofs diverge.

For $(x,t)\in\mathbb{R}\times[0,\infty)$ we choose $(\tilde{x},\tilde{t})\in
G_{C}$ with $x\approx\tilde{x}$ and $t\approx\tilde{t}$, and then define
$u(x,t)=\mathop{\mathrm{St}}(U(\tilde{x},\tilde{t}))$. The continuity property
(31) of the approximate solution $U$ implies that the value of
$\mathop{\mathrm{St}}(U(\tilde{x},\tilde{t}))$ does not depend on how we chose
the grid point $(\tilde{x},\tilde{t})$, for if $(\hat{x},\hat{t})\in G_{C}$
also satisfied $\hat{x}\approx x$, $\hat{t}\approx t$, then
$\tilde{x}\approx\hat{x}$ and $\tilde{t}\approx\hat{t}$, so that
$U(\tilde{x},\tilde{t})\approx U(\hat{x},\hat{t})$. It follows directly that
the function $u:\mathbb{R}\times[0,\infty)\to\mathbb{R}$ is well defined, that
it satisfies the continuity condition (31), and that it satisfies the same
bounds as in (24).

By the transfer principle the standard function $u$ extends in a unique way to
an internal function
${}^{*}\mathbb{R}\times{}^{*}[0,\infty)\to{}^{*}\mathbb{R}$. It is common
practice to abuse notation and use the same symbol $u$ for the extension. The
extended function satisfies the same continuity condition (31).

######  Lemma 4.6.1.

If $(x,t)\in G_{C}$ is limited, then $u(x,t)\approx U(x,t)$.

###### Proof.

If $(x,t)$ is limited, then $x^{\prime}=\mathop{\mathrm{St}}(x)$ and
$t^{\prime}=\mathop{\mathrm{St}}(t)$ are well defined real numbers. By
continuity of both $u$ we have $u(x,t)\approx u(x^{\prime},t^{\prime})$. By
definition of $u$ it follows from $x^{\prime}\approx x$ and $t^{\prime}\approx
t$ that $u(x^{\prime},t^{\prime})\approx U(x,t)$. Combined we get
$u(x,t)\approx U(x,t)$. ∎

###  4.7. Proof that $u$ is a weak solution

We will now show that $u$ is a weak solution whose existence is claimed in
Theorem 4.1.1, i.e. we verify that $u$ satisfies (21) for any test function
$\varphi\in C_{c}^{\infty}(\mathbb{R}^{2})$.

Since $\varphi$ has compact support, there is a positive real $\ell$ such that
$\varphi(x,t)=0$ outside the square $[-\ell,\ell]\times[-\ell,\ell]$. We
therefore have to verify

$\int_{0}^{\ell}\int_{-\ell}^{\ell}\left\\{u(x,t)(-\varphi_{xx}-\varphi_{t})-f(u(x,t))\varphi\right\\}\,dx\,dt=\int_{-\ell}^{\ell}u_{0}(x)\varphi(x,0)\,dx.$

Since the integrands are continuous functions we only make an infinitesimal
error when we replace the two Riemann integrals by Riemann sums over the part
of the grid $G_{C}$ that lies within the square
$[-\ell,\ell]\times[-\ell,\ell]$. Thus we must prove

(32) $\sum_{(x,t)\in
G_{C}}\bigl{\\{}u(x,t)(-\varphi_{xx}-\varphi_{t})-f(u(x,t))\varphi\bigr{\\}}\,dx\,dt\approx\sum_{(x,0)\in
G_{C}}u_{0}(x)\varphi(x,0)\,dx.$

We now intend to replace $u$ by $U$, and the derivatives of $\varphi$ by the
corresponding finite differences. In doing so we make errors that we must
estimate. Let $G_{C\ell}=G_{C}\cap[-\ell,\ell]^{2}$, so that the only nonzero
terms in the two sums come from terms evaluated at points in $G_{C\ell}$. The
intersection of internal sets is again internal, so the set $G_{C\ell}$ is
internal and hyperfinite.

For each $(x,t)\in G_{C\ell}$ the quantities

$|u(x,t)-U(x,t)|,\quad|\varphi_{t}(x,t)-D_{t}^{+}\varphi(x,t)|,\text{ and
}|\varphi_{xx}(x,t)-D^{2}_{x}\varphi(x,t)|$

are infinitesimal. Since they are defined by internal functions, one of the
numbers in the hyperfinite set

$\left\\{|u(x,t)-U(x,t)|,|\varphi_{t}(x,t)-D_{t}^{+}\varphi(x,t)|,|\varphi_{xx}(x,t)-D^{2}_{x}\varphi(x,t)|:(x,t)\in
G_{C\ell}\right\\}$

is the largest. This number, which we call $\varepsilon$, is again
infinitesimal. Therefore we have

(33)
$\max_{G_{C\ell}}\left\\{|u(x,t)-U(x,t)|,|\varphi_{t}(x,t)-D_{t}^{+}\varphi(x,t)|,|\varphi_{xx}(x,t)-D^{2}_{x}\varphi(x,t)|\right\\}\leq\varepsilon$

for some infinitesimal $\varepsilon>0$.

The remainder of the argument is very similar to our proof in § 3.4 that the
distribution $u$ defined was a distribution solution to the linear heat
equation. Namely, if we replace $u$ by $U$ and derivatives of $\varphi$ by
finite differences of $\varphi$ in (32), then (33) implies that we only make
an infinitesimal error on both sides. We therefore only have to prove

$\sum_{(x,t)\in
G_{C}}\bigl{\\{}U(x,t)(-D^{2}_{x}\varphi-D^{+}_{t}\varphi_{t})-f(u(x,t))\varphi\bigr{\\}}\,dx\,dt\approx\sum_{(x,0)\in
G_{C}}u_{0}(x)\varphi(x,0)\,dx.$

This follows after applying summation by parts, and using the finite
difference equation (22) satisfied by $U$. This completes the existence proof.

## References

  * [1] Sergio Albeverio, Raphael Høegh-Krohn, Jens Erik Fenstad, and Tom Lindstrøm.  Nonstandard methods in stochastic analysis and mathematical physics, volume 122 of Pure and Applied Mathematics.  Academic Press, Inc., Orlando, FL, 1986.
  * [2] R. A. Fisher.  The wave of advance of advantageous genes.  Annals of Eugenics, 7(4):355–369, 1937.
  * [3] Gerald B. Folland.  Real analysis.  Pure and Applied Mathematics (New York). John Wiley & Sons, Inc., New York, second edition, 1999.  Modern techniques and their applications, A Wiley-Interscience Publication.
  * [4] Robert Goldblatt.  Lectures on the hyperreals, volume 188 of Graduate Texts in Mathematics.  Springer-Verlag, New York, 1998.  An introduction to nonstandard analysis.
  * [5] D. Henry.  Geometric theory of semilinear parabolic equations.  Lecture notes in mathematics. Springer-Verlag, 1981.
  * [6] F. John.  Partial Differential Equations.  Applied Mathematical Sciences. Springer New York, 1991.
  * [7] H.J. Keisler.  Foundations of Infinitesimal Calculus.  on-line at https://www.math.wisc.edu/~keisler/foundations.html, 2007\.
  * [8] H.J. Keisler.  Elementary Calculus: An Infinitesimal Approach.  Dover Books on Mathematics Series. Dover Publications, 2012.
  * [9] A. Kolmogorov, I. Petrovskii, and N. Piskunov.  A study of the diffusion equation with increase in the amount of substance, and its application to a biological problem.  In V. M. Tikhomirov, editor, Selected Works of A. N. Kolmogorov I, page 248–270. Kluwer, 1991.  ISBN 90-277-2796-1. Translated by V. M. Volosov from Bull. Moscow Univ., Math. Mech. 1, 1–25, 1937.
  * [10] Randall J. LeVeque.  Finite difference methods for ordinary and partial differential equations.  Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2007.  Steady-state and time-dependent problems.
  * [11] Edward Nelson.  Radically elementary probability theory, volume 117 of Annals of Mathematics Studies.  Princeton University Press, Princeton, NJ, 1987.
  * [12] A. Pazy.  Semigroups of Linear Operators and Applications to Partial Differential Equations.  Applied Mathematical Sciences. Springer New York, 2012.
  * [13] W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery.  Numerical Recipes 3rd Edition: The Art of Scientific Computing.  Cambridge University Press, 2007.
  * [14] Terence Tao.  Ultrafilters, nonstandard analysis, and epsilon management.  https://terrytao.wordpress.com/2007/06/25/ultrafilters-nonstandard-analysis-and-epsilon-management/ Accessed: 2018-07-19.