# Online change-point detection for a transient change

J.Jack<EMAIL_ADDRESS>[ School of Mathematics, Cardiff
University, Cardiff, CF24 4AG, UK

(6 4 2021)

###### Abstract

We consider a popular online change-point problem of detecting a transient
change in distributions of i.i.d. random variables. For this change-point
problem, several change-point procedures are formulated and some advanced
results for a particular procedure are surveyed. Some new approximations for
the average run length to false alarm are offered and the power of these
procedures for detecting a transient change in mean of a sequence of normal
random variables is compared.

60G50,

60G35,

60G70,

94C12,

93E20,

Change-point detection,

Statistical quality control,

Boundary crossing probabilities,

###### keywords:

[class=AMS]

###### keywords:

##  1 Introduction

The subject of change-point detection (or statistical quality control) is
devoted to monitoring and detecting changes in the structure of a time series.
This paper considers a popular online change-point problem of detecting a
change in distribution of a sequence of i.i.d. random variables. Online
change-point problems are concerned with monitoring the structure of a random
process(es) whose observations arrive sequentially. For these problems, any
good monitoring procedure should reliably alert the user to unexpected changes
as soon as possible or with highest probability, subject to a tolerance on
false alarms.

Let $y_{1},y_{2},\ldots$ be a sequence of independent random variables
arriving sequentially. The purpose of this paper is to discuss tests for the
hypothesis that $y_{i}\,(i=1,2\ldots)$ are identically distributed with some
probability density function (pdf) $f(y)$ against the alternative that at some
unknown change point $0\leq\nu<\infty$, the random variables
$y_{1},y_{2},\ldots,y_{\nu}$ and $y_{\nu+l+1},y_{\nu+l+2},\ldots$ are
identically distributed with density $f(y)$ and
$y_{\nu+1},y_{\nu+2},\ldots,y_{\nu+l}$ are identically distributed with pdf
$g(y)$ such that $g(y)\neq f(y)$. Here, $l$ is length of the change-point
period (signal) and can be known or unknown. Under a standard hypothesis
testing framework, the null hypothesis is
$\mathbb{H}_{\infty}\\!:\,\nu=\infty$ and hence the pdf $f(y)$ is the density
of $y_{i}$ for all $i=1,2,\ldots$. The alternative hypothesis is
$\mathbb{H}_{\nu}$: $0\leq\nu<l\leq\infty$ and therefore

$\displaystyle\mathbb{H}_{\nu}\\!:\left\\{\begin{array}[]{ll}\mbox{
\\!\\!\\!\\!$y_{i}$ have density $f(y)$}&\mbox{ if $i\leq\nu$ or $i>\nu+l$
}\\\ \mbox{ \\!\\!\\!\\!$y_{i}$ have density $g(y)$}&\mbox{ if
$\nu<i\leq\nu+l$ }\end{array}\right.$

with $i=1,2,\ldots$. Under $\mathbb{H}_{\nu}$, the arrival time of the signal
is $\nu+1$ (it is unknown). Most classical results assume $f$ and $g$ are
known completely; by this, we mean no nuisance parameters are present in the
distributions. In later sections, we will briefly discuss tests designed to
approach the change-point problem in the presence of nuisance parameters.

A thorough introduction to the field of online (quickest) change-point
detection mainly for the case of $l=\infty$ can be found in, for example, [1,
2, 3, 4]. Some of the most popular online change-point algorithms used in
practice are Shewhart’s $\bar{X}$-chart [5], the CUSUM algorithm [6], the
Shiryaev-Roberts procedure [7, 8] and the Exponentially Weighted Moving
Average (EWMA) chart [9]. The case of $l=\infty$, and hence when a change in
distribution occurs it does so permanently, is by far the most popular
scenario considered in the change-point literature; a number of influential
papers are [10, 11, 12, 13, 14, 15]. The CUSUM and Shiryaev Roberts procedures
benefit with their simplicity and proven optimality under suitable optimality
criteria; these two procedures will be the focus of discussion for the case
$l=\infty$. The case of finite $l$, and hence when a change occurs it does so
temporarily, has seen considerable attention in the past, see [16, 17, 18, 19,
20, 21, 22, 23]. More recently it has been the focus of attention in the
papers of [24, 25, 26]. Examples of areas where detecting a transient change
in distributions is extremely important can be found in radar and sonar [27,
28, 29], nondestructive testing [30], and medicine [31]. Non-parametric online
change-point detection methods have also become very popular [32, 33]. For the
state of the art techniques for multiple change-point detection, see [34, 35,
36, 37]. For sequential change-point detection in high-dimensional time
series, a likelihood ratio approach can be found in [38, 39].

This survey is organised as follows. In Section 2, we survey results for
$\l=\infty$ and discuss known optimality results for the CUSUM and Shiryaev-
Roberts procedures. This section contains well known classical results but is
included to introduce the reader to change-point concepts that will be used
when considering the transient change-point problem. In Section 3, we assume
$l<\infty$ and discuss a number of online tests for transient changes; the
likelihood ratio test providing the inspiration behind all tests. In this
Section, we compare procedures when applied for detecting a temporary change
in mean of a sequence of Gaussian random variables. We also apply tests for
monitoring stability of components used in the Oil and Gas industry.

Throughout this survey we shall use the notation ${\rm Pr}_{\infty}$ and
$\mathbb{E}_{\infty}$ to denote probability and expectation under
$\mathbb{H}_{\infty}$. Under the alternative $\mathbb{H}_{\nu}$, we shall use
the notation ${\rm Pr}_{\nu}$ and $\mathbb{E}_{\nu}$ to denote probability and
expectation assuming the change-point occurs at $\nu<\infty$.

##  2 Permanent change in distributions

In this section, we assume $l=\infty$; if a change occurs, it does so
permanently. Suppose $y_{1},y_{2},\ldots,y_{n}$ have been sampled. The
likelihood ratio for testing $\mathbb{H}_{\infty}$ against $\mathbb{H}_{\nu}$
is

$\displaystyle\Lambda_{\nu,n}=\prod_{i=\nu+1}^{n}\frac{g(y_{i})}{f(y_{i})}$

assuming $\nu<n$, otherwise $\Lambda_{\nu,n}=1$.

###  2.1 The CUSUM and Shiryaev-Roberts procedures

By maximising the statistic $\Lambda_{\nu,n}$ over all possible locations of
$\nu$, we obtain the CUSUM statistic

$\displaystyle V_{n}:=\max_{1\leq\nu\leq n}\Lambda_{\nu,n},\,\,\,\,\,n\geq
1\,.$ (2.1)

The CUSUM stopping rule (when to alert the user to a potential change-point)
is

$\displaystyle\tau_{V}(H):=\inf\\{n\geq 1:V_{n}>H\\}\,.$ (2.2)

An appealing property of statistic (2.1) is the recursive property

$\displaystyle
V_{n}=\max\\{V_{n-1},1\\}\cdot\frac{g(y_{n})}{f(y_{n})},\,\,\,\,\,V_{0}=1\,.$

The threshold $H$ in $\tau_{V}(H)$ is chosen on the users tolerance to false
alarm risk. Page [6] and Lorden [10] measured false alarm risk through the
Average Run Length to false alarm (ARL). This corresponds to choosing $H$ such
that $\mathbb{E}_{\infty}\tau_{V}(H)=C$, where $C$ is a pre-defined value
chosen by the user but is typically large. How to compute
$\mathbb{E}_{\infty}\tau_{V}(H)$ will be discussed later in this section.

The famous CUSUM chart of Page [6] introduces a reflective barrier at zero:

$\displaystyle P_{n}\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\max\left\\{P_{n-1}+\log\frac{g(y_{n})}{f(y_{n})},0\right\\},$
(2.3) $\displaystyle P_{0}\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!0\,.$

The statistics (2.3) and $\log V_{n}$ are equivalent on the positive half
plane and hence the rule

$\displaystyle\tau_{P}(\log(H))=\inf\\{n\geq 1:P_{n}>\log H\\}\,,$ (2.4)

and $\tau_{V}(H)$ are equivalent for $H>1$. The stopping rule $\tau_{V}$ is
more general than $\tau_{P}$ as thresholds $H\leq 1$ are permissable. An
approximation for $\mathbb{E}_{\infty}\tau_{P}(H)$ for general distributions
$f$ and $g$ was derived in [40]. Let
$I_{f}:=-\mathbb{E}_{\infty}(\log(g(y_{1})/f(y_{1})))$ and
$I_{g}=\mathbb{E}_{0}(\log(g(y_{1})/f(y_{1})))$ (to compute $I_{g}$ we assume
the change-point occurs at time zero). Then

$\displaystyle\mathbb{E}_{\infty}\tau_{P}(H)\simeq\frac{e^{H}}{I_{g}\zeta^{2}}-\frac{H}{I_{f}}-\frac{1}{I_{g}\zeta}\,.$
(2.5)

Here the constant $\zeta$ is called the limiting exponential overshoot. Let
$Z_{n}=\sum_{i=1}^{n}\log(g(y_{i})/f(y_{i}))$ be a random walk. Then it can be
shown, see [41, Ch. VIII], that

$\displaystyle\zeta\\!=\\!\frac{1}{I_{g}}\exp\left\\{\\!-\\!\sum_{k=1}^{\infty}\frac{1}{k}[{\rm
Pr}_{\infty}(Z_{k}\\!>\\!0)\\!+\\!{\rm
Pr}_{0}(Z_{k}\\!\leq\\!0)]\right\\}\\!.$

The approximation (2.5) seems extremely accurate. For example, suppose pre-
change observations are i.i.d $N(0,1)$ random variables and post-change
observations are i.i.d $N(A,1)$ for some known $A>0$. We have

$\displaystyle f(y)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\frac{1}{\sqrt{2\pi}}\exp(-y^{2}/2),$ (2.6)
$\displaystyle g(y)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\frac{1}{\sqrt{2\pi}}\exp(-(y-A)^{2}/2)\,.$

For $A=1$, Monte Carlo simulations provide
$\mathbb{E}_{\infty}\tau_{P}(4.39)=500$. Application of the approximation in
(2.5) provides 498. The draw back of the approximation in (2.5) is that
$\zeta$ requires expensive numerical evaluation.

To construct the Shiryaev-Roberts (SR) procedure, define the generalised
Bayesian detection statistic as:

$\displaystyle R_{n}:=\sum_{\nu=1}^{n}\prod_{j=\nu+1}^{n}\Lambda_{\nu,n}\,.$
(2.7)

Then the SR test is:

$\displaystyle\tau_{R}(H):=\inf\\{n\geq 1:R_{n}>H\\}\,,$ (2.8)

where $H$ is the solution of $\mathbb{E}_{\infty}\tau_{R}(H)=C$ for some pre-
determined $C$. The SR statistic (2.7) satisfies the following recurrence:

$\displaystyle R_{n}=(1+R_{n-1})\cdot\frac{g(y_{n})}{f(y_{n})},\,\,\,\,\,n\geq
1,\,R_{0}=0\,.$

###  2.2 Evaluating ARL for CUSUM and SR tests

Explicit expressions for $\mathbb{E}_{\infty}\tau_{V}(H)$ and
$\mathbb{E}_{\infty}\tau_{R}(H)$ are not known. However, they can be
numerically obtained by numerically solving particular Fredholm integral
equations as proved in [42]. Here it was shown that
$\mathbb{E}_{\infty}\tau_{V}(H)$ and $\mathbb{E}_{\infty}\tau_{R}(H)$ can be
computed by a unified approach for general Markov statistics. Set $H>0$. For a
sufficiently smooth positive valued function $\xi$ and $s\in[0,H]$, let

$\displaystyle S_{n}=\xi(S_{n-1})\cdot\frac{g(y_{n})}{f(y_{n})}\,\,\,\,\,n\geq
1,\,\,S_{0}=s\in[0,H]$

be a Markov detection statistic with stopping rule

$\displaystyle\tau_{S}(H):=\inf\\{n\geq 1:S_{n}>H\\}\,.$

Let $\phi(s)=\mathbb{E}_{\infty}(\tau_{S}(H))$ be the ARL (note the dependence
on $S_{0}=s$) and set $F(x)={\rm Pr}_{\infty}({g(y_{1})}/f(y_{1})\leq x)$.
Then $\phi(s)$ is the solution of the following Fredholm integral equation:

$\displaystyle\phi(s)=1+\int_{0}^{H}\phi(x)\left[\frac{d}{dx}F\left(\frac{x}{\xi(s)}\right)\right]dx\,.$
(2.9)

For the CUSUM and SR procedures we have $\xi(s)=\max(1,s)$ and $\xi(s)=1+s$,
respectively. To solve this integral equation, we refer to [42].

Approximations for ARL of the CUSUM and SR procedures have been specifically
developed for the problem of detecting the change in mean of normal random
variables. Here we operate under (2.6). To approximate ARL for both the CUSUM
and SR procedures or to narrow the domain of search and more efficiently
numerically solve the Fredholm equation (2.9), one could use the following
simple approximations developed in [43] and [44] respectively:

$\displaystyle\mathbb{E}_{\infty}\tau_{V}(H)$ $\displaystyle\simeq$
$\displaystyle 2H/(A\kappa^{2}(A))\,,$ (2.10)
$\displaystyle\mathbb{E}_{\infty}\tau_{R}(H)$ $\displaystyle\simeq$
$\displaystyle H/\kappa(A)\,,$ (2.11)

where

$\displaystyle\kappa(A)=\frac{2}{A^{2}}\exp\left\\{-2\sum_{\nu=1}^{\infty}\frac{1}{\nu}\Phi\left(-\frac{A}{2}\sqrt{\nu}\right)\right\\}$
$\displaystyle\mbox{ and }\Phi(x)=\int_{-\infty}^{x}f(y)dy\,.$

The approximations in (2.10) and (2.11) are extremely accurate. In Table 1,
one can observe the high accuracy of approximation (2.10) for different
thresholds $H$. In fact, (2.11) is remarkably accurate and frequently leads to
exact values of ARL. The only slight inconvenience of both approximations is
the numerical evaluation required to compute $\kappa(A)$. This quantity can be
approximated with $\kappa(A)\simeq\exp(-\rho\cdot A)$, where the constant
$\rho$ is defined later in (3.2.1) but can be approximated to three decimal
places by $\rho\simeq 0.583$. Using this approximation for $\kappa$ in (2.10)
and (2.11) still results in excellent approximations.

Table 1: Approximations for $\mathbb{E}_{\infty}\tau_{V}(H)$ with $A=1$. $H$ | 9.32 | 17.33 | 80.65 | 159.35 | 788.00
---|---|---|---|---|---
$\mathbb{E}_{\infty}\tau_{V}(H)$ | 50 | 100 | 500 | 1000 | 5000
Approximation (2.10) | 59 | 110 | 513 | 1014 | 5018
Approximation (2.10) with $\kappa(A)\simeq\exp(-\rho\cdot A)$ | 60 | 111 | 517 | 1023 | 5058

###  2.3 Optimality criteria

Denote by $\Delta(C)$ the set of all stopping times of change-point procedures
with ARL of at least $C$. More precisely,
$\Delta(C):=\\{\tau:E_{\infty}\tau\geq C\\},\,\,\,C>1,$ where $\tau=\tau(H)$
is a stopping time for a sequential change-point procedure. A common criterion
for comparing change-point procedures when $l=\infty$ is the supremum Average
Delay to Detection (ADD) introduced by Pollak [12]. Define
$ADD_{\nu}(\tau):=\mathbb{E}_{\nu}(\tau-\nu|\tau>\nu)$. Then

$\displaystyle SADD(\tau):=\sup_{0\leq\nu<\infty}ADD_{\nu}(T)\,.$ (2.12)

An optimal change-point procedure would satisfy
$SADD(\tau_{opt})=\inf_{\tau\in\Delta(C)}SADD(\tau)$ for all $C>~{}1$. Finding
an optimal procedure for this criterion is very difficult, where in general
only asymptotic optimality as $C\rightarrow\infty$ (low false alarm rate) is
known [12]. Another popular criterion is the worst-case minimax scenario of
Lorden [10] defined as

$\displaystyle{\cal L}(\tau):=\sup_{\nu\geq
0}\operatorname*{ess\,sup}\mathbb{E}_{\nu}[(\tau-\nu)^{+}|y_{1},y_{2},\ldots,y_{\nu}]\,.$

This criterion evaluates the average detection delay conditioned on the worst
possible data before the change and then considers the worst possible
deterministic change-point. Asymptotic optimality (as $C\rightarrow\infty$) of
the CUSUM chart of Page was proved in [10]. It was subsequently proved in [11]
that the CUSUM chart of Page is in fact optimal under this criterion for every
$C>1$.

The SR procedure is optimal for every $C>1$ under the Stationary Average Delay
to Detection (STADD) criterion. The STADD criterion praises detection
procedures that detect the change as quickly as possible, at the expense of
raising many false alarms (using a repeated application of the same stopping
rule). Formally, the STADD criterion is defined as follows. Let
$\tau_{1},\tau_{2}\ldots$ be a sequence of independent copies of the stopping
time $\tau$. Let $T_{j}=\tau_{1}+\tau_{2}+\ldots+\tau_{j}$ be the time the
$j^{th}$ alarm is raised. Let $I_{\nu}=\min\\{j>1:T_{j}>\nu\\}$; this is the
index of the first alarm which is not false after $I_{\nu}-1$ false alarms.
Then

$\displaystyle
STADD(\tau):=\lim_{\nu\rightarrow\infty}\mathbb{E}_{\nu}[T_{I_{\nu}}-\nu]\,.$

The STADD criterion is equivalent to the Relative Integral Average Detection
Delay (RIADD) measure, see [42], which is defined as:

$\displaystyle
RIADD(\tau)=\frac{\sum_{\nu=0}^{\infty}\mathbb{E}_{\nu}[(\tau-\nu)^{+}]}{\mathbb{E}_{\infty}[\tau]}\,.$

Both the SR procedure and CUSUM procedure are asymptotically optimal as
$C\rightarrow\infty$ for the Lorden and the STADD criteria. It is discussed in
[42] for both CUSUM and the Shiryaev–Roberts procedure Lorden’s essential
supremum measure (2.3) and Pollak’s supremum measure $SADD$ defined in (2.12)
are attained at $\nu=0$, that is:

$\displaystyle{\cal
L}(\tau_{V}(H))=SADD(\tau_{V}(H))=\mathbb{E}_{0}\tau_{V}(H),$
$\displaystyle{\cal
L}(\tau_{R}(H))=SADD(\tau_{R}(H))=\mathbb{E}_{0}\tau_{R}(H)\,.$

Similarly to the computation of $\mathbb{E}_{\infty}\tau_{V}(H)$ and
$\mathbb{E}_{\infty}\tau_{R}(H)$, to obtain $\mathbb{E}_{0}\tau_{V}(H)$ and
$\mathbb{E}_{0}\tau_{R}(H)$ one can numerically solve a Fredholm equation.
Instead of setting $\phi(s)=\mathbb{E}_{\infty}(\tau(H))$, let
$\phi(s)=\mathbb{E}_{0}(\tau(H))$. Also set $F(x)={\rm
Pr}_{0}({g(y_{1})}/f(y_{1})\leq x)$. Then from [42], $\phi(s)$ is the solution
of the Fredholm integral equation given in (2.9). The computation of STADD
requires solving a slightly more difficult integral equation and we refer the
interested reader to [42] for more discussions.

For the Gaussian example considered in (2.6), the findings of [42] indicate
that for small values of $A$ say $A=0.01$, the CUSUM noticeably outperforms
the SR procedure under Lordens criterion. Vice versa, the SR procedure
noticeably outperforms CUSUM under the STADD framework. When the change in $A$
becomes large, say $A=1$, the benefits a procedure has over the other
diminishes.

##  3 Transient change in distributions

In this section, we assume $1\leq l<\infty$ and therefore study procedures
aimed at detecting a transient change in distributions. Suppose
$y_{1},y_{2},\ldots,y_{n}$ have been sampled. The log likelihood ratio for
testing $\mathbb{H}_{\infty}$ against $\mathbb{H}_{\nu}$ is

$\displaystyle\Lambda_{\nu,\nu+l}=\sum_{i=\nu+1}^{\min\\{\nu+l,n\\}}\log\frac{g(y_{i})}{f(y_{i})}\,.$
(3.1)

###  3.1 A collection of procedures

For $l$ unknown, the log likelihood ratio statistic is obtained by maximising
(3.1) over all possible change point locations $\nu$ and transient change
lengths:

$\displaystyle K_{n}:=\max_{0\leq\nu<\nu+l\leq n}\Lambda_{\nu,\nu+l}\,,$ (3.2)

with the stopping rule

$\displaystyle\tau_{K}(H):=\inf\\{n\geq 1:K_{n}>H\\}\,.$ (3.3)

If there are no nuisance parameters present in $f$ and $g$ that require
estimation, the statistic (3.2) satisfies the recursive property:

$\displaystyle K_{n}\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\max\\{K_{n-1},\max_{0\leq\nu\leq
n-1}\Lambda_{\nu,n}\\},$ (3.4) $\displaystyle K_{0}\\!\\!\\!\\!$
$\displaystyle=$ $\displaystyle\\!\\!\\!\\!0\,.$

For large $n$, the statistic (3.2) is very expensive to compute despite the
recursive property given in (3.4). For offline change-point problems, this
large computational expense may be an inconvenience but it is not a
fundamental problem as time is often not an issue. However, for online
procedures that require calculations in real time, the statistic $K_{n}$ is
not practical. The assumption that no knowledge of the transient change length
is known is unlikely. One can imagine that some prior knowledge about the
length of transient change is likely, for example it may be bounded $l_{0}\leq
l\leq l_{1}$. From here on, this assumption will be made. The log likelihood
ratio statistic is:

$\displaystyle
Z_{n}={Z}_{n}(l_{0},l_{1}):=\max_{\begin{subarray}{c}0\leq\nu<\nu+l\leq n\\\
l_{0}\leq l\leq l_{1}\end{subarray}}\Lambda_{\nu,\nu+l}\,,$ (3.5)

with the stopping rule

$\displaystyle\tau_{{Z}}(H):=\inf\\{n\geq l_{1}:{Z}_{n}(l_{0},l_{1})>H\\}\,.$

If no nuisance parameters require estimation, the statistic ${Z}_{n}$
satisfies the following recursive property:

$\displaystyle{Z}_{n}\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\max\\{{Z}_{n-1},\max_{n\leq\nu\leq
n+l_{1}-l_{0}}\Lambda_{\nu,n+l_{1}}\\},$ $\displaystyle Z_{l_{1}}\\!\\!\\!\\!$
$\displaystyle=$
$\displaystyle\\!\\!\\!\\!\max_{\begin{subarray}{c}0\leq\nu<\nu+l\leq l_{1}\\\
l_{0}\leq l\leq l_{1}\end{subarray}}\Lambda_{\nu,\nu+l}\,.$

This is much easier to compute than (3.2) for $n$ large. This recursive
property means the stopping rule of $Z_{n}$ can be expressed as:

$\displaystyle\tau_{Z}(H)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!l_{1}+\tau_{{{S,l_{0},l_{1}}}}(H)\,,\text{ where }$
(3.6) $\displaystyle\tau_{{{S,l_{0},l_{1}}}}(H)\\!\\!\\!\\!$ $\displaystyle:=$
$\displaystyle\\!\\!\\!\\!\inf\\{n\geq 0:{S}_{n,{l_{0},l_{1}}}>H\\}\,,$

and

$\displaystyle{S}_{n,{l_{0},l_{1}}}:=\max_{n\leq\nu\leq
n-l_{0}+l_{1}}\Lambda_{\nu,n+l_{1}}\,.$

Therefore
$\mathbb{E}_{\infty}\tau_{Z}(H)=\mathbb{E}_{\infty}\tau_{{{S,l_{0},l_{1}}}}(H)+l_{1}$
.

If we make the additional assumption that $l$ is known exactly and is
completely contained within the sample of size $n$, i.e. $\nu+l\leq n$, then
the MOSUM statistic can be obtained by setting $l_{0}=l_{1}=l$ in (3.5). For
this reason, the statistic $Z_{n}$ can be called the generalised MOSUM
procedure. This corresponds to maximising (3.1) over all valid change-point
locations $\nu$:

$\displaystyle M_{n}:=\max_{0\leq\nu\leq n-l}\Lambda_{\nu,\nu+l}\,,$ (3.7)

with the stopping rule

$\displaystyle\tau_{M}(H):=\inf\\{n\geq l:M_{n}>H\\}\,.$

In what follows, we will define the MOSUM test for a general window length
$L$, with $L$ a fixed positive integer. Results for the likelihood ratio test
can be obtained by setting $L=l$. Define the moving sums

$\displaystyle
S_{n,L}:={S}_{n,{L,L}}=\sum_{j=n+1}^{n+L}\log\frac{g(y_{i})}{f(y_{i})}\,\;\;(n=0,1,\ldots)\,.$

Then the stopping rule $\tau_{M}(H)$ for a given window length $L$ can be
expressed as

$\displaystyle\tau_{M}(H)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\tau_{S,L}(H)+L,\,\mbox{ where }$ (3.8)
$\displaystyle\tau_{S,L}(H)\\!\\!\\!\\!$ $\displaystyle:=$
$\displaystyle\\!\\!\\!\\!\inf\\{n\geq 0:S_{n,L}>H\\}\,$

and therefore
$\mathbb{E}_{\infty}\tau_{M}(H)=\mathbb{E}_{\infty}\tau_{S,L}(H)+L$. The
moving sum $S_{n,L}$ is the reason behind the MOSUM name.

For the transient change-point problem, the false alarm risk can be measured
through ARL. However, this is not the only approach taken in the change-point
literature. In [17] and [45] , the false alarm risk is measured through:

$\displaystyle\sup_{k\geq 1}{\rm
Pr}_{\infty}(k\leq\tau<k+m_{\alpha})\leq\alpha\,,$ (3.9)

where $\tau$ is a stopping rule, $\alpha$ is your false alarm tolerance (type
1 error) and $\lim\inf m_{\alpha}/|\log(\alpha)|>I_{g}^{-1}$ but $\log
m_{\alpha}=o(|\log\alpha|)$ as $\alpha\rightarrow 0$; recall
$I_{g}=\mathbb{E}_{0}(\log(g(y_{1})/f(y_{1})))$. Another alternative to the
usual ARL constraint has been proposed in [43, 46]. Here, the suggested
criterion is

$\displaystyle\sup_{k\geq 1}{\rm Pr}_{\infty}(\tau<k+m_{\alpha}|\tau\geq
l)\leq\alpha.$

From now on, false alarm risk will be measured through ARL
$\mathbb{E}_{\infty}\tau$ and we refer to [17, 45, 43, 46] for more
discussions on other approaches. The majority of research has focused on
detecting transient changes in a sequence of Gaussian random variables. The
next section is devoted solely this problem.

###  3.2 Detecting a transient change in Gaussian random variables

Consider the problem of detecting the change in mean of normal random
variables. Suppose pre-change observations are i.i.d $N(\mu,1)$ random
variables and post-change observations are i.i.d $N(\mu+A,1)$ for some $A>0$.
The values of $\mu,l$ and $A$ may be known or unknown, with $\mu$ and $A$
playing the roles of nuisance parameters if unknown. We have

$\displaystyle f(y)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\frac{1}{\sqrt{2\pi}}\exp(-(y-\mu)^{2}/2),$ (3.10)
$\displaystyle g(y)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\frac{1}{\sqrt{2\pi}}\exp(-(y-\mu-A)^{2}/2)\,.$

The offline version of this change-point problem is devoted to testing for
change-points in a sample of fixed length and has seen significant attention
in the past, see [47, 48, 49, 50, 51]. An excellent survey of several
statistics aimed at addressing the offline problem can be found in [52].
Despite the fact $Z_{n}$ defined in (3.5) is a generalisation of $M_{n}$ given
in (3.7), we will initially discuss recent results for $M_{n}$. These results
will provide inspiration for addressing the much more complicated problems
associated with $Z_{n}$.

####  3.2.1 The MOSUM statistic

For the MOSUM test, knowledge of $A$ is not required to set the ARL
constraint; this is because the MOSUM stopping rule given in (3.8) specialised
for this Gaussian example is tantamount to:

$\displaystyle\tau_{M}(H)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\tau_{S,L}(H)+L,$ (3.11)
$\displaystyle\tau_{S,L}(H)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\inf\\{n\geq 0:S_{n,L}>H\\}\,,$ $\displaystyle\text{
with }S_{n,L}\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\sum_{j=n+1}^{n+L}y_{j}\,\,.$

The problem of approximating $\mathbb{E}_{\infty}\tau_{S,L}(H)$ assuming $\mu$
is known was considered in [25]. Here will recall the main steps in the
construction. Define

$\displaystyle h=\frac{H-\mu L}{\sqrt{L}}\;\mbox{ so that }\;H=\mu
L+h\sqrt{L}\,$

and consider the standardised versions of $S_{n,L}$:

$\displaystyle\xi_{n,L}\\!:=\\!\frac{S_{n,L}-\operatorname{\mathbb{E}}_{\infty}S_{n,L}}{\sqrt{{\rm
Var}_{\infty}(S_{n,L})}}\\!=\\!\frac{S_{n,L}-\mu
L}{\sqrt{L}}\,,\mbox{$n=0,1,\ldots\,.$}$

Then the stopping time $\tau_{S,L}(H)$ is equivalent to the stopping time

$\displaystyle{\tau}_{\xi}(h):=\inf\\{n\geq 0:\xi_{n,L}\geq h\\}$ (3.12)

and hence
$\mathbb{E}_{\infty}{\tau}_{\xi}(h)=\mathbb{E}_{\infty}\tau_{S,L}(H)$.

For any integer $M\geq 0$, the discrete time process
$\xi_{0,L},\xi_{1,L},\ldots,\xi_{M,L}$ is approximated by a continuous time
analogue $S(t)$ on $[0,T=M/L]$. The process $S(t)$ is a zero mean, stationary
Gaussian process with correlation function $R(t)=\max\\{0,1-|t|\\}$. The ARL
$\mathbb{E}_{\infty}{\tau}_{\xi}(h)$ then has the continuous-time
approximation

$\displaystyle\mathbb{E}_{\infty}{\tau}_{\xi}(h)\cong-L\int_{0}^{\infty}{s\,dF_{h}(s)}\,,$
(3.13)

where $F_{h}(T):={\rm P}_{\infty}(S(t)<h\text{ for all }t\in[0,T])$.

Explicit formulas for the probability $F_{h}(T)$ with $T\leq 1$ were first
derived in [53]. Here it was shown

$\displaystyle F_{h}(T)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\int_{-\infty}^{h}\\!\\!\Phi\left(\frac{h({Z}\\!+\\!1)\\!-\\!x(-{Z}\\!+\\!1)}{2\sqrt{{Z}}}\right)\varphi(x)dx$
$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!-\frac{2\sqrt{Z}}{Z+1}\varphi(h)\left[{h{\sqrt{Z}}}\,\Phi(h\sqrt{{Z}})\\!+\\!\frac{1}{\sqrt{2\pi}}(\sqrt{2\pi}\varphi(h))^{Z}\right]\,.$

For $T=1$ this reduces to

$\displaystyle
F_{h}(1)=\Phi^{2}(h)-\varphi(h)\big{[}h\Phi(h)+\varphi(h)\big{]}.$ (3.14)

For $T>1$, formulae for $F_{h}(T)$ were first derived in [54]; these
expressions take different forms depending on whether or not $T$ is integer.
The result of [54, p.949] states than if $T=n$ is a positive integer then

$\displaystyle{F}_{h}(n)=\int_{-\infty}^{h}\int_{D_{x}}\det[\varphi(y_{i}-y_{j+1}+h)]^{n}_{i,j=0}\,$
$\displaystyle dy_{2}\ldots dy_{n+1}dx\,$ (3.15)

where $y_{0}=0,y_{1}=h-x,$
$D_{x}=\\{y_{2},\dots,y_{n+1}\>|\>h-x<y_{2}<y_{3}<\ldots<y_{n+1}\\}$. For non-
integer $T\geq 1$, the exact formula for ${F}_{h}(T)$ is even more complex
(the integral has the dimension $\lceil 2T\rceil+1$) see [54, p.950]. For
$T=2$, (3.2.1) yields

$\displaystyle F_{h}(2)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\Phi^{3}(h)-2h\varphi(h)\Phi^{2}(h)$ (3.16)
$\displaystyle+$
$\displaystyle\\!\\!\\!\\!\frac{h^{2}\\!-\\!3\\!+\\!\sqrt{\pi}h}{2}\varphi^{2}(h)\Phi(h)+\frac{h\\!+\\!\sqrt{\pi}}{2}\varphi^{3}(h)$
$\displaystyle+$
$\displaystyle\\!\\!\\!\\!\int_{0}^{\infty}\Phi(h-y)[\varphi(h+y)\Phi(h-y)$
$\displaystyle-\sqrt{\pi}\varphi^{2}(h)\Phi(\sqrt{2}y)\,]dy.$

The complicated nature of these expressions for $F_{h}(T)$ made them
impractical for the use in the ARL approximation (3.13). One simple yet still
very accurate approximation has the form (see [55]):

$\displaystyle F_{h}(T)\simeq F_{h}(2)\left[\theta(h)\right]^{T-2}\,,$ (3.17)

where $\theta(h)={F_{h}(2)}/{F_{h}(1)}$ and the probabilities $F_{h}(1)$ and
$F_{h}(2)$ are given in (3.14) and (3.16) respectively. Here, $\varphi(x)$ and
$\Phi(x)$ are the standard normal density and distribution functions
respectively. The approximation given in (3.17) applied to (3.13) results in
the following continuous-time ARL approximation:

$\displaystyle\mathbb{E}_{\infty}{\tau}_{\xi}(h)$ $\displaystyle\simeq$
$\displaystyle-\frac{L\cdot{F}_{h}(2)}{\theta(h)^{2}\log(\theta(h))}.$

This approximation was then corrected in [25, Section 7] for discrete time to
improve results for small $L$. This amounted to correcting the probabilities
${F}_{h}(1)$ and ${F}_{h}(2)$ for discrete time; this was performed by
specialising results of D. Siegmund; primarily on expected overshoot a
discrete time normal random walk has over a threshold. From [41, p. 225], this
expected overshoot was computed as

$\displaystyle\rho\\!\\!\\!\\!$ $\displaystyle:=$
$\displaystyle\\!\\!\\!\\!-\int_{0}^{\infty}\frac{1}{\pi\lambda^{2}}\log\\{2(1-\exp(-\lambda^{2}/2))/\lambda^{2}\\}\,d\lambda\,$
$\displaystyle\simeq$ $\displaystyle\\!\\!\\!\\!0.582597.$

Define the probability

$\displaystyle{F}_{h}(M;L):={\rm
Pr}\left(\max_{n=0,1,\ldots,M}\xi_{n,L}<h\right).$ (3.19)

From [25, p. 18]:

$\displaystyle\mathbb{E}_{\infty}\tau_{S,L}(H)=\mathbb{E}_{\infty}{\tau}_{\xi}(h)\simeq-\frac{L\cdot{F}_{h}(2L;L)}{\theta_{L}(h)^{2}\log(\theta_{L}(h))}$
$\displaystyle\text{ with
}\theta_{L}(h)=\frac{{F}_{h}(2L;L)}{{{F}_{h}(L;L)}}\,,$

where, for $h_{L}:=h+\omega_{L}$ with $\omega_{L}=\sqrt{2}\rho/\sqrt{L}$, the
probabilities ${F}_{h}(L;L)$ and ${F}_{h}(2L;L)$ can be approximated by:

$\displaystyle{F}_{h}(L;L)\simeq\Phi(h)\Phi({h_{L}})-\varphi({h_{L}})[h\Phi(h)+\varphi(h)]\,,$
$\displaystyle{F}_{h}(2L;L)\\!\\!\\!\\!$ $\displaystyle\simeq$
$\displaystyle\\!\\!\\!\\!\frac{\varphi^{2}(h_{L})}{2}\\![({h}^{2}\\!-\\!1\\!+\\!\sqrt{\pi}h)\Phi\left(h\right)$
(3.22) $\displaystyle+$
$\displaystyle(h\\!+\\!\sqrt{\pi})\varphi\left(h\right)]$ $\displaystyle-$
$\displaystyle\\!\varphi\left(h_{L}\right)\Phi\left(h_{L}\right)\left[\left(h\\!+\\!h_{L}\right)\Phi\left(h\right)\\!+\\!\varphi\left(h\right)\right]$
$\displaystyle+$ $\displaystyle\\!\Phi\left(h\right)\Phi^{2}({h_{L}})$
$\displaystyle+$
$\displaystyle\\!\\!\int_{0}^{\infty}\\!\Phi(h\\!-\\!y)[\varphi(h_{L}+y)\Phi(h_{L}-y)$
$\displaystyle-\sqrt{\pi}\varphi^{2}(h_{L})\Phi(\sqrt{2}y)\,]dy.\,$

Only a one-dimensional integral has to be numerically evaluated for
approximating ${F}_{h}(2L;L)$. Tables 2 and Tables 3 demonstrate that (3.2.1)
using (3.2.1) and (3.22) is extremely accurate.

Table 2: Approximations for $\mathbb{E}_{\infty}{\tau}_{\xi}(h)$ with $L=10$. $h$ | 2 | 2.25 | 2.5 | 2.75 | 3 | 3.25 | 3.5
---|---|---|---|---|---|---|---
(3.2.1) | 126 | 217 | 395 | 759 | 1551 | 3375 | 7837
$\mathbb{E}_{\infty}{\tau}_{\xi}(h)$ | 127 | 218 | 396 | 757 | 1550 | 3344 | 7721
Table 3: Approximations for $\mathbb{E}_{\infty}{\tau}_{\xi}(h)$ with $L=50$. $h$ | 2 | 2.25 | 2.5 | 2.75 | 3 | 3.25 | 3.5
---|---|---|---|---|---|---|---
(3.2.1) | 471 | 791 | 1392 | 2587 | 5099 | 10695 | 23918
$\mathbb{E}_{\infty}{\tau}_{\xi}(h)$ | 472 | 792 | 1397 | 2588 | 5085 | 10749 | 24131

For approximating the boundary-crossing probability ${F}_{h}(M;L)$ for all
$M$, the discrete time corrected form of (3.17) suggests using the
approximation

$\displaystyle{F}_{h}(M;L)\simeq{F}_{h}(2L;L)\left[\theta_{L}(h)\right]^{M/L-2}\,.$
(3.23)

One could then approximate ${F}_{h}(2L;L)$ and $\theta_{L}(h)$ using (3.2.1)
and (3.22); the high accuracy of the resulting approximation was
comprehensively studied in [25].

####  3.2.2 The stopping rule $\tau_{Z}(H)$

Here, we assume $l$ is not known exactly but can be bounded between $l_{0}$
and $l_{1}$. We will initially assume $\mu$ and $A$ are known. The stopping
rule given in (3.6) specialised for this Gaussian example is tantamount to:

$\displaystyle\tau_{{Z}}(H)\\!\\!\\!\\!$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!l_{1}+\tau_{{{S,l_{0},l_{1}}}}(H)\,,\text{ where }$
(3.24) $\displaystyle\tau_{{{S,l_{0},l_{1}}}}(H)\\!\\!\\!\\!$
$\displaystyle:=$ $\displaystyle\\!\\!\\!\\!\inf\\{n\geq
0:{S}_{n,{l_{0},l_{1}}}>H\\}\,,$

and

$\displaystyle{S}_{n,{l_{0},l_{1}}}:=\max_{n\leq\nu\leq
n-l_{0}+l_{1}}\sum_{j=\nu+1}^{n+l_{1}}\left(y_{j}-\mu-\frac{A}{2}\right)\,.$

The short memory of the MOSUM statistic is paramount to the form of the
approximation given in (3.23). This short memory is also present for the
generalised moving sum statistic ${S}_{n,{l_{0},l_{1}}}$ and suggests the form
of approximations (3.2.1) and (3.23) would also be suitable when applied to
${S}_{n,{l_{0},l_{1}}}$. Introduce the probability:

$\displaystyle{F}_{l_{0},l_{1}}(H,M):={\rm
Pr}\\{{S}_{j,{l_{0},l_{1}}}<H\,\,\,\forall\,\,\,j=0\ldots M\\}\,.$

Then the following approximations should also provide high accuracy:

$\displaystyle\,\,\,\,\,\,{F}_{l_{0},l_{1}}(H,M)\\!\simeq\\!{F}_{l_{0},l_{1}}(H,2l_{1})\left[\theta_{l_{1}}(H)\right]^{M/l_{1}-2}$
(3.25) $\displaystyle\,\,\,\,\,\text{ with
}\theta_{l_{1}}(H)=\frac{{F}_{l_{0},l_{1}}(H,2l_{1})}{{F}_{l_{0},l_{1}}(H,l_{1})}\,,$
$\displaystyle\mathbb{E}_{\infty}\tau_{S,l_{0},l_{1}}(H)\simeq-\frac{l_{1}\cdot{F}_{l_{0},l_{1}}(H,2l_{1})}{[\theta_{l_{1}}(H)]^{2}\log(\theta_{l_{1}}(H))}\,.$
(3.26)

Unfortunately, the probability ${F}_{l_{0},l_{1}}(H;M)$ is complex and to the
authors knowledge no formula or approximations are known. The probabilities
${F}_{l_{0},l_{1}}(H;2l_{1})$ and ${F}_{l_{0},l_{1}}(H;l_{1})$ can be
approximated via simulations; this is not too cumbersome as at most $3l_{1}$
random variables need to be simulated at each iteration. As commonly
$\mathbb{E}_{\infty}\tau_{{{S,l_{0},l_{1}}}}(H)=C$ with $C$ large, say
$C=500$, the right tail of the distribution of the random variable
$\max_{j=0\ldots M}{S}_{j,{l_{0},l_{1}}}$ is of the most interest. Large
deviation theory, see [48, 49], could be used to approximate the right tail of
this distribution, however numerical results indicate approximations of these
kind would not be accurate enough for general $l_{0}$ and $l_{1}$ (those that
are not astronomically large). If the prior knowledge that $1\leq l\leq l_{1}$
is known, and an explicit formula to approximate ${F}_{1,l_{1}}(H;M)$ or
$\mathbb{E}_{\infty}\tau_{S}(H)$ is desired, the following simple heuristic
argument could be used. The continuous time analogue of the probability
${F}_{1,l_{1}}(H;M)$ is:

$\displaystyle{\rm Pr}\left\\{\max_{\begin{subarray}{c}0\leq s<t\leq
M+l_{1}\\\ 0\leq t-s\leq
l_{1}\end{subarray}}\left[W(t)-W(s)-\frac{A}{2}(t-s)\right]<H\right\\}\,,$

where $W(t)$, $0\leq t<\infty$, is standard Brownian motion. Ideally, a large
deviation approximation for (LABEL:Brownian_prob) should be computed. However,
for $M=l_{1}$, $M=2l_{1}$ and $A$ large, say $A\geq 1$, simulation studies
indicate that the additional maximisation constraint in (LABEL:Brownian_prob)
of $0<t-s<l_{1}$ has very little influence on this probability. If this
constraint is ignored, the following large deviation result of [49] can be
applied.

######  Lemma 1.

Suppose $\gamma>0$, $m\rightarrow\infty$ and $u\rightarrow\infty$ such that
$m\gamma u^{-1}$ is some fixed number in $(1,\infty)$. Then

$\displaystyle{\rm Pr}\left\\{\max_{0\leq s<t\leq
m}\left[W(t)-W(s)-\gamma(t-s)\right]>u\right\\}$
$\displaystyle=[2\gamma(m\gamma-u)+3+o(1)]\exp(-2\gamma u)\,.$

To subsequently correct this result for discrete time, it is recommended in
[49] to increase the barrier by $H$ by $2\rho$, where $\rho$ is defined in
(3.2.1). This results in the approximations

$\displaystyle{F}_{1,l_{1}}(H,l_{1})\\!\\!\\!\\!$ $\displaystyle\simeq$
$\displaystyle\\!\\!\\!\\!1-(A(Al_{1}-H-2\rho)+3)\times$ (3.29)
$\displaystyle\times$ $\displaystyle\\!\\!\\!\\!\exp\\{-A(H+2\rho)\\}$
$\displaystyle{F}_{1,l_{1}}(H,2l_{1})\\!\\!\\!\\!$ $\displaystyle\simeq$
$\displaystyle\\!\\!\\!\\!1-(A(3Al_{1}/2-H-2\rho)+3)\times$ (3.30)
$\displaystyle\times$ $\displaystyle\\!\\!\\!\\!\exp\\{-A(H+2\rho)\\}\,.$

As a result, using the approximations given in (3.25) and (3.26):

$\displaystyle{F}_{1,l_{1}}(H,M)\\!\\!\\!\\!$ $\displaystyle\simeq$
$\displaystyle\\!\\!\\!\\!1-(A(3Al_{1}/2-H-2\rho)+3)\times$ (3.31)
$\displaystyle\times$
$\displaystyle\\!\\!\\!\\!\exp\\{-A(H+2\rho)\\}\left[\hat{\theta}_{l_{1}}(H)\right]^{M/l_{1}-2}$

with

$\displaystyle\hat{\theta}_{l_{1}}(H)$ $\displaystyle=$
$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\frac{1-(A(3Al_{1}/2-H-2\rho)+3)\exp\\{-A(H+2\rho)\\}}{1-(A(Al_{1}-H-2\rho)+3)\exp\\{-A(H+2\rho)\\}}\,.$

Also

$\displaystyle\mathbb{E}_{\infty}\tau_{S,l_{0},l_{1}}(H)$
$\displaystyle\simeq$ $\displaystyle-$
$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\frac{l_{1}[1\\!-\\!(A(3Al_{1}/2\\!-\\!H\\!-\\!2\rho)\\!+\\!3)\exp\\{-A(H\\!+\\!2\rho)\\}]}{[\hat{\theta}_{l_{1}}(H)]^{2}\log(\hat{\theta}_{l_{1}}(H))}\,.$

The accuracy of the approximation in (3.25) is demonstrated in Figures 1-2 for
different $l_{0},l_{1},M$ and $A$ as a function of $H$. In this approximation,
${F}_{l_{0},l_{1}}(H,2l_{1})$ and ${F}_{l_{0},l_{1}}(H,l_{1})$ have been
approximated using Monte Carlo simulations with 100,000 repetitions. In these
figures, the probability ${F}_{l_{0},l_{1}}(H;M)$ is depicted with a thick
dashed black line and is obtained from simulations. The approximation in
(3.25) is depicted with a solid blue line. From these figures the high
accuracy of approximation (3.25) is clearly demonstrated. In Figures 3-6, we
asses the accuracy of the approximation in (3.31). In these figures, for $A=1$
and various $M$, the probability ${F}_{1,l_{1}}(H;M)$ is depicted with a thick
dashed black line whereas the approximation provided in (3.31) is shown with a
solid red line. The number present on the figure is used to show the value of
$l_{1}$ used. From these figures, we see for large $H$ the approximation in
(3.31) is adequate. In Tables 4-5 the accuracy of the approximations provided
in (3.26) and (LABEL:Approx2) are assessed for different $H$. We see the
approximation in (3.26) is extremely accurate for all $H.$ For large $H$, the
approximation in (LABEL:Approx2) is fairly accurate and has the benefit of
explicit evaluation. For small $H$ and small $A$, the accuracy of
(LABEL:Approx2) should deteriorate.

Figure 1: Empirical probabilities of reaching the barrier $H$ (dashed black)
and approximation (3.25) (solid blue): $A=1$, $M/l_{1}=4$ with $l_{0}=25$ and
$l_{1}=50$.  Figure 2: Empirical probabilities of reaching the barrier $H$
(dashed black) and approximation (3.25) (solid blue): $A=0.5$, $M/l_{1}=25$
with $l_{0}=10$ and $l_{1}=20$.

Figure 3: Empirical probabilities of reaching the barrier $H$ (dashed black)
and corresponding versions of approximation (3.31) (solid red): $A=1$,
$m/l_{1}=1$ with (a) $l_{1}=10$ and (b) $l_{1}=50$.  Figure 4: Empirical
probabilities of reaching the barrier $H$ (dashed black) and corresponding
versions of approximation (3.31) (solid red): $A=1$, $m/l_{1}=2$ with
$l_{1}=10$ and (b) $l_{1}=50$.

Figure 5: Empirical probabilities of reaching the barrier $H$ (dashed black) and corresponding versions of approximation (3.31) (solid red): $A=1$, $m/l_{1}=10$ with $l_{1}=10$ and (b) $l_{1}=50$.  Figure 6: Empirical probabilities of reaching the barrier $H$ (dashed black) and corresponding versions of approximation (3.31) (solid red): $A=1$, $m/l_{1}=5$ with (a) $l_{1}=10$ and (b) $l_{1}=50$. Table 4: Approximations for $\mathbb{E}_{\infty}\tau_{S,l_{0},l_{1}}(H)$ with $l_{0}=25,l_{1}=50,A=1$. $H$ | -5 | -4.5 | -4 | -3.5 | -3 | -2.5 | -2
---|---|---|---|---|---|---|---
(3.26) | 78 | 96 | 116 | 144 | 177 | 218 | 271
$\mathbb{E}_{\infty}\tau_{S,l_{0},l_{1}}(H)$ | 77 | 95 | 115 | 144 | 179 | 217 | 276
Table 5: Approximations for $\mathbb{E}_{\infty}\tau_{S,1,l_{1}}(H)$ with $l_{1}=10,A=1$. $H$ | 2 | 2.25 | 2.5 | 2.75 | 3 | 3.25 | 3.5
---|---|---|---|---|---|---|---
(LABEL:Approx2) | 20 | 32 | 49 | 71 | 100 | 137 | 185
(3.26) | 29 | 43 | 58 | 82 | 109 | 147 | 201
$\mathbb{E}_{\infty}\tau_{S,l_{0},l_{1}}(H)$ | 30 | 43 | 59 | 81 | 111 | 148 | 201

####  3.2.3 The presence of nuisance parameters

Here we briefly consider statistics aimed at detecting a transient change when
certain nuisance parameters require estimation. The brevity of this discussion
is because in practice for online change-point problems, the behaviour of the
time series under the null hypothesis of no change-point is often observed for
a lengthy period of time. This allows for the accurate estimation of certain
nuisance parameters and they can therefore be assumed known. Many of the
following statistics appear in some form in [52] when addressing the offline
change-point problem, and a number of approximations for the false alarm error
are provided. The log likelihood ratio given in (3.1), where $f$ and $g$ are
given in (3.10), is

$\displaystyle\Lambda_{\nu,\nu+l}=A\sum_{j=\nu+1}^{n+l}\left(y_{j}-\mu-\frac{A}{2}\right)\,.$

Using motivation from [47], if $\mu$ is unknown, $l$ is unknown but bounded
$l_{0}\leq l\leq l_{1}$ and $A$ is known, then one can replace $\mu$ with its
maximum likelihood estimator under $H_{\infty}$;
$\hat{\mu}:=\sum_{i=1}^{n}y_{i}/n$ to obtain:

$\displaystyle Z^{1}_{n}:=\max_{\begin{subarray}{c}0\leq\nu<\nu+l\leq n\\\
l_{0}\leq l\leq
l_{1}\end{subarray}}A\sum_{j=\nu+1}^{\nu+l}\left(y_{j}-\hat{\mu}-\frac{A}{2}\right)\,.$

In [48], $\mu$ was replaced with its average over the null and alternative
hypotheses to obtain the true likelihood ratio statistic:

$\displaystyle Z^{2}_{n}:=\max_{\begin{subarray}{c}0\leq\nu<\nu+l\leq n\\\
l_{0}\leq l\leq
l_{1}\end{subarray}}A\sum_{j=\nu+1}^{\nu+l}\left(y_{j}-\hat{\mu}-\frac{A}{2}\left(1-\frac{l}{n}\right)\right).$

If $\mu$ and $A$ are both unknown, the square root of the log likelihood ratio
statistic is:

$\displaystyle Z^{3}_{n}:=\max_{\begin{subarray}{c}0\leq\nu<\nu+l\leq n\\\
l_{0}\leq l\leq
l_{1}\end{subarray}}\frac{\left(\sum_{j=\nu+1}^{\nu+l}y_{j}\right)-l\hat{\mu}}{\sqrt{l(1-\frac{l}{n})}}\,.$

It is not obvious how one can translate the offline change-point results of
[47, 48, 49, 50, 51] to address the online change point problem in the
presence of nuisance parameters. This is because the change-point statistics
$Z^{1}_{n},Z^{2}_{n}$ and $Z^{3}_{n}$ can no longer be written recursively as
the estimators for the unknown parameters get updated at each time $n$.

###  3.3 Optimality criteria

For online detection of transient changes, optimality criteria like (2.12) and
(2.3) do not have much meaning as the change in distributions is not permanent
(signal can be missed). Instead, optimality involving the maximisation of the
probability of detection under a constraint on the false alarm risk is more
applicable, see [56, 57]. One could use a worst-case criterion of the form:

$\displaystyle\inf_{\nu}{\rm
P}_{\nu}\\{\tau(H)-\nu<\\!T\,|\,\tau(H)\\!>\\!\nu\\}\,,$ (3.33)

where $T>1$ is the maximum length of time after the change-point occurs that
it must be detected; this is problem specific and is therefore chosen by the
user. By imposing the condition of a long run with no false alarms, another
possible criterion is

$\displaystyle\lim_{\nu\rightarrow\infty}{\rm
P}_{\nu}\\{\tau(H)-\nu<\\!T\,|\,\tau(H)\\!>\\!\nu\\}.$ (3.34)

Using ARL as the measure of false alarm risk, a stopping rule
$\tau\in\Delta(C)$ is then optimal for a given $C$ if it maximises (3.33) or
(3.34); recall $\Delta(C)=\\{\tau:E_{\infty}\tau\geq C\\},\,\,\,C>1.$

####  3.3.1 MOSUM procedure

For the MOSUM procedure given in (3.11), the quantity (3.34) was the focus of
study in [24] and built on the continuous time results of [26]. For $T=l+L$,
the quantity (3.34) is equivalent to:

$\displaystyle{\cal{P}}_{S}(H,A,L)\\!$ $\displaystyle:=$
$\displaystyle\\!\\!\\!\lim_{\nu\rightarrow\infty}{\rm
P}_{\nu}\\{S_{n,L}\\!>\\!H$
$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\text{
for some
}n\\!\in\\![\nu^{\prime}\\!+\\!1,\nu\\!+\\!l\\!-\\!1]\,|\,\tau_{S,L}(H)\\!>\\!\nu^{\prime}\\},$

with $\nu^{\prime}:=\nu-L$.

Formally, we require $\nu\to\infty$ in (3.3.1). This is to ensure that the
sequence of moving sums $\\{S_{n,L}\\}_{n}$ reaches the stationary behaviour
under the null hypothesis and given that we have not crossed the threshold
$H$. However, as discussed [24, 26], this stationary regime is reached very
quickly and in all approximations it is enough to only require $\nu\geq 2L$.

The reasoning behind the choice $T=l+L$ is as follows. Assume
$\mathbb{H}_{\nu}$ with $\nu<\infty$, and that $\nu$ is suitably large. If the
barrier $H$ is reached for any sum $S_{n,L}$ with $n\leq\nu^{\prime}$ then,
since there are no parts of the signal in the sums $S_{0,L},\ldots
S_{\nu^{\prime},L}$, we classify the event of reaching the barrier as a false
alarm. Each one of the sums $S_{\nu^{\prime}+1,L},\ldots,S_{\nu+l-1,L}$ has
mean larger than $L\mu$ as it contains at least a part of the signal. Reaching
the barrier $H$ by any of these sums will be classified as a correct detection
of the signal. If neither of these sums reaches $H$, then we say that we
failed to detect the signal and further events when $S_{n,L}\geq H$ with
$n\geq\nu+l$ will again be classified as false alarms. In Figure 7 we display
the values $\mathbb{E}_{\nu}S_{n,L}$ as a function of $n$.

0$\mu
L$$(A\\!+\\!\mu)\min(l,L)$$\nu^{\prime}$$\nu\\!+\\!l$$\nu^{\prime}\\!+\\!\max(l,\\!L)$$\nu^{\prime}\\!+\\!\min(l,\\!L)$$n$
Figure 7: $\mathbb{E}_{\nu}S_{n,L}$ as a function of $n$

Define the function

$\displaystyle Q(n;A,L,\nu^{\prime})\\!:=\\!\left\\{\begin{array}[]{cl}0&\\\
A(n-\nu^{\prime})&\\\ A\min(l,L)&\\\
\\!\\!\\!\\!A(\min(l,L)\\!-\\!(n\\!-\\!\nu^{\prime}\\!-\\!\max(l,L))&\end{array}\right.$
$\displaystyle\begin{array}[]{cl}&\text{ for }n\leq\nu^{\prime}\,\text{ or
}n\geq\nu+l\\\ &\text{ for }\nu^{\prime}<n\leq\nu^{\prime}+\min(l,L)\\\
&\text{ for }\nu^{\prime}\\!+\\!\min(l,L)<n\leq\nu^{\prime}\\!+\\!\max(l,L)\\\
&\text{ for }\nu^{\prime}+\max(l,L)<n\leq\nu+l-1\,.\end{array}$

By subtracting $\mathbb{E}_{\nu}S_{n,L}$ from the threshold $H$ and
standardising the random variables $S_{n,L}$ the power of the test given in
(3.3.1) can be expressed in terms of probability under $\mathbb{H}_{\infty}$:

$\displaystyle\\!{\cal
P}_{\xi}(h,A,L)\\!:=\\!\\!\lim_{\nu\rightarrow\infty}\\!{\rm
P}_{\infty}\\!\bigg{\\{}\xi_{n,L}\\!>\\!h\\!-\\!\frac{Q(n;A,L,\nu^{\prime})}{\sigma\sqrt{L}}$
$\displaystyle\text{ for some
}n\\!\in\\![\nu^{\prime}\\!+\\!1,\nu\\!+\\!l\\!-\\!1]\,\big{|}\,{\tau}_{\xi}(h)\\!>\\!\nu^{\prime}\bigg{\\}}\\!,$

where ${\cal{P}}_{S}(H,A,L)={\cal P}_{\xi}(h,A,L)$. To approximate ${\cal
P}_{\xi}(h,A,L)$, the approach taken in [24] was similar to the approach take
to approximate ARL. The approach is as follows. We firstly approximate the
problem in the continuous-time setting and compute probabilities for the
Gaussian process $S(t)$. Then, use the results of D. Siegmund to correct the
continuous time probability for discrete time. Fix $\gamma=A\sqrt{L}/\sigma$,
$\kappa=\nu^{\prime}/L$, $\lambda=l/L$ and define the function

$\displaystyle Q(t;\gamma,\kappa,\lambda)=\left\\{\begin{array}[]{cl}0&\\\
\gamma(t-\kappa)&\\\ \gamma\min(1,\lambda)&\\\
\gamma(\min(1,\lambda)-(t-\kappa-\max(1,\lambda))&\end{array}\right.$
$\displaystyle\begin{array}[]{cl}&\text{ for }t\leq\kappa\text{ or
}t\geq\kappa+1+\lambda.\\\ &\text{ for }\kappa<t\leq\kappa+\min(1,\lambda)\\\
&\text{ for }\kappa+\min(1,\gamma)<t\leq\kappa+\max(1,\lambda)\\\ &\text{ for
}\kappa+\max(1,\lambda)<t\leq\kappa+1+\lambda\,.\end{array}$

The diffusion approximation for the power of the test is

$\displaystyle{\cal P}(h,A):=\lim_{\kappa\rightarrow\infty}{\rm
P}_{\infty}\\{S(t)>h-Q(t;\gamma,\kappa,\lambda)$ $\displaystyle\text{ for some
}n\in[\kappa,\kappa+1+\lambda]\,|\,\tilde{\tau}(h)>\kappa\\},$

where $\tilde{\tau}(h)=\inf\\{t>0:S(t)>h\\}$. That is, we make the
approximation

$\displaystyle{\cal P}_{\xi}(h,A,L)\cong{\cal P}(h,A)$

by assuming $L\to\infty$.

The complexity of computation of the diffusion approximation ${\cal P}(h,A)$
and its discrete-time corrected version depends on the choice of $L$ in
comparison to $l$. Here, we will only consider the scenario of $\lambda=l/L=1$
which corresponds to the case of $l$ known at the MOSUM construction stage.
The two other cases of $\lambda>1$ and $\lambda<1$ are studied in [24].

For $\lambda=1$, the diffusion approximation for ${\cal P}_{\xi}(h,A,L)$ given
in (3.3.1) reduces to

$\displaystyle{\cal P}(h,A)\\!=\\!\lim_{\kappa\rightarrow\infty}{\rm
P}_{\infty}\\{S(t)\geq h-Q(t;\gamma,\kappa)\,\,$ $\displaystyle\mbox{for some
$t\\!\in$ \\!$[\kappa,\kappa+2]$}\big{|}\,\tilde{\tau}(h)>\kappa\\}\\!,$

where $Q(t;\gamma,\kappa)=\gamma\max\left\\{0,1-|t-(\kappa+1)|\right\\}$. The
barrier $h-Q(t;\gamma,\kappa)$ is depicted in Figure 9.

$h$$\kappa\\!+\\!1$$\kappa\\!+\\!2$$\kappa$$h\\!-\\!\gamma(t\\!-\\!\kappa)$$(h\\!-\\!\gamma)+$$\\!\gamma(t\\!-\\!\kappa\\!-\\!1)$$h\\!-\\!\gamma$$t$
Figure 8:  Barrier $h-Q(t;\gamma,\kappa)$ for $\lambda=1$.

$h$$1$$2$$3$$0$$h\\!+\\!\gamma\\!-\\!\gamma t$$h\\!-\\!3\gamma\\!+\\!\gamma
t$$h\\!-\\!\gamma$$t$ Figure 9: Barrier $B(t;h,0,-\gamma,\gamma)$.

The probability (3.3.1) was considered in [26], where approximations accurate
to more than $4$ decimal places were developed. Define the following two
conditional probabilities:

$\displaystyle F_{h,0}(1|x)\\!\\!\\!\\!$ $\displaystyle:=$
$\displaystyle\\!\\!\\!\\!{\rm P}_{\infty}(S(t)<h\text{ for all
}t\in[0,1]\,\,$ $\displaystyle|\,\,S(0)=x)\,,$ $\displaystyle
F_{h,0,-\gamma,\gamma}(3|x)\\!\\!\\!\\!$ $\displaystyle:=$
$\displaystyle\\!\\!\\!\\!{\rm P}_{\infty}(S(t)<B(t;h,0,-\gamma,\gamma)$
$\displaystyle\text{ for all }t\in[0,3]\,\,|\,\,S(0)=x),$

where the barrier $B(t;h,0,-\gamma,\gamma)$ is defined as

$\displaystyle
B(t;h,0,-\gamma,\gamma)=\left\\{\begin{array}[]{ll}\\!\\!\\!h,&0\leq t\leq
1\\\ \\!\\!\\!h-\gamma(t-1),&1<t\leq 2\\\
\\!\\!\\!h-\gamma+\gamma(t-2),&2<t\leq 3\\\
\\!\\!\\!0&\mbox{otherwise,}\end{array}\right.$

and is depicted in Figure 9. From [26] we obtain

$\displaystyle{\cal P}(h,A)\cong
1-\frac{F_{h,0,-\gamma,\gamma}(3|0)}{F_{h,0}(1|0)}\,,$ (3.44)

where

$\displaystyle
F_{h,0}(1\,|\,x)=\Phi(h)-\exp\left(-(h^{2}-x^{2})/2\right)\Phi(x)$

and

$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\\!F_{h,0,-\gamma,\gamma}(3\,|\,x)=\frac{e^{{\gamma}^{2}/2}}{\varphi(x)}\int_{-x-h}^{\infty}\int_{x_{2}-h+\gamma}^{\infty}e^{-\gamma(x_{3}-x_{2})}$
$\displaystyle\\!\\!\\!\\!\times\det\\!\\!\left[\begin{matrix}\varphi(x)&\varphi(-x_{2}\\!-\\!h)\\\
\varphi(h)&\varphi(-x\\!-\\!x_{2})\\\
\varphi(x_{2}\\!+\\!2h\\!+\\!x)&\varphi(h)\\\
\varphi(x_{3}\\!+\\!3h\\!-\\!\gamma\\!+\\!x)\\!\\!&\varphi(x_{3}\\!+\\!2h\\!-\\!\gamma\\!-\\!x_{2})\\\
\end{matrix}\\!\\!\right.$ $\displaystyle\
\\!\\!\left.\begin{matrix}\varphi(-x_{3}\\!-\\!2h\\!+\\!\gamma)&\Phi(-x_{3}\\!-\\!2h\\!+\\!\gamma)\\\
\varphi(-x\\!-\\!x_{3}\\!-\\!h\\!+\\!\gamma)\\!\\!&\\!\Phi(-x\\!-\\!x_{3}\\!-\\!h\\!+\\!\gamma)\\\
\varphi(x_{2}\\!-\\!x_{3}\\!+\\!\gamma)&\Phi(x_{2}\\!-\\!x_{3}\\!+\\!\gamma)\\\
\varphi(h)&\Phi(h)\\\ \end{matrix}\right]\\!\\!dx_{3}dx_{2}\,.$

To compute the approximation (3.44) one needs to numerically evaluate a two-
dimensional integral which is a routine problem for modern computers.

Correcting approximation (3.44) for discrete time can be performed in the same
manner as correcting the ARL approximations in Section 3.2.1. This results in
the approximation

$\displaystyle{\cal P}_{\xi}(h,A,L)\cong
1-\frac{F_{h_{L},0,-\gamma,\gamma}(3|0)}{F_{h_{L},0}(1|0)}\,,$
$\displaystyle\text{ where }h_{L}:=h+\omega_{L}.$ (3.46)

In Figures 10-11, the thicker black dashed line corresponds to the empirical
values of the BCP ${\cal P}_{\xi}(h,A,L)$ computed from 100 000 simulations
with different values of $L$ and $\gamma$, where $\mu=0$ and $\sigma=1$. The
solid red line corresponds to the approximation in (3.3.1). The dot-dashed
blue line corresponds to the diffusion approximation given in (3.44). The axis
are: the $x$-axis shows the value of $\gamma$. The $y$-axis denotes the
probabilities of reaching the barrier. The graphs, therefore, show the
empirical probabilities of ${\cal P}_{\xi}(h,A,L)$ and values of approximation
(3.3.1).

Figure 10: Empirical probabilities of ${\cal P}_{\xi}(h,A,L)$ (thick dashed
black) and its approximations (solid red and solid blue) for $h=3$. Figure 11:
Empirical probabilities of ${\cal P}_{\xi}(h,A,L)$ (thick dashed black) and
its approximations (solid red and solid blue) for $h=4$.

From Figures 10-11, we see that approximation (3.3.1) is very accurate even
for a very small $L=5$. We also see the significance of the discrete-time
correction; whilst the diffusion approximation provides sensible results
should you compare it with $L=100$, for $L=5$ the diffusion approximation is
very far off.

###  3.4 Comparison of tests

In this section, we compare the power of the MOSUM test in (3.11) against the
generalised MOSUM statistic (3.24) and the CUSUM test given in (2.2)
specialised for this Gaussian example when used to detect a transient change.
Secondly, but also simultaneously, we compare the power of the MOSUM test as
$\lambda=l/L$ varies in $[0.5,2]$; the purpose is to demonstrate when the
generalised MOSUM statistic becomes beneficial when the exact value of $l$ is
unknown. Here, we shall consider the power criterion given in (3.34) and set
$T=2l$. For the MOSUM test, the power is then

$\displaystyle{\cal{P}}_{S}(H_{1},A,L)\\!:=\\!\\!\\!\lim_{\nu\rightarrow\infty}{\rm
P}_{\nu}\\{S_{n,L}\\!>\\!H_{1}\text{ for some }$ $\displaystyle
n\\!\in\\![\nu-L\\!+\\!1,\nu\\!-L+\\!2l\\!-\\!1]\,|\,\tau_{S,L}(H_{1})\\!>\\!\nu-L\\}.$

For the generalised MOSUM test, the power is

$\displaystyle{\cal{P}}_{Z}(H_{2},A,l_{0},l_{1})\\!:=\\!\\!\\!\lim_{\nu\rightarrow\infty}{\rm
P}_{\nu}\\{{S}_{n,{l_{0},l_{1}}}\\!>\\!H_{2}\text{ for some }$ $\displaystyle
n\\!\in\\![\nu-
l_{1}\\!+\\!1,\nu\\!-l_{1}+\\!2l\\!-\\!1]\,|\,\tau_{S,l_{0},l_{1}}(H_{2})\\!>\\!\nu-
l_{1}\\}.$

The power of the CUSUM test for the transient change considered in then
equivalent to

$\displaystyle{\cal{P}}_{V}(H_{3},A)\\!:=\\!\\!\\!\lim_{\nu\rightarrow\infty}{\rm
Pr}_{\nu}\\{V_{n}\\!>\\!H_{3}\text{ for some }$ $\displaystyle
n\\!\in\\![\nu+\\!1,\nu\\!+\\!2l\\!-\\!1]\,|\,\tau_{V}(H_{3})\\!>\\!\nu\\}.$

To compare the three tests, the thresholds $H_{1}$, $H_{2}$ and $H_{3}$ have
been set such that
$\mathbb{E}_{\infty}\tau_{M}(H_{1})=\mathbb{E}_{\infty}\tau_{Z}(H_{2})=\mathbb{E}_{\infty}\tau_{V}(H_{3})=500$.
Determination of $H_{1}$ for MOSUM has been computed using the accurate
approximation in (3.2.1). For the generalised MOSUM procedure, $H_{2}$ is
found via Monte Carlo simulations. Determination of $H_{3}$ for CUSUM was
obtained using tabulated values given in [42, p. 3237].

In the first example shown in Figure 12, we have set $A=1$ and $l=10$. For the
MOSUM test, we considered values of $L\in[5,20]$ to ensure
$\lambda\in[0.5,2]$. For each $\lambda$, the values of
${\cal{P}}_{S}(H_{1},A,L)$ can be accurately approximated using results in
[24] or via Monte Carlo methods and are displayed with a solid black line. The
dashed orange line depicts ${\cal{P}}_{Z}(H_{2},A,5,20)$ which corresponds to
prior knowledge that $l$ is between $[5,20]$. The shorter dashed blue line
corresponds to ${\cal{P}}_{V}(H_{3},A)$ which has been obtained via Monte
Carlo simulations. In Figure 13, we set $A=0.5$ and $l=20$. For the MOSUM
procedure, we consider values of $L\in[10,40]$ to ensure $\lambda\in[0.5,2]$.
In this figure, the dashed orange line depicts ${\cal{P}}_{Z}(H_{2},A,10,40)$
which corresponds to prior knowledge that $l$ is between $[10,40]$. The
shorter dashed blue line corresponds to ${\cal{P}}_{V}(H_{3},A)$ obtained via
Monte Carlo simulations.

Figure 12: Power of three tests with $A=1$ and $l=10$ and ARL$=500$.  Figure
13: Power of three tests with $A=0.5$ and $l=20$ and ARL$=500$.

From Figures 12-13, one can observe the advantage of knowing $l$ since the
largest value of ${\cal{P}}_{S}(H_{1},A,L)$ is the largest power of all three
tests and is obtained for $\lambda=l/L=1$. In these figures, the values of
$\lambda=l/L$ such that ${\cal{P}}_{S}(H_{1},A,L)$ exceeds
${\cal{P}}_{Z}(H_{2},A,5,20)$ (Figures 12) and ${\cal{P}}_{Z}(H_{2},A,10,40)$
(Figures 13) shows the freedom in the choice of $L$ such that when $l$ is
unknown, you still benefit over only assuming $l$ is bounded (similarly for
CUSUM case when considering the dashed blue line). From these figures it is
clear that unless you are very fortunate in choosing $L$ close to $l$ for the
MOSUM test, you should use the generalised MOSUM test if $A$ is known.
Unfortunately, there are no convenient analytic results for this test.
Moreover, both the generalised MOSUM procedure and CUSUM procedures require
the additional knowledge of $A$; this is not true for MOSUM. For the choice of
parameters considered in both examples, the additional knowledge of a
transient change leads to obvious benefits in power; those is seen by
comparing the generalised MOSUM orange lines with the blue CUSUM lines. Of
course,
${\cal{P}}_{Z}(H_{2},A,l_{0},l_{1})\rightarrow{\cal{P}}_{S}(H_{1},A,l)$ as
$l_{0},l_{1}\rightarrow l$.

###  3.5 An application to real world data

Hydrostatic pressure testing is important safety precaution for the Oil and
Gas industry, see [58]. Pressure testing is performed to confirm a pressure
containing system is structurally sound and not leaking. Tests are performed
by increasing the pressure in the system, expanding the pressure body, until
the pressure reaches a pre-defined value typically equal to or larger than the
body rated design pressure, then holding it there for a long enough time
period to confirm there are no leaks, until eventually releasing the pressure.
When performing tests offshore on floating Vessel/Drilling Rigs (Rig) this is
complicated by the Rig’s movement due to the ocean waves, which introduce
nearly sinusoidal fluctuations in pressure. Many of these tests are performed
in real time and in parallel. Locating automatically when a test has been
performed is essential for pressure analysis to determine if a leak is present
and this is not obvious when noise is large. Typical example data is shown in
Figure 14. When performing pressure tests, the hold periods can differ in
length and amplitudes (pressure).

Figure 14: Typical pressure data

A sensible way of modelling the data under the null hypothesis of no pressure
test could be $z_{t}=s_{t}+y_{t}$, where $s_{t}$ represents the signal
introduced by the wave motion and $y_{t}$ can be modelled as i.i.d.
$N(\mu,\sigma^{2})$ and reflects the random noise that is present in the
system. In most scenarios, there is significant pre-test data so $s_{t},\mu$
and $\sigma$ can be estimated with great accuracy and therefore assumed known.
How to estimate $s_{t}$ or in general how to remove all main components of a
signal leaving only noise can be performed using Singular Spectrum Analysis,
see [59, 60]. When a pressure test begins, this can be reflected with a change
in mean of the $y_{t}$; that is, under a pressure test
$\mathbb{E}y_{t}=\mu+A$. The value of $A$ is often constant, but can differ
between tests and is generally unknown. Each test can differ in duration but
typical lengths vary between $l\in[50,100]$ units of time. One has to detect a
transient change in mean of $y_{t}=z_{t}-s_{t}$. The behaviour of
$z_{t}-s_{t}$ is shown in Figure 15. In Figure 16, we depict the MOSUM
statistic setting $L=75$. The horizontal line in this figure corresponds to
the threshold required for an ARL of 5000. The MOSUM statistic indicates the
location of three performed pressure tests and has the great advantage of not
requiring knowledge $A$ when determining the ARL threshold unlike the
generalised MOSUM and CUSUM procedures. A similar example is shown in Figures
17-18, where $L=150$ has been selected; three tests have been clearly located.

Figure 15:  Behaviour of $y_{t}$.  Figure 16:  MOSUM statistic based on Figure
15 with $L=50$ and ARL$=5000$.  Figure 17: Behaviour of $y_{t}$.  Figure 18:
MOSUM statistic based on Figure 17 with $L=150$ and ARL$=5000$.

## Acknowledgements

This work was partially supported by EPSRC grant EP/M024830.

## References

  * [1] M. Basseville and I. Nikiforov.  Detection of Abrupt Changes: Theory and Application.  Prentice-Hall Inc, 1993.
  * [2] A. Tartakovsky, I. Nikiforov, and M. Basseville.  Sequential analysis: Hypothesis testing and changepoint detection.  CRC Press, 2014.
  * [3] A. Zhigljavsky and A. Kraskovsky.  Detection of abrupt changes of random processes in radiotechnics problems.  St. Petersburg University Press, St. Petersburg, Russia.(in Russian), 1988\.
  * [4] V. Poor and O. Hadjiliadis.  Quickest detection.  Cambridge University Press, 2008.
  * [5] W. Shewhart.  Economic control of quality of manufactured product.  Macmillan And Co Ltd, London, 1931.
  * [6] E. Page.  Continuous inspection schemes.  Biometrika, 41(1/2):100–115, 1954.
  * [7] A. Shiryaev.  On optimum methods in quickest detection problems.  Theory of Probability & Its Applications, 8(1):22–46, 1963.
  * [8] S. Roberts.  A comparison of some control chart procedures.  Technometrics, 8(3):411–430, 1966.
  * [9] S. Roberts.  Control chart tests based on geometric moving averages.  Technometrics, 42(1):97–101, 2000.
  * [10] G. Lorden.  Procedures for reacting to a change in distribution.  The Annals of Mathematical Statistics, 42(6):1897–1908, 1971.
  * [11] G. Moustakides.  Optimal stopping times for detecting changes in distributions.  Annals of Statistics, 14(4):1379–1387, 1986.
  * [12] M. Pollak.  Optimal detection of a change in distribution.  The Annals of Statistics, pages 206–227, 1985.
  * [13] Y. Ritov.  Decision theoretic optimality of the CUSUM procedure.  The Annals of Statistics, pages 1464–1469, 1990.
  * [14] L. Gordon and M. Pollak.  An efficient sequential nonparametric scheme for detecting a change of distribution.  The Annals of Statistics, pages 763–804, 1994.
  * [15] T. Lai.  Sequential changepoint detection in quality control and dynamical systems.  Journal of the Royal Statistical Society: Series B (Methodological), 57(4):613–644, 1995.
  * [16] B. Guépié, L. Fillatre, and I. Nikiforov.  Detecting a suddenly arriving dynamic profile of finite duration.  IEEE Transactions on Information Theory, 63(5):3039–3052, 2017\.
  * [17] B. Guépié, L. Fillatre, and I. Nikiforov.  Sequential detection of transient changes.  Sequential Analysis, 31(4):528–547, 2012.
  * [18] T. Lai.  Control charts based on weighted sums.  The Annals of Statistics, 2(1):134–147, 1974.
  * [19] C. Chu, K. Hornik, and C. Kaun.  MOSUM tests for parameter constancy.  Biometrika, 82(3):603–617, 1995.
  * [20] P. Bauer and P. Hackl.  The use of MOSUMS for quality control.  Technometrics, 20(4):431–436, 1978.
  * [21] P. Bauer and P. Hackl.  An extension of the MOSUM technique for quality control.  Technometrics, 22(1):1–7, 1980.
  * [22] J. Glaz, V. Pozdnyakov, and S. Wallenstein.  Scan statistics: Methods and applications.  Springer Science & Business Media, 2009.
  * [23] J. Glaz, J. Naus, and X. Wang.  Approximations and inequalities for moving sums.  Methodology and Computing in Applied Probability, 14(3):597–616, 2012.
  * [24] J. Noonan and A. Zhigljavsky.  Power of the MOSUM test for online detection of a transient change in mean.  Sequential Analysis, 39(2):269–293, 2020.
  * [25] J. Noonan and A. Zhigljavsky.  Approximations for the Boundary Crossing Probabilities of Moving Sums of Random Variables.  Methodology and Computing in Applied Probability, 2020.
  * [26] A. Zhigljavsky and J. Noonan.  First passage times for Slepian process with linear and piecewise linear barriers.  Extremes, pages 1–25, 2021.
  * [27] M. Bell.  Information theory and radar waveform design.  IEEE Transactions on Information Theory, 39(5):1578–1597, 1993\.
  * [28] V. Poor.  An introduction to signal detection and estimation.  Springer Science & Business Media, 2013.
  * [29] R. Streit and P. Willett.  Detection of random transient signals via hyperparameter estimation.  IEEE transactions on signal processing, 47(7):1823–1834, 1999.
  * [30] L. Schmerr.  Fundamentals of ultrasonic nondestructive evaluation.  Springer, 2016.
  * [31] A. Bianchi, L. Mainardi, E. Petrucci, M. Signorini, M. Mainardi, and S. Cerutti.  Time-variant power spectrum analysis for the detection of transient episodes in HRV signal.  IEEE Transactions on Biomedical Engineering, 40(2):136–144, 1993\.
  * [32] V. Moskvina and A. Zhigljavsky.  An algorithm based on Singular Spectrum Analysis for change-point detection.  Communications in Statistics—Simulation and Computation, 32(2):319–352, 2003.
  * [33] E. Brodsky and B. Darkhovsky.  Nonparametric methods in change point problems, volume 243.  Springer Science & Business Media, 2013.
  * [34] P. Fryzlewicz.  Wild binary segmentation for multiple change-point detection.  Annals of Statistics, 42(6):2243–2281, 2014.
  * [35] H. Cho and P. Fryzlewicz.  Multiple-change-point detection for high dimensional time series via sparsified binary segmentation.  Journal of the Royal Statistical Society: Series B: Statistical Methodology, pages 475–507, 2015.
  * [36] K. Korkas and P. Fryzlewicz.  Multiple change-point detection for non-stationary time series using wild binary segmentation.  Statistica Sinica, pages 287–311, 2017.
  * [37] P. Fryzlewicz.  Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection.  Journal of the Korean Statistical Society, 49(4):1027–1070, 2020\.
  * [38] H. Dette and J. Gösmann.  A likelihood ratio approach to sequential change point detection for a general class of parameters.  Journal of the American Statistical Association, 115(531):1361–1377, 2020.
  * [39] J. Gösmann, T. Kley, and H. Dette.  A new approach for open-end sequential change point monitoring.  arXiv preprint arXiv:1906.03225, 2019.
  * [40] A. Polunchenko, A. Tartakovsky, and N. Mukhopadhyay.  Nearly optimal change-point detection with an application to cybersecurity.  Sequential Analysis, 31(3):409–435, 2012.
  * [41] D. Siegmund.  Sequential Analysis: Tests and Confidence Intervals.  Springer, 1985.
  * [42] G. Moustakides, A. Polunchenko, and A. Tartakovsky.  Numerical comparison of cusum and shiryaev–roberts procedures for detecting changes in distributions.  Communications in Statistics—Theory and Methods, 38(16-17):3225–3239, 2009.
  * [43] A. Tartakovsky.  Asymptotic performance of a multichart CUSUM test under false alarm probability constraint.  In Proceedings of the 44th IEEE Conference on Decision and Control, pages 320–325. IEEE, 2005.
  * [44] M. Pollak.  Average run lengths of an optimal method of detecting a change in distribution.  The Annals of Statistics, pages 749–779, 1987.
  * [45] T. Lai.  Information bounds and quick detection of parameter changes in stochastic systems.  IEEE Transactions on Information Theory, 44(7):2917–2929, 1998\.
  * [46] A. Tartakovsky.  Discussion on “is average run length to false alarm always an informative criterion?” by Yajun Mei.  Sequential Analysis, 27(4):396–405, 2008.
  * [47] B. Levin and J. Kline.  The CUSUM test of homogeneity with an application in spontaneous abortion epidemiology.  Statistics in Medicine, 4(4):469–488, 1985.
  * [48] D. Siegmund.  Boundary crossing probabilities and statistical applications.  The Annals of Statistics, pages 361–404, 1986.
  * [49] M. Hogan and D. Siegmund.  Large deviations for the maxima of some random fields.  Advances in Applied Mathematics, 7(1):2–22, 1986.
  * [50] D. Siegmund.  Approximate tail probabilities for the maxima of some random fields.  The Annals of Probability, pages 487–501, 1988.
  * [51] Q. Yao.  Large deviations for boundary crossing probabilities of some random fields.  In Journal of Mathematical Research and Exposition, volume 9, pages 181–192, 1989.
  * [52] Q. Yao.  Tests for change-points with epidemic alternatives.  Biometrika, 80(1):179–191, 1993.
  * [53] D. Slepian.  First passage time for a particular Gaussian process.  The Annals of Mathematical Statistics, 32(2):610–612, 1961.
  * [54] L. Shepp.  First passage time for a particular Gaussian process.  Ann. Math. Stat., 42(3):946–951, 1971.
  * [55] J. Noonan and A. Zhigljavsky.  Approximating Shepp’s constants for the Slepian process.  Statistics & Probability Letters, 2019.
  * [56] C. Han, P. Willett, and D. Abraham.  Some methods to evaluate the performance of Page’s test as used to detect transient signals.  IEEE transactions on signal processing, 47(8):2112–2127, 1999.
  * [57] B. Bakhache and I. Nikiforov.  Reliable detection of faults in measurement systems.  International Journal of adaptive control and signal processing, 14(7):683–700, 2000.
  * [58] S. McAleese.  Operational aspects of oil and gas well testing.  Elsevier, 2000.
  * [59] N. Golyandina, V. Nekrutkin, and A. Zhigljavsky.  Analysis of Time Series Structure: SSA and related techniques.  Chapman & Hall/CRC, 2001.
  * [60] N. Golyandina and A. Zhigljavsky.  Singular Spectrum Analysis for time series.  Springer, 2020.