#  Intrinsic Anomaly Detection for Multi-Variate Time Series

Stephan Rabanser
University of Toronto & Vector Institute
<EMAIL_ADDRESS>
Equal contribution.Work done while at Amazon Research. Tim
Januschowski11footnotemark: 1 22footnotemark: 2
Zalando Research
<EMAIL_ADDRESS>
Kashif Rasul
Morgan Stanley
<EMAIL_ADDRESS>
Oliver Borchert22footnotemark: 2
Technical University of Munich
<EMAIL_ADDRESS>
Richard Kurle
Amazon Research
<EMAIL_ADDRESS>
Jan Gasthaus
Amazon Research
<EMAIL_ADDRESS>Michael Bohlke-Schneider
Amazon Research
<EMAIL_ADDRESS>Nicolas Papernot
University of Toronto & Vector Institute
<EMAIL_ADDRESS>
Valentin Flunkert
Amazon Research
<EMAIL_ADDRESS>
###### Abstract

We introduce a novel, practically relevant variation of the anomaly detection
problem in multi-variate time series: _intrinsic anomaly detection_. It
appears in diverse practical scenarios ranging from DevOps to IoT, where we
want to recognize failures of a system that operates under the influence of a
surrounding environment. Intrinsic anomalies are changes in the functional
dependency structure between time series that represent an environment and
time series that represent the internal state of a system that is placed in
said environment. We formalize this problem, provide under-studied public and
new purpose-built data sets for it, and present methods that handle intrinsic
anomaly detection. These address the short-coming of existing anomaly
detection methods that cannot differentiate between expected changes in the
system’s state and unexpected ones, i.e., changes in the system that deviate
from the environment’s influence. Our most promising approach is fully
unsupervised and combines adversarial learning and time series representation
learning, thereby addressing problems such as label sparsity and subjectivity,
while allowing to navigate and improve notoriously problematic anomaly
detection data sets.

##  1 Introduction

Modern applications, appliances, or systems in the physical and virtual world
are equipped with sensors that measure and monitor on the one hand the _state
of a system_ and its sub-components, and on the other hand the _environment_
with which the system interacts. These measurements are typically captured by
monitoring systems as multi-variate time series data sets. Examples can be
found in the Internet-of-Things (IoT) or in the DevOps/AIOps space (Lu et al.,
2009; Lohrmann and Kao, 2011; Nedelkoski et al., 2019; Li et al., 2020;
Krupitzer et al., 2020). Our running example is that of a wind turbine; see
Figure 1. Environmental variables correspond to ambient temperature, wind
speed and direction and system variables correspond to rotator speed and
internal temperature measurements, among many others. We expect internal
temperature measurements to vary naturally with ambient temperature, but a
deviation from the functional relationship between ambient and internal
temperature measurements could point to a problem of the state of the turbine.
It is this change in the functional relationship that is the principal object
of interest of this work. Detecting it allows to understand deteriorations of
state and health of the system independent of the environment.

(a) Wind turbine signal example. The multivariate sensory information for each
entity can be decomposed into environmental variables and system-internal
variables.

(b) Extrinsic anomaly example: The environmental signal is anomalous w.r.t its
typical behaviour. The dependence structure between environmental and system
variables is preserved.

(c) Instrinsic anomaly example: The system signal is anomalous w.r.t the
environment. The dependence structure between environmental varaibles and
system variables is broken.

Figure 1: Structure of time series corpora (left) and different types of
anomalies (middle and right).

To further motivate our approach, consider a simplified system where
_intrinsic normality_ means that a single variable $y$ that represents the
state of a system depends instantaneously on a single environmental signal $x$
via a mapping $h$, i.e., we have $y=h(x)+\varepsilon$, where $\varepsilon$ is
a (not further specified) error term. We then define an _intrinsic anomaly_ as
a point where this functional relationship is broken, i.e. $\varepsilon$
exceeds a threshold $\tau$. The ideal signal to detect an intrinsic anomaly is
the residual $\delta\coloneqq y-h(x)$. Under normal operation this signal
$\delta$ carries no information about the environmental variable $x$ and
thresholding the magnitude of this residual signal allows to define intrinsic
anomalies.

We contrast intrinsic anomalies with anomalies that we would find if we
considered a standard multi-variate anomaly detection problem on environment
variables only, which we refer to as _extrinsic anomalies_. Such problems are
widely studied (see Blázquez-García et al. (2021) for a recent overview). In
the wind turbine example, an extrinsic anomaly could be unexpectedly strong
wind. Detecting it automatically allows to pitch out the blades if they rotate
too fast. However, the pitching out of blades is not an intrinsic anomaly as
it constitutes expected behaviour. Figure 1 provides an illustration of both
anomaly types. Standard anomaly detection methods (Su et al., 2019a; Liu et
al., 2008; Breunig et al., 2000) allow to detect extrinsic anomalies, hence we
refrain from studying them further here.

The data sets for the intrinsic anomaly detection task differ from standard
anomaly detection benchmarks by their division into environment and system
variables. Similar to standard anomaly detection benchmarks (Bogatinovski et
al., 2021; Wu and Keogh, 2021), sparsity and subjectivity of their associated
labels for anomalies are practical concerns. Sparsity may be due to diverse
reasons ranging from practical (e.g., data integration or cost of labelling)
to fundamental (internal system failures are exceedingly rare). Subjectivity
stems from the fact that failure identification often requires human insight
or that alarms come from overly noisy rule-based systems (Bogatinovski et al.,
2021; Wu and Keogh, 2021). Based on our practical experience, it is
unrealistic to obtain a golden data set since both extrinsic and intrinsic
anomalies are domain-specific and subjective. Methods for intrinsic anomaly
detection should allow to iteratively improve the data. A common task is
checking whether time series similar to a labelled intrinsic anomaly are
labelled as such, and if not to potentially (re)label them.

The desiderata for methods for intrinsic anomaly detection are hence as
follows. They need to (i) detect intrinsic anomalies and separate them from
extrinsic anomalies. While an oversimplification, we will consider the problem
of detection extrinsic anomaly detection as solved for the sake of this paper;
(ii) sparsity of labels means that we cannot expect a supervised method to
work well and instead must rely mostly on unsupervised methods; and (iii) the
subjectivity of labels motivates the need for methods to facilitate human
interaction with a potentially high-dimensional, complex data set in an
interactive and iterative fashion (e.g., by being able to identify similar
time series snippets or similar extrinsic/intrinsic anomalies). Given these
desiderata, a natural choice for a method for intrinsic anomaly detection
would be based on unsupervised representation learning which enables the
computation of distances in the embedding space to identify related time
series interactively.

To summarize, our contributions are three-fold. First, we present and
formalize a new anomaly detection task in multi-variate time series: intrinsic
anomaly detection (Sec. 2). Second, we present a method to tackle intrinsic
anomaly detection (Sec. 3) which satisfies the afore-mentioned desiderata. In
particular, we propose a novel adaptation of unsupervised time series
representations (Franceschi et al., 2019) yielding environment-invariant
embeddings. Third, we provide an initial set of diverse data sets that allows
to study this novel problem in a meaningful way: wind turbines as an IoT use
case, a semi-synthetic DevOps data set, a swinging pendulum simulation, and a
fully synthetic data set based on simple generative models. Our experiments in
Sec. 4 show that our approach based on environment-invariant embeddings
performs favorably when compared against standard anomaly detection methods as
well as natural baselines.

##  2 Intrinsic Anomaly Detection: Problem Statement

Let $\mathbf{z}\in\mathbb{R}^{D\times T}$ be a multi-variate, equally spaced
time series of length $T$ with $D$ dimensions corresponding to a
system/environment combination. We denote by
$\mathbf{z}_{t:t+j}=(\mathbf{z}_{t},\mathbf{z}_{t+1},\ldots,\mathbf{z}_{t+j})$
a time series snippet of $\mathbf{z}$. The standard way to define an anomaly
at time $t$ for a multi-variate time series is to say that at time $t$ we have
an anomaly if $\mathbf{z}_{t}$ differs from its expected value, denoted by
$\hat{\mathbf{z}}_{t}$, by more than a threshold $\tau$ (Blázquez-García et
al., 2021). A canonical choice in anomaly detection for multi-variate time
series is a predictive time series model that, given the history of the multi-
variate time series, provides an estimate $\hat{\mathbf{z}}_{t}$ via a
forecast or a probabilistic generative model.

In the context of this paper, we have additional structure that we can take
advantage of by assuming a decomposition $\mathbf{z}=(\mathbf{x},\mathbf{y})$
such that time series $\mathbf{x}\in\mathbb{R}^{N\times T}$ allows to predict
time series $\mathbf{y}\in\mathbb{R}^{M\times T}$, $D=N+M$, via a function $h$
(modulo an error term $\varepsilon$):

$\mathbf{y}_{t}=h(\mathbf{x}_{0:t};\mathbf{y}_{0:t-1})+\varepsilon\,.$ (1)

For all $t$ we have the residual
$\delta_{t}=d(\mathbf{y}_{t},h(\mathbf{x}_{0:t};\mathbf{y}_{0:t-1}))$ for a
distance metric $d$. We call $\mathbf{x}$ the set of _environmental_ and
$\mathbf{y}$ the set of _system_ time series.

We define an _extrinsic anomaly_ at time $t$ as an anomaly solely in
$\mathbf{x}$, i.e., at time $t$ we have
$d(\mathbf{x}_{t},\hat{\mathbf{x}}_{t})>\tau_{x}$ for a threshold $\tau_{x}$.
At the same time, the functional dependency between $\mathbf{x}$ and
$\mathbf{y}$ remains unchanged, meaning that the residual signal $\delta_{t}$
does not exhibit an anomaly: $\delta_{t}\leq\tau_{y}$. Conversely, an
_intrinsic anomaly_ at time $t$ consists of an anomaly in the residual signal
$\delta_{t}$ itself, i.e., for a threshold $\tau_{y}$ we have
$d(\mathbf{y}_{t},\hat{\mathbf{y}}_{t})=d(\mathbf{y}_{t},h(\mathbf{x}_{0:t},\mathbf{y}_{0:t-1}))>\tau_{y}$.

We have not further prescribed what $h$ is apart from being a multi-variate
function; $h$ may be a highly non-linear and/or stochastic function that
depends on the state of the system as well as (lagged) environmental
variables. Depending on the application scenario, $h$ is either given or needs
to be obtained. For example, we may have a known physical model describing a
system’s interaction with the environment or we may assume the need to learn a
complex function $h_{V}\approx h$ from data. In the former case, this
relationship between environment and system state will hold also for extreme
values in the environment (an extrinsic anomaly), but break whenever the
system exhibits internal faults. In the latter case, we may be able to
reliably learn parameters $V$ for $h_{V}$ assuming that intrinsic anomalies
are rare. In the data sets that we consider, it is a realistic assumptions
that we must learn $h_{V}$, so we focus on this scenario.

##  3 Representation Learning for Intrinsic Anomaly Detection

In designing methods for intrinsic anomaly detection, we recall desiderata
from the introduction: they must be able to separate intrinsic from extrinsic
anomalies, rely on unsupervised learning, and allow to identify similar time
series effectively in data quality improvement tasks. We assume that the
decomposition into environmental and system time series is given (and does not
need to be identified), however we do not assume the functional dependence $h$
to be known.

Our approach relies on two main ingredients: (i) learning effective
representations via contrastive learning; and (ii) equipping them to
distinguish intrinsic and extrinsic anomalies by forcing representations to be
invariant to the environment. We motivate both ingredients separately in Sec.
3.1 and 3.2 respectively and then present the proposed models.

###  3.1 Contrastive Losses and Anomaly Detection

Time series representation learning techniques such as (Franceschi et al.,
2019) aim to provide general-purpose representations that are independent of
the downstream task. However, there is a particularly close connection to
popular approaches in anomaly detection. For example, Ruff et al. (2018)
propose to learn representations of data points whose enclosing hyper-sphere
has minimal size, formally

$\min_{W}\sum_{i}(f_{W}(\mathbf{z}_{i:i+\ell})-\mathbf{c})^{2}+\Omega\,.$ (2)

Here, $\mathbf{z}_{i:i+\ell}$ are contiguous time series snippets of length
$\ell$ of $\mathbf{z}$, $f_{W}$ is an embedding function
$R^{D\times\ell}\rightarrow R^{d}$ parameterized by $W$, $\mathbf{c}\in R^{d}$
is a given center of a hyper-sphere, and $\Omega$ is a regularization term.
Ruff et al. (2018) argue that under the common and realistic assumption that
anomalies are rare, a reasonable embedding function $f_{W}$ would be learnt
such that embeddings of normal time series are close to the cluster center
$\mathbf{c}$ while abnormal ones have a larger distance to $\mathbf{c}$. The
distance between $f_{W}(\mathbf{z}_{i:i+\ell})$ and $\mathbf{c}$ can hence be
treated as a metric for the normality of a time series in the embedded space.

The connection to contrastive learning is then intuitively clear as they
construct embeddings equipped with a distance function that fulfills a similar
purpose. Instead of choosing a cluster center explicitly,111Ruff et al. (2018)
use the average of the initially found embeddings of the training set vectors
in the first iteration of the optimization algorithm as cluster centers.
typical contrastive losses flexibly choose points that are (heuristically)
close to each other (e.g., close in time for time series). We formalize this
as

$\min_{W}\sum_{i}\sum_{j}d(f_{W}(\mathbf{z}_{i:i+\ell}),f_{W}(\mathbf{z}_{j:j+k}))+\Omega\,,$
(3)

where $d$ is a distance function in the embedding space (e.g., $\ell_{2}$ as
in (2)). Typically, the cluster centers $\mathbf{z}_{j:j+k}$ are referred to
as _reference_ examples and $\mathbf{z}_{i:i+\ell}$ are _positive_ examples
with $i,\ell$ randomly chosen such that
$\mathbf{z}_{i:i+\ell}\subseteq\mathbf{z}_{j:j+k}$.

In order to guarantee the second property of (2) – that abnormal points are
far from each other – many contrastive losses explicitly require that points
that are (heuristically) different are further away in the embedding space.
These _negative_ examples should be far apart in terms of the cluster centers,
leading to the desired property that individual hyper-spheres have small
volume (normal values are concentrated) and hyper-spheres among themselves are
subject to a repelling effect (summarized by $\Omega$ in (3)).

###  3.2 Representations for Intrinsic Anomaly Detection

While contrastive losses allow for standard anomaly detection in multi-variate
time series (see e.g., Blázquez-García et al. (2021) for an overview), they do
not allow fine-grained distinctions into intrinsic and extrinsic anomalies.
Following the arguments above, we expect extrinsic and intrinsic anomalies to
correspond to outliers in the embedding space because they are rare. It is
unclear however why an embedding of an extrinsic anomaly, should necessarily
be distant from an embedding of an intrinsic anomaly. To guarantee that we can
separate extrinsic and intrinsic anomalies in the embedding space, it is
natural to consider the inclusion of an inductive bias, namely an invariance
of the representations towards the environment, to ensure this separation.
First, note that embeddings that are invariant to changes in the environment
do not carry information about the environment, analogous to the ideal
residual signal in Sec. 2. For an environment-invariant embedding, intrinsic
normality corresponds to proximity in the embedding space (because this is the
frequent case), while intrinsic anomalies violate the standard invariance and
would hence be more distant in the embedding space. Second, we can modify
contrastive losses explicitly to encode that a broken functional relationship
between environment and system variables correspond to greater distance in the
embedding space.

#### Adversarial learning for Environment-Invariance

For obtaining representations of $\mathbf{z}$ that are invariant to the
environment $\mathbf{x}$, a popular choice is to apply an adversarial learning
approach Xie et al. (2018); Long et al. (2017) which we adapt to our
unsupervised setting. We train the predictor $g_{U}$ in an adverserial manner
jointly with the embedding model: The predictor’s parameters are trained to
maximize the likelihood of $\mathbf{x}$ given embeddings $f_{W}(\mathbf{z})$;
the weights of the embedding model $f_{W}$ are trained to maximize the error
of the predictor. This encourages the embeddings to become invariant to
$\mathbf{x}$. Note that for the prediction task, we use a multi-class
classification approach by discretizing $\mathbf{x}$ appropriately. Hence, we
compute the negative log probability of the correct class as our loss, i.e.,

$\mathcal{L}_{\text{adv}}(g_{U}(f_{W}(\mathbf{z}_{i:j})),\mathbf{x}_{i:j})=-\log\frac{1}{g_{U}(f_{W}(\mathbf{z}_{i:j}))_{\mathbf{x}_{i:j}}}.$
(4)

For $g_{U}$ we use a neural network with a softmax function as the final
layer. The full loss is given by

$\mathcal{L}_{\text{adv}}(W,U)=\sum_{i,j}\mathcal{L}_{\text{adv}}(g_{U}(f_{W}(\mathbf{z}_{i:j})),\mathbf{x}_{i:j})\,.$
(5)

#### Negative Examples in Contrastive Losses

For embeddings of $\mathbf{z}$, we can equip (3) with negative examples that
explicitly encode when the dependency structure is broken. We construct these
negative examples as system and environment variables originating from
different time snippets. We denote
$\mathbf{z}^{\text{pos}}=(\mathbf{x}^{\text{pos}},\mathbf{y}^{\text{pos}})=(\mathbf{x}_{i:i+\ell},\mathbf{y}_{i:i+\ell})$
as a positive and
$\mathbf{z}^{\text{ctx}}=(\mathbf{x}^{\text{ctx}},\mathbf{y}^{\text{ctx}})=(\mathbf{x}_{j:j+k},\mathbf{y}_{j:j+k})$
as a reference time series snippet where $i,j,k,\ell$ are chosen (randomly)
such that positive examples are contained in the reference time series. In the
construction of negative examples, we explicitly break the dependency
structure between $\mathbf{x}$ and $\mathbf{y}$ in defining
$\mathbf{z}^{\text{neg}}=(\mathbf{x}^{\text{neg}},\mathbf{y}^{\text{pos}})=(\mathbf{x}_{r:r+\ell},\mathbf{y}_{i:i+\ell})$
where $r$ is (randomly) chosen such that
$\mathbf{x}_{\text{neg}}\cap\mathbf{x}_{\text{pos}}\approx\emptyset$. Figure
2(a) provides an illustration. The contrastive loss uses these three sample
types such that time series with default dependence structure should be
embedded close to each other while time series with varying dependence
structures should be distant from each other. For an embedding network $f_{W}$
with parameters $W$, the loss function takes the following form:

$\displaystyle\begin{split}\mathcal{L}_{\text{contr}}(W)=&\sum_{\text{pos}}\sum_{\text{ctx}}d(f_{W}(\mathbf{z}^{\text{pos}}),f_{W}(\mathbf{z}^{\text{ctx}}))\
-\sum_{\text{neg}}\sum_{\text{ctx}}d(f_{W}(\mathbf{z}^{\text{neg}}),f_{W}(\mathbf{z}^{\text{ctx}}))\;.\end{split}$
(6)

We aim to minimize (6). Note that (6) is amenable to batch optimization and we
can further construct negative examples such that negative environment
variables are chosen from a different series in the batch.

###  3.3 Model Overview

Our approach consists of two main components: a _predictor network_ $g_{U}$
for environment invariance, and an _embedding network_ $f_{W}$ which learns
embeddings using contrastive losses. Figure 2(b) depicts the main components
of our approach. For $g_{U}$, we choose a single-layer feed-forward neural
network to place more onus on $f_{W}$ to become invariant to the environment.
For discretization of $\mathbf{x}$, we choose standard quantile-based binning
with 20 buckets.

The basic building block of our embedding network architecture consists of
stacked temporal dilated causal convolutions (Bai et al., 2018) following
(Franceschi et al., 2019). We choose such a temporal convolutional neural
network (TCN) for its speed of computation and its general robustness to
facilitate experimentation.

(a) Reference, positive, and negative sample selection overview. While
reference and positive sample selection corresponds to Franceschi et al.
(2019), negative samples are selected such that the dependence structure
between environment and system signals is explicitly broken.

(b) Architecture overview with contrastive and adversarial learning for
environment invariance. At its core, we employ a contrastive learning
component that pushes reference and positive samples closer to each other in
the embedding space while trying to push context and negative samples apart.
We further have an adversarial component to obtain environment invariance
that, given a positive embedded sample, tries to reconstruct the context of
the positive sample as badly as possible (red dotted arrows) while
reconstructing the context of the negative sample as well as possible (green
dotted arrows).

Figure 2: Overview of the sample selection process for contrastive learning
and architecture diagram.

The final loss is a combination of a contrastive loss with a regularizer to
obtain environment invariance, which is summarized as follows:

$\mathcal{L}(W,U)=\mathcal{L}_{\text{contr}}(W)-\lambda\mathcal{L}_{\text{adv}}(W,U)\,,$
(7)

where $\mathcal{L}_{\text{contr}}(W)$ is the contrastive loss (6),
$\mathcal{L}_{\text{adv}}(W,U)$ is the environment-invariance loss (5), and
$\lambda$ is a hyperparameter that governs the trade-off between them. We seek
a saddle point $(\hat{W},\hat{U})$ such that
$\hat{W}=\operatorname*{arg\,min}_{W}\mathcal{L}(W,\hat{U})$ and
$\hat{U}=\operatorname*{arg\,min}_{U}\mathcal{L}(\hat{W},U)$. We use a
gradient-reversal layer between $f$ and $g$ to identify the saddle point in
(7) as proposed in (Ganin et al., 2016). Moyer et al. (2018) provide further
theoretical and practical justifications for it.

##  4 Experiments

In our experiments, we empirically evaluate our approach both quantitatively
and qualitatively. Public time series data sets for (standard) anomaly
detection222Wu and Keogh (2021) convincingly argue that many of these datasets
should be abandoned. are not suitable for intrinsic anomaly detection, so we
do not consider them. Instead, we evaluate on four different data sets, each
of which allows for a separation into environmental and system signals. We
discuss these data sets first, then describe our evaluation approach and the
models under consideration, and finally present our results.

Table 1: Data set summary statistics. Each dataset consists of multiple distinct sub-systems. For each system, we have a number of time series snippets available with length $T$. $N$ and $M$ denote the number of environmental and system variables, respectively. Dataset | # Systems | # Series | $T$ | $N$ | $M$
---|---|---|---|---|---
Synthetic | 1 system | 360 | 1440 | 2 | 2
Pendulum | 1 pendulum | 300 | 144 | 1 | 2
Turbine | 4 turbines | 360 | 144 | 5 | 4
DevOps | 1 app | 128 | 128 | 1 | 3

###  4.1 Datasets

The selection of benchmark datasets aims to balance physical and virtual
systems as well as synthetic and real-world data. Note that synthetic data
exhibits perfect labels, while real-world data typically does not. While
unfortunate, we believe that this subjectivity and noisiness must be embraced
as fundamental in the task. Hence, we place additional emphasis on qualitative
experiments. Appendix A contains further details and illustrations on the data
sets and Table 1 captures details of the datasets under consideration.

#### Synthetic

We generate a synthetic dataset based on a simple sinusoidal generative model.
We generate two environmental signals, as well as two system signals. We
inject two types of anomalies into the data: (i) extrinsic anomalies, i.e.,
anomalies in the environmental which are also instantly reflected in the
system variables and (ii) intrinsic anomalies only in the system variables.

#### Pendulum

We consider the case of a swinging pendulum with added control signals, where
we control the dampening of the acceleration from the outside as an
environmental signal (towards which we want to be invariant). As intrinsic
anomalies, we consider those where we inject an anomaly as a change in the
length of the chord, which we capture as part of the system signal. Our aim
with this data set is to understand how well our models handle cases where the
dependency structure between system and environment is significantly more
complex (i.e. time-shifted and non-linear) than in the Synthetic data set.

#### DevOps

is a new, semi-synthetic data set333We will open-source both the raw data as
well as the set-up to produce the data. which resembles data sets behind
corporate firewalls. It is based on a cloud-based microservice demo
app,444https://github.com/microservices-demo/microservices-demo commonly used
in a DevOps context (Wu et al., 2020). The environment for the application are
user interactions with the app, approximated by the network outbound traffic
of an auxiliary application that induces synthetic load/user behaviour on the
app. The system signals consist of metrics such as CPU consumption, memory or
disk consumption of the app. We inject anomalies both in the user behavior
which lead to extrinsic anomalies and in the internal state to obtain
intrinsic anomalies. Note that although we have almost total control of the
application, the labelling of anomalies is still imperfect, thereby adding
further to the complications of public anomaly detection benchmarks (Wu and
Keogh, 2021).

#### Wind Turbine

is a public wind turbine data set by Energias de
Portugal.555https://opendata.edp.com/pages/homepage/ The time series panel can
be separated into environmental signals (e.g., wind speed/direction, ambient
temperature) and system signals (e.g., rotational speeds of generators,
temperature on different components, power output). Note that this data set
contains only few (43) labelled anomalies and visual inspections of the data
reveals inconsistencies with these labels, see e.g., Figure 2(e). Hence,
purely quantitative evaluations cannot be taken at face value. Operating under
this limitation, we illustrate the versatility of our approach qualitatively
in Sec. 4.4.

###  4.2 Model and Evaluation Details

For a comprehensive empirical comparison, we consider the following families
of approaches: (i) approaches based on deep embeddings to varying degree of
complexity, the most closely related to our approach; (ii) approaches based on
classical time series embeddings; and (iii) standard anomaly detection methods
on time series. We consider these approaches on both the original time series
and on the residual signal. We further note that we only perform
hyperparameter tuning on the Synthetic data set as we do not have enough
labels available on other datasets. Appendix B provides more fine-grained
details on particular hyperparameter choices for the models described next.

#### Deep Embeddings

We consider the following models in our comparison which we list in increasing
order of complexity. We can choose to ignore the functional dependency $h$ and
learn representations of $\mathbf{z}$ with a TCN equipped with contrastive
losses. We denote this approach as BasicEmb. Next, we propose a two-step
approach for environment-invariance by explicitly approximating the functional
dependence $h$ between $\mathbf{x}_{t}$ and $\mathbf{y}_{t}$ using a neural
network $h_{V}$ first, and then learning embeddings of the _residual_ signal
$\hat{\delta}:=h_{V}(\mathbf{x})-\mathbf{y}$ in a second step. We refer to
this approach as ResEmbRegr. Finally, we use the approach outlined in Sec. 3
which we call EnvInvEmb. We choose $\lambda=10^{-3}$ as determined by an
ablation study (see Appendix B).

#### Classical Embeddings

We compare against two instances of (Lubba et al., 2019): Catch22 and
ResCatch22 which compute a feature vector per original and residual time
series respectively similar to ResEmbRegr.

#### Anomaly Detection Methods

We consider a simple thresholding mechanism, ResTresh, which thresholds the
residual signal $\hat{\delta}$ that is also obtained in ResEmbRegr;
OmniAnomaly (Su et al., 2019a) as a popular example for deep anomaly
detection; and as examples for classical approaches: Isolation Forests (Liu et
al., 2008) IF, Local Outlier Factor (Breunig et al., 2000) LOF, and One-class
SVM (Schölkopf et al., 1999) OCSVM.

#### Evaluation Details

For each dataset, we break the original multi-variate time series into time
series snippets (length described in Table 1) and classify each snippet as
abnormal if it contains an intrinsic anomaly. Although the length of an
intrinsic anomaly is variable, intrinsic anomalies appear as intervals in our
data sets and not as isolated single-point anomalies. While embedding-based
approaches can handle variable-length input series, classic embeddings and
traditional anomaly detection approaches provide an anomaly score for each
time step. To compare these methods on an equal footing, we aggregate anomaly
scores from individual time steps to a single score for a full snippet. We
report mean results over 5 random seeds.

Table 2: AUROC results on the intrinsic anomaly detection task.

Category | Signal Type | Detection Approach | Synthetic | Pendulum | DevOps | Turbine
---|---|---|---|---|---|---
Deep Embeddings | Original | EnvInvEmb |  0.999 ($\pm$ 0.002) |  0.980 ($\pm$ 0.002) |  0.587 ($\pm$ 0.007) |  0.756 ($\pm$ 0.022)
BasicEmb |  0.512 ($\pm$ 0.022) |  0.969 ($\pm$ 0.013) |  0.535 ($\pm$ 0.041) |  0.632 ($\pm$ 0.015)
2-7 | Residual | ResEmbRegr |  1.000 ($\pm$ 0.000) |  0.951 ($\pm$ 0.015) |  0.532 ($\pm$ 0.036) |  0.725 ($\pm$ 0.018)
Classic Embeddings | Original | Catch22 |  0.494 ($\pm$ 0.008) |  0.904 ($\pm$ 0.000) |  0.573 ($\pm$ 0.000) |  0.512 ($\pm$ 0.000)
2-7 | Residual | ResCatch22 |  1.000 ($\pm$ 0.000) |  0.891 ($\pm$ 0.000) |  0.577 ($\pm$ 0.000) |  0.680 ($\pm$ 0.000)
Anomaly Detection | Original | LOF |  0.557 ($\pm$ 0.016) |  0.541 ($\pm$ 0.042) |  0.518 ($\pm$ 0.033) |  0.578 ($\pm$ 0.026)
IF |  0.500 ($\pm$ 0.014) |  0.502 ($\pm$ 0.024) |  0.489 ($\pm$ 0.013) |  0.512 ($\pm$ 0.018)
OCSVM |  0.499 ($\pm$ 0.022) |  0.500 ($\pm$ 0.013) |  0.569 ($\pm$ 0.025) |  0.521 ($\pm$ 0.017)
OmniAnomaly |  0.788 ($\pm$ 0.108) |  0.642 ($\pm$ 0.082) |  0.513 ($\pm$ 0.023) |  0.561 ($\pm$ 0.026)
2-7 | Residual | LOF |  0.537 ($\pm$ 0.046) |  0.467 ($\pm$ 0.030) |  0.573 ($\pm$ 0.070) |  0.679 ($\pm$ 0.045)
| IF |  0.513 ($\pm$ 0.020) |  0.525 ($\pm$ 0.015) |  0.507 ($\pm$ 0.008) |  0.534 ($\pm$ 0.012)
| OCSVM |  0.546 ($\pm$ 0.012) |  0.534 ($\pm$ 0.009) |  0.571 ($\pm$ 0.013) |  0.533 ($\pm$ 0.010)
| OmniAnomaly |  0.801 ($\pm$ 0.075) |  0.717 ($\pm$ 0.113) |  0.435 ($\pm$ 0.028) |  0.492 ($\pm$ 0.067)
| ResTresh |  1.000 ($\pm$ 0.011) |  0.510 ($\pm$ 0.000) |  0.619 ($\pm$ 0.000) |  0.845 ($\pm$ 0.032)

Table 3: The $\ell_{2}$ distance gap between embeddings of correct and incorrect classes on the anomaly detection task. Environment-invariant embeddings induce the largest gap. | BasicEmb | ResEmbRegr | Catch22 | EnvInvEmb
---|---|---|---|---
Synthetic |  0.051 ($\pm$ 0.041) |  0.641 ($\pm$ 0.314) |  0.008 ($\pm$ 0.01) |  1.944 ($\pm$ 0.039)
Pendulum |  0.446 ($\pm$ 0.059) |  0.433 ($\pm$ 0.074) |  0.019 ($\pm$ 0.00) |  0.486 ($\pm$ 0.178)
DevOps |  0.014 ($\pm$ 0.006) |  0.035 ($\pm$ 0.008) |  0.013 ($\pm$ 0.00) |  0.042 ($\pm$ 0.007)
Turbine |  0.075 ($\pm$ 0.007) |  0.319 ($\pm$ 0.103) |  0.004 ($\pm$ 0.00) |  0.450 ($\pm$ 0.085)

Table 4: F-1 scores for the anomaly type classification task. While primarily designed to capture intrinsic anomalies, the disentanglement yielded by environment-invariant embeddings allows for the identification of extrinsic anomalies. | BasicEmb | ResEmbRegr | Catch22 | ResCatch22 | EnvInvEmb
---|---|---|---|---|---
2-class |  0.85 ($\pm$ 0.04) |  0.98 ($\pm$ 0.01) |  0.85 ($\pm$ 0.02) |  0.99 ($\pm$ 0.01) |  0.99 ($\pm$ 0.01)
3-class |  0.74 ($\pm$ 0.03) |  0.59 ($\pm$ 0.08) |  0.71 ($\pm$ 0.03) |  0.64 ($\pm$ 0.06) |  0.89 ($\pm$ 0.03)

(c) Clean series; labelled clean.

(d) Anomalous series; labelled anomalous.

(e) Anomalous series; labelled clean.

Figure 3: Nearest-neighbor analysis of EnvInvEmb on the Turbine dataset. Each
panel shows environmental signals on the top (blue signals) and system signals
at the bottom (orange signals). The first row shows (a) a normal reference
series; (b) an abnormal reference series (anomaly between time steps 20 and
70); and (c) an anomalous reference series that was mistakenly labeled as
clean in the test set. Below each of these series we show 3 nearest neighbors
in the learned embedding space. For the left and center column, both the
correct label and our predicted label match. For series (c), the reference
time series is labelled as clean, but its nearest neighbors are abnormal.

###  4.3 Quantitative Results

We study both the raw intrinsic anomaly detection performance as well as the
disentanglement of extrinsic and intrinsic anomalies in the learned latent
spaces.

#### Intrinsic Anomaly Detection Performance

Acknowledging recent concern on the validity of time series anomaly benchmarks
(Kim et al., 2021), our main quantitative results report AUROC scores on the
intrinsic anomaly detection task. For the ResThresh method, we compute the
AUROC score based on the maximum residual value over the full residual series.
For the embedding-based approaches, we use a $k$-nearest-neighbor classifier
($k=5$) in the embedding space to determine a discrete anomaly label.

Table 4.2 summarizes our main quantitative results. First, we note that the
difficulty of intrinsic anomaly detection differs widely with the data sets.
For example, intrinsic anomaly detection seems relatively easier on the
Turbine data set compared with the DevOps data set despite the latter being
semi-synthetically generated with controlled labels. Second, we note that
EnvInvEmb leads to overall superior results when comparing the embedding based
approaches (the first three rows in Table 4.2) and is overall competitive
results in many cases, but not always. For the DevOps and Turbine data sets,
ResTresh is overall best by a margin. The explanation here is that the simple
instantaneous mapping of ResTresh is a good inductive bias for these data sets
– however, this is in particular not the case for Pendulum. For the Pendulum
data set, it is worthwhile to note that EnvInvEmb delivers an overall superior
approach despite the complex non-linear interaction between the environmental
and system signals and only a linear environment-predictor component. We
further note that our approach is superior to the other anomaly detection
methods consistently and independently whether we apply the anomaly detection
methods directly on the time series or when taking the residual.

#### Embedding Space Distances

For methods that produce a mapping onto an embedding space, we can further
determine whether any particular (embedded) time series is closer to the mean
embedding of the respective correct/incorrect class. Concretely, we can
compute the gap between the distance to the correct class for the respective
series and the distance to the incorrect class and average this quantity over
all series in the test set. Results are presented in Table 3. This complements
the AUROC results in Table 4.2 by providing an indication of the degree of
entanglement in the embedding space and the resulting nearest-neighborhood
structure. Overall, we observe that environment-invariant embeddings produce
the largest gap.

#### Identifiability of Extrinsic vs Intrinsic Anomalies

We further provide results on the identifiability of both extrinsic anomalies
and intrinsic anomalies. For the Synthetic dataset, we create two distinct
labelings: (i) A 2-class labeling where both normal series snippets and
extrinsic-anomalous series are assigned the label 0 and intrinsic anomalies
only are assigned the label 1; and (ii) a 3-class labeling where normal
snippets are labeled as 0, extrinsic anomalies are labeled as 1, and intrinsic
anomalies are labeled as 2. Performance on the first labeling indicates how
well each method is capable of separating intrinsic anomalies from all
encounterable scenarios (1-vs-all setting) while performance on the second
labeling indicates the level of disentanglement of individual anomaly types in
the embedding space. Note that our evaluation in Table 4.2 relies on 2-class
labeling.

We report the weighted F-1 scores for both labeling strategies as part of
Table 4. We observe that environment-invariant embeddings provide both (i)
strong 1-vs-all performance (2-class) and at the same time still (ii) enable
superior identification of normal and extrinsic-anomalous series (3-class). We
also find that residual-based approaches (i.e. ResEmbRegr and ResCatch22)
provide strong 1-vs-all performance but they do so at the expense of extrinsic
anomaly identification. These findings suggests that environment-invariant
embeddings enable us to learn more complex non-linear dependence structures
and patterns that go beyond simple residual signals.

###  4.4 Qualitative Results: A Case Study On Turbines

The embeddings induce a nearest-neighbor structure which allows us to navigate
the time series corpus via distances. This can be helpful in exploring the
data set and uncovering data or label issues. In Figure 3, we show this in
qualitative results for environment-invariant embeddings on the Turbine
dataset. The first two columns show sanity checks: reference time series (top
row) labelled as normal or abnormal have as their nearest neighbors (same
column as the reference time series) normal and abnormal time series,
respectively. Despite the multi-variate nature of the data, visual inspection
confirms the plausibility of the nearest neighbors. The last column in Figure
3 shows an example where a reference time series is labelled as normal, but
its nearest neighbors consist of time series that are labeled as abnormal.
This could point to a labelling issue, i.e., it may make sense to re-label the
reference time series in column (c) as abnormal for the abnormal stretches
0-40 and 100-130.

##  5 Related work

Anomaly detection in time series has a rich body of literature, see e.g., Su
et al. (2019a); Carmona et al. (2021); Ren et al. (2019); Ayed et al. (2020);
Wu and Keogh (2021); Su et al. (2019b); Guha et al. (2016); Blázquez-García et
al. (2021) for a recent overview. While different variants of anomalies are
the subject of discussion, we are not aware of a prior discussion of intrinsic
anomalies (see e.g., Foorthuis (2021) for a comprehensive overview of anomaly
types). Intrinsic anomaly detection is a subset of the more general class of
contextual anomalies where _context_ is more broadly and fuzzily defined (Han
et al., 2012). The closest relative are conditional anomalies (Song et al.,
2007) defined for tabular data, however they ignore the interplay between
extrinsic and intrinsic anomalies. We are not aware of other publicly
available multi-variate time series data sets that lend themselves to targeted
study of intrinsic anomaly detection. Existing detection methods can be
adopted to the intrinsic anomaly detection problem by relying on the residual
signal, however our experiments show that custom-built methods are superior. A
possibility for future work is the refinement of our problem statement with a
causal treatment, see Janzing et al. (2019) for an example outside the time
series context as well as discovery of the causal dependency structure (Haufe
et al., 2009; Qiu et al., 2020, 2012).

Similar to anomaly detection for time series, representation learning for time
series has a rich body of literature, e.g., Franceschi et al. (2019); Zerveas
et al. (2021); Lubba et al. (2019); Christ et al. (2017). The connection
between anomaly detection and contrastive losses is not new (see e.g., Tack et
al. (2020); Sohn et al. (2021) for recent examples), but it is under-explored
in the time series domain. While we extend Franceschi et al. (2019) for
practical reasons such as speed of experimentation and general robustness, we
remark our approach carries over also to other architectures, such as Zerveas
et al. (2021).

Arguably, representation learning is less prominently discussed in time series
compared to other disciplines such as computer vision or natural language
processing (Chen et al., 2020; van den Oord et al., 2019; He et al., 2020;
Fang et al., 2020; Jaiswal et al., 2021). While these fields consider learning
invariant representations a central question, see e.g., Ganin et al. (2016);
Moyer et al. (2018); Akash et al. (2021); Achille and Soatto (2018), this has
not been the case for the time series domain to the best of our knowledge.
Correspondingly, work on domain adaptation is in its infancy (Jin et al.,
2021). Our approach attempts to change this and relies heavily on Xie et al.
(2018) by considering invariance with respect to variables included in the
data.

##  6 Conclusion

We present a novel variation of the anomaly detection task in multi-variate
time series where an intrinsic anomaly manifests itself as a departure from
the typical functional relationship between an environment and a system. Our
main method for addressing intrinsic anomaly detection relies on unsupervised
time series representation learning. We combine contrastive losses for
representation learning with an adversarial learning component that yields
environment-invariant embeddings. This environment invariance allows to
isolate intrinsic anomalies in the embedding space and leads to favorable
performance in the detection tasks. Our empirical evaluation establishes both
a framework for evaluating intrinsic anomalies and introduces purposefully
designed data sets. We show that the resulting representations allow for
meaningful data interactions by exploring nearest neighbors in the embedding
space, thereby addressing data quality issues ubiquitous in anomaly detection
datasets.

## References

  * Achille and Soatto (2018) Alessandro Achille and Stefano Soatto.  Emergence of invariance and disentanglement in deep representations.  _Journal of Machine Learning Research_ , 19(50):1–34, 2018.  URL http://jmlr.org/papers/v19/17-646.html.
  * Akash et al. (2021) Aditya Kumar Akash, Vishnu Suresh Lokhande, Sathya N. Ravi, and Vikas Singh.  Learning invariant representations using inverse contrastive loss, 2021\.
  * Ayed et al. (2020) Fadhel Ayed, Lorenzo Stella, Tim Januschowski, and Jan Gasthaus.  Anomaly detection at scale: The case for deep distributional time series models, 2020.
  * Bai et al. (2018) Shaojie Bai, J. Zico Kolter, and Vladlen Koltun.  An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, 2018.
  * Blázquez-García et al. (2021) Ane Blázquez-García, Angel Conde, Usue Mori, and Jose A. Lozano.  A review on outlier/anomaly detection in time series data.  _ACM Comput. Surv._ , 54(3), apr 2021.  ISSN 0360-0300.  doi: 10.1145/3444690.  URL https://doi.org/10.1145/3444690.
  * Bogatinovski et al. (2021) Jasmin Bogatinovski, Sasho Nedelkoski, Alexander Acker, Florian Schmidt, Thorsten Wittkopp, Soeren Becker, Jorge Cardoso, and Odej Kao.  Artificial intelligence for it operations (aiops) workshop white paper, 2021.
  * Breunig et al. (2000) Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander.  Lof: identifying density-based local outliers.  In _Proceedings of the 2000 ACM SIGMOD international conference on Management of data_ , pages 93–104, 2000.
  * Carmona et al. (2021) Chris U. Carmona, François-Xavier Aubet, Valentin Flunkert, and Jan Gasthaus.  Neural contextual anomaly detection for time series, 2021.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pages 1597–1607. PMLR, 2020.
  * Christ et al. (2017) Maximilian Christ, Andreas W. Kempa-Liehr, and Michael Feindt.  Distributed and parallel time series feature extraction for industrial big data applications, 2017.
  * Fang et al. (2020) Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie.  Cert: Contrastive self-supervised learning for language understanding, 2020.
  * Foorthuis (2021) Ralph Foorthuis.  On the nature and types of anomalies: a review of deviations in data.  _International Journal of Data Science and Analytics_ , 12(4):297–331, Aug 2021.  ISSN 2364-4168.  doi: 10.1007/s41060-021-00265-1.  URL http://dx.doi.org/10.1007/s41060-021-00265-1.
  * Franceschi et al. (2019) Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi.  Unsupervised scalable representation learning for multivariate time series.  _arXiv preprint arXiv:1901.10738_ , 2019.
  * Ganin et al. (2016) Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  _The journal of machine learning research_ , 17(1):2096–2030, 2016.
  * Guha et al. (2016) Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers.  Robust random cut forest based anomaly detection on streams.  In _Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48_ , ICML’16, page 2712–2721. JMLR.org, 2016.
  * Han et al. (2012) Jiawei Han, Micheline Kamber, and Jian Pei.  Data mining concepts and techniques, third edition, 2012.  URL http://www.amazon.de/Data-Mining-Concepts-Techniques-Management/dp/0123814790/ref=tmm_hrd_title_0?ie=UTF8&qid=1366039033&sr=1-1.
  * Haufe et al. (2009) Stefan Haufe, Guido Nolte, Klaus-Robert Mueller, and Nicole Kraemer.  Sparse causal discovery in multivariate time series, 2009.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning, 2020\.
  * Jaiswal et al. (2021) Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon.  A survey on contrastive self-supervised learning, 2021.
  * Janzing et al. (2019) Dominik Janzing, Kailash Budhathoki, Lenon Minorics, and Patrick Blöbaum.  Causal structure based root cause analysis of outliers, 2019.
  * Jin et al. (2021) Xiaoyong Jin, Youngsuk Park, Danielle C. Maddix, Yuyang Wang, and Xifeng Yan.  Domain adaptation for time series forecasting via attention sharing, 2021\.
  * Kim et al. (2021) Siwon Kim, Kukjin Choi, Hyun-Soo Choi, Byunghan Lee, and Sungroh Yoon.  Towards a rigorous evaluation of time-series anomaly detection.  _arXiv preprint arXiv:2109.05257_ , 2021.
  * Krupitzer et al. (2020) Christian Krupitzer, Tim Wagenhals, Marwin Züfle, Veronika Lesch, Dominik Schäfer, Amin Mozaffarin, Janick Edinger, Christian Becker, and Samuel Kounev.  A survey on predictive maintenance for industry 4.0.  _arXiv preprint arXiv:2002.08224_ , 2020.
  * Kurle et al. (2020) Richard Kurle, Syama Sundar Rangapuram, Emmanuel de Bézenac, Stephan Günnemann, and Jan Gasthaus.  Deep rao-blackwellised particle filters for time series forecasting.  _Advances in Neural Information Processing Systems_ , 33, 2020.
  * Li et al. (2020) Yangguang Li, Zhen Ming (Jack) Jiang, Heng Li, Ahmed E. Hassan, Cheng He, Ruirui Huang, Zhengda Zeng, Mian Wang, and Pinan Chen.  Predicting node failures in an ultra-large-scale cloud computing platform: An aiops solution.  _ACM Trans. Softw. Eng. Methodol._ , 29(2), April 2020.  ISSN 1049-331X.  doi: 10.1145/3385187.  URL https://doi.org/10.1145/3385187.
  * Liu et al. (2008) Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou.  Isolation forest.  In _2008 eighth ieee international conference on data mining_ , pages 413–422. IEEE, 2008.
  * Lohrmann and Kao (2011) Björn Lohrmann and Odej Kao.  Processing smart meter data streams in the cloud.  In _2011 2nd IEEE PES International Conference and Exhibition on Innovative Smart Grid Technologies_ , pages 1–8, 2011.  doi: 10.1109/ISGTEurope.2011.6162747.
  * Long et al. (2017) Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan.  Conditional adversarial domain adaptation.  _arXiv preprint arXiv:1705.10667_ , 2017.
  * Lu et al. (2009) Bin Lu, Yaoyu Li, Xin Wu, and Zhongzhou Yang.  A review of recent advances in wind turbine condition monitoring and fault diagnosis.  In _2009 IEEE Power Electronics and Machines in Wind Applications_ , pages 1–7, 2009.  doi: 10.1109/PEMWA.2009.5208325.
  * Lubba et al. (2019) Carl H Lubba, Sarab S Sethi, Philip Knaute, Simon R Schultz, Ben D Fulcher, and Nick S Jones.  catch22: Canonical time-series characteristics.  _Data Mining and Knowledge Discovery_ , 33(6):1821–1852, 2019.
  * Moyer et al. (2018) Daniel Moyer, Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, and Aram Galstyan.  Invariant representations without adversarial training.  In _Proceedings of the 32nd International Conference on Neural Information Processing Systems_ , NIPS’18, page 9102–9111, Red Hook, NY, USA, 2018. Curran Associates Inc.
  * Nedelkoski et al. (2019) Sasho Nedelkoski, Jorge Cardoso, and Odej Kao.  Anomaly detection from system tracing data using multimodal deep learning.  In _2019 IEEE 12th International Conference on Cloud Computing (CLOUD)_ , pages 179–186, 2019.  doi: 10.1109/CLOUD.2019.00038.
  * Qiu et al. (2012) Huida Qiu, Yan Liu, Niranjan A Subrahmanya, and Weichang Li.  Granger causality for time-series anomaly detection.  In _2012 IEEE 12th international conference on data mining_ , pages 1074–1079. IEEE, 2012.
  * Qiu et al. (2020) Juan Qiu, Qingfeng Du, Kanglin Yin, Shuang-Li Zhang, and Chongshu Qian.  A causality mining and knowledge graph based method of root cause diagnosis for performance anomaly in cloud applications.  _Applied Sciences_ , 10(6), 2020.  ISSN 2076-3417.  doi: 10.3390/app10062166.  URL https://www.mdpi.com/2076-3417/10/6/2166.
  * Ren et al. (2019) Hansheng Ren, Bixiong Xu, Yujing Wang, Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing, Mao Yang, Jie Tong, and Qi Zhang.  Time-series anomaly detection service at microsoft.  In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, KDD ’19, page 3009–3017, New York, NY, USA, 2019. Association for Computing Machinery.  ISBN 9781450362016.  doi: 10.1145/3292500.3330680.  URL https://doi.org/10.1145/3292500.3330680.
  * Ruff et al. (2018) Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft.  Deep one-class classification.  In Jennifer Dy and Andreas Krause, editors, _Proceedings of the 35th International Conference on Machine Learning_ , volume 80 of _Proceedings of Machine Learning Research_ , pages 4393–4402. PMLR, 10–15 Jul 2018.  URL https://proceedings.mlr.press/v80/ruff18a.html.
  * Schölkopf et al. (1999) Bernhard Schölkopf, Robert C Williamson, Alexander J Smola, John Shawe-Taylor, John C Platt, et al.  Support vector method for novelty detection.  In _NIPS_ , volume 12, pages 582–588. Citeseer, 1999.
  * Sohn et al. (2021) Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister.  Learning and evaluating representations for deep one-class classification, 2021.
  * Song et al. (2007) Xiuyao Song, Mingxi Wu, Christopher Jermaine, and Sanjay Ranka.  Conditional anomaly detection.  _IEEE Transactions on Knowledge and Data Engineering_ , 19(5):631–645, 2007.  doi: 10.1109/TKDE.2007.1009.
  * Su et al. (2019a) Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei.  Robust anomaly detection for multivariate time series through stochastic recurrent neural network.  In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, KDD ’19, page 2828–2837, New York, NY, USA, 2019a. Association for Computing Machinery.  ISBN 9781450362016.  doi: 10.1145/3292500.3330672.  URL https://doi.org/10.1145/3292500.3330672.
  * Su et al. (2019b) Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei.  Robust anomaly detection for multivariate time series through stochastic recurrent neural network.  In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, KDD ’19, page 2828–2837, New York, NY, USA, 2019b. Association for Computing Machinery.  ISBN 9781450362016.  doi: 10.1145/3292500.3330672.  URL https://doi.org/10.1145/3292500.3330672.
  * Tack et al. (2020) Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin.  Csi: Novelty detection via contrastive learning on distributionally shifted instances.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, _Advances in Neural Information Processing Systems_ , volume 33, pages 11839–11852. Curran Associates, Inc., 2020.  URL https://proceedings.neurips.cc/paper/2020/file/8965f76632d7672e7d3cf29c87ecaa0c-Paper.pdf.
  * van den Oord et al. (2019) Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding, 2019.
  * Wu et al. (2020) Li Wu, Jasmin Bogatinovski, Sasho Nedelkoski, Johan Tordsson, and Odej Kao.  Performance Diagnosis in Cloud Microservices using Deep Learning.  In _AIOPS 2020 - International Workshop on Artificial Intelligence for IT Operations_ , Dubai, United Arab Emirates, December 2020.  URL https://hal.inria.fr/hal-02948735.
  * Wu and Keogh (2021) Renjie Wu and Eamonn Keogh.  Current time series anomaly detection benchmarks are flawed and are creating the illusion of progress.  _IEEE Transactions on Knowledge and Data Engineering_ , page 1–1, 2021.  ISSN 2326-3865.  doi: 10.1109/tkde.2021.3112126.  URL http://dx.doi.org/10.1109/TKDE.2021.3112126.
  * Xie et al. (2018) Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig.  Controllable invariance through adversarial feature learning.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_. Curran Associates Inc., 2018.
  * Zerveas et al. (2021) George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff.  A transformer-based framework for multivariate time series representation learning.  In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ , KDD ’21, page 2114–2124, New York, NY, USA, 2021\. Association for Computing Machinery.  ISBN 9781450383325.  doi: 10.1145/3447548.3467401.  URL https://doi.org/10.1145/3447548.3467401.

##  Appendix A Additional Dataset Details

#### Synthetic data

We generate two environmental signals ($x_{1}$ and $x_{2}$), as well as two
system signals ($y_{1}$, and $y_{2}$) according to the following set of
equations.

$\displaystyle x_{1}(x)$
$\displaystyle=\sin(\frac{x}{275}-50)+\sin(\frac{x}{200})+\epsilon_{1}\sim\mathcal{N}(0,0.05)$
(8) $\displaystyle x_{2}(x)$
$\displaystyle=\sin(\frac{x}{100})+\epsilon_{2}\sim\mathcal{N}(0,0.1)$ (9)
$\displaystyle y_{1}(x)$
$\displaystyle=x_{1}(x)+\epsilon_{3}\sim\mathcal{N}(0,0.1)$ (10)
$\displaystyle y_{1}(x)$
$\displaystyle=x_{1}(x)+\frac{x_{2}(x)}{2}-2+\epsilon_{4}\sim\mathcal{N}(0,0.08)$
(11)

#### Turbine

While the turbine dataset consists of 82 distinct time series, our
experimentation only operates on a subset of these series. In particular,
after preprocessing, we select ambient wind speed (Amb_WindSpeed_Avg,
Amb_WindSpeed_Est_Avg), ambient wind direction (Amb_WindDir_Relative_Avg,
Amb_WindDir_Abs_Avg), and ambient temperature (Amb_Temp_Avg), as our
environmental variables and generator rotational speed (Gen_RPM_Avg),
generator power output (Prod_LatestAvg_TotActPwr), and two internal
temperature sensors (Gear_Bear_Temp_Avg, Hyd_Oil_Temp_Avg) as our system
variables. We select these variables since they do not contain missing values
and are easy to interpret. Our core experiments are based on data from the
2016 data set.

#### Pendulum

We simulated synthetic data of a pendulum including friction and control
signals (rotational acceleration) following the setup in Kurle et al. (2020).
For simulating the ODE, we used the 4-th order Runge-Kutta method. Noisy
control signals were sampled from an Ornstein-Uhlenbeck process, where the
location of the stationary Gaussian distribution follow a sine signal.

##  Appendix B More Experimental Details

###  B.1 Hyperparameters

We train all embedding-based methods outlined in Table 4.2 using batches of
size 16 and a learning rate of $1.9\cdot 10^{-3}$ for 50 epochs. The trained
TCN encoder model uses 32 channels, a kernel size of 3 and is comprised of 10
causal convolution blocks. The embedding size is dynamically determined based
on the length of the time series ($l$) and the number environmental and system
dimensions ($d_{\text{exog}}$ and $d_{\text{endog}}$) as follows: $l\cdot
d_{\text{exog}}\cdot d_{\text{endog}}\cdot 0.1$. The negative and positive
selection windows mapping into the embedding space are chosen at random
locations and are $l\cdot 0.2$ long. Residuals for all residual-based
approaches are obtained using the MLPRegressor class from sklearn with out-of-
the box parameters. Catch22 and OmniAnomaly are also run using default
parameters.

###  B.2 Qualitative Results

Figure 4 depicts the effect of isolating intrinsic anomalies via a predictive
network on Synthetic data. In Figures 5, we show the effect of learning
embeddings on the residuals vs on the actual values. Figure 7 shows
qualitatively that we need to modify off-the-shelf time series embeddings in
order to be able to isolate intrinsic anomalies. Similar results are available
also for the Turbinedataset in Figures 8 and 9. Figure 6 show that purely
residual-based approaches can fail in more complex prediction scenarios.

Figure 4: The obtained residuals based on the Synthetic dataset. The top panel
shows two distinct environmental variables. For the most part, both series
follow a smooth seasonal pattern. The first series shows a noticeable
extrinsic anomaly at the beginning of the series. In the middle panel, we
depict two endogenous variables along with their predictions. Since the
environmental variables do give us some information about the presence of an
extrinsic anomaly, we can easily predict a similar anomaly for the system
variables. However, each system variable exhibits additional anomalies that
are deviating from typical expected behaviour. Considering the residuals in
the bottom panel, we can see that the residuals only show these intrinsic
anomalies but disregard the predicable extrinsic anomaly at the start.

Figure 5: Nearest-neighbor analysis of ResEmbRegr for the Synthetic dataset.
The top panel depicts an anomalous reference series showing the two residual
series, both of which exhibit anomalous behaviour as indicated by the residual
spikes. By considering the nearest (NN) and farthest (FN) neighbour series in
the embedding space, we can visually inspect the embedding quality. The left
panel shows the 3 nearest neighbour series. Clearly, these series align with
the reference series since the same spiking pattern in both residuals is
present. Considering the farthest neighbours in the right panel, we see that
both residual series are noisy without showing pronounced peaks.

Figure 6: The obtained residuals based on the Pendulum dataset. For this data
set, we show the control signal in blue (top panel) and see that we cannot
predict the system variables well with an instant-effect, multi-variate
regression.

Figure 7: Nearest-neighbor analysis of BasicEmb for the Synthetic dataset. The
top panel depicts a reference series consisting of both environmental and
system signals. The system series do not contain any intrinsic anomalies. In
contrast to Figure 5, there is no agreement in the nearest neighbors, as the
embedding is unable to pick up on the separation into extrinsic anomalies and
intrinsic anomalies.

Figure 8: The obtained residuals based on the Turbine dataset. Similar to
Figure 4.

Figure 9: Nearest-neighbor analysis of ResEmbRegr for the Turbine dataset.
Similar to Figure 5

###  B.3 Quantitative Results

#### Embedding Space Distances

We provide more fine-grained results for the embedding space distance
experiment in Table 6.

#### The importance of $\lambda$

We perform an ablation study to determine the influence of trading off the
contrastive loss with the adversarial environment-invariance loss in (7),
denoted by $\lambda$. The results in Table 5, demonstrate that (i) our
particular choice of negative examples already provide a strong degree of
environment invariance ($\lambda=0$); and that (ii) these results can be
further improved by a careful weighting of the adversarial component. Overall,
$\lambda=10^{-3}$ gives the strongest results, which we also use in our main
experimental results in Table 4.2.

Table 5: Sensitivity study on the Synthetic dataset for EnvInvEmb. Dependency-breaking negative examples alone $(\lambda=0)$ already provide strong performance and adding adversarial training with a carefully chosen $\lambda=10^{-3}$ further improves results. $\lambda$ | 0 | $10^{-5}$ | $10^{-4}$ | $10^{-3}$ | $10^{-2}$ | $10^{-1}$ | 1
---|---|---|---|---|---|---|---
AUROC |  0.85 ($\pm$ 0.12) |  0.69 ($\pm$ 0.15) |  0.86 ($\pm$ 0.06) |  0.94 ($\pm$ 0.03) |  0.84 ($\pm$ 0.14) |  0.58 ($\pm$ 0.09) |  0.55 ($\pm$ 0.07)
F1 |  0.96 ($\pm$ 0.03) |  0.92 ($\pm$ 0.04) |  0.96 ($\pm$ 0.02) |  0.99 ($\pm$ 0.01) |  0.97 ($\pm$ 0.03) |  0.87 ($\pm$ 0.04) |  0.86 ($\pm$ 0.04)

Table 6: Distance results on the anomaly detection task with 5 random seeds. Distances to the correct class (Corr) should be small while distances to the incorrect class (Incorr) should be large. We prefer approaches with a large gap between the mean correct disatance and the mean incorrect distance. | BasicEmb | ResEmbRegr | EnvInvEmb
---|---|---|---
Dataset | Corr | Incorr | Gap | Corr | Incorr | Gap | Corr | Incorr | Gap
Synthetic |  0.483 ($\pm$ 0.249) |  0.534 ($\pm$ 0.257) |  0.051 ($\pm$ 0.041) |  0.634 ($\pm$ 0.216) |  1.284 ($\pm$ 0.122) |  0.641 ($\pm$ 0.314) |  0.034 ($\pm$ 0.012) |  1.978 ($\pm$ 0.029) |  1.944 ($\pm$ 0.039)
DevOps |  0.918 ($\pm$ 0.038) |  0.932 ($\pm$ 0.034) |  0.446 ($\pm$ 0.059) |  0.942 ($\pm$ 0.016) |  0.977 ($\pm$ 0.020) |  0.433 ($\pm$ 0.074) |  0.043 ($\pm$ 0.133) |  0.529 ($\pm$ 0.130) |  0.486 ($\pm$ 0.178)
Pendulum |  0.691 ($\pm$ 0.037) |  1.137 ($\pm$ 0.076) |  0.014 ($\pm$ 0.006) |  0.679 ($\pm$ 0.036) |  1.112 ($\pm$ 0.065) |  0.035 ($\pm$ 0.008) |  0.215 ($\pm$ 0.178) |  0.437 ($\pm$ 0.346) |  0.012 ($\pm$ 0.007)
Turbine |  0.927 ($\pm$ 0.024) |  1.002 ($\pm$ 0.022) |  0.075 ($\pm$ 0.007) |  0.691 ($\pm$ 0.186) |  1.010 ($\pm$ 0.272) |  0.319 ($\pm$ 0.103) |  0.065 ($\pm$ 0.079) |  0.516 ($\pm$ 0.161) |  0.450 ($\pm$ 0.085)


65.44142pt+\frac{+2(1-\widehat{\rho}_{R,\alpha})\left(\dfrac{\widehat{\mu}_{1,R,\alpha}-\mu_{1,0}}{\widehat{\sigma}_{1,R,\alpha}}\right)\left(\dfrac{\widehat{\mu}_{2,R,\alpha}-\mu_{2,0}}{\widehat{\sigma}_{2,R,\alpha}}\right)}{1-\widehat{\rho}_{R,\alpha}^{2}}\Bigg{]}$

Case 7 (Fixing values for variances and covariance of two dependent
populations with normal distribution).
If we consider the function

$\boldsymbol{m}^{T}\left(\boldsymbol{\theta}\right)=\left(\sigma_{1}-\sigma_{1,0},\sigma_{2}-\sigma_{2,0},\sigma_{1}\sigma_{2}\rho-\sigma_{12,0}\right),$

the null hypothesis can be written by
$\boldsymbol{m}\left(\boldsymbol{\theta}\right)=\boldsymbol{0}_{3}$.
Therefore,

$\boldsymbol{M}^{T}(\boldsymbol{\theta})=(\boldsymbol{0}_{3\times
2},\boldsymbol{M}_{22}^{T}(\boldsymbol{\theta})),\qquad\boldsymbol{M}_{22}^{T}(\boldsymbol{\theta})=\left(\begin{array}[]{ccc}1&0&0\\\
0&1&0\\\ \sigma_{2}\rho&\sigma_{1}\rho&\sigma_{1}\sigma_{2}\end{array}\right)$

and we denote by

$\displaystyle\boldsymbol{M}^{T}(\boldsymbol{\theta})\boldsymbol{V}_{\alpha}\left(\boldsymbol{\theta}\right)\boldsymbol{M}(\boldsymbol{\theta})=\boldsymbol{M}_{22}^{T}(\boldsymbol{\theta})\boldsymbol{V}_{2,\alpha}\boldsymbol{\left({\boldsymbol{\theta}}\right)M}_{22}(\boldsymbol{\theta}),$

and

$\displaystyle\left(\boldsymbol{M}^{T}(\boldsymbol{\theta})\boldsymbol{V}_{\alpha}\left(\boldsymbol{\theta}\right)\boldsymbol{M}(\boldsymbol{\theta})\right)^{-1}$
$\displaystyle=\left(\boldsymbol{M}_{22}^{T}(\boldsymbol{\theta})\boldsymbol{V}_{2,\alpha}\boldsymbol{\left({\boldsymbol{\theta}}\right)M}_{22}(\boldsymbol{\theta})\right)^{-1}$
$\displaystyle=\boldsymbol{M}_{22}^{-1}(\boldsymbol{\theta})\boldsymbol{D}_{2,\widehat{\sigma}_{1,R,\alpha},\widehat{\sigma}_{2,R,\alpha}}^{-1}\boldsymbol{V}_{2,\alpha}^{-1}\left(\rho\right)\boldsymbol{D}_{2,\widehat{\sigma}_{1,R,\alpha},\widehat{\sigma}_{2,R,\alpha}}^{-1}\left(\boldsymbol{M}_{22}^{T}(\boldsymbol{\theta})\right)^{-1},$

and

$\boldsymbol{m}^{T}(\widehat{\boldsymbol{\theta}}_{R,\alpha})=(\widehat{\sigma}_{1,R,\alpha}-\sigma_{1,0},\widehat{\sigma}_{2,R,\alpha}-\sigma_{2,0},\widehat{\sigma}_{1,R,\alpha}\widehat{\sigma}_{2,R,\alpha}\widehat{\rho}_{R,\alpha}-\sigma_{12,0}).$

Therefore, we get (58).

##  Appendix G Complementary tables for Simulation (Section 6)

|  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.225 | 0.232 | 0.239 | 0.249 | 0.229 | 0.234 | 0.245 | 0.305 | 0.372 | 0.499
| $0.1$ | 0.227 | 0.231 | 0.236 | 0.243 | 0.229 | 0.232 | 0.239 | 0.268 | 0.316 | 0.438
| $0.2$ | 0.237 | 0.239 | 0.244 | 0.251 | 0.239 | 0.241 | 0.245 | 0.250 | 0.276 | 0.355
| $0.3$ | 0.258 | 0.259 | 0.266 | 0.270 | 0.259 | 0.262 | 0.264 | 0.264 | 0.280 | 0.328
| $0.5$ | 0.347 | 0.360 | 0.361 | 0.368 | 0.353 | 0.363 | 0.353 | 0.364 | 0.366 | 0.396
| $0.7$ | 0.596 | 0.580 | 0.761 | 0.597 | 0.612 | 0.558 | 0.597 | 0.575 | 0.620 | 0.709
$0.3$ | $0$ | 0.214 | 0.226 | 0.230 | 0.237 | 0.220 | 0.225 | 0.234 | 0.299 | 0.371 | 0.496
| $0.1$ | 0.215 | 0.224 | 0.227 | 0.232 | 0.219 | 0.223 | 0.229 | 0.258 | 0.311 | 0.434
| $0.2$ | 0.224 | 0.232 | 0.235 | 0.240 | 0.228 | 0.231 | 0.236 | 0.242 | 0.267 | 0.350
| $0.3$ | 0.244 | 0.251 | 0.254 | 0.261 | 0.247 | 0.251 | 0.256 | 0.255 | 0.269 | 0.322
| $0.5$ | 0.322 | 0.335 | 0.338 | 0.351 | 0.330 | 0.338 | 0.342 | 0.337 | 0.387 | 0.387
| $0.7$ | 0.510 | 0.555 | 0.557 | 0.615 | 0.550 | 0.573 | 0.550 | 0.556 | 0.627 | 0.641
$0.6$ | $0$ | 0.182 | 0.188 | 0.193 | 0.200 | 0.185 | 0.189 | 0.195 | 0.275 | 0.362 | 0.494
| $0.1$ | 0.184 | 0.187 | 0.190 | 0.195 | 0.185 | 0.187 | 0.191 | 0.223 | 0.285 | 0.415
| $0.2$ | 0.192 | 0.193 | 0.197 | 0.201 | 0.192 | 0.193 | 0.196 | 0.205 | 0.233 | 0.317
| $0.3$ | 0.209 | 0.210 | 0.212 | 0.217 | 0.208 | 0.208 | 0.213 | 0.215 | 0.231 | 0.281
| $0.5$ | 0.280 | 0.285 | 0.291 | 0.294 | 0.282 | 0.279 | 0.290 | 0.299 | 0.298 | 0.332
| $0.7$ | 0.468 | 0.460 | 0.485 | 0.496 | 0.460 | 0.468 | 0.455 | 0.468 | 0.512 | 0.513
$0.9$ | $0$ | 0.098 | 0.103 | 0.105 | 0.109 | 0.101 | 0.103 | 0.106 | 0.230 | 0.339 | 0.492
| $0.1$ | 0.098 | 0.102 | 0.104 | 0.107 | 0.101 | 0.102 | 0.104 | 0.131 | 0.188 | 0.336
| $0.2$ | 0.102 | 0.106 | 0.107 | 0.110 | 0.105 | 0.106 | 0.107 | 0.112 | 0.131 | 0.205
| $0.3$ | 0.112 | 0.115 | 0.116 | 0.119 | 0.114 | 0.114 | 0.115 | 0.118 | 0.128 | 0.166
| $0.5$ | 0.149 | 0.153 | 0.155 | 0.161 | 0.153 | 0.151 | 0.157 | 0.159 | 0.169 | 0.195
| $0.7$ | 0.247 | 0.263 | 0.248 | 0.247 | 0.251 | 0.249 | 0.299 | 0.271 | 0.280 | 0.311
Table 9: Simulated mean square error of the MRPDE for ratio of variances, $\widehat{\gamma}_{R,\alpha}$, when $n=15$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.064 | 0.071 | 0.085 | 0.091 | 0.071 | 0.074 | 0.089 | 0.270 | 0.426 | 0.687
| $0.1$ | 0.062 | 0.065 | 0.075 | 0.082 | 0.067 | 0.066 | 0.077 | 0.169 | 0.290 | 0.563
| $0.2$ | 0.067 | 0.068 | 0.075 | 0.085 | 0.069 | 0.067 | 0.077 | 0.103 | 0.167 | 0.354
| $0.3$ | 0.080 | 0.080 | 0.087 | 0.097 | 0.083 | 0.081 | 0.088 | 0.098 | 0.131 | 0.249
| $0.5$ | 0.141 | 0.147 | 0.151 | 0.162 | 0.145 | 0.145 | 0.149 | 0.158 | 0.172 | 0.235
| $0.7$ | 0.253 | 0.263 | 0.267 | 0.280 | 0.255 | 0.257 | 0.269 | 0.273 | 0.288 | 0.338
| MP | 0.049 | 0.057 | 0.066 | 0.073 | 0.058 | 0.058 | 0.071 | 0.250 | 0.405 | 0.666
$0.3$ | $0$ | 0.064 | 0.080 | 0.084 | 0.097 | 0.074 | 0.079 | 0.090 | 0.270 | 0.441 | 0.692
| $0.1$ | 0.062 | 0.074 | 0.076 | 0.086 | 0.068 | 0.071 | 0.080 | 0.167 | 0.299 | 0.567
| $0.2$ | 0.064 | 0.075 | 0.077 | 0.085 | 0.069 | 0.074 | 0.078 | 0.108 | 0.169 | 0.363
| $0.3$ | 0.078 | 0.084 | 0.090 | 0.099 | 0.082 | 0.086 | 0.091 | 0.100 | 0.136 | 0.255
| $0.5$ | 0.138 | 0.147 | 0.150 | 0.164 | 0.142 | 0.145 | 0.151 | 0.157 | 0.174 | 0.236
| $0.7$ | 0.250 | 0.255 | 0.268 | 0.277 | 0.257 | 0.260 | 0.267 | 0.265 | 0.290 | 0.326
| MP | 0.049 | 0.063 | 0.065 | 0.074 | 0.057 | 0.060 | 0.071 | 0.249 | 0.417 | 0.672
$0.6$ | $0$ | 0.070 | 0.081 | 0.083 | 0.100 | 0.075 | 0.080 | 0.092 | 0.286 | 0.474 | 0.735
| $0.1$ | 0.068 | 0.074 | 0.076 | 0.089 | 0.071 | 0.074 | 0.081 | 0.163 | 0.303 | 0.582
| $0.2$ | 0.073 | 0.077 | 0.081 | 0.088 | 0.073 | 0.076 | 0.079 | 0.106 | 0.166 | 0.353
| $0.3$ | 0.085 | 0.088 | 0.092 | 0.099 | 0.087 | 0.087 | 0.092 | 0.102 | 0.134 | 0.248
| $0.5$ | 0.146 | 0.150 | 0.158 | 0.163 | 0.148 | 0.149 | 0.154 | 0.162 | 0.179 | 0.237
| $0.7$ | 0.260 | 0.262 | 0.271 | 0.280 | 0.255 | 0.262 | 0.265 | 0.273 | 0.293 | 0.335
| MP | 0.050 | 0.061 | 0.065 | 0.077 | 0.056 | 0.061 | 0.071 | 0.262 | 0.449 | 0.711
$0.9$ | $0$ | 0.068 | 0.084 | 0.088 | 0.107 | 0.082 | 0.086 | 0.093 | 0.342 | 0.547 | 0.805
| $0.1$ | 0.067 | 0.080 | 0.080 | 0.097 | 0.078 | 0.080 | 0.081 | 0.136 | 0.248 | 0.528
| $0.2$ | 0.070 | 0.082 | 0.083 | 0.099 | 0.079 | 0.082 | 0.083 | 0.092 | 0.124 | 0.276
| $0.3$ | 0.083 | 0.095 | 0.093 | 0.111 | 0.092 | 0.093 | 0.094 | 0.101 | 0.113 | 0.193
| $0.5$ | 0.143 | 0.155 | 0.158 | 0.172 | 0.156 | 0.154 | 0.155 | 0.162 | 0.181 | 0.225
| $0.7$ | 0.257 | 0.268 | 0.273 | 0.285 | 0.270 | 0.273 | 0.268 | 0.279 | 0.300 | 0.345
| MP | 0.047 | 0.061 | 0.063 | 0.081 | 0.060 | 0.061 | 0.068 | 0.312 | 0.519 | 0.786
Table 10: Simulated significance level for testing equal variances through $W_{n,\alpha}(\widehat{\gamma}_{R,\alpha},\widehat{\rho}_{R,\alpha})$ given by (66) and the Morgan-Pitman test, when $n=15$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.217 | 0.224 | 0.230 | 0.239 | 0.221 | 0.224 | 0.233 | 0.338 | 0.431 | 0.596
| $0.1$ | 0.219 | 0.223 | 0.227 | 0.234 | 0.221 | 0.223 | 0.229 | 0.284 | 0.353 | 0.516
| $0.2$ | 0.227 | 0.230 | 0.233 | 0.240 | 0.229 | 0.230 | 0.234 | 0.253 | 0.290 | 0.403
| $0.3$ | 0.244 | 0.247 | 0.251 | 0.257 | 0.246 | 0.247 | 0.250 | 0.258 | 0.282 | 0.352
| $0.5$ | 0.306 | 0.313 | 0.314 | 0.325 | 0.309 | 0.312 | 0.314 | 0.320 | 0.332 | 0.374
| $0.7$ | 0.409 | 0.420 | 0.421 | 0.434 | 0.412 | 0.417 | 0.423 | 0.426 | 0.438 | 0.476
$0.3$ | $0$ | 0.217 | 0.226 | 0.231 | 0.238 | 0.221 | 0.226 | 0.234 | 0.338 | 0.439 | 0.601
| $0.1$ | 0.218 | 0.225 | 0.228 | 0.234 | 0.220 | 0.224 | 0.229 | 0.282 | 0.357 | 0.518
| $0.2$ | 0.226 | 0.232 | 0.235 | 0.240 | 0.228 | 0.231 | 0.235 | 0.253 | 0.292 | 0.404
| $0.3$ | 0.243 | 0.248 | 0.252 | 0.258 | 0.244 | 0.249 | 0.252 | 0.259 | 0.282 | 0.354
| $0.5$ | 0.305 | 0.311 | 0.316 | 0.325 | 0.307 | 0.309 | 0.315 | 0.319 | 0.334 | 0.373
| $0.7$ | 0.410 | 0.413 | 0.424 | 0.432 | 0.412 | 0.417 | 0.422 | 0.421 | 0.441 | 0.468
$0.6$ | $0$ | 0.218 | 0.226 | 0.231 | 0.240 | 0.222 | 0.226 | 0.232 | 0.347 | 0.462 | 0.628
| $0.1$ | 0.220 | 0.225 | 0.228 | 0.235 | 0.222 | 0.224 | 0.228 | 0.279 | 0.360 | 0.527
| $0.2$ | 0.228 | 0.232 | 0.235 | 0.240 | 0.230 | 0.231 | 0.234 | 0.252 | 0.289 | 0.400
| $0.3$ | 0.246 | 0.250 | 0.251 | 0.256 | 0.247 | 0.246 | 0.251 | 0.259 | 0.281 | 0.348
| $0.5$ | 0.311 | 0.314 | 0.319 | 0.324 | 0.311 | 0.311 | 0.317 | 0.322 | 0.335 | 0.376
| $0.7$ | 0.415 | 0.423 | 0.432 | 0.439 | 0.416 | 0.417 | 0.424 | 0.431 | 0.448 | 0.478
$0.9$ | $0$ | 0.216 | 0.226 | 0.230 | 0.239 | 0.223 | 0.228 | 0.232 | 0.377 | 0.507 | 0.688
| $0.1$ | 0.217 | 0.225 | 0.227 | 0.235 | 0.223 | 0.226 | 0.227 | 0.258 | 0.325 | 0.503
| $0.2$ | 0.226 | 0.233 | 0.234 | 0.243 | 0.231 | 0.233 | 0.233 | 0.239 | 0.261 | 0.354
| $0.3$ | 0.244 | 0.251 | 0.253 | 0.264 | 0.250 | 0.251 | 0.252 | 0.255 | 0.267 | 0.319
| $0.5$ | 0.323 | 0.333 | 0.338 | 0.353 | 0.333 | 0.333 | 0.333 | 0.340 | 0.355 | 0.397
| $0.7$ | 0.459 | 0.472 | 0.476 | 0.497 | 0.468 | 0.473 | 0.471 | 0.480 | 0.497 | 0.539
Table 11: Simulated mean square error of the MRPDE for correlation coefficient, $\widehat{\rho}_{R,\alpha}$, when $n=15$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.054 | 0.061 | 0.072 | 0.079 | 0.063 | 0.063 | 0.076 | 0.061 | 0.072 | 0.079
| $0.1$ | 0.052 | 0.055 | 0.063 | 0.070 | 0.057 | 0.056 | 0.066 | 0.055 | 0.063 | 0.070
| $0.2$ | 0.056 | 0.055 | 0.064 | 0.072 | 0.058 | 0.056 | 0.064 | 0.055 | 0.064 | 0.072
| $0.3$ | 0.068 | 0.065 | 0.075 | 0.083 | 0.070 | 0.067 | 0.074 | 0.065 | 0.075 | 0.083
| $0.5$ | 0.122 | 0.128 | 0.133 | 0.143 | 0.127 | 0.128 | 0.130 | 0.128 | 0.133 | 0.143
| $0.7$ | 0.229 | 0.238 | 0.240 | 0.254 | 0.228 | 0.231 | 0.241 | 0.238 | 0.240 | 0.254
| MP | 0.049 | 0.057 | 0.066 | 0.073 | 0.058 | 0.058 | 0.071 | 0.250 | 0.405 | 0.666
$0.3$ | $0$ | 0.053 | 0.067 | 0.070 | 0.081 | 0.061 | 0.065 | 0.077 | 0.067 | 0.070 | 0.081
| $0.1$ | 0.051 | 0.062 | 0.063 | 0.070 | 0.057 | 0.060 | 0.066 | 0.062 | 0.063 | 0.070
| $0.2$ | 0.052 | 0.062 | 0.064 | 0.071 | 0.058 | 0.060 | 0.065 | 0.062 | 0.064 | 0.071
| $0.3$ | 0.065 | 0.071 | 0.074 | 0.085 | 0.067 | 0.073 | 0.076 | 0.071 | 0.074 | 0.085
| $0.5$ | 0.119 | 0.127 | 0.131 | 0.144 | 0.124 | 0.126 | 0.132 | 0.127 | 0.131 | 0.144
| $0.7$ | 0.224 | 0.230 | 0.243 | 0.253 | 0.231 | 0.233 | 0.241 | 0.230 | 0.243 | 0.253
| MP | 0.049 | 0.063 | 0.065 | 0.074 | 0.057 | 0.060 | 0.071 | 0.249 | 0.417 | 0.672
$0.6$ | $0$ | 0.056 | 0.065 | 0.069 | 0.083 | 0.062 | 0.066 | 0.076 | 0.065 | 0.069 | 0.083
| $0.1$ | 0.054 | 0.060 | 0.062 | 0.072 | 0.058 | 0.059 | 0.066 | 0.060 | 0.062 | 0.072
| $0.2$ | 0.058 | 0.060 | 0.065 | 0.072 | 0.060 | 0.061 | 0.064 | 0.060 | 0.065 | 0.072
| $0.3$ | 0.069 | 0.071 | 0.075 | 0.082 | 0.072 | 0.072 | 0.076 | 0.071 | 0.075 | 0.082
| $0.5$ | 0.127 | 0.129 | 0.138 | 0.142 | 0.129 | 0.130 | 0.133 | 0.129 | 0.138 | 0.142
| $0.7$ | 0.233 | 0.241 | 0.249 | 0.258 | 0.231 | 0.236 | 0.242 | 0.241 | 0.249 | 0.258
| MP | 0.050 | 0.061 | 0.065 | 0.077 | 0.056 | 0.061 | 0.071 | 0.262 | 0.449 | 0.711
$0.9$ | $0$ | 0.051 | 0.067 | 0.069 | 0.087 | 0.065 | 0.067 | 0.072 | 0.067 | 0.069 | 0.087
| $0.1$ | 0.051 | 0.062 | 0.063 | 0.076 | 0.061 | 0.062 | 0.063 | 0.062 | 0.063 | 0.076
| $0.2$ | 0.053 | 0.063 | 0.064 | 0.078 | 0.063 | 0.062 | 0.063 | 0.063 | 0.064 | 0.078
| $0.3$ | 0.067 | 0.077 | 0.076 | 0.094 | 0.076 | 0.075 | 0.076 | 0.077 | 0.076 | 0.094
| $0.5$ | 0.147 | 0.160 | 0.164 | 0.185 | 0.159 | 0.155 | 0.158 | 0.160 | 0.164 | 0.185
| $0.7$ | 0.296 | 0.313 | 0.317 | 0.347 | 0.304 | 0.312 | 0.309 | 0.313 | 0.317 | 0.347
| MP | 0.047 | 0.061 | 0.063 | 0.081 | 0.060 | 0.061 | 0.068 | 0.312 | 0.519 | 0.786
Table 12: Simulated significance level for testing null correlation coefficient through $W_{n,\alpha}^{\prime}(\widehat{\rho}_{UV,R,\alpha})$ given by (68) and the Morgan-Pitman test, when $n=15$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.115 | 0.123 | 0.128 | 0.131 | 0.121 | 0.123 | 0.131 | 0.278 | 0.403 | 0.552
| $0.1$ | 0.117 | 0.120 | 0.123 | 0.126 | 0.119 | 0.118 | 0.121 | 0.160 | 0.251 | 0.440
| $0.2$ | 0.121 | 0.123 | 0.125 | 0.128 | 0.122 | 0.122 | 0.124 | 0.134 | 0.168 | 0.296
| $0.3$ | 0.127 | 0.129 | 0.130 | 0.133 | 0.127 | 0.127 | 0.130 | 0.133 | 0.148 | 0.217
| $0.5$ | 0.144 | 0.145 | 0.147 | 0.151 | 0.144 | 0.144 | 0.148 | 0.147 | 0.153 | 0.179
| $0.7$ | 0.168 | 0.171 | 0.173 | 0.178 | 0.171 | 0.168 | 0.175 | 0.172 | 0.176 | 0.193
$0.3$ | $0$ | 0.110 | 0.117 | 0.121 | 0.124 | 0.114 | 0.119 | 0.124 | 0.274 | 0.402 | 0.552
| $0.1$ | 0.111 | 0.114 | 0.116 | 0.119 | 0.112 | 0.114 | 0.116 | 0.155 | 0.242 | 0.438
| $0.2$ | 0.115 | 0.116 | 0.118 | 0.121 | 0.116 | 0.117 | 0.118 | 0.127 | 0.158 | 0.290
| $0.3$ | 0.121 | 0.122 | 0.123 | 0.127 | 0.121 | 0.122 | 0.124 | 0.126 | 0.139 | 0.210
| $0.5$ | 0.137 | 0.137 | 0.139 | 0.143 | 0.138 | 0.137 | 0.140 | 0.139 | 0.143 | 0.171
| $0.7$ | 0.159 | 0.161 | 0.163 | 0.168 | 0.161 | 0.161 | 0.164 | 0.162 | 0.166 | 0.184
$0.6$ | $0$ | 0.093 | 0.097 | 0.102 | 0.106 | 0.097 | 0.100 | 0.105 | 0.270 | 0.402 | 0.553
| $0.1$ | 0.094 | 0.095 | 0.098 | 0.101 | 0.095 | 0.096 | 0.097 | 0.134 | 0.224 | 0.430
| $0.2$ | 0.097 | 0.098 | 0.100 | 0.102 | 0.098 | 0.098 | 0.099 | 0.108 | 0.138 | 0.262
| $0.3$ | 0.101 | 0.102 | 0.104 | 0.106 | 0.102 | 0.103 | 0.103 | 0.107 | 0.121 | 0.180
| $0.5$ | 0.114 | 0.115 | 0.118 | 0.120 | 0.116 | 0.116 | 0.116 | 0.118 | 0.125 | 0.146
| $0.7$ | 0.134 | 0.135 | 0.139 | 0.141 | 0.136 | 0.136 | 0.137 | 0.137 | 0.144 | 0.157
$0.9$ | $0$ | 0.050 | 0.053 | 0.056 | 0.057 | 0.052 | 0.054 | 0.057 | 0.263 | 0.401 | 0.555
| $0.1$ | 0.051 | 0.052 | 0.054 | 0.054 | 0.051 | 0.052 | 0.053 | 0.068 | 0.128 | 0.365
| $0.2$ | 0.053 | 0.053 | 0.054 | 0.055 | 0.053 | 0.053 | 0.054 | 0.056 | 0.066 | 0.138
| $0.3$ | 0.055 | 0.056 | 0.057 | 0.057 | 0.055 | 0.055 | 0.056 | 0.058 | 0.062 | 0.086
| $0.5$ | 0.063 | 0.063 | 0.064 | 0.065 | 0.063 | 0.062 | 0.063 | 0.064 | 0.067 | 0.077
| $0.7$ | 0.073 | 0.074 | 0.075 | 0.076 | 0.074 | 0.074 | 0.074 | 0.075 | 0.079 | 0.088
Table 13: Simulated mean square error of the MRPDE for ratio of variances, $\widehat{\gamma}_{R,\alpha}$, when $n=50$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.054 | 0.072 | 0.081 | 0.089 | 0.063 | 0.071 | 0.091 | 0.535 | 0.809 | 0.980
| $0.1$ | 0.054 | 0.061 | 0.066 | 0.068 | 0.057 | 0.056 | 0.063 | 0.206 | 0.485 | 0.885
| $0.2$ | 0.055 | 0.058 | 0.061 | 0.064 | 0.057 | 0.053 | 0.061 | 0.100 | 0.215 | 0.593
| $0.3$ | 0.055 | 0.059 | 0.061 | 0.065 | 0.058 | 0.055 | 0.062 | 0.079 | 0.131 | 0.348
| $0.5$ | 0.062 | 0.064 | 0.071 | 0.076 | 0.065 | 0.063 | 0.071 | 0.072 | 0.097 | 0.182
| $0.7$ | 0.079 | 0.082 | 0.090 | 0.096 | 0.083 | 0.081 | 0.091 | 0.088 | 0.107 | 0.151
| MP | 0.050 | 0.067 | 0.077 | 0.084 | 0.061 | 0.068 | 0.086 | 0.529 | 0.805 | 0.979
$0.3$ | $0$ | 0.057 | 0.067 | 0.077 | 0.087 | 0.063 | 0.074 | 0.089 | 0.542 | 0.817 | 0.981
| $0.1$ | 0.057 | 0.058 | 0.065 | 0.071 | 0.055 | 0.059 | 0.063 | 0.206 | 0.485 | 0.890
| $0.2$ | 0.056 | 0.056 | 0.061 | 0.066 | 0.054 | 0.056 | 0.061 | 0.099 | 0.205 | 0.592
| $0.3$ | 0.058 | 0.060 | 0.060 | 0.066 | 0.057 | 0.057 | 0.063 | 0.080 | 0.119 | 0.345
| $0.5$ | 0.065 | 0.065 | 0.067 | 0.073 | 0.067 | 0.066 | 0.070 | 0.076 | 0.088 | 0.175
| $0.7$ | 0.082 | 0.081 | 0.086 | 0.092 | 0.082 | 0.082 | 0.090 | 0.090 | 0.099 | 0.146
| MP | 0.052 | 0.063 | 0.072 | 0.081 | 0.059 | 0.070 | 0.084 | 0.536 | 0.814 | 0.980
$0.6$ | $0$ | 0.058 | 0.067 | 0.079 | 0.092 | 0.066 | 0.079 | 0.090 | 0.580 | 0.847 | 0.986
| $0.1$ | 0.056 | 0.057 | 0.068 | 0.072 | 0.058 | 0.059 | 0.061 | 0.197 | 0.487 | 0.891
| $0.2$ | 0.056 | 0.056 | 0.065 | 0.066 | 0.058 | 0.057 | 0.060 | 0.093 | 0.199 | 0.565
| $0.3$ | 0.056 | 0.059 | 0.064 | 0.069 | 0.058 | 0.058 | 0.060 | 0.074 | 0.119 | 0.315
| $0.5$ | 0.065 | 0.066 | 0.072 | 0.073 | 0.066 | 0.064 | 0.067 | 0.075 | 0.090 | 0.161
| $0.7$ | 0.080 | 0.083 | 0.090 | 0.094 | 0.084 | 0.080 | 0.084 | 0.090 | 0.104 | 0.143
| MP | 0.053 | 0.062 | 0.074 | 0.085 | 0.060 | 0.074 | 0.084 | 0.574 | 0.843 | 0.985
$0.9$ | $0$ | 0.054 | 0.068 | 0.085 | 0.088 | 0.064 | 0.075 | 0.090 | 0.679 | 0.907 | 0.996
| $0.1$ | 0.054 | 0.058 | 0.069 | 0.071 | 0.056 | 0.060 | 0.064 | 0.113 | 0.322 | 0.806
| $0.2$ | 0.054 | 0.055 | 0.066 | 0.066 | 0.057 | 0.056 | 0.061 | 0.066 | 0.103 | 0.339
| $0.3$ | 0.056 | 0.055 | 0.067 | 0.066 | 0.060 | 0.058 | 0.062 | 0.061 | 0.079 | 0.164
| $0.5$ | 0.067 | 0.067 | 0.074 | 0.074 | 0.068 | 0.064 | 0.070 | 0.070 | 0.083 | 0.112
| $0.7$ | 0.084 | 0.082 | 0.090 | 0.095 | 0.084 | 0.083 | 0.085 | 0.089 | 0.100 | 0.125
| MP | 0.048 | 0.062 | 0.079 | 0.081 | 0.057 | 0.070 | 0.084 | 0.673 | 0.904 | 0.996
Table 14: Simulated significance level for testing equal variances through $W_{n,\alpha}(\widehat{\gamma}_{R,\alpha},\widehat{\rho}_{R,\alpha})$ given by (66) and the Morgan-Pitman test, when $n=50$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.114 | 0.121 | 0.126 | 0.130 | 0.119 | 0.122 | 0.129 | 0.322 | 0.479 | 0.665
| $0.1$ | 0.115 | 0.119 | 0.121 | 0.124 | 0.117 | 0.117 | 0.120 | 0.174 | 0.288 | 0.525
| $0.2$ | 0.119 | 0.121 | 0.123 | 0.126 | 0.120 | 0.121 | 0.123 | 0.139 | 0.184 | 0.345
| $0.3$ | 0.125 | 0.127 | 0.129 | 0.132 | 0.126 | 0.126 | 0.129 | 0.136 | 0.157 | 0.245
| $0.5$ | 0.141 | 0.142 | 0.145 | 0.148 | 0.141 | 0.142 | 0.146 | 0.147 | 0.157 | 0.193
| $0.7$ | 0.164 | 0.166 | 0.168 | 0.173 | 0.166 | 0.164 | 0.170 | 0.169 | 0.177 | 0.201
$0.3$ | $0$ | 0.115 | 0.121 | 0.125 | 0.128 | 0.118 | 0.123 | 0.129 | 0.326 | 0.487 | 0.673
| $0.1$ | 0.116 | 0.118 | 0.120 | 0.123 | 0.117 | 0.118 | 0.120 | 0.175 | 0.285 | 0.532
| $0.2$ | 0.120 | 0.121 | 0.122 | 0.125 | 0.120 | 0.121 | 0.122 | 0.138 | 0.179 | 0.346
| $0.3$ | 0.125 | 0.126 | 0.127 | 0.131 | 0.126 | 0.126 | 0.128 | 0.135 | 0.153 | 0.244
| $0.5$ | 0.141 | 0.142 | 0.143 | 0.147 | 0.142 | 0.142 | 0.145 | 0.146 | 0.154 | 0.192
| $0.7$ | 0.164 | 0.164 | 0.166 | 0.171 | 0.165 | 0.165 | 0.168 | 0.167 | 0.174 | 0.200
$0.6$ | $0$ | 0.114 | 0.120 | 0.126 | 0.130 | 0.119 | 0.124 | 0.129 | 0.348 | 0.518 | 0.704
| $0.1$ | 0.116 | 0.118 | 0.121 | 0.124 | 0.117 | 0.119 | 0.120 | 0.171 | 0.289 | 0.552
| $0.2$ | 0.119 | 0.121 | 0.123 | 0.125 | 0.121 | 0.122 | 0.122 | 0.136 | 0.177 | 0.338
| $0.3$ | 0.125 | 0.126 | 0.128 | 0.130 | 0.126 | 0.127 | 0.127 | 0.134 | 0.154 | 0.232
| $0.5$ | 0.141 | 0.142 | 0.145 | 0.146 | 0.142 | 0.142 | 0.143 | 0.146 | 0.156 | 0.186
| $0.7$ | 0.163 | 0.165 | 0.169 | 0.171 | 0.165 | 0.165 | 0.166 | 0.169 | 0.178 | 0.196
$0.9$ | $0$ | 0.114 | 0.121 | 0.127 | 0.129 | 0.118 | 0.123 | 0.129 | 0.411 | 0.588 | 0.767
| $0.1$ | 0.115 | 0.118 | 0.122 | 0.123 | 0.116 | 0.118 | 0.120 | 0.141 | 0.228 | 0.539
| $0.2$ | 0.119 | 0.121 | 0.123 | 0.125 | 0.120 | 0.121 | 0.122 | 0.124 | 0.139 | 0.242
| $0.3$ | 0.125 | 0.126 | 0.128 | 0.130 | 0.125 | 0.126 | 0.127 | 0.128 | 0.135 | 0.171
| $0.5$ | 0.141 | 0.141 | 0.144 | 0.146 | 0.141 | 0.141 | 0.142 | 0.143 | 0.149 | 0.164
| $0.7$ | 0.164 | 0.164 | 0.169 | 0.171 | 0.165 | 0.165 | 0.166 | 0.167 | 0.174 | 0.188
Table 15: Simulated mean square error of the MRPDE for correlation coefficient, $\widehat{\rho}_{R,\alpha}$, when $n=50$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.051 | 0.068 | 0.078 | 0.086 | 0.062 | 0.069 | 0.088 | 0.532 | 0.807 | 0.979
| $0.1$ | 0.052 | 0.058 | 0.063 | 0.065 | 0.055 | 0.053 | 0.061 | 0.199 | 0.480 | 0.883
| $0.2$ | 0.052 | 0.055 | 0.058 | 0.061 | 0.054 | 0.051 | 0.058 | 0.096 | 0.210 | 0.588
| $0.3$ | 0.052 | 0.057 | 0.057 | 0.062 | 0.056 | 0.052 | 0.060 | 0.075 | 0.126 | 0.343
| $0.5$ | 0.059 | 0.061 | 0.067 | 0.072 | 0.062 | 0.059 | 0.067 | 0.069 | 0.092 | 0.176
| $0.7$ | 0.074 | 0.078 | 0.085 | 0.092 | 0.079 | 0.075 | 0.085 | 0.084 | 0.101 | 0.145
| MP | 0.050 | 0.067 | 0.077 | 0.084 | 0.061 | 0.068 | 0.086 | 0.529 | 0.805 | 0.979
$0.3$ | $0$ | 0.053 | 0.064 | 0.074 | 0.083 | 0.060 | 0.071 | 0.086 | 0.538 | 0.815 | 0.980
| $0.1$ | 0.054 | 0.055 | 0.062 | 0.069 | 0.051 | 0.056 | 0.060 | 0.201 | 0.480 | 0.888
| $0.2$ | 0.054 | 0.053 | 0.058 | 0.063 | 0.051 | 0.054 | 0.058 | 0.095 | 0.200 | 0.586
| $0.3$ | 0.055 | 0.056 | 0.057 | 0.064 | 0.054 | 0.054 | 0.058 | 0.076 | 0.114 | 0.339
| $0.5$ | 0.062 | 0.061 | 0.063 | 0.070 | 0.063 | 0.061 | 0.067 | 0.072 | 0.084 | 0.168
| $0.7$ | 0.077 | 0.077 | 0.080 | 0.088 | 0.078 | 0.077 | 0.084 | 0.085 | 0.094 | 0.139
| MP | 0.052 | 0.063 | 0.072 | 0.081 | 0.059 | 0.070 | 0.084 | 0.536 | 0.814 | 0.980
$0.6$ | $0$ | 0.054 | 0.063 | 0.076 | 0.086 | 0.061 | 0.076 | 0.086 | 0.575 | 0.845 | 0.985
| $0.1$ | 0.052 | 0.054 | 0.064 | 0.067 | 0.055 | 0.056 | 0.058 | 0.191 | 0.481 | 0.889
| $0.2$ | 0.052 | 0.053 | 0.060 | 0.062 | 0.055 | 0.053 | 0.056 | 0.088 | 0.192 | 0.558
| $0.3$ | 0.053 | 0.055 | 0.061 | 0.064 | 0.055 | 0.054 | 0.055 | 0.070 | 0.114 | 0.306
| $0.5$ | 0.060 | 0.061 | 0.067 | 0.069 | 0.060 | 0.060 | 0.062 | 0.069 | 0.085 | 0.155
| $0.7$ | 0.074 | 0.077 | 0.083 | 0.087 | 0.078 | 0.074 | 0.078 | 0.084 | 0.098 | 0.134
| MP | 0.053 | 0.062 | 0.074 | 0.085 | 0.060 | 0.074 | 0.084 | 0.574 | 0.843 | 0.985
$0.9$ | $0$ | 0.050 | 0.064 | 0.080 | 0.082 | 0.059 | 0.071 | 0.085 | 0.674 | 0.905 | 0.996
| $0.1$ | 0.050 | 0.053 | 0.064 | 0.065 | 0.051 | 0.055 | 0.059 | 0.108 | 0.315 | 0.804
| $0.2$ | 0.051 | 0.051 | 0.061 | 0.061 | 0.053 | 0.052 | 0.056 | 0.060 | 0.096 | 0.331
| $0.3$ | 0.051 | 0.051 | 0.062 | 0.061 | 0.056 | 0.052 | 0.058 | 0.056 | 0.073 | 0.155
| $0.5$ | 0.061 | 0.060 | 0.067 | 0.067 | 0.062 | 0.059 | 0.064 | 0.063 | 0.076 | 0.104
| $0.7$ | 0.077 | 0.075 | 0.083 | 0.088 | 0.079 | 0.076 | 0.079 | 0.082 | 0.093 | 0.118
| MP | 0.048 | 0.062 | 0.079 | 0.081 | 0.057 | 0.070 | 0.084 | 0.673 | 0.904 | 0.996
Table 16: Simulated significance level for testing null correlation coefficient through $W_{n,\alpha}^{\prime}(\widehat{\rho}_{UV,R,\alpha})$ given by (68) and the Morgan-Pitman test, when $n=50$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.081 | 0.086 | 0.089 | 0.093 | 0.085 | 0.089 | 0.094 | 0.288 | 0.429 | 0.569
| $0.1$ | 0.082 | 0.084 | 0.085 | 0.088 | 0.083 | 0.084 | 0.085 | 0.133 | 0.241 | 0.451
| $0.2$ | 0.085 | 0.086 | 0.087 | 0.089 | 0.086 | 0.086 | 0.086 | 0.100 | 0.141 | 0.292
| $0.3$ | 0.089 | 0.090 | 0.090 | 0.092 | 0.090 | 0.089 | 0.089 | 0.095 | 0.114 | 0.197
| $0.5$ | 0.099 | 0.100 | 0.100 | 0.103 | 0.100 | 0.099 | 0.099 | 0.102 | 0.108 | 0.140
| $0.7$ | 0.111 | 0.113 | 0.113 | 0.116 | 0.112 | 0.111 | 0.112 | 0.114 | 0.117 | 0.135
$0.3$ | $0$ | 0.077 | 0.082 | 0.086 | 0.088 | 0.080 | 0.084 | 0.090 | 0.288 | 0.427 | 0.569
| $0.1$ | 0.078 | 0.079 | 0.082 | 0.084 | 0.078 | 0.080 | 0.081 | 0.128 | 0.234 | 0.449
| $0.2$ | 0.081 | 0.081 | 0.083 | 0.084 | 0.080 | 0.082 | 0.082 | 0.094 | 0.134 | 0.284
| $0.3$ | 0.084 | 0.085 | 0.086 | 0.087 | 0.084 | 0.085 | 0.085 | 0.090 | 0.109 | 0.190
| $0.5$ | 0.093 | 0.094 | 0.096 | 0.097 | 0.093 | 0.094 | 0.095 | 0.096 | 0.104 | 0.135
| $0.7$ | 0.105 | 0.107 | 0.108 | 0.110 | 0.105 | 0.107 | 0.106 | 0.108 | 0.113 | 0.130
$0.6$ | $0$ | 0.064 | 0.070 | 0.072 | 0.074 | 0.068 | 0.071 | 0.076 | 0.289 | 0.427 | 0.571
| $0.1$ | 0.065 | 0.068 | 0.069 | 0.070 | 0.066 | 0.067 | 0.068 | 0.109 | 0.210 | 0.443
| $0.2$ | 0.067 | 0.069 | 0.070 | 0.070 | 0.068 | 0.068 | 0.069 | 0.079 | 0.112 | 0.255
| $0.3$ | 0.070 | 0.072 | 0.073 | 0.073 | 0.071 | 0.071 | 0.072 | 0.076 | 0.091 | 0.161
| $0.5$ | 0.077 | 0.080 | 0.081 | 0.081 | 0.079 | 0.079 | 0.080 | 0.081 | 0.088 | 0.114
| $0.7$ | 0.087 | 0.090 | 0.091 | 0.092 | 0.089 | 0.089 | 0.090 | 0.091 | 0.095 | 0.111
$0.9$ | $0$ | 0.035 | 0.038 | 0.039 | 0.040 | 0.037 | 0.039 | 0.041 | 0.286 | 0.428 | 0.570
| $0.1$ | 0.035 | 0.037 | 0.037 | 0.038 | 0.036 | 0.037 | 0.037 | 0.049 | 0.106 | 0.387
| $0.2$ | 0.036 | 0.037 | 0.038 | 0.039 | 0.037 | 0.037 | 0.038 | 0.040 | 0.049 | 0.112
| $0.3$ | 0.038 | 0.039 | 0.039 | 0.040 | 0.039 | 0.039 | 0.039 | 0.040 | 0.044 | 0.064
| $0.5$ | 0.042 | 0.043 | 0.044 | 0.045 | 0.043 | 0.043 | 0.044 | 0.044 | 0.046 | 0.055
| $0.7$ | 0.048 | 0.049 | 0.049 | 0.050 | 0.049 | 0.049 | 0.050 | 0.050 | 0.052 | 0.058
Table 17: Simulated mean square error of the MRPDE for ratio of variances, $\widehat{\gamma}_{R,\alpha}$, when $n=100$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.053 | 0.068 | 0.080 | 0.087 | 0.067 | 0.080 | 0.096 | 0.748 | 0.963 | 1.000
| $0.1$ | 0.054 | 0.059 | 0.063 | 0.067 | 0.057 | 0.056 | 0.061 | 0.280 | 0.684 | 0.983
| $0.2$ | 0.053 | 0.058 | 0.059 | 0.062 | 0.057 | 0.055 | 0.057 | 0.120 | 0.303 | 0.795
| $0.3$ | 0.054 | 0.058 | 0.060 | 0.063 | 0.056 | 0.055 | 0.058 | 0.086 | 0.166 | 0.500
| $0.5$ | 0.057 | 0.059 | 0.063 | 0.066 | 0.057 | 0.058 | 0.059 | 0.069 | 0.097 | 0.227
| $0.7$ | 0.062 | 0.064 | 0.069 | 0.072 | 0.065 | 0.062 | 0.063 | 0.072 | 0.089 | 0.153
| MP | 0.052 | 0.066 | 0.077 | 0.085 | 0.065 | 0.078 | 0.094 | 0.746 | 0.962 | 1.000
$0.3$ | $0$ | 0.051 | 0.069 | 0.079 | 0.086 | 0.063 | 0.077 | 0.095 | 0.760 | 0.964 | 1.000
| $0.1$ | 0.051 | 0.057 | 0.063 | 0.069 | 0.050 | 0.057 | 0.057 | 0.280 | 0.678 | 0.988
| $0.2$ | 0.052 | 0.055 | 0.058 | 0.066 | 0.052 | 0.054 | 0.054 | 0.117 | 0.291 | 0.793
| $0.3$ | 0.052 | 0.055 | 0.058 | 0.065 | 0.051 | 0.055 | 0.056 | 0.083 | 0.161 | 0.494
| $0.5$ | 0.053 | 0.057 | 0.060 | 0.068 | 0.055 | 0.059 | 0.058 | 0.068 | 0.096 | 0.223
| $0.7$ | 0.059 | 0.065 | 0.065 | 0.073 | 0.061 | 0.063 | 0.065 | 0.070 | 0.087 | 0.148
| MP | 0.049 | 0.067 | 0.077 | 0.084 | 0.060 | 0.074 | 0.093 | 0.759 | 0.964 | 1.000
$0.6$ | $0$ | 0.049 | 0.071 | 0.084 | 0.088 | 0.066 | 0.078 | 0.098 | 0.803 | 0.975 | 1.000
| $0.1$ | 0.047 | 0.059 | 0.068 | 0.070 | 0.055 | 0.059 | 0.062 | 0.267 | 0.661 | 0.987
| $0.2$ | 0.049 | 0.058 | 0.065 | 0.063 | 0.054 | 0.056 | 0.056 | 0.102 | 0.262 | 0.773
| $0.3$ | 0.049 | 0.059 | 0.063 | 0.062 | 0.056 | 0.057 | 0.056 | 0.076 | 0.146 | 0.464
| $0.5$ | 0.053 | 0.059 | 0.066 | 0.068 | 0.059 | 0.061 | 0.058 | 0.066 | 0.095 | 0.207
| $0.7$ | 0.058 | 0.065 | 0.070 | 0.074 | 0.063 | 0.066 | 0.065 | 0.070 | 0.089 | 0.144
| MP | 0.047 | 0.069 | 0.081 | 0.085 | 0.063 | 0.076 | 0.096 | 0.801 | 0.974 | 1.000
$0.9$ | $0$ | 0.051 | 0.070 | 0.079 | 0.091 | 0.068 | 0.078 | 0.096 | 0.881 | 0.994 | 1.000
| $0.1$ | 0.050 | 0.058 | 0.062 | 0.072 | 0.055 | 0.057 | 0.060 | 0.131 | 0.423 | 0.948
| $0.2$ | 0.050 | 0.058 | 0.059 | 0.065 | 0.054 | 0.056 | 0.059 | 0.068 | 0.118 | 0.443
| $0.3$ | 0.050 | 0.059 | 0.058 | 0.065 | 0.054 | 0.056 | 0.059 | 0.063 | 0.084 | 0.201
| $0.5$ | 0.056 | 0.062 | 0.058 | 0.067 | 0.057 | 0.057 | 0.063 | 0.064 | 0.071 | 0.116
| $0.7$ | 0.062 | 0.068 | 0.066 | 0.075 | 0.062 | 0.064 | 0.070 | 0.069 | 0.073 | 0.106
| MP | 0.047 | 0.067 | 0.076 | 0.088 | 0.065 | 0.075 | 0.093 | 0.880 | 0.993 | 1.000
Table 18: Simulated significance level for testing equal variances through $W_{n,\alpha}(\widehat{\gamma}_{R,\alpha},\widehat{\rho}_{R,\alpha})$ given by (66) and the Morgan-Pitman test, when $n=100$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.081 | 0.086 | 0.089 | 0.092 | 0.085 | 0.089 | 0.093 | 0.334 | 0.510 | 0.685
| $0.1$ | 0.082 | 0.084 | 0.085 | 0.087 | 0.083 | 0.083 | 0.084 | 0.144 | 0.274 | 0.538
| $0.2$ | 0.085 | 0.086 | 0.086 | 0.088 | 0.085 | 0.085 | 0.085 | 0.104 | 0.154 | 0.337
| $0.3$ | 0.088 | 0.089 | 0.090 | 0.092 | 0.089 | 0.089 | 0.089 | 0.098 | 0.121 | 0.221
| $0.5$ | 0.098 | 0.099 | 0.099 | 0.102 | 0.099 | 0.098 | 0.098 | 0.103 | 0.112 | 0.152
| $0.7$ | 0.110 | 0.111 | 0.112 | 0.115 | 0.111 | 0.110 | 0.111 | 0.114 | 0.120 | 0.144
$0.3$ | $0$ | 0.081 | 0.085 | 0.089 | 0.092 | 0.084 | 0.088 | 0.093 | 0.342 | 0.517 | 0.695
| $0.1$ | 0.082 | 0.083 | 0.085 | 0.087 | 0.081 | 0.083 | 0.084 | 0.143 | 0.274 | 0.545
| $0.2$ | 0.084 | 0.084 | 0.086 | 0.088 | 0.084 | 0.085 | 0.085 | 0.103 | 0.152 | 0.337
| $0.3$ | 0.088 | 0.088 | 0.090 | 0.091 | 0.087 | 0.089 | 0.089 | 0.097 | 0.120 | 0.220
| $0.5$ | 0.097 | 0.098 | 0.099 | 0.101 | 0.097 | 0.098 | 0.098 | 0.102 | 0.112 | 0.152
| $0.7$ | 0.109 | 0.110 | 0.112 | 0.114 | 0.109 | 0.111 | 0.110 | 0.113 | 0.120 | 0.143
$0.6$ | $0$ | 0.080 | 0.086 | 0.089 | 0.092 | 0.085 | 0.088 | 0.094 | 0.373 | 0.550 | 0.726
| $0.1$ | 0.081 | 0.084 | 0.086 | 0.087 | 0.083 | 0.083 | 0.084 | 0.140 | 0.271 | 0.569
| $0.2$ | 0.083 | 0.085 | 0.087 | 0.088 | 0.085 | 0.085 | 0.086 | 0.101 | 0.144 | 0.330
| $0.3$ | 0.087 | 0.089 | 0.091 | 0.091 | 0.089 | 0.089 | 0.089 | 0.096 | 0.116 | 0.208
| $0.5$ | 0.096 | 0.099 | 0.100 | 0.100 | 0.098 | 0.098 | 0.099 | 0.102 | 0.111 | 0.147
| $0.7$ | 0.108 | 0.111 | 0.113 | 0.114 | 0.111 | 0.110 | 0.112 | 0.114 | 0.120 | 0.141
$0.9$ | $0$ | 0.079 | 0.086 | 0.088 | 0.092 | 0.085 | 0.089 | 0.094 | 0.446 | 0.626 | 0.786
| $0.1$ | 0.080 | 0.084 | 0.085 | 0.087 | 0.082 | 0.084 | 0.085 | 0.104 | 0.194 | 0.572
| $0.2$ | 0.083 | 0.085 | 0.086 | 0.088 | 0.084 | 0.085 | 0.086 | 0.088 | 0.104 | 0.205
| $0.3$ | 0.086 | 0.089 | 0.089 | 0.092 | 0.088 | 0.089 | 0.090 | 0.090 | 0.097 | 0.132
| $0.5$ | 0.096 | 0.099 | 0.099 | 0.102 | 0.098 | 0.098 | 0.100 | 0.099 | 0.103 | 0.118
| $0.7$ | 0.109 | 0.111 | 0.112 | 0.115 | 0.110 | 0.111 | 0.113 | 0.112 | 0.115 | 0.126
Table 19: Simulated mean square error of the MRPDE for correlation coefficient, $\widehat{\rho}_{R,\alpha}$, when $n=100$ |  |  | slighly | regular | heavily
---|---|---|---|---|---
$\rho$ | $\alpha$ | pure | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$ | $0.05$ | $0.10$ | $0.20$
$0$ | $0$ | 0.053 | 0.067 | 0.078 | 0.086 | 0.066 | 0.079 | 0.095 | 0.747 | 0.962 | 1.000
| $0.1$ | 0.052 | 0.058 | 0.062 | 0.066 | 0.056 | 0.055 | 0.060 | 0.278 | 0.682 | 0.983
| $0.2$ | 0.052 | 0.057 | 0.058 | 0.061 | 0.056 | 0.053 | 0.056 | 0.118 | 0.301 | 0.794
| $0.3$ | 0.053 | 0.056 | 0.058 | 0.061 | 0.055 | 0.053 | 0.057 | 0.085 | 0.164 | 0.497
| $0.5$ | 0.055 | 0.057 | 0.061 | 0.064 | 0.056 | 0.056 | 0.057 | 0.068 | 0.095 | 0.224
| $0.7$ | 0.061 | 0.063 | 0.067 | 0.070 | 0.063 | 0.060 | 0.062 | 0.070 | 0.087 | 0.150
| MP | 0.052 | 0.066 | 0.077 | 0.085 | 0.065 | 0.078 | 0.094 | 0.746 | 0.962 | 1.000
$0.3$ | $0$ | 0.049 | 0.068 | 0.077 | 0.085 | 0.061 | 0.076 | 0.093 | 0.759 | 0.964 | 1.000
| $0.1$ | 0.050 | 0.055 | 0.061 | 0.068 | 0.049 | 0.056 | 0.055 | 0.276 | 0.675 | 0.988
| $0.2$ | 0.051 | 0.053 | 0.057 | 0.065 | 0.050 | 0.053 | 0.052 | 0.116 | 0.289 | 0.791
| $0.3$ | 0.050 | 0.054 | 0.056 | 0.063 | 0.050 | 0.054 | 0.054 | 0.081 | 0.159 | 0.490
| $0.5$ | 0.052 | 0.056 | 0.059 | 0.066 | 0.053 | 0.057 | 0.056 | 0.066 | 0.094 | 0.220
| $0.7$ | 0.057 | 0.063 | 0.064 | 0.071 | 0.059 | 0.060 | 0.063 | 0.067 | 0.085 | 0.145
| MP | 0.049 | 0.067 | 0.077 | 0.084 | 0.060 | 0.074 | 0.093 | 0.759 | 0.964 | 1.000
$0.6$ | $0$ | 0.048 | 0.070 | 0.082 | 0.086 | 0.064 | 0.077 | 0.096 | 0.801 | 0.974 | 1.000
| $0.1$ | 0.046 | 0.058 | 0.066 | 0.068 | 0.053 | 0.058 | 0.061 | 0.264 | 0.659 | 0.987
| $0.2$ | 0.047 | 0.056 | 0.062 | 0.060 | 0.053 | 0.054 | 0.054 | 0.099 | 0.259 | 0.771
| $0.3$ | 0.047 | 0.058 | 0.060 | 0.060 | 0.054 | 0.056 | 0.054 | 0.074 | 0.143 | 0.460
| $0.5$ | 0.051 | 0.057 | 0.063 | 0.065 | 0.057 | 0.059 | 0.056 | 0.063 | 0.093 | 0.204
| $0.7$ | 0.055 | 0.061 | 0.068 | 0.071 | 0.061 | 0.063 | 0.062 | 0.068 | 0.086 | 0.140
| MP | 0.047 | 0.069 | 0.081 | 0.085 | 0.063 | 0.076 | 0.096 | 0.801 | 0.974 | 1.000
$0.9$ | $0$ | 0.048 | 0.068 | 0.076 | 0.089 | 0.065 | 0.076 | 0.094 | 0.880 | 0.994 | 1.000
| $0.1$ | 0.048 | 0.056 | 0.060 | 0.069 | 0.053 | 0.055 | 0.058 | 0.128 | 0.418 | 0.947
| $0.2$ | 0.047 | 0.056 | 0.056 | 0.063 | 0.052 | 0.054 | 0.057 | 0.066 | 0.115 | 0.438
| $0.3$ | 0.048 | 0.056 | 0.056 | 0.062 | 0.052 | 0.053 | 0.057 | 0.061 | 0.081 | 0.197
| $0.5$ | 0.053 | 0.060 | 0.056 | 0.064 | 0.054 | 0.054 | 0.059 | 0.061 | 0.068 | 0.112
| $0.7$ | 0.059 | 0.065 | 0.062 | 0.071 | 0.059 | 0.061 | 0.067 | 0.066 | 0.069 | 0.101
| MP | 0.047 | 0.067 | 0.076 | 0.088 | 0.065 | 0.075 | 0.093 | 0.880 | 0.993 | 1.000
Table 20: Simulated significance level for testing null correlation
coefficient through $W_{n,\alpha}^{\prime}(\widehat{\rho}_{UV,R,\alpha})$
given by (68) and the Morgan-Pitman test, when $n=100$