# A Review of Open Source Software Tools for Time Series Analysis

Yunus Parvej Faniband
Office of Industrial Collaboration
King Fahd University of Petroleum & Minerals
Dhahran, Saudi Arabia 312613
<EMAIL_ADDRESS>
& Iskandar Ishak
Faculty of Computer Science and Information Technology
Universiti Putra Malaysia
43400 UPM, Serdang, Selangor Darul Ehsan, Malaysia
<EMAIL_ADDRESS>
Sadiq Sait
Office of Industrial Collaboration
King Fahd University of Petroleum & Minerals
Dhahran, Saudi Arabia 312613
<EMAIL_ADDRESS>

###### Abstract

Time series data is used in a wide range of real-world applications. In a
variety of domains, detailed analysis of time series data (via Forecasting and
Anomaly Detection) leads to a better understanding of how events associated
with a specific time instance behave. Time Series Analysis (TSA) is commonly
performed with plots and traditional models. Machine Learning (ML) approaches,
on the other hand, have seen an increase in the state of the art for
Forecasting and Anomaly Detection because they provide comparable results when
time and data constraints are met. A number of time series toolboxes are
available that offer rich interfaces to specific model classes (ARIMA/filters,
neural networks) or framework interfaces to isolated time series modelling
tasks (forecasting, feature extraction, annotation, classification).
Nonetheless, open-source machine learning capabilities for time series remain
limited, and existing libraries are frequently incompatible with one another.
The goal of this paper is to provide a concise and user-friendly overview of
the most important open-source tools for time series analysis. This article
examines two related toolboxes: (1) forecasting and (2) anomaly detection.
This paper describes a typical Time Series Analysis (TSA) framework with an
architecture and lists the TSA framework’s main features. The tools are
categorized based on the criteria of analysis tasks completed, data
preparation methods employed, and evaluation methods for results generated.
This paper presents quantitative analysis and discusses the current state of
actively developed open-source Time Series Analysis frameworks. Overall, this
article considered 60 time series analysis tools, and 32 of which provided
forecasting modules, and 21 packages included anomaly detection.

_Keywords_ Time Series Analysis $\cdot$ Forecasting $\cdot$ Anomaly detection
$\cdot$ Open Source Software $\cdot$

##  1 Introduction

Time series are defined as “a collection of observations made sequentially
through time” Chatfield (2005). An important area of statistical analysis is
the methods of time series analysis. The study of events over time occurs in a
wide range of fields ranging from marketing to finance and engineering, but
also for IoT data, climate reading and renewable energy Faniband and Shaahid
(2021) Faniband and Shaahid (2020) . Identifying how specific aspects perform,
and being ready to foresee their expansion, aids to diagnose issues, foresee
actions, and conclusively strengthen greater outcomes. Although this leads to
several specifications, forecast of instances based on prior behavior
(forecasting) and the detection of sporadic observations, in relationship to
the generalization of the data items (anomaly detection) are two familiar
areas of time series analysis.

Categorization of time series becomes necessary because no approach has the
necessary generalization capabilities to address every type of time series.
Seasonality, Trend, and Outliers are the main features to consider when
classifying time series Chatfield (2003). The instances of time series
analysis scenarios constitutes an important problem in areas such as
sociology, scientific science, economy, economics, stock control, marketing,
production planning and engineering as argued by the authors in Brockwell et
al. (2016). The objective of the analysis of the time series is to draw
conclusions from these structures. Following the establishment of a convenient
model collection, parameters can be evaluated, data fitness can be
investigated, and fitted models may be used to gain a better understanding of
the series generating process. The models can be applied to characterize data,
extract noise from signals (via time plots) and test hypotheses, such as
global warming using climate data. These goals can be divided into four
categories: description, explanation, prediction (forecasting), and control.

Rapid Prototyping, Reproducibility and Transparency are the three main areas
where Toolboxes aid the researcher community. Toolboxes are essential for
Rapid Prototyping, as they provide a quick and efficient way to implement and
experiment with new models. Users and researchers can evaluate and compare
models quickly and systematically. The other critical feature is
Reproducibility, which allows researchers to reliably reproduce the results of
existing models and then compare them with new models. Machine learning and
forecasting research are very much dependent on reproducibility Makridakis et
al. (2018) Arnold et al. (2019). Toolboxes make algorithms and workflows more
readable and transparent by providing a consistent interface for algorithms
and composition functionality.

Forecasting is the prediction of unknown values inside an observable time
series. For example, given an observed series x1, x2,…, xn, the objective is
to estimate values, xn+h, where _h_ is the forecasting horizon. This is a
significant difficulty in various aspects of the economy, such as inventory
control, market strategy, and production planning. In real-world applications,
time series forecasting is omnipresent. One example is forecasting of demand
to supply inventory space, along with long-term economic growth forecasts to
inform government policies. Other examples include predicting stock prices to
aid financial decisions, and forecasting of demand to fill up inventory.
Forecasting is also trending area for machine learning research, with recent
advances in pure and hybrid machine learning approaches Smyl (2020).
Forecasting in practice entails a number of steps: we must first specify, fit,
and select an appropriate model, before evaluating and deploying it. We can
implement these steps using various open-source toolboxes. In most cases,
existing toolboxes tend to be severely limited in key areas. While some models
support specific model families (e.g. Autoregressive Integrated Moving Average
(ARIMA) or neural networks), others support more generic frameworks for
forecasting. Some other options offer functionality only for specific parts of
the process (e.g. feature extraction). Others provide more generic forecasting
frameworks but lack interfaces to well-known machine learning toolkits such as
scikit-learn Pedregosa et al. (2011). It appears, however, that despite the
successful implementation of machine learning for forecasting, there is a lack
of open-source toolbox that connects existing machine learning packages,
enabling model development, tuning, and evaluation.

Anomaly detection is a "important analysis task, which detects anomals or
abnormals in a certain set of data" Ahmed et al. (2016a). It is also known as
outlier or novelty detection and Outliers are essential because they
illustrate uncommon events in a number of fields, which commonly speed up
decisive actions to address. Recent trends indicate a sharp increase in the
number of computer systems and digital gadgets over the last decade, due to
the wide range of technologies present in the same time period. As the
networks grew, security concerns increased and It is expected that unusual
network traffic may be indicative of a compromised device. A common anomaly in
a credit card report could indicate fraud, for instance. If there are visual
anomalies in an MRI image, it could indicate the presence of a tumor.
Similarly anomalies in frequencies on an EEG scan indicate seizures and/or
their pre/post conditions. Anomalies from a wide variety of areas can be
extrapolated and are categorized into following three groups namely Point
Anomaly, Contextual Anomaly and Collective Anomaly Ahmed et al. (2016b).

Based on the literature, performing time series analysis can be a time-
consuming endeavor Petropoulos et al. (2022). In actuality, it’s not as simple
as writing an algorithm and handing it over to a machine. To begin, the data
must be prepared and cleaned. Manually conducting repetitive, time-consuming,
and hands-on trial and error experiments must follow data processing. TSA
frameworks, on the other hand, must be scalable and accommodate a variety of
time series approaches, as well as automate them and allow them to be used
iteratively to develop the model. The current ecosystem of Tools is fragmented
, with many specialized tool kits, but no overarching framework, and difficult
to understand, use and inter-operate.

Following are the contributions of this article:

  * •

Introduces the basic requirements for a typical Time Series Analysis (TSA)
framework ( Section 3) with architecture and describe the main features of TSA
framework in relation to the differences expressed between Univariate Time
Series (UTS) and Multivariate Time Series (MTS) approaches (Section 4) .

  * •

A list of Tools for TSA which provide automation in time series analysis, for
forecasting and anomaly detection ( See Section 5)

  * •

This paper presents quantitative analysis and discusses the state of the
current actively developed open-source frameworks for Time Series Analysis
(Section 7), including Forecasting, and Anomaly Detection

##  2 Methodology

This work conducted systematic literate review according to the guidelines
specified in Kitchenham and Brereton (2013). These guidelines, on the other
hand, are geared toward printed materials (academic research articles) rather
than software packages. As a result, some adjustments are made to these
procedures and explored the literature databases and open source code
repositories (e.g., Bitbucket, SourceForge, Github).

Literature Search _(SC:1, 3)_ Checking Open Source Code Repository (e.g.,
Bitbucket, SourceForge, Github) _(SC2.*_) Removing duplicates + Analysis
Generic vs Domain specific _(SC4)_ Snowballing _(SC1, 2.*, 3)_ Focuses on Time
Series Analysis? _(SC3)_ Removing duplicates Is Tool implented in Python,
Java, R or Julia ? _(SC2.3)_ Open Source Code Repository (e.g., Bitbucket,
SourceForge, Github) Search _(SC:1, 2.1, 2.2)_
$112$$22$$12$$60$$79$$127$$90$$71$$48$$60$ Figure 1: Overview of the search
and filtering procedure for Tools code repository. The number of repositories
left after each stage is indicated by edge labels.

###  2.1 Selection Criteria for Tools:

Based on the literature review the TSA tasks fall into following categories:
Forecasting (Prediction) (_TSK1_) ( Fakhrazari and Vakilzadian (2017)
Hendikawati et al. (2020) Mahalakshmi et al. (2016) Panigrahi and Behera
(2020) Tealab (2018) ), Classification (_TSK2_) (Abanda et al. (2019) Bagnall
et al. (2017)), Clustering (_TSK3_) (Aghabozorgi et al. (2015)), Anomaly
Detection (_TSK4_) (Wu (2016), Cook et al. (2019), Ayadi et al. (2017) ,
Segmentation (Summarization) (TSK5), Pattern Recognition (_TSK6_) ( Torkamani
and Lohweg (2017), Wang et al. (2019)) and Change Point Detection(_TSK7_) (
Sharma et al. (2016), Truong et al. (2020), Aminikhanghahi and Cook (2017) ).
This research work consider the open source tools implementing these tasks.

In terms of providing the data preparation capabilities, tools providing
following set of methods are considered : Dimensionality reduction methods
(_DP1_), Tools explicitly providing missing values imputation methods (_DP2_),
tools serving decomposition methods (_DP3_) (e.g., decomposing time series
into trends, seasonal components, or frequency components), tools integrated
with generic transformation and features generation methods (_DP4_) and tools
providing methods for computing similarity measures (_DP5_).

With respect to the Evaluation criteria followed by the TSA Tools, the
following categories are identified : Tools providing the methods for model
selection, hyperparameter search, or feature selection (_EVL1_), tools
Providing evaluation metrics and statistical tests(_EVL2_) and tools Providing
visualization methods (_EVL3_). Some of the tools provide functions for either
generating synthetic time series data (_DS1_) or integrate functionality to
download existing datasets (_DS2_).

The tools should be open source to explicitly target time series analysis
(SC3) and implemented using popular programming language like Python,Java,
C++, Julia or R (SC1). Some tools that can be used as building blocks for time
series analysis and whose primary purpose is not time series analysis per se
were excluded from consideration (e.g., scikit-learn, scipy , numpy in case of
Python). The search process narrowed down to tools that provide methods which
are generally domain-agnostic (IC4), and domain-specific tools were excluded.

With respect to source code repository of open source tools following points
are considered. The tools should have been evolved over a time with continuous
improvement and actively maintained(SC2.1) (last commit within less than 6
months). For source repositories available in Github , the tool should have
more than 100 GitHub stars (SC2.2). For python based tools, it should be
listed in PyPI and be installable via pip or conda (SC2.3).

##  3 Functional Specifications of TSA framework

The four baseline functional specifications of a TSA framework are as follows
Costa (2019)( See Appendix:Table LABEL:tab:requirments).

  1. 1.

Data processing capabilities: The framework must be able to provide and apply
data cleaning and preparation strategies for the input dataset.

  2. 2.

A Forecasting framework: The framework must be able to select an adequately
suited approach based on the characteristics of the given time series, which
will cover both basic and optimized approaches to forecasting for single and
multi-value time series.

  3. 3.

Anomaly Detection framework: The framework must be able to detect anomalous or
abnormal data from a given dataset employing specialized anomaly detection
approaches for single and multi-value time series.

  4. 4.

Exposing framework operations through REST API: The framework must include a
REST API that supports all operations required for forecasting and anomaly
detection.

#### Build capabilities for data processing (FS.01)

Certain time series analysis techniques require the data to contain, or not
contain, specific characteristics. Some time series are classified according
to seasonal patterns, trends, and anomalies (cite references ). The necessity
for implementing techniques that identify and, if necessary, remove seasonal
and trend patterns is enforced by this requirement. This specification also
incorporates the handling of other time series characteristics, including
missing values and noise.

  1. 1.

Physical data processing characteristics: It is critical that the data be
formatted properly for the framework operations. Keeping this in mind, this
requirement is in charge of resolving inconsistencies caused by data entry
errors.

  2. 2.

Processing time series data characteristics : To ensure that the dataset is
error-free, it is also critical to adopt measures to protect time series
structures against data anomalies. This criteria stipulates that prior to the
data being used in a model, concerns such as missing data and noise must be
dealt with effectively. Some of the integrated approaches can be considered
more robust, however, the final model has to take into account the rest, whose
performance is affected by phenomenon such as a high noise level, outliers,
gaps in the data (FS.01.02.02) and non-stationarity (FS.01.02.02)

  3. 3.

Transformation of processed data : Data transformations are commonly used to
improve visual readability and rescale data instances into relative values in
the event of normalization This transformation requirement dictates that the
framework should handle these transformations.

#### Forecasting Framework (FR.02)

  1. 1.

Integrate models: Baseline methods can be used regardless of the problem
context, but they only offer a bare minimum in terms of quality. On the other
hand, more sophisticated approaches can outperform the simpler ones. This
requirement implies combining those methods. The optimized models should
support both Univariate Time Series (UTS) (FS.02.01.01) and Multivariate Time
Series (MTS) (FS.02.01.02)

  2. 2.

Implementation of Autonomous Approach Selection: According to the features of
the time series, the forecasting framework should be able to select the most
appropriate method and determine the optimum parameters to match the time
series’ specifications (FS.02.02).

When deciding on a model, one must take into consideration the likelihood of
the model fitting. The performance of ARIMA and Long Short Term Memory (LSTM)
techniques is projected to be lower when applied to small datasets.
Requirements for how the autonomous approach selector manages the model’s
decision-making and parameterization are generated by the autonomous approach
selector.

  * •

Select the most fitting model: This requirement (FR.02.01) states that
approach selector should be able to determine the overall quality of the
different approaches within a dataset, taking into consideration the
characteristics of the dataset.

  * •

Select the most fitting parameter values: Following the selection of a
particular approach, it must be properly parametrized. This is important for
all of the methods that are available. In case of k-NN, a few parameters
should be anticipated (just the number of adjacent neighbours), whereas other
approaches may warrant a greater number of parameters to minimize errors.

#### Anomaly Detection framework (FS.03)

The requirements for the anomaly detection framework (FS.03) are identical to
those for the framework for forecasting. The primary difference is the
selection of the models the selector chooses from.

#### Exposing framework operations through REST API

A combination of the forecasting and anomaly detection frameworks constitute a
time series analysis framework results. TSA framework also aggregate their
data handling and model management potentials. Wrapping and exposing this
framework through a REST API that offers all operations of the forecasting and
anomaly detection allows flexible integration into other services (FS.04).

#### Performance Specifications

Taking processing time into account is important, as some algorithms are more
complex than others, which takes longer to complete. The Approach Selector
should be time-bound in order to select the most appropriate approach.

  1. 1.

Processing time for Approach Selection: It is evident that some Machine
Learning (ML) techniques have drastically varying computational complexity
since some techniques require much longer computation time. For the model to
be usable, it is necessary that the model’s implementation achieves a
satisfactory performance threshold, according to this requirement. In this
regard, a ratio that compares the quality of the algorithm with the complexity
of the algorithm should be calculated and should be available for the user for
adjustment.

  2. 2.

Selection processing time: There are many different algorithms available to
each of the selectors, which gives them the capabilities to deal with a
variety of time series analysis challenges. The decision on the most
appropriate method should have the least amount of impact on the overall
execution time as possible. In other words, this requirement should act as a
ceiling to set the Autonomous Selector’s execution time.

#### Quality Requirements

A review of the literature reveals that different approaches, with different
parametrization, show differing results for both prediction and anomaly
detection practices. However, because some of the data sets collected already
provide successful solutions,it is worthwhile to achieve a relative quality
threshold, as explained in the below requirement.

#### Optimum Threshold

This requirement exploits datasets that are already in use and leverages
already existing solutions to stipulate a quality threshold using metrics
employed by the solution. This is advantageous because the process can be
adjusted to generate results of comparable quality to the M4-competitive
processes and other solutions provided by other developers using other sources
like the Irvine (UCI) University Repository.

#### Technological Specifications

The framework should be implemented using popular programming language like
Java or Python and Licenses for all other software used should enable its use
without additional charge.

##  4 General architecture of TSA framework

Figure 2: General architecture of Time Series Analysis framework.

The primary modules of a typical TSA framework comprise of Forecasting
Selector, Anomaly Detection Selector, Data Processing and Third Party
Libraries.

This framework accepts time series as input, and then returns the best
optimized and performing model, based on the type of analysis being performed
(forecasting or anomaly detection). As shown in Figure 2, both UTS and MTS can
be used in case of a forecasting and MTS, if anomaly detection is the
preferred analysis.

The Forecasting Selector manages operations based on the data for which
predictions are requested by the user. It maintains a number of predictive
approaches which can be selected and correctly parameterised.

Like the Forecasting Selector, the Anomaly Detection Selector uses anomaly
detection approaches, but applies them to Multivariate Time Series (MTS).

A variety of techniques are employed by the Data Processing module in order to
ensure that data is correctly processed by the analysis selectors, while also
improving the accuracy and performance of the approaches. This component also
comes mainly from the third-party library supplied, which includes operations
dealing with data characteristics. Scikit-learn Pedregosa et al. (2011) and
Pandas McKinney et al. (2011) are such popular third-party libraries.

In order to maximize performance while minimizing information loss, the
Scikit-learn library is used for dataset feature reduction, which facilitates
feature ranking, where the features with the most discriminating power are the
most relevant.

Pandas McKinney et al. (2011) is another powerful library that aids in data
processing of Tabular data integrating other Python scientific computing
libraries such as NumPy Van Der Walt et al. (2011) and helps to eliminate
inconsistencies that are incompatible with provided selection of approaches.

##  5 Open source Tools for Time Series Analysis

A rising interest in machine learning and data mining has been seen in the
past few years in many different fields. In response to this surge in
opportunities, there is a substantial increase in the number of machine
learning packages, including both open source and proprietary. There are time
series toolboxes that provide rich interfaces to specific models
(ARIMA/filters Seabold and Perktold (2010), neural networks Alexandrov et al.
(2019) ) or framework interfaces to individual time series models tasks
(forecasting Taylor and Letham (2018) Guecioueur , feature extraction Kanter
and Veeramachaneni (2015) Tavenard et al. (2017) Christ et al. (2018), Anomaly
detection Ayadi et al. (2017) Cook et al. (2019) Wu (2016), annotation Burns
and Whyne (2018), classification Burns and Whyne (2018) Tavenard et al.
(2017)).

###  5.1 Time Series Analysis Tools

A number of effective toolboxes dedicated to time series analysis exist, which
offer common interfaces to apply modeling strategies of fitting, predicting,
and hyper-parameter estimation and fine-tuning , as well as support for the
building and tuning of composite models. More notably scikit-learn Pedregosa
et al. (2011), Orange Demšar et al. (2013) (Python), Weka Holmes et al. (1994)
Hall et al. (2009) (Java), MLj Blaom et al. (2020) (Julia), and mlr Bischl et
al. (2016), mlr3 Lang et al. (2019) or caret Kuhn (2015) (R).

Scikit-learn Pedregosa et al. (2011) is a Python module that incorporates a
diverse set of cutting-edge machine learning algorithms for medium-scale
supervised and unsupervised problems. This package is aimed at providing non-
specialists with a high-level, general use of machine learning. Although most
of the code is implemented in Python, there are also included C++ libraries
with reference implementations of support vector machines (SVMs) Cristianini
et al. (2000) and generalized linear models with compatible licenses. It
incorporates compiled code for efficiency, with a lower reliance on NumPy Van
Der Walt et al. (2011) and scipy Jones et al. (2001) only to facilitate easy
distribution, and is primarily concerned with imperative programming.

Prophet Taylor and Letham (2018) is a Facebook-developed open-source
forecasting library that is available in Python and R. It includes a single
generic forecasting model based on the exponential-smoothing method family,
but with a linear regression component as well.

Structural Time Series (STS) Tensorflow Harvey (1990) is a Google’s open-
source deep-learning framework recently announced. In Tensorflow Probability,
STS is a part of Tensorflow’s probabilistic modeling subpackage and is able to
use its specialized machinery there.

Gluon Time Series (GluonTS) Alexandrov et al. (2019) is an open-source
modeling tool kit from Amazon Web Services.Based on MXNet’s deep learning
framework, GluonTS uses a diverse set of neural forecasting models. Due to its
ease of use, GluonTS allows researchers to focus on developing models instead
of dealing with tedious tasks like data input and model creation and testing.

Cesium Naul et al. (2016), Seglearn Burns and Whyne (2018) , tsfresh Christ et
al. (2018), and tslearn Tavenard et al. (2017) are some notable Python
libraries for processing time series and sequences with machine learning.
These tools mainly assist time series data to feed models of other Python open
source libraries such as Scikit-learn, pandasn numpy, scipy.

Cesium is a complete time series analysis platform composed of a Python
library, along with a web-based front-end, and includes features such as the
ability to examine data, apply machine learning algorithms, and provide
simple, repeatable, and extendable results. The Seglearn package is a scikit-
learn compatible open-source Python module that enables the efficient
processing of time series, sequence and contextual data for classification,
regression, and forecasting issues. The ’tsfresh’ (abbreviated for Time Series
FeatuRe Extraction on basis of Scalable Hypothesis tests) is a python package
that accelerates Time series feature engineering process by combining 63 time
series characterization methods with the feature selection on the base of
automatically defined testing hypotheses that by default calculate a total of
794 time series features. ’tslearn’ is a Python package which includes
preprocessor and functional extractor tools, as well as suitable models for
the time series data (written as Scikit-learn compliant transformers).

Waikato Environment for Knowledge Analysis (Weka) (Version$\geq$3.7.3) Hall et
al. (2009) includes a dedicated time series analysis environment which enables
the creation, evaluation, and visualization of forecasting models. The time
series framework of Weka takes a machine learning/data mining approach to time
series modeling by transforming data into a form that can be used by standard
proposing learning algorithms. In Weka’s graphical user interface, this
environment takes the form of an online plugin tab and can be installed via
the package manager. The information is encoded into additional input fields (
referred to as "lagged" variables), which removes the temporal ordering of
individual input instances. The algorithms can model trends and seasonality by
using a variety of other fields that are automatically computed. Any
regression algorithms of Weka can be used to learn a model after data has been
transformed. There are many methods that can be used to make predictions for a
continuous target, such as multiple linear regression, support vector machines
for regression and model trees (decision trees with linear regression
functions at the leaves).

The Orange Demšar et al. (2013) open-source machine learning and data mining
framework is made up of scripting and visual programming elements. When it
comes to visual interface development, Orange utilizes the concept of widgets;
widgets, in this context, are essentially a graphical wrapper for complex data
analysis algorithms that are developed in Orange and Python. Widgets
communicate with one another through channels, and a collection of connected
widgets is referred to as a schema. Either orange schemas can be set up in
Python scripts or, preferably, designed in an application called Orange Canvas
through visual programming. The ’As Time Series’ widget in orange,
reinterprets any data table as a time series data and allows to choose which
data attribute represents time in the widget.

TSML Palmes et al. (2020) is an IBM package implemented in Julia to process,
classify, and forecast data for time series. TSML library provides common ML
(Machine Learning) libraries from Python’s Scikit-Learn, R’s Caret and Julia
Native MLs, to create complex assemblies that allow for robust time series
predictions, clustering, and classification, in order to seamlessly integrate
heterogeneous libraries.

The MLR3 Lang et al. (2019) package for R language provides a framework for
classification, regression, survival analysis, and clustering. It has a
object-oriented uniform interface to more than 160 basic learners and contains
feature selection, hyperparameter tuning, and ensemble creation meta-
algorithms to improve and extend the capability of basic learners. By
providing a single interface that allows for the extension and combination of
current learners, as well as the intelligent selection and tuning of the most
suited technique for a job, this approach effectively aids in the process of
meta-learning. Natively enabled parallel high-performance computing through
the ’parallelMap’ package Par  allows local multicore, socket, and MPI
computation modes. Explicitly specifying which operations to parallelize is
possible on compute clusters using ’BatchJobs’ Bischl et al. (2015) package.
’mlr3temporal’ package extends the mlr3 package framework by time-series
forecasting and resampling methods.

The package named “Caret” Kuhn (2015) (abbreviated for classification and
regression training) offers several tools for constructing predictive models
in R, which have a vast variety of machine learning models available. The
software includes features like as data splitting and pre-processing for
projects in the early phases, as well as models for feature selection and
resampling to help detect overfitting.

Some open source Feature-based time series foecasting tools in R language are
tsfeatures Hyndman , fforma Montero-Manso et al. (2020), gratis Kang et al.
(2020), seer Talagala et al. (2018). ’tsfeatures’ provides methods for
extracting various features from time series data. The ’fforma’ forecasting
tools use a model combination approach to provide forecasting and can be used
for model averaging or model selection. ’fforma’ works by training a
’classifier,’ or machine learning algorithm, how to select and combine
forecast models. n feature-based time series forecasting, the ’gratis’ package
provides efficient algorithms for generating time series with a variety of
controllable traits, which are helpful in creation of training data sets.. For
forecast model selection, the "seer" package uses time series features and
FFORMS (Feature-based FORecast Model Selection) to implement a novel
framework.

###  5.2 TSA Tools on Anomaly or Outlier Detection

Environment for Developing KDD-Applications Supported by Index-Structures
(ELKI for short) Achtert et al. (2008) is a Java-based open source (AGPLv3)
data mining software. ELKI’s primary focus is algorithm research, with a
particular emphasis on unsupervised methods in cluster analysis and outlier
detection. ELKI can also integrate a geographic/geospatial information system
(GIS) and a data mining system (DMS) into a single framework.

Anomaly Detection Extension in RapidMiner Hofmann and Klinkenberg (2016)
include popular unsupervised anomaly detection algorithms(implemented in
Java). Using this tool, it is possible to find that data that is significantly
different from normal, without having to label the data. RapidMiner’s
graphical user interface (GUI) can be used to create data mining processes
that are made up of arbitrary nested operators that are described in XML files
(GUI). Massive Online Analysis (MOA) Bifet et al. (2010) is yet another Java-
based software environment for developing algorithms and running experiments
for online learning from changing data streams. MOA is intended to address the
difficult problem of scaling up the implementation of cutting-edge algorithms
to real-world dataset size and also supports bi-directional interaction with
Weka Hall et al. (2009),.

Machine learning toolboxes like the Shogun Sonnenburg et al. (2010) is built
in C++ and have interfaces to Python and other languages like R, Octave and
Java. This software includes support vector machines, dimensionality reduction
and online learning as well as clustering and implemented kernels for numeric
data analysis algorithms.

The Scikit-learn Pedregosa et al. (2011) library also has a set of machine
learning tools that can be used for both novelty detection and outlier
detection. There are two methods for detecting anomalies: outliers and novelty
detection. Outlier detection is also referred to as unsupervised anomaly
detection, and novelty detection is referred to as semi-supervised anomaly
detection in this context.

Python Outlier Detection (PyOD) Zhao et al. (2019) is a multivariate data
outlier detection toolkit written in Python. Outlier ensembles and newly
emerging deep learning models are included among the more than 20 detection
algorithms that make up the toolkit. On top of PyOD, another acceleration
framework called Scalable Unsupervised Outlier Detection (SUOD) Zhao et al.
(2020a) is built for large-scale unsupervised outlier detector training and
prediction.

PyNomaly Constantinou (2018) is a single algorithm implementation package in
Python that implements a modified approach to Local Outlier Probabilities
(LoOP) method. PyNomaly can be used for applications involving streaming data
or when speedy calculations are required. Similary Jubatus Hido (2012) is a
distributed processing framework and streaming machine learning library
developed in C++ and support the Local Outlier Factor (LOF) method. The other
two Python based packages implementing the Half Space Trees method are Creme
Halford et al. (2019) and scikit-multiflow Montiel et al. (2018). The new
library called River Montiel et al. (2020) merges the features of Creme and
scikit-multiflow.

The Anomaly Detection Toolkit (ADTK) Arundo (2020) is a Python package that
detects anomalies in time series using unsupervised / rule-based methods. This
package provides a set of unified APIs for common detectors, transformers, and
aggregators, as well as pipe classes that connect them into models. It also
has some tools for processing and visualizing time series and anomaly events.

Python Streaming Anomaly Detection (PySAD) Yilmaz and Kozat (2020) is a Python
framework for conducting anomaly detection experiments that provides a
comprehensive set of tools. Over 15 online anomaly detection algorithms are
available in the current (v0.1.1) version of PySAD as well as two alternative
techniques for integrating PyOD detectors into the streaming environment.

TODS Lai et al. (2021) is an automated Time Series Outlier Detection System
designed for use in research and industry. Unlike other pipeline-building
systems, TODS is extremely modular that enables the rapid construction of
pipelines. TODS’ primitive is an implementation of a function with hyper-
parameters, and it serves as the foundation for all other components.
Currently, TODS supports over 70 primitives, including functions for data
manipulation and time series analysis, feature analysis, as well as algorithms
for detection and a reinforcement module for training data. End-to-end outlier
detection can be performed with the constructed pipeline by users who can
freely build a pipeline using these primitives. Drag-and-drop functionality is
provided by TODS with a Graphical User Interface (GUI), that enables users to
create flexible pipelines.

Skyline Stanway (2013) is a real-time anomaly detection, time series analysis,
and performance monitoring system designed to enable passive monitoring of
metrics without the need to configure individual models/thresholds. It is
intended for use in situations where a large number of high-resolution time
series must be continuously monitored. Once a metrics stream is configured,
Skyline automatically adds additional metrics for analysis.

Banpei Tsuruta  is a Python package for anomaly detection that was created
with the objective of establishing a real-time abnormality monitoring
environment. Banpei has a function that corresponds to the streaming data in
order to accomplish this goal. A simple monitoring system can also be built
with the help of Bokeh Bok , which is an excellent visualization library.

Telemanom Hundman et al. (2018) is a framework for detecting anomalies in
multivariate time series data using Long Short-Term Memory (LSTMs). To find
anomalies in multivariate sensor data, Telemanom uses vanilla LSTMs built with
Keras/Tensorflow. These artificial neural networks (LSTMs) are taught to
recognize common system behaviors by feeding them encoded command information
and historical telemetry data. Every time step, predictions are generated, and
predictions’ errors represent deviations from expected behavior. Telemanom
then employs a novel nonparametric, unsupervised approach to thresholding and
identifying anomalous sequences of errors for Telemanom’s algorithms.

For anomaly detection on time series data, DeepADoTS Fischer et al. (2019) is
a bench-marking pipeline for many state-of-the-art deep learning approaches.
On time series data, its main purpose is to create a benchmarking pipeline
that can be used to compare the performance of various deep learning
approaches.

In streaming, real-time applications, the Numenta Anomaly Benchmark (NAB)
Lavin and Ahmad (2015) offers a fresh benchmark for testing algorithms for
anomaly detection. It consists of more than 50 annotated real-world and
artificial timeseries data files, as well as a revolutionary scoring method
built specifically for real-time application development.

An access point to a list of ’R’ packages that are mostly used for anomaly
detection is provided by the ’CRAN Task View’ Hyndman (2021). A collection of
time series outlier detection methods is provided by the ’composits’ package,
which may be applied to compositional, multivariate, and univariate data. To
find time series outliers, this package employs the R packages forecast,
tsoutliers, otsad, and anomalize. The ’forecast’ package provides some simple
heuristic methods for identifying and correcting outliers. The ’tsoutliers’
package is primarily intended for detecting outliers in time series. This
package assist to discover patterns such as level shifts, temporary changes,
seasonal level shifts, innovative outliers and additive outliers. The ’otsad’
package contains a set of online defect (anomaly) detectors for time series
that use prediction-based and window-based algorithms. It’s capable of working
in both stationary and non-stationary context. ’anomalize’ is a “tidy”
workflow for detecting anomalies in data. An early detection package, referred
to as a ’oddstream,’ helps identify anomalous time series patterns in large
collections of streaming time series data.

##  6 Related Work

#### Research topics and Application domains

The field of time series analysis encompasses a wide range of research topics
and applications domains. The literature contains numerous reviews on a
variety of TSA topics, including forecasting ( Hendikawati et al. (2020)
Mahalakshmi et al. (2016) Panigrahi and Behera (2020) Tealab (2018) ), anomaly
detection(Wu (2016), Cook et al. (2019), Ayadi et al. (2017) ). Some of the
reviews focus on pattern recognition ( Torkamani and Lohweg (2017), Wang et
al. (2019)) , dimensionality reduction(Badhiye and Chatur (2018)) and change
point analysis ( Sharma et al. (2016), Truong et al. (2020), Aminikhanghahi
and Cook (2017) ). Some reviews cover specific application domains such as IoT
and Industry 4.0 ( Lepenioti et al. (2020), Mohammadi et al. (2018), Zhao et
al. (2020b) ), health (Zeger et al. (2006)) and finance (Sezer et al. (2020)).

Few papers conduct systematic reviews of language-specific packages or
libraries. There are several reviews of Python packages for various domains,
such as data mining Stančin and Jović (2019), topological data analysis Ray
and Trovati (2017), or social media content scraping Thivaharan et al. (2020).
With respect to R language, there are reviews related to packages for
analyzing animal movement data in R Joo et al. (2020) , as well as hydrology-
related R packages Slater et al. (2019). The present article covers the open
source Toolboxes for both Time Series Analysis and Anomaly Detection.

##  7 Discussion

###  7.1 Tools for Time Series Analysis

The tools for Time Series Analysis are largely divided into four groups based
on the Graphical User Interface (GUI) functionality. First and foremost, tools
that provide APIs but are primarily focused on the usage of a GUI that allows
novices to quickly and simply apply machine learning algorithms (e.g Weka).
Second, there are tools that focus exclusively on developing and presenting an
useful and consistent API, implying that its intended users are capable of
programming (e.g. scikit-learn and other compatible tools). The majority of
tools explored in this article belong to this group. The third type of tools
provide features both scripting and visual programming (e.g Orange). The
fourth type of tools are intended to be used as command-line tools (and
sometimes do not offer any type of API) (e.g sofia-ml).

With regard to time series learning capabilities, the tools shows different
features. sktime Löning et al. (2019) and tslearn Tavenard et al. (2020) are
generally devoted to the analysis of time series while tsfresh, cesium and
seglearn concentrate on the extraction of time series statistical
characteristics.Table 1 shows the comparison of time series learning
capabilities of following tools: cesium-ml, seglearn, sktime, tsfresh and
tslearn.

ceseium-ml (v0.9.6) and tsfresh (v0.11.1) provide multi-variable time series
feature representation learning and are currently implementing more
functionality than seglearn. Although the feature representation transformers
are built as a pre-processing step that is distinct from the pipeline, the
pipeline can nevertheless be used by other sklearn compatible tools. By
design, this results in the inability to utilize end-to-end model selection.
No obvious support exists for situations that affect a sequence or time
series, or where deep learning models are integrated.

Time series specific classical methods for clustering, classification, and
barycenter computation for time series with variable lengths have been
implemented in the latest version of tslearn (v0.1.18.4). Feature
representation learning, context learning or deep learning are not supported.
Pyts Faouzi and Janati (2020) is an entirely time series classification
package for Python.

Table 1:  Comparison of Time Series Learning features for Packages (versions):
cesium (0.9.12), seglearn (1.2.2), sktime (0.7.0), tsfresh (0.18.0), & tslearn
(0.5.2).

Feature cesium-ml seglearn sktime ts-fresh tslearn Time series target ✕ ✓ ✓ ✕
✕ Sliding window segmentation ✕ ✓ ✓ ✕ ✕ Temporal folds ✕ ✓ ✓ ✕ ✕ Context data
✓ ✓ ✓ ✕ ✕ Multivariate time series ✓ ✓ ✓ ✓ ✓ Feature representation learning ✓
✓ - ✕ ✕ Number of implemented features 112 27 - 77 N/A Classification ✓ ✓ ✓ ✓
✓ Clustering ✓ ✓ ✓ ✓ ✓ Regression ✓ ✓ ✓ ✓ ✓ Forecasting ✓ ✓ ✓ ✓ ✕ Deep
learning ✕ ✓ ✕ ✕ ✕ sklearn compatible model selection ✕ ✓ ✓ ✕ ✕

sktime and pyts stand out with their numbers of implementations of algorithms
published in the literature. sktime provides more tree-based algorithms, while
pyts is more focused on dictionary-based and image-based models.

Practitioners with a basic understanding of forecasting models but
insufficient statistical knowledge should use Prophet if they have daily data
Januschowski et al. (2019). To handle data at a different granularity, you
might need a different software program.In any circumstance, Prophet should be
used for bench-marking. The Stan model code underlying Prophet provides
greater programming flexibility for the more scientifically minded user.

The STS Tensorflow programming paradigm is beneficial for individuals who want
to experiment with a programming paradigm that is connected with the
TensorFlow environment Januschowski et al. (2019). Tensorflow’s momentum and
speed of development mean that STS is bound to get better as the community
around the tool grows and expands. At this point, it does not offer a complete
forecasting tool package. The user still has to implement standard components
like evaluation techniques and backtesting procedures, which is a task for any
but the most skilled programmer.

For users undertaking research with deep learning models and developing
scenarios for operational forecasting challenges, GluonTS can serve as the go-
to tool set of choice Januschowski et al. (2019). In terms of strategic and
tactical forecasting, Prophet is better suited for users.

In many time series algorithms, the task of reducing a complex learning task
to a simpler one is a common part of the process. Some of the frameworks allow
to take advantage of reduction relationships that exist among various time
series-related tasks to the fullest extent Beygelzimer et al. (2008). Many
time series-related tasks, such as time series regression, multivariate (or
panel) forecasting, but also forecasting and tabular (or cross-sectional)
regression and time series annotation, anomaly detection have been shown to
have reduction relationships Bontempi (2013).

Frameworks handle the time series dataset that can be either univariate,
multivariate, or panel (sometimes referred to as longitudinal data), depending
on the number and interrelation between time series variables and the number
of instances for which each variable is observed.

Frameworks implement standard interfaces for fitting, predicting and hyper-
parameters, and support composite model development and tuning along with
support for tabular (or cross-sectional) configuration. For fitting,
forecasting, and hyper-parameters, almost all of the Toolboxes or libraries
examined provide the same interface for composite model creation and
adjustment. Toolbox capabilities, however, are still constrained when used
outside of a cross-sectional context. Toolbox capabilities, however, are still
constrained when used outside of a cross-sectional context. Toolbox
capabilities, however, are still constrained when used outside of a cross-
sectional context. The problem is that none of them have a dedicated
forecasting API built in. Another class of toolboxes extends the capabilities
of tabular toolboxes by giving functionality to handle certain parts of a time
series modelling process, most notably feature extraction toolboxes such as
Featuretools Kanter and Veeramachaneni (2015), tsfresh Christ et al. (2018)
and hctsa Fulcher and Jones (2017). Various smaller toolkits for specific
reduction approaches, such as time series regression and forecasting, are also
available, ranging from tabular toolboxes.

Additionally, there are several toolboxes devoted specifically to forecasting.
However, the majority of them have significant drawbacks. Forecast library
Hyndman and Khandakar (2008) in R is a widely used and comprehensive toolbox
for forecasting. Forecast and its partner libraries provide significant
capabilities for statistical and encapsulated machine learning algorithms, as
well as for pre-processing, model selection, and assessment, when used in
conjunction. Deep-learning models for probabilistic forecasting are also
available in Python with GluonTS Alexandrov et al. (2019) and it also
interfaces with other programs such as forecast. The support for composite
model generation, however, is restricted in both programs, and neither
integrates with widely accessible machine learning libraries, such as scikit-
Learn.

Certain model families can only be used with certain forecasting toolkits in
Python. For example Time series analysis, including forecasting, can be done
with statsmodels Seabold and Perktold (2010), but it is limited to statistical
models (e.g. ARIMA, exponential smoothing and state space models). However,
pmdarima Smith et al. (2017) provides additional tools for seasonality
testing, pre-processing and pipelining, but is limited to the ARIMA family of
models. Likewise Prophet Taylor and Letham (2018) is restricted to generalised
additive models whereas PyFlux Taylor (2016) only support generalised auto-
regressive models(e.g. GAS, GARCH).

In addition, a number of repositories, such as AtsPy Snow (2020) and the
Microsoft forecasting repository, gather and aggregate common forecasting
models through interfaces to existing libraries and tools to simplify
workflows, but none of them permit the construction of composite models.

###  7.2 Discussion of TSA Tools on Anomaly detection

In the current literature, existing frameworks are divided into two
categories: (1) streaming frameworks and (2) anomaly detection frameworks.
Unlike batch models, which may access all data with a finite number of
instances during training, a streaming anomaly detection model takes data in
the form of new instances and either retains the instance for a limited length
and memory or trains itself immediately. Streaming frameworks are used for a
variety of machine learning tasks using streaming data, including
classification and regression, in addition to anomaly detection. Due to the
requirement to maintain different methods for other activities at the same
time, streaming frameworks have a limited number of specialized streaming
anomaly detection methods. A pipeline made up of projectors, ensemblers, and
probability calibrators can be used. In a streaming context, projectors
project supplied information to a (potentially) lower dimensional space so
that models may better distinguish abnormalities. Ensemblers create a score by
combining scores from numerous models for a single instance. Using various
techniques, probability calibrators transform target score into the
probability of being abnormal.

Anomaly detection software is available in a variety of programming languages,
including ELKI Data Mining, Massive Online Analysis (MOA) Bifet et al. (2010)
, and RapidMiner in Java. The ’CRAN Task View’ Hyndman (2021) provides access
to a list of ’R’ packages dedicated to Anomaly detection like tsoutliers,
anomalize and oddstream. There is also a large ecosystem of tools implemented
in Python.

For anomaly detection, existing Toolboxes are mainly divided into three types:
(1) standalone tools implementing single algorithm (like PyNomaly Constantinou
(2018) , JubatusHido (2012)), (2) part of a general larger framework that
doesn’t specifically cater to anomaly detection (Novelty and Outlier Detection
in Scikit-learn Pedregosa et al. (2011)) and (3) toolbox dedicated to Anomaly
detection (PyOD Zhao et al. (2019), PySAD Yilmaz and Kozat (2020) etc). Also
some frameworks focused on anomaly detection, such as PyOD and ADTK only
target anomaly detection on batch data, whereas some framework ( like PySAD,
Jubatus, MOA) targets anomaly detection on streaming data.

Table 2 shows the comparison of frameworks for streaming data and anomaly
detection Packages of following tools: PySAD (v0.1.1), PyOD (v0.9.5), ADTK
(v0.6.2), River (v0.8.0), scikit-multiflow (v0.5.3), Jubatus (v1.1.1) & MOA
(v2021.07).

Table 2:  Comparison of some existing frameworks for streaming data and
anomaly detection Packages (versions): PySAD (0.1.1), PyOD (0.9.5), ADTK
(0.6.2), River (0.8.0), scikit-multiflow (0.5.3), Jubatus (1.1.1) & MOA
(2021.07).

Feature PySAD PyOD ADTK River scikit-multiflow Jubatus MOA Implementation
Language Python Python Python Python Python C$++$ Java # of Models for
Streaming Data 16 0 0 2 1 1 6 Streaming ✓ ✕ ✕ ✓ ✓ ✓ ✓ Transformers ✓ ✕ ✓ ✓ ✓ ✓
✓ Projectors ✓ ✕ ✕ ✓ ✕ ✕ ✓ Ensemblers ✓ ✓ ✓ ✓ ✓ ✕ ✓ Calibrators ✓ ✕ ✕ ✕ ✕ ✕ ✕

Out of the compared frameworks some support streaming data (PySAD, River,
scikit-multiflow, Jubatus and MOA), while others support batch data only
(ADTK, PyOD). Except Jubatus, all packages support Ensemble. Projector
component is supported in only River, MOA and PySAD. The PySAD has
unsupervised probability calibrators that convert anomaly scores to
probabilities. Anomaly scores are rarely interpretable in a probabilistic
manner, and they cannot be easily transformed into anomalousness decisions.

###  7.3 Overall analysis of tools

This research work carried out the analysis of 60 tools which provide
different time analysis tasks (See Table
LABEL:tab:all_tools_summary_tsa_v1_1). The analysis found that 32 tools
explicitly providing forecasting functinality (TSK1). Forecasting is by far
the most frequently implemented task by most of the tools. Classification
tasks (TSK2) are implemented by 14 Tools. 13 Tools are classsfied to provide
clustering methods (TSK3). Anomaly Detection (TSK4) is the second most
frequently implemented task by 21 tools. Five Tools also provided segmentation
methods (TSK5). Eight Tools are classified under the category pattern
recognition (TSK6), consisting of both indexing and motif discovery tasks.
Eight Tools are identified to provide change point detection (TSK7)
functionality.

In terms of providing the data preparation capabilities, only seven tools are
identified to explicitly provide functionality of dimensionality reduction
methods (DP1). The facility of imputing missing values (DP2) exist in 23
Tools. 24 Tools support the Decomposition methods (DP3) such as decomposing
time series into trends, seasonal components or frequency components. Explicit
generic transformation and features generation methods (DP4) are implemented
in 24 Tools. The are limited number of tools (only nine) that support the
functionality of providing methods for computing similarity measures (DP5). It
has to be noted that there are less tools providing the data preparation
capabilities of DP1 and DP5. There is no significant difference, in terms of
number of tools providing other data preparation capabilities DP2, DP3 and
DP4.

During the analysis on tools that provide generation of synthetic time series
data (DS1), it is found that 22 tools provide this feature, while 31 tools
contain specific data-sets for time series. The vast majority of these tools
also include metric to evaluate the output generated. After the analysis it is
found that there are 27 tools that explicitly provide methods for model
selection (EVL1). Evaluation metric and statistical tests (EVL2) are included
in 41 tools. Standalone visualization methods (EVL3) are provided in 39 tools.

##  8 Conclusion

This paper describes a typical Time Series Analysis (TSA) framework with
architecture and listed out the main features of TSA framework. Further this
article conducted a comprehensive review of open source tools for time series
analysis. The tools are categorized based on the criteria of analysis tasks
completed, data preparation methods employed, and evaluation methods for
results generated. However this review work does not carry out evaluation of
implementations or results for these tools, such as those on benchmark
dataset. Overall, this article considered 60 time series analysis tools and 32
tools provided forecasting module and 21 packages include anomaly detection. A
list and comparison of Tools for TSA which provide automation in time series
analysis, for forecasting and anomaly detection are provided. Majority of the
tools in ecosystem exists in Python, since it is the programming languages of
choice for data scientists. The overall review activity showed that the
existing Tool ecosystem is fragmented, with several specialized tool kits for
Time series analysis but no overall framework, making it difficult to
understand, utilize, and inter operate with others tools.

## References

  * Chatfield [2005] Chris Chatfield.  Time-series forecasting.  _Significance_ , 2(3):131–133, 2005.
  * Faniband and Shaahid [2021] Yunus Parvej Faniband and SM Shaahid.  Univariate time series prediction of wind speed with a case study of yanbu, saudi arabia.  _International Journal_ , 10(1), 2021.
  * Faniband and Shaahid [2020] Yunus Parvej Faniband and SM Shaahid.  Forecasting wind speed using artificial neural networks–a case study of a potential location of saudi arabia.  In _E3S Web of Conferences_ , volume 173, page 01004. EDP Sciences, 2020.
  * Chatfield [2003] Chris Chatfield.  The analysis of time series : An introduction, sixth edition.  2003\.
  * Brockwell et al. [2016] Peter J Brockwell, Peter J Brockwell, Richard A Davis, and Richard A Davis.  _Introduction to time series and forecasting_.  Springer, 2016.
  * Makridakis et al. [2018] Spyros Makridakis, Vassilios Assimakopoulos, and Evangelos Spiliotis.  Objectivity, reproducibility and replicability in forecasting research.  _International Journal of Forecasting_ , 34(4):835–838, 2018.
  * Arnold et al. [2019] Becky Arnold, Louise Bowler, Sarah Gibson, Patricia Herterich, Rosie Higman, Anna Krystalli, Alexander Morley, Martin O’Reilly, Kirstie Whitaker, et al.  The turing way: a handbook for reproducible data science.  _Zenodo_ , 2019.
  * Smyl [2020] Slawek Smyl.  A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting.  _International Journal of Forecasting_ , 36(1):75–85, 2020.
  * Pedregosa et al. [2011] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al.  Scikit-learn: Machine learning in python.  _the Journal of machine Learning research_ , 12:2825–2830, 2011.
  * Ahmed et al. [2016a] Mohiuddin Ahmed, Abdun Naser Mahmood, and Jiankun Hu.  A survey of network anomaly detection techniques.  _Journal of Network and Computer Applications_ , 60:19–31, 2016a.
  * Ahmed et al. [2016b] Mohiuddin Ahmed, Abdun Naser Mahmood, and Md Rafiqul Islam.  A survey of anomaly detection techniques in financial domain.  _Future Generation Computer Systems_ , 55:278–288, 2016b.
  * Petropoulos et al. [2022] Fotios Petropoulos, Daniele Apiletti, Vassilios Assimakopoulos, Mohamed Zied Babai, Devon K Barrow, Souhaib Ben Taieb, Christoph Bergmeir, Ricardo J Bessa, Jakub Bijak, John E Boylan, et al.  Forecasting: theory and practice.  _International Journal of Forecasting_ , 2022.
  * Kitchenham and Brereton [2013] Barbara Kitchenham and Pearl Brereton.  A systematic review of systematic review process research in software engineering.  _Information and software technology_ , 55(12):2049–2075, 2013.
  * Fakhrazari and Vakilzadian [2017] Amin Fakhrazari and Hamid Vakilzadian.  A survey on time series data mining.  In _2017 IEEE International Conference on Electro Information Technology (EIT)_ , pages 476–481. IEEE, 2017.
  * Hendikawati et al. [2020] P Hendikawati et al.  A survey of time series forecasting from stochastic method to soft computing.  In _Journal of Physics: Conference Series_ , volume 1613, page 012019\. IOP Publishing, 2020.
  * Mahalakshmi et al. [2016] Ganapathy Mahalakshmi, S Sridevi, and Shyamsundar Rajaram.  A survey on forecasting of time series data.  In _2016 International Conference on Computing Technologies and Intelligent Data Engineering (ICCTIDE’16)_ , pages 1–8. IEEE, 2016.
  * Panigrahi and Behera [2020] Sibarama Panigrahi and Himansu Sekhar Behera.  Fuzzy time series forecasting: a survey.  _Computational Intelligence in Data Mining_ , pages 641–651, 2020\.
  * Tealab [2018] Ahmed Tealab.  Time series forecasting using artificial neural networks methodologies: A systematic review.  _Future Computing and Informatics Journal_ , 3(2):334–340, 2018.
  * Abanda et al. [2019] Amaia Abanda, Usue Mori, and Jose A Lozano.  A review on distance based time series classification.  _Data Mining and Knowledge Discovery_ , 33(2):378–412, 2019.
  * Bagnall et al. [2017] Anthony Bagnall, Jason Lines, Aaron Bostrom, James Large, and Eamonn Keogh.  The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances.  _Data mining and knowledge discovery_ , 31(3):606–660, 2017.
  * Aghabozorgi et al. [2015] Saeed Aghabozorgi, Ali Seyed Shirkhorshidi, and Teh Ying Wah.  Time-series clustering–a decade review.  _Information Systems_ , 53:16–38, 2015.
  * Wu [2016] Hu-Sheng Wu.  A survey of research on anomaly detection for time series.  In _2016 13th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP)_ , pages 426–431. IEEE, 2016.
  * Cook et al. [2019] Andrew A Cook, Göksel Mısırlı, and Zhong Fan.  Anomaly detection for iot time-series data: A survey.  _IEEE Internet of Things Journal_ , 7(7):6481–6494, 2019.
  * Ayadi et al. [2017] Aya Ayadi, Oussama Ghorbel, Abdulfattah M Obeid, and Mohamed Abid.  Outlier detection approaches for wireless sensor networks: A survey.  _Computer Networks_ , 129:319–333, 2017.
  * Torkamani and Lohweg [2017] Sahar Torkamani and Volker Lohweg.  Survey on time series motif discovery.  _Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery_ , 7(2):e1199, 2017.
  * Wang et al. [2019] Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu.  Deep learning for sensor-based activity recognition: A survey.  _Pattern Recognition Letters_ , 119:3–11, 2019.
  * Sharma et al. [2016] Shilpy Sharma, David A Swayne, and Charlie Obimbo.  Trend analysis and change point techniques: a survey.  _Energy, Ecology and Environment_ , 1(3):123–130, 2016.
  * Truong et al. [2020] Charles Truong, Laurent Oudre, and Nicolas Vayatis.  Selective review of offline change point detection methods.  _Signal Processing_ , 167:107299, 2020.
  * Aminikhanghahi and Cook [2017] Samaneh Aminikhanghahi and Diane J Cook.  A survey of methods for time series change point detection.  _Knowledge and information systems_ , 51(2):339–367, 2017.
  * Costa [2019] Pedro Moreira Costa.  _Time-Series Analysis Framework_.  PhD thesis, Universidade de Coimbra, 2019.
  * McKinney et al. [2011] Wes McKinney et al.  pandas: a foundational python library for data analysis and statistics.  _Python for high performance and scientific computing_ , 14(9):1–9, 2011.
  * Van Der Walt et al. [2011] Stefan Van Der Walt, S Chris Colbert, and Gael Varoquaux.  The numpy array: a structure for efficient numerical computation.  _Computing in science & engineering_, 13(2):22–30, 2011.
  * Seabold and Perktold [2010] Skipper Seabold and Josef Perktold.  Statsmodels: Econometric and statistical modeling with python.  In _Proceedings of the 9th Python in Science Conference_ , volume 57, page 61. Austin, TX, 2010.
  * Alexandrov et al. [2019] Alexander Alexandrov, Konstantinos Benidis, Michael Bohlke-Schneider, Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle C Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, et al.  Gluonts: Probabilistic time series models in python.  _arXiv preprint arXiv:1906.05264_ , 2019.
  * Taylor and Letham [2018] Sean J Taylor and Benjamin Letham.  Forecasting at scale.  _The American Statistician_ , 72(1):37–45, 2018\.
  * [36] Ahmed Guecioueur.  pysf: Supervised forecasting of sequential data in python, 2018.  _URL: https://github. com/alan-turing-institute/pysf_.
  * Kanter and Veeramachaneni [2015] James Max Kanter and Kalyan Veeramachaneni.  Deep feature synthesis: Towards automating data science endeavors.  In _2015 IEEE international conference on data science and advanced analytics (DSAA)_ , pages 1–10. IEEE, 2015.
  * Tavenard et al. [2017] Romain Tavenard, Johann Faouzi, Gilles Vandewiele, Felix Divo, Guillaume Androz, Chester Holtz, Marie Payne, Roman Yurchak, Marc Rußwurm, Kushal Kolar, et al.  tslearn: A machine learning toolkit dedicated to time-series data, 2017\.
  * Christ et al. [2018] Maximilian Christ, Nils Braun, Julius Neuffer, and Andreas W Kempa-Liehr.  Time series feature extraction on basis of scalable hypothesis tests (tsfresh–a python package).  _Neurocomputing_ , 307:72–77, 2018.
  * Burns and Whyne [2018] David M Burns and Cari M Whyne.  Seglearn: a python package for learning sequences and time series.  _The Journal of Machine Learning Research_ , 19(1):3238–3244, 2018.
  * Demšar et al. [2013] Janez Demšar, Tomaž Curk, Aleš Erjavec, Črt Gorup, Tomaž Hočevar, Mitar Milutinovič, Martin Možina, Matija Polajnar, Marko Toplak, Anže Starič, et al.  Orange: data mining toolbox in python.  _the Journal of machine Learning research_ , 14(1):2349–2353, 2013.
  * Holmes et al. [1994] Geoffrey Holmes, Andrew Donkin, and Ian H Witten.  Weka: A machine learning workbench.  In _Proceedings of ANZIIS’94-Australian New Zealnd Intelligent Information Systems Conference_ , pages 357–361. IEEE, 1994.
  * Hall et al. [2009] Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten.  The weka data mining software: an update.  _ACM SIGKDD explorations newsletter_ , 11(1):10–18, 2009.
  * Blaom et al. [2020] Anthony D Blaom, Franz Kiraly, Thibaut Lienart, Yiannis Simillides, Diego Arenas, and Sebastian J Vollmer.  Mlj: A julia package for composable machine learning.  _arXiv preprint arXiv:2007.12285_ , 2020.
  * Bischl et al. [2016] Bernd Bischl, Michel Lang, Lars Kotthoff, Julia Schiffner, Jakob Richter, Erich Studerus, Giuseppe Casalicchio, and Zachary M Jones.  mlr: Machine learning in r.  _The Journal of Machine Learning Research_ , 17(1):5938–5942, 2016.
  * Lang et al. [2019] Michel Lang, Martin Binder, Jakob Richter, Patrick Schratz, Florian Pfisterer, Stefan Coors, Quay Au, Giuseppe Casalicchio, Lars Kotthoff, and Bernd Bischl.  mlr3: A modern object-oriented machine learning framework in r.  _Journal of Open Source Software_ , 4(44):1903, 2019.
  * Kuhn [2015] Max Kuhn.  Caret: classification and regression training.  _Astrophysics Source Code Library_ , pages ascl–1505, 2015.
  * Cristianini et al. [2000] Nello Cristianini, John Shawe-Taylor, et al.  _An introduction to support vector machines and other kernel-based learning methods_.  Cambridge university press, 2000.
  * Jones et al. [2001] Eric Jones, Travis Oliphant, Pearu Peterson, et al.  Scipy: Open source scientific tools for python.  2001\.
  * Harvey [1990] Andrew C Harvey.  Forecasting, structural time series models and the kalman filter.  1990\.
  * Naul et al. [2016] Brett Naul, Stéfan van der Walt, Arien Crellin-Quick, Joshua S Bloom, and Fernando Pérez.  cesium: Open-source platform for time-series inference.  _arXiv preprint arXiv:1609.04504_ , 2016.
  * Palmes et al. [2020] Paulito Palmes, Joern Ploennigs, and Niall Brady.  Tsml (time series machine learnng).  _arXiv preprint arXiv:2005.13191_ , 2020.
  * [53] parallelmap:r package to interface some popular parallelization backends with a unified interface.  URL https://github.com/mlr-org/parallelMap.
  * Bischl et al. [2015] Bernd Bischl, Michel Lang, Olaf Mersmann, Jörg Rahnenführer, and Claus Weihs.  Batchjobs and batchexperiments: Abstraction mechanisms for using r in batch environments.  _Journal of Statistical Software_ , 64(1):1–25, 2015.
  * [55] Rob J. Hyndman.  tsfeatures.  URL https://github.com/robjhyndman/tsfeatures.  original-date: 2017-11-22T01:55:59Z.
  * Montero-Manso et al. [2020] Pablo Montero-Manso, George Athanasopoulos, Rob J Hyndman, and Thiyanga S Talagala.  Fforma: Feature-based forecast model averaging.  _International Journal of Forecasting_ , 36(1):86–92, 2020.
  * Kang et al. [2020] Yanfei Kang, Rob J Hyndman, and Feng Li.  Gratis: Generating time series with diverse and controllable characteristics.  _Statistical Analysis and Data Mining: The ASA Data Science Journal_ , 13(4):354–376, 2020.
  * Talagala et al. [2018] Thiyanga S Talagala, Rob J Hyndman, George Athanasopoulos, et al.  Meta-learning how to forecast time series.  _Monash Econometrics and Business Statistics Working Papers_ , 6:18, 2018.
  * Achtert et al. [2008] Elke Achtert, Hans-Peter Kriegel, and Arthur Zimek.  Elki: a software system for evaluation of subspace clustering algorithms.  In _International Conference on Scientific and Statistical Database Management_ , pages 580–585. Springer, 2008.
  * Hofmann and Klinkenberg [2016] Markus Hofmann and Ralf Klinkenberg.  _RapidMiner: Data mining use cases and business analytics applications_.  CRC Press, 2016.
  * Bifet et al. [2010] Albert Bifet, Geoff Holmes, Bernhard Pfahringer, Philipp Kranen, Hardy Kremer, Timm Jansen, and Thomas Seidl.  Moa: Massive online analysis, a framework for stream classification and clustering.  In _Proceedings of the First Workshop on Applications of Pattern Analysis_ , pages 44–50. PMLR, 2010.
  * Sonnenburg et al. [2010] Sören Sonnenburg, Gunnar Rätsch, Sebastian Henschel, Christian Widmer, Jonas Behr, Alexander Zien, Fabio de Bona, Alexander Binder, Christian Gehl, and Vojtěch Franc.  The shogun machine learning toolbox.  _The Journal of Machine Learning Research_ , 11:1799–1802, 2010.
  * Zhao et al. [2019] Yue Zhao, Zain Nasrullah, and Zheng Li.  Pyod: A python toolbox for scalable outlier detection.  _arXiv preprint arXiv:1901.01588_ , 2019.
  * Zhao et al. [2020a] Yue Zhao, Xueying Ding, Jianing Yang, and Haoping Bai.  Suod: Toward scalable unsupervised outlier detection.  _arXiv preprint arXiv:2002.03222_ , 2020a.
  * Constantinou [2018] Valentino Constantinou.  Pynomaly: Anomaly detection using local outlier probabilities (loop).  _Journal of Open Source Software_ , 3(30):845, 2018.
  * Hido [2012] S Hido.  Jubatus: Distributed online machine learning framework for big data.  _Proc. of the 1st Extremely Large Databases (XLDB) Asia_ , 2012.
  * Halford et al. [2019] Max Halford, Geoffrey Bolmier, Raphael Sourty, Robin Vaysse, and Adil Zouitine.  creme, a python library for online machine learning, 2019.
  * Montiel et al. [2018] Jacob Montiel, Jesse Read, Albert Bifet, and Talel Abdessalem.  Scikit-multiflow: A multi-output streaming framework.  _The Journal of Machine Learning Research_ , 19(1):2915–2914, 2018.
  * Montiel et al. [2020] Jacob Montiel, Max Halford, Saulo Martiello Mastelini, Geoffrey Bolmier, Raphael Sourty, Robin Vaysse, Adil Zouitine, Heitor Murilo Gomes, Jesse Read, Talel Abdessalem, and Albert Bifet.  River: machine learning for streaming data in python, 2020.
  * Arundo [2020] Arundo.  Anomaly detection toolkit (adtk).  _Online Code Repos_ , 2020.  URL https://github.com/arundo/adtk.
  * Yilmaz and Kozat [2020] Selim F Yilmaz and Suleyman S Kozat.  Pysad: A streaming anomaly detection framework in python.  _arXiv preprint arXiv:2009.02572_ , 2020.
  * Lai et al. [2021] Kwei-Herng Lai, Daochen Zha, Guanchu Wang, Junjie Xu, Yue Zhao, Devesh Kumar, Yile Chen, Purav Zumkhawaka, Minyang Wan, Diego Martinez, and Xia Hu.  Tods: An automated time series outlier detection system.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 35(18):16060–16062, May 2021.
  * Stanway [2013] A Stanway.  Etsy skyline.  _Online Code Repos_ , 2013.  URL https://github.com/etsy/skyline.
  * [74] Hirofumi Tsuruta.  banpei - anomaly detection library based on singular spectrum transformation(sst).  URL https://github.com/tsurubee/banpei.
  * [75] bokeh : Interactive data visualization in the browser, from python.  URL https://github.com/bokeh/bokeh.
  * Hundman et al. [2018] Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom.  Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding.  In _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_, pages 387–395, 2018.
  * Fischer et al. [2019] Maxi Fischer, Willi Gierke, Thomas Kellermeier, Ajay Kesar, Axel Stebner, and Daniel Thevessen.  Anomaly detection on time series: An evaluation of deep learning methods, 2019.  URL https://github.com/KDD-OpenSource/DeepADoTS.
  * Lavin and Ahmad [2015] Alexander Lavin and Subutai Ahmad.  Evaluating real-time anomaly detection algorithms–the numenta anomaly benchmark.  In _2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA)_ , pages 38–44. IEEE, 2015.
  * Hyndman [2021] Rob J Hyndman.  Cran task view: Time series analysis.  _Online Code Repos_ , 2021.  URL https://github.com/pridiltal/ctv-AnomalyDetection.
  * Badhiye and Chatur [2018] Sagar S Badhiye and PN Chatur.  A review on time series dimensionality reduction.  _HELIX_ , 8(5):3957–3960, 2018.
  * Lepenioti et al. [2020] Katerina Lepenioti, Alexandros Bousdekis, Dimitris Apostolou, and Gregoris Mentzas.  Prescriptive analytics: Literature review and research challenges.  _International Journal of Information Management_ , 50:57–70, 2020.
  * Mohammadi et al. [2018] Mehdi Mohammadi, Ala Al-Fuqaha, Sameh Sorour, and Mohsen Guizani.  Deep learning for iot big data and streaming analytics: A survey.  _IEEE Communications Surveys & Tutorials_, 20(4):2923–2960, 2018.
  * Zhao et al. [2020b] Yang Zhao, Chaobo Zhang, Yiwen Zhang, Zihao Wang, and Junyang Li.  A review of data mining technologies in building energy systems: Load prediction, pattern identification, fault detection and diagnosis.  _Energy and Built Environment_ , 1(2):149–164, 2020b.
  * Zeger et al. [2006] Scott L Zeger, Rafael Irizarry, and Roger D Peng.  On time series analysis of public health and biomedical data.  _Annu. Rev. Public Health_ , 27:57–79, 2006.
  * Sezer et al. [2020] Omer Berat Sezer, Mehmet Ugur Gudelek, and Ahmet Murat Ozbayoglu.  Financial time series forecasting with deep learning : A systematic literature review: 2005–2019.  _Applied Soft Computing_ , 90:106181, 2020.  ISSN 1568-4946.  doi:https://doi.org/10.1016/j.asoc.2020.106181.  URL https://www.sciencedirect.com/science/article/pii/S1568494620301216.
  * Stančin and Jović [2019] Igor Stančin and Alan Jović.  An overview and comparison of free python libraries for data mining and big data analysis.  In _2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)_ , pages 977–982. IEEE, 2019.
  * Ray and Trovati [2017] Jeffrey Ray and Marcello Trovati.  A survey of topological data analysis (tda) methods implemented in python.  In _International conference on intelligent networking and collaborative systems_ , pages 594–600. Springer, 2017.
  * Thivaharan et al. [2020] S Thivaharan, G Srivatsun, and S Sarathambekai.  A survey on python libraries used for social media content scraping.  In _2020 International Conference on Smart Electronics and Communication (ICOSEC)_ , pages 361–366. IEEE, 2020.
  * Joo et al. [2020] Rocio Joo, Matthew E Boone, Thomas A Clay, Samantha C Patrick, Susana Clusella-Trullas, and Mathieu Basille.  Navigating through the r packages for movement.  _Journal of Animal Ecology_ , 89(1):248–267, 2020\.
  * Slater et al. [2019] Louise J Slater, Guillaume Thirel, Shaun Harrigan, Olivier Delaigue, Alexander Hurley, Abdou Khouakhi, Ilaria Prosdocimi, Claudia Vitolo, and Katie Smith.  Using r in hydrology: a review of recent developments and future directions.  _Hydrology and Earth System Sciences_ , 23(7):2939–2963, 2019.
  * Löning et al. [2019] Markus Löning, Anthony Bagnall, Sajaysurya Ganesh, Viktor Kazakov, Jason Lines, and Franz J Király.  sktime: A unified interface for machine learning with time series.  _arXiv preprint arXiv:1909.07872_ , 2019.
  * Tavenard et al. [2020] Romain Tavenard, Johann Faouzi, Gilles Vandewiele, Felix Divo, Guillaume Androz, Chester Holtz, Marie Payne, Roman Yurchak, Marc Rußwurm, Kushal Kolar, et al.  Tslearn, a machine learning toolkit for time series data.  _J. Mach. Learn. Res._ , 21(118):1–6, 2020.
  * Faouzi and Janati [2020] Johann Faouzi and Hicham Janati.  pyts: A python package for time series classification.  _J. Mach. Learn. Res._ , 21:46–1, 2020.
  * Januschowski et al. [2019] Tim Januschowski, Jan Gasthaus, and Yuyang Wang.  Open-source forecasting tools in python.  _Foresight: The International Journal of Applied Forecasting_ , (55), 2019.
  * Beygelzimer et al. [2008] Alina Beygelzimer, John Langford, and Bianca Zadrozny.  Machine learning techniques—reductions between prediction quality metrics.  In _Performance Modeling and Engineering_ , pages 3–28. Springer, 2008.
  * Bontempi [2013] Gianluca Bontempi.  Machine learning strategies for time series prediction.  _European Business Intelligence Summer School, Hammamet, Lecture_ , pages 1–128, 2013.
  * Fulcher and Jones [2017] Ben D Fulcher and Nick S Jones.  hctsa: A computational framework for automated time-series phenotyping using massive feature extraction.  _Cell systems_ , 5(5):527–531, 2017.
  * Hyndman and Khandakar [2008] Rob J Hyndman and Yeasmin Khandakar.  Automatic time series forecasting: the forecast package for r.  _Journal of statistical software_ , 27(1):1–22, 2008.
  * Smith et al. [2017] Taylor G Smith et al.  pmdarima: Arima estimators for python.  _Online Code Repos_ , 309, 2017.  URL http://www.alkaline-ml.com/pmdarima.
  * Taylor [2016] Ross Taylor.  Pyflux: An open source time series library for python, 2016.
  * Snow [2020] Derek Snow.  Atspy: Automated time series forecasting in python.  _Available at SSRN 3580631_ , 2020.

Table 3: Technical Specifications of a Time Series Analysis Framework. ID |  Requirement Title |  Description |  Dependencies
---|---|---|---
Functional Specifications
FS.01 |  Build capabilities for data processing |  Data preparation and processing should be handled within the framework. |  -
FS.02 |  Build a framework for Forecasting |  Provide an interface for forecast operations that manages different forecast approaches for different time series and selects the most suited, properly parametersed approach. |  FS.01
FS.03 |  Build a Anomaly Detection framework |  IIncorprate an anomaly detection framework that manages different anomaly detection approaches and performances,& outputs the most appropriate, based on the features of the input dataset. |  FS.01
FS.04 |  Exposing framework operations through REST API |  The framework should be available as REST APIs wrapped in an API interface. |  -
Specifications for Implementation of data processing techniques
FS.01.01 |  Physical data processing characteristics |  Raw data should be organized into a tabular format,by the framework, so that the approaches can understand. |  -
FS.01.02 |  Processing time series data characteristics |  The implementation of features that are specific to time series data should be provided. |  FS.01.01
FS.01.03 |  Transformation of processed data |  Allow techniques for dealing with both conventional statistical data transformations and machine learning transformations. |  FS.01.01, FS.01.02
Specifications for Processing time series data characteristics
FS.01.02.01 |  Non-stationarity is addressed |  Non-stationarity characerstics like Seasonality and trend should be detected and removed using various data processing techniques. |  -
FS.01.02.02 |  Detect and manage missing data |  When there are missing data occurances, interpolation should be used to mask these instances. |  -
Specifications to Build a Forecasting framework (FS.02)
FS.02.01 |  Integrate models |  The framework should incorporate a variety of specialized approaches, the selection of which should be determined by the data. |  -
FS.02.02 |  Implementation of Autonomous Approach Selection |  The implementation should include support for automating the decision process of optimal forecasting approach which includes the step of model parameterization. |  FS.02.01
Specifications for models’ integration (FS.02.01) )
FS.02.01.01 |  Integrate models for UTS |  Optimized models that have been implemented should be suitable for Univariate Time Series (UTS). |  -
FS.02.01.02 |  Integrate models for MTS |  Optimized models that have been implemented should be suitable for Multivariate Time Series (MTS) |  -
Specifications for implementation of Autonomous Approach Selection (FS.02.02)
FS.02.02.01 |  Select the most fitting model |  Given the dataset and its characteristics, the approach selector can evaluate approaches available in terms of their quality. |  -
FS.02.02.02 |  Select the most fitting parameter values |  Once an approach is adopted, the selector should choose an approach to parametrize the results so as to maximize the quality of the results. |  FS.02.03.01
Performance Specifications
PS.01 |  Processing time for Approach Selection |  The Approach Selector should take into consideration the processing time for each approach. |  -
PS.02 |  Selection processing time |  The Approach Selector should be implemented in such a way that it minimizes the time required for selection. |  -
Quality Specifications
QS.01 |  Optimum Threshold |  According to a given benchmark, the framework should define a minimum accuracy. |  -
Table 4: Summary of Tools for TSA. | Tasks | Data Preparation | Dataset | Evaluation
---|---|---|---|---
Tool Name |  TSK1 |  TSK2 |  TSK3 |  TSK4 |  TSK5 |  TSK6 |  TSK7 |  DP1 |  DP2 |  DP3 |  DP4 |  DP5 |  DS1 |  DS2 |  EVL1 |  EVL2 |  EVL3
ADTK ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  ✣ |  |  |  |  ✣ |  |  |  |  |  |  |  ✣
arch ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  |  |  |  |  ✣ |  ✣ |  |  ✣ |
atspy ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |  |  |  |  ✣ |  ✣ |  ✣
banpei ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  ✣ |  |  |  |  |  |  |  |  |  |
cesium ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  |  |  ✣ |  |  |  |  |  |
Caret in R ${}^{\@add@centering\textnormal{ \centering\char 72}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  |  |  |  |  ✣ |  ✣ |  ✣ |  ✣
CRAN Task View ${}^{\@add@centering\textnormal{ \centering\char 72}}$ |  ✣ |  |  ✣ |  ✣ |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣ |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣ |  ✣
darts ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |  |  ✣ |  ✣ |  |  ✣ |  ✣
DeepADoTS ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  |  |  |  |  |  |  |  |  |  ✣ |  ✣
deeptime ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  ✣ |  |  |  |  |  ✣ |  |  ✣ |  ✣ |  |  ✣ |  |  |  ✣ |  ✣
deltapy ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  ✣ |  |  |  |  |  |  ✣ |  |  ✣ |  ✣ |  ✣ |  |  |  |  |
dtaidistance ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  |  |  |  |  ✣
EMD-signal ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  |  ✣ |  |  |  |  |  |  |  ✣
ELKI Data Mining ${}^{\@add@centering\textnormal{ \centering\char 63}}$ |  |  |  ✣ |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |  |
flood-forecast ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  |  |  |  |  ✣ |  ✣
gluonts ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  |  ✣ |  ✣ |  |  ✣ |  ✣
hcrystalball ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣
hmmlearn ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  |  ✣ |  |  |  |
hypertools ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  ✣ |  |  |  |  |  ✣ |  ✣ |  |  ✣ |  |  |  |  |  |  ✣
Jubatus ${}^{\@add@centering\textnormal{ \centering\char 115}}$ |  ✣ |  ✣ |  ✣ |  ✣ |  |  |  |  |  |  |  |  |  |  |  |  ✣ |  ✣
linearmodels ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  |  |  |  |  |  ✣ |  |  ✣ |
luminaire ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  ✣ |  |  |  ✣ |  |  ✣ |  ✣ |  ✣ |  |  |  |  ✣ |  |
Massive Online Analysis ${}^{\@add@centering\textnormal{ \centering\char 63}}$ |  ✣ |  ✣ |  ✣ |  ✣ |  |  |  |  |  ✣ |  ✣ |  ✣ |  |  |  ✣ |  ✣ |  ✣ |  ✣
matrixprofile ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  |  |  ✣ |  |  |  ✣ |  |  ✣ |  |  |  ✣
mcfly ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  ✣ |  |  |  |  |  |  |  |  |  |  |  |  |  ✣ |  |  ✣
MLR3 ${}^{\@add@centering\textnormal{ \centering\char 72}}$ |  ✣ |  ✣ |  ✣ |  |  |  |  |  |  ✣ |  ✣ |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣
neuralprophet ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |  |  |  |  |  |  ✣
nolds ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  ✣ |  |  |  |  ✣ |  ✣ |  |  ✣ |
Numenta AB ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  ✣ |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |
Orange ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  |  |  |  |  |  ✣ |  ✣
pmdarima ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  ✣ |  |  |  ✣ |  ✣ |  ✣ |  ✣
prophet ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  ✣ |  |  |  ✣ |  |  |  |  |  |  ✣ |  ✣
pyaf ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |  |  |  ✣ |  ✣ |  ✣ |  ✣
pycwt ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  |  ✣ |  |  |  |  ✣ |  |  |
pydlm ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  |  |  |  |  ✣ |  ✣ |  ✣
pyFTS ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣
PyNomaly ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  |  |  |  |  |  |  |  |  |  ✣ |
PySAD ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  |  |  |  |  |  |  |  |  ✣ |  ✣ |
PyOD ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  ✣ |  ✣
pyodds ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣
pytorchts ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  |  ✣ |  ✣ |  |  |  ✣
pyts ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  ✣ |  |  |  ✣ |  ✣ |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣ |  ✣ |  |  ✣ |  ✣
PyWavelets ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  |  ✣ |  ✣ |  |  ✣ |  ✣ |  |  |
ruptures ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  ✣ |  |  |  |  |  |  |  |  ✣ |  ✣ |  ✣
scikit-multiflow ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  ✣ |  |  ✣ |  |  |  ✣ |  |  ✣ |  |  |  ✣ |  ✣ |  |  |  ✣ |  ✣
seglearn ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  |  |  ✣ |  |  |  ✣ |  |  |
Shogun ${}^{\@add@centering\textnormal{ \centering\char 115}}$ |  ✣ |  ✣ |  ✣ |  |  |  |  |  |  |  |  |  |  |  ✣ |  |  ✣ |
sktime ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  ✣ |  |  ✣ |  ✣ |  |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  |  ✣ |  ✣ |  ✣ |  ✣
sktime-dl ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  ✣ |  |  |  |  |  |  |  |  |  |  |  |  |  ✣ |  |
statsmodels ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  ✣ |  |  |  |  |  |  |  ✣ |  ✣ |  ✣ |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣
stumpy ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  ✣ |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  |
SUOD ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  ✣ |  ✣
Telemanom ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  ✣
TODS ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  ✣ |  |  ✣ |  |  ✣ |  |  ✣ |  |  |  |  ✣ |  ✣ |  ✣ |
tftb ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  |  ✣ |  ✣ |  |  ✣ |  |  |  ✣ |  ✣
tsfresh ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  |  |  |  |  |  |  |  ✣ |  |  ✣ |  |  |  |  ✣ |  ✣ |
tslearn ${}^{\@add@centering\textnormal{ \centering\char 65}}$ |  |  ✣ |  ✣ |  |  |  ✣ |  |  |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  |  ✣ |
tsml ${}^{\@add@centering\textnormal{ \centering\char 169}}$ |  ✣ |  ✣ |  |  |  |  |  |  |  ✣ |  |  |  |  |  ✣ |  ✣ |  ✣ |
Weka ${}^{\@add@centering\textnormal{ \centering\char 63}}$ |  ✣ |  ✣ |  ✣ |  ✣ |  |  |  |  |  ✣ |  ✣ |  |  |  ✣ |  ✣ |  ✣ |  ✣ |  ✣
Total (60) |  32 |  14 |  13 |  21 |  5 |  8 |  8 |  7 |  23 |  24 |  24 |  9 |  22 |  31 |  27 |  41 |  39
Note: TSA Tasks: TSK1 - Forecasting methods (FS.02), TSK2 - Classification
methods, TSK3 - Clustering methods, TSK4 - Anomaly Detection methods (FS.03),
TSK5 - Segmentation methods, TSK6 - Pattern Recognition, TSK7 - Change Point
Detection. Data Preparation Modules: DP1 - Dimensionality reduction methods,
DP2 - Missing values imputation methods (FS.01.02.02), DP3 - Decomposition
methods (e.g., decomposing time series into trends, seasonal components, or
frequency components) (FS.01.02.01), DP4 - preprocessing, DP5 - Similarity
measures. Datasets: D1 - Generating synthetic time series data, D2- Providing
access to time series datasets. Evaluation Components: EVL1 - Methods for
model selection, hyperparameter search, or feature selection, EVL2 - Providing
evaluation metrics and statistical tests, EVL3 - Providing visualization. ✣ -
Module Exists. Primary Implementation language:
${}^{\@add@centering\textnormal{ \centering\char 65}}$\- Python,
${}^{\@add@centering\textnormal{ \centering\char 72}}$\- R,
${}^{\@add@centering\textnormal{ \centering\char 63}}$\- Java,
${}^{\@add@centering\textnormal{ \centering\char 169}}$\- Julia,
${}^{\@add@centering\textnormal{ \centering\char 115}}$\- C++.