# Towards Robust Representation of Limit Orders Books
for Deep Learning Models

Yufei Wu  J.P. Morgan AI ResearchLondonUnited Kingdom<EMAIL_ADDRESS>,
Mahmoud Mahfouz  J.P. Morgan AI Research,
Imperial College LondonLondonUnited Kingdom<EMAIL_ADDRESS>,
Daniele Magazzeni  J.P. Morgan AI ResearchLondonUnited Kingdom
<EMAIL_ADDRESS>and  Manuela Veloso  J.P. Morgan AI ResearchNew
YorkUnited States of America<EMAIL_ADDRESS>
(2021)

###### Abstract.

The success of machine learning models is highly reliant on the quality and
robustness of representations. The lack of attention on the robustness of
representations may boost risks when using data-driven machine learning models
for trading in the financial markets. In this paper, we focus on
representations of the limit order book (LOB) data and discuss the
opportunities and challenges of representing such data in an effective and
robust manner. We analyse the issues associated with the commonly-used LOB
representation for machine learning models from both theoretical and
experimental perspectives. Based on this, we propose new LOB representation
schemes to improve the performance and robustness of machine learning models
and present a guideline for future research in this area.

deep learning, knowledge representation, limit order books

††copyright: acmcopyright††journalyear: 2021††doi:
10.1145/1122445.1122456††conference: ICAIF-2021: ACM International Conference
on AI in Finance; November 3–5, 2021; Online††booktitle: ICAIF-2021: ACM
International Conference on AI in Finance, November 3–5, 2021, Online††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Computing methodologies Neural
networks††ccs: Applied computing Forecasting

##  1\. Introduction

Figure 1. (A) A snapshot of the limit order book. (B) Workflow of a price
forecasting task using LOB data with machine learning models.

Limit order books (LOBs) are used by financial exchanges to match buyers and
sellers of a particular instrument and act as an indicator of the supply and
demand at a given point in time. It can be described as a self-evolving
process with complex spatial and temporal structures revealing the price
dynamics at the microstructural level. Market making, optimal execution and
statistical arbitrage strategies, all require a good understanding of the LOB
and its dynamics. Figure 1 (A) shows a snapshot of the LOB with both the bid
(buy) and ask (sell) order volumes accumulated at each price level. The mid-
price is the average of the best (lowest) ask price and the best (highest) bid
price and the difference between them is referred to as the bid-ask spread.
The LOB gets updated continuously with order placements, cancellations and
executions.

The use of algorithmic trading strategies and the digitisation of exchange
activities has made available a tremendous amount of LOB data for
practitioners and researchers to study the market dynamics using data-driven
approaches. This led to a surge in interest for big data applications in the
financial markets and machine learning (including deep learning) models have
become a trend in the quantitative finance domain (Buehler et al., 2019),
(Wiese et al., 2020). One of the classical and popular tasks using limit order
books is the short-term price forecasting task, which is studied extensively
in the academic literature and is valuable from the commercial perspective for
markets including equities, currencies and commodities.

Limit order book data come in different degrees of granularity, including
_Level-1_ data providing the best bid/ask prices and volumes, _Level-2_ data
providing the same data across a certain number of price levels and _Level-3_
data containing the non-aggregated orders placed by market participants. This
information is captured and represented in vectors or matrices before they are
fed into machine learning models. In our work, we focus on LOB data
representations to be used as input signals to machine learning models. As we
mentioned, a vector/matrix organisation of the raw limit order book
information is necessary for upcoming machine learning processes. This
information organisation scheme leads to the initial representation, which is
the foundation of model performance.

Sometimes, this transformation from raw data to feature vectors is referred to
as _feature engineering_ , if feature extraction techniques are applied
manually to the raw data. This requires a good and comprehensive domain
knowledge to make sure the extracted features match the learning task. By
contrast, _representation learning_ , also called _feature learning_ , is an
automated approach to discover an optimal representation for the data. The
major difference between feature engineering and representation learning is
whether the representation is formed in a purely data-driven way. Also, it is
common for a machine learning system to involve both feature engineering and
representation learning with multiple levels of representation appearing at
different stage of processing (see figure 1 (B)).

The performance of machine learning models is heavily influenced by the data
representation scheme (Bengio et al., 2013). For neural networks, the
representation learning and the predictor are combined within the network
structure and are trained together towards the same target function. In this
case, the original representation of LOB, i.e. the input representation to
neural networks, becomes the foundation of the entire model. Presently, the
price level-based data representation scheme is used in almost all recent
studies (Tsantekidis et al., 2017b, a; Tran et al., 2018; Zhang et al., 2019;
Mahfouz et al., 2019; Sirignano, 2019; Tsantekidis et al., 2020; Wallbridge,
2020) applying machine learning including deep learning models on LOB data.
However, this representation scheme is rarely discussed or investigated
towards its compatibility with machine learning especially deep learning
models and robustness under unexpected situations. This lack of investigation
would lead to potential risks under adversarial perturbations. We claim that,
the robustness of LOB representation is the foundation to all LOB-related
machine learning tasks and in this paper, we propose an insight to challenge
the commonly-used level-based LOB representation for machine learning models.

#### Our contributions

This paper is, to our knowledge, the first work bringing adversarial
robustness to LOB representations and pointing out the critical flaws of the
commonly-used LOB representation with machine learning models. It proposes a
perturbation paradigm to the LOB to examine the robustness of data
representation in price forecasting tasks. The experimental results confirm
our concerns about the current level-based LOB representation as well as
machine learning models designed based on this representation scheme. Based on
these concerns, this paper presents desiderata of LOB representations and
proposes new schemes to represent LOBs, which lead to more effective and
robust machine learning models.

##  2\. Representing Limit Order Books

Figure 2. (A) Original LOB data with 10 levels on ask and bid side without
perturbation. (B) LOB data with 10 levels after data perturbation. Red blocks
represent intentionally placed perturbation orders with order volume = 1.
Compared with the original one, the new 10-level data representation has a
much narrower vision on the market.

###  2.1. Drawbacks of Current Representation

The current LOB representation is a time-series of multiple levels of orders
in LOBs, which is commonly-used as inputs to most machine learning models for
LOB predictions and financial benchmark datasets. Each input data point is in
the format of $\vec{x}\in\mathbb{R}^{T\times 4L}$. Temporally, a history of
$T$ LOB snapshots is stacked to reflect the evolution of the market based on
events such as order placement, cancellation, execution etc. Spatially, a LOB
snapshot in this level-based representation is a vector
$x_{t}=\left\\{p_{a}^{i}(t),v_{a}^{i}(t),p_{b}^{i}(t),v_{b}^{i}(t)\right\\}_{i=1}^{L}$
containing $L$ price levels in each side of the LOB. $p_{a}^{i}(t)$,
$p_{b}^{i}(t)$ are the ask and bid prices for price level $i$ and
$v_{a}^{i}(t)$, $v_{b}^{i}(t)$ are the ask and bid volumes respectively.
Reflected in the data format, $4L$ is the length of the vector representing
each snapshot of LOB at a certain time point.

From the data perspective, this representation has some particular
characteristics. The most intuitive one is that the price and volume for each
LOB level are tied together - any disentanglement or distortion to this would
result in invalid representations. In addition, the spatial structure across
different levels is not homogeneous since there is no assumption for adjacent
price levels to have fixed intervals. We also realise the instability and non-
smoothness of the representation due to occasional shifts in price levels -
the previous best bid/ask data can suddenly shift to second best bid/ask
channel if a new order is placed with a better price.

These characteristics may not be a big problem for human understanding but
they explain the drawbacks of this LOB representation when treated as inputs
to machine learning models. Firstly, it is fundamentally assumed in machine
learning that signals from the same channel (input dimension) are from the
same source. In this case, ‘level’ is a manually defined concept based on the
price ranks of current orders in LOB, which can hardly be treated as the same
source especially when the information of a level shifts to the channel of
another level due to certain actions. Secondly, homogeneous relationship is a
basic assumption for convolutional neural networks (CNN) due to the parameter
sharing mechanism. Thus, the heterogeneous features of LOB data representation
may reduce model robustness when learning with CNN models. Furthermore, the
way how the information is organised from multiple levels makes it vulnerable
to perturbations - a small perturbation would leads to shift of price levels
and thus the representation would be affected dramatically due to this shift.

###  2.2. Risks under Adversarial Perturbations

Adversarial perturbations are common and inevitable in most real-world
systems, which can happen in any component of the data processing chain.
Perturbations can be very subtle but still significantly harmful to delicate
systems. If such systems are built without considering risks under possible
perturbations during the model design and development stage, it would lead to
instability and vulnerability after deployment when facing real-world
conditions.

In the financial domain, it is always important to ensure that the potential
risks are noticed and controlled to ensure the systematic reliability. One way
to examine this is by designing hypothetical conditions which might not have
been seen before but can possibly happen in the future. Especially, when we
have a relatively good understanding of the drawbacks of a certain system
based on its design, we can develop scenarios with _adversarial perturbations_
to better identify risks. Unlike adversarial samples, which are targeted
specifically on individual samples, adversarial perturbations are considered
as a general condition that can be applied to all inputs in the same manner.

We present the perturbations by assuming that the data is perturbed by small
size orders at empty price levels beyond the best ask/bid prices. This
perturbation ensures no change is made to the mid-price before and after
perturbation to make sure the prediction labels are not affected. In some LOB
data for equities, the price difference between adjacent price levels is
sometimes larger than the tick size (the minimum price increment change
allowed). This is especially prevalent in small-tick stocks and can result in
the entire LOB shifting even if a small order of the minimum allowable size is
placed at a price in between the existing price levels.

We illustrate this data perturbation with a synthetic LOB example as shown in
Fig. 2. Fig. 2 (A) shows the synthetic LOB snapshot with 10 price levels in
both ask and bid sides of the LOB (marked as L1-L10) before any perturbation.
We assume the tick size is 0.01 and the minimum order size present in our data
is 1. In this LOB snapshot, the mid-price is 10.00 with bid-ask spread equal
to 0.04. We can observe some price levels where no orders are placed, such as
10.03, 10.06 in the ask side and 9.96, 9.94 in the bid side. To perturb this
LOB data, one can place orders with allowed minimum order size to fill these
empty price levels. These minimum size orders may seem to be not influential
since 1) they do not effect the mid price, 2) their volumes are tiny. However,
the LOB representation changed dramatically after this perturbation (see Fig.
2 (B)). Approximately half of the original price level information is no
longer visible after perturbation (e.g. ask-side L5 to L10 information is not
included in representation after perturbation) and while the rest are
preserved, they are shifted to different levels in the LOB representation
(e.g., the ask-side L2 appears in ask-side L3 after perturbation).

Intuitively, this perturbation has two impacts from the machine learning point
of view. Firstly, it shifts the 40-dimensional input space dramatically. For
example, the Euclidean distance between these two 40-dimensional vectors
before and after perturbation is 344.623 whereas actually the total volume of
orders applied is only 10. This means that the level-based representation
scheme does not bring local smoothness. Furthermore, it narrows the scope of
vision of machine learning models to ‘observe’ the market. As shown in the LOB
data visualisation plot in Fig. 2, the gray areas are masked out for the model
input after perturbation.

We assume that, such perturbation with tiny amount of orders would ideally
have limited impact to the future price movement trend. However, risks under
perturbations to be even dilated when limit order book data is presented in
the current way (level-based). In the following sections, we are going to
prove this with experimental results.

###  2.3. Desiderata of Robust Representation

As we mentioned, the performance of machine learning models relies hugely on
representations, either learned via representation learning or designed via
feature engineering. We would like to propose some desiderata for improving
the robustness of LOB-related data representations and machine learning models
designed on top of certain representations. These desiderata come from two
perspectives. The first perspective is from the information transformation,
focusing on whether the representation is a meaningful, comprehensive and
efficient way to reflect the original information. The second perspective is
from the machine learning point of view, concerning about whether this
representation is compatible and appropriate for the machine learning model to
be used in real tasks. Note that, data can be represented differently in
storage, transition or analysis and our desiderata only applies on the
representation directly fed to machine learning models as inputs.

  * •

Region of interests The entire limit order book may contain hundreds of price
levels with a large range of price. A complete representation including all
price levels leads is not always necessary for all the tasks. Thus, an
appropriate region of interests needs to be placed to the limit order book to
reach a balance between complexity and performance.

  * •

Efficiency: A LOB representation should organise data in a efficient manner to
reduce the _curse of dimensionality_. For example, limit order books snapshot
can be represented with extremely sparse vector including all price ticks
appeared in the history. This kind of representation is complete, easy to
understand but very inefficient both in storage and in computation.

  * •

Validity: A LOB data representation should include a clear and simple
definition of validity.

  * •

Smoothness: A LOB Data representation should not change dramatically under
subtle perturbations in the market.

  * •

Compatibility: Basic assumptions needs to be matched between data
representations and learning models. If not, these models may contain unknown
risks due to invalid fundamental settings. For example, convolutional neural
networks (CNN) assumes homogeneous spatial (or temporal depending on the
convolution direction) relationship due to its parameter sharing mechanism.
Thus, if the input representation does not match the assumption of
homogeneosity, the learned shared features may suffer from the risk of being
invalid or non-meaningful.

##  3\. Spatial-temporal Representation in mid-price-centred Moving Windows

We propose here to represent limit order books with fixed size moving windows
centred at the mid-price of the current time point, which is referred to as
the moving window representation (MW). For predicting short-term price
movement, limit orders near the mid-price plays an important role compared
with orders placed far away from the mid-price. Thus, we are more concern
about limit order information near the mid-price. We set a 2-dimensional
window for the region of interests containing $N$ LOB histories and $2W+1$
continuous price levels stepped by the tick size $\Delta p$. This window
provides a vision for limit orders within price range $[p(t)-W\Delta
p,p(t)+W\Delta p]$ and within a short history.

Figure 3. Spatial-temporal Representation in mid-price-centred Moving Windows.
Red/blue represents ask/bid volumes. (A) Moving window representation. (B)
Accumulated moving window representation. (C) Smoothed moving window
representation.

In the two dimensional matrix $x\in\mathbb{R}^{N\times 2W+1}$, each element
$x_{n,i},n=\\{1,...,N\\},i=\\{0,...,2W\\}$ of the moving window representation
indicates the volume of limit orders at price $p(t)-W\Delta p+i$ and at LOB
snapshot $t-N+n$. To distinguish from ask and bid orders, we mark $x_{n,i}>0$
for ask-side limit orders and $x_{n,i}<0$ for bid side limit orders and the
volume size is given by $|x_{n,i}|$. Fig. 3 (A) visualises this moving window
representation of a example in real FI-2010 LOB dataset (N = 40 and W = 20).
This representation is re-organisation of limit order book information like
level-based representation scheme but avoids some of its drawbacks. First, all
numerical numbers in this representation are volumes instead of volume-price
couples, which avoids the risks of invalidity if disentanglement happens in
future black-box models. This representation is also spatially homogeneous
since distances between adjacent elements (spatially) all equals to the tick
size. Thus, it doesn’t have the issue of level shifts when empty ticks are
filled. Because it includes all the empty ticks within the vision scope, this
representation may look sparse, i.e. with a considerable amount of zero
elements in the representation.

We can analyse the moving window representation based on the desiderata we
proposed before. We choose the regions of interests around the mid-price
instead of considering a wide range of price levels appeared in the history.
This is a compromise of representational comprehensiveness and efficiency.
With such compromise, our representation shows similar space requirement as
the commonly-used representation with 10 price levels at each side. The
determination of the region of interests can be depending on the tick-size,
stock price, accuracy requirement, computational resources etc. of specific
tasks. By encoding price information implicitly in element location and volume
information explicit in element value, we now have homogeneosity in this
moving window representation and should be compatible with majority of machine
learning models including CNNs. Furthermore, the advantage of this
disentanglement is obvious - each change to the limit order book would be
reflected reasonably in the representation without level shifts and similar
LOBs would have similar representations.

Based on this moving window representation, we introduce two variations - the
accumulated moving window representation (accumulated MW), and the smoothed
moving window representation (smooth MW). The accumulated moving window
representation is a equivalent variation of the moving window representation,
which can be easily transformed mutually. Each element
$x_{n,i},n=\\{1,...,N\\},i=\\{0,...,2W\\}$ in the accumulated moving window
representation is the sum of total volumes up to the corresponding price level
on each side in the n-th snapshot. In financial area, this accumulation of
limit orders refers to the _market depth_ , which considers the entire levels
of the limit order book. The market depth reflects the market’s capability to
absorb market orders without having the price being affected dramatically by
large-scale orders. This accumulated volume representation is a time series
version of market depth information and aligned regarding to the current mid-
price (see Fig. 1 (B)). The smoothed moving window representation is a
processed version of the moving window representation using Gaussian kernels.
Via Gaussian kernels, we approximately create a potential distribution of
limit orders across different prices and make the matrix less sparse. Fig. 3
(C) illustrates an example of smoothed volume representation which is edited
based on Fig. 3 (A).

##  4\. Related Works

Early works using high frequency limit order book data usually use
handcrafting features from the LOB combined with naive learning models like
support vector machines (SVMs) to indicate future price movement(Kercheval and
Zhang, 2015). As deep learning methods start to show their power in various
other areas like computer vision, researches also seek for deeper
architectures to solve large-scale problems in the financial area - multi-
layer perceptrons (Mahfouz et al., 2019), recurrent neural networks (Sirignano
and Cont, 2019), long short-term memory (Tsantekidis et al., 2017b),
convolutional neural networks (Zhang et al., 2019, 2018) self-attention
transformer networks (Wallbridge, 2020) and recently Seq2Seq networks(Zhang
and Zohren, 2021).

Although deep learning is gaining attention in the financial area, it is
rarely discussed in literature about representation schemes of financial data.
A vast majority of (even not all) the machine learning models, including those
we mentioned above, and benchmark datasets arrange LOB in the level-based
representation (e.g. (Ntakaris et al., 2018; Huang and Polak, 2011)). As we
emphasize in previous sections, a LOB representation which is efficient and
convenient from the perspective of human understanding and matching engine
does not necessarily mean it is an appropriate representation scheme for
machine learning models to learn features from. The study of the importance of
robust data representation, the criteria for evaluating the quality of the
representation, and the variety of methods for learning these representations
is studied extensively in the machine learning literature with (Bengio et al.,
2013) providing a survey of these methods.

In our work, we focus on the representation of financial market microstructure
data. (Bouchaud et al., 2018), (Abergel et al., 2016) study the structure and
empirical properties of limit order books and provide a set of statistical
properties (referred to as stylized facts) using NASDAQ exchange data. On the
other hand, (Lehalle and Laruelle, 2018) discusses the practical aspects and
issues of market structure, design, price formation, discovery and the
behaviour of different actors in limit order book markets. A significant
amount of research in recent years focused on applying deep learning models on
limit order book data for the purposes of price forecasting or price movements
classification.

##  5\. Experimental Design

###  5.1. Benchmark Dataset and Model Inputs

We use the FI-2010 dataset (Ntakaris et al., 2018) as the benchmark dataset,
which consists of LOB data from 5 stocks in the Helsinki Stock Exchange during
normal trading hours (no auctions) for 10 trading days. This dataset takes
into account 10 price levels on the bid/ask sides of the limit order book,
which are updated according to events such as order placement, executions and
cancellations. In our experiments, we take into account the history of the LOB
snapshots for future price movement prediction. Thus, each input data point is
a short time series with input dimension $T\times 40$ where T is the total
amount of historical snapshots. The prediction target is the micro-movement
$l_{t}=\frac{m_{+}(t)-p_{t}}{p_{t}}$ where
$m_{+}(t)=\frac{1}{k}\sum^{k}_{i=1}p_{t+i}$ is the smoothed mid-price with
prediction horizon $k$. The movement is further categorised into three classes
- 1:up ($l_{t}>0.002$), 2:stationary ($-0.002\leq l_{t}\leq 0.002$), 3:up
($l_{t}<-0.002$). We choose the FI-2010 prediction labels with prediction
horizon k-50 as the targets for model training and testing. Notice that, the
FI-2010 dataset provides inputs and outputs already being pre-processed with
the approaches mentioned above.

We consider 4 different representations - the level-based representations
which is commonly used in the literature (Level-based), moving window (MW),
Accumulated moving window (Accumulated MV), Smoothed moving window (Smoothed
MW). All these representation schemes encode exactly the same information -
spatially, price and volume of 10 price levels on each side and temporally, 10
most recent LOB snapshots as market history. For level-based representations,
the input data dimension is $10$ and for moving window representations, the
input data dimensions are $10\times 41$.

We additionally generate LOB testing datasets with adversarial perturbations
by adding orders to limit order books. All 4 input representations now need to
represent each perturbed LOB (original LOB + order perturbation) in their own
manner. We design 4 perturbation paradigms - the LOB data is not perturbed
(‘None’) and the LOB data is perturbed by placing minimum-size orders to fill
the ticks on the ask side only (‘ask side’), on the bid side only (‘bid
side’), on both the ask and bid side (‘both sides’). The idea of adversarial
perturbations is to examine the model’s robustness under unexpected subtle
perturbations. Robust models should show good generalisation against subtle
perturbations and their prediction should not be influenced dramatically even
they never seen exact examples like the perturbed ones before. Thus, all these
methods are trained with the same FI-2010 training dataset (unperturbed) as
how people usually train these models in production but only being tested in
perturbation paradigms mimic unexpected adversarial perturbations or even
attacks.

###  5.2. Price Forecasting Models

In this paper, we apply 5 machine learning models for price forecasting tasks,
including 3 basic benchmark models, i.e. linear (logistic regression), multi-
layer perceptron (MLP), long short-term memory (LSTM), and 2 deep learning
models, DeepLOB (Zhang et al., 2019) and temporal convolutional networks (TCNs
(Bai et al., 2018)). The baseline linear model is demonstrated as a multi-
class logistic regression. It includes a linear layer coming up with a softmax
activation evaluating the probability of multiple categories. Multilayer
perceptrons are generic solutions to machine learning problems without
requiring prior knowledge about the spatial-temporal structure of the data.
The multi-layer perceptron model we apply has two hidden layers with 100 and
50 neurons respectively with ReLU activation. LSTMs are recurrent neural
networks focusing on capturing temporal dynamics of the data. We apply a
shallow LSTM model in this paper with only 1 LSTM layer consisting of 20
units. The DeepLOB model is a deep learning solution for price forecasting
using LOBs. It combines convolutional neural networks with LSTM and is
designed specifically based on the level-based inputs. It’s hidden layers
consists of 3 convolutional layers, 1 inception module and 1 LSTM layer in a
sequence, aiming to capture both temporal dynamics and spatial structure of
LOB data. Since DeepLOB is not compatible with other LOB representations, we
apply temporal convolutional networks to our moving window representations to
demonstrate their performance under deep learning settings. TCNs, which
capture the spatial-temporal information, are CNN alternatives of conventional
RNNs when solving time series problems. The most important structure within
TCNs are causal convolutional layers .In this implementation, we stack 3
causal layers (32 channels each) to create a relatively deep but not over-
complicated model.

##  6\. Results

Perturbation | Level-based (%) | MW (%) | Accumulated MW (%) | Smoothed MW (%)
---|---|---|---|---
Accuracy | F-score | Accuracy | F-score | Accuracy | F-score | Accuracy | F-score
| Linear
None |  52.98$\pm$0.14 |  38.50$\pm$0.38 | 59.57$\pm$0.00 | 53.66$\pm$0.01 |  54.24$\pm$0.00 |  41.12$\pm$0.01 |  58.85$\pm$0.00 |  52.66$\pm$0.00
Ask side |  49.81$\pm$0.09 |  42.47$\pm$0.25 | 59.57$\pm$0.00 | 53.66$\pm$0.00 |  54.23$\pm$0.00 |  41.09$\pm$0.00 |  58.85$\pm$0.00 |  52.65$\pm$0.00
Bid side |  51.65$\pm$0.10 |  35.83$\pm$0.16 | 59.57$\pm$0.00 | 53.66$\pm$0.01 |  54.24$\pm$0.01 |  41.12$\pm$0.01 |  58.85$\pm$0.00 |  52.66$\pm$0.00
Both sides |  49.40$\pm$0.14 |  41.54$\pm$0.04 | 59.57$\pm$0.00 | 53.66$\pm$0.00 |  54.24$\pm$0.00 |  41.10$\pm$0.00 |  58.85$\pm$0.00 |  52.66$\pm$0.00
| MLP
None |  60.14$\pm$4.73 |  53.96$\pm$8.41 |  69.52$\pm$0.68 |  67.88$\pm$0.65 | 71.27$\pm$0.36 | 69.59$\pm$0.35 |  65.59$\pm$0.30 |  63.69$\pm$0.32
Ask side |  55.75$\pm$2.30 |  47.08$\pm$4.71 |  69.52$\pm$0.68 |  67.88$\pm$0.65 | 71.27$\pm$0.36 | 69.59$\pm$0.35 |  65.59$\pm$0.30 |  63.69$\pm$0.32
Bid side |  55.05$\pm$4.08 |  45.74$\pm$8.01 |  69.54$\pm$0.68 |  67.90$\pm$0.65 | 71.28$\pm$0.36 | 69.59$\pm$0.35 |  65.60$\pm$0.30 |  63.70$\pm$0.32
Both sides |  50.26$\pm$0.89 |  38.88$\pm$5.11 |  69.53$\pm$0.68 |  67.89$\pm$0.65 | 71.28$\pm$0.36 | 69.60$\pm$0.35 |  65.59$\pm$0.30 |  63.69$\pm$0.32
| LSTM
None |  70.74$\pm$0.22 |  68.45$\pm$0.33 |  75.15$\pm$0.40 |  73.89$\pm$0.38 | 77.46$\pm$0.17 | 76.18$\pm$0.19 |  65.90$\pm$0.41 |  64.44$\pm$0.38
Ask side |  65.85$\pm$1.43 |  63.09$\pm$1.16 |  75.14$\pm$0.40 |  73.89$\pm$0.38 | 77.46$\pm$0.17 | 76.18$\pm$0.19 |  65.90$\pm$0.41 |  64.44$\pm$0.38
Bid side |  63.33$\pm$1.58 |  60.39$\pm$1.78 |  75.16$\pm$0.40 |  73.88$\pm$0.38 | 77.47$\pm$0.17 | 76.19$\pm$0.19 |  65.90$\pm$0.41 |  64.44$\pm$0.38
Both sides |  57.79$\pm$3.14 |  54.35$\pm$2.74 |  75.15$\pm$0.40 |  73.89$\pm$0.38 | 77.47$\pm$0.17 | 76.19$\pm$0.19 |  65.90$\pm$0.41 |  64.44$\pm$0.38
|  DeepLOB (Zhang et al., 2019)
None |  77.30$\pm$0.08 |  77.23$\pm$0.10 | / | / | / | / | / | /
Ask side |  69.02$\pm$0.33 |  67.42$\pm$0.25 | / | / | / | / | / | /
Bid side |  63.94$\pm$0.93 |  60.70$\pm$1.33 | / | / | / | / | / | /
Both sides |  51.35$\pm$1.74 |  39.59$\pm$3.73 | / | / | / | / | / | /
| TCN
None | / | / |  77.83$\pm$0.33 |  76.47$\pm$0.34 | 78.81$\pm$0.38 | 77.29$\pm$0.47 |  65.73$\pm$0.34 |  63.63$\pm$0.46
Ask | / | / |  77.82$\pm$0.32 |  76.21$\pm$0.34 | 78.80$\pm$0.38 | 77.27$\pm$0.47 |  65.72$\pm$0.34 |  63.63$\pm$0.46
Bid | / | / |  77.83$\pm$0.33 |  76.22$\pm$0.35 | 78.82$\pm$0.38 | 77.30$\pm$0.46 |  65.72$\pm$0.34 |  63.63$\pm$0.46
Both sides | / | / |  77.83$\pm$0.33 |  76.21$\pm$0.34 | 78.80$\pm$0.38 | 77.28$\pm$0.46 |  65.72$\pm$0.34 |  63.63$\pm$0.46
Table 1. Price forecasting model performance under data perturbation. Each
model is trained with a non-perturbed training set and when testing the model,
we apply various data perturbation. None: no perturbation. Ask-side:
perturbation only applied to the ask-side of data. Bid-side: perturbation only
applied to the bid-side of the data. Both sides: perturbation applied to both
ask and bid sides.

###  6.1. Price Forecasting Model Performance

Table 1 demonstrates the testing performance of these machine learning models
in the price movement forecasting tasks. Since the testing set is unbalanced,
we use 4 different metrics (scores) to evaluate and compare the performance -
Accuracy (%), Precision (%), Recall (%) and F-score (%). Among these metrics,
Accuracy (%) is measured as the percentage of predictions of the test samples
exactly matches the ground truth, which is the unbalanced accuracy score where
as the rest metrics are all averaged across classes in an unweighted manner to
eliminate the influence of data imbalance. All the experiments are repeated 5
times with different random seeds to get an averaged performance with standard
deviations.

It can be observed from Table. 1 that the ranking of model performance
regardless of the input representation or testing paradigm - deep models
(DeepLOB, TCN) -> LSTM -> MLP -> Linear. We take the model performance using
level-based representation as an instance. The linear model (accuracy =
52.98%, F-score = 38.50%) is not capable to learn complex features either
spatially or temporally due to its simplicity. The MLP model with multiple
hidden layers in theory can learn arbitrary features. However, the feature
extraction in MLP are not that effective under limited parameter capacity due
to the lack of explicitly defined data structure. Thus, the MLP model
performance (accuracy = 60.14%, F-score = 53.96%) is much lower than LSTM
(accuracy = 60.14%, F-score = 53.96%), which has clear definition over the
temporal structure during learning. The DeepLOB model builds an additional
convolutional architecture on top of a LSTM to enable both spatial and
temporal feature extraction and significantly outperforms those relatively
simple models (Linear, MLP, LSTM) with accuracy = 77.30%, F-score = 77.23%.
Similarly, the TCN model also shows leading performance over other comparing
methods.

In Table. 1, we can compare the performance horizontally with different input
representations. By just replacing the level-based representation with moving
windows, the forecasting performance of the same model is boosted by 7% for
Linear (Level-based v.s. MW), 11% for MLP (Level-based v.s. Accumulated MW)
and 7% for LSTM (Level-based v.s. Accumulated MW). Especially, LSTM with the
accumulated MW representation can already reach an approximate level of
performance as the much more complex deepLOB model. This means that both MW
and Accumulated MW are more effective representations than the commonly-used
level-based one for machine learning models to learn meaningful features for
forecasting future price movements. In general, accumulated MW demonstrates
the best performance among all the representation schemes. This proves from
the data-driven perspective that the market depth is important information for
understanding the market.

###  6.2. Robustness under Perturbations

For the level-based representation, we observe a performance decay of all the
machine learning models under unexpected perturbations, from the simplest
linear model to the most sophisticated DeepLOB model. In addition, the
performance decline alters under different types of perturbations. When the
perturbation is applied to both sides, the performance decrease becomes more
severe - 11% accuracy decrease on MLP, 12% on LSTM and over 25% on DeepLOB.
Similar trends can also be viewed for other evaluation metrics. By contrast,
our moving window representations show no obvious performance decay under
perturbations in all experiments with different models.

From the these performance decay results, we find that DeepLOB, the best
performed model under normal condition as well as the most complicated one, is
also the most vulnerable one under perturbation (the largest performance
decay). Its predictive accuracy decreases to 47.5% and the F-score is only
22.2%, which even underperforms logistic regression. The reason behind this
phenomenon may be a combination of various factors. On one hand, the
complexity of model is related to overfitting, which may reduce the
generalisation ability and become unstable under the perturbation. Also as we
mentioned in earlier sections, CNN assumes homogeneous spatial relationship
but the level-based LOB representation is obviously heterogeneous, which leads
to a mismatch between the data representation and the network characteristics.
Once the spatial relationship is further broken due to perturbation, the CNN
descriptors may not be able to extract meaningful features and thus cause
malfunction of the entire predictor.

Figure 4. Confusion matrices for corresponding experimental results of the
level-based representation in Table. 1

Fig. 4 further illustrates more details behind the numerical performance
metrics in the form of a confusion matrix about the performance decay with the
level-based representation. The logistic regression model basically classify a
majority of samples as ‘Stationary’ no matter whether perturbation is applied.
Similarly in MLP, about half of the ‘Up’ and ‘Down’ samples are misclassified
as ‘Stationary’ ones. Both LSTM and DeepLOB shows confusion matrices with
obvious diagonal feature without perturbation - more than half of the samples
from each class are classified the same as their true labels. When the
perturbation is applied, LSTM shows performance decrease but the still near
half of samples are correctly classified. DeepLOB, however, fails in the
perturbation condition by misclassifying almost all the data to ‘Stationary’
class (see DeepLOB+Both in Fig. 4).

##  7\. Conclusion & Future Work

In this paper, we discussed the importance of data representations to machine
learning models applied to LOB-related tasks and highlighted the drawbacks and
risks when using non-robust representations. Further, we proposed new
representation schemes that avoid these drawbacks. To reveal this, we
implemented price forecasting tasks with multiple benchmark models and data
representations. We also designed data perturbation scenarios to test not only
the performance but also the robustness of these machine learning models with
various representation schemes including the commonly-used level-based
representation and our moving window representations.

We show that how the information is organised and represented as the input has
large impact to the model performance. In our case, by replacing the level-
based representation with our moving window representations, performance of
the same model increases significantly. This reveals that our moving window
representations are more effective and suitable for machine learning models.
In addition, the level-based representation brings vulnerability to models
even under subtle perturbations, which leads to significant performance decay
especially when models are more sophisticated. Our moving window
representations, on the contrary, are almost immune to this perturbation and
thus are more stable and reliable.

Like previous literature, we also show that machine learning models especially
deep learning models can be a promising solution to financial problems.
Especially, we can adopt existing machine learning solutions (e.g. TCNs) which
was designed to solve similar problems in other areas to solve financial
problems. The TCN structure we used in this paper was originally designed for
video-based segmentation but matches nicely with price forecasting problem
here and delivers satisfying results. Our future work would focus on extending
representation-related robustness to more tasks dimensionality reduction,
reinforcement learning etc.

###### Acknowledgements.

The authors would like to acknowledge our colleagues Vacslav Gluckov, Jeremy
Turiel, Rui Silva and Thomas Spooner and for their input and suggestions at
various key stages of the research. This paper was prepared for informational
purposes in part by the Artificial Intelligence Research group of JPMorgan
Chase & Co and its affiliates ("J.P. Morgan"), and is not a product of the
Research Department of J.P. Morgan. J.P. Morgan makes no representation and
warranty whatsoever and disclaims all liability, for the completeness,
accuracy or reliability of the information contained herein. This document is
not intended as investment research or investment advice, or a recommendation,
offer or solicitation for the purchase or sale of any security, financial
instrument, financial product or service, or to be used in any way for
evaluating the merits of participating in any transaction, and shall not
constitute a solicitation under any jurisdiction or to any person, if such
solicitation under such jurisdiction or to such person would be unlawful.

## References

  * (1)
  * Abergel et al. (2016) Frédéric Abergel, Marouane Anane, Anirban Chakraborti, Aymen Jedidi, and Ioane Muni Toke. 2016\.  _Limit order books_.  Cambridge University Press.
  * Bai et al. (2018) Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018\.  An empirical evaluation of generic convolutional and recurrent networks for sequence modeling.  _arXiv preprint arXiv:1803.01271_ (2018).
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013\.  Representation learning: A review and new perspectives.  _IEEE transactions on pattern analysis and machine intelligence_ 35, 8 (2013), 1798–1828.
  * Bouchaud et al. (2018) Jean-Philippe Bouchaud, Julius Bonart, Jonathan Donier, and Martin Gould. 2018.  _Trades, quotes and prices: financial markets under the microscope_.  Cambridge University Press.
  * Buehler et al. (2019) Hans Buehler, Lukas Gonon, Josef Teichmann, and Ben Wood. 2019\.  Deep hedging.  _Quantitative Finance_ 19, 8 (2019), 1271–1291.
  * Huang and Polak (2011) Ruihong Huang and Tomas Polak. 2011.  Lobster: Limit order book reconstruction system.  _Available at SSRN 1977207_ (2011).
  * Kercheval and Zhang (2015) Alec N Kercheval and Yuan Zhang. 2015.  Modelling high-frequency limit order book dynamics with support vector machines.  _Quantitative Finance_ 15, 8 (2015), 1315–1329.
  * Lehalle and Laruelle (2018) Charles-Albert Lehalle and Sophie Laruelle. 2018.  _Market microstructure in practice_.  World Scientific.
  * Mahfouz et al. (2019) Mahmoud Mahfouz, Angelos Filos, Cyrine Chtourou, Joshua Lockhart, Samuel Assefa, Manuela Veloso, Danilo Mandic, and Tucker Balch. 2019\.  On the Importance of Opponent Modeling in Auction Markets.  _arXiv preprint arXiv:1911.12816_ (2019).
  * Ntakaris et al. (2018) Adamantios Ntakaris, Martin Magris, Juho Kanniainen, Moncef Gabbouj, and Alexandros Iosifidis. 2018.  Benchmark dataset for mid-price forecasting of limit order book data with machine learning methods.  _Journal of Forecasting_ 37, 8 (2018), 852–866.
  * Sirignano and Cont (2019) Justin Sirignano and Rama Cont. 2019.  Universal features of price formation in financial markets: perspectives from deep learning.  _Quantitative Finance_ 19, 9 (2019), 1449–1459.
  * Sirignano (2019) Justin A Sirignano. 2019\.  Deep learning for limit order books.  _Quantitative Finance_ 19, 4 (2019), 549–570.
  * Tran et al. (2018) Dat Thanh Tran, Alexandros Iosifidis, Juho Kanniainen, and Moncef Gabbouj. 2018.  Temporal attention-augmented bilinear network for financial time-series data analysis.  _IEEE transactions on neural networks and learning systems_ 30, 5 (2018), 1407–1418.
  * Tsantekidis et al. (2017a) Avraam Tsantekidis, Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, and Alexandros Iosifidis. 2017a.  Forecasting stock prices from the limit order book using convolutional neural networks. In _2017 IEEE 19th Conference on Business Informatics (CBI)_ , Vol. 1. IEEE, 7–12.
  * Tsantekidis et al. (2017b) Avraam Tsantekidis, Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, and Alexandros Iosifidis. 2017b.  Using deep learning to detect price change indications in financial markets. In _2017 25th European Signal Processing Conference (EUSIPCO)_. IEEE, 2511–2515.
  * Tsantekidis et al. (2020) Avraam Tsantekidis, Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, and Alexandros Iosifidis. 2020.  Using deep learning for price prediction by exploiting stationary limit order book features.  _Applied Soft Computing_ 93 (2020), 106401.
  * Wallbridge (2020) James Wallbridge. 2020\.  Transformers for limit order books.  _arXiv preprint arXiv:2003.00130_ (2020).
  * Wiese et al. (2020) Magnus Wiese, Robert Knobloch, Ralf Korn, and Peter Kretschmer. 2020.  Quant gans: Deep generation of financial time series.  _Quantitative Finance_ 20, 9 (2020), 1419–1440.
  * Zhang and Zohren (2021) Zihao Zhang and Stefan Zohren. 2021.  Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units.  _arXiv preprint arXiv:2105.10430_ (2021).
  * Zhang et al. (2018) Zihao Zhang, Stefan Zohren, and Stephen Roberts. 2018\.  Bdlob: Bayesian deep convolutional neural networks for limit order books.  _arXiv preprint arXiv:1811.10041_ (2018).
  * Zhang et al. (2019) Zihao Zhang, Stefan Zohren, and Stephen Roberts. 2019\.  Deeplob: Deep convolutional neural networks for limit order books.  _IEEE Transactions on Signal Processing_ 67, 11 (2019), 3001–3012.