# Deep Learning in EEG: Advance of the Last Ten-Year Critical Period

Shu Gong, Kaibo Xing , Andrzej Cichocki , and Junhua Li S. Gong and K. Xing
are with the College of Life Sciences, Sichuan University, Chengdu 610065,
China.A. Cichocki is with the Skolkovo Institute of Science and Technology
(Skoltech), Moscow, 121205, Russia, and also with the Nicolaus Copernicus
University (UMK), Torun, 87-100, Poland.J. Li is with the School of Computer
Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ,
UK, and also with the Laboratory for Brain-Bionic Intelligence and
Computational Neuroscience, Wuyi University, Jiangmen 529020, China.This work
was supported by the National Natural Science Foundation of China (No.
61806149), the Guangdong Basic and Applied Basic Research Foundation (No.
2020A1515010991), the Ministry of Education and Science of the Russian
Federation (No. 14.756.31.0001), and Polish National Science Center (No.
UMO-2016/20/W/NZ4/00354). Corresponding author: Junhua Li (e-mail:
<EMAIL_ADDRESS>
Abstract:

Deep learning has achieved excellent performance in a wide range of domains,
especially in speech recognition and computer vision. Relatively less work has
been done for EEG, but there is still significant progress attained in the
last decade. Due to the lack of a comprehensive survey for deep learning in
EEG, we attempt to summarize recent progress to provide an overview, as well
as perspectives for future developments. We first briefly mention the
artifacts removal for EEG signal and then introduce deep learning models that
have been utilized in EEG processing and classification. Subsequently, the
applications of deep learning in EEG are reviewed by categorizing them into
groups such as brain-computer interface, disease detection, and emotion
recognition. They are followed by the discussion, in which the pros and cons
of deep learning are presented and future directions and challenges for deep
learning in EEG are proposed. We hope that this paper could serve as a summary
of past work for deep learning in EEG and the beginning of further
developments and achievements of EEG studies based on deep learning.

Keywords:

Deep Learning, Electroencephalogram (EEG), Classification, Brain Computer
Interface, Disease, Emotion, Sleep, Mental State

##  1 Introduction

Machine learning technology has benefited to diverse domains in our modern
society [1], [2]. Deep learning, a subcategory of machine learning technology,
has been showing excellent performance in pattern recognition [3],
dramatically improving classification accuracy. It is worth noting that new
world records were created by using deep learning in many competitions such as
ImageNet Competition [4]. Deep learning was successfully applied to speech
recognition[5], computer vision[6], and others[7], which remarkably influenced
our life and might change our lifestyle.

Deep learning was established based on inspiration from the neural mechanism
of hierarchical network existing in visual cortex of human brain. The key word
’deep’ in the term of deep learning represents a deep hierarchical structure
of multiple processing layers, which is specialized in information extraction.
The information extracted becomes more and more abstract through layers.
Compared to conventional machine learning methods, deep learning features
could be extracted and refined through layers of deep learning structure.
However, deep learning suffers from slow convergence and high requirement in
computing resource. These drawbacks have been released by the introduction of
training strategies into the deep learning structure and availability of high-
performance computers, such as dropout[8] and batch normalization[9]. Or, a
pooling layer that is embedded into the deep learning structure to have
feature representation while reducing feature dimension [10]. From the side of
high-performance computer, not only there is improved power in central
processing units, but also new types of computing units such as graphic
processing units and tensor processing units. These new types of computing
units were designed to fit the matrix manipulation, which could greatly reduce
the training time of deep learning model. Moreover, the availability of large
scale of data and increased capacity of data storage contribute to the
development of deep learning because deep learning usually needs more training
data in order to obtain expected performance.

Figure 1: Cumulative numbers of published papers since the year of 2010.

Electroencephalogram (EEG) signal was first recorded by Hans Berger in the
year of 1924 [11], which manifests underlying brain activity. Multiple
electrodes can be set to record EEG signal by placing them on different
locations of the brain scalp and temporal fluctuations in voltage can be
captured in a high resolution (e.g., in millisecond) by setting a high
sampling rate. With the advantages of multi-channel recording and high
temporal resolution, EEG has been applied to numerous domains from brain-
computer interface [12, 13, 14, 15], to emotion [16, 17], to cognition [18],
to brain diseases [19]. The analysis of EEG signal is evolved from simple
methods such as mean and amplitude comparison to complicated methods such as
connectivity topology and deep learning. In particular, deep learning exhibits
better performance in EEG classification (recognition) compared to
conventional methods (e.g., support vector machine). By using deep learning,
discriminative features could be extracted without manual engagement, which
requires specific knowledge and expertise. This could release the problem that
classification performance largely depends on how well features are
handcrafted in the conventional methods. However, deep learning is not the
final best solution as we still need to select suitable model structure and
parameter setting according to a specific classification situation. A good
classification performance usually is not obtained by just feeding data into a
deep learning model. Due to the low amplitude of EEG signal relative to noise
and the characteristic of non-stationarity, it is not easy to mine brain
activity information from EEG signal. Usually, artifacts removal is
implemented to remove artifacts to improve signal-to-noise ratio before
feeding into a deep learning model. To date, different kinds of deep learning
models have been employed to process or classify EEG signal. Cecotti et al.
used convolutional neural network (CNN) to extract features of steady-state
visual evoked potential early in 2008 [20]. Li et al. employed denoising
autoencoder to classify two classes of motor imagery using EEG recorded from
14 electrodes on the sensorimotor cortex [21]. Tsiouris et al. applied
recurrent neural network (RNN) to capture sequential relationships for seizure
detection [22]. A summary of deep learning applications in EEG can be found in
[23].

Significant progress has been done since deep learning applied to EEG signal,
but it is far behind compared to the advance of deep learning in other domains
such as computer vision [24] and speech recognition[25]. It is time to
summarize the achievements of deep learning in EEG for the past 10 years and
discuss current existing issues and future directions. The key words ”deep
learning” AND ”EEG” OR ”classification” OR ”recognition” OR ”identification”
were used for literature retrieval in the Web of Science in March 2020. After
manual selection, 193 papers were included in this survey. As shown in Fig. 1,
the majority of these papers were published after 2015 while there was a rapid
increase in 2019. The topics of brain-computer interface and disease detection
are mostly investigated compared to the others, especially in the years of
2019 and 2020. The remainder of the survey is organized as follows. In Section
II, artifacts removal is briefly introduced. This is followed by the detailed
descriptions of all deep learning models which have been applied to EEG in
Section III. In this section, we also mention the advantages and limitations
of each deep learning model. Subsequently, the applications of deep learning
in EEG are detailed along with publicly available EEG datasets used in these
applications in Section IV. Finally, discussions are given and future
directions are drawn at the end of the survey.

Figure 2: (A) Generic framework of a deep learning model. (B) Classical units
that are employed in a deep learning model.

##  2 Artifacts Removal

In general, artifacts are larger than that we intend to extract from EEG
signal in terms of scale, leading to a low signal-to-noise ratio (SNR). In
order to improve the SNR, EEG signal is preprocessed to remove or mitigate the
effect of artifacts on the signal before the signal is further processed. For
example, a notch filter [18] is effective for eliminating the interference of
power line. Independent component analysis [26] is usually utilized to remove
eye movements-related and muscular activity-related artifacts. Classical
methods of artifacts removal and their targeted artifacts are summarized in
Supplementary Table LABEL:tab_artifacts.

When deep learning emerges, the step of artifacts removal is kept. EEG signal
is preprocessed as usual to remove artifacts before inputting into a deep
learning model. This is an effective way as all artifacts removal methods can
be applied with deep learning models to be of both benefits inherited from the
artifacts removal methods and deep learning models. This is also a natural and
straightforward way that researchers are able to easily implement. However, a
deep learning model consisted of layers enables to remove artifacts by that
features are refined through the layers. A few attempts were made. For
example, Supratak et al. inputted raw EEG data into a CNN for sleep stage
classification without any hand-crafted features. Their study showed that an
acceptable performance can be achieved without a separate step of artifacts
removal [27].

##  3 Deep Learning Models

In this section, we describe each fundamental deep learning model. Their
variants and combinations are not included as they share the similar rationale
with fundamental models. A deep learning model is a hierarchical structure,
comprising layers through which data are mapped into more and more abstract.
Whatever a deep learning model is, there are an input layer, an output layer,
and one or more hidden units (see Fig. 2(A)). The hidden unit might be one of
the layer structures illustrated in Fig. 2(B) or their combinations. In the
following subsections, we introduce classical deep learning models where
typical units illustrated in Fig. 2(B) are embedded.

###  3.1 Restricted Boltzmann Machine and Deep Belief Networks

A restricted Boltzmann machine (RBM) [28] is an undirected graph model (see
Fig. 2(B): RBM Unit), which has a visible layer
$\mathbf{v}=(v_{1},v_{2},\dots,v_{n})$ and a hidden layer
$\mathbf{h}=(h_{1},h_{2},\dots,h_{n})$. Connections exist only between visible
layer $\mathbf{v}$ and hidden layer $\mathbf{h}$ and there are no connections
between nodes within the visible layer or hidden layer. The energy function
for an RBM is defined as:

$E(\mathbf{v},\mathbf{h})=-\mathbf{v}^{T}\mathbf{W}\mathbf{h}-\mathbf{a}^{T}\mathbf{v}-\mathbf{b}^{T}\mathbf{h}$
(1)

where $\mathbf{W}$ is the weight matrix, $\mathbf{a}$ and $\mathbf{b}$ are
bias vectors. The joint probability of $\mathbf{v}$ and $\mathbf{h}$ is
constructed in terms of $E$:

$P(\mathbf{v},\mathbf{h})=\frac{1}{Z}e^{-E(\mathbf{v},\mathbf{h})}$ (2)

where $Z$ is a normalizing constant defined as:

$Z=\sum_{\mathbf{v},\mathbf{h}}e^{-E\left(\mathbf{v},\mathbf{h}\right)}$ (3)

The marginal distribution over the visible variables is obtained as:

$P(\mathbf{v})=\frac{1}{Z}\sum_{\mathbf{h}}e^{-E(\mathbf{v},\mathbf{h})}$ (4)

The conditional probabilities can be described as:

$P\left(h_{j}=1|\mathbf{v}\right)=\sigma\left(\mathbf{W}_{j}\mathbf{v}+b_{j}\right)$
(5)
$P\left(v_{i}=1|\mathbf{h}\right)=\sigma\left(\mathbf{W}_{i}\mathbf{h}+a_{i}\right)$
(6)

where $\sigma$ is logistic function defined as:

$\sigma(x)=\left(1+e^{-x}\right)^{-1}$ (7)

A deep belief network (DBN) is constructed by stacking multiple RBMs [29].
Each RBM in the DBN is trained using an unsupervised manner at first. Then,
the output of previous RBM is inputted into the next RBM. All RBMs are fine-
tuned together by supervised optimization.

###  3.2 Convolutional Neural Network

Convolutional neural network (CNN) [30] is good at capturing spatial
information of data (see Fig. 2(B): Convolutional Unit). Most CNNs consist of
two types of layers: convolutional layer, pooling layer, and some include
fully-connected layer.

In specific, a convolutional layer has filters $k_{ij}^{l}$, the size of which
is smaller than the dimension of input data and forms a locally connected
structure. Filter at layer $l$ can produce feature maps $\mathbf{X}_{j}^{l}$
by convolving with the input $\mathbf{X}_{i}^{l-1}$ using biases $b_{j}^{l}$.
These features are subjected to a non-linear transform $f(\cdot)$ and can be
mathematically expressed as:

$\mathbf{X}_{j}^{l}=f\left(\sum_{i=1}^{M^{l-1}}\mathbf{X}_{i}^{l-1}*k_{ij}^{l}+b_{j}^{l}\right)$
(8)

Where $M^{l-1}$ represents the number of feature maps in layer $l-1$, and $*$
denotes convolution operation.

A pooling layer is responsible for feature selection and information
filtering. Two kinds of pooling operations are widely used: max pooling and
average pooling. In max pooling, maximum value is mapped from a sub-region by
pooling operator. In average pooling, the average value of a sub-region is
selected as the result. A fully-connected layer is usually located at the last
part of a CNN. It transforms a 1D vector and sent the output to its following
layer through an activation function.

Weight sharing and sparse connections are two basic concepts of CNN, which
lead to dramatic reduction in the number of parameters, thus accelerating the
training process. Meanwhile, these concepts also help mitigate overfitting
problem while maintaining the ability of complex feature extraction.

###  3.3 Recurrent Neural Networks

Recurrent neural network (RNN) [31] was developed to deal with sequential data
because of its unique recurrent structure (see Fig. 2(B): Recurrent Unit),
which allows previous outputs to be used as inputs while having hidden states.
It is widely used in applications that need to extract sequential information,
such as natural language processing, speech recognition, and EEG
classification.

####  3.3.1 GRU

Gated Recurrent Unit(GRU) [32] has two gates, reset $\mathbf{r}_{t}$ and
update $\mathbf{z}_{t}$. Let $\mathbf{x}_{t}$ be the input at time step $t$ to
a GRU layer and $\mathbf{h}_{t}$ be the output vector. The output activation
is a linear interpolation between the activation from the previous time step
and a candidate activation $\hat{\mathbf{h}}_{t}$.

${\mathbf{h}_{t}=\mathbf{z}_{t}\odot\mathbf{h}_{t-1}+\left(1-\mathbf{z}_{t}\right)\odot\tilde{\mathbf{h}}_{t}}$
(9)

where $\mathbf{z}_{t}$ decides the interpolation weight, which is computed by:

${\mathbf{z}_{t}=f\left(\mathbf{W}_{z}\mathbf{x}_{t}+\mathbf{U}_{z}\mathbf{h}_{t-1}+\mathbf{b}_{z}\right)}$
(10)

where $\mathbf{W}$ and $\mathbf{U}$ are weight matrices for the update gate,
$\mathbf{b}$ is bias vector, and $f(\cdot)$ is a non-linear function. The
candidate activation is also controlled by an additional reset gate and
computed as follows:

${\tilde{\mathbf{h}}_{t}=g\left(\mathbf{W}_{h}\mathbf{x}_{t}+\mathbf{U}_{h}\left(\mathbf{r}_{t}\odot\mathbf{h}_{t-1}\right)+\mathbf{b}\right)}$
(11)

where $\odot$ represents an element-wise multiplication and $g(\cdot)$ is
often a tanh non-linearity. The reset gate is computed in a similar manner as
the update gate:

${\mathbf{r}_{t}=f\left(\mathbf{W}_{r}\mathbf{x}_{t}+\mathbf{U}_{r}\mathbf{h}_{t-1}+\mathbf{b}_{r}\right)}$
(12)

####  3.3.2 LSTM

Different from GRU, Long Shor-Term Memory (LSTM) [33] has three gates, input
$\mathbf{i}_{t}$, output $\mathbf{o}_{t}$, and forget gates $\mathbf{f}_{t}$.
Each LSTM cell has an additional memory component $\mathbf{c}_{t}$. The gates
are calculated in a similar manner as the GRU unit but LSTM has additional
memory components.

$\mathbf{i}_{t}=f\left(\mathbf{W}_{i}\mathbf{x}_{t}+\mathbf{U}_{i}\mathbf{h}_{t-1}+\mathbf{b}_{i}\right)$
(13)

$\mathbf{o}_{t}=f\left(\mathbf{W}_{o}\mathbf{x}_{t}+\mathbf{U}_{o}\mathbf{h}_{t-1}+\mathbf{b}_{o}\right)$
(14)

$\mathbf{f}_{t}=f\left(\mathbf{W}_{f}\mathbf{x}_{t}+\mathbf{U}_{f}\mathbf{h}_{t-1}+\mathbf{b}_{f}\right)$
(15)

A memory component is updated by forgetting the existing content and adding a
new memory component as:

$\mathbf{c}_{t}=\mathbf{f}_{t}\odot\mathbf{c}_{t-1}+\mathbf{i}_{t}\odot\hat{\mathbf{c}}_{t}$
(16)

where $\hat{\mathbf{c}}_{t}$ can be computed by:

$\hat{\mathbf{c}}_{t}=g\left(\mathbf{W}_{c}\mathbf{x}_{t}+\mathbf{U}_{c}\mathbf{h}_{t-1}+\mathbf{b}_{c}\right)$
(17)

The updated equation for the memory component is controlled by the forget and
input gates. Then, the output of the LSTM unit is computed from the memory
modulated by the output gate according to the following equation:

$\mathbf{h}_{t}=\mathbf{o}_{t}\odot g\left(\mathbf{c}_{t}\right)$ (18)

###  3.4 Autoencoder and Stacked Autoencoder

Stacked autoencoder is an unsupervised model and consists of several single
autoencoders. It can learn discriminative and informative features
automatically using unlabeled data. The single autoencoder is a symmetrical
structure with two layers [34] (see Fig. 2(B): Autoencoder Unit). An encoder
layer that learns representative features of the input data, and a decoder
layer that reconstructs the features. The goal of an autoencoder is to
minimize the reconstruction error between the input and the output.

Given the inputs $\mathbf{x}\in\mathbf{R}$, the encoding process first maps it
into a hidden layer $\mathbf{h}\in\mathbf{R}$ with a weight matrix
$\mathbf{W}_{v}$, bias $\mathbf{b}_{v}$, and an activation function
$f(\cdot)$:

$\mathbf{h}=f\left(\mathbf{W}_{v}\mathbf{x}+\mathbf{b}_{v}\right)$ (19)

Then the decoding process transforms hidden layer’s representation
$\mathbf{h}$ into reconstructed $\mathbf{y}$ with a weight matrix
$\mathbf{W}_{h}$, bias $\mathbf{b}_{h}$, and an activation function
$g(\cdot)$:

$\mathbf{y}=g\left(\mathbf{W}_{h}\mathbf{h}+\mathbf{b}_{h}\right)$ (20)

To simplify the network architecture, the tied weights strategy
$\mathbf{W}_{v}=\mathbf{W}_{h}=\mathbf{W}$ are usually employed. The
parameters to be determined are
$\\{\mathbf{W},\mathbf{b}_{v},\mathbf{b}_{h}\\}$. The objective function to
train an autoencoder is to minimize the loss:

$\arg\min_{\mathbf{W},\mathbf{b}_{v},\mathbf{b}_{h}}\mathcal{J}\left(\mathbf{W},\mathbf{b}_{v},\mathbf{b}_{h}\right)$
(21)

Given the training samples $\mathbf{D}_{n}$, the loss function is defined as:

$\mathcal{J}\left(\mathbf{W},\mathbf{b}_{v},\mathbf{b}_{h}\right)=\sum_{\mathbf{x}\in\mathbf{D}_{n}}L(\mathbf{x},\mathbf{y})$
(22)

where $L$ is the reconstruction error.

###  3.5 Others

In addition to models we mentioned above, there are other models aiming to
solve shortcomings of the above models. For example, capsule network (CapsNet)
was presented to overcome the shortcomings of CNN that discards relationship
information between parts of an image. When it applied to EEG [17], it is
expected to capture comprehensive information among channels, frequencies, and
more. Other shortcomings are long training time and the lack of
interpretability. Extreme learning machine (ELM) algorithm was proposed to
expedite the training course [35], where weights of the input layer and hidden
layers were randomly generated. The concept of random weights is also applied
to echo state network (ESN) [36]. ESN is a recurrent neural network where the
weights of hidden layers are randomly assigned and fixed while the weights of
output layer can be learned. Spiking neural network (SNN) is a biologically
inspired model and was used to explore brain activity patterns in [37]. Deep
polynomial network (DPN) uses a quadratic function to process its inputs and
is able to learn features between different samples and dimensions. It was
implemented in [38] to utilize features from multiple views for motor imagery
classification, including common spatial pattern, power spectral density, and
wavelet packet transform.

##  4 Applications

We summarized applications, in which deep learning was utilized for EEG
processing and classification, in this section. For your convenience, we group
diverse applications into six topics, which are brain-computer interface (see
Supplementary Table LABEL:tab_bci for the details of studies), disease
detection (see Supplementary Table LABEL:tab_disease), emotion recognition(see
Supplementary Table LABEL:tab_emotion), operator functional states (see
Supplementary Table LABEL:tab_ofs), sleep stage classification (see
Supplementary Table LABEL:tab_slepp), as well as the applications other than
above topics (see Supplementary Table LABEL:tab_other_applications). According
to statistics, the majority of published papers belong to the topics of brain-
computer interface (account for 27%) and disease detection (account for 25%).
The proportions of each topic and the proportions of each model used in each
topic are illustrated in Fig. 3. In addition, we collected the information of
the publicly available datasets which had been used in the studies and listed
them in Supplementary Table LABEL:tab_datasets.

Figure 3: Proportions of topics and models. The outer ring represents paper
proportions for each topic. The models within each topic are distinguished
from the darkest to lightest colors, which represent CNN, RNN, SAE, DBN, and
other models, respectively.

###  4.1 Brain-Computer Interface

A brain-computer interface (BCI) can be defined as a system that decodes brain
activity and translate user’s intentions into messages or commands for the
purposes of communication or the control of external devices, and more. In
this topic, deep learning was mainly applied to establish motor imagery (MI)-
and P300-based BCIs (see Fig. 4).

Figure 4: (A) Paradigms of brain-computer interface. (B) Proportions of the
published papers for each paradigm by the year of 2020

Transfer learning is introduced into deep learning model to provide a solution
for the lack of EEG data. Therefore, the data from a whole session or a
subject could be used for training, rather than dividing into training set and
testing set. Lu et al. implemented the session-to-session transfer and
subject-to-subject transfer and found that it is feasible to use a model
trained on a session to classify the data of another session, while subject-
to-subject transfer is inefficient [39]. With the combination of transfer
learning and CNN, Hang et al. proposed a deep domain adaption network [40].
They used maximum mean discrepancy to minimize the distribution discrepancy
between target and source subjects and used the center-based discriminative
feature learning method to make deep features closer to corresponding class
centers. The evaluation on BCI Competition datasets (i.e., Dataset IVa of
Competition III and Dataset IIa of Competition IV) demonstrated a good
classification performance. In the study of the subject-to-subject transfer
[41], network weights were transferred. Dose et al. used a pool of data to
obtain a universal model of CNN [42]. This model was then adapted based on a
small amount of data from a particular subject before applying to this
subject. Their results showed that an average improvement of 6$\sim$9% in
terms of classification accuracy was achieved for motor imagery
classification.

Moreover, transfer was done between domains. A CNN-based model (VGG-16)
trained on image data (data from ImageNet) was transferred to recognize EEG
data by freezing the parameters in the first several layers and fine-tuning
the parameters in the last several layers using an EEG dataset [43]. The
performance was better than that of support vector machine. Similar to the
domain of image recognition, the amount of EEG data can also be increased by
augmentation procedure. Li et al. produced new samples by adding noise into
EEG data [44]. They claimed that adding noise into amplitudes of power spectra
was superior to that adding noise into EEG time series in terms of
classification accuracy. Zhang et al. used intrinsic mode functions derived
from empirical mode decomposition to generate new EEG samples so that the
total number of samples was increased [45].

Classical models such as CNN and RNN were originally developed for image or
speech recognition, so they did not well match the characteristics of EEG
signal. They should be adapted before applying to EEG recognition. Li et al.
designed a CNN-based network consisted of three blocks to capture spatial and
temporal dependencies [44]. Multi-channel raw EEG signals were fed into
temporal convolutional layer and spatial convolutional layer successively in
the first block. In the second block, a standard convolutional layer and a
dilated convolutional layer were utilized to extract temporal information at
different scales while reducing the number of parameters. The extracted
features were finally used for motor imagery classification in the third
block. In another CNN-based network [46], a layer was fed by all outputs from
previous layers and its output was inputted to all following layers. By using
such dense inter-layer connections, information loss could be reduced. In
[45], EEG signals were transformed into tensors and fed into a CNN-like
network where convolution were replaced with complex Morlet wavelets,
resulting in parameter reduction. Wavelet kernel was also used to learn time-
frequency features [41]. Their results demonstrated that wavelet kernels can
provide faster convergence rate and higher classification accuracy compared to
plain CNN. Alazrai et al. used CNN to extract features from time-frequency
images, which were transformed using a quadratic time-frequency distribution
[47]. The methods were compared to a support vector machine, and it suggested
that CNN can achieve good performance in MI tasks of the same hand. In order
to accelerate the training course and alleviate the overfitting problem, Liu
et al. adjusted the number and position of batch normalization layers in a
CNN-based network for P300 detection [9]. Kshirsagar et al. employed leaky
rectified linear unit activation function at each convolutional layer [48]. To
evaluate whether the number of convolutional layers needs to be adjusted for
different BCI tasks and find out an optimal structure, Lawhern et al. compared
networks with different numbers of convolutional layers [49]. Their results
showed that deep CNN (i.e., five convolutional layers) tended to perform
better on the oscillatory BCI dataset than on the event-related potential BCI
dataset, while shallow CNN (i.e., two convolutional layers) achieved better
performance on the event-related potential BCI dataset. Apart from CNN, Lu et
al. used a DBN (i.e., three RBMs and an output layer) to extract features of
motor imagery [39]. Some studies aimed to compare performances of different
deep learning models. For example, Pei et al. compared SAE and CNN in the
classification of reaching movements [50]. They found that SAE was better than
CNN and suggested that poorer performance in CNN might be due to the lack of
training data. One year later, another study comparing between these two
models showed that SAE had satisfactory performance in some trials, but
inefficient to treat those trials from subjects who are less attentive in P300
detection, while CNN performed well in terms of accuracy and information
transfer rate [48].

The combination of deep learning model and traditional model or the mixture of
two or more types of deep learning models is applied in EEG classification.
For example, SAE was combined with support vector machine to classify EEG
signal [51]. SAE was also combined with CNN to develop a new model [52], where
CNN layers were used to extract features from 2D time-frequency images
(obtained by Fourier transform over EEG signals) and SAE layers were used to
further extract features. In [53], the features extracted by CNN were fed into
an autoencoder for cross-subject MI classification. This combination achieved
a better accuracy for the cross-subject classification, but worse for the
subject-specific classification, compared to the combination of CNN and
multilayer perceptron (MLP). Zhang et al. presented a hybrid network comprised
of CNN and LSTM, in which EEG signals were sequentially processed through
common spatial pattern, CNN, and LSTM [54]. The idea of using CNN and LSTM to
extract spatial and temporal features was also conceived by Yang et al. [55].
However, they inserted a discrete wavelet transformation (DWT) between CNN and
LSTM, which led to better performance in the MI classification compared to
that of pure combination of CNN and LSTM.

In addition to P300- and MI-based BCIs, deep learning models also applies to
the other BCIs, including motion-onset visual evoked potentials [56] and self-
paced reaching movements [50]. Nguyen et al. developed an SSVEP-based BCI
speller system that used only one channel [57]. They segmented single-channel
EEG signals to windowed EEG and used fast Fourier transform to extract
features, then features were fed into a CNN model for classification.
According to this paper, frequency resolutions and lengths of time window
influence classification performance. It was found that frequency resolution
of 0.0625 Hz and time window of 2S were optimal for the classification of
five-class SSVEP [57]. Waytowich et al. proposed a CNN-based network to deal
with asynchronous problem in SSVEP classification [58]. This network had three
convolutional layers and each convolutional layer was followed by batch
normalization, pooling and dropout layers. The network was evaluated on a
12-class SSVEP dataset. The results demonstrated its robustness and efficiency
in both slow-wave and fast-wave frequency activity.

Figure 5: Proportions of the published papers across diseases.

###  4.2 Disease Detection

Machine learning could benefit disease diagnosis by providing assistant
information and preliminary diagnostic results. In this topic, deep learning
models were also widely employed to detect a variety of diseases or disorders
(see the distribution of the published papers over diseases in Fig. 5). In
this subsection, commonly used models designing strategies were introduced at
first, including examples of single or hybrid models, as well as the detailed
structures (e.g., layers). Afterwards, we described other techniques that have
an influence on the performance of deep learning.

CNN was a deep learning model which was widely introduced into the detection
of brain diseases and disorders (e.g. detection of seizures [59] and
schizophrenia [60]). To enhance CNN classification performance, original
binary labels, namely interictal epileptiform discharge (IED) and non-IED,
were converted to a multiple labels for model training [61]. Specifically,
samples were given more detailed tags (divided into five subclasses) according
to spatial distribution and morphology of EEG waveforms and were then fed into
a CNN model. At the last layer of the CNN model, preliminary subclasses were
thresholded to obtain final binary classification (IED versus non-IED).
Compared to the CNN trained with binary labels, the CNN with extra threshold
layer trained with more detailed tags was better in classification performance
for most subjects.

CNN was combined with other models to further enhance the classification
capability. In [62], CNN and AE were combined to learn robust features in an
unsupervised way. The integrated network had an encoder consisting of
convolution and down-sampling and a decoder consisting of deconvolution and
up-sampling. These extracted features were fed into several classifiers
including k-nearest neighbor (KNN), SVM, and MLP. The results demonstrated
that CNN+AE was superior to the others in epilepsy detection. In [63], a
hybrid model combining CNN, AE, and LSTM performed remarkable performance of
seizure prediction. However, model combination is not always positive to the
performance improvement. Some studies showed that performance may decline in
some cases. For instance, Mumtaz et al. combined CNN and LSTM to detect
unipolar depression. Their results showed that the hybrid model did not
outperform single model of CNN [64].

The inner settings of a model varied across published papers. Tsiouris et al.
discarded intermediate dropout layer since it did not show significant
influence on the basis of shuffle [22]. Qiu et al. applied the corrupting
operation to stacked autoencoder for seizure detection [65]. Specifically,
they designed a denoising sparse autoencoder whose inputs can be set to zero,
improving robustness and reducing overfitting. In addition to the inner
settings, disease detection performance is influenced by data. Mumtaz et al.
found that unipolar depression can be more accurately detected using the EEG
recorded under the condition of eyes open compared to eyes closed [64]. In the
paper of ADHD detection using a CNN model, EEG signals at different channels
were rearranged to make adjacent channels together in the connectivity matrix
to improve accuracy [66]. Moreover, Tsiouris et al. shuffled interictal and
preictal segments of EEG to avoid overfitting in seizure detection [22]. As a
result, non-adjacent preictal segments were randomly selected as an input of
their LSTM-based networks. Yuan et al. used a channel-aware module to enhance
the capability of feature learning and focus on important and relevant EEG
channels [67]. Daoud et al. computed the statistical variance and entropy for
channels, and selected these with the highest variance entropy for seizure
prediction [63].

EEG data collection and preprocessing also affect the performance of deep
learning. For example, EEG data are reshaped into 2D format before inputting
into a deep learning model. In [10], EEG data were transformed into 2D images
of spectral powers. Then, these images were fed into a CNN network for
distinguishing Alzheimer’s disease and mild cognitive impairment from healthy
controls. To differentiate patients with schizophrenia [68], Pearson
correlation coefficients were calculated between channels and assembled as a
correlation matrix. Correlation matrices of each subject were fed into a CNN
network. Moreover, fast Fourier transform [69] and continuous wavelet
transform [70] were used to transform EEG data into 2D images for motor
impairment neural disorders and epilepsy classification, respectively. Wei et
al. further converted 2D images into 3D stacked images according to the mutual
correlation intensity between channels [71]. Brain networks derived from EEG
(in the form of connectivity matrices) were stacked on four frequency bands in
[66]. To utilize comprehensive information from different data forms, Tian et
al. used three CNNs to obtain deep features in time, frequency, and time-
frequency domain, then utilized these features to generate a multi-view
classifier [72]. By comparing with single-view classifiers, multi-view
classifiers that utilized features from different views exhibited better
performance in epilepsy detection. In a comparison study among raw EEG
signals, Fourier transform, wavelet transform, and empirical mode
decomposition, raw signals and empirical mode decomposition were better than
the others in distinguishing focal EEG from non-focal EEG, while Fourier
transform was best in ictal and non-ictal classification [73]. To handle the
problem of lack of data, EEG signals were split into sub-signals using a fixed
window, sliding from the start to the end in an overlapping manner in [74].
Cao et al. presented a novel integration system to help collect and label more
EEG data, which consisted of a visualization-based labeling module and a deep
learning-based active learning module [19]. This system allows users to
interactively explore and label EEG segments in a 2D map view, in which there
are 350 million EEG segments from over 2500 subjects. Its deep learning module
can leverage the results to progressively improve the accuracy of
classification.

Figure 6: Four illustrative emotions classified based on the scores of arousal
and valence.

###  4.3 Emotion Recognition

Emotion conveys lots of underlying information during conversations and is
part of communication between people. People can understand emotion by reading
facial expression, voice tone, and behaviors. From the perspective of
artificial intelligence, emotion can be recognized by using facial expression
[75], eye movement [76], EEG [77], or galvanic skin response signal [78].
According to different rules, emotion can be categorised into different
classes. In general, two dimensions of arousal and valence are used to
classify emotion (see Fig. 6). In the literature included in this survey,
emotion is mainly categorised into three classes (i.e., positive, neutral, and
negative) or more classes based on the scores of arousal and valence. In these
papers, the datasets named ’SEED’ [79] and ’DEAP’ [80] are frequently used to
build and evaluate deep learning models of emotion recognition.

SEED dataset was published by the BCMI laboratory at the Shanghai Jiao Tong
University [79]. In this dataset, EEG data were collected from 15 subjects
when they were watching positive, negative, and neutral clips using 62 EEG
channels. Using this dataset, Li et al. trained a hierarchical CNN and
accomplished around 88% of recognition accuracy based on features of the gamma
band [81]. Zhang et al. proposed a two-layer RNN model to extract spatial and
temporal features, respectively. The first layer of their network is an RNN
that takes EEG signals at a certain moment from various electrodes as inputs.
The outputs of the first layer were concatenated by time and fed to the second
RNN layer. The performance of the architecture was reported reliable on SEED,
achieving an accuracy of 89.5% [75]. In [82]. Zeng et al. used an architecture
that adapted from SincNet (a CNN-based network proposed for speaker
recognition [83]) to classify EEG data from the SEED. Their results
demonstrated that adapted SincNet (i.e. three convolutional layers and three
fully connected layers) was promising for emotion classification, reaching an
accuracy of around 95%.

Another dataset named ’DEAP’ [80], was collected from 32 subjects when they
watched 40 one-minute long music videos. Perceptual emotion was assessed in
terms of arousal, valence, liking, and dominance. Studies using this dataset
have showed that deep learning methods were successful and effective to
classify emotion categories based on EEG. [77], [84]. Particularly, LSTM was
used to classify emotions based on raw EEG in [85], it reported an acceptable
accuracy of around 85%. In [86], various hand-crafted EEG features (e.g.
sample entropy, mean, and power spectral density) were fed to three stacked
autoencoders in a parallel way for voting. Chao et al. also designed a
parallel architecture to process EEG signal. However, they used DBN as the
basic unit. To improve the classification performance and utilize strengths of
different models. Li et al. combined CNN and LSTM to extract representations
from multi-channel EEG, in which CNN was used to learn inter-channel and
inter-frequency correlation while LSTM was used to extract contextual
information [87].

Because deep learning models have many parameters, training is time-consuming.
Various techniques aim to deal with this issue. Hemantha et al. modified the
back-propagation neural network by arranging layers in a circular way that the
output can access the parameters of input and hidden layers [88]. This
modification reduced the convergence time by around 20% compared to the back-
propagation neural network without modification. Jirayucharoensak et al. used
principal component analysis for dimension reduction to lower computation
cost. [89]. Gao et al. utilized gradient priority particle swarm optimization
to optimize parameters of a CNN model [90].

###  4.4 Operator Functional States

The operator functional states (OFS) describe the mental states of operators
in specific working conditions [91]. Two of them are mental workload and
mental fatigue. In specific, mental workload is a measure of cognitive
resources consumed in the human working memory while mental fatigue is
identified by an accumulated process of a disinclination of effort and
drowsiness. To date, deep learning was used to identify mental states based on
EEG signal. For example, drivers’ [92] [93] [94] [95] and pilots’ [96] fatigue
was monitored for the purposes of preventing fatigued operation.

Generalization is one of the important metrics to evaluate a model. In the
classification of operator functional states, large variance across subjects
is challenging. Many studies used the strategy that classifiers were subject-
specific for each subject. For example, Tao et al. fused multiple ELMs and
Naive Bayesian model to build a subject-specific classifier. This ensemble
model with fine-tuned hyper-parameters was of the higher subject-specific
accuracy in mental workload assessment [97]. In the study of [98], Zhang et
al. selected the most relevant EEG channels for each subject and used these
subject-specific channels for calculating weights between the input layer and
the first hidden layer in the DBN. In contrast to the subject-specific models,
the cross-subject model aims to have a general model for tolerating variance
of subjects. For example, Heron et al. used multi-path convolutional layers
and bi-directional LSTM layers to learn frequency and temporal features over
subjects. This model achieved low variance in performance across subjects and
showed better generalization compared to subject-specific models [99]. Another
cross-subject model was proposed using an adaptive DBN with the weights of the
first hidden layer iteratively updated to track the EEG changes in a new
subject [100]. When different tasks were used to induce mental workload, the
induced workload might be variable across tasks. The cross-task workload
classification was made by using a CNN+RNN model [101]. Another study used
transfer learning strategy to improve model generalization for the
classification of mental workload [102].

Multiple kinds of features can be fused to improve assessment performance of
mental workload. Gao et al. presented a temporal convolutional block to
extract sequential information of EEG. The block orderly consists of a 1D
convolution, a rectified linear activation, and a batch normalization.
Temporal convolutional blocks and dense layers for spatial feature fusion were
combined to form a novel network. Their results showed that this architecture
can achieve higher accuracy for fatigue classification, when compared to these
networks that replace convolutional block by 1D convolution [94]. Zhang et al.
proposed a two-stream CNN network to learn spectral and temporal features
[103]. One stream of CNN was fed by power spectral density topographic maps
and the other was fed by topographic maps of amplitude distributions. At the
same year (2019), They designed another network for the same propose of
learning spectral and temporal features for mental workload classification. In
this network, CNN with 3D kernels were first applied to EEG cubes, then
extracted features from CNN were flatten to 1D vectors and fed to a
bidirectional LSTM for further processing and classification [101]. Both
models (i.e. two-stream CNN and CNN+LSTM) showed a significant improvement in
mental workload classification.

###  4.5 Sleep Stage Classification

Sleep stage classification helps us understand the quality of sleep and sleep-
related disorders. Supplementary Table. 1 briefly summarized the
characteristics of each sleep stage. With the aid of EEG recording, sleep
quality can be assessed objectively. In the processing of sleep quality, sleep
staging is a precedent step. In recent years, sleep staging can be performed
automatically using deep learning models. For instance, LSTM model was used
for single-channel sleep stage classification [104]. CNN+LSTM model was
presented to classify sleep stages [105] [27] and detect sleep spindles [106].

Sleep is a process consisting of a sequence of stages. Temporal information is
critical for sleep stage classification. Morlet wavelets [107] and time-
frequency representation [104] [108] were applied to retain temporal
information in the extraction of spectral features. These extracted features
were then learned by deep learning models for sleep stage classification,
showing promising performance. Using the time-frequency representation of EEG,
CNN model achieved good performance [109]. In another study, the CNN was
combined with LSTM to capture both temporal and spatial information for sleep
stage classification [110]. In supervised learning, training data are labelled
by experts so that deep learning models can learn discriminative information
among sleep stages. Labelling is time-consuming, so unsupervised deep learning
models were developed to learn from unlabeled EEG data. Zhang et al. presented
a CNN model with a greedy layer-wise training strategy, in which complex-
valued K-means was utilized to train filters used in the convolution with
unlabeled EEG data [111]. In [112], unsupervised sparse DBN was used to
extract features. Subsequent classifiers (e.g. KNN, SVM) based on these
unsupervised-extracted features performed well on sleep stage classification.
Sleep stage classification has unbalanced sample numbers for each sleep stage.
To relieve this issue, the class-balanced random sampling [107], data
augmentation [113], class-balance training set design [27], and synthetic
minority oversampling technique [114] were used to make samples more balanced
across sleep stages.

###  4.6 Others

Those studies which cannot be grouped into the above topics are presented in
this subsection. A summary table with key information of those studies is
prepared (see Supplementary Table LABEL:tab_other_applications). On the one
hand, EEG with deep learning can be used for person identification [115],
[116], age and gender prediction [117]. On the other hand, it can also be used
to decode brain activity related to vision, audio [118], and pain [119]. In a
study of image classification [120], LSTM was used to extract EEG features
while CNN was used to extract image features. This study claimed that features
extracted from EEG could help image classification so that classification
performance was improved. In [121], a CNN+LSTM hybrid network was used to
extracted visual representations from EEG, and a generative adversarial
network was applied to reconstruct images from the learnt EEG representations.
Deep learning and EEG were also applied to understand brain functions and
structure. These studies aimed to understand functional brain connectivity
[122], speech laterality [123], as well as memory under specific conditions.
In specific, Baltatzis et al. investigated the brain’s activity of different
people (ever experienced school bullying or not) to different stimuli (2D
videos or Virtual Reality) [124]. Doborjeh et al. used EEG and spiking neural
network to decode how the brain react to various commercial brands (locally
familiar or not) [125]. Arora et al. studied the memory loss after seizure
surgery [126].

##  5 Discussion

In this survey, we reviewed the researches of deep learning in EEG for the
last ten years, which is a critical period for the development of deep
learning used in EEG. An introduction about deep learning in EEG was first
presented in the first section. Subsequently, we presented classical methods
of artifacts removal which is an important step in EEG processing. We detailed
prevalent deep learning models, followed by the comprehensive reviews on
different applications that used deep learning to process and classify EEG
signals. These applications were categorised into several topics for
presentation. The increase in the number of published papers suggested that
the research of deep learning in EEG are expanding over time. Although
remarkable achievements were obtained, challenges and limitations still exist,
which need to be addressed. We discuss them below and provide our
perspectives.

The performance of deep learning-based classification should be further
improved. Although the published papers showed the advantages of deep learning
in EEG classification and demonstrated that deep learning is superior to
conventional methods, the performance is much lower compared to the
performance achieved by deep learning in image or speech classification [24],
[25]. The reasons for the lower performance are mainly from two aspects, EEG
data itself and models. On the one hand, the EEG signal is non-stationary and
much variable over time, which makes the extraction of robust features
difficult. An effective solution for this problem is to partition time series
into short segments, which can be seen as a stationary signal. However, this
is only an approximation but not a final solution. When performing cross-
participant classification or cross-session classification, EEG over
participants or sessions is largely variable, making the above problem more
dominant. Besides, available data size in EEG studies is significantly smaller
than the data size used in image or speech studies [24], [25]. As we know, the
deep learning model requires extensive training and a large data size can
benefits the model training to a great extent. Compared to those domains with
millions of training data, EEG signals are collected from tens, hundreds, at
most thousands of participants. These limitations of existing models and the
characteristics of EEG lead to overall relatively low performance. On the
other hand, most deep models are originally proposed to process other signals
(e.g., images) rather than EEG. Though modifications can be utilized to these
models, it is still difficult to reach its maximum performance because of the
nature of unfitness.

One potential solution for the lack of EEG data in the model training is the
use of transfer learning. Deep learning model can be trained by the data which
are not collected at the moment and the trained model can be used for
recognition or classification on the new collected EEG data after fine-tuning
or even without fine-tuning [39], [40], [41]. However, the effect of transfer
learning on classification performance improvement in EEG is not as good as in
image classification [127]. This might be due to the lack of effective
training framework and strategies that are suitable for the transfer of EEG
pattern. As we know, the underlying brain activity could be detected by other
neurophysiological signals besides EEG. Therefore, other neurophysiological
signals (e.g., fNIRS) can be used to pre-train a deep learning model to learn
a pattern which is relevant to the brain activity. Then, this pre-training
model can be fine-tuned by EEG signal so that other neurophysiological signals
could compensate the lack of EEG. Alternatively, multiple data modalities
(e.g., EEG, fMRI, and DTI) could be fed into deep learning model and extract
their respective information contained in each data modality so that the deep
learning model is able to learn information from all those data. With the help
of multiple data modalities, the performance can be improved. It has been seen
that classification performance was elevated by feature fusion in the case of
using conventional classifiers [128]. The fusion could be done at the
different stages of the classification process (e.g., at the beginning of
initial feature fusion or at the later stage of decision fusion). In deep
learning models, how the fusion maximises the performance should be studied.
Moreover, data modalities might be heterogeneous in time resolution and
dimension. It requires alignment among modalities. Sometimes, the final
classification performance might be dramatically affected by the alignment.
For example, poor alignment results in low classification accuracy. Lastly, a
straightforward solution for the lack of EEG data is to collect more data.
However, it could lead to new issues such as cost increase and time delay. If
many institutes collaboratively collect EEG data, they have to define data
standards before data collection to make all data from different institutes
comparable.

As more data modalities and more data from different institutes are used for
the training of deep learning model, computation is definitely increased
further from a high demand. Researchers must upgrade or replace hardware
partially or fully after a certain period to meet the computational
requirements of deep learning. Every institute spends lots of money to do so,
resulting in a financial burden and possible inefficiency of facility usage.
With advances in communication technology and cloud computation, an
alternative option that hardware can be more efficiently used, and the
financial burden could be mitigated is to shift computation to a cloud server
which is shared across many institutes. This emerges and progresses commercial
services for cloud computation, such as Amazon Web Services and Google Cloud.

When designing models for EEG signal processing, we must take characteristics
of EEG (e.g., non-stationarity and small amplitude) into consideration, as
well as external circumstances such as electrode layout, interferences
introduced by the surrounding environment during data collection. A possible
instance may be tensor deep learning, as it can utilise comprehensive
information (time, channels, frequencies, etc.) and stored the information in
tensor. These novel deep models are expected to update dynamically in response
to data collection over time. Appropriate data retraining schemes or parameter
tuning strategies should also be developed to promote the advance of deep
learning in EEG. Models can be updated in a real-time manner by tuning
parameters or even changing architecture. Another concern that meets our needs
is to develop general models that can be applied to different applications of
EEG. A prominent advance we need to mention is the development of EEGNet [49],
which is proven effective and accurate in different BCI paradigms. Another
promising model is SincNet, which was initially proposed for speaker
recognition and also well for the classification of EEG signal [82]. New deep
learning architectures, such as capsule network [129, 130], are also required
to enhance the chance of success of EEG applications.

Apart from various applications that use deep learning to classify EEG
signals, deep learning may also be a useful tool to reveal neural mechanisms
of the brain. On the one hand, deep learning was developed with the reference
to the brain structure so that it might be possible to reflect information
process in the brain to some extent. On the other hand, EEG is a signal
representing the underlying brain activity. Therefore, deep learning might be
used to reversely decompose EEG signal back into representations in the brain.
It seems unrealistic at current stage, but some efforts towards this aim were
made. For example, Goh et al. proposed spatio-spectral representation learning
to classify walking conditions. Meanwhile, they demonstrated spatial
distribution of brain activations related to the movement of lower limbs
[131].

Our survey is a glimpse of what have been done for the deep learning in EEG
over the past ten years. There are still many researches currently on-going at
laboratories and hospitals, dealing with challenges we mentioned above and
beyond. We hope that our survey can provide the researchers who are working in
this field with a summary and facilitate their researches.

## References

  * [1] M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspectives, and prospects,” _Science_ , vol. 349, no. 6245, pp. 255–260, 2015.
  * [2] D. Michie, D. J. Spiegelhalter, C. Taylor _et al._ , “Machine learning,” _Neural and Statistical Classification_ , vol. 13, no. 1994, pp. 1–298, 1994\.
  * [3] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” _nature_ , vol. 521, no. 7553, pp. 436–444, 2015.
  * [4] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein _et al._ , “Imagenet large scale visual recognition challenge,” _International journal of computer vision_ , vol. 115, no. 3, pp. 211–252, 2015.
  * [5] D. Amodei, S. Ananthanarayanan, R. Anubhai, J. Bai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, Q. Cheng, G. Chen _et al._ , “Deep speech 2: End-to-end speech recognition in english and mandarin,” _International conference on machine learning_ , pp. 173–182, 2016.
  * [6] Z.-Q. Zhao, P. Zheng, S.-t. Xu, and X. Wu, “Object detection with deep learning: A review,” _IEEE transactions on neural networks and learning systems_ , vol. 30, no. 11, pp. 3212–3232, 2019.
  * [7] D. Zhan, S. Yi, and D. Jiang, “Small-scale demographic sequences projection based on time series clustering and lstm-rnn,” _2018 IEEE International Conference on Data Mining Workshops (ICDMW)_ , pp. 803–809, 2018.
  * [8] W. P. He, G. Wang, J. Hu, C. Li, B. L. Guo, and F. P. Li, “Simultaneous Human Health Monitoring and Time-Frequency Sparse Representation Using EEG and ECG Signals,” _IEEE Access_ , vol. 7, pp. 85 985–85 994, 2019.
  * [9] M. F. Liu, W. Wu, Z. H. Gu, Z. L. Yu, F. F. Qi, and Y. Q. Li, “Deep learning based on Batch Normalization for P300 signal detection,” _Neurocomputing_ , vol. 275, pp. 288–297, 2018.
  * [10] C. Ieracitano, N. Mammone, A. Bramanti, A. Hussain, and F. C. Morabito, “A Convolutional Neural Network approach for classification of dementia stages based on 2D-spectral representation of EEG recordings,” _Neurocomputing_ , vol. 323, pp. 96–107, 2019.
  * [11] L. F. Haas, “Hans berger (1873–1941), richard caton (1842–1926), and electroencephalography,” _Journal of Neurology, Neurosurgery & Psychiatry_, vol. 74, no. 1, pp. 9–9, 2003.
  * [12] Y. Wang, R. Wang, X. Gao, B. Hong, and S. Gao, “A practical vep-based brain-computer interface,” _IEEE Transactions on neural systems and rehabilitation engineering_ , vol. 14, no. 2, pp. 234–240, 2006.
  * [13] J. Li, Y. Wang, L. Zhang, and T.-P. Jung, “Combining erps and eeg spectral features for decoding intended movement direction,” _2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society_ , pp. 1769–1772, 2012.
  * [14] Y. Zhang, G. Zhou, J. Jin, Q. Zhao, X. Wang, and A. Cichocki, “Sparse bayesian classification of eeg for brain–computer interface,” _IEEE transactions on neural networks and learning systems_ , vol. 27, no. 11, pp. 2256–2267, 2015.
  * [15] J. Li, C. Li, and A. Cichocki, “Canonical polyadic decomposition with auxiliary information for brain–computer interface,” _IEEE journal of biomedical and health informatics_ , vol. 21, no. 1, pp. 263–271, 2015.
  * [16] W.-L. Zheng, J.-Y. Zhu, Y. Peng, and B.-L. Lu, “Eeg-based emotion classification using deep belief networks,” _2014 IEEE International Conference on Multimedia and Expo (ICME)_ , pp. 1–6, 2014.
  * [17] H. Chao, L. Dong, Y. L. Liu, and B. Y. Lu, “Emotion Recognition from Multiband EEG Signals Using CapsNet,” _Sensors_ , vol. 19, no. 9, 2019.
  * [18] F. Li, G. F. Zhang, W. Wang, R. Xu, T. Schnell, J. Wen, F. McKenzie, and J. Li, “Deep models for engagement assessment with scarce label information,” _IEEE Transactions on Human-Machine Systems_ , vol. 47, no. 4, pp. 598–605, 2017.
  * [19] L. Cao, W. B. Tao, S. T. E. An, J. Jin, Y. Z. Yan, X. Y. Liu, W. D. Ge, A. Sah, L. Battle, J. M. Sun, R. Chang, B. Westover, S. Madden, and M. Stonebrakerl, “Smile: A System to Support Machine Learning on EEG Data at Scale,” _Proceedings of the VLDB Endowment_ , vol. 12, no. 12, pp. 2230–2241, 2019\.
  * [20] H. Cecotti and A. Graeser, “Convolutional neural network with embedded fourier transform for eeg classification,” _2008 19th International Conference on Pattern Recognition_ , pp. 1–4, 2008.
  * [21] J. Li and A. Cichocki, “Deep learning of multifractal attributes from motor imagery induced eeg,” _International Conference on Neural Information Processing_ , pp. 503–510, 2014.
  * [22] K. M. Tsiouris, V. C. Pezoulas, M. Zervakis, S. Konitsiotis, D. D. Koutsouris, and D. I. Fotiadis, “A Long Short-Term Memory deep learning network for the prediction of epileptic seizures using EEG signals,” _Computers in Biology and Medicine_ , vol. 99, pp. 24–37, 2018.
  * [23] A. Craik, Y. He, and J. L. Contreras-Vidal, “Deep learning for electroencephalogram (eeg) classification tasks: a review,” _Journal of neural engineering_ , vol. 16, no. 3, p. 031001, 2019.
  * [24] W. Rawat and Z. Wang, “Deep convolutional neural networks for image classification: A comprehensive review,” _Neural computation_ , vol. 29, no. 9, pp. 2352–2449, 2017.
  * [25] A. B. Nassif, I. Shahin, I. Attili, M. Azzeh, and K. Shaalan, “Speech recognition using deep neural networks: A systematic review,” _IEEE Access_ , vol. 7, pp. 19 143–19 165, 2019.
  * [26] M. P. Hosseini, D. Pompili, K. Elisevich, and H. Soltanian-Zadeh, “Optimized Deep Learning for EEG Big Data and Seizure Prediction BCI via Internet of Things,” _IEEE Transactions on Big Data_ , vol. 3, no. 4, pp. 392–404, 2017\.
  * [27] A. Supratak, H. Dong, C. Wu, and Y. K. Guo, “DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-Channel EEG,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 25, no. 11, pp. 1998–2008, 2017.
  * [28] R. Salakhutdinov, A. Mnih, and G. Hinton, “Restricted boltzmann machines for collaborative filtering,” _Proceedings of the 24th international conference on Machine learning_ , pp. 791–798, 2007.
  * [29] G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” _Neural computation_ , vol. 18, no. 7, pp. 1527–1554, 2006\.
  * [30] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Advances in neural information processing systems_ , pp. 1097–1105, 2012.
  * [31] Z. C. Lipton, J. Berkowitz, and C. Elkan, “A critical review of recurrent neural networks for sequence learning,” _arXiv preprint arXiv:1506.00019_ , 2015.
  * [32] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” _arXiv preprint arXiv:1412.3555_ , 2014.
  * [33] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [34] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” California Univ San Diego La Jolla Inst for Cognitive Science, Tech. Rep., 1985.
  * [35] S. F. Ding, N. Zhang, X. Z. Xu, L. L. Guo, and J. Zhang, “Deep Extreme Learning Machine and Its Application in EEG Classification,” _Mathematical Problems in Engineering_ , 2015.
  * [36] L. Bozhkov, P. Koprinkova-Hristova, and P. Georgieva, “Reservoir computing for emotion valence discrimination from EEG signals,” _Neurocomputing_ , vol. 231, pp. 28–40, 2017.
  * [37] M. G. Doborjeh, G. Y. Wang, N. K. Kasabov, R. Kydd, and B. Russell, “A Spiking Neural Network Methodology and System for Learning and Comparative Analysis of EEG Data From Healthy Versus Addiction Treated Versus Addiction Not Treated Subjects,” _IEEE Transactions on Biomedical Engineering_ , vol. 63, no. 9, pp. 1830–1841, 2016.
  * [38] B. Y. Lei, X. L. Liu, S. Liang, W. L. Hang, Q. Wang, K. S. Choi, and J. Qin, “Walking Imagery Evaluation in Brain Computer Interfaces via a Multi-view Multi-level Deep Polynomial Network,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 27, no. 3, pp. 497–506, 2019.
  * [39] N. Lu, T. F. Li, X. D. Ren, and H. Y. Miao, “A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 25, no. 6, pp. 566–576, 2017.
  * [40] W. L. Hang, W. Feng, R. Y. Du, S. Liang, Y. Chen, Q. Wang, and X. J. Liu, “Cross-Subject EEG Signal Recognition Using Deep Domain Adaptation Network,” _IEEE Access_ , vol. 7, pp. 128 273–128 282, 2019.
  * [41] D. Y. Zhao, F. Z. Tang, B. L. Si, and X. S. Feng, “Learning joint space-time-frequency features for EEG decoding on small labeled data,” _Neural Networks_ , vol. 114, pp. 67–77, 2019.
  * [42] H. Dose, J. S. Moller, H. K. Iversen, and S. Puthusserypady, “An end-to-end deep learning approach to MI-EEG signal classification for BCIs,” _Expert Systems with Applications_ , vol. 114, pp. 532–542, 2018.
  * [43] G. W. Xu, X. A. Shen, S. R. Chen, Y. S. Zong, C. Y. Zhang, H. Y. Yue, M. Liu, F. Chen, and W. L. Che, “A Deep Transfer Convolutional Neural Network Framework for EEG Signal Classification,” _IEEE Access_ , vol. 7, pp. 112 767–112 776, 2019.
  * [44] Y. Li, X. R. Zhang, B. Zhang, M. Y. Lei, W. G. Cui, and Y. Z. Guo, “A Channel-Projection Mixed-Scale Convolutional Neural Network for Motor Imagery EEG Decoding,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 27, no. 6, pp. 1170–1180, 2019.
  * [45] Z. W. Zhang, F. Duan, J. Sole-Casals, J. Dinares-Ferran, A. Cichocki, Z. L. Yang, and Z. Sun, “A Novel Deep Learning Approach With Data Augmentation to Classify Motor Imagery Signals,” _IEEE Access_ , vol. 7, pp. 15 945–15 954, 2019.
  * [46] D. L. Li, J. H. Wang, J. C. Xu, and X. K. Fang, “Densely Feature Fusion Based on Convolutional Neural Networks for Motor Imagery EEG Classification,” _IEEE Access_ , vol. 7, pp. 132 720–132 730, 2019.
  * [47] R. Alazrai, M. Abuhijleh, H. Alwanni, and M. I. Daoud, “A Deep Learning Framework for Decoding Motor Imagery Tasks of the Same Hand Using EEG Signals,” _IEEE Access_ , vol. 7, pp. 109 612–109 627, 2019.
  * [48] G. B. Kshirsagar and N. D. Londhe, “Improving Performance of Devanagari Script Input-Based P300 Speller Using Deep Learning,” _IEEE Transactions on Biomedical Engineering_ , vol. 66, no. 11, pp. 2992–3005, 2019\.
  * [49] V. J. Lawhern, A. J. Solon, N. R. Waytowich, S. M. Gordon, C. P. Hung, and B. J. Lance, “EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces,” _Journal of Neural Engineering_ , vol. 15, no. 5, 2018.
  * [50] D. Y. Pei, M. Burns, R. Chandramouli, and R. Vinjamuri, “Decoding Asynchronous Reaching in Electroencephalography Using Stacked Autoencoders,” _IEEE Access_ , vol. 6, pp. 52 889–52 898, 2018.
  * [51] S. Kundu and S. Ari, “P300 based character recognition using sparse autoencoder with ensemble of SVMs,” _Biocybernetics and Biomedical Engineering_ , vol. 39, no. 4, pp. 956–966, 2019.
  * [52] Y. R. Tabar and U. Halici, “A novel deep learning approach for classification of EEG motor imagery signals,” _Journal of Neural Engineering_ , vol. 14, no. 1, 2017.
  * [53] S. U. Amin, M. Alsulaiman, G. Muhammad, M. A. Mekhtiche, and M. S. Hossain, “Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion,” _Future Generation Computer Systems-the International Journal of Escience_ , vol. 101, pp. 542–554, 2019.
  * [54] R. L. Zhang, Q. Zong, L. Q. Dou, and X. Y. Zhao, “A novel hybrid deep learning scheme for four-class motor imagery classification,” _Journal of Neural Engineering_ , vol. 16, no. 6, 2019.
  * [55] J. Yang, S. W. Yao, and J. Wang, “Deep Fusion Feature Learning Network for MI-EEG Classification,” _IEEE Access_ , vol. 6, pp. 79 050–79 059, 2018\.
  * [56] T. Ma, H. Li, H. Yang, X. L. Lv, P. Y. Li, T. J. Liu, D. Z. Yao, and P. Xu, “The extraction of motion-onset VEP BCI features based on deep learning and compressed sensing,” _Journal of Neuroscience Methods_ , vol. 275, pp. 80–92, 2017.
  * [57] T. H. Nguyen and W. Y. Chung, “A Single-Chanel SSVEP-Based BCI Speller Using Deep Learning,” _IEEE Access_ , vol. 7, pp. 1752–1763, 2019.
  * [58] N. Waytowich, V. J. Lawhern, J. O. Garcia, J. Cummings, J. Faller, P. Sajda, and J. M. Vettel, “Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials,” _Journal of Neural Engineering_ , vol. 15, no. 6, 2018.
  * [59] U. R. Acharya, S. L. Oh, Y. Hagiwara, J. H. Tan, and H. Adeli, “Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals,” _Computers in Biology and Medicine_ , vol. 100, pp. 270–278, 2018.
  * [60] S. L. Oh, J. Vicnesh, E. J. Ciaccio, R. Yuvaraj, and U. R. Acharya, “Deep Convolutional Neural Network Model for Automated Diagnosis of Schizophrenia Using EEG Signals,” _Applied Sciences-Basel_ , vol. 9, no. 14, 2019.
  * [61] A. Antoniades, L. Spyrou, D. Martin-Lopez, A. Valentin, G. Alarcon, S. Sanei, and C. C. Took, “Detection of Interictal Discharges With Convolutional Neural Networks Using Discrete Ordered Multichannel Intracranial EEG,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 25, no. 12, pp. 2285–2294, 2017.
  * [62] T. X. Wen and Z. N. Zhang, “Deep Convolution Neural Network and Autoencoders-Based Unsupervised Feature Learning of EEG Signals,” _IEEE Access_ , vol. 6, pp. 25 399–25 410, 2018.
  * [63] H. Daoud and M. A. Bayoumi, “Efficient Epileptic Seizure Prediction Based on Deep Learning,” _IEEE Transactions on Biomedical Circuits and Systems_ , vol. 13, no. 5, pp. 804–813, 2019.
  * [64] W. Mumtaz and A. Qayyum, “A deep learning framework for automatic diagnosis of unipolar depression,” _International Journal of Medical Informatics_ , vol. 132, 2019.
  * [65] Y. Qiu, W. D. Zhou, N. N. Yu, and P. D. Du, “Denoising Sparse Autoencoder-Based Ictal PEG Classification,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 26, no. 9, pp. 1717–1726, 2018.
  * [66] H. Chen, Y. Song, and X. L. Li, “A deep learning framework for identifying children with ADHD using an EEG-based brain network,” _Neurocomputing_ , vol. 356, pp. 83–96, 2019.
  * [67] Y. Yuan, G. X. Xun, K. B. Jia, and A. D. Zhang, “A Multi-View Deep Learning Framework for EEG Seizure Detection,” _IEEE Journal of Biomedical and Health Informatics_ , vol. 23, no. 1, pp. 83–94, 2019.
  * [68] C. A. T. Naira and C. J. L. Del Alamo, “Classification of People who Suffer Schizophrenia and Healthy People by EEG Signals using Deep Learning,” _International Journal of Advanced Computer Science and Applications_ , vol. 10, no. 10, pp. 511–516, 2019.
  * [69] G. Vrbancic and V. Podgorelec, “Automatic Classification of Motor Impairment Neural Disorders from EEG Signals Using Deep Convolutional Neural Networks,” _Elektronika Ir Elektrotechnika_ , vol. 24, no. 4, 2018.
  * [70] O. Turk and M. S. Ozerdem, “Epilepsy Detection by Using Scalogram Based Convolutional Neural Network from EEG Signals,” _Brain Sciences_ , vol. 9, no. 5, 2019.
  * [71] X. Y. Wei, L. Zhou, Z. Y. Chen, L. J. Zhang, and Y. Zhou, “Automatic seizure detection using three-dimensional CNN based on multi-channel EEG,” _Bmc Medical Informatics and Decision Making_ , vol. 18, 2018.
  * [72] X. B. Tian, Z. H. Deng, W. H. Ying, K. S. Choi, D. R. Wu, B. Qin, J. Wang, H. B. Shen, and S. T. Wang, “Deep Multi-View Feature Learning for EEG-Based Epileptic Seizure Detection,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 27, no. 10, pp. 1962–1972, 2019.
  * [73] R. San-Segundo, M. Gil-Martin, L. F. D’Haro-Enriquez, and J. M. Pardo, “Classification of epileptic EEG recordings using signal transforms and convolutional neural networks,” _Computers in Biology and Medicine_ , vol. 109, pp. 148–158, 2019.
  * [74] I. Ullah, M. Hussain, E. U. Qazi, and H. Aboalsamh, “An automated system for epilepsy detection using EEG brain signals based on deep learning approach,” _Expert Systems with Applications_ , vol. 107, pp. 61–71, 2018\.
  * [75] T. Zhang, W. M. Zheng, Z. Cui, Y. Zong, and Y. Li, “Spatial-Temporal Recurrent Neural Network for Emotion Recognition,” _IEEE Transactions on Cybernetics_ , vol. 49, no. 3, pp. 839–847, 2019.
  * [76] W. L. Zheng, W. Liu, Y. F. Lu, B. L. Lu, and A. Cichocki, “EmotionMeter: A Multimodal Framework for Recognizing Human Emotions,” _IEEE Transactions on Cybernetics_ , vol. 49, no. 3, pp. 1110–1122, 2019.
  * [77] A. Al-Nafjan, A. Al-Wabil, M. Hosny, and Y. Al-Ohali, “Classification of Human Emotions from Electroencephalogram (EEG) Signal using Deep Neural Network,” _International Journal of Advanced Computer Science and Applications_ , vol. 8, no. 9, pp. 419–425, 2017.
  * [78] Y. H. Kwon, S. B. Shin, and S. D. Kim, “Electroencephalography Based Fusion Two-Dimensional (2D)-Convolution Neural Networks (CNN) Model for Emotion Recognition System,” _Sensors_ , vol. 18, no. 5, 2018.
  * [79] W.-L. Zheng and B.-L. Lu, “Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks,” _IEEE Transactions on Autonomous Mental Development_ , vol. 7, no. 3, pp. 162–175, 2015\.
  * [80] S. Koelstra, C. Muhl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras, “Deap: A database for emotion analysis; using physiological signals,” _IEEE transactions on affective computing_ , vol. 3, no. 1, pp. 18–31, 2011.
  * [81] J. P. Li, Z. X. Zhang, and H. G. He, “Hierarchical Convolutional Neural Networks for EEG-Based Emotion Recognition,” _Cognitive Computation_ , vol. 10, no. 2, pp. 368–380, 2018.
  * [82] H. Zeng, Z. H. Wu, J. M. Zhang, C. Yang, H. Zhang, G. J. Dai, and W. Z. Kong, “EEG Emotion Classification Using an Improved SincNet-Based Deep Learning Model,” _Brain Sciences_ , vol. 9, no. 11, 2019.
  * [83] M. Ravanelli and Y. Bengio, “Speaker recognition from raw waveform with sincnet,” _2018 IEEE Spoken Language Technology Workshop (SLT)_ , pp. 1021–1028, 2018.
  * [84] E. J. Choi and D. K. Kim, “Arousal and Valence Classification Model Based on Long Short-Term Memory and DEAP Data for Mental Healthcare Management,” _Healthcare Informatics Research_ , vol. 24, no. 4, pp. 309–316, 2018.
  * [85] S. Alhagry, A. A. Fahmy, and R. A. El-Khoribi, “Emotion Recognition based on EEG using LSTM Recurrent Neural Network,” _International Journal of Advanced Computer Science and Applications_ , vol. 8, no. 10, pp. 355–358, 2017\.
  * [86] S. Bagherzadeh, K. Maghooli, J. Farhadi, and M. Z. Soroush, “Emotion Recognition from Physiological Signals Using Parallel Stacked Autoencoders,” _Neurophysiology_ , vol. 50, no. 6, pp. 428–435, 2018.
  * [87] X. Li, D. W. Song, P. Zhang, Y. X. Hou, and B. Hu, “Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring,” _International Journal of Data Mining and Bioinformatics_ , vol. 18, no. 1, pp. 1–27, 2017.
  * [88] D. J. Hemanth, J. Anitha, and L. H. Son, “Brain signal based human emotion analysis by circular back propagation and Deep Kohonen Neural Networks,” _Computers & Electrical Engineering_, vol. 68, pp. 170–180, 2018.
  * [89] S. Jirayucharoensak, S. Pan-Ngum, and P. Israsena, “EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation,” _Scientific World Journal_ , 2014.
  * [90] Z. K. Gao, Y. L. Li, Y. X. Yang, X. M. Wang, N. Dong, and H. D. Chiang, “A GPSO-optimized convolutional neural networks for EEG-based emotion recognition,” _Neurocomputing_ , vol. 380, pp. 225–235, 2020.
  * [91] Z. Yin and J. H. Zhang, “Task-generic mental fatigue recognition based on neurophysiological signals and dynamical deep extreme learning machine,” _Neurocomputing_ , vol. 283, pp. 266–281, 2018.
  * [92] Y. L. Ma, B. Chen, R. H. Li, C. S. Wang, J. Wang, Q. S. She, Z. Z. Luo, and Y. C. Zhang, “Driving Fatigue Detection from EEG Using a Modified PCANet Method,” _Computational Intelligence and Neuroscience_ , vol. 2019, 2019\.
  * [93] H. Zeng, C. Yang, G. J. Dai, F. W. Qin, J. H. Zhang, and W. Z. Kong, “EEG classification of driver mental states by deep learning,” _Cognitive Neurodynamics_ , vol. 12, no. 6, pp. 597–606, 2018.
  * [94] Z. K. Gao, X. M. Wang, Y. X. Yang, C. X. Mu, Q. Cai, W. D. Dang, and S. Y. Zuo, “EEG-Based Spatio-Temporal Convolutional Neural Network for Driver Fatigue Evaluation,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 30, no. 9, pp. 2755–2763, 2019.
  * [95] R. F. Chai, S. H. Ling, P. P. San, G. R. Naik, T. N. Nguyen, Y. Tran, A. Craig, and H. T. Nguyen, “Improving EEG-Based Driver Fatigue Classification Using Sparse-Deep Belief Networks,” _Frontiers in Neuroscience_ , vol. 11, 2017\.
  * [96] E. Q. Wu, X. Y. Peng, C. Z. Z. Zhang, J. X. Lin, and R. S. F. Sheng, “Pilots’ Fatigue Status Recognition Using Deep Contractive Autoencoder Network,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 68, no. 10, pp. 3907–3919, 2019.
  * [97] J. D. Tao, Z. Yin, L. Liu, Y. Tian, Z. Q. Sun, and J. H. Zhang, “Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling,” _Symmetry-Basel_ , vol. 11, no. 7, 2019.
  * [98] J. H. Zhang and S. A. Li, “A deep learning scheme for mental workload classification based on restricted Boltzmann machines,” _Cognition Technology & Work_, vol. 19, no. 4, pp. 607–631, 2017.
  * [99] R. Hefron, B. Borghetti, C. S. Kabban, J. Christensen, and J. Estepp, “Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks,” _Sensors_ , vol. 18, no. 5, 2018.
  * [100] Z. Yin and J. H. Zhang, “Cross-subject recognition of operator functional states via EEG and switching deep belief networks with adaptive weights,” _Neurocomputing_ , vol. 260, pp. 349–366, 2017.
  * [101] P. B. Zhang, X. Wang, W. H. Zhang, and J. F. Chen, “Learning Spatial-Spectral-Temporal EEG Features With Recurrent 3D Convolutional Neural Networks for Cross-Task Mental Workload Assessment,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 27, no. 1, pp. 31–42, 2019.
  * [102] Z. Yin, M. Y. Zhao, W. Zhang, Y. X. Wang, Y. G. Wang, and J. H. Zhang, “Physiological-signal-based mental workload estimation via transfer dynamical autoencoders in a deep learning framework,” _Neurocomputing_ , vol. 347, pp. 212–229, 2019.
  * [103] P. B. Zhang, X. Wang, J. F. Chen, W. You, and W. H. Zhang, “Spectral and Temporal Feature Learning With Two-Stream Neural Networks for Mental Workload Assessment,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 27, no. 6, pp. 1149–1159, 2019.
  * [104] H. Dong, A. Supratak, W. Pan, C. Wu, P. M. Matthews, and Y. Guo, “Mixed Neural Network Approach for Temporal Sleep Stage Classification,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 26, no. 2, pp. 324–333, 2018.
  * [105] E. Bresch, U. Grossekathofer, and G. Garcia-Molina, “Recurrent Deep Neural Networks for Real-Time Sleep Stage Classification From Single Channel EEG,” _Frontiers in Computational Neuroscience_ , vol. 12, 2018.
  * [106] P. M. Kulkarni, Z. D. Xiao, E. J. Robinson, A. S. Jami, J. P. Zhang, H. C. Zhou, S. E. Henin, A. A. Liu, R. S. Osorio, J. Wang, and Z. Chen, “A deep learning approach for real-time detection of sleep spindles,” _Journal of Neural Engineering_ , vol. 16, no. 3, 2019.
  * [107] O. Tsinalis, P. M. Matthews, and Y. K. Guo, “Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders,” _Annals of Biomedical Engineering_ , vol. 44, no. 5, pp. 1587–1597, 2016\.
  * [108] S. Hartmann and M. Baumert, “Automatic A-Phase Detection of Cyclic Alternating Patterns in Sleep Using Dynamic Temporal Information,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 27, no. 9, pp. 1695–1703, 2019.
  * [109] J. M. Zhang, R. X. Yao, W. G. Ge, and J. F. Gao, “Orthogonal convolutional neural networks for automatic sleep stage classification based on single-channel EEG,” _Computer Methods and Programs in Biomedicine_ , vol. 183, 2020.
  * [110] Y. Jeon, S. Kim, H. S. Choi, Y. G. Chung, S. A. Choi, H. Kim, S. Yoon, H. Hwang, and K. J. Kim, “Pediatric Sleep Stage Classification Using Multi-Domain Hybrid Neural Networks,” _IEEE Access_ , vol. 7, pp. 96 495–96 505, 2019.
  * [111] J. M. Zhang and Y. Wu, “Complex-valued unsupervised convolutional neural networks for sleep stage classification,” _Computer Methods and Programs in Biomedicine_ , vol. 164, pp. 181–191, 2018.
  * [112] J. M. Zhang, Y. Wu, J. Bai, and F. Q. Chen, “Automatic sleep stage classification based on sparse deep belief net and combination of multiple classifiers,” _Transactions of the Institute of Measurement and Control_ , vol. 38, no. 4, pp. 435–451, 2016.
  * [113] Z. Mousavi, T. Y. Rezaii, S. Sheykhivand, A. Farzamnia, and S. N. Razavi, “Deep convolutional neural network for classification of sleep stages from single-channel EEG signals,” _Journal of Neuroscience Methods_ , vol. 324, 2019.
  * [114] P. Chriskos, C. A. Frantzidis, P. T. Gkivogkli, P. D. Bamidis, and C. Kourtidou-Papadeli, “Automatic Sleep Staging Employing Convolutional Neural Networks and Cortical Connectivity Images,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 31, no. 1, pp. 113–123, 2020.
  * [115] O. Ozdenizci, Y. Wang, T. Koike-Akino, and D. Erdogmus, “Adversarial Deep Learning in EEG Biometrics,” _IEEE Signal Processing Letters_ , vol. 26, no. 5, pp. 710–714, 2019.
  * [116] M. Wang, H. El-Fiqi, J. K. Hu, and H. A. Abbass, “Convolutional Neural Networks Using Dynamic Functional Connectivity for EEG-Based Person Identification in Diverse Human States,” _IEEE Transactions on Information Forensics and Security_ , vol. 14, no. 12, pp. 3259–3272, 2019.
  * [117] P. Kaushik, A. Gupta, P. P. Roy, and D. P. Dogra, “EEG-Based Age and Gender Prediction Using Deep BLSTM-LSTM Network Model,” _IEEE Sensors Journal_ , vol. 19, no. 7, pp. 2634–2641, 2019.
  * [118] N. Huang, M. Slaney, and M. Elhilali, “Connecting Deep Neural Networks to Physical, Perceptual, and Electrophysiological Auditory Signals,” _Frontiers in Neuroscience_ , vol. 12, 2018.
  * [119] M. X. Yu, Y. C. Sun, B. F. Zhu, L. Q. Zhu, Y. Z. Lin, X. Y. Tang, Y. K. Guo, G. K. Sun, and M. L. Dong, “Diverse frequency band-based convolutional neural networks for tonic cold pain assessment using EEG,” _Neurocomputing_ , vol. 378, pp. 270–282, 2020.
  * [120] J. M. Jiang, A. Fares, and S. H. Zhong, “A Context-Supported Deep Learning Framework for Multimodal Brain Imaging Classification,” _IEEE Transactions on Human-Machine Systems_ , vol. 49, no. 6, pp. 611–622, 2019.
  * [121] X. Zheng, W. Z. Chen, M. Y. Li, T. Zhang, Y. You, and Y. Jiang, “Decoding human brain activity with deep learning,” _Biomedical Signal Processing and Control_ , vol. 56, 2020.
  * [122] C. C. Hua, H. Wang, H. Wang, S. W. Lu, C. Liu, and S. M. Khalid, “A Novel Method of Building Functional Brain Network Using Deep Learning Algorithm with Application in Proficiency Detection,” _International Journal of Neural Systems_ , vol. 29, no. 1, 2019.
  * [123] S. Toraman, S. A. Tuncer, and F. Balgetir, “Is it possible to detect cerebral dominance via EEG signals by using deep learning?” _Medical Hypotheses_ , vol. 131, 2019.
  * [124] V. Baltatzis, K. M. Bintsi, G. K. Apostolidis, and L. J. Hadjileontiadis, “Bullying incidences identification within an immersive environment using HD EEG-based analysis: A Swarm Decomposition and Deep Learning approach,” _Scientific Reports_ , vol. 7, 2017.
  * [125] Z. G. Doborjeh, N. Kasabov, M. G. Doborjeh, and A. Sumich, “Modelling Peri-Perceptual Brain Processes in a Deep Learning Spiking Neural Network Architecture,” _Scientific Reports_ , vol. 8, 2018.
  * [126] A. Arora, J. J. Lin, A. Gasperian, J. Maldjian, J. Stein, M. Kahana, and B. Lega, “Comparison of logistic regression, support vector machines, and deep learning classifiers for predicting memory encoding success using human intracranial EEG recordings,” _Journal of Neural Engineering_ , vol. 15, no. 6, 2018.
  * [127] L. Shao, F. Zhu, and X. Li, “Transfer learning for visual categorization: A survey,” _IEEE transactions on neural networks and learning systems_ , vol. 26, no. 5, pp. 1019–1034, 2014.
  * [128] T. K. K. Ho, J. Gwak, C. M. Park, and J. I. Song, “Discrimination of Mental Workload Levels From Multi-Channel fNIRS Using Deep Leaning-Based Approaches,” _IEEE Access_ , vol. 7, pp. 24 392–24 403, 2019.
  * [129] S. Sabour, N. Frosst, and G. E. Hinton, “Dynamic routing between capsules,” _Advances in neural information processing systems_ , pp. 3856–3866, 2017\.
  * [130] T. Wang, A. Bezerianos, A. Cichocki, and J. Li, “Multi-kernel capsule network for schizophrenia identification,” _arXiv preprint arXiv:1907.12827_ , 2019\.
  * [131] S. K. Goh, H. A. Abbass, K. C. Tan, A. Al-Mamun, N. Thakor, A. Bezerianos, and J. H. Li, “Spatio-Spectral Representation Learning for Electroencephalographic Gait-Pattern Classification,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 26, no. 9, pp. 1858–1867, 2018.
  * [132] X. L. Ma, S. Qiu, W. Wei, S. P. Wang, and H. G. He, “Deep Channel-Correlation Network for Motor Imagery Decoding From the Same Limb,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 28, no. 1, pp. 297–306, 2020.
  * [133] X. Y. Zhu, P. Y. Li, C. B. Li, D. Z. Yao, R. Zhang, and P. Xu, “Separated channel convolutional neural network to realize the training free motor imagery BCI systems,” _Biomedical Signal Processing and Control_ , vol. 49, pp. 396–403, 2019.
  * [134] A. M. Chiarelli, P. Croce, A. Merla, and F. Zappasodi, “Deep learning for hybrid EEG-fNIRS brain-computer interface: application to motor imagery classification,” _Journal of Neural Engineering_ , vol. 15, no. 3, 2018\.
  * [135] Z. Tayeb, J. Fedjaev, N. Ghaboosi, C. Richter, L. Everding, X. W. Qu, Y. Y. Wu, G. Cheng, and J. Conradt, “Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals,” _Sensors_ , vol. 19, no. 1, 2019.
  * [136] M. X. Dai, D. Z. Zheng, R. Na, S. Wang, and S. L. Zhang, “EEG Classification of Motor Imagery Using a Novel Deep Learning Framework,” _Sensors_ , vol. 19, no. 3, 2019.
  * [137] K. W. Ha and J. W. Jeong, “Motor Imagery EEG Classification Using Capsule Networks,” _Sensors_ , vol. 19, no. 13, 2019.
  * [138] T. W. Shi, L. Ren, and W. H. Cui, “Feature recognition of motor imaging EEG signals based on deep learning,” _Personal and Ubiquitous Computing_ , vol. 23, no. 3-4, pp. 499–510, 2019.
  * [139] Z. J. Wang, L. Cao, Z. Zhang, X. L. Gong, Y. R. Sun, and H. R. Wang, “Short time Fourier transformation and deep neural networks for motor imagery brain computer interface recognition,” _Concurrency and Computation-Practice & Experience_, vol. 30, no. 23, 2018.
  * [140] S. U. Amin, M. Alsulaiman, G. Muhammad, M. A. Bencherif, and M. S. Hossain, “Multilevel Weighted Feature Fusion Using Convolutional Neural Networks for EEG Motor Imagery Classification,” _IEEE Access_ , vol. 7, pp. 18 940–18 950, 2019.
  * [141] R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, M. Glasstetter, K. Eggensperger, M. Tangermann, F. Hutter, W. Burgard, and T. Ball, “Deep learning with convolutional neural networks for eeg decoding and visualization,” _Human brain mapping_ , vol. 38, no. 11, pp. 5391–5420, 2017\.
  * [142] A. Hassanpour, M. Moradikia, H. Adeli, S. R. Khayami, and P. Shamsinejadbabaki, “A novel end-to-end deep learning scheme for classifying multi-class motor imagery electroencephalography signals,” _Expert Systems_.
  * [143] Q. S. She, B. Hu, Z. Z. Luo, T. Nguyen, and Y. C. Zhang, “A hierarchical semi-supervised extreme learning machine method for EEG recognition,” _Medical & Biological Engineering & Computing_, vol. 57, no. 1, pp. 147–157, 2019.
  * [144] L. F. S. Uribe, C. A. Stefano, V. A. de Oliveira, T. B. D. Costa, P. G. Rodrigues, D. C. Soriano, L. Boccato, G. Castellano, and R. Attux, “A correntropy-based classifier for motor imagery brain-computer interfaces,” _Biomedical Physics & Engineering Express_, vol. 5, no. 6, 2019.
  * [145] L. J. Duan, M. H. Bao, S. Cui, Y. H. Qiao, and J. Miao, “Motor Imagery EEG Classification Based on Kernel Hierarchical Extreme Learning Machine,” _Cognitive Computation_ , vol. 9, no. 6, pp. 758–765, 2017.
  * [146] H. Wu, Y. Niu, F. Li, Y. C. Li, B. X. Fu, G. M. Shi, and M. H. Dong, “A Parallel Multiscale Filter Bank Convolutional Neural Networks for Motor Imagery EEG Classification,” _Frontiers in Neuroscience_ , vol. 13, 2019\.
  * [147] I. Majidov and T. Whangbo, “Efficient Classification of Motor Imagery Electroencephalography Signals Using Deep Learning Methods,” _Sensors_ , vol. 19, no. 7, 2019.
  * [148] X. L. Tang, W. C. Ma, D. S. Kong, and W. Li, “Semisupervised Deep Stacking Network with Adaptive Learning Rate Strategy for Motor Imagery EEG Recognition,” _Neural Computation_ , vol. 31, no. 5, pp. 919–942, 2019\.
  * [149] B. G. Xu, L. L. Zhang, A. G. Song, C. C. Wu, W. L. Li, D. L. Zhang, G. Z. Xu, H. J. Li, and H. Zeng, “Wavelet Transform Time-Frequency Image and Convolutional Network-Based Motor Imagery EEG Classification,” _IEEE Access_ , vol. 7, pp. 6084–6093, 2019.
  * [150] W. Gao, J. A. Guan, J. F. Gao, and D. Zhou, “Multi-ganglion ANN based feature learning with application to P300-BCI signal classification,” _Biomedical Signal Processing and Control_ , vol. 18, pp. 127–137, 2015.
  * [151] A. Farahat, C. Reichert, C. M. Sweeney-Reed, and H. Hinrichs, “Convolutional neural networks for decoding of covert attention focus and saliency maps for EEG feature visualization,” _Journal of Neural Engineering_ , vol. 16, no. 6, 2019.
  * [152] A. J. Solon, V. J. Lawhern, J. Touryan, J. R. McDaniel, A. J. Ries, and S. M. Gordon, “Decoding P300 Variability Using Convolutional Neural Networks,” _Frontiers in Human Neuroscience_ , vol. 13, 2019.
  * [153] L. Vareka and P. Mautner, “Stacked Autoencoders for the P300 Component Detection,” _Frontiers in Neuroscience_ , vol. 11, 2017.
  * [154] S. Morabbi, M. Keyvanpour, and S. V. Shojaedini, “A new method for P300 detection in deep belief networks: Nesterov momentum and drop based learning rate,” _Health and Technology_ , vol. 9, no. 4, pp. 615–630, 2019.
  * [155] A. Ditthapron, N. Banluesombatkul, S. Ketrat, E. Chuangsuwanich, and T. Wilaiprasitporn, “Universal Joint Feature Extraction for P300 EEG Classification Using Multi-Task Autoencoder,” _IEEE Access_ , vol. 7, pp. 68 415–68 428, 2019.
  * [156] L. Citi, R. Poli, and C. Cinel, “Documenting, modelling and exploiting p300 amplitude changes due to variable target delays in donchin’s speller,” _Journal of Neural Engineering_ , vol. 7, no. 5, p. 056006, 2010.
  * [157] M. Schreuder, B. Blankertz, and M. Tangermann, “A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue,” _PloS one_ , vol. 5, no. 4, 2010.
  * [158] L. Acqualagna and B. Blankertz, “Gaze-independent bci-spelling using rapid serial visual presentation (rsvp),” _Clinical Neurophysiology_ , vol. 124, no. 5, pp. 901–908, 2013.
  * [159] M. S. Treder, H. Purwins, D. Miklody, I. Sturm, and B. Blankertz, “Decoding auditory attention to instruments in polyphonic music using single-trial eeg classification,” _Journal of neural engineering_ , vol. 11, no. 2, p. 026009, 2014.
  * [160] B. Boloukian and F. Safi-Esfahani, “Recognition of words from brain-generated signals of speech-impaired people: Application of autoencoders as a neural Turing machine controller in deep neural networks,” _Neural Networks_ , vol. 121, pp. 186–207, 2020.
  * [161] U. Hoffmann, J.-M. Vesin, T. Ebrahimi, and K. Diserens, “An efficient p300-based brain–computer interface for disabled subjects,” _Journal of Neuroscience methods_ , vol. 167, no. 1, pp. 115–125, 2008.
  * [162] J. X. Chen, Z. J. Mao, R. Zheng, Y. F. Huang, and L. F. He, “Feature Selection of Deep Learning Models for EEG-Based RSVP Target Detection,” _Ieice Transactions on Information and Systems_ , vol. E102D, no. 4, pp. 836–844, 2019.
  * [163] J. Touryan, G. Apker, S. Kerick, B. Lance, A. J. Ries, and K. McDowell, “Translation of eeg-based performance prediction models to rapid serial visual presentation tasks,” _International Conference on Augmented Cognition_ , pp. 521–530, 2013.
  * [164] R. Manor and A. B. Geva, “Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI,” _Frontiers in Computational Neuroscience_ , vol. 9, 2015.
  * [165] R. Manor, L. Mishali, and A. B. Geva, “Multimodal Neural Network for Rapid Serial Visual Presentation Brain Computer Interface,” _Frontiers in Computational Neuroscience_ , vol. 10, 2016.
  * [166] Q. Q. Liu, Y. Jiao, Y. Y. Miao, C. L. Zuo, X. Y. Wang, A. Cichocki, and J. Jin, “Efficient representations of EEG signals for SSVEP frequency recognition based on deep multiset CCA,” _Neurocomputing_ , vol. 378, pp. 36–44, 2020\.
  * [167] M. Nakanishi, Y. Wang, Y.-T. Wang, and T.-P. Jung, “A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials,” _PloS one_ , vol. 10, no. 10, 2015.
  * [168] X. J. Bi and H. B. Wang, “Early Alzheimer’s disease diagnosis based on EEG spectral images using deep learning,” _Neural Networks_ , vol. 114, pp. 119–135, 2019.
  * [169] F. C. Morabito, M. Campolo, N. Mammone, M. Versaci, S. Franceschetti, F. Tagliavini, V. Sofia, D. Fatuzzo, A. Gambardella, A. Labate, L. Mumoli, G. G. Tripodi, S. Gasparini, V. Cianci, C. Sueri, E. Ferlazzo, and U. Aguglia, “Deep Learning Representation from Electroencephalography of Early-Stage Creutzfeldt-Jakob Disease and Features for Differentiation from Rapidly Progressive Dementia,” _International Journal of Neural Systems_ , vol. 27, no. 2, 2017.
  * [170] K. Hayase, K. Hayashi, and T. Sawa, “Hierarchical Poincare analysis for anaesthesia monitoring,” _Journal of Clinical Monitoring and Computing_.
  * [171] Q. Liu, J. F. Cai, S. Z. Fan, M. F. Abbod, J. S. Shieh, Y. C. Kung, and L. S. Lin, “Spectrum Analysis of EEG Signals Using CNN to Model Patient’s Consciousness Level Based on Anesthesiologists’ Experience,” _IEEE Access_ , vol. 7, pp. 53 731–53 742, 2019.
  * [172] S. Kim, J. Kim, and H. W. Chun, “Wave2Vec: Vectorizing Electroencephalography Bio-Signal for Prediction of Brain Disease,” _International Journal of Environmental Research and Public Health_ , vol. 15, no. 8, 2018.
  * [173] H. Chen, Y. Song, and X. L. Li, “Use of deep learning to detect personalized spatial-frequency abnormalities in EEGs of children with ADHD,” _Journal of Neural Engineering_ , vol. 16, no. 6, 2019.
  * [174] R. Boshra, K. I. Ruiter, C. DeMatteo, J. P. Reilly, and J. F. Connolly, “Neurophysiological Correlates of Concussion: Deep Learning for Clinical Assessment,” _Scientific Reports_ , vol. 9, 2019.
  * [175] H. Q. Sun, E. Kimchi, O. Akeju, S. B. Nagaraj, L. M. McClain, D. W. Zhou, E. Boyle, W. L. Zheng, W. D. Ge, and M. B. Westover, “Automated tracking of level of consciousness and delirium in critical illness using deep learning,” _Npj Digital Medicine_ , vol. 2, 2019.
  * [176] B. Ay, O. Yildirim, M. Talo, U. B. Baloglu, G. Aydin, S. D. Puthankattil, and U. R. Acharya, “Automated Depression Detection Using Deep Representation and Sequence Learning with EEG Signals,” _Journal of Medical Systems_ , vol. 43, no. 7, 2019.
  * [177] U. R. Acharya, S. L. Oh, Y. Hagiwara, J. H. Tan, H. Adeli, and D. P. Subha, “Automated eeg-based screening of depression using deep convolutional neural network,” _Computer methods and programs in biomedicine_ , vol. 161, pp. 103–113, 2018.
  * [178] U. R. Acharya, S. Oh, Y. Hagiwara, J. H. Tan, H. Adeli, and D. P. Subha, “Automated EEG-based screening of depression using deep convolutional neural network,” _Computer Methods and Programs in Biomedicine_ , vol. 161, pp. 103–113, 2018.
  * [179] X. W. Li, X. Zhang, J. Zhu, W. D. Mao, S. T. Sun, Z. H. Wang, C. Xia, and B. Hu, “Depression recognition using machine learning methods with different feature generation strategies,” _Artificial Intelligence in Medicine_ , vol. 99, 2019.
  * [180] J. Zhu, Y. Wang, R. La, J. W. Zhan, J. H. Niu, S. Zeng, and X. P. Hu, “Multimodal Mild Depression Recognition Based on EEG-EM Synchronization Acauisition Network,” _IEEE Access_ , vol. 7, pp. 28 196–28 210, 2019\.
  * [181] A. M. Karim, M. S. Guzel, M. R. Tolun, H. Kaya, and F. V. Celebi, “A new framework using deep auto-encoder and energy spectral density for medical waveform data classification and processing,” _Biocybernetics and Biomedical Engineering_ , vol. 39, no. 1, pp. 148–159, 2019.
  * [182] R. G. Andrzejak, K. Lehnertz, F. Mormann, C. Rieke, P. David, and C. E. Elger, “Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state,” _Physical Review E_ , vol. 64, no. 6, p. 061907, 2001.
  * [183] D. K. Thara, B. G. PremaSudha, and F. Xiong, “Epileptic seizure detection and prediction using stacked bidirectional long short term memory,” _Pattern Recognition Letters_ , vol. 128, pp. 529–535, 2019.
  * [184] M. Abu Sayeed, S. P. Mohanty, E. Kougianos, and H. P. Zaveri, “Neuro-Detect: A Machine Learning-Based Fast and Accurate Seizure Detection System in the IoMT,” _IEEE Transactions on Consumer Electronics_ , vol. 65, no. 3, pp. 359–368, 2019.
  * [185] M. Stead, M. Bower, B. H. Brinkmann, K. Lee, W. R. Marsh, F. B. Meyer, B. Litt, J. Van Gompel, and G. A. Worrell, “Microseizures and the spatiotemporal scales of human partial epilepsy,” _Brain_ , vol. 133, no. 9, pp. 2789–2797, 2010.
  * [186] B. H. Brinkmann, M. R. Bower, K. A. Stengel, G. A. Worrell, and M. Stead, “Large-scale electrophysiology: acquisition, compression, encryption, and storage of big data,” _Journal of neuroscience methods_ , vol. 180, no. 1, pp. 185–192, 2009.
  * [187] R. Hussein, H. Palangi, R. K. Ward, and Z. J. Wang, “Optimized deep neural network architecture for robust detection of epileptic seizures using EEG signals,” _Clinical Neurophysiology_ , vol. 130, no. 1, pp. 25–37, 2019\.
  * [188] A. M. Abdelhameed and M. Bayoumi, “Semi-Supervised EEG Signals Classification System for Epileptic Seizure Detection,” _IEEE Signal Processing Letters_ , vol. 26, no. 12, pp. 1922–1926, 2019.
  * [189] R. Akut, “Wavelet based deep learning approach for epilepsy detection,” _Health Information Science and Systems_ , vol. 7, 2019.
  * [190] A. Emami, N. Kunii, T. Matsuo, T. Shinozaki, K. Kawai, and H. Takahashi, “Seizure detection by convolutional neural network-based analysis of scalp electroencephalography plot images,” _Neuroimage-Clinical_ , vol. 22, 2019\.
  * [191] U. B. Baloglu and O. Yildirim, “Convolutional Long-Short Term Memory Networks Model For Long Duration EEG Signal Classification,” _Journal of Mechanics in Medicine and Biology_ , vol. 19, no. 1, 2019.
  * [192] C. Jansen, S. Hodel, T. Penzel, M. Spott, and D. Krefting, “Feature relevance in physiological networks for classification o obstructive sleep apnea,” _Physiological Measurement_ , vol. 39, no. 12, 2018.
  * [193] G. Klosh, B. Kemp, T. Penzel, A. Schlogl, P. Rappelsberger, E. Trenker, G. Gruber, J. Zeithofer, B. Saletu, W. Herrmann _et al._ , “The siesta project polygraphic and clinical database,” _IEEE Engineering in Medicine and Biology Magazine_ , vol. 20, no. 3, pp. 51–57, 2001.
  * [194] S. Jonas, A. O. Rossetti, M. Oddo, S. Jenni, P. Favaro, and F. Zubler, “EEG-based outcome prediction after cardiac arrest with convolutional neural networks: Performance and visualization of discriminative features,” _Human Brain Mapping_ , vol. 40, no. 16, pp. 4606–4617, 2019.
  * [195] M. C. Tjepkema-Cloostermans, C. D. Lourenco, B. J. Ruijter, S. C. Tromp, G. Drost, F. H. M. Kornips, A. Beishuizen, F. H. Bosch, J. Hofmeijer, and M. van Putten, “Outcome Prediction in Postanoxic Coma With Deep Learning*,” _Critical Care Medicine_ , vol. 47, no. 10, pp. 1424–1432, 2019\.
  * [196] S. U. Amin, M. S. Hossain, G. Muhammad, M. Alhussein, and M. A. Rahman, “Cognitive Smart Healthcare for Pathology Detection and Monitoring,” _IEEE Access_ , vol. 7, pp. 10 745–10 753, 2019.
  * [197] G. Ruffini, D. Ibanez, M. Castellano, L. Dubreuil-Vall, A. Soria-Frisch, R. Postuma, J. F. Gagnon, and J. Montplaisir, “Deep Learning With EEG Spectrograms in Rapid Eye Movement Behavior Disorder,” _Frontiers in Neurology_ , vol. 10, 2019.
  * [198] A. Piryatinska, B. Darkhovsky, and A. Kaplan, “Binary classification of multichannel-eeg records based on the e-complexity of continuous vector functions,” _Computer methods and programs in biomedicine_ , vol. 152, pp. 131–139, 2017.
  * [199] Z. Yin, M. Y. Zhao, Y. X. Wang, J. D. Yang, and J. H. Zhang, “Recognition of emotions using multimodal physiological signals and an ensemble deep learning model,” _Computer Methods and Programs in Biomedicine_ , vol. 140, pp. 93–110, 2017.
  * [200] T. Chen, S. H. Ju, X. H. Yuan, M. Elhoseny, F. J. Ren, M. Y. Fan, and Z. G. Chen, “Emotion recognition using empirical mode decomposition and approximation entropy,” _Computers & Electrical Engineering_, vol. 72, pp. 383–392, 2018.
  * [201] H. Chao, H. L. Zhi, L. Dong, and Y. L. Liu, “Recognition of Emotions Using Multichannel EEG Data and DBN-GC-Based Ensemble Deep Learning Framework,” _Computational Intelligence and Neuroscience_ , 2018.
  * [202] S. K. Kim and H. B. Kang, “An analysis of smartphone overuse recognition in terms of emotions using brainwaves and deep learning,” _Neurocomputing_ , vol. 275, pp. 1393–1406, 2018.
  * [203] J. Teo, L. H. Chew, J. T. Chia, and J. Mountstephens, “Classification of Affective States via EEG and Deep Learning,” _International Journal of Advanced Computer Science and Applications_ , vol. 9, no. 5, pp. 132–142, 2018\.
  * [204] J. X. Chen, D. M. Jiang, and N. Zhang, “A Hierarchical Bidirectional GRU Model With Attention for EEG-Based Emotion Classification,” _IEEE Access_ , vol. 7, pp. 118 530–118 540, 2019.
  * [205] O. Balan, G. Moise, A. Moldoveanu, M. Leordeanu, and F. Moldoveanu, “Fear Level Classification Based on Emotional Dimensions and Machine Learning Techniques,” _Sensors_ , vol. 19, no. 7, 2019.
  * [206] J. H. Jeong, B. W. Yu, D. H. Lee, and S. W. Lee, “Classification of Drowsiness Levels Based on a Deep Spatio-Temporal Convolutional Bidirectional LSTM Network Using Electroencephalography Signals,” _Brain Sciences_ , vol. 9, no. 12, 2019.
  * [207] Z. Yin and J. H. Zhang, “Cross-session classification of mental workload levels using EEG and an adaptive deep learning model,” _Biomedical Signal Processing and Control_ , vol. 33, pp. 30–47, 2017.
  * [208] Z. C. Jiao, X. B. Gao, Y. Wang, J. Li, and H. J. Xu, “Deep Convolutional Neural Networks for mental load classification based on EEG data,” _Pattern Recognition_ , vol. 76, pp. 582–595, 2018.
  * [209] S. Yang, Z. Yin, Y. G. Wang, W. Zhang, Y. X. Wang, and J. H. Zhang, “Assessing cognitive mental workload via EEG signals and an ensemble deep learning classifier based on denoising autoencoders,” _Computers in Biology and Medicine_ , vol. 109, pp. 159–170, 2019.
  * [210] O. Yildirim, U. B. Baloglu, and U. R. Acharya, “A Deep Learning Model for Automated Sleep Stages Classification Using PSG Signals,” _International Journal of Environmental Research and Public Health_ , vol. 16, no. 4, 2019.
  * [211] A. Patanaik, J. L. Ong, J. J. Gooley, S. Ancoli-Israel, and M. W. L. Chee, “An end-to-end framework for real-time automatic sleep stage classification,” _Sleep_ , vol. 41, no. 5, 2018.
  * [212] Y. Yuan, K. B. Jia, F. L. Ma, G. X. Xun, Y. Q. Wang, L. Su, and A. D. Zhang, “A hybrid self-attention deep learning framework for multivariate sleep stage classification,” _Bmc Bioinformatics_ , vol. 20, 2019.
  * [213] L. D. Zhang, D. Fabbri, R. Upender, and D. Kent, “Automated sleep stage scoring of the Sleep Heart Health Study using deep neural networks,” _Sleep_ , vol. 42, no. 11, 2019.
  * [214] F. Chapotot and G. Becq, “Automated sleep-wake staging combining robust feature extraction, artificial neural network classification, and flexible decision rules,” _International Journal of Adaptive Control and Signal Processing_ , vol. 24, no. 5, pp. 409–423, 2010.
  * [215] A. Malafeev, D. Laptev, S. Bauer, X. Omlin, A. Wierzbicka, A. Wichniak, W. Jernajczyk, R. Riener, J. Buhmann, and P. Achermann, “Automatic Human Sleep Stage Scoring Using Deep Neural Networks,” _Frontiers in Neuroscience_ , vol. 12, 2018.
  * [216] H. Phan, F. Andreotti, N. Cooray, O. Y. Chen, and M. De Vos, “Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification,” _IEEE Transactions on Biomedical Engineering_ , vol. 66, no. 5, pp. 1285–1296, 2019.
  * [217] S. Chambon, M. N. Galtier, P. J. Arnal, G. Wainrib, and A. Gramfort, “A Deep Learning Architecture for Temporal Sleep Stage Classification Using Multivariate and Multimodal Time Series,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 26, no. 4, pp. 758–769, 2018.
  * [218] S. Biswal, H. Q. Sun, B. Goparaju, M. B. Westover, J. M. Sun, and M. T. Bianchi, “Expert-level sleep scoring with deep neural networks,” _Journal of the American Medical Informatics Association_ , vol. 25, no. 12, pp. 1643–1650, 2018.
  * [219] A. Sors, S. Bonnet, S. Mirek, L. Vercueil, and J. F. Payen, “A convolutional neural network for sleep stage scoring from raw single-channel EEG,” _Biomedical Signal Processing and Control_ , vol. 42, pp. 107–114, 2018.
  * [220] J. B. Blank, P. M. Cawthon, M. L. Carrion-Petersen, L. Harper, J. P. Johnson, E. Mitson, and R. R. Delay, “Overview of recruitment for the osteoporotic fractures in men study (mros),” _Contemporary clinical trials_ , vol. 26, no. 5, pp. 557–568, 2005.
  * [221] S. Redline, R. Amin, D. Beebe, R. D. Chervin, S. L. Garetz, B. Giordani, C. L. Marcus, R. H. Moore, C. L. Rosen, R. Arens _et al._ , “The childhood adenotonsillectomy trial (chat): rationale, design, and challenges of a randomized controlled trial evaluating a standard surgical procedure in a pediatric population,” _Sleep_ , vol. 34, no. 11, pp. 1509–1517, 2011.
  * [222] M. H. AlMeer, H. Hassen, and N. Nawaz, “ROM-based Inference Method Built on Deep Learning for Sleep Stage Classification,” _Tem Journal-Technology Education Management Informatics_ , vol. 8, no. 1, pp. 28–40, 2019.
  * [223] S. Charnbon, V. Thorey, P. J. Arnal, E. Mignot, and A. Gramfort, “DOSED: A deep learning approach to detect multiple sleep micro-events in EEG signal,” _Journal of Neuroscience Methods_ , vol. 321, pp. 64–78, 2019\.
  * [224] O. Andlauer, H. Moore, L. Jouhier, C. Drake, P. E. Peppard, F. Han, S.-C. Hong, F. Poli, G. Plazzi, R. O’Hara _et al._ , “Nocturnal rapid eye movement sleep latency for identifying patients with narcolepsy/hypocretin deficiency,” _JAMA neurology_ , vol. 70, no. 7, pp. 891–902, 2013.
  * [225] T. Young, L. Finn, P. E. Peppard, M. Szklo-Coxe, D. Austin, F. J. Nieto, R. Stubbs, and K. M. Hla, “Sleep disordered breathing and mortality: eighteen-year follow-up of the wisconsin sleep cohort,” _Sleep_ , vol. 31, no. 8, pp. 1071–1078, 2008.
  * [226] B. Kaur, D. Singh, and P. P. Roy, “Age and gender classification using brain–computer interface,” _Neural Computing and Applications_ , vol. 31, no. 10, pp. 5887–5900, 2019.
  * [227] D. F. Wulsin, J. R. Gupta, R. Mani, J. A. Blanco, and B. Litt, “Modeling electroencephalography waveforms with semi-supervised deep belief nets: fast classification and anomaly measurement,” _Journal of Neural Engineering_ , vol. 8, no. 3, 2011.
  * [228] J. Anem, G. S. Kumar, and R. Madhu, “Cat Swarm Fractional Calculus optimization-based deep learning for artifact removal from EEG signal,” _Journal of Experimental & Theoretical Artificial Intelligence_.
  * [229] S. Jacob, V. G. Menon, F. Al-Turjman, P. G. Vinoj, and L. Mostarda, “Artificial Muscle Intelligence System With Deep Learning for Post-Stroke Assistance and Rehabilitation,” _IEEE Access_ , vol. 7, pp. 133 463–133 473, 2019.
  * [230] B. H. Yang, K. W. Duan, C. C. Fan, C. X. Hu, and J. L. Wang, “Automatic ocular artifacts removal in EEG using deep learning,” _Biomedical Signal Processing and Control_ , vol. 43, pp. 148–158, 2018.
  * [231] C. Spampinato, S. Palazzo, I. Kavasidis, D. Giordano, N. Souly, and M. Shah, “Deep learning human mind for automated visual classification,” _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 6809–6817, 2017.
  * [232] P. Croce, F. Zappasodi, L. Marzetti, A. Merla, V. Pizzella, and A. M. Chiarelli, “Deep Convolutional Neural Networks for Feature-Less Automatic Classification of Independent Components in Multi-Channel Electrophysiological Brain Recordings,” _IEEE Transactions on Biomedical Engineering_ , vol. 66, no. 8, pp. 2372–2380, 2019.
  * [233] Y. R. Ming, W. P. Ding, D. Pelusi, D. R. Wu, Y. K. Wang, M. Prasad, and C. T. Lin, “Subject adaptation network for EEG data analysis,” _Applied Soft Computing_ , vol. 84, 2019.
  * [234] P. Nagabushanam, S. T. George, and S. Radha, “EEG signal classification using LSTM and improved neural network algorithms,” _Soft Computing_.
  * [235] A. Fares, S. H. Zhong, and J. M. Jiang, “EEG-based image classification via a region-level stacked bi-directional deep learning framework,” _Bmc Medical Informatics and Decision Making_ , vol. 19, 2019.
  * [236] H. Akbari, B. Khalighinejad, J. L. Herrero, A. D. Mehta, and N. Mesgarani, “Towards reconstructing intelligible speech from the human auditory cortex,” _Scientific Reports_ , vol. 9, 2019.
  * [237] A. Antoniades, L. Spyrou, D. Martin-Lopez, A. Valentin, G. Alarcon, S. Sanei, and C. C. Took, “Deep Neural Architectures for Mapping Scalp to Intracranial EEG,” _International Journal of Neural Systems_ , vol. 28, no. 8, 2018.
  * [238] J. J. Bird, D. R. Faria, L. J. Manso, A. Ekart, and C. D. Buckingham, “A Deep Evolutionary Approach to Bioinspired Classifier Optimisation for Brain-Machine Interaction,” _Complexity_ , vol. 2019, 2019.
  * [239] V. Singhal, A. Majumdar, and R. K. Ward, “Semi-Supervised Deep Blind Compressed Sensing for Analysis and Reconstruction of Biomedical Signals From Compressive Measurements,” _IEEE Access_ , vol. 6, pp. 545–553, 2018.
  * [240] A. Gogna, A. Majumdar, and R. Ward, “Semi-supervised Stacked Label Consistent Autoencoder for Reconstruction and Analysis of Biomedical Signals,” _IEEE Transactions on Biomedical Engineering_ , vol. 64, no. 9, pp. 2196–2205, 2017.
  * [241] H. J. Jang and K. O. Cho, “Dual deep neural network-based classifiers to detect experimental seizures,” _Korean Journal of Physiology & Pharmacology_, vol. 23, no. 2, pp. 131–139, 2019.
  * [242] T. Ogawa, Y. Sasaka, K. Maeda, and M. Haseyama, “Favorite Video Classification Based on Multimodal Bidirectional LSTM,” _IEEE Access_ , vol. 6, pp. 61 401–61 409, 2018.
  * [243] A. Ben Said, M. F. Al-Sa’d, M. Tlili, A. A. Abdellatif, A. Mohamed, T. Elfouly, K. Harras, and M. D. O’Connor, “A Deep Learning Approach for Vital Signs Compression and Energy Efficient Delivery in mhealth Systems,” _IEEE Access_ , vol. 6, pp. 33 727–33 739, 2018.

Supplementary Tables

Table 1: A Brief Summary of Sleep Stages Sleep Stages | Main Features of EEG in Each Stage | Brief Description
---|---|---
Wake | Alpha Waves | Before Sleep
Stage N1 NREM | Low-Voltage Theta Waves | Blood Pressure Falls
Stage N2 NREM | Theta Waves with K Complexes and Sleep Spindles | Cardiac Activity Decrease
Stage N3 NREM | High-Amplitude Delta Waves | High Threshold for Arousal
Stage REM Sleep | Low-Amplitude Theta Waves | Blood Pressure and Pulse Rate Increase

Table 2: A Summary of Datasets Mentioned in This Survey Dataset Name |  Modality |  Data Information |  Category |  URL
---|---|---|---|---
BCI Challenge |  EEG |  26 Participants, 56 Channels, 600 Hz |  P300 and Non-P300 |  https://www.kaggle.com/c/inria-bci-challenge
BCI Competition |  EEG |  Vary with datasets in the competition |  |  http://www.bbci.de/competition
BNCI Horizon |  EEG |  Vary with datasets in the database |  |  http://bnci-horizon-2020.eu/database/data-sets
CAP Sleep Database |  EEG, EOG, EMG, ECG |  16 Participants, 3 EEG Channels |  W, S1, S2, S3, S4, and REM |  https://physionet.org/content/capslpdb/1.0.0
CHB-MIT Scalp EEG Database |  EEG |  22 Participants, 23 Channels, 256 Hz |  Ictal Activity, Siezure Onset, and Offset |  https://physionet.org/content/chbmit/1.0.0
CSU BCI Collection |  EEG |  Vary with data sets in the database |  Normal and Motor Impairments |  https://www.cs.colostate.edu/eeg
DEAP Dataset |  EEG and Physiological Signals |  32 Participants, 32 Channels, 512 Hz |  Scores For Arousal, Valence, Iiking, Dominance and Familiarity |  http://www.eecs.qmul.ac.uk/mmv/datasets/deap
EEG Brainwave Dataset: Feeling Emotions |  EEG |  2 Participants, 4 Channels, - I |  Positive, Neutral, and Negative |  https://www.kaggle.com/birdy654/eeg-brainwave-dataset-feeling-emotions
EEG Brainwave Dataset: Mental State |  EEG |  4 Participants, 4 Channels, - Hz |  Relaxed, Concentrating, and Neutral |  https://www.kaggle.com/birdy654/eeg-brainwave-dataset-mental-state
EEG Database Data Set/UCI EEG dataset |  EEG |  122 Participants, 64 Channels, 256 Hz |  Alcoholic and Control |  https://archive.ics.uci.edu/ml/datasets/eeg+database
EEG Motor Movement/Imagery Dataset |  EEG |  109 Participants, 64 Channels, 160 Hz |  Left/Right Fist or Both Fists/Both Feet |  https://physionet.org/content/eegmmidb/1.0.0
EPFL BCI Dataset |  EEG |  9 Participants, 34 Channels, 2047 Hz |  P300 and Non-P300 |  https://www.epfl.ch/labs/mmspg/research/page-58317-en-html/bci-2/bci_datasets
Epileptic Seizure Recognition Data Set |  EEG |  500 Participants, - Channels, 173.61 Hz |  A,B,C,D, and E II |  https://archive.ics.uci.edu/ml/datasets/Epileptic+Seizure+Recognition
MASS Database |  EEG, EOG, EMG, ECG |  200 Participants, 4–20 EEG Channels, 256Hz |  W, N1, N2, N3, and REM |  http://www.ceams-carsm.ca/en/MASS
MESA |  EEG, ECG, EMG |  2,056 Participants, Fz-Cz, Cz-Oz, C4, 256Hz |  W, N1, N2, N3, and REM |  https://www.sleepdata.org/datasets/mesa
MindBigData |  EEG |  Vary with data sets in the dataset |  Brain Reaction from Seeing A Digit (0 to 9) |  http://www.mindbigdata.com/opendb
MIT-BIH Polysomnographic Database |  EEG, ECG, EOG, EMG, Respiration Signals, and Physiological Signals |  16 subjects, 7 PSG Channels, 250 Hz |  W, N1, N2, N3, N4, and REM |  https://www.physionet.org/content/slpdb/1.0.0
SEED Dataset |  EEG and Eye Movement |  15 Participants, 62 Channels, 1000Hz |  Positive/Neutral/Negative and Happy/Sad/Neutral/Fear |  http://bcmi.sjtu.edu.cn/ seed
SHHS |  EEG, EOG, EMG |  6,441 Participants, C4-A1 and C3-A2, 125 Hz |  W, N1, N2, N3, N4, and REM |  https://sleepdata.org/datasets/shhs/
Sleep-EDF Database Expanded |  EEG, EOG, EMG |  61 Participants, Fpz-Cz and Pz-Oz, 100 Hz |  W, S1, S2, S3, S4, and REM |  https://physionet.org/content/sleep-edfx/1.0.0
Sleep-EDF Database |  EEG, EOG, EMG |  20 Participants, Fpz-Cz and Pz-Oz, 100 Hz |  W, N1, N2, N3, and REM |  https://physionet.org/content/sleep-edf/1.0.0
The Bern-Barcelona EEG Database |  EEG |  5 Participants, 7500 Pairs of Signals, 512 or 1024 Hz |  Focal and Non-Focal |  https://www.upf.edu/web/mdm-dtic/datasets
The DREAMS Sleep Spindles Database |  EEG, EOG, EMG |  8 Participants, FP1-A1, C3-A1, O1-A1, 50–200 Hz |  Start Time or Duration of N2 Stage Spindles, and Non-Spindles |  http://www.tcts.fpms.ac.be/ devuyst/Databases/DatabaseSpindles
The SIESTA Normative Database |  EEG, EOG, EMG, ECG |  292 Participants, 6 EEG Channels, >100Hz |  W, N1, N2, N3, and REM |  http://ofai.at/siesta/database.html
TUH EEG Abnormal Corpus Dataset |  EEG |  1488 Abnormal and 1529 Normal EEG Sessions |  Clinically Abnormal and Normal |  https://www.isip.piconepress.com/projects/tuh_eeg
UCD Database |  EEG and Physiological Signals |  25 Participants, C3–A2 and C4–A1, 128Hz |  W, S1, S2, SWS, and REM |  https://physionet.org/content/ucddb/1.0.0
I. Dash ’-’ indicates information unavailable
| II. The letters A-E represent five sets of EEG. Set A and set B were
collected from healthy volunteers with eyes open (A) and eyes closed (B).
---
Sets C, D, and E were from epileptic patients, of which set C and set D were
seizure free intervals from two opposite hemisphere of the brain, and set E
was seizure data.

Table 3: Typical Methods for Artifacts Removal Methods |  Target Artifacts |  Property
---|---|---
Notch Filter |  Line Noise |  Signal distortion in specific frequencies
Band-Pass Filter |  Artifacts concentrated on a particular frequency band |  Preclude certain frequency signals
Independent Component Analysis |  Ocular and muscular noise removal |  Decompose channels into independent components
Reject Contaminated Data Segments |  Ocular noise, muscular noise etc., which are difficultly mitigated |  Reject gross eye movement and occasional recording artifacts
Wavelet Transformation Analysis |  Ocular and muscular noise removal |  Signals are reconstructed based on the corrected coefficient
Common Average Reference |  Artifacts equivalently affect all channels |  Amplitudes can be overall reduced
Z-Score Calculation |  Noisy channels or time periods |  Generates zero-mean data with unitary variance
Denoise AutoEncoder |  General Noises |  Denoise in an unsupervised manner

Table 4: Key Information of Papers about Brain-Computer Interface Authors |  Models |  Paradigms |  Classes |  Data (Private/Public: No. of Participants, No. of Channels, Sampling Rate)
---|---|---|---|---
Ma et al. 2020 [132] |  CNN |  MI |  Rest, Right Hand, and Right Elbow |  Private: 25 Participants, 64 Channels, 1000 Hz
Zhang et al. 2019 [45] |  CNN |  MI |  Left, Right Hand |  BCI Competition II Dataset III
Xu et al. 2019 [43] |  CNN |  MI |  Left, Right Hand |  BCI Competition IV Dataset 2b
Zhu et al. 2019 [133] |  CNN |  MI |  Left, Right Hand |  1\. Private: 25 Participants, 15 Channels, 1000 Hz 2\. BCI Competition IV Dataset 2b
Lu et al. 2017 [39] |  DBN |  MI |  Left, Right Hand |  BCI Competition IV Dataset 2b
Chiarelli et al. 2018 [134] |  DNN |  MI |  Left, Right Hand |  Private: 15 Participants, 128 Channels, 250 Hz
Tayeb et al. 2019 [135] |  CNN,LSTM, CNN+LSTM |  MI |  Left, Right Hand |  1\. Private: 20 Participants, 32 Channels, 256 Hz 2\. BCI Competition IV Dataset 2b
Dai et al. 2019 [136] |  CNN+AE |  MI |  Left, Right Hand |  BCI Competition IV Dataset 2b
Ha et al. 2019 [137] |  CapsNet |  MI |  Left, Right Hand |  BCI Competition IV Dataset 2b
Shi et al. 2019 [138] |  CNN |  MI |  Left, Right Hand |  Private: - Participants, 118 Channels, - Hz
Wang et al. 2018 [139] |  CNN, LSTM |  MI |  Left, Right Hand |  Private: 14 Participants, 11 Channels, 256 Hz
Tabar et al. 2017 [52] |  CNN,SAE, CNN+SAE |  MI |  Left, Right Hand |  1\. BCI Competition II Dataset III 2\. BCI Competition IV Dataset 2b
Amin et al. 2019 [140] |  CNN |  MI |  Left Hand, Right Hand, Feet, and Tongue |  1\. High Gamma Dataset [141] 2\. BCI Competition IV Dataset 2a
Amin et al. 2019 [53] |  CNN, MLP, AE |  MI |  Left Hand, Right Hand, Feet, and Tongue |  1\. BCI Competition IV Dataset 2a 2\. High Gamma Dataset [141]
Li et al. 2019 [46] |  CNN |  MI |  Left Hand, Right Hand, Feet, and Tongue |  BCI Competition IV Dataset 2a
Hassanpour et al. 2019 [142] |  DBN, SAE |  MI |  Left Hand, Right Hand, Feet, and Tongue |  BCI Competition IV Dataset 2a
Zhang et al. 2019 [54] |  CNN+LSTM |  MI |  Left Hand, Right Hand, Feet, and Tongue |  BCI Competition IV Dataset 2a
She et al. 2018 [143] |  ELM |  MI |  Left Hand, Right Hand, Feet, and Tongue |  BCI Competition IV Dataset 2a
Uribe et al. 2019 [144] |  ELM |  MI |  Left Hand, Right Hand, Feet, and Tongue |  BCI Competition IV Dataset 2a
Lei et al. 2019 [38] |  DPN |  MI |  Idle, Preparation, Walking Imagery, and Restoration |  Private: 9 Participants, 32 Channels, 512 Hz
Duan et al. 2017 [145] |  ELM |  MI |  Cortical Positivity and Negativity |  BCI Competition II Dataset Ia
Alazrai et al. 2019 [47] |  CNN |  MI |  Rest, Grasp-Related (Small Diameter, Lateral, and Extension-Type), Wrist-Related (Ulnar/Radial Deviation. Flexion/Extension), Fingers-Related ( Flexion and Extension of The Index, The Middle, The Ring, The Little, and The Thumb Finger) |  Private: 22 Participants (18 Able-Bodied and 4 with Transradial Amputations), 16 Channels, 2048 Hz
Hang et al. 2019 [40] |  CNN |  MI |  1\. Right Hand, Foot 2\. Left Hand, Right Hand, Feet, and Tongue |  1\. BCI Competition III Dataset IVa 2\. BCI Competition IV Dataset IIa
Yang et al. 2018 [55] |  CNN+LSTM |  MI |  1\. Left Hand, Right Foot 2\. Left, Right Hand 3\. Left Hand, Tongue |  1\. Private: 6 Participants, 64 Channels, 500 Hz 2\. BCI Competition III Dataset - 3\. BCI Competition IV Dataset -
Zhao et al. 2019 [41] |  CNN |  MI |  1\. Left Hand, Right Hand, Feet, and Tongue 2\. Left, Right Hand 3\. Elbow Flexion/Extension, Forearm Supination/Pronation, Hand Open/Close |  1\. BCI Compeition IV Dataset 2a 2\. BCI Compeition IV Dataset 2b 3\. From Ofner et al., 15 Participants, 61 Channels, 512 Hz
Wu et al. 2019 [146] |  CNN |  MI |  1\. Left Hand, Right Hand, Feet, and Tongue 2\. Left, Right Hand |  1\. BCI Competition IV Dataset 2a 2\. BCI Competition IV Dataset 2b 3\. High Gamma Dataset [141]
Majidov et al. 2019 [147] |  CNN |  MI |  1\. Left Hand, Right Hand, Feet, and Tongue 2\. Left, Right Hand |  1\. BCI Competition IV Dataset 2a 2\. BCI Competition IV Dataset 2b
Li et al. 2019 [44] |  CNN |  MI |  1\. Left Hand, Right Hand, Feet, and Tongue 2\. Left Hand, Right Hand, Feet, and Rest |  1\. BCI Competition IV Dataset 2a 2\. High Gamma Dataset [141]
Dose et al. 2018 [42] |  CNN |  MI |  Left/Right Fist or Both Fists/Both Feet |  EEG Motor Movement/MI Dataset
Tang et al. 2019 [148] |  DBN |  MI |  Left, Right Hand |  Private: 7 Participants, 14 Channels, 128 Hz
Li et al. 2014 [21] |  SAE |  MI |  Left, Right Hand |  Private: 3 Participants, 14 Channels, 250 Hz
Xu et al. 2018 [149] |  CNN |  MI |  1\. Left, Right Hand 2\. Left Hand, Right Hand, Feet, and Tongue |  1\. BCI Competition II Dataset III 2\. BCI Competition IV Dataset 2a
Ding et al. 2015 [35] |  ELM |  - |  Cortical Positivity and Negativity |  BCI Competition II Dataset Ia
Ma et al. 2017 [56] |  DBN |  mVEP |  Target Stimulus Signal and The Standard Stimulus Signal |  Private: 11 Participants, 10 Channels, 1000 Hz
Gao et al. 2015 [150] |  ANN |  P300 |  P300 and Non-P300 |  Private: 5 Participants, 32 Channels, 2048 Hz
Kundu et al 2019 [51] |  SAE |  P300 |  P300 and Non-P300 |  1\. BCI Competition II Dataset IIb 2\. BCI Competition III Dataset II 3\. BNCI Horizon Dataset
Kshiragar et al. 2019 [48] |  SAE,CNN |  P300 |  P300 and Non-P300 |  Private: 10 Participants, 16 Channels, 500 Hz
Liu et al. 2018 [9] |  CNN |  P300 |  P300 and Non-P300 |  1\. BCI Competition III Dataset II 2\. BCI Competition II Dataset IIb
Farahat et al. 2019 [151] |  CNN |  P300 |  P300 and Non-P300 |  Private: 19 Participants, 29 Channels, 508.63 Hz
Solon et al. 2019 [152] |  CNN |  P300 |  P300 and Non-P300 |  Private: 67 Participants, 64 Channels, - Hz
Vareka et al. 2017 [153] |  SAE |  P300 |  P300 and Non-P300 |  Private: 25 Participants, 19 Channels, 1000 Hz
Morabbi et al. 2018 [154] |  DBN |  P300 |  P300 and Non-P300 |  EPFL BCI Dataset
Ditthapron et al. 2019 [155] |  CNN+LSTM+AE |  P300 |  P300 and Non-P300 |  1\. From Citi et al. [156], 12 Participants, 64 Channels, 2048 Hz 2\. BCI Competition III Dataset II 3\. From Schreuder et al. [157], 10 Participants, 60 Channels, 240 Hz 4\. From Acqualagna et al. [158], 13 Participants, 63 Channels, 250 Hz 5\. EEG Database Data Set 6\. From Treder et al. [159], 11 Participants, 63 Channels, 200 Hz
Lawhern et al. 2018 [49] |  CNN |  P300,MI,etc |  1\. P300 and Non-P300 2\. Correct and Incorrect 3\. The Left Index, Left Middle, Right Index, and Right Middle Finger 4\. Left Hand, Right Hand, Feet, and Tongue |  1\. Private: 15 Participants, 64 Channels, 512 Hz 2\. BCI Challenge 3\. Private: 13 Participants, 256 Channels, 1024 Hz 4\. BCI Competition IV Dataset 2a
Boloukian et al. 2020 [160] |  DN-AE-NTMI |  P300,MI,etc |  1\. P300 and Non-P300 2\. Alcoholic and Control 3\. Left/Right Fist or Both Fists/Both Feet |  1\. From Hoffmann et al. [161], 9 Participants (5 with disablement 4 able-bodied), - Channel, - Hz 2\. EEG Database Data Set 3\. EEG Motor Movement/Imagery Dataset
Pei et al. 2018 [50] |  SAE |  Reaching movements |  Left, Central and Right |  Private: 5 Participants, 32 Channels, 256 Hz
Chen et al. 2019 [162] |  CNN |  RSVP |  Target and Non-Target |  From Touryan et al. [163], 10 Participants, 64 Channels, 512Hz
Manor et al. 2015 [164] |  CNN |  RSVP |  Target and Non-Target |  Private: 15 Participants, 64 Channels, 256 Hz
Manor et al. 2016 [165] |  CNN |  RSVP |  Target and Non-Target |  Private: 15 Participants, 64 Channels, 256 Hz
Nguyen et al. 2019 [57] |  CNN |  SSVEP |  6.67, 7.5, 8.57, 10, and 12 Hz |  Private: 8 Participants, 1 Channel, 128 Hz
Liu et al. 2020 [166] |  DMCCAII |  SSVEP |  6, 8, 9, and 10 Hz |  Private: 10 Participants, 8 Channels, 250 Hz
Waytowich et al. 2018 [58] |  CNN |  SSVEP |  12 SSVEP Stimuli Flashed at Frequencies Ranging from 9.25 Hz To 14.75 Hz in Steps of 0.5 Hz |  From Nakanishi et al. [167], - Participants, - Channel, 2048 Hz
I. DN-AE-NTM: Deep Network Autoencoder Neural Turing Machine
II. DMCCA: Deep Multiset Canonical Correlation Analysis

Table 5: Key Information of Papers about Disease Detection Author |  Models |  Categories |  Classes |  Data (Private/Public: No. of Participants, No. of Channels, Sampling Rate)
---|---|---|---|---
Doborjeh et al. 2016 [37] |  SNN |  Addiction |  Healthy, Addiction Treated , and Addiction Not Treated Subjects |  Private: 74 Participants, 26 Channels, - Hz
Ieracitano et al. 2019 [10] |  CNN |  Alzheimer’s Disease |  1\. ADI vs. HCII, AD vs. MCIIII, MCI vs. HC 2\. AD, MCI, and HC |  Private: 189 Participants (63 AD, 63 MCI, 63 HC), 19 Channels, 1024 Hz
Bi et al. 2019 [168] |  DBN |  Alzheimer’s Disease |  1\. AD, HC, and MCI 2\. Identification: determine EEG spectral image come from which person 3\. Verification: wheather two EEG spectral images come from the same person |  Private: 12 Participants (4 HC, 4 MCI, and 4 AD), 64 Channels, 500 Hz
Morabito et al. 2016 [169] |  SAE, MLP |  Alzheimer’s Disease |  CJDIV/RPDV, CJD/HC, and CJD/AD |  Private: 76 Participants, 19 Channels, - Hz
Hayase et al. 2019 [170] |  MLP |  Anaesthesia |  - |  Private: 30 Participants, - Channels, 128 hZ
Liu et al. 2019 [171] |  CNN |  Anaesthesia |  Anesthetic Ok, Deep, and Light |  Private: 50 Participants, - Channel, - Hz
Kim et al. 2018 [172] |  CNN,LSTM, DNN |  Brain Disease |  1\. Normal and Dementia 2\. Normal and Alcoholism |  UCI EEG Dataset
Chen et al. 2019 [66] |  CNN |  Children with ADHDVI |  Adhd and Controls |  Private: 107 Participants (50 Children with ADHD and 57 Controls), 128 Channels, 1000 Hz