# INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-
Native Speech Recognition

Eunseop Yoon1, Hee Suk Yoon111footnotemark: 1, John Harvill2,
Mark Hasegawa-Johnson2, and Chang D. Yoo1
1Korea Advanced Institute of Science and Technology (KAIST)
2University of Illinois at Urbana-Champaign (UIUC)
{esyoon97, hskyoon<EMAIL_ADDRESS>
{harvill2<EMAIL_ADDRESS>Equal contribution Corresponding author

###### Abstract

Automatic Speech Recognition (ASR) systems have attained unprecedented
performance with large speech models pre-trained based on self-supervised
speech representation learning. However, these pre-trained speech models
suffer from representational bias as they tend to better represent those
prominent accents (i.e., native (L1) English accent) in the pre-training
speech corpus than less represented accents, resulting in a deteriorated
performance for non-native (L2) English accents. Although there have been some
approaches to mitigate this issue, all of these methods require updating the
pre-trained model weights. In this paper, we propose Information Theoretic
Adversarial Prompt Tuning (INTapt), which introduces prompts concatenated to
the original input that can re-modulate the attention of the pre-trained model
such that the corresponding input resembles a native (L1) English speech
without updating the backbone weights. INTapt is trained simultaneously in the
following two manners: (1) adversarial training to reduce accent feature
dependence between the original input and the prompt-concatenated input and
(2) training to minimize CTC loss for improving ASR performance to a prompt-
concatenated input. Experimental results show that INTapt improves the
performance of L2 English and increases feature similarity between L2 and L1
accents.

##  1 Introduction

Self-supervised learning has improved input data representation without
requiring extensive human-labeled data. Powerful pre-trained models providing
high-performing representations for various data types (e.g., text, images,
and audio) have been proposed. For instance, in speech, self-supervised pre-
trained models such as HuBERT (Hsu et al., 2021) have advanced state-of-the-
art performance of automatic speech recognition (ASR).

However, one major challenge in using pre-trained speech models for ASR is the
representational bias towards prominent accents present in the dataset during
pre-training. Consequently, there will be a disparity in ASR performance
between native and non-native speakers. More specifically, pre-training using
a large dataset such as the LibriSpeech (Panayotov et al., 2015), which
comprises a large proportion of utterances from native (L1) English speakers,
leads to a less satisfactory recognition rate for non-native (L2) English
accented speech. This phenomenon can curtail the effectiveness of current
high-performing ASR systems for real-world applications.

Figure 1: Illustration of a hypothetical accent feature space ($\mathcal{H}$).
Distinctive accent features between native and L2 accents lead to degraded
performance of ASR systems on L2 accents. INTapt concatenates a prompt to the
input space to reduce this distinction. Figure 2: Overview of INTapt. INTapt
incorporates a two-step training process where the first step involves
training the Accent Module to get the accent feature of a particular input
speech and the second step involves training the Prompt Generator capable of
making the non-native (L2) English speech input have a better ASR performance
by re-modulating the attention of the Backbone Model so that it resembles the
accent of a native (L1) English speech.

There have been several ways to address this issue in ASR, including fine-
tuning the model on diverse accents (Winata et al., 2019; Shibano et al.,
2021), having a separate model for each accent (Yang et al., 2018) or using
regularization losses that guide the fine-tuning process to achieve robustness
to accents (Chen et al., 2020), all of which require updating the pre-trained
model weights.

We propose a different solution for improving L2 speech recognition in
transformer-based speech models that introduces a small number of learnable
parameters into the input space while keeping the backbone weights of the
model untouched. Our approach is guided by Information-Theoretic Adversarial
Learning; thus, we refer to it as INTapt (Information-Theoretic Adversarial
Prompt Tuning). INTapt aims to introduce auxiliary embeddings (i.e., prompt)
concatenated to the original input, which can re-modulate the attention and
adapt the pre-trained weights so that the corresponding input looks like
speech with an accent seen during pre-training (Figure 1). To achieve this,
INTapt incorporates (1) adversarial training, which tries to minimize the
mutual information between the accent feature of the original input and that
obtained by concatenating the prompt embeddings in front of the initial input,
and (2) CTC loss training to improve the ASR performance of the prompt-
concatenated input. Essentially the prompt is trained such that the accent of
the concatenation is pushed away from the input accent and the concatenation
achieves native CTC loss performance. Unlike the previous use-case of prompts
in NLP or Computer vision (CV), where a single prompt embedding is learned for
each discrete task or input domain, the intensity of an accent is continuous.
Thus, we propose an input-dependent prompt embedding by training a prompt
generator that outputs an input-specific prompt. Through extensive
experiments, we show that the proposed dual objectives of INTapt not only lead
to better performance on L2 English accents but result in a higher similarity
between the accent feature of the prompt-concatenated input and that of L1
English accents.

##  2 INTapt

Figure 2 depicts the overall process of INTapt. INTapt incorporates a two-step
training process. In the first step, we train an Accent Module (AM) capable of
isolating the accent feature from a given audio feature $\bm{a}$ of an input
speech $\bm{x}$. In the second step, we train a Prompt Generator (PG), which
outputs a prompt $\bm{p}$ for a given audio feature $\bm{a}$, using two
objectives: (1) Minimize the mutual information between the accent feature
$\bm{z^{\prime}}$ and $\bm{z}$, where the former is obtained using the prompt-
concatenated input $(\bm{p};\bm{a})$ and the latter is obtained from the
original audio feature $\bm{a}$, (2) Minimize CTC loss to improve the ASR
performance of the input $(\bm{p};\bm{a}).$

###  2.1 Accent Module (AM)

Since our method requires direct access to the isolated accent feature of the
corresponding audio feature input, we propose an Accent Module (AM) capable of
extracting the accent feature $\bm{z}$ from the input $\bm{a}$. The module
consists of an accent feature extractor $f_{\theta_{1}}$ which is trained with
an accent classification head $f_{\theta_{2}}$ to isolate the accent feature
and an accent intensity regression head $f_{\theta_{3}}$ to capture the
intensity of the accent into the obtained feature.

#### Accent Classification Head

The role of the accent classification head $f_{\theta_{2}}$ is to isolate the
accent feature of a given speech 111We show in Appendix B that the proposed
way effectively isolate the accent feature from other features.. Given the
hidden state representation $\bm{h}$ of an audio feature input $\bm{a}$, the
feature extractor outputs the accent feature (i.e.,
$\bm{z}=f_{\theta_{1}}(\bm{h}))$ and the accent classification head
$f_{\theta_{2}}$ tries to assign it to the correct accent label $y$.

#### Accent Intensity Regression Head

The intensity of an accent could vary among different people even though there
are in the same L2 group, and it could also vary between utterances from the
same speaker. Thus, an accent intensity regression head is introduced to
incorporate the accent intensity into the obtained accent feature $\bm{z}$.
Based on the assumption that the intensity of the accent affects ASR
performance, making the accent intensity regression head predict the CTC loss
222Connectionist Temporal Classification (CTC) (Graves et al., 2006) is the
primary loss used to train deep neural networks in speech recognition.,
obtained by inputting the corresponding speech into the backbone speech model,
will allow the extracted accent feature $\bm{z}$ to capture the intensity of
the accent.

Given a batch $B$, the training of the Accent Module with the two
aforementioned heads could be summarized as:

$\begin{split}&\min_{\theta_{1},\theta_{2}}\frac{1}{|B|}\sum_{i\in B}-\log
p(y_{i}|f_{\theta_{2}}(f_{\theta_{1}}(h_{i})))+\\\
&\lambda\min_{\theta_{1},\theta_{3}}\frac{1}{|B|}\sum_{i\in
B}[\,f_{\theta_{3}}(f_{\theta_{1}}(h_{i}))-\text{CTC}(x_{i})]\,^{2}\;\;\end{split}$
(1)

###  2.2 Prompt Generator (PG)

Building on the success of prompts in NLP (Liu et al., 2021; Li and Liang,
2021) and CV (Dosovitskiy et al., ), we introduce a prompt tuning method to
improve the ASR performance for L2 English speech by efficiently utilizing a
pre-trained model that already shows good performance for L1 English speech.
In contrast to traditional NLP or CV applications, where a single, discrete
prompt embedding is learned for each specific task or input domain, the
intensity of an accent is continuous. To address this, we propose an input-
dependent prompt embedding by training prompt generator $PG_{\theta_{4}}$ that
generates an input-specific prompt guided by Information-Theoretic Adversarial
Learning. More specifically, given a hidden state
$\bm{h}=[h_{1},h_{2},...,h_{L}]$ with length $L$ we produce a prompt of length
$L^{\prime}$,

$\bm{p}=PG_{\theta_{4}}(\bm{h})\;\;$ (2)

#### Mutual Information Minimization

Mutual Information meausures the co-dependence between two random variables
$X$ and $Y$. Belghazi et al. (2018) recently proposed a gradient descent based
method for estimating this property, allowing the use of neural networks for
the estimation of mutual information between high dimensional random
variables. The estimation is done using a neural network parameterized by
$\phi$ as below:

$I(X,Y)\geq I_{\phi}(X,Y),$ (3)

where maximizing $I_{\phi}(X,Y)$ provides a tight lower bound of the original
mutual information $I(X,Y).$ We use this to adversarially train the prompt
generator $PG_{\theta_{4}}$ to minimize the mutual information between the
accent feature of the original L2 speech input and the prompt-concatenated
input.

#### CTC Loss Minimization

We train the prompt generator $PG_{\theta_{4}}$ to minimize the CTC loss
obtained for the prompt-concatenated input $(\bm{p};\bm{a})$.

The two minimization objectives wrt. the prompt generator, along with the
maximization objective wrt. the Mutual Information Neural Estimator, are done
jointly in the second training step (Equation 4). We show in Section 3.2 and 4
that the aforementioned objectives not only improve the ASR performance of L2
speech but also effectively make it resemble the accent feature of the L1
speech.

$\min_{\theta_{4}}\max_{\phi}\frac{1}{|B|}\sum_{i\in
B}\text{CTC}(\bm{p}_{\theta_{4}};\bm{a})+\lambda
I_{\phi}(\bm{z^{\prime}}_{\theta_{4}},\bm{z})\;$ (4)

##  3 Experiments

###  3.1 Experimental setting

#### Dataset

We use the L2-ARCTIC (Zhao et al., 2018), which is a speech corpus of non-
native (L2) English speakers - Mandarin (ZH), Hindi (HI), Vietnamese (VI),
Korean (KO), Spanish (ES), and Arabic (AR). Each L2 group contains two male
and two female speakers, and all the speakers read the same 1132 texts. The
train/dev/test set is configured by dividing the data into 0.8/0.1/0.1 splits
with no overlapping texts between each splits. Additionally, since we would
like to simulate a natural data collection situation where the amount of data
varies across groups, we randomly divided the training data into More Frequent
Accent (MFA) (ZH, HI), Less Frequent Accent (LFA) (VI, KO), and Unseen Accent
(UA) (ES, AR) - For MFA we keep all the training data, for LFA we keep half of
the data, and for UA we remove all the training data.

Backbone | #.params | Methods | MFA | LFA | UA | ALL
---|---|---|---|---|---|---
ZH | HI | VI | KO | ES | AR
HUBERT Large | - | Backbone | 18.71 | 8.80 | 25.8 | 10.98 | 14.12 | 14.92 | 15.55
315M | +Finetune | 15.46 | 7.91 | 22.26 | 9.95 | 14.19 | 13.94 | 13.95
12.5M | +Promptctc | 13.93 | 7.20 | 21.93 | 9.69 | 12.64 | 12.38 | 12.96
12.9M | +INTapt | 13.09 | 6.64 | 21.25 | 8.97 | 12.18 | 11.92 | 12.34
HUBERTXLarge | - | Backbone | 17.03 | 7.48 | 26.02 | 10.49 | 13.65 | 13.52 | 14.69
958M | +Finetune | 15.49 | 7.53 | 24.09 | 10.02 | 13.48 | 12.56 | 13.86
19.7M | +Promptctc | 13.02 | 7.31 | 19.26 | 8.05 | 10.46 | 10.38 | 11.41
19.9M | +INTapt | 11.67 | 6.63 | 18.41 | 7.17 | 10.44 | 10.55 | 11.00
|  |  |  |  |  |  |  |  |
Table 1: Comparison of WER (%) (lower is better) on the created subset of
L2-ARCTIC (MFA, LFA, UA). #.params denote the number of parameters that were
updated for training. The std. are reported in Table 3 of Appendix C.2.

#### Models

For the backbone pre-trained speech models we try two different settings,
HuBERTLarge and HuBERTXLarge Hsu et al. (2021). We consider three different
training situations: 1) Finetune denotes a standard finetuning method where we
update the pre-trained model weights to minimize the CTC loss, 2) Promptctc is
the case of training the prompt generator without the minimization of mutual
information, and 3) INTapt trains the prompt generator with our proposed
objective in equation 4. We include the training details in Appendix A.

###  3.2 Results

Table 1 shows the Word Error Rate (WER) across different L2 groups on the ASR
task. We find that the performance improvement of the prompt tuning approaches
(Promptctc and INTapt) are more significant compared to standard finetuning
despite updating small number of parameters (2-4%). INTapt shows the lowest
WER on all L2 groups, obtaining 12.34% for HuBERTLarge and 11.00% for
HuBERTXLarge on the aggregated all speakers, outperforming the finetuned by
1.62%p and 2.86%p, respectively 333We show some examples of improved ASR
results using INTapt in Appendix C.1. This conforms to the previous findings
(Lester et al., 2021) that larger model size can benefit more from prompt
tuning methods.

In Table 2, we report the WER on LibriSpeech (Panayotov et al., 2015) test-
clean and test-other, which consists mainly of L1 speech. Compared with the
backbone model, the WER after finetuning increased by 5.81%p. However, since
Promptctc and INTapt does not change the backbone weights, the WER on test-all
increased only by 0.48%p and 0.37%p, respectively. This shows one of the key
benefits of prompt tuning methods in that it only slightly degrades the
performance of the backbone model on tasks it already excels at while
improving performance on others.

Methods | test-clean | test-other | test-all
---|---|---|---
Backbone | 2.15 | 4.42 | 3.29
+Finetune | 8.10 | 10.08 | 9.10
+Promptctc | 2.56 | 4.93 | 3.77
+INTapt | 2.41 | 4.94 | 3.66
Table 2: WER (%) (lower is better) on LibriSpeech. test-all denotes the
aggregation of test-clean and test-other. Figure 3: Cosine similarity between
L1 accent feature and L2 accent features obtained from different methods.

##  4 Ablation Study

We analyze whether INTapt allows the L2 speech input to resemble the accent of
L1 speech (Figure 3). Using the Accent Module, we extract the L1 accent
feature, and L2 accent features obtained using the Backbone model, Promptctc,
and INTapt. INTapt showed the highest cosine similarity for all L2 groups,
meaning that INTapt effectively adjusts the attention of the pre-trained model
so that L2 speech resemble L1 speech in terms of accent.

##  5 Conclusion

We introduced Information Theoretic Adversarial Prompt Tuning (INTapt) for
improving non-native ASR performance. To achieve this, INTapt re-modulates the
attention of the pre-trained speech models by concatenating input-dependent
prompt embeddings to the original input, without updating the model weights.
Throughout the experiment, we show that INTapt is capable of outperforming
standard finetuning of the pre-trained model on L2 speech, without degradation
on L1 speech, by allowing the L2 input to resemble a L1 accent.

## Limitations

INTapt adopts a prompt tuning method which utilizes the inherent information
from pre-trained models that already shows good ASR performance on L1 English
speakers. Therefore, in order to apply our method we need a pre-trained model
that already has good performance on a specific task which might not be
available for other langauges. Also, our method might potentially need
sufficiently large pre-trained model size in order for prompt to utilize the
internal information of the model.

## Ethics Statement

Since pre-trained speech models usually show better performance on native (L1)
speech automatic speech recognition (ASR) due to the nature of the pre-
training data used, this work have contributed to improve the ASR performance
for non-native (L2) English speakers and mitigating the performance gap
between them. This has the potential to construct a fair ASR machine well-
operating not only on L1 English speakers but L2 speakers, which is an
important feature to have for its deployment in real-life. Additionally, since
we utilize the pre-trained model, it is possible to have ethical issue
depending on the pre-trained model.

## Acknowledgements

This work was supported by Institute of Information & communications
Technology Planning & Evaluation (IITP) grant funded by the Korea government
(MSIT) (No. 2022-0-00951, Development of Uncertainty-Aware Agents Learning by
Asking Questions), Institute of Information & communications Technology
Planning & Evaluation (IITP) grant funded by the Korea government(MSIT)
[NO.2021-0-02068, Artificial Intelligence Innovation Hub (Seoul National
University)], and Institute of Information & communications Technology
Planning & Evaluation (IITP) grant funded by the Korea government(MSIT)
(No.2022-0-00184, Development and Study of AI Technologies to Inexpensively
Conform to Evolving Policy on Ethics).

## References

  * Belghazi et al. (2018) Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. 2018.  Mine: Mutual information neural estimation.
  * Chen et al. (2020) Yi-Chen Chen, Zhaojun Yang, Ching-Feng Yeh, Mahaveer Jain, and Michael L Seltzer. 2020.  Aipnet: Generative adversarial pre-training of accent-invariant networks for end-to-end speech recognition.  In _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 6979–6983. IEEE.
  * (3) A Dosovitskiy, L Beyer, A Kolesnikov, D Weissenborn, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.
  * Graves et al. (2006) Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006.  Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks.  In _Proceedings of the 23rd International Conference on Machine Learning_ , ICML ’06, page 369–376, New York, NY, USA. Association for Computing Machinery.
  * Hsu et al. (2021) Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021.  Hubert: Self-supervised speech representation learning by masked prediction of hidden units.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 29:3451–3460.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.  The power of scale for parameter-efficient prompt tuning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 3045–3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4582–4597.
  * Liu et al. (2021) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _arXiv preprint arXiv:2107.13586_.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_.
  * Panayotov et al. (2015) Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015.  Librispeech: An asr corpus based on public domain audio books.  In _2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 5206–5210.
  * Shibano et al. (2021) Toshiko Shibano, Xinyi Zhang, Mia Taige Li, Haejin Cho, Peter Sullivan, and Muhammad Abdul-Mageed. 2021.  Speech technology for everyone: Automatic speech recognition for non-native english.  In _Proceedings of The Fourth International Conference on Natural Language and Speech Processing (ICNLSP 2021)_ , pages 11–20.
  * van der Maaten and Hinton (2008) Laurens van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-sne.  _Journal of Machine Learning Research_ , 9(86):2579–2605.
  * Winata et al. (2019) Genta Indra Winata, Zhaojiang Lin, and Pascale Fung. 2019.  Learning multilingual meta-embeddings for code-switching named entity recognition.  In _Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)_ , pages 181–186, Florence, Italy. Association for Computational Linguistics.
  * Yang et al. (2018) Xuesong Yang, Kartik Audhkhasi, Andrew Rosenberg, Samuel Thomas, Bhuvana Ramabhadran, and Mark Hasegawa-Johnson. 2018.  Joint modeling of accents and acoustics for multi-accent speech recognition.  In _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 1–5. IEEE.
  * Zhao et al. (2018) Guanlong Zhao, Sinem Sonsaat, Alif Silpachai, Ivana Lucic, Evgeny Chukharev-Hudilainen, John Levis, and Ricardo Gutierrez-Osuna. 2018.  L2-arctic: A non-native english speech corpus.  In _INTERSPEECH_ , pages 2783–2787.

##  Appendix A Experiment Details

#### Model Architecture

We use a 3-layer Multi Layer Perceptron (MLP) for the Accent Feature Extractor
and 1-layer, 3-layer MLP for the Accent Classification Head and the Accent
Intensity Regression Head in the Accent Module, respectively. The Prompt
Generator (PG) is composed of a single layer transformer. Since we adopt the
transformer architecture for PG, its maximum output length is same as the
length $L$ of the input audio feature $a$. The specific length of prompt can
be set by taking the first $L^{\prime}$ output embeddings from the front of
the transformer output. For the Mutual Information Neural Estimator (MINE), we
use a 3-layer MLP as well.

#### Training Details

For pre-processing L2-ARCTIC, we utilized the huggingface resampling tool
444https://huggingface.co/docs/datasets/audio_process to downsample the audio
files from 44.1kHz to 16kHz. The hidden state representation obtained from the
3rd layer of the backbone model is used as the input to the Accent Module and
Prompt Generator for both HuBERTlarge and HuBERTXLarge. The dimension of the
accent feature $\bm{a}$ is set to $d=256$, the length of prompt $L^{\prime}$
is 40, and the dimension of the prompt is set to 1024 and 1280, same as that
of the input embedding size for HuBERTlarge and HuBERTXLarge, respectively. We
use the AdamW optimizer (Loshchilov and Hutter, 2019) with
$\beta_{1}=0.9,\beta_{2}=0.999,\epsilon=1e-8$, and weight decay
$\lambda=0.005$ with different learning rates for all trainable model (i.e.,
AM, PG, MINE, finetuned backbone). The learning rate used for both AM and MINE
is 1e-3, and 5e-6, 1e-4, 1e-4 are used for Fintune, promptctc, and INTapt,
respectively. For all the methods, the batch size is set to 16 for HuBERTlarge
and 8 for HuBERTXLarge. We use $\lambda=0.5$ for Equation 1 and
$\lambda=0.003$ for Equation 4. The best model is selected by the lowest WER
on the validation set. All experiments was done on NVIDIA Quadro RTX 8000.

##  Appendix B Accent Feature Isolation

We visually analyze the accent feature extracted from AM to validate that the
feature does not contain any other information except accent. We plot the 2-D
representation of extracted accent feature using t-SNE (van der Maaten and
Hinton, 2008) with the label for gender for three L2 groups (i.e., HI, KO,
ES). Figure 5 shows that the scatter points are distinctive between L2 groups
but difficult to distinguish gender, which means our AM successfully isolates
the accent feature from audio.

Figure 4: Latent space visualization showing that Accent Module extracts
isolated accent feature

##  Appendix C Results of INTapt

###  C.1 Examples

In Figure 5, we show some examples of improved speech recognition results
using INTapt. HuBERTLarge is used as backbone model for both cases and the
INTapt having lowest WER on val set is selected to predict samples. Red
represents wrong transcription output from the backbone pre-trained
HuBERTLarge, and green represents enhanced recognition through the use of
INTapt to re-modulate the backbone attention.

Figure 5: Examples of enhanced recognition though the use of INTapt. Red
represents the wrong transcription output prediction from the backbone model
and green represents the correct output prediction using INTapt.

###  C.2 Standard Deviation of Results

Backbone | #.params | Methods | MFA | LFA | Unseen Accent | Avg.
---|---|---|---|---|---|---
ZH | HI | VI | KO | ES | AR
HUBERT Large | 315M | +Finetune | 0.31 | 0.70 | 0.51 | 0.50 | 0.73 | 0.38 | 0.36
12.5M | +Promptctc | 0.51 | 0.56 | 0.99 | 0.74 | 0.78 | 0.38 | 0.30
12.9M | +INTapt | 0.66 | 0.72 | 0.73 | 0.69 | 0.59 | 0.27 | 0.13
HUBERTXLarge | 958M | +Finetune | 0.41 | 0.21 | 1.79 | 0.06 | 0.33 | 0.12 | 0.29
19.7M | +Promptctc | 0.15 | 0.64 | 0.19 | 0.51 | 0.24 | 0.56 | 0.12
19.9M | +INTapt | 0.33 | 0.31 | 0.41 | 0.30 | 0.25 | 0.12 | 0.18
|  |  |  |  |  |  |  |  |
Table 3: Standard deviation values for the experimental results in Table 1.
The values were obtained by running the same experiments with five different
random seeds.

In Table 3, we report the standard deviation of the results in Table 1 with
five different random seeds. As the backbone experiment in Table 1 is obtained
without any training, we do not contain the standard deviation for those.


# The History of Speech Recognition to the Year 2030

Awni Hannun111 Send correspondence to<EMAIL_ADDRESS>
###### Abstract

The decade from 2010 to 2020 saw remarkable improvements in automatic speech
recognition. Many people now use speech recognition on a daily basis, for
example to perform voice search queries, send text messages, and interact with
voice assistants like Amazon Alexa and Siri by Apple. Before 2010 most people
rarely used speech recognition. Given the remarkable changes in the state of
speech recognition over the previous decade, what can we expect over the
coming decade? I attempt to forecast the state of speech recognition research
and applications by the year 2030. While the changes to general speech
recognition accuracy will not be as dramatic as in the previous decade, I
suggest we have an exciting decade of progress in speech technology ahead of
us.

##  1 Recap

The decade from 2010 to 2020 saw remarkable progress in speech recognition and
related technology. Figure 1 is a timeline of some of the major developments
in the research, software, and application of speech recognition over the
previous decade. The decade saw the launch and spread of phone-based voice
assistants like Apple Siri. Far-field devices like Amazon Alexa and Google
Home were also released and proliferated.

These technologies were enabled in-part by the remarkable improvement in the
word error rates of automatic speech recognition as a result of the rise of
deep learning. The key drivers of the success of deep learning in speech
recognition have been 1) the curation of massive transcribed data sets, 2) the
rapid rate of progress in graphics processing units, and 3) the improvement in
the learning algorithms and model architectures.

Thanks to these ingredients, the word error rate of speech recognizers
improved consistently and substantially throughout the decade. On two of the
most commonly studied benchmarks, automatic speech recognition word error
rates have surpassed those of professional transcribers (see figure 3).

This remarkable progress invites the question: what is left for the coming
decade to the year 2030? In the following, I attempt to answer this question.
But, before I begin, I’d first like to share some observations on the general
problem of predicting the future. These findings are inspired by the
mathematician (as well as computer scientist and electrical engineer) Richard
Hamming, who also happened to be particularly adept at forecasting the future
of computing.

Figure 1: A timeline of some of the major developments in speech recognition
from the years 2010 to 2020. The decade saw the launch of voice-based devices
and voice assistants, open-source and widely used speech recognition software
like Kaldi [23], and larger benchmarks like LibriSpeech [22]. We also saw
speech recognition models improve starting from hybrid neural network
architectures [15] to more end-to-end models including Deep Speech [11], Deep
Speech 2 [2], encoder-decoder models with attention [5], and transducer-based
speech recognition [14].

##  2 On Predicting the Future

Richard Hamming in _The Art of Doing Science and Engineering_ [10] makes many
predictions, many of which have come to pass. Here are a few examples222Quotes
and predictions are from chapters 2, 4, and 21 of Hamming [10].

  * •

He stated that by “the year 2020 it would be fairly universal practice for the
expert in the field of application to do the actual program preparation rather
than have experts in computers (and ignorant of the field of application) do
the program preparation.”

  * •

He predicted that neural networks “represent a solution to the programming
problem,” and that “they will probably play a large part in the future of
computers.”

  * •

He predicted the prevalence of general-purpose rather than special-purpose
hardware, digital over analog, and high-level programming languages all long
before the field had decided one way or another.

  * •

He anticipated the use of fiber-optic cables in place of copper wire for
communication well before the switch actually took place.

These are just a few examples of Hamming’s extraordinary prescience. Why was
he so good at predicting the future? Here are a few observations which I think
were key to his ability:

Practice: One doesn’t get good at predicting the future without actually
practicing at it. Hamming practiced. He reserved every Friday afternoon “great
thoughts” in which he mused on the future. But he didn’t just predict in
isolation. He made his predictions public, which forced him to put them in a
communicable form and held him accountable. For example, in 1960 Hamming gave
a talk titled “The History of Computing to the Year 2000” (you may recognize
the title) in which he predicted how computing would evolve over the next
several decades.

Focus on fundamentals: In some ways, forecasting the future development of
technology is just about understanding the present state of technology more
than those around you. This requires both depth in one field as well as non-
trivial breadth. This also requires the ability to rapidly assimilate new
knowledge. Mastering the fundamentals builds a strong foundation for both.

Open mind: Probably the most important trait Hamming exhibited, and in my
opinion the most difficult to learn, was his ability to keep an open mind.
Keeping an open mind requires constant self-vigilance. Having an open mind one
day does not guarantee having it the next. Having an open mind with respect to
one scientific field does not guarantee having it with respect to another.
Hamming recognized for example that one may be more productive in an office
with the door closed, but he kept his office door open as he believed an “open
mind leads to the open door, and the open door tends to lead to the open mind”
[10, chp. 30].

I’ll add to these a few more thoughts. First, the rate of change of progress
in computing and machine learning is increasing. This makes it harder to
predict the distant future today than it was 50 or 100 years ago. These days
predicting the evolution of speech recognition even ten years out strikes me
as a challenge. Hence my choosing to work with that time frame.

A common saying about technology forecasting is that short-term predictions
tend to be overly optimistic and long-term predictions tend to be overly
pessimistic. This is often attributed to the fact that progress in technology
has been exponential. Figure 2 shows how this happens if we optimistically
extrapolate from the present assuming progress is linear with time. Progress
in speech recognition over the previous decade (2010-2020) was driven by
exponential growth along two key axes. These were compute (e.g. floating-point
operations per second) and data set sizes. Whether or not figure 2 applies to
speech recognition for the coming decade remains to be seen.

Figure 2: The graph depicts progress as a function of time. The linear
extrapolation from the present (dashed line) initially results in overly
optimistic predictions. However, eventually the predictions become pessimistic
as they are outstripped by the exponential growth (solid line).

I’m sure a lot of the following predictions will prove wrong. In some ways,
particularly when it comes to the more controversial predictions, these are
really more of an optimistic wishlist for the future. On that note, let me
close this section with the famous words of the computer scientist Alan
Kay333Alan Kay is best known for developing the modern graphical user
interface and also object-oriented programming in the Smalltalk programming
langauge.:

> _The best way to predict the future is to invent it._

##  3 Research Predictions

###  3.1 Semi-supervised Learning

Prediction: Semi-supervised learning is here to stay. In particular, self-
supervised pretrained models will be a part of many machine-learning
applications, including speech recognition.

Part of my job as a research scientist is hiring, which means a lot of
interviews. I’ve interviewed more than a hundred candidates working on a
diverse array of machine-learning applications. Some large fraction,
particularly of the natural language applications, rely on a pretrained model
as the basis for their machine-learning enabled product or feature. Self-
supervised pretraining is already pervasive in language applications in
industry. I predict that by 2030 self-supervised pretraining will be just as
pervasive in speech recognition.

The past three years of deep learning have been the years of semi and self-
supervision. The field has undoubtedly learned how to improve machin-learning
models using unannotated data. Self-supervised learning [18] has benefited
many of the most challenging machine learning benchmarks. In language tasks,
state-of-the-art records have been repeatedly set and surpassed by self-
supervised models [6, 24, 33]. Self and semi-supervision are now commonplace
and setting records in computer vision [13, 4, 9], abstractive summarization
[34] and machine translation [27].

Speech recognition has also benefited from semi-supervised learning. Two
approaches are commonly used, both of which work well. The first approach is
self-supervised pretraining [26, 35] with a loss function based on contrastive
predictive coding [21]. The idea is simple: train the model to predict the
future frame(s) of audio given the past. Of course, the devil is in the
details and the scale. The second approach is pseudo-labeling [19, 16, 32].
Again the idea is simple: use the trained model to predict the label on
unlabeled data, then train a new model on the predicted labels as if they were
the ground truth. And again the devil is in the details and the scale. The
fact that pseudo-labeling leads to better models is remarkable. It feels as if
we are getting something for nothing, a free lunch. The reason and the regime
in which pseudo-labeling works are interesting research questions.

The main challenges with self-supervision are those of scale, and hence
accessibility. Right now only the most highly endowed industry research labs
(_e.g._ Google Brain, Google DeepMind, Facebook AI Research, OpenAI, _etc._)
have the funds to burn on the compute required to research self-supervision at
scale. As a research direction, self-supervision is only becoming less
accessible to academia and smaller industry labs.

Research implications: Self-supervised learning would be more accessible given
lighter-weight models which could be trained efficiently on less data.
Research directions which could lead to this include sparsity for lighter-
weight models, optimization for faster training, and effective ways of
incorporating prior knowledge for sample efficiency.

(a) LibriSpeech

(b) Switchboard Hub5’00

Figure 3: The improvement in word error rate over time on the LibriSpeech [22]
and Switchboard Hub5’00 benchmarks. The data for these figures is from
%****␣research_predictions.tex␣Line␣75␣****https://github.com/syhw/wer_are_we.
The dashed lines indicate human-level performance. The human-level results on
LibriSpeech are reported in Amodei et al. [2], and those on Switchboard are
reported in Xiong et al. [31].

###  3.2 On Device

Prediction: Most speech recognition will happen on the device or at the edge.

There are a few reasons I predict this will happen. First, keeping your data
on your device rather than sending it to a central server is more private. The
trend towards data privacy will encourage on-device inference whenever
possible. If the model needs to learn from a user’s data, then the training
should happen on the device.

The second reason to prefer on-device inference is latency. In absolute terms,
the difference between 10 milliseconds and 100 milliseconds is not much. But
the former is well below the perceptual latency of a human, and the latter
well above [17, 20]. Google has already demonstrated an on-device speech
recognition system with accuracy nearly as good as a server-side system [14].
The latency differences are easily noticeable.444For an example of the
perceptual difference in latencies see the blog post on Google’s on-device
speech recognizer: https://ai.googleblog.com/2019/03/an-all-neural-on-device-
speech.html From a pragmatic standpoint, the latency of the server-side
recognizer is probably sufficient. However, the imperceptible latency of the
on-device system makes the interaction with the device feel much more
responsive and hence more engaging.

A final reason to prefer on-device inference is 100% availability. Having the
recognizer work even without an internet connection or in spotty service means
it will work all the time. From a user interaction standpoint there is a big
difference between a product which works most of the time and a product which
works every time.

Research implications: On-device recognition requires models with smaller
compute and memory requirements and which use less energy in order to preserve
battery life. Model quantization and knowledge distillation (training a
smaller model to mimic the predictions of a more accurate larger model) are
two commonly used techniques. Sparsity, which is less commonly used, is
another approach to generate lighter-weight models. In sparse models, most of
the parameters (_i.e._ connections between hidden states) are zero and can be
effectively ignored. Of these three techniques, I think sparsity is the most
promising research direction.

I believe we have extracted most of the value that quantization has to offer.
Even in the unlikely best possible scenario of further reducing quantization
from 8-bit to 1-bit, we only get a factor-of-eight gain. With distillation, we
still have a lot to learn. However, I believe uncovering the mechanism through
which distillation works will subsequently enable us to train small models
directly rather than taking the circuitous path of training a large model and
then a second small model to mimic the large model.

This leaves sparsity as the most promising research direction for lighter-
weight models. As findings like the “lottery ticket hypothesis” demonstrate
[7], we have a lot to learn about the role of sparsity in deep learning. In
theory, the computational gains from sparsity could be substantial. Realizing
these gains will require developments in the software, and possibly hardware,
used to evaluate sparse models.

Weak supervision will be an important research direction for on-device
training for applications which typically require labeled data. For example, a
users interaction with the output of a speech recognizer or the actions they
take immediately afterward could be useful signal from which the model can
learn in a weakly-supervised manner.

###  3.3 Word Error Rate

Prediction: By the end of the decade, possibly much sooner, researchers will
no longer be publishing papers which amount to “improved word error rate on
benchmark X with model architecture Y.”

As you can see in figure 3, word error rates on the two most commonly studied
speech recognition benchmarks have saturated. Part of the problem is that we
need harder benchmarks for researchers to study. Two recently released
benchmarks may spur further research in speech recognition [3, 8]. However, I
predict that these benchmarks will quickly saturate by scaling up models and
computation.

Another part of the problem is that we have reached a regime where word error
rate improvements on academic benchmarks no longer correlate with practical
value. Speech recognition word error rates on both benchmarks in figure 3
surpassed human word error rates several years ago.555Estimates of human-level
word error rates on the CallHome portion of Hub5’00 vary considerably. For
example Saon et al. [25] report a best result 6.8 out of three transcribers
whose results vary by nearly 2.0 absolute word error rate. However, in most
settings humans understand speech better than machines do. This implies that
word error rate as a measure of the quality our speech recognition systems
does not correlate well with an ability to understand human speech.

A final issue is research in state-of-the-art speech recognition is becoming
less accessible as models and data sets are getting larger, and as computing
costs are increasing. A few well-funded industry labs are rapidly becoming the
only places that can afford this type of research. As the advances become more
incremental and further from academia, this part of the field will continue to
shift from research labs to engineering organizations.

###  3.4 Richer Representations

Figure 4: An example lattice used to encode mutliple hypotheses output from a
speech recognizer with differing weights.

Prediction: Transcriptions will be replaced by richer representations for
downstream tasks which rely on the output of a speech recognizer. Examples of
such downstream applications include conversational agents, voice-based search
queries, and digital assistants.

Downstream applications often don’t care about a verbatim transcription; they
care about semantic correctness. Hence, improving the word error rate of a
speech recognizer often does not improve the objective of the downstream task.
One possibility is to develop a _semantic error rate_ and use it to measure
the quality of the speech recognizer. This is a challenging albeit interesting
research problem.

I think a more likely outcome is to give downstream applications richer
representations from the speech recognizer. For example, instead of passing a
single transcription, passing a lattice of possibilities (as in figure 4)
which captures the uncertainty for each could be much more useful.

Research implications: The exact structure used to encode the representation
is an open question. One possibility could be some sort of weighted transducer
which if differentiable could allow for fine-tuning the recognizer to specific
applications [1, 12]. This type of representation also requires models which
are able to ingest variable-sized graphs as input.

###  3.5 Personalization

Prediction: By the end of the decade, speech recognition models will be deeply
personalized to individual users.

One of the main distinctions between the automatic recognition of speech and
the human interpretation of speech is in the use of context. Humans rely on a
lot of context when speaking to one another. This context includes the topic
of conversation, what was said in the past, the noise background, and visual
cues like lip movement and facial expressions. We have, or will soon reach,
the Bayes error rate for speech recognition on short (_i.e._ less than ten
second long) utterances taken out of context. Our models are using the signal
available in the data to the best of their ability. To continue to improve the
machine understanding of human speech will require leveraging context as a
deeper part of the recognition process.

One way to do this is with personalization. Personalization is already used to
improve the recognition of utterances of the form “call <NAME>”. Sim et al.
[29] found personalizing a model with a user’s contact list improves named
entity recall from 2.4% to 73.5% – a massive improvement. Personalizing models
to individual users with speech disorders improves word error rates by 64%
relative [30]. Personalization can make a huge difference in the quality of
the recognition, particularly for groups or domains that are underrepresented
in the training data. I predict we will see much more pervasive
personalization by the end of the decade.

Research implications: On-device personalization requires on-device training
which in itself requires lighter-weight models and some form of weak
supervision (see section 3.2). Personalization requires models which can be
easily customized to a given user or context. The best way to incorporate such
context into a model is still a research question.

Table 1: Predictions for the progress in speech recognition research and
applications by the year 2030. Prediction
---
Self-supervised learning and pretrained models are here to stay.
Most speech recognition (inference) will happen at the edge.
On-device model training will be much more common.
Sparsity will be a key research direction to enable on-device inference and
training.
Improving word error rate on common benchmarks will fizzle out as a research
goal.
Speech recognizers will output richer representations (graphs) for use by
downstream tasks.
Personalized models will be commonplace.
Most transcription services will be automated.
Voice assistants will continue to improve, but incrementally.

##  4 Application Predictions

###  4.1 Transcription Services

Prediction: By the end of the decade, 99% of transcribed speech services will
be done by automatic speech recognition. Human transcribers will perform
quality control and correct or transcribe the more difficult utterances.
Transcription services include, for example, captioning video, transcribing
interviews, and transcribing lectures or speeches.

###  4.2 Voice Assistants

Prediction: Voice assistants will get better, but incrementally, not
fundamentally. Speech recognition is no longer the bottleneck to better voice
assistants. The bottlenecks are now fully in the language understanding domain
including the ability to maintain conversations, multi-ply contextual
responses, and much wider domain question and answering. We will continue to
make incremental progress on these so-called AI-complete problems,666Shapiro
[28, sec. 4] defines an AI task as AI-complete if solving it is equivalent to
“solving the general AI problem”, which he defines as “producing a generally
intelligent computer program”. but I don’t expect them to be solved by 2030.

Will we live in smart homes that are always listening and can respond to our
every vocal beck and call? No. Will we wear augmented reality glasses on our
faces and control them with our voice? Not by 2030.

##  5 Conclusion

Table 1 summarizes my predictions for the progress in speech recognition to
the year 2030. The predictions show that the coming decade could be just as
exciting and important to the development of speech recognition and spoken
language understanding as the previous one. We still have many research
problems to solve before speech recognition will reach the point where it
works all the time, for everyone. However, this is a goal worth working
toward, as speech recognition is a key component to more fluid, natural, and
accessible interactions with technology.

## Acknowledgements

Thanks to Chris Lengerich, Marya Hannun, Sam Cooper, and Yusuf Hannun for
their feedback on this article.

## References

  * k2 [2020] K2.  https://github.com/k2-fsa/k2, 2020.
  * Amodei et al. [2016] Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al.  Deep speech 2: End-to-end speech recognition in english and mandarin.  In _International conference on machine learning_ , pages 173–182. PMLR, 2016.
  * Chen et al. [2021] Guoguo Chen, Shuzhou Chai, Guanbo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, et al.  Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio.  _arXiv preprint arXiv:2106.06909_ , 2021.
  * Chen et al. [2020] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pages 1597–1607. PMLR, 2020.
  * Chorowski et al. [2015] Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio.  Attention-based models for speech recognition.  In _Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 1_ , pages 577–585, 2015.
  * Devlin et al. [2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL-HLT (1)_ , 2019.
  * Frankle and Carbin [2019] Jonathan Frankle and Michael Carbin.  The lottery ticket hypothesis: Finding sparse, trainable neural networks.  In _International conference on learning representations_ , 2019.
  * Galvez et al. [2021] Daniel Galvez, Greg Diamos, Juan Manuel Ciro Torres, Keith Achorn, Anjali Gopi, David Kanter, Max Lam, Mark Mazumder, and Vijay Janapa Reddi.  The people’s speech: A large-scale diverse english speech recognition dataset for commercial usage.  2021\.
  * Grill et al. [2020] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al.  Bootstrap your own latent: A new approach to self-supervised learning.  _arXiv preprint arXiv:2006.07733_ , 2020.
  * Hamming [1997] Richard R Hamming.  _Art of doing science and engineering: Learning to learn_.  CRC Press, 1997.
  * Hannun et al. [2014] Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al.  Deep speech: Scaling up end-to-end speech recognition.  _arXiv preprint arXiv:1412.5567_ , 2014.
  * Hannun et al. [2020] Awni Hannun, Vineel Pratap, Jacob Kahn, and Wei-Ning Hsu.  Differentiable weighted finite-state transducers.  _arXiv preprint arXiv:2010.01003_ , 2020.
  * He et al. [2020] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 9729–9738, 2020.
  * He et al. [2019] Yanzhang He, Tara N Sainath, Rohit Prabhavalkar, Ian McGraw, Raziel Alvarez, Ding Zhao, David Rybach, Anjuli Kannan, Yonghui Wu, Ruoming Pang, et al.  Streaming end-to-end speech recognition for mobile devices.  In _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 6381–6385. IEEE, 2019.
  * Hinton et al. [2012] Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al.  Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups.  _IEEE Signal processing magazine_ , 29(6):82–97, 2012.
  * Kahn et al. [2020] Jacob Kahn, Ann Lee, and Awni Hannun.  Self-training for end-to-end speech recognition.  In _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 7084–7088. IEEE, 2020.
  * Lago and Kon [2004] Nelson Posse Lago and Fabio Kon.  The quest for low latency.  In _ICMC_ , 2004.
  * LeCun and Misra [2021] Y LeCun and I Misra.  Self-supervised learning: The dark matter of intelligence.  https://ai.facebook.com/blog/self-supervised-learning-the-dark-matter-of-intelligence/, 2021\.
  * Lee et al. [2013] Dong-Hyun Lee et al.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.  In _Workshop on challenges in representation learning, ICML_ , volume 3, page 896, 2013.
  * Levitin et al. [2000] Daniel J Levitin, Karon MacLean, Max Mathews, Lonny Chu, and Eric Jensen.  The perception of cross-modal simultaneity (or “the greenwich observatory problem” revisited).  In _AIP Conference Proceedings_ , volume 517, pages 323–329. American Institute of Physics, 2000.
  * Oord et al. [2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding.  _arXiv preprint arXiv:1807.03748_ , 2018.
  * Panayotov et al. [2015] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur.  Librispeech: an asr corpus based on public domain audio books.  In _2015 IEEE international conference on acoustics, speech and signal processing (ICASSP)_ , pages 5206–5210. IEEE, 2015.
  * Povey et al. [2011] Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al.  The kaldi speech recognition toolkit.  In _IEEE 2011 workshop on automatic speech recognition and understanding_ , number CONF. IEEE Signal Processing Society, 2011.
  * Radford et al. [2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9, 2019.
  * Saon et al. [2017] George Saon, Gakuto Kurata, Tom Sercu, Kartik Audhkhasi, Samuel Thomas, Dimitrios Dimitriadis, Xiaodong Cui, Bhuvana Ramabhadran, Michael Picheny, Lynn-Li Lim, et al.  English conversational telephone speech recognition by humans and machines.  2017\.
  * Schneider et al. [2019] Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli.  wav2vec: Unsupervised pre-training for speech recognition.  In _INTERSPEECH_ , 2019.
  * Sennrich et al. [2016] Rico Sennrich, Barry Haddow, and Alexandra Birch.  Improving neural machine translation models with monolingual data.  In _54th Annual Meeting of the Association for Computational Linguistics_ , pages 86–96. Association for Computational Linguistics (ACL), 2016\.
  * Shapiro [1992] Stuart C Shapiro.  _Encyclopedia of artificial intelligence second edition_.  John, 1992.
  * Sim et al. [2019a] Khe Chai Sim, Françoise Beaufays, Arnaud Benard, Dhruv Guliani, Andreas Kabel, Nikhil Khare, Tamar Lucassen, Petr Zadrazil, Harry Zhang, Leif Johnson, et al.  Personalization of end-to-end speech recognition on mobile devices for named entities.  In _2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)_ , pages 23–30. IEEE, 2019a.
  * Sim et al. [2019b] Khe Chai Sim, Petr Zadrazil, and Françoise Beaufays.  An investigation into on-device personalization of end-to-end automatic speech recognition models.  _arXiv preprint arXiv:1909.06678_ , 2019b.
  * Xiong et al. [2016] Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Mike Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig.  Achieving human parity in conversational speech recognition.  _arXiv preprint arXiv:1610.05256_ , 2016.
  * Xu et al. [2020] Qiantong Xu, Tatiana Likhomanenko, Jacob Kahn, Awni Hannun, Gabriel Synnaeve, and Ronan Collobert.  Iterative pseudo-labeling for speech recognition.  pages 1006–1010, 2020.
  * Yang et al. [2019] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32, 2019.
  * Zhang et al. [2020a] Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  In _International Conference on Machine Learning_ , pages 11328–11339. PMLR, 2020a.
  * Zhang et al. [2020b] Yu Zhang, James Qin, Daniel S Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V Le, and Yonghui Wu.  Pushing the limits of semi-supervised learning for automatic speech recognition.  _arXiv preprint arXiv:2010.10504_ , 2020b.


# Bibliography management: biblatex package

ShareLaTeX

A Comparative Approach to Explainable Artificial Intelligence (XAI) Methods in
Application to High-Dimensional Electronic Health Records: Examining the
Usability of XAI

Jamie A. Duell

##  1 Introduction

Machine Learning approaches are ubiquitous across many domains of application,
altering the way that we approach real-world tasks, firstly, we acknowledge
the two common forms of Machine Learning model designs, these are _Black-box_
and _Glass-box_.

  1. (i)

_Black-box_ models are a commonality for most ML algorithms, as they contain a
sense of dissonance due to the unseen nature of mathematical model computation
and impact on each input combined with a lack of understanding from domain
experts, providing ambiguity in most cases to obtain the given output, these
models often provide the best performance in trade for explainablility, as
they do not provide an explanation as to why a solution is provided. Black-box
models see the application of model-agnostic methods provided to interpret
such relationships, there are various model-agnostic methods that are applied
to provide an interpretation of machine learning methods extending the model
to initialize a form of transparency regarding output, providing an
explainable solution to a model that is otherwise not understood due to the
black-box nature [9], [19].

  2. (ii)

_Glass-box_ models imply that all necessary information is readily available,
the trade-off for such method is often the models performance for an
explainable solution, the term glass-box can often be thought of as synonymous
with the term XAI. Glass-box methods have seen recent implementation with the
likes of models such as the Explainable Boosting Machine (EBM) and DeepLIFT.
EBM is a glass-box model introduced by Microsoft in 2019 [17], model
application provided in section 2.2. The architecture of DeepLIFT can be used
in junction with model-agnostic methods such as SHAP an explanatory method,
this can be seen in the DeepSHAP implementation [8], this concatenation of
both Shapley values and DeepLIFT is out of scope for the implementation of
this paper.

The aim of this paper is to apply Explainable Artificial Intelligence (XAI)
methods to demonstrate the usability of an explainable tertiary layer
appending black-box models. XAI aims to define the feature importance
regarding a models prediction indicative of contribution. Therefore, the use
XAI allows us to extract interpretable knowledge from high-dimensional data-
driven questions to inform domain experts of trends that can be identified
through the categorical and numeric features that can be difficult for a human
expert due to data quantity, this intending to help inform decisions and place
as a assistant - not a substitute to a human expert, forming a tertiary layer
of knowledge, where we have: the patient data, the patient data observations
from a domain expert; and an explainable solution to act as the aforementioned
subsidiary tertiary layer.

Notably, explainable solutions can also be regarded as interpretable, let us
differentiate _interpretable_ and _explainable_ , in some cases these terms
are used interchangeably, but for the purpose of clarity we define

  1. (i)

_Interpretable models_ provide a visual representation of the data that
determines the model findings, allowing the user to gauge an understanding of
the output, this giving room to user interpretation as there’s minimal clarity
to reasoning, allowing a flexible view on the results, this can differ between
individuals.

  2. (ii)

_Explainable models_ are models that provide direct description as to the
results and relationships that support the model prediction, as such
simplifying and communicating a complex domain, introducing clarity and
transparency, though human interpretation is not redundant as the provided
explanations are model dependent and can see different results giving room for
a human-expert.

This work will present state-of-the-art models that can be defined as
explainable AI solutions, covering the usability of model-agnostic Machine
Learning extensions to black-box models and glass-box model integration to
provide data interpretability for large-scale Electronic Health Records (EHR).

The medical field for one has seen a pursuit of interest for the use of
Machine Learning (ML) algorithms, we can see challenges arise when dealing
with the circumstance of human-subjects, the ideology behind trusting a
machine to tend towards the livelihood of a human poses an ethical conundrum –
leaving trust as the basis from humans towards the machine capabilities with
extrapolation from patients to external parties such as insurance companies
who hold investment in procedural success [18], the aforementioned paper
highlights that human trust is a key element for human-AI collaboration
ranging from the medical field to vehicle automation [2], as we idealise the
application of autonomous behaviours it does pose questions to human response
and acceptance of the integration of AI ubiquitous across domains of study and
practicality, with the intent to provide a better performing solution than a
human-expert alone, but a commonality arises in algorithm application, to
where the model can exhibit different behaviours in a non-deterministic manner
compromising the trust factor to where black-box models become an adversary to
trust, though machines do often outperform human experts - how can we assure
the AI solution is trustworthy? The work of Tonekaboni et al. [5] surrounds
the question of ’what clinicians want?’ It is identified that merely having a
highly accurate Machine Learning model is not sufficient to be adopted by
clinical staff, notably a single metric such as classification accuracy does
not provide insight to how the solution was obtained or provide depth to the
models effectiveness [7], interpretability is not heavily required when a
problem has been well studied, this indicating a per-case dependency varying
in necessity when approaching fields such as medicine, where there a
requisition of clarity needed due to the fragile nature of data in the field,
with the field being less familiar with DL implementations. There’s a subset
of algorithms that are commonly used as interpretable models, examples being:
Linear regression, logistic regression and a decision tree, these providing
interpretability on a modular level, with the Naïve Bayes classifier and
k-Nearest Neighbours also being considered as interpretable due to distance
observations on local data points providing a form of observational
interpretability, but are less reliable explainable solutions [15].

###  1.1 XAI and Understanding Explainability

The recent field of study XAI, the term was first introduced in 2004 [11] by
Lent et al., a subset of AI that aims to provide intelligible explanations to
the end user, the need for XAI as defined in [1] applies as milestone of
mitigation towards model doubt, to adhere to ethical concern or regulatory
considerations that need to be made within the domain, should there be bias or
discriminatory results – but, should we gain trust trust, we as humans could
tend towards reliance and complacency [16] as we become comfortable with such
systems as opposed to a supplement to human-expert domain knowledge. Appending
to the given concerns, in an attempt supplement notable negative contributors
with AI application, there is likelihood of regulation application for AI
systems from a legality standpoint regarding black-box model integration as
identified by the work of R. Hoffman et al. [4] where it is deemed a “right to
explanation” maybe a necessary protocol and the user rights must be safe
guarded, as such XAI provides a milestone towards the direction of machine-
based reasoning.

The work of [6] defines an explanation as a social process subject to two
roles taking fruition and two or more participants, the roles consisting of 1)
a subject explaining and 2)a subject receiving the explanation, given this,
human subject participation can hold two forms of contribution with
explanation experimentation as identified in [6], where one case – is the
receival of feedback, provided to determine the usability of the explanation
ensuring clarification of a pragmatic understanding of XAI explanation; the
opposing approach being feedforward, this intends for the human participant to
provide a general explanation, that the algorithmic explanation is compared
against providing a benchmark of comparison. The research of Miller [14]
pinpoints the term XAI as a subset of Human-Agent Interaction located at the
intersection of Social Science, Human-Computer Interaction and Artificial
intelligence, the paper a strong endeavour, linking XAI with social science,
where there’s exploration of the different levels that categorise a potential
explanation, these used to identify where in the pipeline we’re defining the
posed question that derives our explanation – with regards to answering the
question. The intent of this study, is to provide XAI generated explanations
for the given predicted results of the patient case decision, producing a
determinant as to why the case is predicted like-so for further domain-expert
evaluation. Referring to user interaction, we must obtain the correct
mechanisms of approach to gain useful feedback, one form of posing a question
is providing confidence intervals that can be assessed through the feedback,
this seen for a medical case-study, with the implementation seen in the works
of [3], where domain-experts have been assessed, by being presented cases to
determine on a 5-value Likert scale the confidence of the decision made for a
given explainable solution, Tonekaboni et al. [5] identify three key metrics
for evaluating XAI, these being

  1. 1.

_Domain Appropriate Representation_ ; identifying the coherence of
representation with respect to a domain specific task, ideally nullifying
redundancy in explanation, therefore, we can identify from the applied area
what information would be valuable to present in explanation, in essence
filtering the response to the user.

  2. 2.

_Potential Actionability_ ; idealistically inform clinical workflow via
explanation, should the model itself present an account of basic trust
principles, as factors such as patient similarity is not informative, usable,
and can in times cause unnecessary cognitive overload, as such, XAI should
place explanations that can lead to decisions with identification of factors
that allow for follow up workflow.

  3. 3.

_Consistency_ ; aligns with logical mapping of approximate 1-1 domain
knowledge to model explanation, where changes in prediction can yield a
representational change in explanation.

It’s stated that there’s already awareness in the clinical field that there is
no perfect prediction, but sole focus from this paper defines an acceptable
explanation as one that falls in line with justification of their clinical
decision making that align with medical practice, it is also noted that local
explanations are highly valued, with a preference for patient individuality
and feature importance with regards to the singular instance. A method of
approach for future work would be the introduction of a user-study in response
to being provided explanations, this would ideally be an amalgamation of the
aforementioned feed-forward and feed-backward methods, as the initial approach
of feedforward will define a benchmark of explanatory comparisons based on
input feature predictions to where we can gauge a human-expert opinion as a
blind comparison, the human-expert will then see the given explanation to see
if the explanations holds a form of usability and if anything is newly
identified from the given explanations, the intent to provide explanations
from each algorithm implementation to determine the most useful. The purpose
behind such intent is that curiosity is noted as a key aspect of seeking an
explanation [4], as, if the explanation is provided initially, it could skew
the natural thought process of human reasoning and modify the answer post
explanation. Therefore, to mitigate such risks, the intent in future work
would be to first gain user explanation, as factors such as trust and
curiosity can be measured through user study to gain insight on the XAI model
applicability, as agreeing with the model is one step, but the key element of
progress for XAI application is the acceptance of explanation from human-
emotional response, this an identifiable gap for future work. For the
demonstrative purpose of this paper, we define the objectives of the study to
be 1) To demonstrate the explainable visualization of XAI for high-dimensional
medical data. 2) To contrast the different model-agnostic methods and the
extraction of feature importance, providing commonalities and differentiation
amongst each. 3) To determine pros and mitigate cons of the XAI model to
provide aid to human-experts approaching high-dimensional data. Given these
objectives, we introduce four XAI models; three being model-agnostic black-box
extensions and one a glass-box method, with the ideology surrounding the
demonstration of XAI as a supplementary tertiary layer for domain-expert
interpretation, enabling the consideration of practical application in the
medical field, the XAI methods are namely: Shapley Additive Explanations
(SHAP), Local Interpretable Model-Agnostic Explanations (LIME), Scoped Rules
(Anchors) and Explainable Boosting Machine (EBM).

XAI proposes a form of reasoning, this is often communicated in natural
language and through the use of diagrammatic representation, identifying the
route associative to the decision making process of the ML model, given that
the representation of a models performance in traditional ML application does
not provide complete reasoning on feature importance and the correlation
between the model inputs and the model output..

##  2 Method

###  2.1 Case Study and Data Pre-processing

Though existing models are effective in providing a predictive accuracy that
can be deemed as successful, there is a certain scepticism when it comes to
interpreting data from any Machine Learning (ML) model, vulnerabilities can be
projected in the field of medicine from a misinterpretation of results and
findings, it’s critical when dealing with patient data that clarity is
portrayed by the model alone to validate the given results so relationships
between direct patient features and external associations such as regimen
alterations, this issue can be approached by developing a transparent and
idealistically transferable model, not a specific domain altered model, this
defined as an Explainable Artificial Intelligence (XAI) solution, with the
intent to communicate model correlations determining the route taken to obtain
the results. Conclusively, XAI needs to be explored in the field of healthcare
to enable better transparency giving clear reasoning-based performance.

The systematic intent behind the development of the model is to obtain useful
information from a given domain. This environment space will contain features
of large dimensionality to deduce a prognostication of trends, patterns and
reasoning once the models have been applied, to extract valuable information
to answer specific questions surrounding a given domain, this will be an
aggregation of many feature inputs, to obtain useful information from a given
research question.

Data for this paper used artificial data from the Simulacrum, a synthetic
dataset developed by Health Data Insight CiC derived from anonymous cancer
data provided by the National Cancer Registration and Analysis Service
(NCRAS), which is part of Public Health England, this will be used to develop
the XAI framework. The Simulacrum data set consists of 1,322,100 synthetic
patients allowing for model development for researchers whilst maintaining
clear patient confidentiality, reflecting a high degree of accuracy reflective
of properties found in the NCRAS data set, allowing for transferable developed
models from synthetic data sets to real-world data sets.

The adoption of the Simulacrum data set enables the capabilities for ML model
development on big data, with initial steps of the project involving data pre-
processing to allow for the data to use ML effectively, the initial phase of
development prioritised the creation of a master table concatenating both
Cancer Registration (AV) and Systemic Anti-Cancer Therapy (SACT) tables with a
total of 63 columns, the available tables with the associated number of
columns is illustrated in table 4.

Table Name | No. Associated Columns
---|---
sim av patient | 6
sim av tumour | 16
sim sact patient | 2
sim sact tumour | 5
sim sact cycle | 6
sim sact drug detail | 10
sim sact regimen | 10
sim sact outcome | 8
Table 1: An overview of the available Simulacrum data set tables with the
numeric value of associated columns

Firstly, we must deduce a set of different medical questions to apply the base
ML methods and XAI framework, therefore, we identified a set of four
supervised classification problems, two with a focus on all given patients,
and two reduced to a specific cohort of patients, these namely:

  * •

_Predicting the likelihood of reducing any patients drug dose._

  * •

_Predicting the likelihood of mortality in any given patient._

  * •

_Predicting the likelihood of reducing a lung-cancer patients drug dose._

  * •

_Predicting the likelihood of mortality for lung-cancer patients._

Following this, the intervention of data set manipulation allows for the
construction of a machine teachable model, this is antecedent to the
application of ML methods themselves, in the sense of morphing the existing
data with methods such as encoding, this step is to provide data that removes
null values or hard to teach unexplanatory patterns, instances in this case
could include logical inconsistencies such as having a patients weight or
height that is illogical of being, and cases where undergoing a regimen after
death, or vital status being claimed as alive at such time, this being
reflective of real-world noisy data allowing for beneficial practice of
technique implementation, allowing for a viable model demonstrator. Following
this, the data set is then balanced to the lower bounds bias for each binary
classification output, covering lung cancer (LC) patients for both mod dose
reduction (MD) and mortality prediction (DA), and all patients.

Bounds | LC-DA | LC-MD | MD | DA
---|---|---|---|---
Lower | 25456 | 21567 | 181626 | 384375
Higher | 95944 | 27752 | 1034230 | 830434
Updated Higher | 24000 | N/A | 180000 | 380000
Table 2: An overview of the updated output balance for each question

We compose the given data set in a tabular format where each row corresponds
to a singular patient visit, this singular input will be comprised of a set of
features that will be applied to each question posed without selecting
features based on question bias, therefore, the set of features fed into the
model, question dependant should extract a different level of feature
importance given the direction of the question. Ideally, feature dependency
will alternate between each question focus, as such we can assume association
has been appropriately made regarding domain specificities. Therefore, we
define a set of input features from the Simulacrum data set, these identified
in table 3 with the name in the original data set to the altered name for
output clarity, supported with a description of the features, the feature name
’ _NEW VITAL STATUS DESC_ ’ is used as the supervised output for mortality,
whereas ’ _REGIMEN MOD DOSE REDUCTION_ ’ is used for the supervised output for
the reduction of drug dosage, these are not inversely applied as input
features to the opposing models, to keep inputs to the model synchronous.

###  2.2 Baseline Model Implementation

To obtain results from model-agnostic methods we must first execute black-box
algorithms on the given data sets, the two decided algorithms which will
inherit an explanation are Logistic Regression and XGBoost, this to which we
can compare against a glass-box method an Explainable Boosting Machine.
Therefore, provided in the summative bar charts is a comparative approach
identifying base performance metrics being precision, recall and accuracy of
each applied baseline model, these shown in figure 1.

Figure 1: Comparison of algorithm performance across each available data set Dataset |  |  Precision (%) |  Recall(%) |  Accuracy(%)
---|---|---|---|---
| _Logistic Regression_ | 75 | 75 | 75
DA | _XGBoost_ | 76 | 76 | 76
| _EBM_ | 74 | 74 | 74
| _Logistic Regression_ | 68 | 68 | 68
LC-DA | _XGBoost_ | 78 | 78 | 78
| _EBM_ | 67 | 67 | 67
| _Logistic Regression_ | 64 | 64 | 64
MD | _XGBoost_ | 68 | 68 | 68
| _EBM_ | 62 | 62 | 62
| _Logistic Regression_ | 68 | 67 | 67
LC-MD | _XGBoost_ | 75 | 71 | 71
| _EBM_ | 64 | 62 | 62
Table 3: Summary table of baseline performances for logistic regression,
XGBoost an EBM tested on each dataset, where DA = All patients for mortality
prediction, LC-DA = Lung Cancer patients for morality prediction, MD =
Reduction of drug dosage for all patients and LC-MD = Lung Cancer patients for
the reduction of mod dosage

From this comparison, we can deduce that the XGBoost algorithm is the best
performing across all the given datasets, this can be applied as the baseline
algorithm for the extension of model-agnositc solutions.

###  2.3 Model-Agnostic Techniques and Application

For the formulation of XAI techniques on the posed questions, we apply three
different model-agnostic techniques to obtain the decisive input features for
each question further evaluated in section 2.3.4, for the introduction of the
model-agnostic models - SHAP, LIME and Scoped Rules (Anchors), the
visualization examples will be provided from a single case-study from the LC-
DA dataset and apply each method with the supported explanations as a
demonstrator, the case defined in figure 2, with supporting explanations in
sections 2.3.1, 2.3.2 and 2.3.3.

Figure 2: Case study for explanation demonstrations for predicted case:
Deceased — Actual case: Deceased

####  2.3.1 Shapley Additive Explanations

Shapley Additive exPlanations (SHAP) is a model-agnostic framework introduced
by Lundberg et al. [10] that is intended to explain ML algorithms through
visualization, introduced 2 years posterior to LIME, with a similar intent as
LIME aiming to highlight feature relevancy, re-instantiating how there’s an
common goal of gaining user trust by obtaining an explanation.

The intent of SHAP is to explain the given prediction through calculating the
feature importance for the ML prediction. The use of Shapley values directs
the model on how to fairly distribute the feature or a group of features
importance which we determine as players in game theory, therefore, importance
is based on the player investment, the implementation of Shapley value sees
the incorporation of a characteristic function game G, for a number of players
N = 0,1,2,..$n$, ensuring that coalitions $G\subseteq N$, where the shapley
value calculates average contribution over a permutation of players. Notably,
additive explanations can be defined as a linear function of binary inputs,
represented

$g(z^{\prime})=\phi_{0}+\sum_{k=1}^{M}\phi{{}_{k}}z{{}^{\prime}_{k}}$

Where we identify g as the given explanation for an original prediction f(x)
where $z{{}^{\prime}}\subseteq\\{0,1\\}^{M}$, to where M is the number of
simplified inputs, explanatory models use simplified feature inputs due to the
complexity of the original input data, finally we have $\phi_{k}$, this is
feature attribution for feature $k$, we then obtain $g(z^{\prime})$ the sum of
all feature contributions for the linear model g which is fit for the
simplified application of $z^{\prime}$. The localised contribution of features
on the identified case produces the output displayed in 3.

Figure 3: SHAP providing an explanation for a patient, with the width of each
descriptive block and colour are indicative of the shift in probability to a
given case. In this case Red indicating a shift towards death which is
countered by a small adjustment of blue, providing an overall function of (x)
providing the likelihood of the patient being deceased.

####  2.3.2 Local Interpretable Model-agnostic Explanations

Regarding extensions to ML models to provide an explainable model, Local
Interpretable Model-agnostic Explanations (LIME) is an example of such model
agnostic method that was proposed by Ribeiro et al. [12] that explains a
choice indicating feature importance towards a decision, providing somewhat of
an explanation towards the solution, LIME can be applied to any ML model, with
the intent to understand the model based on how the prediction changes as it
passes through the model to obtain the output based on the input data samples,
notably, when LIME is used, it’s still required that the end-user must
interpret the output correctly.

Surrogate models have the goal of providing an approximation for the
predictions of a black-box machine learning model, the explanation model
minimizes the loss and measures whether or not the explanation is close to the
prediction of the original model, it’s notable that the model only optimizes
the loss [15]. LIME can be applied to any type of given data, the model of
LIME namely focuses on the local interpretablility of a model, this by
accessing a single input feature a fitting a line of linearity using a
regularization constraint to the linear regression model, we try to find
$\phi$ by minimising loss, where we measure the faithfulness of the
explanation $g$ to the original $f(x)$ for a local prediction, we can view
this as $L(f,g,\pi{{}_{x}})$ applying the regularization parameters
$\Omega(g)$ where L denotes the square loss function to which we try minimize,
therefore, we define the explanation for a local point $x$ using LIME as

$E=arg\ min\ L(f,g,\pi{{}_{x}})+\Omega(g)$

In practice, we can extract local explanations using LIME determining the
considered most important features for the local linear model, the left of
Figure 4 showing the probability for towards each classification, with the
right displaying each feature importance towards the outputs, in the given
example we see M Best and Behaviour decreasing the likelihood of survival,
with the largest value of importance, represented by the block length and
contributing value 0.21, 0.12, contrasting to that of the feature that shift
probability of living, in turn concluding to a large death probability,
notably, the initial 8 priority features are represented, it is possible to
show the entirety of feature contribution, the figure intent demonstrates the
communicative capabilities of XAI visualization.

Figure 4: LIME providing an explanation given probability of Alive / Deceased
cases providing in the top left, supported by each feature value and
supporting importance value with the corresponding class to which each feature
is weighted towards.

####  2.3.3 Scoped Rules

Scoped rules (Anchors) introduced in 2018 by Ribeiro et al. [13] is used to
explain individual predictions of a classification model by prioritising a
search for a decision rule to increase the prediction likelihood. The intent
behind anchors to integrate, much like LIME in section 2.3.2 a perturbation-
based strategy is used to create an explanatory output of black-box machine
learning models. Anchors follow the idea of factual explanations, identifying
cases where certain anchors given a condition fit to obtain the an output.

Differentiation from LIME is evident in the explanation structure, providing a
rule-based format of communication with base IF-THEN rule, these being the
anchors. The introduction to such extension of LIME provides a better
localized understanding. An Anchor can be defined as followed:

$\mathbb{E}_{D_{x}}{}_{(z|A)}[\mathbbm{1}_{f(x)=f(z)}]\geq\tau,A(x)=1$

Where predicate $A$ is an anchor if the expected evaluation of the neighbours
of instance $x$ of the distribution $D$ matching $A$ is greater-than or equal-
to the precision boundary set on $\tau$, therefore, only rules that achieve
this local fidelity are viable and then will be considered an anchor.

An example of Anchor usability demonstrated in [13] shows the prevalence of
Anchors for natural language processing application, with local instance of a
word such as ”bad” or ”not” have negative connotation, though not reflecting
upon the full statements sentiment, an example being ”This movie is not bad”
this could be misinterpreted to have negative sentiment due to isolated words
with an unseen region, this could put salience using weights on the word ”not”
and correlate the statement to a negative sentiment, whereas, the introduction
of anchors attempts to associate word relations through coverage of segmented
phrases such as ”not”, ”bad” having a positive connotation explanation and
”not”, ”good” having a negative connotation explanation, which could otherwise
be misinterpreted, especially if the instances are presented to the user with
no structure, we will proceed to identify the explanation on a tabular data
set, as presented by the given case.

Figure 5: Anchors gives a conditional conjunction of cases these being Anchors
which identify given certain elements are true, these Anchors play importance
on the outcome. In this case there is a single Anchor, but this can be a
string of ’AND’ conditions that would be prevalent in some other cases.

####  2.3.4 Analysis of model-agnostic Application

We apply these listed three model-agnostic solutions, to the given four
questions to provide a comparative approach for feature importance each a
binary classification problem. Through the use of SHAP and EBM, we can
introduce the extraction of global feature importance, the given example was
trained using XGBoost on Lung Cancer patients in the binary classification
[”Dead”, ”Alive”] introduced in section 2, the SHAP model in Figure 6 denoting
negative weighting towards output solution on the left or positive on the
right, of 0 on the $x$ axis, shifts towards an output, with a colour gradient
representation of the feature value, with the next image in Figure 7
demonstrating the absolute value of feature importance through the use of EBM
to provide a comparison on feature ordering and representation of global
importance.

Figure 6: SHAP global explanation, the x-axis provides a weighting with
positive SHAP values shifting towards deceased and negative SHAP values
shifting towards death, with the width of the given data point scattering to
the level of importance it has for each instance, therefore ‘0.0’ indicative
of minimal impact. The feature value is also represented from low to high
based on colour as shown on the right vertical bar, this for each feature over
the entire dataset. Figure 7: EBM global explanation, the x-axis shows the
mean absolute importance for the given features, indicative of the impact of
each feature over the entire LC-DA dataset.

Firstly, we have LIME this isolating an individual patient instance as a local
linear fit. Through the LIME framework we can provide a view of feature
importance for a single instance as interpreted by the LIME model, this then
explained the end user,

Secondly, we then apply SHAP, where we view each instance of both the feature
and feature value of a singular record as the ’player’ to determine the
players contribution to the obtained output, therefore, in our context we view
each tabular category e.g ”Site=C50”, ”Age=70”etc. as the players, we can then
obtain the contribution towards one of the two output classes, in this case
being either [’Y’, ’N’].

We can view the shared features by absolute value irrespective of positive or
negative connotation towards to the model result, where they are contributing
in order of importance such that the named data set e.g. MD, for models SHAP
and LIME share a named feature $F$ concluding that $SHAP(F_{k})$ ==
$LIME(F_{k})$ where $k$ is the selected feature index, we can conclude this
for multiple features by returning shared features up to $k$, for each data
set we allow for where k = 1, k = 2 and k = 3, these demonstrated for each
data set independently shown in Figures 8 and 9.

Figure 8: Comparing the shared features across both DA data sets using SHAP
and LIME

Figure 9: Comparing the shared features across both MD data sets using SHAP
and LIME

From these shared features we can see inconsistencies amongst the shared
features, though given this fact, inverse comparisons where e.g. $SHAP(F_{1})$
== $LIME(F_{2})$ there are a high majority of features, as priority on the
most important feature, as shown in the first 1000 iterations in figure 11, it
graphs that the most important feature is the same feature for a high
percentile of the designated population, it would seem that the most important
feature in some cases, aren’t inline with one another, to draw such comparison
we view the MD data set as an example where the shared features for k = 1
provides the smallest percentage, therefore we compare $SHAP(F_{2})$ and
$SHAP(F_{3})$ against $LIME(F_{1})$ providing evidence of differential in
priority order, returning the shared features shown in Figure 10.

Figure 10: Comparing the shared features on the MD data set using SHAPs second
and third feature list and LIMEs first

Given such information, we can also determine priority features from the data
sets, though they may not be shared for each instance, it’s optimal that the
identified important features are relational to a certain extent for
validation supporting the consistency of knowledge representation. Therefore,
we provide a comparison across the four data sets, where we take the first
1000 instances from the test data set to extract the most important feature
where $k=1$ for each LIME and SHAP as well as the first Scoped Rules anchor,
where we determine the most influential factors towards a patient either
[”Dead”, ”Alive”] or have reduced drug modification for the current regimen
[’Y’, ’N’].

Evidently, commonalities arise across the shared most important feature for
each XAI application, only matched ordering for features priority is less
common for each model, this differentiating in balance across the data sets,
where we see SHAP holding priority on weight for the LC-MD data set conversely
to LIME having Time Delay as the greatest priority, where we see a more
balanced dissemination across the three features for the SHAP model, notably,
LIME shares percentiles of the same importance regarding features, we can see
that Scoped Rules also determines shared anchors across the same set, this
displayed in Figure 11.

Figure 11: Most important feature returned or the first anchor (scoped rules)
for the first 1000 instances on the test data set for both LC-DA and LC-MD
data sets

##  3 Future Work and Conclusion

Determining feature priority from a local interpretation provides a sense of
clarity for non-domain experts with a base clarity of feature contribution to
a given case. As appointed in section 2.3 we derive a slight aberration in the
feature attribution priority order amongst the model-agnostic solutions,
though still sharing a level of importance across three features. For future
work, to support the discernible necessity of human-expert trust, we aim to
carry out a user study to determine effectiveness of the given explanations,
the direct extraction of $k=3$ feature importance comparison provides
discomfort in reliability as it poses questions towards the validity of
explanation, as such it was then shown a majority of features are shared
across those 3 important features, only not in junction for all use cases,
this providing support of clear commonality, given the data is synthetic, the
provided explanations are that of demonstrative visualization and explanation
usability, that from future work can determine validity.

The ability to extract explanations of black-box model predictions is
essential for high-risk applications of machine learning; medical application
being one example, from such information of patterns across features and
different explanation models we can determine data set and supporting
predictions fairness. Explanation locality allows for new instances to be
communicated to the domain-expert with reason. This tertiary layer of
knowledge could improve the rate of case deduction and support human-expert
reasoning.

## References

  * [1] A. Adadi and Mohammed Berrada  “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)”  In _IEEE Open Access Journal_ , 2018  DOI: 10.1109/ACCESS.2018.2870052
  * [2] C. al.  “Trust in Automation – Before and After the Experience of Take-over Scenarios in a Highly Automated Vehicle”  In _Procedia Manufacturing Volume 3, 2015, Pages 3025-3032_ , 2015  DOI: https://doi.org/10.1016/j.promfg.2015.07.847
  * [3] J. al.  “Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach”  In _Artificial Intelligence in Medicine Volume 94, March 2019, Pages 42-53_ , 2019  DOI: doi.org/10.1016/j.artmed.2019.01.001
  * [4] R.R Hoffman al.  “Metrics for Explainable AI: Challenges and Prospects”  In _arXiv preprint arXiv:1812.04608_ , 2018
  * [5] S. al.  “What Clinicians Want: Contextualizing Explainable Machin e Learning for Clinical End Use”  In _arXiv preprint 1905.05134_ , 2019
  * [6] M. Chromik and M. Schuessler  “A Taxonomy for Human Subject Evaluation of Black-Box Explanations in XAIs”  In _ExSS-ATEC@ IUI_ , 2020
  * [7] F. Doshi-Velez and B. Kim  “Towards A Rigorous Science of Interpretable Machine Learning”  In _arXiv:1702.08608_ , 2017
  * [8] S. H. and S. Lee  “Explaining Models by Propagating Shapley Values”  In _arXiv:1911.11888_ , 2019
  * [9] O. Loyola-González  “Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View”, 2019  DOI: 10.1109/ACCESS.2019.2949286
  * [10] S. Lundberg and S.Lee  “A Unified Approach to Interpreting Model Predictions”  In _arXiv:1705.07874_ , 2017
  * [11] William Fisher M. and Michael Mancuso  “An Explainable Artificial Intelligence System for Small-unit Tactical Behavior”  In _Proceedings of the Nineteenth National Conference on Artificial Intelligence, Sixteenth Conference on Innovative Applications of Artificial Intelligence_ , 2004
  * [12] S. M. and C. Guestrin  ““Why Should I Trust You?” Explaining the Predictions of Any Classifier”  In _arXiv:1602.04938_ , 2016
  * [13] S. M. and C. Guestrin  “Anchors: High-Precision Model-Agnostic Explanations”, 2018
  * [14] T. Miller  “Explanation in Artificial Intelligence: Insights from the Social Sciences”  In _Artificial Intelligence Volume 267, February 2019, Pages 1-38_ , 2019  DOI: doi.org/10.1016/j.artint.2018.07.007
  * [15] C. Molnar  “Interpretable Machine Learning: A Guide for Making Black Box Models Explainable.” https://christophm.github.io/interpretable-ml-book/index.html, 2020
  * [16] S.T Mueller  “Explanation in Human-AI Systems: A Literature Meta-Review Synopsis of Key Ideas and Publications and Bibliography for Explainable AI”  In _arXiv preprint 1902.01876_ , 2019
  * [17] Harsha Nori, Samuel Jenkins, Paul Koch and Rich Caruana  “InterpretML: A Unified Framework for Machine Learning Interpretability”  In _arXiv preprint arXiv:1909.09223_ , 2019
  * [18] A.E Bayrak O. and A. Choudhury  “Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians”  In _Journal of Medical Internet Research (JMIR)_ , 2019  DOI: 10.2196/15154
  * [19] C. Rudin  “Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead”  In _arXiv preprint arXiv:”1811.10154_ , 2019

##  4 Appendix

Feature Name in Data set | Feature Name Transformed | Feature Description
---|---|---
AGE | Age | Age of the patient at the time of instance
SEX DESC y | Sex | Sex of the patient
WEIGHT AT START OF REGIMEN | Weight | Weight in kg. at the start of the SACT regimen
HEIGHT AT START OF REGIMEN | Height | Height in metres at the start of the SACT regimen
SITE ICD10 O2 DESC | Site | The given ICD-10 Code
MORPH ICD10 O2 | Morph | Morphology ICD-1O code
GRADE | Grade | Indicative of Tumor grade
CANCER CARE PLAN INTENT | CANCER PLAN | The cancer plan
ACTUAL DOSE PER ADMINISTRATION | Dose Administration | Dose in mg for each administration in the SACT cycle
CHEMO RADIATION | Chemo Radiation | Whether or not a patient is undergoing radiotherapy
BEHAVIOUR ICD10 O2 DESC | Behaviour | Behaviour description
REGIMEN OUTCOME SUMMARY DESC | Outcome | A description of regimen outcome
NEW VITAL STATUS DESC | STATUS | Whether the patient is Dead or Alive
T BEST | T Best | The size and extent of the tumor
N BEST | N Best | No. of nearby lymph nodes that have cancer
M BEST | M Best | Whether the cancer has spread from the primary tumor
DRUG GROUP | Drug Group | The drug group name
ADMINISTRATION ROUTE DESC | Administration route | Method of delivery for each cycle administration
REGIMEN MOD TIME DELAY | Time Delay | Time delay between administration
REGIMEN MOD STOPPED EARLY | Stopped Early | Regimen stopped early
MAPPED REGIMEN | Regimen | The mapped regimen name
CLINICAL TRIAL | Clinical Trial | Patient is in an active Systemic Anti Cancer therapy trial
CYCLE NUMBER | Cycle | The current cycle
CNS | CNS | CNS
REGIMEN MOD DOSE REDUCTION | Reduction | Identifies the reduction of the drug dose during regimen
ACE27 | ACE | Adult Co-morbidity Evaluation
Table 4: Features with given description