# UKP-SQuARE: An Interactive Tool for Teaching Question Answering

Haishuo Fang, Haritz Puerto, Iryna Gurevych
Ubiquitous Knowledge Processing Lab (UKP Lab),
Department of Computer Science and Hessian Center for AI (hessian.AI),
Technical University of Darmstadt
www.ukp.tu-darmstadt.de

###### Abstract

The exponential growth of question answering (QA) has made it an indispensable
topic in any Natural Language Processing (NLP) course. Additionally, the
breadth of QA derived from this exponential growth makes it an ideal scenario
for teaching related NLP topics such as information retrieval, explainability,
and adversarial attacks among others. In this paper, we introduce UKP-SQuARE
as a platform for QA education. This platform provides an interactive
environment where students can run, compare, and analyze various QA models
from different perspectives, such as general behavior, explainability, and
robustness. Therefore, students can get a first-hand experience in different
QA techniques during the class. Thanks to this, we propose a learner-centered
approach for QA education in which students proactively learn theoretical
concepts and acquire problem-solving skills through interactive exploration,
experimentation, and practical assignments, rather than solely relying on
traditional lectures. To evaluate the effectiveness of UKP-SQuARE in teaching
scenarios, we adopted it in a postgraduate NLP course and surveyed the
students after the course. Their positive feedback shows the platform’s
effectiveness in their course and invites a wider adoption.

UKP-SQuARE: An Interactive Tool for Teaching Question Answering

Haishuo Fang, Haritz Puerto, Iryna Gurevych Ubiquitous Knowledge Processing
Lab (UKP Lab), Department of Computer Science and Hessian Center for AI
(hessian.AI), Technical University of Darmstadt www.ukp.tu-darmstadt.de

##  1 Introduction

Question Answering (QA) is one of the overarching research topics in Natural
Language Processing (NLP). QA pipelines have been developed to address
different types of questions, knowledge sources, and answer formats, including
extractive, abstractive, knowledge base, multiple-choice, generative, and
open-domain QA. Such a massive number of QA systems and relevant NLP
techniques are making QA lectures more important in NLP courses. However,
despite QA being an application-oriented topic (e.g., chatbots, virtual
assistants, etc.), classes are usually theoretically driven. Thus, in this
paper, we propose the use of the UKP-SQuARE platform as a tool for QA
education. This platform integrates most QA formats, popular models, datasets,
and analysis tools, such as explainability, adversarial attacks, and graph
visualizations.

Compared with conventional teacher-led classes, we propose a learner-centered
class following the flipped classroom (Bishop and Verleger, 2013) with UKP-
SQuARE as the driving tool of the lecture. This tool provides an interface for
users to interact with different QA models and analysis tools. Therefore,
students can actively learn about QA systems and get hands-on experience by
interacting with models on the platform. Concretely, students can flexibly
compare multiple architectures that model different QA formats, analyze their
outputs with explainability tools, and even analyze their robustness against
adversarial attacks. Prior studies have shown that flipped classroom lectures
improve the learning process of students in programming courses (Alhazbi,
2016). Thus, we believe that teaching and learning QA through a live demo with
this platform can also make NLP lectures more engaging, drawing students’
attention, and interest in the topics.

To investigate the effectiveness of UKP-SQuARE in QA education, we adopted it
for the first time in a postgraduate NLP course111Master’s level course and
conducted a survey afterward. The positive feedback from the students
encourages us to continue adopting this platform and education method in more
NLP courses. The contributions of this paper are: i) a novel interactive
learner-centered methodology to teach QA and relevant NLP topics, ii)
extending the UKP-SQuARE platform for teaching QA, and iii) the design of a
syllabus for interactive QA lectures.

##  2 UKP-SQuARE

UKP-SQuARE (Baumgärtner et al., 2022; Sachdeva et al., 2022; Puerto et al.,
2023) is an extendable and interactive QA platform that integrates numerous
popular QA models such as deeepset’s roberta-base-
squad2222https://huggingface.co/deepset/roberta-base-squad2, SpanBERT (Joshi
et al., 2020) for HotpotQA, and QAGNN (Yasunaga et al., 2021). It provides an
ecosystem for QA research, including comparing different models, explaining
model outputs, adversarial attacks, graph visualizations, behavioral tests,
and multi-agent models. In addition, this platform provides a user-friendly
interface333https://square.ukp-lab.de/ that enables users to interact. Users
can run available models, deploy new ones, compare their behaviors, and
explain outputs.

##  3 Learning Question Answering with UKP-SQuARE

In this section, we present the syllabus of a lecture focused on QA and
relevant NLP topics that use the platform UKP-SQuARE following the flipped
classroom methodology Bishop and Verleger (2013). The flipped classroom is an
effective learner-centered educational methodology in which students study
pre-recorded lectures and materials in advance to engage in more interactive
and collaborative learning activities in class. UKP-SQuARE can be the driving
tool for the flipped classroom in QA education. With our platform, lecturers
can introduce the topics by interacting with the students and then proceed to
an in-depth explanation of the technical details behind the methods of each
topic. We propose dividing the lecture into three topics in the QA field:
basic QA concepts, trustworthy QA, and multi-agent QA systems. With these
topics, students can learn about QA and related NLP topics such as information
extraction, explainability, adversarial attacks, and multi-agent systems.

###  3.1 Learning Basic QA Components

QA systems include two main components, i.e., Readers and Retrievers. Readers
are QA models responsible for obtaining answers from the context retrieved by
retrievers. In UKP-SQuARE, students can easily learn various readers (QA
models) within different QA formats and information retrieval techniques via
interacting with the interface.

####  3.1.1 Contrasting Different QA Formats

With UKP-SQuARE, students can get first-hand experience by interacting with
multiple models on our platform. The home readings would include descriptions
of the main QA datasets and their baselines. In class, the lecturer can show
the different QA formats with real demonstrations of the models and explain on
the fly the architectural differences needed to model each QA format. An
example is shown in Figure 1 where a span-extraction QA model, i.e., Span-
BERT, and a multiple-choice QA model, i.e., CommonsenseQA model are presented
to show the difference between these two QA formats. Such interactions can
make theoretical explanations of the architectures easier to digest and,
therefore, the class more engaging.

(a) A span-extraction QA model

(b) A multiple-choice QA model

Figure 1: Different QA formats in UKP-SQuARE

####  3.1.2 Learning Information Retrieval

To learn Information Retrieval (IR) methods, the user interface of UKP-SQuARE
offers a compelling approach to help students differentiate between different
IR methods, e.g., lexical retrieval and semantic retrieval, and understand how
they affect the final performance of QA models. The home readings would
include book chapters or slides describing the main IR methods such as TF-IDF
(Sparck Jones, 1988), BM25 (Robertson et al., 1995), Sentence-BERT (Reimers
and Gurevych, 2019), and Dense Passage Retrieval (DPR; Karpukhin et al.,
2020). Like the above section, the lecturer can guide students to find the
difference between lexical retrieval (e.g., BM25) and semantic retrieval
(e.g., DPR) via playing with UKP-SQuARE by themselves. As shown in Figure 2,
for the question When was Barack Obama’s inauguration?, the BM25 retriever
returns a passage covering all keywords but irrelevant to the question, while
the DPR retriever returns the correct document, which contains the answer to
the question. By providing this example in class, students can easily
understand that DPR retrieves semantically similar passages while BM25 only
retrieves passages that contain the query tokens and, thus, may retrieve
unrelated passages. This could be further explored by comparing two open-
domain QA models implementing these retrieval methods and the same reader
model to demonstrate the error propagation due to irrelevant passages. This
active learning method can prevent the issue of students losing attention that
commonly occurs in traditional lectures Felder and Brent (2003).

Figure 2: Example of difference between using BM25 retriever and DPR
retriever. The red boxes represent keywords in the retrieved passages

###  3.2 Learning Trustworthy QA Systems

In addition to learning basic QA components, it is important to understand how
to identify and evaluate trustworthy QA systems. This involves several related
NLP topics, such as explainability, transparency, and robustness. UKP-SQuARE
provides such analysis tools to facilitate students’ learning process of
trustworthy QA systems.

####  3.2.1 Explainability Methods

The exponential adoption of AI is pushing regulators to adopt policies to
regulate its use. One of the key points they aim to address is the
explainability of these methods to make AI safer444https://digital-
strategy.ec.europa.eu/en/policies/european-approach-artificial-intelligence.
Thus, it is of utmost importance to include explainability methods on AI
courses in Universities. In terms of the explainability of QA models, UKP-
SQuARE includes BertViz Vig (2019) and a suite of saliency map methods to
facilitate the understanding of the model’s decision-making process. Saliency
maps employ attribution-weighting techniques such as gradient-based Simonyan
et al. (2014); Sundararajan et al. (2017) and attention-based Jain et al.
(2020); Serrano and Smith (2019) methods to determine the relative importance
of each token for the model prediction. The descriptions of these methods
would form part of the home readings and to make the classes more active, the
class would be driven by real examples of saliency maps using our platform and
their interpretation. In this way, students can learn how to explain the
output of a QA model based on saliency maps.

An example of a saliency map is shown in Figure 3. The color level of the
highlighted text reflects its importance for the answer. As we can see, of
what celestial body? is the most important part of the question, while sun
gets the most attention in the context, which is the final answer. This means
the model successfully understands the main point of the question and can link
them to the context. Making this type of interpretation can help students
identify potential problems or biases in the models.

Figure 3: An attention-based saliency map of a question in UKP-SQuARE.

####  3.2.2 Behavioral Tests in QA models

The next important component in trustworthy QA is behavioral tests of models.
Machine learning models do not throw errors as regular software programs.
Instead, an error in machine learning is usually an unwanted behavior, such as
a misclassification that may pass inadvertently to a person (Ribeiro et al.,
2020). This makes testing machine learning models challenging. To simplify the
behavioral analysis of machine learning models, Ribeiro et al. (2020) proposes
CheckList, a list of inputs and expected outputs that aims to analyze general
linguistic capabilities and NLP models mimicking the unit tests in software
engineering. The integration of CheckList into UKP-SQuARE offers a simple
method to analyze the performance of QA models beyond traditional benchmarks,
such as MRQA tasks (Fisch et al., 2019).

Figure 4: The result of running CheckList for SQuAD 2.0 RoBERTa Adapter and
BERT Adapter. The number of failed and succeeded test cases are highlighted in
green and red.

As illustrated in Figure 4, we test the SQuAD 2.0 RoBERTa Adapter and SQuAD
2.0 BERT Adapter using the CheckList in which multiple NLP capabilities are
tested like coreference, negation, and robustness. As we can see SQuAD 2.0
BERT Adapter performs worse than RoBERTa Adapter in the above dimensions. Such
an example can be used by the lecturer in class to introduce the idea of
behavioral tests on the fly. In addition, the behavioral tests of UKP-SQuARE
can be used to foster the students’ analytical skills. A potential assignment
could be to train a QA model and deploy it on our platform to analyze it with
the provided ecosystem of QA tools. In particular, thanks to the behavioral
tests in UKP-SQuARE, students can provide a deeper analysis of their model
based on the quantitative results of their test set and a qualitative analysis
based on the behavioral test results.

####  3.2.3 Adversarial Attacks

Policymakers are also designing a regulatory framework that guarantees users
that their AI models are resilient to adversarial attacks555See footnote 3.
Therefore, AI curriculums should also include adversarial attacks to prepare
students for these new regulations.

UKP-SQuARE provides tools to conduct adversarial attacks, such as HotFlip
Ebrahimi et al. (2018), input reduction Feng et al. (2018), and sub-span Jain
et al. (2020). Thus, the home readings should include a theoretical
introduction to these methods. Then, the lecture would use the platform to
exploit the interactive nature of adversarial attacks. In particular, the need
to analyze examples to understand different types of attacks makes this part
of the topic especially practical. Therefore, the lecturer can introduce the
topic through UKP-SQuARE and delve deeper into the technical details
afterward.

Figure 5: A HotFlip example where only flipping . (full stop) to 670 changes
the answer.

An exemplary case is that students can attack real models with examples by
tuning different parameters, such as the number of flips in HotFlip, to see
how the output changes when they subtly change the input data. In Figure 5,
only flipping . (full stop) to wore can directly change the answer. In class,
a small experiment can be set up by lecturers in which students need to
manually manipulate the input to see if it can trick the model into making
incorrect answers and compare it with adversarial attack tools to deepen their
understanding of those adversarial attacks and the importance of building up
trustworthy QA systems.

####  3.2.4 Graph-based QA Models

Knowledge Graph Question Answering (KGQA) systems can have strong explanatory
power thanks to the reasoning paths that can be extracted from the graph. Such
transparency can enhance the interpretability and trustworthiness of the
system. UKP-SQuARE currently offers QA-GNN Yasunaga et al. (2021), a KGQA
model that makes use of ConceptNet (Speer et al., 2017), and provides a
visualization interface to explore the subgraph used by the model.

Although a reasoning path in a graph may provide a clear explanation of a
model’s prediction, we believe that interpreting graph-based models is not
straightforward because, usually, that path contains many irrelevant nodes and
edges that may obscure the actual reasoning of the model. Thus, we propose to
teach KGQA models with real examples of graphs. In this way, the lecturer, or
even the students themselves, have to show the process of cleaning the graph
to obtain and interpret the reasoning path. This process would be much more
valuable for the future endeavor of the students than using a set of slides
with examples of preprocessed clean graphs because they will be able to
reproduce what they learn in real-use cases in companies.

Figure 6: A visualized reasoning graph of the question Where would you find a
basement that can be accessed with an elevator?

###  3.3 Learning Multi-Agent Systems

Lastly, the current progress in QA is pushing toward creating robust models
across multiple domains. To do this, there are two types of approaches: multi-
dataset models and multi-agent models. While the former aims to train a single
architecture on multiple datasets, the latter does the opposite. It trains
multiple models (agents) on single datasets and combines the agents. UKP-
SQuARE is compatible with both approaches; therefore, it is an ideal platform
to teach them.

Thanks to UKP-SQuARE, we can also follow a flipped classroom methodology to
teach multi-agent systems. After reading class materials explaining the models
of this topic at home, the class time can be used as an explanation of the
topic with a live demonstration of these models. In particular, we can easily
show that multi-agent systems such as MetaQA (Puerto et al., 2021) select
different agents depending on the input question. Figure 7 shows that the
first answer selected by MetaQA, which is the correct one, is from an out-of-
domain agent, while the second answer, which is not correct, is from the in-
domain agent. This example illustrates the collaboration between agents
achieved by multi-agent systems and can be an ideal way of starting the
lecture on this topic before explaining the architectural details of MetaQA.
Similarly, the platform can be used to introduce multi-dataset systems such as
UnifiedQA (Khashabi et al., 2020), before delving into in-detail explanations
of the model. In particular, the lecturer can explain the multiple accepted QA
formats by UnifiedQA through real examples, and then, continue the explanation
with the training details of the model with the support of slides.

Figure 7: Multi-Agent QA in UKP-SQuARE: different agents are selected to
predict the answer based on the input

###  3.4 Assignments with UKP-SQuARE

In addition to the above teaching scenarios in class, we also propose a
homework assignment based on UKP-
SQuARE666https://colab.research.google.com/drive/17qw1dLWmU5EDxf9TLR29zIG9-EGKmNxP?usp=share_link
that leverages the insights and knowledge they acquire from the class. The
students need to train their own QA model using the popular Hugging Face’s
Transformer library (Wolf et al., 2020), deploy the model on our platform, and
then write an in-detail report where they analyze their model from multiple
perspectives. This report must include a quantitative analysis of the
performance of their model on the test set and also a qualitative analysis
that includes an explanation of the outputs of the model to a series of input
questions, adversarial attacks that shows errors of their model, and an
analysis of the possible behavioral errors obtain from CheckList. Furthermore,
the students should also compare their model with other available models and
identify the type of questions where their model fails. This would help them
understand that models overfit the domain of their training data and,
therefore, may fail in other domains. This assignment requires students to
truly understand each component they learned during the class, which will help
them consolidate their knowledge and develop a deeper understanding of the
inner workings of different QA techniques. Additionally, the assignment can
serve as a useful assessment tool, enabling teachers to gauge students’
understanding of the material and provide targeted feedback and support as
needed.

###  3.5 User Study

To quantitatively evaluate the effectiveness of UKP-SQuARE in teaching the
above QA techniques, we designed a questionnaire to collect feedback from
students. The questionnaire was administered to a group of students who had
completed a graduate NLP course that used our platform in both class time and
for the assignment. All participants are 20-to-30 years-old graduate students
in computer science. The questionnaire mainly focuses on two aspects: whether
UKP-SQuARE deepens their understanding of techniques in QA systems and whether
it makes it easier to get hands-on experience in UKP-SQuARE. The majority of
questions require students to rate on a scale of 1 to 5. The complete
questionnaire can be found in Appendix A.

Figure 8: Students feedback towards UKP-SQuARE used in QA education.

Figure 8 shows the Likert scale chart with the responses of seven students who
participated in the survey. As we can see, students have very positive
attitudes towards all aspects of UKP-SQuARE for their QA learning. All
participants think that the platform makes the class more engaging and
interesting. In particular, most of them (91%) think UKP-SQuARE helps them
better distinguish different QA formats. For information retrieval, the
majority of the responders do not think that the platform can help them
understand better the difference between lexical retrieval and semantic
retrieval. The main reason behind this is that the difference between lexical
and semantic retrievers is challenging to distinguish only via visualization
unless students actively compare the documents by themselves. Besides, it also
requires students to have a good understanding of semantic similarity and
lexical similarity. Therefore, we plan to improve it by showing the difference
between vector similarity and keyword matching between questions and retrieved
documents. Regarding explainability and adversarial attack tools, around two-
thirds of students believe that the platform facilitates their learning
process of these topics. When it comes to hands-on experience, the vast
majority of students agree that UKP-SQuARE is easy to use. Our platform
provides an infrastructure that dramatically lowers the bar for students to
get hands-on experience. All students think that without UKP-SQuARE, they
would spend more time finding suitable open-source software to compare
different models, analyze the output, and conduct adversarial attacks.
Moreover, the respondents estimated that without UKP-SQuARE, the average time
spent on homework would increase from 2-5 hours to more than 8 hours. One
student also commented that doing experiments with the platform was
straightforward and allowed him to try different ideas without any overhead.
Therefore, although the survey sample is small and limits the conclusions,
this overall positive feedback invites us to continue investigating how to
conduct our QA and NLP classes more interactively with UKP-SQuARE and suggests
that our students would benefit from extending this interactive class to other
NLP topics such as generative pre-trained large language models, prompting
with reinforcement learning from human feedback, word embeddings, parsing
trees, and machine translation among others.

##  4 Related Work

The most relevant tool is the AllenNLP
demo777https://demo.allennlp.org/reading-comprehension/, which provides a user
interface to the main components of the AllenNLP library (Gardner et al.,
2018). This website includes an interface where users can interact with five
extractive QA models. However, their goal is to have a showcase of their
library rather than an extensive platform for teaching QA. Thus, their
functionalities are limited. Most of their deployed models are outdated, only
cover extractive QA settings, and do not provide information retrieval
methods. Moreover, their explainability and adversarial attacks are not
compatible with their transformer-based model. Furthermore, they do not
provide graph-based models, which can be useful to explain graph neural
networks and explainability methods based on graphs. Additionally, it cannot
be used for our homework assignment because users cannot deploy and analyze
their own models with explainability and adversarial attack tools as in our
platform. However, they do provide demos for other NLP topics, such as Open
Information Extraction and named entity recognition, and parsing trees, among
others.

##  5 Conclusion

In this paper, we present a novel method to teach question-answering to
postgraduate NLP students following the learner-centered method of flipped
classrooms. We propose to provide reading materials to the students before the
class and use the UKP-SQuARE platform as a driving tool to conduct the class.
This platform integrates the most popular QA pipelines and an ecosystem of
tools to analyze the available models. These tools include explainability
methods, behavioral tests, adversarial attacks, and graph visualizations. We
provide a series of use cases for teaching based on the provided models and
methods by UKP-SQuARE, showing that classes can become much more interactive
by using UKP-SQuARE than in conventional lectures. To evaluate the
effectiveness of the platform and our methodology, we conducted a survey to
collect feedback from students who took our class. The results show that most
of the students think UKP-SQuARE accelerates their learning process and
reduces the overhead to get hands-on experience. We plan to extend our
platform to support prompting large language models, and therefore, we leave
as future work creating a curriculum to teach prompting methods.

## Acknowledgements

We thank Max Eichler, Martin Tutek, Thomas Arnold, Tim Baumgärtner, and the
anonymous reviewers for their insightful comments on a previous draft of this
paper. This work has been funded by the German Research Foundation (DFG) as
part of the UKP-SQuARE project (grant GU 798/29-1), the QASciInf project (GU
798/18-3), and by the German Federal Ministry of Education and Research and
the Hessian Ministry of Higher Education, Research, Science and the Arts
(HMWK) within their joint support of the National Research Center for Applied
Cybersecurity ATHENE.

## References

  * Alhazbi (2016) Saleh Alhazbi. 2016.  Using flipped classroom approach to teach computer programming.  In _2016 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)_ , pages 441–444.
  * Baumgärtner et al. (2022) Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva, Gregor Geigle, Max Eichler, Clifton Poth, Hannah Sterz, Haritz Puerto, Leonardo F. R. Ribeiro, Jonas Pfeiffer, Nils Reimers, Gözde Şahin, and Iryna Gurevych. 2022.  UKP-SQUARE: An online platform for question answering research.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_ , pages 9–22, Dublin, Ireland. Association for Computational Linguistics.
  * Bishop and Verleger (2013) Jacob Bishop and Matthew A Verleger. 2013.  The flipped classroom: A survey of the research.  In _2013 ASEE Annual Conference & Exposition_, pages 23–1200.
  * Ebrahimi et al. (2018) Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018.  HotFlip: White-box adversarial examples for text classification.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 31–36, Melbourne, Australia. Association for Computational Linguistics.
  * Felder and Brent (2003) Richard M Felder and Rebecca Brent. 2003.  Designing and teaching courses to satisfy the abet engineering criteria.  _Journal of Engineering education_ , 92(1):7–25.
  * Feng et al. (2018) Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber. 2018.  Pathologies of neural models make interpretations difficult.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3719–3728, Brussels, Belgium. Association for Computational Linguistics.
  * Fisch et al. (2019) Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. 2019\.  MRQA 2019 shared task: Evaluating generalization in reading comprehension.  In _Proceedings of the 2nd Workshop on Machine Reading for Question Answering_ , pages 1–13, Hong Kong, China. Association for Computational Linguistics.
  * Gardner et al. (2018) Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018.  AllenNLP: A deep semantic natural language processing platform.  In _Proceedings of Workshop for NLP Open Source Software (NLP-OSS)_ , pages 1–6, Melbourne, Australia. Association for Computational Linguistics.
  * Jain et al. (2020) Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, and Byron C. Wallace. 2020.  Learning to faithfully rationalize by construction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4459–4473, Online. Association for Computational Linguistics.
  * Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020.  SpanBERT: Improving pre-training by representing and predicting spans.  _Transactions of the Association for Computational Linguistics_ , 8:64–77.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6769–6781, Online. Association for Computational Linguistics.
  * Khashabi et al. (2020) Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. 2020.  UNIFIEDQA: Crossing format boundaries with a single QA system.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1896–1907, Online. Association for Computational Linguistics.
  * Puerto et al. (2023) Haritz Puerto, Tim Baumgärtner, Rachneet Sachdeva, Haishuo Fang, Hao Zhang, Sewin Tariverdian, Kexin Wang, and Iryna Gurevych. 2023.  UKP-SQuARE v3: A Platform for Multi-Agent QA Research.  _arXiv preprint arXiv:2303.18120_.
  * Puerto et al. (2021) Haritz Puerto, Gözde Gül Şahin, and Iryna Gurevych. 2021.  Metaqa: Combining expert agents for multi-skill question answering.  _arXiv preprint arXiv:2112.01922_.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Ribeiro et al. (2020) Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020.  Beyond accuracy: Behavioral testing of NLP models with CheckList.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4902–4912, Online. Association for Computational Linguistics.
  * Robertson et al. (1995) Stephen Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, and M. Gatford. 1995\.  Okapi at trec-3.  In _Overview of the Third Text REtrieval Conference (TREC-3)_ , pages 109–126. Gaithersburg, MD: NIST.
  * Sachdeva et al. (2022) Rachneet Sachdeva, Haritz Puerto, Tim Baumgärtner, Sewin Tariverdian, Hao Zhang, Kexin Wang, Hossain Shaikh Saadi, Leonardo F. R. Ribeiro, and Iryna Gurevych. 2022.  UKP-SQuARE v2: Explainability and adversarial attacks for trustworthy QA.  In _Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing: System Demonstrations_ , pages 28–38, Taipei, Taiwan. Association for Computational Linguistics.
  * Serrano and Smith (2019) Sofia Serrano and Noah A. Smith. 2019.  Is attention interpretable? In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2931–2951, Florence, Italy. Association for Computational Linguistics.
  * Simonyan et al. (2014) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014.  Deep inside convolutional networks: Visualising image classification models and saliency maps.  In _2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Workshop Track Proceedings_.
  * Sparck Jones (1988) Karen Sparck Jones. 1988.  _A Statistical Interpretation of Term Specificity and Its Application in Retrieval_ , page 132–142. Taylor Graham Publishing, GBR.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_ , AAAI’17, page 4444–4451. AAAI Press.
  * Sundararajan et al. (2017) Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017.  Axiomatic attribution for deep networks.  In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 3319–3328. PMLR.
  * Vig (2019) Jesse Vig. 2019.  Bertviz: A tool for visualizing multihead self-attention in the bert model.  In _ICLR workshop: Debugging machine learning models_.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Yasunaga et al. (2021) Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021.  QA-GNN: Reasoning with language models and knowledge graphs for question answering.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 535–546, Online. Association for Computational Linguistics.

##  Appendix A Questionnaire

The questionnaire includes two parts:

  * •

Whether UKP-SQuARE deepens their understanding of QA topic. Some exemplary
questions are:

    * –

Does UKP-SQuARE help you understand different types of QA systems better (e.g.
extractive QA, abstractive QA)?

    * –

Does the adversarial attack tool in UKP-SQuARE help you understand the
potential vulnerability of QA models better?

    * –

Does the explainability tool in UKP-SQuARE help you understand better how the
model generates answers based on the input?

    * –

Does using UKP-SQuARE in the classroom make the lecture more dynamic and
engaging?

  * •

Whether UKP-SQuARE makes it easier to get hands-on experience. Some exemplary
questions are:

    * –

How long did you spend on the assignment?

    * –

If you don’t use UKP-SQuARE, what will you use to finish your assignment
(which involves comparing different models, and adversarial attacks)?

    * –

Without UKP-SQuARE, how long do you think you need to finish your
assignment(including searching for platforms or building a small service by
yourself)?

    * –

How easy it is to use UKP-SQuARE to do adversarial attacks against models?

    * –

How easy it is to use UKP-SQuARE to explain the model output?

    * –

If you don’t use UKP-SQuARE and you need to perform adversarial attacks on
your model, would you be able to complete the assignment? If so, how much more
difficult would it be?

    * –

If you don’t use UKP-SQuARE and you need to interpret the answers of your
model using saliency maps, would you be able to do it? if so, how much more
difficult would it be?

    * –

Does UKP-SQuARE UI help you compare models easier? (eg: compared to using
Jupyter Notebooks)?


# Remaining Useful Life Estimation Under Uncertainty with Causal GraphNets

Charilaos Mylonas, Eleni Chatzi  Manuscript received Month xx, 2xxx; revised
Month xx, xxxx; accepted Month x, xxxx. This work was supported by ERC
Starting Grant (ERC-2015-StG #679843) on the topic of ”Smart Monitoring,
Inspection and Life-Cycle Assessment of Wind Turbines” at ETH Zürich.
Charilaos Mylonas and Eleni Chatzi are with the Department of Civil and
Environmental Engineering, ETH Zürich, Zurich 8093, Switzerland (e-mail:
<EMAIL_ADDRESS>chatzi@ibk.baug.ethz.ch).

###### Abstract

In this work, a novel approach for the construction and training of time
series models is presented that deals with the problem of learning on large
time series with non-equispaced observations, which at the same time may
possess features of interest that span multiple scales. The proposed method is
appropriate for constructing predictive models for non-stationary stochastic
time series. The efficacy of the method is demonstrated on a simulated
stochastic degradation dataset and on a real-world accelerated life testing
dataset for ball-bearings. The proposed method, which is based on GraphNets,
implicitly learns a model that describes the evolution of the system at the
level of a state-vector rather than of a raw observation. The proposed
approach is compared to a recurrent network with a temporal convolutional
feature extractor head (RNN-tCNN) which forms a known viable alternative for
the problem context considered. Finally, by taking advantage of recent
advances in the computation of reparametrization gradients for learning
probability distributions, a simple yet effective technique for representing
prediction uncertainty as a Gamma distribution over remaining useful life
predictions is employed.

###### Index Terms:

Ball Bearings, Condition Monitoring, Equipment Failure, Forecast Uncertainty,
Graph Neural Network, GraphNet, Implicit Reparametrization Gradients, Long-
Term Recurrent Convolutional Network, Machine Learning Algorithms, Nonuniform
Sampling, Remaining Life Assessment, Time Series Analysis

##  I Introduction

This work has been submitted to the IEEE for possible publication. Copyright
may be transferred without notice, after which this version may no longer be
accessible.

Predictive tasks relying on time series data form a focal area overarching
diverse technological and scientific fields. Settings where observations are
available in non-equispaced and sparse intervals require approximations on the
evolution of the time series. Physics-based models able to simulate the
evolution of a system could offer such predictive capabilities, but are
typically either unavailable, of lower precision, or associated with
prohibitively expensive numerical computations and/or modeling effort. On the
other hand, there exist settings, where readily available measurement data
correlate in a non-trivial manner with quantities of interest. Moreover, when
the evolution of the system at hand is non-deterministic, even if a perfect
knowledge of the instantaneous system state is somehow achieved, a
deterministic estimate of the long-term evolution of the system is not
possible. Therefore, it is of utmost importance to represent the uncertainty
in the predictions involving stochastically evolving systems. This work
focuses on the problem of Remaining Useful Life (RUL) prediction, which
encompases these characteristics.

In many real-world applications, as in the case-study examined herein, a model
of degradation and final failure is not available or not reliable enough111At
this point a clear distinction of the model of degradation of a component and
a model of the time series of the component should be made. We consider
settings where we have neither but have raw measurements of the latter..
Therefore, such a model has to be learned directly from field or experimental
observations. Although the physics of the considered problem are relatively
well understood, the uncertainty in various parameters involved in analyzing
such systems, such as geometric deviations, effect of environmental conditions
on lubricant properties, material and manufacturing imperfections and the
effect of not fully observable loading conditions do not allow for a treatment
of the problem where all physical processes are accounted for. In the same
context, it is expected that features related to the damage of the components
evolve stochastically and the damage state has an indirect effect on the
observed raw time series.

The method proposed herein is inspired by the recent advances in GraphNets
(GNs) and the flexibility these allow for in terms of defining inductive
biases. The GraphNets framework, as introduced in [1], is a generalization on
possible computations on attributed graphs, which covers Graph Neural Network
(GNN) techniques, such as Message-Passing Neural Networks (MPNNs) [2] and Non-
local Neural Networks (NLNNs) [3]. An inductive bias (or learning bias) is any
belief or assumption that, when incorporated in the training procedure, can
facilitate a machine learning algorithm to learn with fewer data or better
generalize in unseen settings. In practice, for the problem of RUL estimation,
due to interruptions in transmission or storage limitations, monitoring time
series contain gaps [4]. The non-regular sampling of the time series data is
routinely treated as a missing data problem; a task most commonly referred to
as time series imputation [5]. This requires to impose an explicit evolution
model that reproduces the raw time series itself in regular intervals, so that
algorithms designed to work with data observed in regular intervals can be
used. This approach biases the subsequent treatment of the data with
predictive algorithms. The present work, in addition to providing a solution
to long time series, yields a radically different approach to the problem of
non-regular observations for building predictive models. Instead of completing
the missing data and subsequently employing a time series technique that
operates on equispaced data, a model that accumulates the information of the
available non-regularly spaced data is learned directly. Instead of an
explicit model that reproduces the time series, the temporal ordering of the
observations is incorporated in the learning algorithm as an inductive bias.

Incorporation of inductive biases is useful in constructing machine learning
models that perform well when trained on relatively small datasets and for
building smaller and more computationally efficient models. Recurrent neural
networks (RNNs) impose a chain-structure of dependence, which constrains RNN
algorithms to sequential computations and typically require $N$ sequential
steps of computation to propagate information from observations that lie $N$
steps before the current time step. When considering very large $N$, this
becomes a significant computational disadvantage both in training and
evaluation of RNNs for long time series. Other recent original approaches to
sequence modeling, such as NeuralODEs [6] and Legendre Memory Units [7] offer
a solution to the issue of non-equispaced data, but do not facilitate the
easier propagation of information from arbitrary past steps since they retain
the chain structure of RNNs. In contrast, the architecture proposed in this
work does not assume a chain graph for processing the past time-steps but a
more general causal graph. Thus, the proposed architecture can learn in a more
parallelized manner with a constant (and adjustable) number of sequential
computational steps, as will be detailed later in the text.

Classical machine learning techniques for general sequence datasets consist of
separate feature extraction & selection and predictive model construction and
selection pipelines. The most widely used feature extraction techniques,
naturally fitting to time series models, are (1) Discrete Fourier Transforms
(DFT), due to the intuitive decomposition of the signal to coefficients (2),
Wavelet transforms, owing to the multi-scale time-frequency characteristics of
some signals, and (3) Dynamic Time-Warping (DTW), when the main source of
variation among signals is due to some temporal distortion (i.e. non-
stationarity), such as different heart-rates in EEG classification [8] or
different rotational speeds in machinery [9]. In several applications of
machine learning for predictive time series models simple moments of the
signals are used, such as kurtosis and standard deviation of time series
segments [10]. In several application fields, special expert-guided feature
extraction techniques have been proposed to facilitate downstream tasks. One
successful representative example of this class of models in time series
analysis are mel-cepstral features [11] in human speech and music processing.
For most other applications, the classical machine learning workflow is
followed, where a large set of features are pre-computed and, in a second
stage, features are selected by inspecting the generalization performance of
the model (for instance with cross-validation). When physical intuition is not
easy to draw from for the problem at hand, features are extracted by
unsupervised learning techniques [12], such as autoencoders, or special
negative-sampling based losses, such as time-contrastive learning [13].
Combinations of unsupervised learning techniques (such as autoencoders and
deep Boltzmann machines) and hand-crafted pre-processing with DCT are also
used [14].

A number of works apply deep learning for the RUL prediction problem from time
series data. In [15] two CNN-based predictors are trained. One classifier
predicts the point in time where the sudden increase in the amplitude of
accelerations occurs which is close to failure and subsequently a second
classifier predicts the time-to-failure after that point. The same approach is
followed in [20], where Random Forests and XGBoost are used as predictive
models. In [16], a recurrent convolutional network is adopted [17] and Monte-
Carlo Dropout [18] is used as a simple and effective way of representing the
uncertainty in the predictions. In [19], instead of recurrent connections, as
applied in [17], attention layers are used to enhance the performance of CNNs.
All aforementioned approaches are not appropriate for arbitrarily spaced data,
as there is no explicit representation of the time between the observations.
In the present work a uniform treatment of the different stages of
degradation, is proposed without attempting to classify different stages of
degradation since they are not clearly defined and this approach could bias
unfavorably the results.

##  II Degradation Time Series Datasets

###  II-A A Simulated Degradation Process Dataset

In order to verify the efficacy of the method for remaining useful life
prediction tasks over long time series, a non-stationary degradation process
was simulated. The underlying process governing the degradation is a non-
stationary Markov process with Gamma distributed increments [21]. The
parameters of the Gamma distributions producing the increments are assumed to
depend on the previous steps, since damage propagation does depend on previous
damage states. In physical terms, this simulates the path dependence of
irreversible processes. The random process presented herein does not have a
direct physical analog and is only designed to demonstrate the properties of
the proposed algorithm. The process generating the latent space as follows

$\displaystyle\delta\eta_{t_{i}}^{(\alpha,\beta)}$ $\displaystyle\sim
Gamma(\alpha(t_{i},c),\beta),\quad\alpha(t,c)=0.02+t^{c}$ $\displaystyle
z_{t_{k}}^{(\alpha,\beta)}$
$\displaystyle=\sum_{i=0}^{t_{k}}\delta\eta_{t_{i}}^{(\alpha,\beta)},\quad
z_{t_{k}}^{(\alpha,\beta)}<z_{f}$ (1)

where $\\{t_{0},t_{1},\cdots t_{N}\\}$ are consecutive, discrete time steps,
$\eta_{t}^{(\alpha,\beta)}$ is a random variable with a non-linear dependence
on time and $c$ a random variable different for each experiment. The
parameters $\alpha>0$ and $\beta>0$ are termed the concentration or shape and
rate parameters of the Gamma distribution. The probability density function of
a Gamma distribution is defined as
$f(x;\alpha,\beta)=\frac{\beta}{\Gamma(\alpha)}x^{\alpha-1}e^{-x\beta}$ where
$\Gamma(\cdot)$ is the Gamma function. Failure occurs when the latent
accumulating damage variable $z_{t_{k}}$ reaches a threshold value, which is
the same for all experiments, denoted as $z_{f}$. It is assumed that the
different experiments have slightly different evolutions for their damage,
which may arise from variations in manufacturing. This is simulated by
sampling $c$ from a Gaussian distribution. The non-linear dependence is
realized through the shape parameter $\alpha(t,c)$ of the Gamma distribution
controlling the size of the increments. It should be noted that the non-linear
dependence on time is used to simulate the non-stationarity of the process,
due to dependence of “$\alpha$” on the accumulated $z_{t_{k}}$. The high-
frequency instantaneous measurement of the signal is denoted as $x_{t_{k}}$.
The observations of the process consist of 1000 samples that contain randomly
placed spikes with amplitude that non-linearly depends on $z_{t_{k}}$, a
process denoted with $G(\cdot)$ for conciseness.

$\begin{aligned} \tilde{z}_{t_{k}}&=z_{t_{k}}+\epsilon\\\
x_{t_{k}}&=G(\tilde{z}_{t_{k}})+\zeta\\\ \end{aligned}\qquad\begin{aligned}
\epsilon&\sim\mathcal{N}(0,{\sigma_{z}}^{2})\\\
\zeta&\sim\mathcal{N}(0,{\sigma_{x}}^{2})\\\ \end{aligned}$

Gaussian noise is added both to the raw signal observation $x_{t_{k}}$ and
directly to the latent variable $z_{t_{k}}$. Noise $\zeta$ is the observation
noise. Noise $\epsilon$, is added to the instantaneous latent damage state
$z_{t_{k}}$ in order to model the fact that $z_{t_{k}}$ may not be accurately
determinable from $x_{t}$ even in the absence of $\zeta$. Each process
underlying the observations of each experiment, evolves in the long-term in a
similar yet sufficiently varied manner as shown in Figure 1. A set of $x_{t}$
signals (raw observations) are shown in Figure 1(b). Although this process
does not correspond directly to some actual physical problem, it is argued
that it possesses all the necessary characteristics of a prototypical RUL
problem and a useful test-case.

Figure 1: (a) Simulated latent variable $z_{t_{k}}$ and (b) raw high-frequency
time series realizations $x_{t_{k}}$.

###  II-B An experimental dataset on accelerated fatigue of ball bearings

The dataset in [22] consists of run-to-failure experiments of a total of 17
bearings, loaded in 3 different conditions. Only 2-axis acceleration
measurements are used in the present work. Temperature measurements are also
available. Importantly, no artificial damage is introduced to the components
for accelerating failure, thus rendering the accelerated testing scenario a
better representation of real-world settings, where the failure mode is not
known a-priori. The experimental conditions are summarized in Table II.

In order to test generalization on un-seen experiments a test-set containing
whole experiments is used. A different train/test split is adopted from [22],
which is detailed in Table II.

TABLE I: Available experiments and loading conditions.

Conditions $i$ |  $\dot{\phi}_{i}$ [rpm] |  $F_{i}$ [kN] | Number of experiments
---|---|---|---
A | 1800 | 4.0 | 7
B | 1650 | 4.2 | 7
C | 1500 | 5.0 | 3

TABLE II: FEMTO bearings dataset, training/test split

Set | Experiment | Conditions | Failure time [s] | Num. Obs.
---|---|---|---|---
Training | 1_2 | A | 8700 | 871
| 1_3 | A | 23740 | 2375
| 1_4 | A | 14270 | 1428
| 1_5 | A | 24620 | 2463
| 2_1 | B | 9100 | 911
| 2_5 | B | 23100 | 2311
| 2_6 | B | 7000 | 701
| 3_3 | C | 4330 | 434
Testing | 1_1 | A | 28072 | 2803
| 1_6 | A | 24470 | 2448
| 1_7 | A | 22580 | 2259
| 2_2 | B | 7960 | 797
| 2_3 | B | 19540 | 1955
| 2_4 | B | 7500 | 751
| 2_7 | B | 2290 | 230
| 3_1 | C | 5140 | 515
| 3_2 | C | 16360 | 1637

Fatigue damage on roller bearings, manifests as frictional wear of the
bearings and/or the surrounding ring. Empirically, higher lateral loads
$F_{i}$ and rotational speeds $\dot{\phi}_{i}$ are associated with faster wear
for bearings. The $\ddot{u}_{x}$ and $\ddot{u}_{y}$ accelerometer data are
available in 0.1 second segments, sampled at 25.56kHz (2556 samples per
segment). Temporal convolution networks are used, to automatically detect and
use features that potentially are useful to tracking degradation in these
experiments.

##  III Model Architectures

###  III-A GraphNets for arbitrary inductive biases

GraphNets (GNs) are a class of machine learning algorithms operating with
(typically pre-defined) attributed graph data, which generalize several graph
neural network architectures. An attributed graph, in essence, is a set of
nodes (vertices) $V:\\{\mathbf{v}_{1},\cdots\mathbf{v}_{k}\\}$ and edges
$E:\\{(\mathbf{e}_{1},r_{1},s_{1})\cdots(\mathbf{e}_{k},r_{k},s_{k})\\}$ where
$\mathbf{e}_{k}\in\mathbb{R}^{N^{e}}$ and
$\mathbf{v}_{i}\in\mathbb{R}^{N^{v}}$. Each edge is a triplet
$(\mathbf{e}_{j},r_{j},s_{j})$ (or equivallently
$(\mathbf{e},\mathbf{v}_{r_{j}},\mathbf{v}_{s_{j}})$) and it contains a
reference to a receiver node $\mathbf{v}_{r_{j}}$, to a sender node
$\mathbf{v}_{s_{j}}$ as well as a (vector) attribute $\mathbf{e}_{j}$. Self-
edges, i.e. when $r_{i}:=s_{i}$ are allowed. In [1] a more general class of
GraphNets is presented where global variables which affect all nodes and edges
are allowed. A GN with no global variables consists of a node-function
$\phi^{v}$, an edge function $\phi^{e}$, and an edge aggregation function
$\rho^{e\rightarrow v}$. The function $\rho^{e\rightarrow v}$ should be (1)
invariant to the permutation of its inputs and (2) able to accept a variable
number of inputs. In the following this will be referred to as the edge
aggregation function. Simple valid aggregation functions are $Min(\cdot)$,
$Max(\cdot)$, $Sum(\cdot)$ and $Mean(\cdot)$. Inventing more general
aggregation functions (for instance by combining them) and investigating how
they affect the approximation properties of GNs is an active current research
subject [23].

Ignoring global graph attributes, the GraphNet computation procedure is as
detailed in algorithm 1. First, the new edge states are evaluated using the
sender and receiver vertex attributes ($\mathbf{v}_{s_{i}}$ and
$\mathbf{v}_{r_{i}}$ correspondingly) and the previous edge state
$\mathbf{e_{i}}$ as arguments to the edge function $\phi^{e}$. The arguments
of the edge function may contain any combination of the source and target node
attributes and the edge attribute. Afterwards, the nodes of the graph are
iterated and the incoming edges for each node are used to compute an
aggregated incoming edge message $\mathbf{\bar{e}}^{\prime}_{i}$ The
aggregated edge message together with the node attributes are used to compute
an updated node state. Typically, small Multi-Layer Perceptrons (MLPs) are
used for the edge and node GraphNet functions $\phi^{e}$ and $\phi^{v}$. It is
possible to compose GN blocks by using the output of a GN as the input to
another GN block. Since a single GN block allows only first order neighbors to
exchange messages, GN blocks are composed as

$\displaystyle GN_{K}(GN_{K-1}(\cdots(GN_{0}(G)\cdots)))=$ $\displaystyle
GN_{K}\circ GN_{K-1}\circ\cdots\circ GN_{0}(G)$

where “$\circ$” denotes composition. The first GN block may cast the input
graph data to a lower dimension so as to allow for more efficient computation.
The first GN block may have edge functions that depend only on edge states
$\phi^{e_{0}}(\mathbf{e})$ and correspondingly node functions that depend only
on node states $\phi^{u_{0}}(\mathbf{v})$. This is refered to a Graph
Independent GN block and it is used as the type of layer for the first and the
last GN block. The inner GN steps (i.e. $GN_{1}$ to $GN_{K-1}$) are full GN
blocks, where message passing takes place. This general computational pattern
is refered to as encode-process-decode [1]. The inner GN blocks may have
shared weights, yielding smaller memory footprint for the whole model or
different weights, ammounting to different GN functions that need to be
trained for each level. Sharing weights and repeatedly applying the same GN
block helps propagate and combine information from more connected nodes in the
graph.

function GraphNetwork($E$, $V$)

for $k\in\\{1\ldots{}N^{e}\\}$ do

$\mathbf{e}_{k}^{\prime}\leftarrow\phi^{e}\left(\mathbf{e}_{k},\mathbf{v}_{r_{k}},\mathbf{v}_{s_{k}}\right)$
$\triangleright$ 1\. Compute updated edges

end for

for $i\in\\{1\ldots{}N^{n}\\}$ do

let
$E^{\prime}_{i}=\left\\{\left(\mathbf{e}^{\prime}_{k},r_{k},s_{k}\right)\right\\}_{r_{k}=i,\;k=1:N^{e}}$

$\mathbf{\bar{e}}^{\prime}_{i}\leftarrow\rho^{e\rightarrow
v}\left(E^{\prime}_{i}\right)$ $\triangleright$ 2\. Aggregate edges per node

$\mathbf{v}^{\prime}_{i}\leftarrow\phi^{v}\left(\mathbf{\bar{e}}^{\prime}_{i},\mathbf{v}_{i},\right)$
$\triangleright$ 3\. Compute updated nodes

end for

let $V^{\prime}=\left\\{\mathbf{v}^{\prime}\right\\}_{i=1:N^{v}}$

let
$E^{\prime}=\left\\{\left(\mathbf{e}^{\prime}_{k},r_{k},s_{k}\right)\right\\}_{k=1:N^{e}}$

return $(E^{\prime},V^{\prime})$

end function

Algorithm 1 GN block without global variables [1].

In the present work, as is the case with RNNs [24] and causal CNNs [25], the
causal structure of time series is also exploited, which is a good inductive
bias for the problem at hand, although without requiring that the data is
processed as a chain-graph or that the data are regularly sampled. Instead, an
arbitrary causal graph for the underlying state is built, together with
functions to infer the quantity of interest which is the remaining useful life
of a component given a set of non-consecutive short-term observations.

###  III-B Incorporation of Causal Inductive Biases using GraphNets

The variable dependencies of the proposed model are schematically depicted in
Figure 2 for 3 observations. The computational architecture is depicted in
more detail in Figure 3. The variable $Z_{K}$ represents the current estimate
for the latent state of the system222It can be considered that the variable
contains values that represent, for instance, sufficient statistics or a re-
parametrization of a probability distribution, allowing its interpretation as
a representation of a probability distribution.. The variable $T_{K\rightarrow
L}$, which represents the propagated latent state from past observations,
depends on the latent state $Z_{K}$, an exogenous input $F_{K\rightarrow L}$
that controls the propagation of state $Z_{K}$ to $Z_{L}$ and potentially
other propagated latent state estimates from instants before $t_{L}$. The
exogenous input $F_{K\rightarrow L}$ to the state propagation function can be
as simple as the elapsed time between two time instants $t_{K\rightarrow
L}=t_{L}-t_{K}$ or incorporate more prior inductive biases, such as the values
representing different operating conditions during the interval between
observations. An arbitrary number of past states can be propagated from past
observations and aggregated in order to yield better estimates for a latent
state $Z_{L}$. In adition to propagated latent states, instantaneous
observations of raw data $X_{K}$ inform the latent state $Z_{K}$. For
instance, in Figure 2 $Z_{C}$ depends on $T_{B\rightarrow C}$ but at the same
time on $T_{A\rightarrow C}$ and potentially more propagated states from past
observations (other yellow nodes in the graph) and at the same time to an
instantaneous observation $X_{C}$. Each inferred latent state $Z_{i}$ can be
transformed to a distribution for the quantity of interest $Y_{i}$. The value
of the propagated state variable from state $s$ to state $d$, $T_{s\rightarrow
t}$, depends jointly on the edge attributes and on the latent state of the
source node. In a conventional RNN model, $F_{K\rightarrow L}$ corresponds to
an exogenous input for the RNN cell. In contrast to an RNN model, in this work
the dependence of the estimate of each state depends on multiple states by
introducing a propagated state that is modulated by the exogenous input. In
that manner an arbitrary and variable number of past states can be used
directly for refining the estimate of the current latent state, instead of the
estimate summarized in the latent cell state of the last RNN cell state. In
the proposed model, the parameters of the functions relating the variables of
the model are learned directly from data while only defining the inductive
biases following naturally from the temporal ordering of the observations.
This approach allows for uniform treatment of all observations from the past
and allows for the consideration of an arbitrary number of such observations
to yield an estimate of current latent state.

The connections from all observable past states and the ultimate one, where
prediction (read-out) is performed, are implemented as a node-to-edge
transformation and subsequent aggregations. Aggregation corresponds to the
edge-aggregation function $\rho^{e\rightarrow u}(\cdot)$ of the GraphNet. In
this manner, it is possible to propagate information from all distant past
states on a single computation step. The computation of all available past
states would be innefficient. To remedy that, it is possible to randomly
sample the past states used in order to perform inference for the current
step. Similarly, during training it is possible to yield unbiased estimates of
gradients for the propagation and feature extraction model by randomly
sampling the past states. It was found that for the presented use-cases this
was an effective strategy for training. In GN terms, the “encode” GraphNet
block ($GN_{enc}:\\{\phi^{u_{0}},\phi^{e_{0}}\\}$) is a graph-independent
block consisting of the node function $\phi^{u_{0}}$ and edge function
$\phi^{e_{0}}$. The node function is a temporal convolutional neural network
(temporal CNNs), with architecture detailed in Table III.

TABLE III: Details on temporal CNN which acts as the node-function $\phi^{u_{0}}$ of the graph independent $GN_{enc}$ GraphNet. $n^{k},n^{s},n^{f}$ corresponds to kernel size, stride and number of filters. For dense layers $n^{f}$ corresponds to the layer width. Layer type | $(n^{k},n^{s},n^{f})$ | Activation
---|---|---
Conv1D | $(1\times 1,1,50)$ | -
Conv1D | $(1\times 3,2,18)$ | -
Conv1D | $(1\times 3,2,18)$ | Dropout $20\%$ $ReLU$
Conv1D | $(1\times 3,2,50)$ |
Average Pool | $(1\times 2,2,1)$ | -
Conv1D | $(1\times 1,1,50)$ | -
Conv1D | $(1\times 3,2,18)$ | -
Conv1D | $(1\times 3,2,18)$ | Dropout $20\%$ $ReLU$
Conv1D | $(1\times 3,2,50)$ | -
Avg. Pool | $(1\times 2,2,1)$ | -
Conv1D | $(1\times 1,1,50)$ | -
Conv1D | $(1\times 3,2,18)$ | -
Conv1D | $(1\times 3,2,18)$ | Dropout $20\%$ $ReLU$
Conv1D | $(1\times 3,2,50)$ | -
Global Avg. Pool | $(1\times 2,2,1)$ | -
Feed-forward | $(-,-,15)$ | $Leaky\,ReLU$

The edge update function is a feed-forward neural network. The input of the
edge function is the temporal difference between observations. Both networks
cast their inputs to vectors of the same size. The
$GN_{core}:\\{\phi^{u_{c}},\phi^{e_{c}},\rho^{e\rightarrow u}\\}$ network,
consists of small feed-forward neural networks for the node MLP $\phi^{u_{c}}$
and the edge MLP $\phi^{e_{c}}$. The input of the edge MLP is the sender and
receiver state and the previous edge state. The MLP is implemented with a
residual connection to allow for better propagation of gradients through
multiple steps [26].

$\mathbf{e}^{\prime}_{i}\leftarrow\phi^{e_{c}}(\mathbf{e}_{i},\mathbf{u}_{s_{i}},\mathbf{u}_{r_{i}})=\bar{\phi}^{e_{c}}(\mathbf{e}_{i},\mathbf{u}_{s_{i}},\mathbf{u}_{r_{i}})+\mathbf{e}_{i}$

In this work, the $Mean(\cdot)$ aggregation function was chosen, which does
not depend strongly on the in-degree of the state nodes $Z_{i}$ (i.e. number
of incoming messages) which corresponds to step $2$ in algorithm 1. The node
MLP of the core network is also implemented as a residual MLP.

$\mathbf{u^{\prime}}_{i}\leftarrow\phi^{u_{c}}(\mathbf{u}_{i},\mathbf{\bar{e}}_{i})=\bar{\phi}^{u_{c}}(\mathbf{u}_{i},\mathbf{\bar{e}}_{i})+\mathbf{u}_{i}$

The $GN_{core}$ network is applied multiple times to the output of $GN_{enc}$.
This ammounts to the shared weights variant of GNs which allow for propagation
of information from multiple steps while costing a small memory footprint.
After the last $GN_{core}$ step is applied, a final graph-independent layer is
applied. At this point, for further computation only the final state of the
last node is needed, which is the one corresponding to the last observation.
The state of the last node is passed through two MLPs that terminate with
$softplus$ activation functions

$Softplus(x)=log(exp(x)+1).$ (2)

The $Softplus$ activation is needed for forcing the outputs to be in
$(0,+\inf)$, since they are used as parameters for a $Gamma$ distribution
which in turn is used to represent the RUL estimates. The GraphNet computation
procedure detailed above is denoted as

$g_{out}=GN_{tot}(g)=GN_{dec}\circ GN_{core}^{(N_{c})}\circ GN_{enc}(g_{in})$
(3)

where $GN_{core}^{(N_{c})}$ denotes $N_{c}$ compositions of the $GN_{core}$
GraphNet and “$g_{in},\,g_{out}$” are the input and output graphs. The vertex
attribute of the final node is in turn used as rate
($\alpha(GN_{tot}(g_{in}))$) and concentration ($\beta(GN_{tot}(g_{in}))$)
parameters of a $Gamma(\alpha,\beta)$ distribution. For ease of notation, the
parameters (weights) of all the functions involved are denoted by
“$\bm{\theta}$” and the functions that return the rate and concentration are
denoted as $f_{\alpha;{\mathbf{\theta}}}$ and $f_{\beta;{\mathbf{\theta}}}$
correspondingly to denote explicitly their dependence on “$\bm{\theta}$”. The
$Gamma$ distribution was chosen for the distribution of the output values
since they correspond to remaining time and they are necessarily positive. The
GN described above is trained so as to maximize directly the expected
likelihood of the remaining useful life estimates. For numerical reasons,
equivalently, the negative log-likelihood (nll) is maximized. The optimization
problem reads,

$\displaystyle\operatorname*{arg\,max}_{\bm{\theta}}\mathbb{E}_{(\mathcal{P},\mathcal{S})}[p(\mathbf{y}|\mathbf{g})]\propto\operatorname*{arg\,max}_{\bm{\theta}}\prod_{i=1}^{N^{s,p}}p(y_{i}|g_{i})\equiv$
$\displaystyle=\operatorname*{arg\,min}_{\bm{\theta}}\sum_{i=1}^{N^{s,p}}\Big{(}-\log
p(y_{i}|f_{\alpha;\bm{\theta}}(g_{i}),f_{\beta;{\bm{\theta}}}(g_{i}))\Big{)}$
(4)

where $\mathbf{g}$ corresponds to the sets of input graphs, and $\mathbf{y}$
corresponds to the estimate of RUL for the last observation of each graph. The
input graphs in our case consist of nodes, which correspond to observations
and edges with time-difference as their features. Correspondingly $g_{i}$ and
$y_{i}$ are single samples from the aformentioned set of causal graphs and
remaining useful life estimates and $N^{s,p}$ denotes the number of sampled
causal graphs from experiment $p$ used for computing the loss (i.e. batch
size). The expectation symbol is approximated by an expectation over the set
of available training experiments denoted as $\mathcal{P}$ and the random
causal graphs created for training $\mathcal{S}$. The gradients of Equation 4
are computable through implicit re-parametrization gradients [27]. This
technique allows for low-variance estimates for the gradient of the nll loss
with respect to the parameters of the distribution, which in turn allows for a
complete end-to-end differentiable training procedure for the proposed
architecture.

As in recurrent neural network models [29, 24], and [25], a gated-tanh
activation function was used for the edge update and node update core
networks.

$h(y)=sigmoid(W_{g}y)\odot tanh(W_{a}y)$

GraphNets using this activation strongly outperformed the ones using tanh but
showed similar performance to the ones using relu activation. Networks for the
edge and node MLPs were tested with widths $30,\,50,$ and $100$. The smaller
networks tested (size $30$) consistently outperformed networks with size $50$
and for the most part had similar performance with networks with size $100$
for some cases. The $30-$unit networks were selected for the presented
results.

Figure 2: Dependency graph for the variables associated with the proposed
model. $X_{A}$ represents the raw observed variable at time $t_{A}$. Variable
$Z_{A}$ represents the (unobserved) state that can be translated to the
quantity of interest $Y_{C}$ or a probabilistic estimate. Figure 3: Detail of
the GNN-tCNN. States are propagated with $\phi^{e}$ and accumulated with
$\rho^{e\rightarrow u}$.

###  III-C Recurrent Neural Network with Temporal CNN Feature Extractors
(LSTM-tCNN)

The Causal GNN component of the architecture detailed in subsection III-B is
used to satisfy the following desiderata: (1) to allow for computationally
efficient and parallelized propagation of information from time-instants in
the distant past with respect to the current timestep and (2) to allow for
learning a state-propagation function and hence dealing with arbitrarily
spaced points in a consistent manner. Although gated RNNs, such as GRUs and
LSTMs, rely on sequential computation between timesteps, and therefore less
parallelizeable, they are known to be relatively effective in dealing with
long dependencies. Moreover, by appending the time difference between
observations in the input gate of the RNN the RNN can learn to condition the
predictions for the propagated state not only on the previous state and the
CNN feature extractor input but also to the time-difference between different
RNN steps [28]. One such model, using an LSTM cell, is depicted in Figure 4.

Figure 4: Model architecture of LSTM-tCNN.

##  IV Results

###  IV-A Simulated Dataset

Although it is easy to create a large number of training and test set
experiments from the simulated dataset, in order to keep the simulated use-
case realistic, only 12 experiments were used for training and a set of 3
experiments were used as a test set. Representative prediction results for the
test-set experiments are shown in 5a. The accuracy of the model is inspected
in terms of the expected negative log likelihood (smaller is better).When more
observations are used, the estimates for the RUL of the fictitious processes
are more accurate for a larger portion of the entirety of the observations.
When a single observation is used, which completely neglects the long-term
evolution of damage but uses short-term features extracted by the learned
graph-independent $\phi^{u_{0}}$ which is a temporal CNN, the RUL estimates
are inaccurate and fluctuate at the beginning of the experiment (top-right
side of the first column of the plots). It is also observed that when using
more observations, although the uncertainty bounds are not becoming smaller at
the beginning of the fictitious experiment, the estimates of the trend of
degradation, and consequently the remaining useful life of the component are
accurate and smooth early on in the course of the experiment.

This observation provides evidence that the proposed architecture accurately
captures both features of the high-frequency time series through the CNNs of
the first graph-independent processing step, and the long-term evolution of
the time series through the GraphNet processing steps. Moreover, the estimated
probability distributions of RUL become more concentrated closer to failure,
while they are wider at the beginning of the experiments. This observation
aligns with the intuition that it is not possible to have sharp estimates in
the beginning of the experiments.

###  IV-B Bearings Dataset

Results for representative experiments from the test-set of the FEMTO-bearing
dataset are shown in 5b. Similarly to the simulated experiments, predictions
are characterized by smaller uncertainty closer to failure and the degradation
trends are captured effectively. Predictions employing up to 30 arbitrarily
spaced observations from the past 2000 seconds are shown. Using more
observations (more than 30) did not signifficantly improve the accuracy or the
uncertainty bounds of the predictions. This may be due to the fact that the
damage phenomenon is slowly evolving, thus using a larger number of points
does not offer more information on the evolution of the phenomenon.

(a) GNN-tCNN, Simulated

(b) GNN-tCNN, Bearings

(c) LSTM-tCNN, Bearings

Figure 5: (a) Results on the simulated dataset using GNN-tCNN. (b), (c)
Results on the bearing dataset using GNN-tCNN and LSTM-tCNN.

##  V Conclusion

Two neural network architectures (GNN-tCNN and RNN-tCNN) were applied to the
problem of remaining life assessment on real-world experimental data and on
artificial data from a stochastic degradation process. Both architectures seem
to capture the long-term dependences on features of the considered time
series. The GraphNet architecture proposed features a causal connectivity
structure that can capture with less sequential computation long-term
dependencies in the time series. Finally, an effective gradient-based
technique, which employs low-variance reparametrization-based gradient
estimators for fitting distributions with positive quantities of interest,
such as RUL estimates, was employed. The proposed architectures are intuitive
and easy to implement.333Code to reproduce the experiments in the paper will
be made available upon publication

Although only RUL estimation problems were considered in this work, the non-
sequential causal approach to dealing with long-term dependencies may be
applicable to further applications where non-regularly sampled time series
arise (e.g. electronic health records). In future works, other time-series
tasks such as time-series generation or unsupervised/self-supervised learning
[13] are to be attempted, employing the GNN-tCNN architecture.

## References

  * [1] P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner _et al._ , “Relational inductive biases, deep learning, and graph networks,” _arXiv preprint arXiv:1806.01261_ , 2018.
  * [2] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” _arXiv preprint arXiv:1704.01212_ , 2017.
  * [3] X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 7794–7803, 2018.
  * [4] J. Sikorska, M. Hodkiewicz, and L. Ma, “Prognostic modelling options for remaining useful life estimation by industry,” _Mechanical systems and signal processing_ , vol. 25, no. 5, pp. 1803–1836, 2011.
  * [5] R. Razavi-Far, S. Chakrabarti, M. Saif, and E. Zio, “An integrated imputation-prediction scheme for prognostics of battery data with missing observations,” _Expert Systems with Applications_ , vol. 115, pp. 709–723, 2019.
  * [6] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” in _Advances in neural information processing systems_ , pp. 6571–6583, 2018.
  * [7] A. Voelker, I. Kajić, and C. Eliasmith, “Legendre memory units: Continuous-time representation in recurrent neural networks,” in _Advances in Neural Information Processing Systems_ , pp. 15 570–15 579, 2019.
  * [8] B. Raghavendra, D. Bera, A. S. Bopardikar, and R. Narayanan, “Cardiac arrhythmia detection using dynamic time warping of ecg beats in e-healthcare systems,” in _2011 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks_ , pp. 1–6. IEEE, 2011.
  * [9] D. Zhen, T. Wang, F. Gu, and A. Ball, “Fault diagnosis of motor drives using stator current signal analysis based on dynamic time warping,” _Mechanical Systems and Signal Processing_ , vol. 34, no. 1-2, pp. 191–202, 2013.
  * [10] B. Rouet-Leduc, C. Hulbert, N. Lubbers, K. Barros, C. J. Humphreys, and P. A. Johnson, “Machine learning predicts laboratory earthquakes,” _Geophysical Research Letters_ , vol. 44, no. 18, pp. 9276–9282, 2017.
  * [11] T. Kitamura, E. Hayahara, and Y. Simazciki, “Speaker-independent word recogniton in noisy environments using dynamic and averaged spectral features based on a two-dimensional mel-cepstrum,” in _First International Conference on Spoken Language Processing_ , 1990.
  * [12] Z. Yang, P. Baraldi, and E. Zio, “Automatic extraction of a health indicator from vibrational data by sparse autoencoders,” in _2018 3rd International Conference on System Reliability and Safety (ICSRS)_ , pp. 328–332. IEEE, 2018.
  * [13] A. Hyvarinen and H. Morioka, “Unsupervised feature extraction by time-contrastive learning and nonlinear ica,” in _Advances in Neural Information Processing Systems_ , pp. 3765–3773, 2016.
  * [14] L. Deng, M. L. Seltzer, D. Yu, A. Acero, A.-r. Mohamed, and G. Hinton, “Binary coding of speech spectrograms using a deep auto-encoder,” in _Eleventh Annual Conference of the International Speech Communication Association_ , 2010\.
  * [15] B. Yang, R. Liu, and E. Zio, “Remaining useful life prediction based on a double-convolutional neural network architecture,” _IEEE Transactions on Industrial Electronics_ , vol. 66, no. 12, pp. 9521–9530, 2019.
  * [16] B. Wang, Y. Lei, T. Yan, N. Li, and L. Guo, “Recurrent convolutional neural network: A new framework for remaining useful life prediction of machinery,” _Neurocomputing_ , vol. 379, pp. 117–129, 2020.
  * [17] M. Liang and X. Hu, “Recurrent convolutional neural network for object recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 3367–3375, 2015.
  * [18] Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in _international conference on machine learning_ , pp. 1050–1059, 2016.
  * [19] B. Wang, Y. Lei, N. Li, and W. Wang, “Multi-scale convolutional attention network for predicting remaining useful life of machinery,” _IEEE Transactions on Industrial Electronics_ , 2020.
  * [20] J. Shi, T. Yu, K. Goebel, and D. Wu, “Remaining useful life prediction of bearings using ensemble learning: The impact of diversity in base learners and features,” _Journal of Computing and Information Science in Engineering_ , vol. 21, no. 2, 2020.
  * [21] J. L. Bogdanoff and F. Kozin, “A New Cumulative Damage Model - Part 4,” _Journal of Applied Mechanics_ , vol. 47, DOI 10.1115/1.3153635, no. 1, pp. 40–44, 03 1980. [Online]. Available: https://doi.org/10.1115/1.3153635
  * [22] P. Nectoux, R. Gouriveau, K. Medjaher, E. Ramasso, B. Chebel-Morello, N. Zerhouni, and C. Varnier, “Pronostia: An experimental platform for bearings accelerated degradation tests.” 2012.
  * [23] G. Corso, L. Cavalleri, D. Beaini, P. Liò, and P. Veličković, “Principal neighbourhood aggregation for graph nets,” _arXiv preprint arXiv:2004.05718_ , 2020.
  * [24] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [25] A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” _arXiv preprint arXiv:1609.03499_ , 2016.
  * [26] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * [27] M. Figurnov, S. Mohamed, and A. Mnih, “Implicit reparameterization gradients,” in _Advances in Neural Information Processing Systems_ , pp. 441–452, 2018.
  * [28] Y. Zhu, H. Li, Y. Liao, B. Wang, Z. Guan, H. Liu, and D. Cai, “What to do next: Modeling user behaviors by time-lstm.” in _IJCAI_ , vol. 17, pp. 3602–3608, 2017.
  * [29] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using rnn encoder-decoder for statistical machine translation,” _arXiv preprint arXiv:1406.1078_ , 2014.

|  Charilaos Mylonas was born in Thessaloniki, Greece. He holds a Dipl. Ing.
Structural Engineering degree from Aristotle University of Thessaloniki and a
MSc on Computational Science and Engineering from ETH Zürich. He previously
worked as a scientific software developer at ETH Zürich and as a full-stack
software developer in the banking sector. He is currently working towards his
Ph.D. on the topic of remaining life assessment and uncerainty quantification
for wind turbines.
---|---
|  Eleni Chatzi was born in Athens, Greece. She received her PhD (2010) from
the Department of Civil Engineering and Engineering Mechanics at Columbia
University. She is Chair of Structural Mechanics and Monitoring at the
Department of Civil, Environmental and Geomatic Engineering of ETH Zürich. Her
research interests include the fields of Structural Health Monitoring (SHM)
and structural dynamics, nonlinear system identification, and intelligent
assessment for engineered systems. She is leading the ERC Starting Grant
WINDMIL on smart monitoring of Wind Turbines. Her work in the domain of self-
aware infrastructure was recognized with the 2020 Walter L. Huber Research
prize, awarded by the American Society of Civil Engineers (ASCE).
---|---


# Searching for modifications to the exponential radioactive decay law with
the Cassini spacecraft

Peter. S. Cooper
Fermi National Accelerator Laboratory, Batavia, IL 60510, U.S.A

###### Abstract

Data from the power output of the radioisotope thermoelectric generators
aboard the Cassini spacecraft are used to test the conjecture that small
deviations observed in terrestrial measurements of the exponential radioactive
decay law are correlated with the Earth-Sun distance. No significant
deviations from exponential decay are observed over a range of $0.7-1.6A.U.$ A
$90\%$ Cl upper limit of $0.84\times 10^{-4}$ is set on a term in the decay
rate of ${}^{238}Pu$ proportional to $1/R^{2}$ and $0.99\times 10^{-4}$ for a
term proportional to $1/R$.

PACS numbers: 23.60.+e, 23.40.-s, 95.55.Pe, 96.60.Vg, 0620.Jr.

A recent archive preprint reports evidence for a correlation between nuclear
decay rates and the Earth-Sun distanceFischbach . This correlation is
extracted from an annual modulation in the observed decay rates of
${}^{32}Si/^{36}Cl$, both $\beta$ emitters, and ${}^{226}Ra$, an $\alpha$
emitter. Reference 1 analyzes this as a correlation with $1/R^{2}(t)$, the
variation in the Earth-Sun distance due to the eccentricity of the Earth’s
orbit. To set a scale for this correlation the amplitude of the decay rate
variation is $\sim 0.1\%$ roughly in phase with the $3\%$ annual modulation in
$1/R^{2}(t)$ suggesting a $(3\times 10^{-2})/R^{2}$ term in the decay rate.

In the conclusion of this paper the authors observe that: _These conclusions
can be tested…_ [by] _measurements on radioactive samples carried aboard
spacecraft to other planets_ [which] _would be very useful since the sample-
Sun distance would vary over a much wider range_. I report here the results of
exactly such a measurement based on the power output of the Radioisotope
Thermoelectric Generators (RTG) aboard the Cassini spacecraft which launched
in 1997 and reached Saturn in 2004.

Cassini is powered by three RTGs each of which is a very large ($7.7Kg$,
$130KCu$) ${}^{238}Pu$ radioactive source, an $\alpha$ emitter with an $87.7y$
half lifeNNDC . The heat from these sources are converted to electric power
with thermoelectric piles. Together these sources produced $878w$ of
electrical power from $\sim 13Kw$ of radioactive decay heat at launch. The
power output of these RTGs were literally the lifeblood of the Cassini
mission. Their power output was monitored carefully and often.

Figure 1: Left, suppressed zero, scale: solid red curve, Heliocentric distance
[$R(t)(A.U.)$], Black points are the 5 times when R(t)=1 (green line), Right,
suppressed zero, scale: blue diamonds, RTG electrical power.

The trajectory of the Cassini spacecraft is available on the webTRAJ . I’ve
used these data, converted to astronomical units ($A.U.$), to compute
$1/R(t)^{2}=R_{e}^{2}/[x(t)^{2}+y(t)^{2}+z(t)^{2}]$. Over the first 2 years
Cassini went from R(0)=1 at launch, made 2 visits to Venus at R=0.7 and
crossed the orbit of the earth a total of 4 times before finally gaining
enough speed to reach Saturn at R=9.

JPL kindly providedJPL  P(t), the total electrical power from the three RTGs
aboard, measured daily since launch and the expected power output from their
RTG modeling. The distance of Cassini from the Sun and the electrical power
output are plotted in Figure 1 for the first $2$ years of the mission. R
ranged from $0.7-1.6A.U.$ $(0.35<1/R^{2}<2.2)$. The power dropped from $878w$
at launch to $815w$ over this period.

The thermal power output of an RTG is directly proportional to the decay rate
of the radioisotope generating the heat: $P_{th}(t)=N_{0}\lambda exp(-\lambda
t)E_{d}$, where $E_{d}$ is the energy released per decay. The electrical power
output, $P(t)=P_{th}(t)\epsilon_{0}\epsilon(t)$ is modified by an initial
efficiency, $\epsilon_{0}$, and a time (or power) dependent thermoelectric
conversion efficiency; $\epsilon(t),\epsilon(0)=1$.

I cannot safely use the RTG efficiency modelDEGRA  here since it assumed only
exponential behavior for radioactive decay. Any new physics effect might be
inadvertently subsumed into that model. The Cassini trajectory provides a
natural calibration for $\epsilon(t)$ using the measured power at the 5 points
(shown in Figure 1) where the spacecraft was $1A.U.$ from the sun. Fitting
these measurements to $P_{R=1}(t)=P(0)2^{(-t/87.7y)}2^{(-t/T_{eff})}$ yields
$T_{eff}=21.2\pm 1.9y$. This simple model agrees in shape with Reference 4 and
obeys the requirements of Carnot thermal efficiency: as the power decreases,
and the temperature difference across the thermoelectric piles decrease, the
efficiency can only decrease. Something as complicated as a space-born
thermoelectric pile requires more than one parameter to accurately describe
its behavior. The single exponential reduces a $5\%$ power drop in the first 2
years to a $\pm 1\%$ variation from unity.

Following Reference 1 I’ve plotted $\epsilon(t)U(t)=P(t)/P(0)2^{t/87.7y}$, the
normalized electrical power corrected for ${}^{238}Pu$ decay, and U(t), the
normalized thermal power, as a functions of time since launch in Figure 2 for
the first 2 years of Cassini’s voyage to Saturn.

Figure 2:  RTG power; green open circles, decay corrected [$\epsilon(t)U(t)$],
black solid circles, efficiency and decay corrected [$U(t)$], cyan curve, 3rd
order polynomial correction function described the in text. red solid line,
expected effect extrapolated from Reference 1: $1-(1/R^{2}(t)-1)/30$.

To compare to the rough magnitude of the effect reported by reference 1 I’ve
plotted $U_{ref1}(t)=1+(0.1\%/3\%)(1/R(t)^{2}-1)$ on Figure 2 to extrapolate
the small R variation of reference 1 for comparison with the larger R range
available in this study. Extrapolating a $0.1\%$ decay rate change for a $3\%$
change in $1/R^{2}$ to a $50\%$ change in $1/R^{2}$ ($R=0.7A.U.$, $t=0.43y$)
should cause a $+4\%$ change in the power output of the RTG 5 months after
launch. In fact $U(t)$ decreases by $\sim 1\%$. Changes this large are
excluded by the Cassini efficiency corrected data both in magnitude and by the
absence of any reflection of the shape of $1/R^{2}(t)$ in either normalized
power curve.

In order to set quantitative limits I’ve fit the efficiency and half-life
corrected $U(t)$ normalized power data (suppressing the two obvious outlying
points in Figure 1) to $P_{3}(t)$, a 3th order polynomial in time, to
phenomenologically describe the last $1\%$ variation in the $U(t)$. As shown
in Figure 2, this polynomial smoothly interpolates the $U(t)$ measurements. A
3th order polynomial make a very poor fit to $1/R^{2}(t)$; these two shapes
are approximately orthogonal. The error assigned to each U(t) measurement by
requiring this fit to have $\chi^{2}/\nu=1$ is $0.0015$. The individual
relative power measurements have a resolution of $0.15\%$ and the polynomial
is a $\sim 6\sigma$ systematic correction beyond the simple exponential
efficiency model.

Figure 3: Black points; $\Delta(t)$, RTG normalized thermal power less a 3rd
order polynomial correction, Red curve; $10$ times the $90\%$ CL limit,
$10\alpha/R^{2}$, Blue curve; $10$ times the $90\%$ CL limit, $10\beta/R$.

The difference of the data from the polynomial fit; $\Delta(t)=U(t)-P_{3}(t)$
are plotted in Figure 3. Some structure at the $1\sigma$ level and some
outlying measurements remain. These difference data are fit to $\alpha/R^{2}$
and $\beta/R$ to give limits on the contribution of a term in the ${}^{238}Pu$
decay rate dependent on the Earth-Sun distance. The $90\%CL$ limit on from
these fits are $|\alpha|<0.84\times 10^{-4}$ and $|\beta|<0.99\times 10^{-4}$
respectively. The limiting functions, scale up by a factor of $10$ for
visibility, are also shown in Figure 3. $\alpha$ is to be compared with the
correlation seen in Reference 1 for ${}^{226}Rn$ decay of $\sim+3\times
10^{-2}$.

The Cassini RTG power data exclude any variation of the ${}^{238}Pu$ nuclear
decay rate correlated with the distance of the source from the Sun to a level
$350\times$ smaller than the effect reported by Reference 1. ${}^{238}Pu$ and
${}^{226}Ra$ are similar $\alpha$ emitters. Another physical or experimental
cause of the reported annual variations in nuclear decay rates appears to be
necessary. More generally Rutherford, Chadwick, and Ellis’s 1930 conclusion
that _The rate of transformation of an element has been found to be constant
under all conditions._Rutherford  now has solid experimental support at least
from Venus (R=0.7) to Mars (R=1.5).

I am indebted to several of my colleagues for calling this paper and physics
issue to my attention; Chris Quigg and Martin Hu of Fermilab and Jurgen
Engelfried of the Universidad Autǿnoma de San Luis Potosí, Mexico. I am also
thankful for very useful lunchtime conversations on this subject with several
of my Fermilab colleagues. I an indebted to Richard Ewell and Torrence Johnson
of JPL for making the RTG data available and Stephen Parke of Fermilab for
critical comments on this manuscript.

## References

  * (1) Jere H. Jenkins et al., _Evidence for Correlations Between Nuclear Decay Rates and the Earth-Sun Distance_ , arXiv:astro-ph:0808.3283v1.
  * (2) National Nuclear Data Center, http://www.nndc.bnl.gov/.
  * (3) http://www.lepp.cornell.edu/ seb/celestia/cassini-all.zip
  * (4) private communication, R. Ewell, JPL.
  * (5) R. Ewell, D. Hanks, J. Lozano, V. Shields, $\&$ E. Wood, _DEGRA - A computer Model for Predicting Long Term Thermoelectric Generator Performance_ , Space Technologies and Applications International Forum (STAIF), Albuquerque, New Mexico, February 12-16, 2005., [http://hdl.handle.net/2014/38760]
  * (6) S. E. Rutherford, J. Chadwick, and C. Ellis, _Radiations from Radioactive Substances_ (Cambridge University Press, 1930).