# Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical
Trials

Anna L. Trella
<EMAIL_ADDRESS>
School of Engineering and Applied Sciences
Harvard University &Kelly W. Zhang
<EMAIL_ADDRESS>
Graduate School of Business
Columbia University &Inbal Nahum-Shani
<EMAIL_ADDRESS>
Institute for Social Research
University of Michigan &Vivek Shetty
<EMAIL_ADDRESS>
Schools of Dentistry & Engineering
University of California, Los Angeles &Iris Yan
<EMAIL_ADDRESS>
School of Engineering and Applied Sciences
Harvard University &Finale Doshi-Velez
<EMAIL_ADDRESS>
School of Engineering and Applied Sciences
Harvard University &Susan A. Murphy
<EMAIL_ADDRESS>
School of Engineering and Applied Sciences
Harvard University

###### Abstract

Online reinforcement learning (RL) algorithms offer great potential for
personalizing treatment for participants in clinical trials. However,
deploying an online, autonomous algorithm in the high-stakes healthcare
setting makes quality control and data quality especially difficult to
achieve. This paper proposes algorithm fidelity as a critical requirement for
deploying online RL algorithms in clinical trials. It emphasizes the
responsibility of the algorithm to (1) safeguard participants and (2) preserve
the scientific utility of the data for post-trial analyses. We also present a
framework for pre-deployment planning and real-time monitoring to help
algorithm developers and clinical researchers ensure algorithm fidelity. To
illustrate our framework’s practical application, we present real-world
examples from the Oralytics clinical trial. Since Spring 2023, this trial
successfully deployed an autonomous, online RL algorithm to personalize
behavioral interventions for participants at risk for dental disease.

##  1 Introduction

Reinforcement learning (RL) algorithms are increasingly used in digital health
clinical trials to determine the delivery of interventions via devices like
wearables or mobile apps (Liao et al., 2019; Yom-Tov et al., 2017; Figueroa et
al., 2021; Forman et al., 2019). Recently, there has been growing interest in
the online class of RL algorithms that update their policies using incoming
participant data throughout the trial. This is because online RL algorithms
have greater potential to learn more effectively (e.g., handle user
heterogeneity, distribution shift, non-stationarity, etc.) and provide
personalized interventions. However, online algorithms require real-time
updates and interactions with complex systems. They can result in issues that,
if not properly monitored and planned for, jeopardize the entire trial. These
issues are especially concerning because clinical trials can be expensive and
take years to develop with multiple stakeholders.

Undetected issues or issues solved in an untimely manner can result in (1)
insufficient data for conducting post-trial analyses and (2) compromise the
effectiveness of the intervention and participant experience. Given the
complexity and real-time nature of implementing online RL algorithms in
healthcare settings, unexpected issues may arise even with careful planning
(e.g., updates with overflow values result in an incorrect policy; state or
reward data does not properly write to the database). There are well-
established criteria and a variety of existing frameworks for real-world
machine learning (ML) deployment (Oala et al., 2021; Challen et al., 2019;
Loftus et al., 2022; Sambasivan et al., 2021). While some of these frameworks
are specific to health, the inherent properties of online RL algorithms
introduce unique challenges that this literature does not address.

We first introduce the critical concept of algorithm fidelity in clinical
trials (Section 3), which refers to the responsibility of the online RL
algorithm to (1) safeguard participants and (2) preserve the scientific
utility of data for post-trial analyses. Then, we provide a framework for
helping research teams ensure algorithm fidelity through proper planning and
building a monitoring system around any online RL algorithm (Section 4). Our
work brings to the RL community’s attention the critical gap between fast-
paced algorithmic research and the feasibility of deploying these algorithms
effectively (i.e., in a way that does more good than harm) in high-stakes
clinical trials. To showcase the utility of the algorithm fidelity framework,
we share insights and issues identified by the real-time monitoring system
built for Oralytics. Oralytics successfully deployed an online RL algorithm in
an oral health clinical trial that has been in the field since Spring 2023.

##  2 Running Example: Supporting Oral Health

To facilitate the use of our algorithm fidelity framework (Section 4), we show
concrete examples of how we used our framework to plan and develop an
autonomous monitoring system for a real RL algorithm involved in an oral
health clinical trial. As our running example, we use Oralytics (Shetty, 2022;
Trella et al., 2022; 2023; Nahum-Shani et al., 2024): a digital behavioral
intervention trial with participants at risk for dental disease. The trial
involved an online RL algorithm to optimize the delivery of prompts to
encourage participants to engage in oral self-care behaviors.

Each trial participant received a commercially available electric toothbrush
with Bluetooth connectivity and built-in sensors and was asked to download the
Oralytics mobile application onto their smartphone. The RL algorithm
determined the delivery of prompts (e.g., brushing feedback and motivational
messages) via push notifications from the Oralytics app. In Spring 2023, a
pilot version of the RL algorithm was deployed for Oralytics with a cohort of
10 participants. In Fall 2023, the finalized RL algorithm was deployed in the
main clinical trial of Oralytics with 70 participants, recruited incrementally
with around 5 participants starting in the trial every 2 weeks. The trial
duration for each participant in the main trial was 70 days. Every day, the RL
algorithm determined the delivery of prompts for each participant. There were
two decision times when a prompt may be delivered: (1) one hour before the
participant-specified morning brushing time, and (2) one hour before the
participants-specified evening brushing time. The RL algorithm updated weekly
on Sundays with the entire history of data across all participants and up to
that time.

###  2.1 Defining the RL Problem

We review the definitions of state, action, and reward. For each of the
components, we use subscript $i\in[1\colon N]$ to denote the participant and
subscript $t\in[1\colon T]$ to denote the decision time. In addition, online
RL algorithms can be decomposed into two main parts: 1. a model of the
environment (participant behavior) and 2. a policy informed by the environment
model. The RL algorithm fits the model of the environment and iteratively
updates that model using a history of states, actions, and rewards from
participants. In Oralytics, the RL algorithm was a generalized contextual
bandit that performed Thompson sampling (Russo et al., 2018). Therefore, the
model of the environment was a model of the expected reward and the policy was
an action-selection probability formed using a posterior distribution.

The state $S_{i,t}$ for participant $i$ at decision time $t$ consists of
relevant features of the participant environment that provide the algorithm
with the current context of the participant. The state features could include
information such as current weather, the participant’s current location, or
recent adherence to the trial protocol. In Oralytics, the state features
included the time of day, how well a participant brushed in the past week,
amount of prompts sent to the participant (dosage) in the past week, and
recent app engagement. Notice that while some features are deterministically
calculated (i.e., time of day) other features require current sensor
information from the toothbrush and app. See Appendix A.1 for a full
discussion of Oralytics state features.

An action $A_{i,t}$ (i.e., intervention) is decided by the algorithm for
participant $i$ at decision time $t$. For Oralytics, the actions
$A_{i,t}\in\\{0,1\\}$ were binary and referred to whether to send a prompt
$(A_{i,t}=1)$ or not send one $(A_{i,t}=0)$. To select actions, the algorithm
observed state $S_{i,t}$, formed an action-selection probability $\pi_{i,t}$,
and selected $A_{i,t}=1$ with that probability. The RL algorithm in Oralytics
was a Thompson sampling-based algorithm so this action-selection probability
$\pi_{i,t}$ was formed using the posterior distribution maintained by the
algorithm. See Appendix A.2 for more details on the Oralytics action space and
policy.

The reward $R_{i,t}$ is a function of the participant’s health outcome
following the selection of $A_{i,t}$. The goal of the algorithm is to maximize
the total expected rewards collected across participants over the entire
trial, $\mathbb{E}\left[\sum_{t=1}^{T}\sum_{i=1}^{N}R_{i,t}\right]$. In
Oralytics, the reward was a function of brushing quality (proximal health
outcome) and a proxy for participant burden (cost of current action on the
effectiveness of future actions) (Trella et al., 2023). See Appendix A.3 for
more details on this reward.

Figure 1: Oralytics System and RL System Architecture. Brushing data is
captured by sensors in the toothbrush and uploaded to the commercial cloud via
a dock. The main controller gathers this data, along with app engagement data
from the Oralytics app, and feeds it to the RL service and the dashboards.
This sensor data is provided to the RL service to select actions and update.
Using the actions selected by the RL service, the main controller populates
intervention prompt content, and schedules prompts onto each participant’s
Oralytics app.

###  2.2 Oralytics System Components

The RL algorithm is one component of the RL software service. The RL service
consists of (1) an API to communicate with other services, (2) a dependency
module to request sensor data from other services, (3) a module to update the
RL algorithm based on newly collected participant data, (4) an action-
selection module that takes in the participant’s current state and assigns an
intervention based on the algorithm’s current policy, and (5) an internal
database that stores all the data necessary for the RL algorithm to function
and perform post-trial analyses.

To function properly, the RL service depends on stable interactions with a
variety of other services. We refer to this larger ecosystem of software
components as the Oralytics system (Figure 1). When a participant brushed, the
toothbrush recorded brushing sensor data from that session and when docked
properly, this data was sent to the commercial cloud. Similarly, when a
participant uses the Oralytics app, any app analytics data collected by the
Oralytics app was sent to the backend main controller. For each participant’s
decision-time, the backend main controller collected sensor data, made this
data available for the RL service to request, and called the RL API’s action
selection module. Once called, the RL service obtained the recent sensor data,
constructed the participants’ state features, then returned the actions
selected by the RL algorithm’s policy. The RL service saved the necessary data
corresponding to the state and action to the internal database. The backend
main controller then scheduled prompts accordingly. During algorithm update
time, the backend main controller collected sensor data needed for reward
construction from the commercial cloud, made this data available for the RL
service to request, and called the RL API’s algorithm update module. The RL
service obtained the recent sensor data, constructed rewards for each new
decision-time, and then updated its policy using the entire history of state,
action, and reward data. The RL service saved the necessary data corresponding
to reward and policy to the internal database. As one can see, proper RL
algorithm functionality requires frequent real-time communication between a
variety of components. While the trial is running, many things could go wrong.

##  3 Defining Algorithm Fidelity

To help research teams best prepare for RL algorithm deployment in the
challenging healthcare setting, we introduce algorithm fidelity: the
responsibility of the online RL algorithm in (1) safeguarding participants and
(2) preserving the scientific utility of the data in clinical trials. These
concepts build upon quality control and data quality, which are two well-
defined criteria for ML deployment. Quality control, used widely throughout
engineering, encompasses a comprehensive set of standards or predefined set of
requirements that a product must meet before launching. Data quality involves
best practices of handling or reporting the data used to train models. Here,
we extend quality control and data quality to concerns involving online RL
algorithms in clinical trials. Often times, other system components (see
Appendix B) are also responsible for ensuring quality control and data
quality. However, algorithm fidelity specifically refers to areas that are
directly controlled or caused by the RL algorithm. For example, monitoring for
app crashes and failing to collect app analytics falls under the purview of
the clinician dashboard. However, detecting errors in saving state values used
to select actions would come under monitoring of algorithm fidelity.

###  3.1 Related Work

We build on quality control and data quality discussed in the existing
literature on best-practice frameworks for deploying ML models in general and
discuss how they can be adapted to online RL algorithms in clinical trials.
Many of the ML frameworks for ensuring quality control or data quality are in
the offline or in batch settings where the algorithm is trained on previous
batch data and the model is fixed before deployment. In these settings,
quality control standards could include achieving baseline prediction accuracy
(Oala et al., 2021), handling distribution shifts and reporting levels of
confidence (Challen et al., 2019), explainability, fairness, and
reproducibility (Oala et al., 2020; Loftus et al., 2022), or preparing for
accidents and unintended risks (Amodei et al., 2016). Data quality could
include ensuring that models trained on the data can accurately represent and
predict for their task (Sambasivan et al., 2021), or forming a checklist of
what to report about AI algorithms in clinical trials such as which version of
the algorithm was used, how input data was acquired, and how the sample size
was determined (Liu et al., 2020).

###  3.2 Quality Control

The online nature and inherent properties of RL algorithms require additional
aspects of quality control. Unlike prediction models, validating the quality
of RL algorithms goes beyond straightforward performance metrics like
prediction accuracy. Also, online algorithms autonomously update using
incoming data from participants in the trial. Previous work covers monitoring
the deployment of between-trial algorithm updates with a fixed policy (Oala et
al., 2020), but online settings require identifying and addressing issues in
real-time of a policy that is frequently updated throughout the trial.
Furthermore, validation in the online setting is more complicated than in the
offline setting. In the offline setting, training can involve a human-in-the-
loop (Wu et al., 2022) for performing hyperparameter search, checking for
convergence, or performing clinical evaluations (Oala et al., 2020). An online
algorithm, however, can only be validated once the trial has completed (e.g.,
via off-policy analyses, one can evaluate whether an action was effective in
certain states for a participant).

Considering our clinical trial setting, we also extend quality control to
include standards that involve participant experience and safety. In any
clinical trial, participant protection from harm and adverse experiences is
the most important consideration. Although there is a list of ethical
guidelines that the clinical research team must follow (i.e., overseen by
Institutional Review Boards (Grady, 2015)) and guidelines that must be
followed to register the trial (i.e., at clinicaltrials.gov) 111The framework
considers participant protection responsible by the RL algorithm as opposed to
general participant protection guidelines in the trial. Note that ethics are
priority consideration prior to algorithmic development. For example, in
trials where the algorithm actions may have negative side effects (e.g.,
administering a drug), the action space is restricted by the current state of
the participant. , unintended harm can occur via the inappropriate choice of
actions by the RL algorithm. For example, the algorithm could deliver too many
interventions that take the participant away from normative activities and may
lead to burden, habituation, and subsequent dropout. On the other hand, the
algorithm could deliver too few interventions or none at all, thus diminishing
the intended health benefits of the intervention actions. Furthermore, the
algorithm could reduce the functionality of the participant’s smartphone by
depleting the battery or taking up a large amount of storage space.

###  3.3 Data Quality

Achieving data quality for online RL algorithms is also not straightforward.
RL algorithms use and store additional types of data as compared to ML
algorithms, such as the action-selection probability $\pi_{i,t}$ for every
participant $i$ at decision-time $t$. Online algorithms also require extra
considerations for data reporting and reproducibility. Unlike offline settings
where previous work suggests stating the version of the model used (Liu et
al., 2020), frequent updates of the RL algorithm in the online setting means
needing to index and save each policy update, the data that was used in each
update, and code versions and software packages if a code change needed to be
made during the trial.

We also extend data quality in our setting to focus on preserving the
scientific utility of data for post-trial analyses. Post-trial analyses refers
to the broad range of analyses conducted after the trial is completed. These
kinds of analyses can (1) improve the design of the intervention for future
iterations; (2) inform policy and decisions for rolling out the intervention
more broadly; and (3) support the development and refinement of health
behavior theories. We now highlight some important analyses of interest. At a
minimum, the clinical trial must collect data to perform a primary analysis
(Furberg & Friedman, 2012; Qian et al., 2022), a statistical analysis that
addresses the foremost scientific question of the trial. For example, the
primary analysis for Oralytics concerns the causal excursion effect (Boruvka
et al., 2018) of sending versus not sending a prompt on the participant’s
proximal oral self-care behaviors. Secondary analysis may focus on
investigating whether the benefits of sending (versus not sending) a prompt
vary by the participant’s baseline characteristics (e.g., age and gender), as
well as their current state (e.g., recent brushing quality or app engagement).
Additional analyses such as off-policy evaluation (Thomas & Brunskill, 2016;
Levine et al., 2020), other types of causal inference (Hadad et al., 2021),
and continual learning (Parisi et al., 2019; Khetarpal et al., 2022) assess
the benefit of using such an algorithm and improve the design of future
versions of the RL algorithm. Furthermore, sharing and publishing high-quality
trial data helps the broader scientific community design their own algorithm
for future trials.

##  4 A Pragmatic Framework for Ensuring Algorithm Fidelity

To maintain algorithm fidelity, we offer guidelines for both a pre-trial
planning phase (Section 4.1) and a real-time monitoring phase (Section 4.2)
while the algorithm is running during the trial. We view our framework as a
necessary step in the design and pre-deployment process when developing the RL
algorithm. Our framework helps research teams determine possible errors and
prioritize issues, developing a monitoring framework that ensures a stable
system for participant experience. In addition, our framework also helps
research teams solidify a data collection procedure that ensures the validity
of trial results and for the data to be usable to stakeholders and other
research teams after the trial.

###  4.1 Planning Phase

If you fail to plan, you plan to fail. Many issues that arise during the trial
can be avoided with proper planning. We will see in the later section (Section
4.2) that planning can also reduce the severity of certain issues. Here we
examine several topics related to quality control and data quality to consider
planning for.

##### Implementing Fallback Methods (Quality Control)

Ensuring that participants have consistent functionality throughout the trial
is an important part of ensuring a quality participant experience. Even if
worst-case issues arise (i.e., software system fails or sensor data is
unavailable for state and reward construction), participants should still get
some intervention and the algorithm should not update with incorrect or
corrupted data. To guarantee a baseline level of functionality, the RL
algorithm can be paired with fallback methods. The fallback method is a
procedure that is executed in lieu of the standard action-selection or update
procedure by the RL algorithm. For action selection, it is critical that each
participant receives at least some intervention throughout the trial, even if
the intervention is not personalized. For Oralytics, this manifests in two
ways. First, instead of providing a single action for the current decision
time, the Oralytics RL system constructs and provides a full schedule of
actions for the entire full trial length (i.e., 70 days for Oralytics)
starting at that decision time. For the first day in the schedule
($t^{\prime}=t,t+1$), the algorithm uses the current state $S_{i,t},S_{i,t+1}$
as input to the action-selection probability $\pi_{i,t}$. For decision times
within the first 2 weeks from the first day, the algorithm will use a modified
state space instead (See Appendix A.1.1). Finally, for all decision times
afterward, the algorithm selects actions with fixed probability
$\pi_{i,t}=0.5$. Second, if the main controller requests the schedule of
actions from the RL service and any step in the RL action-selection procedure
fails, a schedule of actions with fixed uniform probability $\pi_{i,t}=0.5$ is
deployed. For updating, if issues arise such as malformed data or if recent
data is not readily available, then the algorithm stores the data point but
does not update its policy with that data point.

##### Setting Restrictions (Quality Control and Data Quality)

Setting proper restrictions or constraints is necessary for safeguarding
participants and preserving the scientific utility of the data for post-trial
analyses. Here, the restrictions designed to safeguard participants are ones
the RL algorithm must adhere to and are not the system restrictions specified
in the trial protocol (See footnote in Section 3.2). For Oralytics, one of
these restrictions for participant protection included specifying thresholds
for too many or too few interventions. The number of interventions a
participant received was available on the clinician dashboard and was checked
by the staff every day. If a staff member noticed that the amount of
interventions exceeded the upper and lower thresholds, then they would contact
the RL software engineering team. To facilitate post-trial analyses, other
restrictions set in Oralytics included purposely deploying a stochastic RL
policy to enable continual learning, and restricting the stochastic RL policy
to select actions within a pre-specified range (bounded away from $0$ and $1$)
to enhance the ability to address scientific questions with sufficient power
(Yao et al., 2021) (See Appendix A.2 for further details).

##### Collecting Critical Data (Data Quality)

Recall from Section 3.3 that one of the main goals of algorithm fidelity is
maintaining the scientific utility of the data. Some analyses require
reproducing the algorithm (i.e., re-running or recreating the decisions made
by the RL algorithm), which is also important for validating results and
ensuring confidence in trial findings within the scientific community. Digital
intervention clinical trials require the coordination of many complex systems
(i.e., RL system, smartphones and/or wearables, data storage systems, etc.)
and some types of data cannot be recovered if not explicitly saved. Although
some data values could be reconstructed after the trial concludes, unexpected
issues could cause reconstruction to be difficult (e.g., bugs or code failures
due to OS updates may require code changes during the course of the trial,
which could make it difficult to reconstruct action selection probabilities).
Therefore, being able to run post-trial analyses hinges on how well the
scientific team set up the data collection and storage plan. This data
collection plan includes the variety of data fields that need to be collected
and plans regarding how to handle/denote missing and delayed data. For
Oralytics, the primary analysis involves brushing quality, which is a
component of the reward, but not exactly the reward itself. Therefore, the RL
algorithm saved all possible data and raw sensory information used in the
construction of the reward (See Appendix A.3). For reproducibility, Oralytics
saved the most basic data fields such as states, actions, and rewards as well
as action-selection probabilities used by the RL algorithm, seeds for various
random processes, code changes and versions of the algorithm.

###  4.2 Real-Time Monitoring Phase

Autonomously monitoring an online RL algorithm during the trial is important
for detecting, alerting, and preventing failures. Failures and errors could
arise during the trial that leads to critical situations (e.g., participants
did not get the correct intervention or database failed to save important data
fields). A quality monitoring system prevents incidents and detects problems
as soon as they arise. This allows the research team to immediately identify,
triage, and solve the issue which (1) minimizes the time and negative impact
on participants and (2) reduces the burden off of the staff in manually
monitoring the clinical trial system. To set up the monitoring system,
research teams should first specify and assign priority to potential issues
that could occur during the trial. Once these issues are specified, one can
use built-in packages (e.g., exception handling and Flask email for Python) to
send automatic emails when an issue occurs. To assist with this specification
and prioritization, we suggest categorizing issues into one of three levels of
severity: red, yellow, and green. For each category of issues, we also provide
a concrete example of how considering this issue helped with the development
of Oralytics.

##### Red Severity (Compromises Algorithm Fidelity)

Red severity issues are the most severe issues that compromise participant
experience or the scientific utility of the data and require immediate
attention. Red severity issues result in an immediate email to the RL software
engineering team and appear on the clinician dashboard. Participant experience
can be compromised if they receive an unreasonable amount of prompts. This
could be caused by communication errors between system components (RL
algorithm, main controller, app), out-of-range action-selection probabilities
due to numerical instability in the RL algorithm, or failure to recognize that
a participant has started in the trial. The scientific utility of the data
could be compromised if the RL internal data storage fails to properly write
values due to integration issues with the database management system. These
issues could include failed/timed out/closed connection with the database
and/or the database exceeded storage. Examples of red severity checks in
Oralytics included checking in the code if the RL algorithm selected an
unreasonable amount of prompts (less than the minimum amount or more than the
maximum amount) for a participant or checking for failures in the RL internal
database saving values. As of Spring 2024, there have been no red severity
issues in Oralytics that were caused by the RL algorithm.

##### Yellow Severity (Compromises RL Algorithm Functionality)

Yellow severity issues are issues that compromise the ability of the RL
algorithm to learn or personalize interventions for participants. Yellow
severity issues also result in an immediate email and appear on the monitoring
dashboard. However, the urgency to fix a yellow severity issue is lower,
allowing more time for the RL software engineering team to resolve. Notice
that issues impacting RL algorithm learning and personalizing are not red
severity issues because of fallback methods (Section 4.1). RL algorithm
learning and personalizing can be compromised if the algorithm (1) does not
obtain sensor data in a stable or timely manner and (2) fails to fetch
necessary data from the RL internal database. In Oralytics, examples of yellow
severity checks included checking if requests to obtain sensor data failed, if
the request succeeded but there were issues parsing the data, and if there
were issues connecting to and fetching data from the RL database. During the
Oralytics trial, the monitoring system detected the yellow severity issue of
the RL algorithm failing to construct the current state for a participant.
This issue involved the RL algorithm failing to obtain app analytics data for
a participant, which is part of the state space. The monitoring system
automatically detected this issue through the check for issues parsing the
sensor information and sent an email to the RL software engineering team. The
team triaged and discovered that the max limit of requests per minute to the
main controller was exceeded when more participants entered the trial.
Exceeding this limit causes the returned response to be empty and therefore
un-parseable. The team fixed the issue by coordinating a more efficient
request strategy with the main controller team.

##### Green Severity (Impacts Post-Trial Data Analysis)

Green severity issues are problems that need to be documented so that post-
trial analyses can be adjusted. If not properly accounted for, these issues
could compromise the validity of the primary analysis or the opportunity to
improve the system for future trials. Green severity issues are only problems
if not documented. Handling these issues could involve a list of items to
investigate after the trial is over (e.g., consistency in data saved by the
main controller with data in the RL internal database). Sometimes handling
these issues involves components outside of the RL system (e.g., main
controller or app). Therefore, we suggest making contracts with other
component teams prior to the trial to agree on who will document what data and
which issues. In Oralytics, some documented items included all issues that
arose in the RL service and its dependencies, timestamps and any associated
data when the RL algorithm successfully updated and conversely when it did
not, and timestamps when the main controller called the RL API and the
attempted call has failed. During the Oralytics trial, a green severity issue
was detected by clinical staff when the dashboards showed that only a small
fraction of participants had prompts scheduled on their apps. As part of the
contract between system components, the staff contacted both the main
controller and the RL software engineering teams. Both teams triaged and
discovered that the RL API was functioning properly, but the main controller
was timing out (i.e., did not wait long enough to get the full schedule for
every participant currently in the trial) when calling the RL API. Therefore,
the main controller incorrectly pushed a blank schedule of prompts for certain
participants. The main controller team fixed the issue the next business day
and the RL team documented this issue, the participants affected, and the
decision times that were affected.

##  5 Discussion

In this paper, we discuss adherence to algorithm fidelity, an important
problem that research teams will inevitably face when implementing an online
RL algorithm in a digital intervention clinical trial. Due to the
unpredictable nature of the real-world environment and the dynamic nature of
online RL algorithms, maintaining algorithm fidelity can become difficult and
expensive unless planned for. We used our successful trial to ground pragmatic
solutions from our framework for pre-trial planning and building an autonomous
monitoring system. While the focus of this work is on clinical trials, we
expect many ideas in this paper to be transferable to general digital
experimentation (e.g., online education, ads, UI testing, etc.). By offering
the algorithm fidelity definition and a framework for achieving algorithm
fidelity, we hope to smooth the transition from algorithm development to real-
world application: giving other researchers the confidence to deploy their
innovative online RL algorithms in real life.

## References

  * Amodei et al. (2016) Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané.  Concrete problems in ai safety.  _arXiv preprint arXiv:1606.06565_ , 2016.
  * Boruvka et al. (2018) Audrey Boruvka, Daniel Almirall, Katie Witkiewitz, and Susan A Murphy.  Assessing time-varying causal effect moderation in mobile health.  _Journal of the American Statistical Association_ , 113(523):1112–1121, 2018.
  * Challen et al. (2019) Robert Challen, Joshua Denny, Martin Pitt, Luke Gompels, Tom Edwards, and Krasimira Tsaneva-Atanasova.  Artificial intelligence, bias and clinical safety.  _BMJ quality & safety_, 2019.
  * Figueroa et al. (2021) Caroline A Figueroa, Adrian Aguilera, Bibhas Chakraborty, Arghavan Modiri, Jai Aggarwal, Nina Deliu, Urmimala Sarkar, Joseph Jay Williams, and Courtney R Lyles.  Adaptive learning algorithms to optimize mobile applications for behavioral health: guidelines for design decisions.  _Journal of the American Medical Informatics Association_ , 28(6):1225–1234, 2021.
  * Forman et al. (2019) Evan M Forman, Stephanie G Kerrigan, Meghan L Butryn, Adrienne S Juarascio, Stephanie M Manasse, Santiago Ontañón, Diane H Dallal, Rebecca J Crochiere, and Danielle Moskow.  Can the artificial intelligence technique of reinforcement learning use continuously-monitored digital data to optimize treatment for weight loss?  _Journal of behavioral medicine_ , 42:276–290, 2019.
  * Furberg & Friedman (2012) Curt D Furberg and Lawrence M Friedman.  Approaches to data analyses of clinical trials.  _Progress in cardiovascular diseases_ , 54(4):330–334, 2012.
  * Grady (2015) Christine Grady.  Institutional review boards: Purpose and challenges.  _Chest_ , 148(5):1148–1155, 2015.
  * Hadad et al. (2021) Vitor Hadad, David A Hirshberg, Ruohan Zhan, Stefan Wager, and Susan Athey.  Confidence intervals for policy evaluation in adaptive experiments.  _Proceedings of the National Academy of Sciences_ , 118(15), 2021.
  * Khetarpal et al. (2022) Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup.  Towards continual reinforcement learning: A review and perspectives.  _Journal of Artificial Intelligence Research_ , 75:1401–1476, 2022.
  * Levine et al. (2020) Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu.  Offline reinforcement learning: Tutorial, review, and perspectives on open problems.  _arXiv preprint arXiv:2005.01643_ , 2020.
  * Liao et al. (2019) Peng Liao, Kristjan H. Greenewald, Predrag V. Klasnja, and Susan A. Murphy.  Personalized heartsteps: A reinforcement learning algorithm for optimizing physical activity.  _CoRR_ , abs/1909.03539, 2019.  URL http://arxiv.org/abs/1909.03539.
  * Liu et al. (2020) Xiaoxuan Liu, Samantha Cruz Rivera, David Moher, Melanie J Calvert, Alastair K Denniston, Hutan Ashrafian, Andrew L Beam, An-Wen Chan, Gary S Collins, Ara DarziJonathan J Deeks, et al.  Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the consort-ai extension.  _The Lancet Digital Health_ , 2(10):e537–e548, 2020.
  * Loftus et al. (2022) Tyler J Loftus, Patrick J Tighe, Tezcan Ozrazgat-Baslanti, John P Davis, Matthew M Ruppert, Yuanfang Ren, Benjamin Shickel, Rishikesan Kamaleswaran, William R Hogan, J Randall Moorman, et al.  Ideal algorithms in healthcare: explainable, dynamic, precise, autonomous, fair, and reproducible.  _PLOS digital health_ , 1(1):e0000006, 2022.
  * Nahum-Shani et al. (2024) Inbal Nahum-Shani, Zara M Greer, Anna L Trella, Kelly W Zhang, Stephanie M Carpenter, Dennis Ruenger, David Elashoff, Susan A Murphy, and Vivek Shetty.  Optimizing an adaptive digital oral health intervention for promoting oral self-care behaviors: Micro-randomized trial protocol.  _Contemporary Clinical Trials_ , pp. 107464, 2024.
  * Oala et al. (2020) Luis Oala, Jana Fehr, Luca Gilli, Pradeep Balachandran, Alixandro Werneck Leite, Saul Calderon-Ramirez, Danny Xie Li, Gabriel Nobis, Erick Alejandro Muñoz Alvarado, Giovanna Jaramillo-Gutierrez, et al.  Ml4h auditing: From paper to practice.  In _Machine learning for health_ , pp. 280–317. PMLR, 2020.
  * Oala et al. (2021) Luis Oala, Andrew G Murchison, Pradeep Balachandran, Shruti Choudhary, Jana Fehr, Alixandro Werneck Leite, Peter G Goldschmidt, Christian Johner, Elora DM Schörverth, Rose Nakasi, et al.  Machine learning for health: algorithm auditing & quality control.  _Journal of medical systems_ , 45:1–8, 2021.
  * Parisi et al. (2019) German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter.  Continual lifelong learning with neural networks: A review.  _Neural networks_ , 113:54–71, 2019.
  * Qian et al. (2022) Tianchen Qian, Ashley E Walton, Linda M Collins, Predrag Klasnja, Stephanie T Lanza, Inbal Nahum-Shani, Mashfiqui Rabbi, Michael A Russell, Maureen A Walton, Hyesun Yoo, et al.  The microrandomized trial for developing digital interventions: Experimental design and data analysis considerations.  _Psychological methods_ , 2022.
  * Russo et al. (2018) Daniel J Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen, et al.  A tutorial on thompson sampling.  _Foundations and Trends® in Machine Learning_ , 11(1):1–96, 2018.
  * Sambasivan et al. (2021) Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo.  “everyone wants to do the model work, not the data work”: Data cascades in high-stakes ai.  In _proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ , pp. 1–15, 2021.
  * Shetty (2022) Vivek Shetty.  Micro-randomized trial to optimize digital oral health behavior change interventions.  Identifier NCT02747927. U.S. National Library of Medicine. https://clinicaltrials.gov/ct2/show/NCT05624489 (accessed 2023-06-16), Nov 2022.
  * Thomas & Brunskill (2016) Philip Thomas and Emma Brunskill.  Data-efficient off-policy policy evaluation for reinforcement learning.  In _International Conference on Machine Learning_ , pp. 2139–2148. PMLR, 2016.
  * Trella et al. (2022) Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, and Susan A. Murphy.  Designing reinforcement learning algorithms for digital interventions: Pre-implementation guidelines.  _Algorithms_ , 15(8), 2022.  ISSN 1999-4893.  doi: 10.3390/a15080255.  URL https://www.mdpi.com/1999-4893/15/8/255.
  * Trella et al. (2023) Anna L Trella, Kelly W Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, and Susan A Murphy.  Reward design for an online reinforcement learning algorithm supporting oral self-care.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 37, pp. 15724–15730, 2023.
  * Wu et al. (2022) Xingjiao Wu, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma, and Liang He.  A survey of human-in-the-loop for machine learning.  _Future Generation Computer Systems_ , 135:364–381, 2022.
  * Yao et al. (2021) Jiayu Yao, Emma Brunskill, Weiwei Pan, Susan Murphy, and Finale Doshi-Velez.  Power constrained bandits.  In _Machine Learning for Healthcare Conference_ , pp. 209–259. PMLR, 2021.
  * Yom-Tov et al. (2017) Elad Yom-Tov, Guy Feraru, Mark Kozdoba, Shie Mannor, Moshe Tennenholtz, and Irit Hochberg.  Encouraging physical activity in patients with diabetes: intervention using a reinforcement learning system.  _Journal of medical Internet research_ , 19(10):e338, 2017.
  * Zhang et al. (2022) Kelly W Zhang, Lucas Janson, and Susan A Murphy.  Statistical inference after adaptive sampling for longitudinal data.  _arXiv preprint arXiv:2202.07098_ , 2022.

##  Appendix A RL Algorithm Supplementary Information

For completeness, we offer an overview of components of the Oralytics RL
algorithm to make ideas in the main paper clear. We again use $i$ to denote
the participant and $t$ to denote the decision time. See Appendix A of Nahum-
Shani et al. (2024) for full details on the Oralytics RL algorithm.

###  A.1 State Space of the RL Algorithm

$S_{i,t}\in\mathbb{R}^{d}$ represents the state of dimension $d$ (num. of
features).

##### Baseline and Advantage State Space

$f(S_{i,t})\in\mathbb{R}^{5}$ denotes the features used to model both the
baseline reward function and the advantage. For Oralytics, the baseline and
advantage features are the same (i.e., all $f(S_{i,t})$), but this is a design
choice, and they do not have to be.

  1. 1.

Bias / Intercept Term $\in\mathbb{R}$

  2. 2.

Time of Day (Morning/Evening) $\in\\{0,1\\}$

  3. 3.

$\bar{B}$: Exponential Average of Brushing Quality Over Past 7 Days
(Normalized) $\in\mathbb{R}$

  4. 4.

$\bar{A}$: Exponential Average of Messages Sent Over Past 7 Days $\in[0,1]$

  5. 5.

Prior Day App Engagement (Opened App / Not Opened App) $\in\\{0,1\\}$

####  A.1.1 Modified State Space for Fallback Method

As a fallback method, instead of providing a single action for the current
decision time, the Oralytics RL system constructed and provided a full
schedule of actions for the entire 70-day study length starting at the current
decision time. For decision times within the first 2 weeks from the first day,
the algorithm uses the state space as specified above with the following
modifications:

  * •

Feature 3 is replaced with the Most Recent Exponential Average Brushing
Quality Over Past 7 Days (Normalized) $\in\mathbb{R}$

  * •

Feature 5 is replaced with the Best Guess of Prior Day App Engagement (Opened
App / Not Opened App) $=0$

Of the state features presented earlier, features 1, 2, and 4 were known
except for features 3 ($\bar{B}$) and 5 (prior day app engagement). Since
future values of $\bar{B}$ were unknown, we imputed this feature with the most
recent value of $\bar{B}_{i,t}$ known when the schedule was formed. Namely,
the $\bar{B}$ value for decision times $j=t+2,...,t+27$ used the same
$\bar{B}$ value as decision times $t,t+1$. For the prior day app engagement
feature, we imputed the value to 0, because our best guess was that the
participant does not getting a fresh schedule because they did not open the
app.

###  A.2 Action Space and Policy of the RL Algorithm

The algorithm decided between sending a prompt $A_{i,t}=1$ or not $A_{i,t}=0$.
If $A_{i,t}=1$, the main controller (described in Section 2.2) randomly
selected a prompt from the intervention content component. There were 3
different prompt categories: (1) participant winning a gift (direct
reciprocity), (2) participant winning a gift for their favorite charity
(reciprocity by proxy), and (3) Q&A for the morning decision time or feedback
on prior brushing for the evening decision time. To decide on the exact
prompt, the main controller first sampled a category with equal probability
and then sampled with replacement a prompt from that category. Due to the
large number of prompts in each category, it is highly unlikely that a
participant received the same prompt twice.

We now discuss the policy. Since the RL algorithm in Oralytics was a
generalized contextual bandit, the model of the participant environment was a
model of the mean reward. To model the reward, the Oralytics RL algorithm used
a Bayesian Linear Regression with action centering model:

$R_{i,t}=f(S_{i,t})^{T}\alpha_{0}+\pi_{i,t}f(S_{i,t})^{T}\alpha_{1}+(A_{i,t}-\pi_{i,t})f(S_{i,t})^{T}\beta+\epsilon_{i,t}$
(1)

where $\pi_{i,t}$ is the probability that the RL algorithm selects action
$A_{i,t}=1$ in state $S_{i,t}$. $\epsilon_{i,t}\sim\mathcal{N}(0,\sigma^{2})$
and there are priors on
$\alpha_{0}\sim\mathcal{N}(\mu_{\alpha_{0}},\Sigma_{\alpha_{0}})$,
$\alpha_{1}\sim\mathcal{N}(\mu_{\beta},\Sigma_{\beta})$,
$\beta\sim\mathcal{N}(\mu_{\beta}\Sigma_{\beta})$.

##### Posterior Updating

Let $\tau(i,t)$ be the update-time for participant index $i$ that included the
current decision time $t$. We needed an additional index because update times
$\tau$ and decision times $t$ are not on the same cadence. At update time, the
reward model updated the posterior with all history of state, action, and
rewards up to that point. Since the RL algorithm used Thompson sampling, the
algorithm updated parameters $\mu_{\tau(i,t)}^{\text{post}}$ and
$\Sigma_{\tau(i,t)}^{\text{post}}$ of the posterior distribution.

##### Action-Selection

The RL algorithm for Oralytics was a modified posterior sampling algorithm
called the smooth posterior sampling algorithm. The algorithm selected action
$A_{i,t}\sim\text{Bern}(\pi_{i,t})$:

$\displaystyle\pi_{i,t}=\mathbb{E}_{\tilde{\beta}\sim\mathcal{N}(\mu_{\tau(i,t)-1}^{\text{post}},\Sigma_{\tau(i,t)-1}^{\text{post}})}\left[\rho(s^{\top}\tilde{\beta})\big{|}\mathcal{H}_{1:n,\tau(i,t)-1},S_{i,t}=s\right]$
(2)

$\tau(i,t)-1$ was the last update-time for the posterior parameters. Notice
that the last expectation above is only over the draw of $\tilde{\beta}$ from
the posterior distribution.

In smooth posterior sampling, $\rho$ is a smooth function chosen to enhance
the replicability of the randomization probabilities if the study is repeated
(Zhang et al., 2022). The Oralytics RL algorithm set $\rho$ to be a
generalized logistic function:

$\rho(x)=L_{\min}+\frac{L_{\max}-L_{\min}}{\big{[}1+c\exp(-bx)\big{]}^{k}}$
(3)

where asymptotes $L_{\min},L_{\max}$ are clipping values (bounded away from 0
and 1) to enhance the ability to answer scientific questions with sufficient
power (Yao et al., 2021).

###  A.3 Reward for the RL Algorithm

We defined the reward $R_{i,t}$ given to the algorithm to be a function of
brushing quality $Q_{i,t}$ (i.e., proximal health outcome) in order to improve
the algorithm’s learning (Trella et al., 2023). The reward $R_{i,t}$ was used
to update the posterior distribution of the parameters in the reward model
described in Appendix A.2. The reward for the $i$th participant at decision
time $t$ was:

$R_{i,t}:=Q_{i,t}-C_{i,t}$ (4)

The cost term $C_{i,t}$ allowed the RL algorithm to optimize for immediate
healthy brushing behavior, while also considering the delayed effects of the
current action on the effectiveness of future actions. The cost term is a
function (with parameters $\xi_{1},\xi_{2}$) which took in current state
$S_{i,t}$ and action $A_{i,t}$ and outputted the delayed negative effect of
currently sending a prompt.

Let $\bar{B}_{i,t}$ and $\bar{A}_{i,t}$ be state features 3 and 4 defined
earlier. The cost of sending a prompt was:

$C_{i,t}:=\begin{cases}\xi_{1}\mathbb{I}[\bar{B}_{i,t}>b]\mathbb{I}[\bar{A}_{i,t}>a_{1}]&\\\
\hskip
28.45274pt+\xi_{2}\mathbb{I}[\bar{A}_{i,t}>a_{2}]&\smash{\raisebox{6.88889pt}{if
$A_{i,t}=1$}}\\\ 0&\mathrm{if~{}}A_{i,t}=0\end{cases}$ (5)

##  Appendix B Additional Forms of Fidelity

In this paper, we focus mainly on RL algorithm fidelity; however, there can be
other types of fidelity that are important to consider when running an online
algorithm in a clinical trial. In addition to algorithm fidelity, there is
also system fidelity (i.e., how each component in the clinical trial works
together) and participant fidelity (i.e., participant adherence when
participating in the trial). System fidelity can involve verifying correct
communication between the sensory-collection device, the data storage system,
the computation system where the algorithm runs, and the device that
administers the action. Notice that algorithm fidelity is part of system
fidelity. On the other hand, participant fidelity can involve concerns such as
ensuring the participants are able to correctly download or update the app,
keep their smart device charged, actions needed from the participant to obtain
sensory data, etc. We make the distinction that issues can arise that impact
post-trial analyses and participant experiences that are caused by other
components of the system failing (e.g., expired credentials cause participants
to lose access to the mobile app), however, this paper focuses on issues that
are directly controlled or caused by the RL algorithm.


# Effect of the dynamic pressure on the shock structure and sub-shocks
formation in a mixture of polyatomic gases

Tommaso Ruggeri<EMAIL_ADDRESS>Department of Mathematics $\&$ Alma
Mater Research Center on Applied Mathematics, University of Bologna, Via
Saragozza, 8, Bologna, 40123, Italy  Shigeru Taniguchi
<EMAIL_ADDRESS>Department of Creative Engineering, National
Institute of Technology, Kitakyushu College, 5-20-1 Shii, Kitakyushu,
802-0985, Japan

###### Abstract

We study the shock structure and the sub-shocks formation in a binary mixture
of rarefied polyatomic gases, considering the dissipation only due to the
dynamic pressure. We classify the regions depending on the concentration and
the Mach number for which there may exist the sub-shock in the profile of
shock structure in one or both constituents or not for prescribed values of
the mass ratio of the constituents and the ratios of the specific heats. We
compare the regions with the ones of the corresponding mixture of Eulerian
gases and we perform the numerical calculations of the shock structure for
typical cases previously classified and confirm whether sub-shocks emerge.

keywords: Shock structure, Mixture of gases, Rational extended thermodynamics,
Polyatomic gases, Dynamic pressure, Sub-shock formation, Bulk viscosity

MSC Classification: 35L67, 76L05, 76N30, 35L02

##  1 Introduction

The qualitative and quantitative analysis of the shock structure is important
for many applications and a challenging subject for physics to develop a valid
theory to explain the rapid and steep change of the physical quantities near
the shock front [1, 2]. In order to understand the structure of strong shock
waves involving a highly-nonequilibrium state beyond the validity range of the
_local equilibrium_ assumption, we need to establish an appropriate theory.
Non-equilibrium thermodynamics based on the local equilibrium assumption,
including the Navier-Stokes-Fourier (NSF) theory, is not able to explain the
shock structure with large Mach numbers.

The kinetic approaches based on the Boltzmann equation, such as DSMC (Direct
Simulation Monte Carlo), provide successful results on the shock structure in
a rarefied monatomic gas [3]. However, due to internal modes, modeling the
collision terms becomes difficult for polyatomic gases. Particularly difficult
for a mixture of rarefied polyatomic gases object of the present study. On
these subjects, see the recent works [4, 5] and the review paper [6].

Rational Extended Thermodynamics (RET) [7, 8, 9] is a phenomenological theory
that is valid beyond the validity range of the local equilibrium assumption.
In addition to the usual equilibrium quantities, the RET theory adopts
dissipative fluxes, such as the viscous stress and the heat flux, as
independent variables. The closure of the system is achieved by using
universal principles: objectivity, entropy principle, and thermodynamic
stability.

In the case of rarefied polyatomic gas with large bulk viscosity, as the Mach
number increases, the shock structure changes from the nearly symmetric
profile (Type A) to an asymmetric profile (Type B) and further to the profile
with thick and thin layers (Type C) [1]. RET can explain these changes of the
shock wave structure in a unified way [10, 11] in contrast to previous
approaches such as the Bethe-Teller theory [12] and the prediction by Navier-
Stokes-Fourier theory [13]. In particular, the shock structure in a rarefied
polyatomic gas with large bulk viscosity (large relaxation time for the
dynamic pressure) can be predicted by RET with only $6$ independent fields
(RET6) [14, 15]; the mass density $\rho$, the velocity $\mathbf{v}$, the
temperature $T$, and the dynamic pressure $\Pi$. Within the limited resolution
of RET6, the thin layer is described as a sub-shock [11, 16] while RET14 with
$14$ independent variables can explain the fine structure with very steep but
continuous change. After the successful explanation of the shock structure by
RET, numerical analysis based on the kinetic theory was done. The agreement
between the theoretical predictions by RET and kinetic theory strongly
supports the usefulness of the analysis based on RET [17, 18].

The next step is to analyze the shock structure in a mixture of polyatomic
gases. In a recent paper [19], by using the framework of RET, the field
equations of a mixture of polyatomic gases are derived in the context of the
multi-temperature model where each constituent has $6$ fields.

For a generic quasi-linear hyperbolic system of partial differential equations
it has been shown that the profile of the shock structure may not be
continuous only when the shock velocity meets a characteristic velocity [20].
However, this necessary condition becomes also sufficient when the shock speed
is greater than the maximum characteristic velocity evaluated in the
unperturbed field, as was proved in a theorem by Boillat and Ruggeri [21]. The
present status of the sub-shock formation is summarized in a recent survey
[22] and in the book [9].

A sub-shock emerges only after the maximum characteristic velocity in the
context of RET theory for a single fluid of rarefied gases according to the
numerical calculations [23, 24]. On the other hand, sub-shock may appear
before the maximum characteristic velocity, and multiple sub-shock may also
arise in the context of mixture theories [25, 26, 27] and toy models [24, 28].

In the case of the binary Eulerian mixture of rarefied polyatomic gases, the
authors have given a complete classification of the regions in the plane of
concentration and Mach number where both scenarios of sub-shocks before and
after the maximum characteristic velocity arises [29, 30]. For a dissipative
mixture of gases, we started the equivalent analysis of Eulerian mixtures in a
recent paper [31] but examined only some particular values of the structural
parameters and have not given a complete classification of the possible
regions, that is, the aim of the present paper.

The present paper is organized as follows: In Section 2, we summarize the
basic equations for the present analysis. In Section 3, the shock structure
problem is presented, and in Section 4 are classified the possible regions
depending on the concentration and Mach number in which there is the
possibility of sub-shock formation. In Section 5, the numerical calculations
of the shock structure are given and are confirmed whether the sub-shocks
arise or not. Section 6 shows the Conclusions.

##  2 Basic equations for a binary mixture of RET6 gases

We consider a binary mixture of rarefied polytropic gases described by the
following thermal and caloric equations of state:

$p_{\alpha}=\frac{k_{B}}{m_{\alpha}}\rho_{\alpha}T_{\alpha},\qquad\varepsilon_{\alpha}=\frac{k_{B}}{m_{\alpha}(\gamma_{\alpha}-1)}T_{\alpha},$

where the quantities with suffix $\alpha$ $(=1,2)$ represent the corresponding
quantities for the constituent $\alpha$ and $p_{\alpha}$, $k_{B}$,
$m_{\alpha}$, $T_{\alpha}$, $\varepsilon_{\alpha}$, and $\gamma_{\alpha}$ are,
respectively, the pressure, the Boltzmann constant, the mass of a molecule,
the temperature, the specific internal energy, and the ratio of the specific
heats. In the present analysis, we consider polytropic gases in which the
specific heat is independent of the temperature, and therefore also
$\gamma_{\alpha}$ is constant.

In the present study, we analyze plane shock waves propagating along with
$x$-direction and hereafter, we focus on the one-dimensional problem.
Furthermore, it is assumed that the mixture is inert and no chemical reactions
occur.

The system of a binary mixture of RET6 gases without shear viscosity and heat
conductivity was given in the paper [19] 111In [32] the production terms
$\hat{e}_{1}$ and $\hat{\omega}_{1}$ differ from the present one by a factor
$2$. The present choice is better to compare with previous results in the case
of Eulerian gases.:

$\displaystyle\frac{\partial\rho_{1}}{\partial
t}+\frac{\partial\rho_{1}v_{1}}{\partial x}=0,$
$\displaystyle\frac{\partial\rho_{1}v_{1}}{\partial
t}+\frac{\partial}{\partial
x}\left\\{\rho_{1}v^{2}_{1}+p_{1}+\Pi_{1}\right\\}=\hat{m}_{1},$
$\displaystyle\frac{\partial}{\partial
t}\left(\rho_{1}v^{2}_{1}+2\rho_{1}\varepsilon_{1}\right)+\frac{\partial}{\partial
x}\left\\{\left[\rho_{1}v^{2}_{1}+2\rho_{1}\varepsilon_{1}+2p_{1}+2\Pi_{1}\right]v_{1}\right\\}=2(\hat{e}_{1}+\hat{m}_{1}v),$
$\displaystyle\frac{\partial}{\partial
t}\left\\{\rho_{1}v_{1}^{2}+3(p_{1}+\Pi_{1})\right\\}+\frac{\partial}{\partial
x}\left\\{\left[\rho_{1}v_{1}^{2}+5(p_{1}+\Pi_{1})\right]v_{1}\right\\}=-\frac{3\Pi_{1}}{\tau_{1}}+2(\hat{\omega}_{1}+\hat{m}_{1}v),$
$\displaystyle\frac{\partial\rho_{2}}{\partial
t}+\frac{\partial\rho_{2}v_{2}}{\partial x}=0,$ (1)
$\displaystyle\frac{\partial\rho_{2}v_{2}}{\partial
t}+\frac{\partial}{\partial
x}\left\\{\rho_{2}v^{2}_{2}+p_{2}+\Pi_{2}\right\\}=-\hat{m}_{1},$
$\displaystyle\frac{\partial}{\partial
t}\left(\rho_{2}v^{2}_{2}+2\rho_{2}\varepsilon_{2}\right)+\frac{\partial}{\partial
x}\left\\{\left[\rho_{2}v^{2}_{2}+2\rho_{2}\varepsilon_{2}+2p_{2}+2\Pi_{2}\right]v_{2}\right\\}=-2(\hat{e}_{1}+\hat{m}_{1}v),$
$\displaystyle\frac{\partial}{\partial
t}\left\\{\rho_{2}v_{2}^{2}+3(p_{2}+\Pi_{2})\right\\}+\frac{\partial}{\partial
x}\left\\{\left[\rho_{2}v_{2}^{2}+5(p_{2}+\Pi_{2})\right]v_{2}\right\\}=-\frac{3\Pi_{2}}{\tau_{2}}-2(\hat{\omega}_{1}+\hat{m}_{1}v),$

where $v$ is the velocity of center of mass and $\tau_{\alpha}>0$ is the
relaxation time for the dynamic pressure $\Pi_{\alpha}$ of the constituent
$\alpha$. The production terms, $\hat{m}_{\alpha}$, $\hat{e}_{\alpha}$, and
$\hat{\omega}_{\alpha}$ represent the interchange of the momentum, the energy,
the momentum flux, respectively evaluated for $v=0$.

The expressions of $\hat{m}_{1}$, $\hat{e}_{1}$, and $\hat{\omega}_{1}$ are
obtained in such that the entropy production is positive and quadratic as
usual in non-equilibrium thermodynamics [19]

$\begin{split}\hat{m}_{1}=\psi_{11}\hat{\Lambda}_{1},\quad\hat{e}_{1}=2(\theta_{11}\hat{\eta}_{1}+\kappa_{11}\hat{\zeta}_{1}),\quad\hat{\omega}_{1}=2(\kappa_{11}\hat{\eta}_{1}+\phi_{11}\hat{\zeta}_{1}),\end{split}$
(2)

with

$\begin{split}&\hat{\Lambda}_{1}=-\frac{u_{1}}{T_{1}}\left(1-\frac{\Pi_{1}}{p_{1}}\right)+\frac{u_{2}}{T_{2}}\left(1-\frac{\Pi_{2}}{p_{2}}\right),\\\
&\hat{\eta}_{1}=\frac{1}{2T_{1}}\left(1-\frac{\Pi_{1}}{p_{1}}\right)-\frac{1}{2T_{2}}\left(1-\frac{\Pi_{2}}{p_{2}}\right),\\\
&\hat{\zeta}_{1}=-\frac{1}{(5-3\gamma_{1})T_{1}}\frac{\Pi_{1}}{p_{1}}+\frac{1}{(5-3\gamma_{2})T_{2}}\frac{\Pi_{2}}{p_{2}},\end{split}$
(3)

where $u_{\alpha}=v_{\alpha}-v$ is the diffusion velocity. The coefficient
$\psi_{11}>0$ and the matrix

$\displaystyle\begin{pmatrix}\theta_{11}&\kappa_{11}\\\
\kappa_{11}&\phi_{11}\end{pmatrix},\qquad\text{is positive definite}.$ (4)

In general, the phenomenological coefficients $\psi_{11}$, $\theta_{11}$,
$\kappa_{11}$, and $\omega_{11}$ depend on the mass densities and temperatures
and the functional forms of the coefficients may be determined by kinetic
theoretical consideration and/or experimental data.

###  2.1 Field equations for global quantities

By taking sum of (2), we have the following equivalent system

$\displaystyle\frac{\partial\rho}{\partial t}+\frac{\partial\rho v}{\partial
x}=0,$ $\displaystyle\frac{\partial\rho v}{\partial
t}+\frac{\partial}{\partial x}\left(\rho v^{2}+p+\Pi-\sigma\right)=0,$
$\displaystyle\frac{\partial}{\partial t}\left(\frac{1}{2}\rho
v^{2}+\rho\varepsilon\right)+\frac{\partial}{\partial
x}\left\\{\left(\frac{1}{2}\rho
v^{2}+\rho\varepsilon+p+\Pi-\sigma\right)v+q\right\\}=0,$
$\displaystyle\frac{\partial}{\partial t}\left\\{\rho
v^{2}+3(p+\Pi)\right\\}+\frac{\partial}{\partial x}\left\\{\left[\rho
v^{2}+5(p+\Pi)-2\sigma\right]v+2Q\right\\}=-3\left(\frac{\Pi_{1}}{\tau_{1}}+\frac{\Pi_{2}}{\tau_{2}}\right),$
$\displaystyle\frac{\partial\rho_{1}}{\partial
t}+\frac{\partial\rho_{1}v_{1}}{\partial x}=0,$ (5)
$\displaystyle\frac{\partial\rho_{1}v_{1}}{\partial
t}+\frac{\partial}{\partial
x}\left\\{\rho_{1}v^{2}_{1}+p_{1}+\Pi_{1}\right\\}=\hat{m}_{1},$
$\displaystyle\frac{\partial}{\partial
t}\left(\rho_{1}v^{2}_{1}+2\rho_{1}\varepsilon_{1}\right)+\frac{\partial}{\partial
x}\left\\{\left[\rho
v^{2}_{1}+2\rho_{1}\varepsilon_{1}+2p_{1}+2\Pi_{1}\right]v_{1}\right\\}=2(\hat{e}_{1}+\hat{m}_{1}v),$
$\displaystyle\frac{\partial}{\partial
t}\left\\{\rho_{1}(v_{1})^{2}+3(p_{1}+\Pi_{1})\right\\}+\frac{\partial}{\partial
x}\left\\{\left[\rho_{1}v_{1}^{2}+5(p_{1}+\Pi_{1})\right]v_{1}\right\\}=-\frac{3\Pi_{1}}{\tau_{1}}+2(\hat{\omega}_{1}+\hat{m}_{1}v),$

where the global quantities: mass density $\rho$, pressure $p$, specific
internal energy $\varepsilon$, shear stress $\sigma$, dynamic pressure $\Pi$,
heat flux $q$, and a new extra variable $Q$ are given by

$\displaystyle\rho=\sum_{\alpha=1}^{2}\rho_{\alpha},\qquad
p=\sum_{\alpha=1}^{2}p_{\alpha},$
$\displaystyle\rho\varepsilon=\rho\varepsilon_{\rm
int}+\frac{1}{2}\sum_{\alpha=1}^{2}\rho_{\alpha}u_{\alpha}^{2},\quad\left(\varepsilon_{\rm
int}=\frac{1}{\rho}\sum_{\alpha=1}^{2}\rho_{\alpha}\varepsilon_{\alpha}\right),$
$\displaystyle\sigma={-\frac{2}{3}\sum_{\alpha=1}^{2}\rho_{\alpha}u_{\alpha}^{2}}=-\frac{2}{3}\frac{\rho_{1}\rho}{\rho_{2}}u_{1}^{2},$
$\displaystyle\Pi=\sum_{\alpha=1}^{2}\left(\Pi_{\alpha}+\frac{1}{3}\rho_{\alpha}u_{\alpha}^{2}\right)=\Pi_{1}+\Pi_{2}-\frac{\sigma}{2},$
$\displaystyle
q=\sum_{\alpha=1}^{2}\left(\frac{1}{2}\rho_{\alpha}u_{\alpha}^{2}+\rho_{\alpha}\varepsilon_{\alpha}+p_{\alpha}+\Pi_{\alpha}\right)u_{\alpha},$
$\displaystyle
Q=\sum_{\alpha=1}^{2}\left\\{\frac{1}{2}\rho_{\alpha}u_{\alpha}^{2}+\frac{5}{2}\left(p_{\alpha}+\Pi_{\alpha}\right)\right\\}u_{\alpha}.$

Here $\varepsilon_{\rm int}$ represents the intrinsic part of the global-
specific internal energy. We note that the global shear stress and global heat
flux appear in the system (2.1) due to the diffusion velocity, even if we
neglect these dissipative quantities in the field equations for each
constituent.

In order to discuss the behavior of non-equilibrium average temperature
$\mathcal{T}$, we use the definition proposed in [33, 34, 35]. This average
temperature is introduced such that the intrinsic internal energy
$\varepsilon_{\mathrm{int}}$ of the multi-temperature mixture have the same
expression of a single-temperature:

$\displaystyle\rho\varepsilon_{\rm
int}=\rho_{1}\varepsilon_{1}(\rho_{1},\mathcal{T})+\rho_{2}\varepsilon_{2}(\rho_{2},\mathcal{T})=\rho_{1}\varepsilon_{1}(\rho_{1},T_{1})+\rho_{2}\varepsilon_{2}(\rho_{2},T_{2}).$

###  2.2 Equilibrium subsytem

The production terms $\hat{m}_{\alpha}$, $\hat{e}_{\alpha}$, and
$\hat{\omega}_{\alpha}$ vanish in equilibrium and from (2) and (3), we obtain
$v_{1}=v_{2}=v$, $T_{1}=T_{2}=T$, and $\Pi_{1}=\Pi_{2}=0$, where $T$ is the
average temperature in equilibrium. In a generic equilibrium state, the total
(equilibrium) pressure and the total internal intrinsic energy are the same
forms as in the case of a single-component gas

$p^{(\rm eq)}=\frac{k_{B}}{m}\rho T,\qquad\varepsilon^{(\rm eq)}_{\rm
int}=\frac{k_{B}}{m(\gamma-1)}T$

provided that we introduce following [36] an average mass $m\equiv m(c)$ and
an average ratio of the specific heats $\gamma\equiv\gamma(c)$

$\begin{split}\frac{1}{m}=\frac{c}{m_{1}}+\frac{1-c}{m_{2}},\quad\frac{1}{\gamma-1}=\frac{m}{m_{1}}\frac{c}{\gamma_{1}-1}+\frac{m}{m_{2}}\frac{1-c}{\gamma_{2}-1},\end{split}$
(6)

where $c=\rho_{1}/\rho\in]0,1[$ denoting the concentration.

The associate equilibrium subsystem of (2.1) according to the definition given
in [37] is given by

$\displaystyle\begin{split}&\frac{\partial\rho}{\partial t}+\frac{\partial\rho
v}{\partial x}=0,\\\ &\frac{\partial\rho v}{\partial
t}+\frac{\partial}{\partial x}\left(\rho v^{2}+p^{(\rm eq)}\right)=0,\\\
&\frac{\partial}{\partial t}\left(\frac{1}{2}\rho v^{2}+\rho\varepsilon^{(\rm
eq)}_{\rm int}\right)+\frac{\partial}{\partial x}\left\\{\left(\frac{1}{2}\rho
v^{2}+\rho\varepsilon^{(\rm eq)}_{\rm int}+p^{(\rm eq)}\right)v\right\\}=0,\\\
&\frac{\partial\rho_{1}}{\partial t}+\frac{\partial\rho_{1}v}{\partial
x}=0.\end{split}$ (7)

##  3 Shock Structure

The shock structure is a traveling wave solution with constant velocity $s$ in
which all components of the field depend on a single variable $\varphi=x-st$
connecting two constant equilibrium state $\mathbf{u}_{0}$ and
$\mathbf{u}_{\mathrm{I}}$ at $\varphi=\pm\infty$ respectively.

We can choose without loss of generality the unperturbed velocity $v_{0}=0$.
Taking into account the conservation laws, we have the following boundary
conditions

$\mathbf{u}_{0}=\left[\begin{array}[]{c}\rho_{0}\\\ v_{0}\\\ T_{0}\\\
\Pi_{0}\\\ \left(\rho_{1}\right)_{0}\\\ \left(v_{1}\right)_{0}\\\
\left(T_{1}\right)_{0}\\\ \left(\Pi_{1}\right)_{0}\\\
\end{array}\right],\quad\mathbf{u}_{\mathrm{I}}=\left[\begin{array}[]{c}\rho_{\mathrm{I}}\\\
v_{\mathrm{I}}\\\ T_{\mathrm{I}}\\\ \Pi_{\mathrm{I}}\\\
\left(\rho_{1}\right)_{\mathrm{I}}\\\ \left(v_{1}\right)_{\mathrm{I}}\\\
\left(T_{1}\right)_{\mathrm{I}}\\\ \left(\Pi_{1}\right)_{\mathrm{I}}\\\
\end{array}\right].$

In the unperturbed equilibrium state $\mathbf{u}_{0}$, we have:

$\left(\rho_{1}\right)_{0}=c_{0}\rho_{0},\quad\left(T_{1}\right)_{0}=T_{0},\quad
v_{0}=\left(v_{1}\right)_{0}=0,\quad\Pi_{0}=\left(\Pi_{1}\right)_{0}=0,$ (8)

and the perturbed equilibrium state $\mathbf{u}_{\mathrm{I}}$ is the solution
of the Rankine-Hugoniot equations of the equilibrium sub-system (7):

$\displaystyle\rho_{\mathrm{I}}=\frac{(\gamma_{0}+1)M_{0}^{2}}{2+(\gamma_{0}-1)M_{0}^{2}}\rho_{0},\quad
c_{\mathrm{I}}=c_{0},\quad\left(\rho_{1}\right)_{\mathrm{I}}=c_{0}\rho_{\mathrm{I}}$
$\displaystyle
T_{\mathrm{I}}={T_{1}}_{\mathrm{I}}=\frac{\\{2+(\gamma_{0}-1)M_{0}^{2}\\}\\{1+\gamma_{0}(2M_{0}^{2}-1)\\}}{(\gamma_{0}+1)^{2}M_{0}^{2}}T_{0},$
(9) $\displaystyle
v_{\mathrm{I}}={(v_{1})}_{\mathrm{I}}=\frac{2(M_{0}^{2}-1)}{(\gamma_{0}+1)M_{0}}a_{0},\quad\Pi_{\mathrm{I}}=\left(\Pi_{1}\right)_{\mathrm{I}}=0,$

where $\gamma_{0}$ is the function $\gamma$ given in (6) evaluated in $c_{0}$:
$\gamma_{0}=\gamma(c_{0})$. The unperturbed Mach number $M_{0}$ is defined by

$M_{0}=\frac{s-v_{0}}{a_{0}},$

with $a_{0}$ being the sound velocity in the unperturbed state

$a_{0}=\sqrt{\gamma_{0}\frac{k_{B}}{m_{0}}T_{0}},$

and $m_{0}=m(c_{0})$ being the equilibrium average mass of the mixture.

##  4 Regions classified by the possibility of sub-shock formation

Let us analyze whether the continuous shock-structure solution exists or not.
Without any loss of generality, hereafter, we assume $m_{1}\leq m_{2}$.
Following the paper [31], we will analyze the shock wave propagating in the
positive direction with respect to the fluid flow introducing the
characteristic velocities in an equilibrium state for constituents $1$ and $2$

$\lambda_{1}=v+\sqrt{\frac{5}{3}\frac{k_{\mathrm{B}}}{m_{1}}T},\quad\lambda_{2}=v+\sqrt{\frac{5}{3}\frac{k_{\mathrm{B}}}{m_{2}}T},$
(10)

and the characteristic velocities of the equilibrium subsystem

$\bar{\lambda}=v+\sqrt{\gamma\frac{k_{\mathrm{B}}}{m}T},$ (11)

with $m$ and $\gamma$ given in (6).

Let’s introduce the dimensionless characteristic velocities at the unperturbed
and perturbed states

$M_{10}\equiv\frac{\lambda_{10}}{a_{0}},\quad
M_{20}\equiv\frac{\lambda_{20}}{a_{0}},\quad
M_{\mathrm{1I}}\equiv\frac{\lambda_{\mathrm{1I}}}{a_{0}},\quad
M_{\mathrm{2I}}\equiv\frac{\lambda_{\mathrm{2I}}}{a_{0}},\quad$

where, taking into account (10), (11) and (8), (3) the explicit expressions of
these quantities are given by

$\displaystyle\begin{split}&M_{10}=\sqrt{\frac{5}{3\gamma_{0}\left\\{c_{0}+(1-c_{0})\mu\right\\}}},\quad
M_{20}=\sqrt{\frac{5\mu}{3\gamma_{0}\left\\{c_{0}+(1-c_{0})\mu\right\\}}},\\\
&M_{\mathrm{1I}}=\frac{2(M_{0}^{2}-1)}{(\gamma_{0}+1)M_{0}}+\sqrt{\frac{5\\{2+(\gamma_{0}-1)M_{0}^{2}\\}\\{1+\gamma_{0}(2M_{0}^{2}-1)\\}}{3\gamma_{0}(\gamma_{0}+1)^{2}M_{0}^{2}}\frac{m_{0}}{m_{1}}},\\\
&M_{\mathrm{2I}}=\frac{2(M_{0}^{2}-1)}{(\gamma_{0}+1)M_{0}}+\sqrt{\frac{5\\{2+(\gamma_{0}-1)M_{0}^{2}\\}\\{1+\gamma_{0}(2M_{0}^{2}-1)\\}}{3\gamma_{0}(\gamma_{0}+1)^{2}M_{0}^{2}}\frac{m_{0}}{m_{2}}},\end{split}$
(12)

with $\mu$ being the ratio of the masses of the constituents defined by

$\mu=\frac{m_{1}}{m_{2}},\quad 0<\mu\leq 1.$

The analogous expressions for a mixture of Eulerian gases are given in [31]
and indicated with an apex ${(E)}$ in the Figures 1-7.

In the previous paper [31], we have identified sufficient and necessary
conditions for the sub-shock formation:

According to the theorem given in [21], _the necessary and sufficient
condition for the sub-shock formation for the shock structure of the
constituent $1$ (light species) is given by _

$s>\lambda_{10}\quad\Leftrightarrow\quad M_{0}>M_{10}.$

While, _the necessary condition but in general not sufficient for the sub-
shock formation for the shock structure of constituent $2$ (heavy species) is
given by _

Case A) $\displaystyle\,\text{If}\quad\mu\leq{3\gamma_{1}}/{5},\text{and }$
$\displaystyle\text{when}\,\,\,0<c_{0}\leq c_{0}^{*},\,\,\text{for
}\,\,s>\lambda_{20}\quad\Leftrightarrow\quad\text{for }\,\,M_{0}>M_{20},$
$\displaystyle\text{when}\,\,\,c_{0}^{*}<c_{0}<1,\,\,\text{for
}\,\,s>s^{*},\,\,\text{solution of }\,\,\lambda_{\rm 2I}(s^{*})=s^{*},$
$\displaystyle\qquad\Leftrightarrow\quad\text{for
}\,\,M_{0}>M^{*}_{20},\,\,\text{solution of }\,\,M_{\rm
2I}(M^{*}_{20})=M^{*}_{20},$ (13)
$\displaystyle\text{where}\,\,{c}_{0}^{*}(\mu)\,\,\text{is the solution
of}\,\,\lambda_{20}=\bar{\lambda}_{0}\,\,\Leftrightarrow
M_{20}(c_{0}^{*},\mu)=1.$ Case B)
$\displaystyle\,\text{If}\quad\mu>{3\gamma_{1}}/{5},\,\,\text{for
}\,\,s>\lambda_{20}\,\Leftrightarrow\,\text{for }\,\,M_{0}>M_{20},\,\text{for
any}\,\,c_{0}\in\,]0,1[\,.$

###  4.1 Regions in Case A)

Case A) has four possible types of regions depending on $\mu$:

  * •

A1) in which $M^{*}_{20}$ intersects $M_{10}$ and has an asymptote like in the
example reported in Fig.1. Taking into account (12) this situation is possible
if

$0<\mu\leq\frac{3}{10}(\gamma_{1}-1).$

  * •

A2) in which $M^{*}_{20}$ intersects $M_{10}$ and doesn’t have an asymptote
like in the example reported in Fig.2. Taking into account (12) this situation
is possible if

$\frac{3}{10}(\gamma_{1}-1)<\mu<\frac{15-33\gamma_{1}}{15\gamma_{1}-65}.$

  * •

A3) in which $M^{*}_{20}$ does not intersect $M_{10}$ like in the example
reported in Fig.3. Taking into account (12) this situation is possible if

$\frac{15-33\gamma_{1}}{15\gamma_{1}-65}\leq\mu<\frac{3}{5}\gamma_{1}.$

  * •

A4) in which $M^{*}_{20}$ does not intersect $M_{10}$ and the curves of
$M_{20}$ and $M_{10}^{(E)}$ coincide with each other and $c_{0}^{*}=1$ like in
the example reported in Fig.4. Taking into account (12) this situation is
possible if

$\mu=\frac{3}{5}\gamma_{1}.$

Figure 1: Regions in the plane $(c_{0},M_{0})$ of possible sub-shocks in Case
A1 (right); $\gamma_{1}=7/5$, $\gamma_{2}=9/7$, and $\mu=0.1$. The vertical
dotted line represents the asymptote of $M^{*}_{20}$. The magnification of the
regions around $(c_{0},M_{0})=(0,1)$ is also shown (left).  Figure 2: Regions
in the plane $(c_{0},M_{0}$) of possible sub-shocks in Case A2;
$\gamma_{1}=7/5$, $\gamma_{2}=9/7$, and $\mu=0.4$. The circles represent the
parameters adopted for numerical calculations shown in Sec. 5.  Figure 3:
Regions classified by the possibility of the sub-shock formation in the
$M_{0}$ – $c_{0}$ plane in Case A3; $\gamma_{1}=7/6$, $\gamma_{2}=7/5$, and
$\mu=0.65$. Figure 4: The regions classified by the possibility of the sub-
shock formation in the $M_{0}$ – $c_{0}$ plane in Case A4; $\gamma_{1}=7/6$,
$\gamma_{2}=7/5$, and $\mu=0.7$. In this case, $c_{0}^{*}=1$.

###  4.2 Regions in Case B)

Case B) has three possible topologies of regions depending on $\mu$:

  * •

B1) in which $M_{20}$ is always greater than one in the whole range of
$0<c_{0}<1$, and therefore $M_{20}^{*}$ does not appear like in the example
reported in Fig.5. Taking into account (12), this situation is possible if

$\frac{3}{5}\gamma_{1}<\mu<\frac{\gamma_{1}}{\gamma_{2}}.$

  * •

B2) as in case B1) with $M^{(E)}_{10}=M^{(E)}_{20}$ like in the exapmle
reported in Fig. 6. Taking into account (12) and the results in [30], this
situation is possible if

$\mu=\frac{\gamma_{1}}{\gamma_{2}}.$

  * •

B3) as in case B1) but $M^{(E)}_{10}<M^{(E)}_{20}$ like in the exapmle
reported in Fig. 7. Taking into account (12) and the results in [30], this
situation is possible if

$\mu>\frac{\gamma_{1}}{\gamma_{2}}.$

Figure 5: The regions classified by the possibility of the sub-shock formation in the $M_{0}$ – $c_{0}$ plane in Case B1; $\gamma_{1}=7/6$, $\gamma_{2}=7/5$, and $\mu=0.75$. Figure 6: Regions classified by the possibility of the sub-shock formation in the $M_{0}$ – $c_{0}$ plane in Case B2; $\gamma_{1}=7/6$, $\gamma_{2}=7/5$, and $\mu=5/6$. Figure 7: The regions classified by the possibility of the sub-shock formation in the $M_{0}$ – $c_{0}$ plane in Case B3; $\gamma_{1}=7/6$, $\gamma_{2}=7/5$, and $\mu=0.9$. Table 1: Summary of the possibility of the sub-shock formation for the classified regions in the plane $(c_{0},M_{0}$). The symbols $S_{1}$ and $S_{2}$ represent the sub-shock for the shock structure of the constituent $1$ and $2$, respectively. The corresponding conclusions for a binary mixture of Eulerian gases are also shown in the columns $S_{1}^{(E)}$ and $S_{2}^{(E)}$.  Region | $S_{1}$ | $S_{2}$ | $S_{1}^{(E)}$ | $S_{2}^{(E)}$
---|---|---|---|---
1 | No | No | No | No
2 | No | No | No | Maybe
3 | No | No | No | Yes
4 | No | No | Maybe | No
5 | No | No | Maybe | Yes
6 | No | No | Yes | No
7 | No | No | Yes | Maybe
8 | No | No | Yes | Yes
9 | No | Maybe | No | No
10 | No | Maybe | No | Maybe
11 | No | Maybe | Maybe | Yes
12 | No | Maybe | Yes | No
13 | No | Maybe | Yes | Maybe
14 | No | Maybe | Yes | Yes
15 | Yes | No | Yes | No
16 | Yes | Maybe | Maybe | Yes
17 | Yes | Maybe | Yes | No
18 | Yes | Maybe | Yes | Maybe
19 | Yes | Maybe | Yes | Yes

In Figures 1-7, we enumerate the different regions, and the conclusions
whether we can have or no sub-shock according to (4) are summarized in Table
1. The word “Maybe” indicates that the condition is necessary but not
sufficient for the existence of a sub-shock. In fact, the singularity is maybe
apparent due to an indeterminate form $0/0$. For more details, see [30, 31].

##  5 Numerical simulation of shock structure

In the present section, we solve the system of field equations numerically and
clarify the possibility of the sub-shock formation.

For convenience, we introduce the following dimensionless variables to scale
the state variables and the independent variable $\varphi$ with the values
evaluated in the unperturbed state:

$\displaystyle\hat{\rho}=\frac{\rho}{\rho_{0}},\quad\hat{v}=\frac{v}{a_{0}},\quad\hat{\mathcal{T}}=\frac{\mathcal{T}}{T_{0}},\quad\hat{\Pi}=\frac{\Pi}{\rho_{0}\frac{k_{B}}{m_{0}}T_{0}},$
$\displaystyle\hat{\varphi}=\frac{\varphi}{t_{c}\,a_{0}},\quad\hat{x}=\frac{x}{t_{c}\,a_{0}},\quad\hat{t}=\frac{t}{t_{c}},\quad\hat{\tau}=\frac{\tau}{t_{c}},$
$\displaystyle\hat{\psi}=\frac{t_{c}}{\rho_{0}T_{0}}\psi_{11},\quad\hat{\theta}=\frac{t_{c}}{\rho_{0}\frac{k_{B}}{m_{0}}T_{0}^{2}}\theta_{11},\quad\hat{\kappa}=\frac{t_{c}}{\rho_{0}\frac{k_{B}}{m_{0}}T_{0}^{2}}\kappa_{11},\quad\hat{\phi}=\frac{t_{c}}{\rho_{0}\frac{k_{B}}{m_{0}}T_{0}^{2}}\phi_{11},\quad$

where $t_{c}$ is an arbitrary characteristic time for numerical computations.

The stability conditions, namely, $\psi_{11}>0$ and that the matrix of (4) is
positive definite, can be written in terms of the dimensionless
phenomenological coefficients as follows:
$\hat{\psi}>0,\,\hat{\theta}>0,\,\hat{\theta}\,\hat{\phi}-\hat{\kappa}^{2}>0$.

As done previously in the literature [38, 39, 40, 24, 28, 30], we perform
numerical calculations on the Riemann problem consisting of two equilibrium
states $\mathbf{u}_{0}$ and $\mathbf{u}_{\rm I}$ satisfying (8) and (3) and
obtain the profiles of the physical quantities after a long time. Following an
idea of Liu [41], a conjecture about the large-time behavior of the Riemann
problem and the Riemann problem with structure [42, 43] for a system of
balance laws was proposed by Ruggeri and Coworkers [38, 44, 40]. According to
this conjecture, if the Riemann initial data correspond to a shock family
$\mathcal{S}$ of the equilibrium sub-system, for a large time, the solution of
the Riemann problem of the full system converges to the corresponding shock
structure.

We have developed numerical code adopting the Uniformly accurate Central
Scheme of order 2 (UCS2) [45] to solve the Riemann problem numerically. As we
do not know to refer in this paper to a particular real mixture of gases also
because it seems difficult to find experimental data for shock profile in a
mixture of polyatomic gases with large bulk viscosity, we only analyze the
qualitative behavior of shock and sub-shocks, and we chose for our numerical
experiments the following parameters:
$\hat{\psi}=\hat{\theta}=\hat{\kappa}=0.2$, $\hat{\phi}=0.3$,
$\hat{\tau}_{1}=1$, and $\hat{\tau}_{2}=2$.

We show the shock structure for several Mach numbers in the case of
$\gamma_{1}=7/5$, $\gamma_{2}=9/7$, $\mu=0.4$, and $c_{0}=0.1$ in Figure 8.
These parameters correspond to circles Nos. I – III in Figure 2. It is
confirmed that the shock-structure solution predicted by the RET6 theory is
continuous for $M_{0}=1.2$ while a sub-shock for the shock structure of the
constituent $2$ emerges for $M_{0}=1.3$. If we increase the Mach number more,
we observe another sub-shock for the shock structure of the constituent $1$ as
shown in Figure 8 for $M_{0}=1.9$.

For comparison, we also depicted the theoretical predictions by the Eulerian
theory with the same parameters. As expected, the shock structure predicted by
the RET6 theory is broader than the ones predicted by the Eulerian theory.
Moreover, although the necessary condition for the sub-shock formation for
constituent $2$ is satisfied in both theories for $M_{0}=1.2$, the sub-shock
is observed only in the Eulerian theory. These results indicate the
dissipative effect of the dynamic pressure on the shock structure and the sub-
shock formation.

Figure 8: Shock structure of the dimensionless global mass density, mixture
velocity, and average temperature (Solid curves) with $\gamma_{1}=7/5$,
$\gamma_{2}=9/7$, $\mu=0.4$, $c_{0}=0.1$ for several Mach numbers; $M_{0}=1.2$
indicated by circle No. I in Region $10$ in Figure 2 (top row), $M_{0}=1.3$
indicated by circle No. II in Region $10$ (middle row), and $M_{0}=1.9$
indicated by circle No. III in Region $18$ (bottom row). The corresponding
predictions by the theory of a mixture of Eulerian gases are also shown
(dashed curves).

Next, let us analyze the shock structure with $\gamma_{1}=7/5$,
$\gamma_{2}=9/7$, $\mu=0.4$, and $c_{0}=0.15$ for $M_{0}=1.06$ and for
$M_{0}=1.2$ shown in Figures 9. The parameters correspond to circles Nos. IV
and V in Figure 2. The result for $M_{0}=1.06$ is interesting because the
necessary condition $M_{0}>M_{20}$ is satisfied only in the RET6 theory, in
spite of the fact that the RET6 theory incorporates the dissipative effect.
Figure 9 shows that both theories predict continuous solutions without sub-
shock formation for $M_{0}=1.06$. If we increase the Mach number, the sub-
shock formation is observed in a mixture of the Eulerian gases immediately
after $M_{20}^{*(E)}$ while the RET6 theory predicts a continuous shock-
structure solution for $M_{0}=1.2$.

Figure 9: Shock structure predicted by the RET6 theory (Solid curves) and the
Eulerian theory (dashed curves) with $\gamma_{1}=7/5$, $\gamma_{2}=9/7$,
$\mu=0.4$, $c_{0}=0.15$ for $M_{0}=1.06$ indicated by circle No. IV in Region
$9$ (top row) and for $M_{0}=1.2$ indicated by circle No. V in Region $10$
(bottom row).

Similar situation is observed in the the shock structure with
$\gamma_{1}=7/5$, $\gamma_{2}=9/7$, $\mu=0.4$, and $c_{0}=0.3$ for $M_{0}=1.2$
and for $M_{0}=1.3$ shown in Figure 10. In these cases, parameters are
indicated by circles No. VI and VII in Figure 2. Although the necessary
condition $M_{0}>M_{20}^{*}$ is satisfied in the RET6 theory, the theoretical
predictions by the RET6 theory are continuous for both $M_{0}=1.2$ and
$M_{0}=1.3$. The sub-shock is observed only in the Eulerian theory for
$M_{0}=1.3$.

Figure 10: Shock structure predicted by the RET6 theory (Solid curves) and the
Eulerian theory (dashed curves) with $\gamma_{1}=7/5$, $\gamma_{2}=9/7$,
$\mu=0.4$, $c_{0}=0.3$ for $M_{0}=1.2$ indicated by circle No. VI in Region
$9$ in Figure 2 (top row), and for $M_{0}=1.3$ indicated by circle No. VII in
Region $10$ (bottom row).

Another example is the shock structure with $\gamma_{1}=7/5$,
$\gamma_{2}=9/7$, $\mu=0.4$, and $c_{0}=0.45$ for $M_{0}=1.21$ and for
$M_{0}=1.3$ shown in Figure 11. These parameters correspond to circles No.
VIII and IX in Figure 2. Again, the RET6 theory predicts continuous solutions
for both both $M_{0}=1.2$ and $M_{0}=1.3$ while the Eulerian theory predicts a
sub-shock $M_{0}=1.3$ in agreement with the theorem.

Figure 11: Shock structure predicted by the RET6 theory (Solid curves) and the
Eulerian theory (dashed curves) with $\gamma_{1}=7/5$, $\gamma_{2}=9/7$,
$\mu=0.4$, $c_{0}=0.45$ for $M_{0}=1.21$ indicated by circle No. VIII in
Region $9$ in Figure 2 (top row), and for $M_{0}=1.3$ indicated by circle No.
IX in Region $12$ (bottom row).

The last example is the shock structure with $\gamma_{1}=7/5$,
$\gamma_{2}=9/7$, $\mu=0.4$, and $c_{0}=0.7$ for $M_{0}=1.5$ and for
$M_{0}=1.7$ shown in Figure 12. These parameters are indicated by circles No.
X and XI in Figure 2. According to the theorem, a sub-shock for the shock
structure of the constituent $1$ arises in both theories. No sub-shock for the
shock structure of the constituent $2$ is observed in the prediction by both
theories for $M_{0}=1.5$, and multiple sub-shocks are predicted only in the
Eulerian theory for $M_{0}=1.7$.

Figure 12: Shock structure predicted by the RET6 theory (Solid curves) and the
Eulerian theory (dashed curves) with $\gamma_{1}=7/5$, $\gamma_{2}=9/7$,
$\mu=0.4$, $c_{0}=0.75$, and $M_{0}=1.5$ indicated by circle No. X in Region
$17$ in Figure 2 (top row), and for $M_{0}=1.7$ indicated by circle No. XI in
Region $18$ (bottom row).

##  6 Conclusions

In this paper, we have analyzed the shock structure in a binary mixture of
rarefied polyatomic gases based on rational extended thermodynamics. We have
used the system of the field equations derived in the context of the multi-
temperature model in which the dynamic pressure’s contribution is relevant. We
have classified the possible parameters by considering the necessary
conditions to observe the sub-shock formation and have tested the sub-shock
formation numerically. As we expect, the shock profile is more regularized
with respect to the mixture of Eulerian gas, and in the case that exists sub-
shocks, the amplitude of the sub-shock is reduced compared to those of non-
dissipative gases. We have also observed from the numerical simulation that
for very small Mach numbers in the regions in which the existence is only
necessary, the singularity is only apparent, while increasing the Mach number
always we have sub-shocks in this kind of region.

The present results are in agreement with the fact that the present theory
satisfies the so-called K-condition [29] that, together with the convexity of
entropy, take the system in the assumptions of general theorems for which if
the initial data are sufficiently small, there exist smooth global solutions
[46, 47, 31, 9, 48].

It remains open to compare with experimental data for mixtures in which
constituents have large bulk viscosity. However, at the moment, the authors
were not able to find experimental data on the shock-wave profiles for this
kind of mixture in the literature. Instead, there are several single gases
with this characteristic in which, as we say in precedence, the RET6 theory is
very successful.

## Acknowledgments

This paper is dedicated to Gerald Warnecke on the occasion of his 65th
birthday. The work has been partially supported by JSPS KAKENHI Grant Numbers
JP19K04204 (S.T.). The paper has also been carried out in the framework of the
activities of the Italian National Group of Mathematical Physics of the
Italian National Institute of High Mathematics GNFM/INdAM (T.R.).

## Compliance with Ethical Standards

### Funding

Partially supported by JSPS KAKENHI Grant Numbers JP19K04204 (S.T.)

### Conflict of Interest

On behalf of all authors, the corresponding author states that there is no
conflict of interest

### Ethical Approval in the manuscript

Not Applicable

## References

  * [1] W. G. Vincenti and C. H. Kruger, Jr, Introduction to Physical Gas Dynamics.  John Wiley and Sons, New York, London, Sydney, 1965.
  * [2] Y. B. Zel’dovich and Y. P. Raizer, Physics of Shock Waves and High-Temperature Hydrodynamic Phenomena.  Dover Publications, Mineola, New York, 2002.
  * [3] G. A. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows.  Oxford Univ Press, Oxford, 1994.
  * [4] V. Djordjić, M. Pavić-C̆olić, and M. Torrilhon, Consistent, explicit, and accessible Boltzmann collision operator for polyatomic gases, Physical Review E 104 (2021) 025309.
  * [5] I. M. Gamba and M. Pavić-C̆olić, On the Cauchy problem for Boltzmann equation modeling a polyatomic gas, J. Math. Phys. 64 (2023) 013303.
  * [6] M. Pirner, A review on BGK models for gas mixtures of mono and polyatomic molecules, Fluids 6 (2021) 393.
  * [7] I. Müller and T. Ruggeri, Rational Extended Thermodynamics.  Springer, New York, 1998.
  * [8] T. Ruggeri and M. Sugiyama, Rational Extended Thermodynamics beyond the Monatomic Gas.  Springer, Cham, 2015.
  * [9] T. Ruggeri and M. Sugiyama, Classical and Relativistic Rational Extended Thermodynamics of Gases.  Springer, Cham, 2021.
  * [10] S. Taniguchi, T. Arima, T. Ruggeri, and M. Sugiyama, Thermodynamic theory of the shock wave structure in a rarefied polyatomic gas: Beyond the bethe-teller theory, Phys. Rev. E 89 (Jan, 2014) 013025.
  * [11] S. Taniguchi, T. Arima, T. Ruggeri, and M. Sugiyama, Effect of the dynamic pressure on the shock wave structure in a rarefied polyatomic gas, Physics of Fluids 26 (2014) 016103.
  * [12] H. A. Bethe and E. Teller, Deviations from Thermal Equilibrium in Shock Waves.  reprinted by Engineering Research Institute. University of Michigan, Michigan, 1941.
  * [13] D. Gilbarg and D. Paolucci, The structure of shock waves in the continuum theory of fluids., J. Rat. Mech. Anal. 2 (1953) 617\.
  * [14] T. Arima, S. Taniguchi, T. Ruggeri, and M. Sugiyama, Extended thermodynamics of real gases with dynamic pressure: An extension of Meixner’s theory, Physics Letters A 376 (2012) 2799–2803.
  * [15] T. Arima, T. Ruggeri, M. Sugiyama, and S. Taniguchi, Non-linear extended thermodynamics of real gases with 6 fields, International Journal of Non-Linear Mechanics 72 (2015) 6–15.
  * [16] S. Taniguchi, T. Arima, T. Ruggeri, and M. Sugiyama, Overshoot of the non-equilibrium temperature in the shock wave structure of a rarefied polyatomic gas subject to the dynamic pressure, International Journal of Non-Linear Mechanics 79 (2016) 66–75.
  * [17] S. Kosuge, K. Aoki, and T. Goto, Shock wave structure in polyatomic gases: Numerical analysis using a model Boltzmann equation, AIP Conference Proceedings 1786 (2016) 180004.
  * [18] S. Kosuge and K. Aoki, Shock-wave structure for a polyatomic gas with large bulk viscosity, Phys. Rev. Fluids 3 (Feb, 2018) 023401\.
  * [19] T. Arima, T. Ruggeri, M. Sugiyama, and S. Taniguchi, Galilean invariance and entropy principle for a system of balance laws of mixture type, Atti Accad. Naz. Lincei Cl. Sci. Fis. Mat. Natur. 28 (2017) 66–75.
  * [20] T. Ruggeri, Breakdown of shock-wave-structure solutions, Phys. Rev. E 47 (Jun, 1993) 4135–4140.
  * [21] G. Boillat and T. Ruggeri, On the shock structure problem for hyperbolic system of balance laws and convex entropy, Continuum Mechanics and Thermodynamics 10 (Oct, 1998) 285–292.
  * [22] T. Ruggeri and S. Taniguchi, Shock waves in hyperbolic systems of nonequilibrium thermodynamics, in Applied Wave Mathematics II, A. Berezovski and T. Soomere, eds., Mathematics of Planet Earth, ch. 8, pp. 167–186.  Springer, Cham, 2019.
  * [23] W. Weiss, Continuous shock structure in extended thermodynamics, Phys. Rev. E 52 (Dec, 1995) R5760–R5763.
  * [24] S. Taniguchi and T. Ruggeri, On the sub-shock formation in extended thermodynamics, International Journal of Non-Linear Mechanics 99 (2018) 69–78.
  * [25] F. Conforto, A. Mentrelli, and T. Ruggeri, Shock structure and multiple sub-shocks in binary mixtures of Eulerian fluids, Ricerche di Matematica 66 (Jun, 2017) 221–231.
  * [26] M. Bisi, G. Martalò, and G. Spiga, Shock wave structure of multi-temperature Euler equations from kinetic theory for a binary mixture, Acta Applicandae Mathematicae 132 (Aug, 2014) 95–105.
  * [27] V. Artale, F. Conforto, G. Martalò, and A. Ricciardello, Shock structure and multiple sub-shocks in grad 10-moment binary mixtures of monoatomic gases, Ricerche di Matematica 68 (Dec, 2019) 485–502.
  * [28] S. Taniguchi and T. Ruggeri, A 2 $\times$ 2 simple model in which the sub-shock exists when the shock velocity is slower than the maximum characteristic velocity, Ricerche di Matematica 68 (Jun, 2019) 119–129.
  * [29] T. Ruggeri and S. Taniguchi, Sub-shock formation in shock structure of a binary mixture of polyatomic gases, Atti Accad. Naz. Lincei Cl. Sci. Fis. Mat. Natur. 32 (2021) 167–179.
  * [30] T. Ruggeri and S. Taniguchi, A complete classification of sub-shocks in the shock structure of a binary mixture of Eulerian gases with different degrees of freedom, Physics of Fluids 34 (2022) 066116.
  * [31] T. Ruggeri and S. Taniguchi, Shock structure and sub-shocks formation in a mixture of polyatomic gases with large bulk viscosity, Ricerche di Matematica, in press (2023) .
  * [32] D. Madjarević, T. Ruggeri, and S. Simić, Shock structure and temperature overshoot in macroscopic multi-temperature model of mixtures, Physics of Fluids 26 (2014) 106102.
  * [33] T. Ruggeri and S. Simić, Mixture of gases with multi-temperature: Identification of a macroscopic average temperature, in Memorie dell’Accademia delle Scienze, Lettere ed Arti di Napoli, Proceedings Mathematical Physics Models and Engineering Sciences, pp. 455–465.  2008\.  http://www.societanazionalescienzeletterearti.it/pdf/Memorie.
  * [34] H. Gouin and T. Ruggeri, Identification of an average temperature and a dynamical pressure in a multitemperature mixture of fluids, Phys. Rev. E 78 (Jul, 2008) 016303.
  * [35] T. Ruggeri and S. Simić, Average temperature and Maxwellian iteration in multitemperature mixtures of fluids, Phys. Rev. E 80 (Aug, 2009) 026317.
  * [36] T. Ruggeri and S. Simić, On the hyperbolic system of a mixture of Eulerian fluids: a comparison between single- and multi-temperature models, Mathematical Methods in the Applied Sciences 30 (2007) 827–849.
  * [37] G. Boillat and T. Ruggeri, Hyperbolic principal subsystems: Entropy convexity and subcharacteristic conditions, Archive for Rational Mechanics and Analysis 137 (Jun, 1997) 305–320.
  * [38] F. Brini and T. Ruggeri, On the Riemann problem in extended thermodynamics, in Proceedings of the 10th International Conference on Hyperbolic Problems (HYP2004), pp. 319–326.  Yokohama Publisher Inc., Yokohama, 2006.
  * [39] F. Brini and T. Ruggeri, The Riemann problem for a binary non-reacting mixture of Euler fluids, in Proceedings XII Int. Conference on Waves and Stability in Continuous Media, R. Monaco, S. Pennisi, S. Rionero, and T. Ruggeri, eds., pp. 102–108.  World Scientific, Singapore, 2004.
  * [40] A. Mentrelli and T. Ruggeri, Asymptotic behavior of Riemann and Riemann with structure problems for a 2$\times$2 hyperbolic dissipative system, Suppl. Rend. Circ. Mat. Palermo II 78 (2006) 201–225.
  * [41] T.-P. Liu, Nonlinear hyperbolic-dissipative partial differential equations, pp. 103–136.  Springer, Berlin, Heidelberg, 1996.
  * [42] T.-P. Liu, Linear and nonlinear large-time behavior of solutions of general systems of hyperbolic conservation laws, Communications on Pure and Applied Mathematics 30 (1977) 767–796.
  * [43] T.-P. Liu, Large-time behavior of solutions of initial and initial-boundary value problems of a general system of hyperbolic conservation laws, Communications in Mathematical Physics 55 (Jun, 1977) 163–177.
  * [44] F. Brini and T. Ruggeri, On the Riemann problem with structure in extended thermodynamics, Suppl. Rend. Circ. Mat. Palermo II 78 (2006) 31–43.
  * [45] S. F. Liotta, V. Romano, and G. Russo, Central schemes for balance laws of relaxation type, SIAM Journal on Numerical Analysis 38 (2001) 1337–1356.
  * [46] T. Ruggeri and D. Serre, Stability of constant equilibrium state for dissipative balance laws system with a convex entropy, Quart. Appl. Math. 62 (2004) 163–179.
  * [47] W.-A. Yong, Entropy and global existence for hyperbolic balance laws, Arch. Rational Mech. Anal. 172 (2004) 247–266.
  * [48] S. Bianchini, B. Hanouzet, and R. Natalini, Asymptotic behavior of smooth solutions for partially dissipative hyperbolic systems with a convex entropy, Comm. Pure Appl. Math. 60 (2007) 1559–1622.