# When Should the FDA Inspect Pharmaceutical Manufacturing Facilities to
Better Mitigate Drug Shortages?

Daniel Kosmas  Department of Mechanical and Industrial Engineering,
Northeastern University, Boston, MA 02115, USA Özlem Ergun  Department of
Mechanical and Industrial Engineering, Northeastern University, Boston, MA
02115, USA

###### Abstract

Problem Definition: Drug shortages have been a persistent problem in American
healthcare for decades, and the resulting lack of access to necessary drugs
has been disastrous to patient health. A majority of these shortages were
caused by quality issues related to problems in the manufacturing process.
More frequent inspections can help reduce quality concerns, but deciding when
to inspect is a complex problem; strict regulation enforcement can force low-
profit facilities to close due to excessive maintenance costs, while lax
enforcement allows for regulation violations to persist, both of which can
cause drug shortages.

Methodology/Results: We propose a novel model to assist the FDA in determining
when to inspect manufacturing facilities. We formulate this problem as a
finite-horizon partially observable Markov decision process (POMDP) based on
the classifications the FDA assigns to each facility after inspection, as well
two disruptive events: a manufacturing failure occurring or the facility
closing for non-mandatory maintenance. We theoretically show that this problem
can be reduced to only needing to consider whether or not to inspect
immediately, which is independent of the time horizon. We additionally
determine the sensitivity of the optimal inspection time on the penalty
incurred for an unexpected disruptive event occurring.

Managerial Implications: Our computational study demonstrates a quadratic
relationship between the relative difference in average value accumulated
between inspecting based on the optimal inspection time produced by our model
and inspecting based on the expected time to an unexpected disruptive event,
highlighting the importance of allocating more inspection resources to high-
risk facilities that produce drugs that highly impact public health. We
additionally find that optimal inspection time is more sensitive to changes in
the penalty incurred from a disruptive event occurring the longer it has been
since the last inspection.

##  1 Introduction

Drug shortages have plagued American health care for decades, with no signs of
improvement in sight (Tucker et al., 2020). Over 680 active drug shortages
were reported between 2011 and 2020 (U.S. Food & Drug Administration, 2020a).
The number of active drug shortages increased by 30% between 2021 and 2022,
reaching a 5-year record high of 295 active drug shortages at the end of 2022.
The average drug shortage lasts 1.5 years, and at least 15 critical drugs have
been in shortage for over a decade (U.S. Senate Committee on Homeland Security
& Governmental Affairs, 2023). Drug shortages are disastrous for patients and
health care providers, as a lack of access to necessary drugs can result in
medication errors and treatment delays or cancellations that can lead to
adverse patient health outcomes or even death (U.S. Senate Committee on
Homeland Security & Governmental Affairs, 2023; Ventola, 2011). In 2019, the
Food and Drug Administration (FDA) found that quality issues were the leading
cause of drug shortages, accounting for 62% of shortages (U.S. Food & Drug
Administration, 2019). These quality issues are often a result of
manufacturing problems in pharmaceutical manufacturing facilities (Francas et
al., 2023; Jensen et al., 2002).

Pharmaceutical manufacturing facilities can experience numerous problems that
lead to unexpected disruptions in the manufacturing process, such as quality
issues with ingredients, outdated manufacturing equipment and natural
disasters damaging the facilities. These disruptions are costly for both the
manufacturer, who is responsible for resolving these regulatory issues in
order to resume production, and the consumers, who receive a lower quality
product or no product at all (U.S. Food & Drug Administration, 2019). The true
cost and length of an unexpected disruption are often incredibly difficult to
predict, making it difficult for the FDA to determine if they need to
intervene (National Academies of Sciences, Engineering, and Medicine, 2022).
In the worst case scenario, the capacity reduction caused by the unexpected
disruption can result in a drug shortage (U.S. Food & Drug Administration,
2019). Since facilities are only required to report when production is
interrupted or permanently discontinued, inspections are essential to ensuring
that risk factors that could lead to disruptions are appropriately mitigated
(Ergun et al., 2022; U.S. Food & Drug Administration, 2020c; National
Academies of Sciences, Engineering, and Medicine, 2022). Through routine
inspections, the FDA can help prevent unexpected disruptions from occurring,
whether it be from preventing low quality products entering the market that
may later need to be recalled or from identifying issues in the production
line that may result in the facility needing to shut down for maintenance
(Anand et al., 2012).

If the FDA inspects a facility and finds that production conditions are not in
line with the current regulations, they are equipped with various regulatory
tools to encourage companies to address these concerns (Woodcock, 2019).
However, the decision to ramp down production when issues are identified is
not without drawbacks. If the FDA is too penalizing with their decisions or
the manufacturing process issues are too costly for the manufacturer to
address, the facility may be forced to stop production for maintenance or
decide to discontinue production of the product altogether, which is another
disruptive event that reduces production capacity. This may unintentionally
result in a shortage due to a number of factors, such as the combination of a
sudden ramp down in production and limited FDA resources delaying re-
inspection or other manufacturers of similar products being unable to ramp up
their production to account for the facility’s closure (Kimball, 2023;
Ventola, 2011; Woodcock & Kopcha, 2020). However, delayed re-inspections are
also problematic when managing unexpected disruptions, so it is better for the
FDA to be aware of the severity of the facility’s problems. If the FDA is
aware of the severity of the facility’s problems, they can take action to
mitigate the impact of the disruption they cause, such as working with the
facility to address their concerns without fully halting production or
expediting the process of enabling new manufacturers to enter the market
(Jensen et al., 2002; Kimball, 2023; Ventola, 2011). Additionally, it is
better for the FDA to force a facility to halt production if the defective
product can negatively impact patient health (Buckley et al., 2013; Pew Health
Group, 2012; White, 2020). A product negatively impacting patient heath is
especially a concern when it has few (or no) substitute-able alternatives
since the lack of alternatives forces patients to use a product that may be
more harmful than not receiving treatment at all (Edney, 2019; Wang et al.,
2023). Henceforth, we distinguish between a disruption caused by the FDA
inspection resulting in the facility needing to close for mandatory
maintenance and a disruption caused by an event that the FDA is not aware of,
such as manufacturing failures and non-mandatory maintenance, which we refer
to as an _unexpected disruptive event_.

If the FDA does not inspect frequently enough, manufacturers have less
incentive to maintain high quality standards (Wu & Wang, 2020). Declining
quality can lead to products that need to be recalled, which, if caught, can
cause shortages if there are few (or no) alternative products. If not caught,
the product can negatively impact patient health. Thus, the FDA must strike a
careful balance of inspecting frequently enough to incentivize pharmaceutical
manufacturing facilities to maintain high production quality and identify
production issues before they can cause a shortage, while not inspecting too
frequently, wasting their resources and potentially causing shortages through
enforcing regulatory requirements so strictly that the manufacturer deems
production to be unprofitable. The FDA must additionally balance how
inspection resources are allocated across all pharmaceutical manufacturing
facilities, forcing the FDA to identify which facilities are more critical for
public health, and thus need more resources allocated to monitoring them.

We propose a finite horizon partially observable Markov decision process
(POMDP) (Krishnamurthy, 2016) to help support the FDA in determining when to
inspect pharmaceutical manufacturing facilities that balances the reward of
allowing a facility to continue production uninhibited with the risks of
unexpected disruptive events. We analytically show that the optimal inspection
time suggested by our model can be deduced by simple computations each time
period. We additionally analyze how the sensitivity in the severity of the
impact caused by an unexpected disruptive events will change the optimal
inspection time.

The remainder of this paper is organized as follows. Section 2 reviews
relevant literature around applying tools from operations research to
decision-making for inspections. Section 3 describes the model of determining
how much time should pass between sequential inspections of manufacturing
facilities, and derives closed-form equations for the value functions of the
POMDP. Section 4 demonstrates theoretical properties of this model that allow
us to determine the optimal inspection time only by determining if we should
inspect now or wait one time period, and then analyzes the sensitivity of the
optimal inspection time suggested by the model on the penalties incurred by
the facility experiencing an unexpected disruptive event. Section 5 proposes a
variant to this model to account for the outcome of the inspection,
additionally resulting in a penalty for the inspector if the facility needs to
close for mandatory maintenance. Section 6 computationally explores the trade-
offs of different inspection rules for varying transition matrices and shows
numerically the sensitivity of the optimal inspection time and total value
accumulated on various parameters. Section 7 concludes the paper and proposes
avenues for future research.

##  2 Literature Review

The FDA inspects pharmaceutical manufacturing facilities to verify compliance
with applicable laws and regulations, known as the current good manufacturing
practices (CGMP) (U.S. Food & Drug Administration, 2023). When the FDA
inspects a facility, they assign one of three classifications: No Action
Indicated, Voluntary Action Indicated, or Official Action Indicated. No Action
Indicated means that no violations of regulations were identified during the
inspection. Voluntary Action Indicated means that the inspector has identified
violations, but the FDA chooses to not take regulatory action. Official Action
Indicated means that the violations identified are severe enough for the FDA
to take regulatory action (U.S. Food & Drug Administration, 2020b). Under the
Federal Food, Drug and Cosmetic Act, the FDA was required to inspect
pharmaceutical manufacturing facilities at least once every two years.
However, the FDA struggled to meet these requirements for all facilities due
to limited resources (Woodcock, 2019). In 2005, a risk-based site-selection
model (SSM) was developed by the Center for Drug Evaluation and Research
(CDER) to help support the FDA’s decision-making around determining which
facilities to inspect (National Academies of Sciences, Engineering, and
Medicine, 2022; Woodcock, 2019). The SSM incorporates factors such as facility
type, time since last inspection, compliance history and inherent product risk
to determine a “risk score” for each facility, which helps the FDA determine
which facilities are more in need of being inspected. While the specifics of
how the scores are determined is not publicly known, it is known that these
risk scores are based on factors intrinsic to the facility and product(s)
being produced and do not account for how a disruption would impact public
health (Center for Drug Evaluation and Research, 2018). The two-year minimum
requirement was replaced in the Food and Drug Administration Safety and
Innovation Act (FDASIA) of 2012. Under FDASIA, the inspection requirement was
expanded to include both domestic and foreign pharmaceutical manufacturing
facilities, but the fixed minimum inspection interval was fully removed,
leaving inspection scheduling solely dictated by the SSM (Woodcock, 2019). Our
analytical results allow us to directly supplement the SSM since the optimal
inspection time suggested by our model can be deduced by simple computations
each time period. These computations can be converted into a score which can
be integrated into the SSM.

There has been some quantitative work to support the FDA in determining when
to inspect different pharmaceutical manufacturing facilities. Klimberg et al.
(1992) developed an integer program where facilities are either inspected in
year one or year two, and the set of facilities is partitioned into these two
sets. Macher et al. (2006) investigated how different factors, such as days
between inspection, compliance history, and the quality of training of the
inspectors, impact the classification a facility receives. Similarly,
Pazhayattil et al. (2020) explored how violations of different sections of the
CGMP were correlated with the inspection classifications. Hein et al. (2019)
compared various data-driven methods to predict the classification facilities
would receive. Ahuja et al. (2021) used data-driven methods to determine the
safety of pharmaceutical products, helping determine if FDA intervention for
specific products is necessary. Anand et al. (2012) demonstrated how FDA
inspections act as an inhibitor to operational decay. They were also
proponents of implementing targeting high-risk facilities for inspection. Ball
et al. (2017) showed that inspection frequency is predictive of future
facility quality and highlighted that inspector complacency can be detrimental
ensuring that inspections enforce quality standards. Wu & Wang (2020) explore
the impact of FDA surveillance inspections on domestic pharmaceutical
manufacturing facilities producing generic drugs. They suggest that more
frequent monitoring of facilities improved product quality, subject to
diminishing returns. Their analysis found that more frequent inspections were
most beneficial in reducing recalls for products from facilities that were
previously inspected less than once every two years. They additionally note
that cost concerns can cause high-risk facilities to close, rather than
perform the necessary maintenance to comply with the CGMP, suggesting that the
FDA must consider how manufacturers will react to inspections. Zhang et al.
(2019) proposed a game theoretic model to determine how strictly a supervisor
should monitor a pharmaceutical manufacturing facility to prevent drug fraud.
While pharmaceutical inspection has received some academic attention, there
has been more work on supporting the FDA in inspecting food manufacturing
facilities which have focused more on data-driven models for predicting
inspection results and determining how increased inspection improved self-
regulation amongst food vendors (Babich & Tang, 2012; Chang et al., 2017; Jin
et al., 2021; Levi et al., 2019; Zhou et al., 2022). Our work differs from
past work on FDA inspection by analytically determining when a facility should
be inspected, as opposed to using data-driven methods or determining a rate of
inspection frequency. Our model may also be applied to the inspection of food
manufacturing facilities as well, as FDA uses the same classification system
to determine their compliance with relevant laws and regulations.

While quantitative models supporting FDA inspection decision making has
primarily focused on game theoretic models or data driven models, determining
the optimal inspection and maintenance time for general maintenance has also
been well studied through the lens of condition-based maintenance models
(Ahmad & Kamaruddin, 2012; Zhu & Xiang, 2021). Condition-based maintenance has
been popularized for its ability to react to the observed conditions of the
system, preventing system failure when degradation is observed sooner than
expected. Markov decision processes (MDPs) have commonly been leveraged for
condition-based maintenance due to their ability to characterize multiple
maintenance actions and multiple degradation states (Papakonstantinou &
Shinozuka, 2014b, c). In particular, POMDPs have been utilized in the optimal
maintenance of numerous systems (eg., (Byon et al., 2010; Ge et al., 2007;
Ghandali et al., 2020; Papakonstantinou & Shinozuka, 2014a; Zhang & Revie,
2017; Zouch et al., 2011)). More recently, Morato et al. (2022) proposed
integrating Bayesian networks with POMDPs to provide a joint framework for
inspection and maintenance planning. Xu et al. (2022) introduces risk-averse
variants to the traditional MDP model for maintenance, and proves that there
exists an optimal maintenance policy when optimizing Value-at-Risk (VaR) and
Conditional Value-at-Risk (CVaR). Liu et al. (2023) proposed a framework for
the sequential inspection and maintenance of multi-state systems, where a
fixed number of inspection or maintenance actions can be taken each break and
the actions are implemented sequentially, allowing for the decisions to be
updated as inspections occur. Our problem differs from traditional inspection
and maintenance problems because the inspector does not have the capabilities
to perform the maintenance needed. As such, the inspector can only plan when
they will inspect and try to account for whether the outcome of the inspection
will determine that maintenance is needed. Additionally, most work on applying
POMDPs to maintenance and inspection focuses on numerical experiments. We
additionally provide a theoretical analysis of our problem to determine
optimal inspection time without the need of more complex solvers.

Our problem is also related to the single-component mission abort problem
(Levitin et al., 2019). The goal of mission abort problems is to determine if
a degrading system is able to complete its mission before failure occurs, and
if not, when to terminate the mission. The operator only receives a reward if
the mission is successfully completed. If the operator chooses to abort the
mission, the penalty of failure is less than that of if the system fails. In
this problem, the system is observed at regular time intervals. The state of
the system is healthy, unhealthy, or failed. However, when inspected, the only
information about the state that is revealed is whether or not it is
operational, leaving ambiguity between whether or not the system is healthy or
unhealthy. This is congruent with current regulations for pharmaceutical
manufacturing facilities; no information regarding the quality of these
facilities is shared with FDA unless a manufacturing failure has occurred
(National Academies of Sciences, Engineering, and Medicine, 2022). Typically,
if some issue has come to the attention of the FDA, they will conduct a “for-
cause” inspection to immediately investigate the severity of the issue (U.S.
Food & Drug Administration, 2018). Myers (2009) was the first to study the
mission abort problem, which considers a multi-component system. Later,
Levitin et al. (2019) explored the single-component mission abort problem,
where they model the mission abort as the initiation of the rescue operation
to prevent the destruction of the system. Qiu & Cui (2019) derives
degradation-based and duration-based abort policies for systems that degrade
according to a Gamma process. Qiu et al. (2023) formulates the mission abort
problem as a stochastic dynamic program and compares the effectiveness of
mission abort decisions against various heuristic measures, such as the length
of time spent in the unhealthy state. They numerically show that heuristic
abort strategies were less economically efficient then strategies provided by
solving the stochastic dynamic program.

To the best of our knowledge, very few works formulate the mission abort
problem as a MDP. Zhao et al. (2021) determines the optimal mission abort time
for a system that degrades according to a stationary Gamma process. This
degradation process is then converted to a continuous-state MDP. Cheng et al.
(2023) utilizes a hidden Markov model to model the system’s degradation, which
is then transformed into a continuous-state MDP. Both of these works solve
their models via value iteration (Krishnamurthy, 2016), whereas we derive a
closed-form equation for our value functions. Moreover, our results allow us
to perform simple calculations to determine the optimal inspection time. Our
problem differs from existing literature on MDP formulations of the mission
abort problems in the following ways. First, these models only receive a
reward if the mission is fully completed, whereas we accumulate value each
time period the manufacturing facility is operating. Second, while some works
generalize the number of quality states the system has, previous works on
mission abort problems only have theoretical results for two operational
states and one absorbing state. The model we consider has three operational
states and two absorbing states, and we provide theoretical results on the
optimal inspection time for our model.

##  3 POMDP Model

We consider a finite time POMDP model to determine when the FDA should inspect
a pharmaceutical manufacturing facility. We focus on determining when to
inspect a single facility independently of other facilities since the SSM
generates scores for each individual facility based on risk factors intrinsic
to that facility and does not consider them interdependently (Center for Drug
Evaluation and Research, 2018). In our model, the FDA must inspect a facility
by a pre-specified time. While in reality the formal requirement to inspect in
by a certain time has been removed, it is reasonable to assume that the FDA
would not allow a facility to go uninspected after a certain amount of time
has passed. We consider two unexpected disruptive events in our model: a
manufacturing failure occurring and the facility closing for non-mandatory
maintenance. A manufacturing failure is an unexpected event that results in a
reduction or elimination of production capacity due to issues in the
manufacturing process, while a facility closing to perform non-mandatory
maintenance is a proactive event by the facility to shut down production for a
set timeline, but the details of this event are unknown to the FDA. While we
only consider two unexpected disruptive events, this model can easily be
generalized to multiple unexpected disruptive events.

In each time period, the facility may be in operation, may have closed due to
a manufacturing failure, or may have closed to perform non-mandatory
maintenance. The FDA receives a reward if the facility is in operation.
Otherwise, the FDA incurs a large penalty if the facility has experienced a
manufacturing failure, or a small penalty if the facility closes for non-
mandatory maintenance. These penalties reflect the impact of facility no
longer manufacturing their pharmaceutical product(s) on social well-being.
While a facility undergoing maintenance is positive in long-term planning, the
immediate halt in production without prior knowledge by the FDA is disruptive
and can potentially cause a shortage. We assume that closing for non-mandatory
maintenance has a smaller penalty than closing due to a manufacturing failure
because the true impact and timeline of a manufacturing failure is difficult
to predict, while non-mandatory maintenance will have a more consistent
timeline.

The FDA accrues rewards until they decide to inspect (or must inspect if the
end of the time horizon is reached) or one of the disruptive events occurs.
The goal of the FDA is to maximize the total reward accrued. We first consider
a model where there is no penalty for inspecting, regardless of quality of the
facility. We also consider a model where inspecting can also result in the
facility closing for mandatory maintenance. As with closing for non-mandatory
maintenance, closing for mandatory maintenance as a result of an FDA
inspection can potentially cause a shortage. However, in this case the FDA
will be aware that the closure will be occurring, allowing them to better
mitigate the impact of the closure. As opposed to defining the states of the
POMDP to be arbitrary quality levels of the facility, then determining what
classification the FDA should assign to each quality level, we instead define
the states to be the classifications that the FDA will give after inspecting a
facility with a certain quality level. Since the FDA cannot directly control
the quality levels of the facility, their primary concern is identifying the
classification the facility should receive. This assumes that facility quality
levels are able to be coherently clustered into the classification system, and
this clustering is not influenced by external factors.

In each time period the facility is in operations, the FDA must decide whether
or not they will inspect the facility at the end of the time period. In order
to determine when is best to inspect, the FDA maintains their private belief
about how likely the facility is to receive each classification rating in each
time period. If they do not inspect, there is a chance that an unexpected
disruptive event will occur in the next time period. When they inspect, the
true classification of the facility is revealed and no future reward is
received. We assume that, when the FDA decides to inspect, they observe the
quality of the facility accurately and will assign the appropriate
classification to a facility based on the conditions it is in, regardless of
the implications on public health. Outside of a manufacturing failure
occurring or inspecting, the FDA receives no information on the status of the
facility (National Academies of Sciences, Engineering, and Medicine, 2022).

We now mathematically define our model; all notation is summarized in Appendix
A. Given a finite horizon $T$, the FDA must choose when to inspect the
facility. They may do so any time before $T$, but they are required to inspect
the facility by $T$. Once an inspection occurs, the time requirement on
inspection resets. Thus, the FDA choosing to inspect the facility acts as the
end of the game. Additionally, if an unexpected disruptive event occurs, the
FDA receives the appropriate penalty for not inspecting before the event
occurred, and then must inspect upon the facility reopening. Since we are only
considering the length of time between sequential inspections for a single
facility, we only need to consider whether an unexpected disruptive event has
occurred or not. Additionally, since the time requirement on inspection resets
after each inspection, we only need to model the FDA’s decision making when
they should next inspect, and do not need to determine the long-term decision
making over an infinite time horizon. When the FDA inspects the facility, they
would gain the information necessary to update model parameters as appropriate
before restarting the model.

The classifications the FDA assigns serve as the basis for the states of the
Markov chain: No Action Indicated is $N$, Voluntary Action Indicated is $V$,
and Official Action Indicated is $O$. We additionally include states
indicating that the facility has experienced a manufacturing failure ($D$),
has closed for non-mandatory maintenance ($C$) or has been inspected ($I$).
Each period, the FDA can receive one of three observations regarding facility
quality. If the facility experiences an unexpected disruptive event, it must
be reported to the FDA (U.S. Food & Drug Administration, 2020c). If the
facility has experienced a manufacturing failure, the FDA observes that a
manufacturing failure has occurred $(dr)$, and if the facility has closed for
non-mandatory maintenance, the FDA observes that the facility is closing for
non-mandatory maintenance $(cr)$, indicating that they know if the facility is
in states $D$ or $C$ with full certainty. Otherwise, the FDA observes no
reported issues from the facility, $(nr)$, indicating that the facility is
still operating. The FDA receives a reward $r$ each period that the facility
is successfully producing. If the facility experiences a manufacturing
failure, then the FDA receives a penalty of $-d$. If the facility closes for
non-mandatory maintenance, then the FDA receives a penalty of $-c$. This
penalty is to reflect that the facility is no longer producing, which may
cause a shortage. After the facility has been inspected, the FDA receives a
reward of $0$, indicating that the facility has been inspected before an
unexpected disruptive event occurred and the game is over. The focus of this
initial model is on inspecting before an unexpected disruptive event occurs,
and does not account for the outcome of the inspection. This model represents
the ideal situation for the FDA: where inspection focuses on identifying
whether or not a facility is in compliance with the CGMP, and does not need to
consider how the outcome of the inspection can negatively impact public
health. We later extend our model to account for the outcome of the inspection
in Section 5. For simplicity, we set $r=1$ and re-scale other parameters to
reflect this.

At each state, the FDA can either choose to inspect $(i)$ or not inspect
$(ni)$. We assume the cost of inspection has already been budgeted for, and
thus do not include it in our model. Figure 1 presents the state space of the
Markov chain, as well as the transition between states. Solid arcs represent
the transitions that can occur when the FDA does not inspect, while dashed
arcs represent transitions that occur when the FDA does inspect.

Figure 1: State space of the Markov chain

If the FDA does not inspect, the quality of the facility can degrade, with the
following transition probabilities:

$P(ni)=\left[\begin{array}[]{c|cccccc}&N&V&O&D&C&I\\\ \hline\cr
N&p_{NN}&p_{NV}&p_{NO}&p_{ND}&p_{NC}&0\\\ V&0&p_{VV}&p_{VO}&p_{VD}&p_{VC}&0\\\
O&0&0&p_{OO}&p_{OD}&p_{OC}&0\\\ D&0&0&0&1&0&0\\\ C&0&0&0&0&1&0\\\
I&0&0&0&0&0&1\end{array}\right]$

Once the FDA has inspected, the game is over and no more reward can be
collected. Thus, inspecting at any state enforces that the next state will be
$I$, which is an absorbing state, i.e., $P(i)_{SI}=1$ for all $S$ and
$P(i)_{SS^{\prime}}=0$ for $S^{\prime}\neq I$.

In each time period, the FDA maintains their private belief about how likely
it is for the facility to be in each state. Let
$b^{t}=(b^{t}_{N},b^{t}_{V},b^{t}_{O},b^{t}_{D},b^{t}_{C},b^{t}_{I})$ be the
vector indicating the belief probabilities of being in each state at time
period $t$. Note that, based on our assumptions, we can classify these belief
vectors into four categories. Once the facility has been inspected, we have
that $b^{t}=(0,0,0,0,0,0,1)$. Likewise, if the facility has experienced a
manufacturing failure, we have that $b^{t}=(0,0,0,1,0,0)$, and if the facility
has closed for non-mandatory maintenance, $b^{t}=(0,0,0,0,1,0)$. If no
unexpected disruptive event has been reported and we have not inspected, then
the facility is still manufacturing their product, and thus
$b^{t}=(b^{t}_{N},b^{t}_{V},b^{t}_{O},0,0,0)$.

We model the FDA determining when they should choose to inspect a facility as
them inspecting in the time period that would maximize the expected rewards
accrued over the entire time horizon. As opposed to the traditional
stochastic-dynamic formulation of the value functions, we focus on the
conditional plan formulation. This choice allows us to compute the expected
values of sub-optimal decisions, as opposed to just determining that those
decisions are sub-optimal. We can reasonably generate all conditional plans
since choosing to inspect and two of the three observations we can receive
after deciding to not inspect result in the game ending (i.e., no future
decision needs to be made in response to those observations). Thus, all
conditional plans can be described by when the FDA chooses to inspect the
facility or is forced to inspect after an unexpected disruptive event has
occurred. Let $\sigma_{j}^{t}$ be the conditional plan starting at time $t$
where we inspect after $j$ periods of the facility being in operation, where
$0\leq j\leq T-t$. To be clear, $\sigma_{0}^{t}$ inspects immediately and
$\sigma_{T-t}^{t}$ does not inspect the facility until the end of the time
horizon. Similarly, we can describe a conditional plan as the sequence of
actions taken in each time period, i.e., $(a_{1},\ldots,a_{T})$, where $a_{t}$
is the action taken at time $t$. Since no actions taken after an inspection
occurs, we can reduce this notation to
$(a_{1},\ldots,a_{k-1},a_{k})=(ni,\ldots,ni,i)$, where $k$ is the time period
when we decide to inspect. For example, the conditional plan $(i)$ is the plan
where we inspect immediately (i.e., $\sigma^{t}_{0}$) and the conditional plan
$(ni,i)$ is the plan where we wait a single time period before inspecting.

We now demonstrate how the belief probabilities will update over time in the
event that the FDA observes $(nr)$. To do so, we first determine the
probabilities of each observation occurring in each time period. Let
$P[(dr)]^{t}$, $P[(cr)]^{t}$, and $P[(nr)]^{t}$, be the probabilities that we
believe we will observe $(dr)$, $(cr)$ and $(nr)$, respectively, in time
period $t$. We can compute these probabilities as

$\displaystyle
P[(dr)]^{t}=p_{ND}b_{N}^{t-1}+p_{VD}b_{V}^{t-1}+p_{OD}b_{O}^{t-1}\text{,}$
$\displaystyle
P[(cr)]^{t}=p_{NC}b_{N}^{t-1}+p_{VC}b_{V}^{t-1}+p_{OC}b_{O}^{t-1}\text{, and}$
$\displaystyle
P[(nr)]^{t}=(p_{NO}+p_{NV}+p_{NN})b_{N}^{t-1}+(p_{VO}+p_{VV})b_{V}^{t-1}+p_{OO}b_{O}^{t-1}\text{.}$

Thus, in the event that the FDA observes $(nr)$, the belief probabilities
update as

$b^{t+1}_{N}=\frac{p_{NN}b^{t}_{N}}{P[(nr)]^{t+1}}\text{,
}b^{t+1}_{V}=\frac{p_{NV}b^{t}_{N}+p_{VV}b^{t}_{V}}{P[(nr)]^{t+1}}\text{, and
}b^{t+1}_{O}=\frac{p_{NO}b^{t}_{N}+p_{VO}b^{t}_{V}+p_{OO}b^{t}_{O}}{P[(nr)]^{t+1}}\text{.}$

We next construct the value functions of the model. Let
$V^{t}(b^{t},\sigma^{t}_{j})$ to be the expected reward accumulated from time
$t$ to $T$ given the current belief vector $b^{t}$ and implementing
conditional plan $\sigma^{t}_{j}$. Value functions are defined as the sum of
the expected reward for the state the FDA believes they are in at time $t$ and
expected future rewards from time periods $t+1$ to $T$ following the actions
dictated in $\sigma^{t}_{j}$. We start by defining the value functions that
represent the end of the game. Recall that, if the facility has already been
inspected, i.e., it is in state $I$, then the game is over and the FDA
receives a reward of zero. We define $V^{t}((0,0,0,0,0,0,1),\sigma^{t}_{j})=0$
for any $\sigma^{t}_{j}$. The FDA receives a reward of $1$ if the facility is
in state $N$, $V$ or $O$, a reward of of $-d$ if the facility is in state $D$,
and a reward of $-c$ if the facility is in state $C$. Note that, no reward is
collected after an inspection has occurred, and inspecting incurs no cost. To
match our assumption that a penalty for an unexpected disruptive event is only
incurred once, if the facility is in state $D$ or $C$, it will then be optimal
to “inspect” to transition the Markov chain to state $I$, which indicates the
game is over. This likewise reflects that a facility must be re-inspected by
the FDA in order to restart production. Thus, for any $t$,
$V^{t}((0,0,0,0,1,0,0),(i))=-d$ and $V^{t}((0,0,0,0,0,1,0),(i))=-c$, both of
which are the optimal policy for the respective belief states.

What remains to be constructed are the value functions for when the facility
is in operations, i.e., $b^{t}=(b^{t}_{N},b^{t}_{V},b^{t}_{O},0,0,0)$. Note
that, when the facility is in operation and the FDA chooses to inspect, the
FDA will receive the reward for the facility being open and then will not
receive any future rewards. Thus, $V^{t}(b^{t},(i))=1$ for any $t$. To define
the remaining $V^{t}$, we first note that
$V^{T-1}(b^{T-1},(ni,i))=1-P[(dr)]^{T}d-P[(cr)]^{T}c+P[(nr)]^{T}V^{T}(b^{t+1},(i))=1-P[(dr)]^{T}d-P[(cr)]^{T}c+P[(nr)]^{T}$.
In this equation, the expected value accumulated in time periods $T-1$ and $T$
is the immediate value received for the facility being in operation in time
period $T-1$ ($1$), and the expected reward for choosing to wait to inspect in
time period $T$ ($-P[(dr)]^{T}d-P[(cr)]^{T}c+P[(nr)]^{T}$).

Observe that $p_{NO}+p_{NV}+p_{NN}=1-p_{ND}+p_{NC}$,
$p_{VO}+p_{VV}=1-p_{VD}-p_{VC}$ and $p_{OO}=1-p_{OD}-p_{OC}$. Thus,
$V^{T-1}(b^{T-1},(ni,i))=1+(1-(d+1)p_{ND}-(c+1)p_{NC})b_{N}^{T-1}+(1-(d+1)p_{VD}-(c+1)p_{VC})b_{V}^{T-1}+(1-(d+1)p_{OD}-(c+1)p_{OC})b_{O}^{T-1}$.
By defining $k_{S}=1-(d+1)p_{SD}-(c+1)p_{SC}$ for $S\in\\{N,V,O\\}$, we can
more concisely say
$V^{T-1}(b^{T-1},(ni,i))=1+\sum_{S\in\\{N,V,O\\}}k_{S}b_{S}^{T-1}$.

We start by recursively defining the value functions for $\sigma^{t}_{T-t}$,
when the FDA does not inspect until they are required to do so. For $t<T$, the
value functions are defined as

$V^{t}(b^{t},\sigma_{T-t}^{t})=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}V^{t+1}(b^{t+1},\sigma_{T-t-1}^{t+1}).$
(1)

Let $\Theta^{i}_{SS^{\prime}}$ be the set of all paths from $S$ to
$S^{\prime}$ in $i-1$ time steps for $i\geq 2$ where
$\theta^{i}\in\Theta^{i}_{SS^{\prime}}$ is defined as
$\theta^{i}=(\theta^{i}_{1},\ldots,\theta^{i}_{i})$ with $\theta^{i}_{j}$ as
the state after $(j-1)$ time steps. Let
$\Theta^{i}=\bigcup_{S,S^{\prime}\in\\{N,V,O\\}}\Theta^{i}_{SS^{\prime}}$. We
can additionally define a closed form equivalent of
$V^{t}(b^{t},\sigma_{T-t}^{t})$ as:

$V^{t}(b^{t},\sigma_{T-t}^{t})=1+\sum_{S\in\\{N,V,O\\}}\sum_{\begin{subarray}{c}S^{\prime}\in\\{N,V,O\\}\\\
S^{\prime}\text{ is reachable from
}S\end{subarray}}k_{SS^{\prime}}^{T-t}b_{S}^{t},$ (2)

where
$k_{SS^{\prime}}^{T-t}=k_{S^{\prime}}\sum_{\theta^{T-t}\in\Theta^{T-t}_{SS^{\prime}}}f_{\theta^{T-t}},$
$f_{\theta_{1}^{2},\theta_{2}^{2}}=\begin{cases}p_{\theta_{1}^{2}\theta_{1}^{2}}+1\text{
if }\theta_{1}^{2}=\theta_{2}^{2},\\\ p_{\theta_{1}^{2}\theta_{2}^{2}}\text{
otherwise.}\end{cases}$

and

$f_{\theta^{T-t}}=\begin{cases}p_{\theta_{1}^{T-t}\theta_{2}^{T-t}}f_{(\theta_{2}^{T-t},\ldots,\theta_{T-t}^{T-t})}+1\text{
if }\theta_{1}^{T-t}=\theta_{2}^{T-t}=\theta_{T-t}^{T-t},\\\
p_{\theta_{1}^{T-t}\theta_{2}^{T-t}}f_{(\theta_{2}^{T-t},\ldots,\theta_{T-t}^{T-t})}\text{
otherwise,}\end{cases}$

The proof of this equivalency, as well as all future proofs, appears in
Appendix B.

######  Theorem 1.

Equation (1) is equivalent to Equation (2) for $1\leq t\leq T-2$, i.e., there
are closed form equations for the value functions that incorporate the dynamic
nature of the equations explicitly.

We lastly need to define $V^{t}(b^{t},\sigma_{j}^{t})$ for $j\neq T-t$. Since
no value is accumulated after an inspection occurs and the reward accumulated
each time period is constant over the entire time horizon, the closed form
definition of $V^{t}(b^{t},\sigma_{j}^{t})$ will be similar to the closed form
definition of $V^{T-j}(b^{T-j},\sigma_{T-j}^{T-j})$. This is because both
conditional plans accumulated the rewards of $j$ time periods. The only
adjustment that needs to be made is to use the belief probabilities at time
$t$ instead of $T-j$. Thus, the closed form definition is
$V^{t}(b^{t},\sigma_{j}^{t})=V^{T-j}(b^{t},\sigma_{T-j}^{T-j})$. This
completes the definition of the value functions.

##  4 Analysis of Conditional Plans

Similarly to Cheng et al. (2023),Qiu et al. (2023) and Zhao et al. (2021), we
demonstrate structural properties of our problem when we enforce additional
structure on the sequences $(P[(dr)]^{t})$, $(P[(cr)]^{t})$ and
$(P[(nr)]^{t})$. We assume that $(P[(dr)]^{t})$ and $(P[(cr)]^{t})$ are non-
decreasing over $t$, and thus that $(P[(nr)]^{t})$ is non-increasing over $t$.
This is a reasonable assumption, as the quality of the facility will degrade
over time, which in turn will make the facility more susceptible to a
manufacturing failure. As the facility becomes more susceptible to a
manufacturing failure, the facility is in turn more likely to halt operations
to allow for non-mandatory maintenance to occur.

With this additional structure, we can directly compare the values of
conditional plans. We first compare two conditional plans where one plan
inspects in time period $j$ and the other inspects in time period $j+1$. Lemma
1 states that once inspecting earlier has a greater expected value, it will
continue to have a greater expected value for the remainder of the time
horizon. Likewise, if inspecting later has a greater expected value, it will
also have a greater expected value in earlier time periods.

######  Lemma 1.

Suppose $(P[(dr)]^{t})$ and $(P[(cr)]^{t})$ are non-decreasing sequences and
$(P[(nr)]^{t})$ is a non-increasing sequence. If, for some $t$ and $j\leq
T-t$, $V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$, then
$V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$.
If $V^{t}(b^{t},\sigma_{j}^{t})\leq V^{t}(b^{t},\sigma_{j+1}^{t})$, then
$V^{t-1}(b^{t-1},\sigma_{j}^{t-1})\leq V^{t-1}(b^{t-1},\sigma_{j+1}^{t-1})$.

Lemma 1 allows us to prove the following theorem about the optimal inspection
time. Theorem 2 shows that, if there exists a time period $t$ where inspecting
immediately is better than waiting to inspect until time period $t+1$, then
waiting $j$ time periods after $t$ to inspect will be better than waiting
$j+1$ time periods after $t$.

######  Theorem 2.

Suppose $(P[(dr)]^{t})$ and $(P[(cr)]^{t})$ are non-decreasing sequences,
$(P[(nr)]^{t})$ is a non-increasing sequence and there exists a $t$ such that
$V^{t}(b^{t},(i))\geq V^{t}(b^{t},(ni,i))$. Then
$V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$ for $1\leq
j<T-t$.

From Theorem 2, we can further deduce that the optimal inspection time can be
determined by comparing $V^{t}(b^{t},(i))$ and $V^{t}(b^{t},(ni,i))$.

######  Corollary 1.

Suppose $(P[(dr)]^{t})$ and $(P[(cr)]^{t})$ are non-decreasing sequences and
$(P[(nr)]^{t})$ is a non-increasing sequence and there exists a $t$ such that
$V^{t}(b^{t},(i))\geq V^{t}(b^{t},(ni,i))$. Then
$V^{t}(b^{t},(ni,i))\geq V^{t}(b^{t},\sigma_{j}^{t})$ for all $2\leq j\leq
T-t$.

Corollary 1 indicates that, at any time period, we only need to compare the
value functions for the plans $(i)$ and $(ni,i)$. Hence, making a simple
comparison at each time period is sufficient to deduce the optimal inspection
time for the overall problem, significantly reducing the complexity of the
problem. If $V^{t}(b^{t},(i))\leq V^{t}(b^{t},(ni,i))$, then inspecting now is
not optimal, and thus the immediate choice of not inspecting will be the
decision in the optimal conditional plan. However, if $V^{t}(b^{t},(i))\geq
V^{t}(b^{t},(ni,i))$, then by Corollary 1 we know that the optimal conditional
plan is to inspect immediately. Lemma 1 further confirms that this is indeed
the correct decision, as it implies $V^{\bar{t}}(b^{\bar{t}},(i))\geq
V^{\bar{t}}(b^{\bar{t}},(ni,i))$ for $t\leq\bar{t}\leq T$. This result is
similar to the identification of critical thresholds in the mission abort
literature, despite the difference in reward structures (Cheng et al., 2023;
Qiu et al., 2023; Zhao et al., 2021). For any time period $t$, this reduces
the number of conditional plans we need to consider from $T-t+1$ to $2$,
drastically reducing the computational cost. Thus, we have an algorithm to
identify the optimal inspection time, which requires at most $2T-2$
computations. The maximum number of computations is only required when the FDA
does not inspect until time periods $T-1$ or $T$. Additionally, the FDA could
integrate the score $V^{t}(b^{t},(ni,i))-V^{t}(b^{t},(i))$ into their existing
SSM model. This score indicates that a facility is more in need to being
inspected as the score decreases. Given that the score is easy to compute and
does not require the use of any complex algorithms, it will be simple to
integrate into the SSM.

From Lemma 1, we can derive further structure to the value of the conditional
plans. The expected value from inspecting in time period $t$ is non-decreasing
as $t$ increases until the optimal inspection time $t^{*}$, then is non-
increasing as $t$ increases.

######  Corollary 2.

Let $t^{*}$ be the optimal inspection time, i.e.,
$V^{1}(b^{1},\sigma^{1}_{t^{*}})\geq V^{1}(b^{1},\sigma^{1}_{t})$ for all $t$.
Then $V^{1}(b^{1},\sigma^{1}_{t})\leq V^{1}(b^{1},\sigma^{1}_{t+1})$ for
$1\leq t<t^{*}$ and $V^{1}(b^{1},\sigma^{1}_{t})\geq
V^{1}(b^{1},\sigma^{1}_{t+1})$ for $t^{*}\leq t\leq T-1$.

###  4.1 Sensitivity of Optimal Inspection Time to Penalty Parameters

We additionally explore how varying $d$ and $c$ impact the optimal inspection
time $t^{*}$. We consider two scenarios: one where we are interested in
inspecting at time $t^{*}-1$, and one where $\bar{t}$ is a user-provided input
and we want to determine what values $d$ and $c$ can take such that
$t^{*}=\bar{t}$. For the first scenario, we derive an inequality that
determines the minimum change in $d$ and $c$ such that inspecting at time
$t^{*}-1$ is strictly better than waiting until time $t^{*}$ to inspect. We
analytically find that the optimal inspection time is more sensitive to change
in penalty parameters as the probabilities of unexpected disruptive events
increase. For the second case, we derive a pair of inequalities on $d$ and $c$
that ensure that $\bar{t}$ is the optimal inspection time. Our computational
study of these inequalities shows that the range these penalty parameters can
lie in decreases as $\bar{t}$ increases. More details about these inequalities
are in Appendix C.

##  5 Modeling Accounting for Inspection Outcomes

We next consider a model where inspection may result in the facility needing
to close for mandatory maintenance, resulting in a small penalty for the FDA.
This model considers how FDA intervention can also cause a disruption that the
FDA must account for. The FDA would force a facility that receives the OA
classification to halt production; receiving the OA classification means that
there are too many regulatory violations in the manufacturing process, and the
drug produced is either ineffective or could cause harm when used. To
incorporate this variation into our model, we add a new state $IC$, which can
be transitioned to when the FDA inspects from the states $N$, $V$, or $O$.

Let $p_{SIC}$ be the probability of closing after being inspected in state
$S$, and let $\tilde{c}\geq 0$ be the penalty for being in state $IC$, where
$\tilde{c}\leq c\leq d$. We choose $\tilde{c}$ in this manner since closure
does increase vulnerability to shortages, but, unlike in the event of closing
for non-mandatory maintenance, this closure is not unexpected by the FDA, and
thus the FDA can take appropriate measures to mitigate further vulnerabilities
to shortages.

With this adaptation, if $b_{N}^{t}+b_{V}^{t}+b_{O}^{t}=1$, then
$V^{t}(b^{t},(i))=1-\tilde{c}(p_{NIC}b_{N}^{t}+p_{VIC}b_{V}^{t}+p_{OIC}b_{O}^{t})$.
Under this definition, if the facility is in operations in time period $t$,
then inspecting will result in a penalty in expectation. Thus,

$\displaystyle V^{T-1}(b^{T-1},(ni,i))$
$\displaystyle=1-P[(dr)]^{T}d-P[(cr)]^{T}c+P[(nr)]^{T}V^{T}(b^{T},(i))$
$\displaystyle=1+k_{N}b_{N}^{T-1}+k_{V}b_{V}^{T-1}+k_{O}b_{O}^{T-1}$
$\displaystyle-\tilde{c}(p_{NIC}(p_{NN}b_{N}^{T-1})+p_{VIC}(p_{NV}b_{N}^{T-1}+p_{VV}b_{V}^{T-1})$
$\displaystyle+p_{OIC}(p_{NO}b_{N}^{T-1}+p_{VO}b_{V}^{T-1}+p_{OO}b_{O}^{T-1}))$
$\displaystyle=1+(k_{N}-\tilde{c}(p_{NN}p_{NIC}+p_{NV}p_{VIC}+p_{NO}p_{OIC}))b_{N}^{T-1}$
$\displaystyle+(k_{V}-\tilde{c}(p_{VV}p_{VIC}+p_{VO}p_{OIC}))b_{V}^{T-1}+(k_{O}-\tilde{c}p_{OO}p_{OIC})b_{O}^{T-1}$

Let
$\tilde{c}_{SS^{\prime}IC}^{i}=\tilde{c}p_{S^{\prime}IC}\sum_{\theta^{i}\in\Theta^{i}_{SS^{\prime}}}\left(\prod_{j=1}^{i-1}p_{\theta_{j}^{i}\theta_{j+1}^{i}}\right)$.
Under this adaptation, we can again define a closed form equivalent for the
value functions as:

$V^{t}(b^{t},\sigma_{t}^{T-t})=1+\sum_{S\in\\{N,V,O\\}}\sum_{\begin{subarray}{c}S^{\prime}\in\\{N,V,O\\}\\\
S^{\prime}\text{ is reachable from
}S\end{subarray}}(k_{SS^{\prime}}^{T-t}-\tilde{c}_{SS^{\prime}IC}^{T-t+1})b_{S}^{t}.$
(3)

######  Theorem 3.

Under the definition of
$V^{t}(b^{t},(i))=1-\tilde{c}(p_{NIC}b_{N}^{t}+p_{VIC}b_{V}^{t}+p_{OIC}b_{O}^{t})$
when $b^{t}_{N}+b^{t}_{V}+b^{t}_{O}=1$, Equation (1) is equivalent to Equation
(3) for $i\geq 2$.

Let $P[(cr)|(i)]^{t}=p_{NIC}b_{N}^{t}+p_{VIC}b_{V}^{t}+p_{OIC}b_{O}^{t}$ i.e.,
$P[(cr)|(i)]^{t}$ is the probability we believe that inspecting in time period
$t$ will result in the facility needing to close for mandatory maintenance. To
prove an equivalent version of Theorem 2, we need an equivalent version of
Lemma 1. Let $d=\alpha_{d}\tilde{c}$ and $c=\alpha_{c}\tilde{c}$, and note
that $\alpha_{d}\geq 1$ and $\alpha_{c}\geq 1$. As before, we assume
$(P[(dr)]^{t})$, $(P[(cr)]^{t})$ and $(P[(cr)|(i)]^{t})$ are non-decreasing
sequences and $(P[(nr)]^{t})$ is a non-increasing sequence. We additionally
assume that $(P[(nr)]^{t}P[(cr)|(i)]^{t})$ is a non-decreasing sequence, i.e.,
the probability of a facility needing to close after an inspection increases
faster than probability of no issues being reported decreases. Since
$P[(nr)]^{t}+P[(dr)]^{t}+P[(cr)]^{t}=1$ for all $t$, this assumption is
equivalent to saying, the probability of a facility closing after an
inspection increases faster than the probability of a facility experiencing an
unexpected disruptive event increases. This is a reasonable assumption since
we expect the FDA to be as sensitive to CGMP violations as facility managers
would be.

######  Lemma 2.

Suppose $(P[(dr)]^{t})$, $(P[(cr)]^{t})$ and $(P[(cr)|(i)]^{t})$ are non-
decreasing sequences and $(P[(nr)]^{t})$ is a non-increasing sequence.
Additionally, suppose that $(P[(nr)]^{t}P[(cr)|(i)]^{t})$ is a non-decreasing
sequence and
$\alpha_{d}(P[(dr)]^{t+1}-P[(dr)]^{t})+\alpha_{c}(P[(cr)]^{t+1}-P[(cr)]^{t})\geq(P[(cr)|(i)]^{t+1}-P[(cr)|(i)]^{t})$.
If, for some $t$ and $j\leq T-t$, $V^{t}(b^{t},\sigma_{j}^{t})\geq
V^{t}(b^{t},\sigma_{j+1}^{t})$, then $V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq
V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$.

We note that the bound
$\alpha_{d}(P[(dr)]^{t+1}-P[(dr)]^{t})+\alpha_{c}(P[(cr)]^{t+1}-P[(cr)]^{t})\geq(P[(cr)|(i)]^{t+1}-P[(cr)|(i)]^{t})$
is not tight, but instead provides a nice practical interpretation. As long as
the expected penalty of closure due to inspection is increasing at a slower
rate than the expected penalty of a manufacturing failure or unexpected
closure occurring, then a conditional plan $\sigma_{j}^{t}$, once preferred
over $\sigma_{j+1}^{t}$, will remain preferred as $t$ increases. We now prove
an equivalent version of Theorem 2.

######  Theorem 4.

Suppose $(P[(dr)]^{t})$, $(P[(cr)]^{t})$ and $(P[(cr)|(i)]^{t})$ are non-
decreasing sequences and $\\\ (P[(nr)]^{t})$ is a non-increasing sequence.
Additionally, suppose that $(P[(nr)]^{t}P[(cr)|(i)]^{t})$ is a non-decreasing
sequence,
$\alpha_{d}(P[(dr)]^{t+1}-P[(dr)]^{t})+\alpha_{c}(P[(cr)]^{t+1}-P[(cr)]^{t})\geq(P[(cr)|(i)]^{t+1}-P[(cr)|(i)]^{t})$.
If there exists a $t$ such that $V^{t}(b^{t},(i))\geq V^{t}(b^{t},(ni,i))$,
then $V^{t}(b^{t},\sigma^{t}_{j})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$ for
$1\leq j\leq T-t$.

Theorem 4 again allows us to further deduce an equivalent version of Corollary
1, and we can thus again determine the optimal inspection time in this variant
model solely by comparing $V^{t}(b^{t},(i))$ and $V^{t}(b^{t},(ni,i))$.

######  Corollary 3.

Suppose $(P[(dr)]^{t})$, $(P[(cr)]^{t})$ and $(P[(cr)|(i)]^{t})$ are non-
decreasing sequences and $\\\ (P[(nr)]^{t})$ is a non-increasing sequence.
Additionally, suppose that $(P[(nr)]^{t}P[(cr)|(i)]^{t})$ is a non-decreasing
sequence,
$\alpha_{d}(P[(dr)]^{t+1}-P[(dr)]^{t})+\alpha_{c}(P[(cr)]^{t+1}-P[(cr)]^{t})\geq(P[(cr)|(i)]^{t+1}-P[(cr)|(i)]^{t})$.
If there exists a $t$ such that $V^{t}(b^{t},(i))\geq V^{t}(b^{t},(ni,i))$,
then $V^{t}(b^{t},(ni,i))\geq V^{t}(b^{t},\sigma_{j}^{t})$ for all $2\leq
j\leq T-t$.

As with the base model, we only need to compare the value of inspecting
immediately against the value of inspecting after waiting one time period to
determine the optimal solution. This again reduces our problem to a series of
simple computations to determine when to inspect. Thus, the algorithm proposed
in Section 4 will still identify the optimal inspection time, and the score
$V^{t}(b^{t},(ni,i))-V^{t}(b^{t},(i))$ can still be integrated into the FDA’s
existing SSM model.

##  6 Computational Study

We simulate how the quality of a facility will degrade over time to test how
effective our model is at maximizing the value of a facility while being
cognizant of the penalties of unexpected disruptive events occurring. For
these experiments, we assume the facility starts in the No Official Action
($N$) state with full certainty, i.e., $b^{1}=(1,0,0,0,0,0)$, and we iterate
the Markov chain until an unexpected disruptive event occurs. We choose this
starting belief probability under the assumption that the facility was just
inspected and given the No Action Indicated classification, so the state is
known with full certainty. We say an inspection rule _catches an unexpected
disruptive event_ if the time of inspection calculated by the inspection rule
is strictly less than the time the Markov chain reaches either states $D$ or
$C$ in the simulation, which we denote to be $t_{F}$. We follow the procedure
outlined in Section 4 to identify the optimal inspection time. Let $t_{V}$ be
the optimal inspection time when not accounting for inspection outcomes, and
let $t_{VC}$ be the optimal inspection time when accounting for inspection
outcomes. As a baseline, we compare these inspection times against inspecting
before the expected time to reach states $D$ or $C$ from the state $N$. Since
we cannot compare against SSM-recommended inspection times, this serves as the
best existing means of comparison that accounts for qualities intrinsic to the
facility and inherent risks from production. The expected time to reach state
$D$ or $C$ from a state $S$ is computed as
$\mu_{S}=1+\sum_{S^{\prime}\in\\{N,V,O\\}:p_{SS^{\prime}}>0}p_{SS^{\prime}}\mu_{S^{\prime}}$
(Brémaud, 2020). Then given the expected time to reach state $D$ or $C$, in
the baseline setting we inspect in the time period
$t_{E}=\lceil\mu_{N}\rceil-1$, which is the largest integer that is strictly
less than $\mu_{N}$. We refer to this rule as the ETD inspection rule. We
refer to these three inspection rules as the risk-based inspection rules.

We perform $100,000$ experiments for each set of fixed penalty parameters, and
we record the percentage of instances where each inspection rule inspects
before an unexpected disruptive event occurs and the total accumulated value
for each instance. We additionally compare the total accumulated values
against the values accumulated when there is no inspection, i.e., when the
Markov chain runs until it reaches either states $D$ or $C$, and when the FDA
inspects after a fixed number of time periods: $T_{1}=24$, $T_{2}=60$ and
$T_{3}=120$. If each time period were to be representative of a month, then
these time horizons would be equivalent to two years, five years and ten
years, respectively. These comparisons shed light on the value of planning
inspections based on risk. Numerical experiments were conducted in R on a
laptop with an Intel® CoreTM i7-11370H processor and 16 GB RAM running Windows
11. The iGraph package (Csardi & Nepusz, 2006) was used to conduct the penalty
parameter sensitivity analysis.

###  6.1 Catching Unexpected Disruptive Events

We explore how different transition probability matrices and penalty
parameters will impact the percent of unexpected disruptive events caught and
the total value accumulated over time. We first analyze the results of our
inspection rules with a fixed transition probability matrix and fixed penalty
parameters $d$ and $c$. For these experiments, we set the penalty parameters
to be $c=5$ and $\tilde{c}=1$, and we set the probabilities of closure due to
inspection to be $p_{NIC}=0$, $p_{VIC}=0.3$ and $p_{OIC}=1$. A facility
classified as No Action Needed has no need to close for maintenance, while a
facility classified as Official Action Needed will have regulatory action
taken against it if it does not close for maintenance. We speculate that a
facility classified as Voluntary Action Needed will be more likely to stay in
operations, as closing for non-mandatory maintenance will reduce profits
through maintenance costs and lost sales. We choose $\tilde{c}$ such that the
assumptions outlined in Section 5 are satisfied for all tested values of $d$.
We test three different probability matrices with different expected times to
unexpected disruptive events and various penalty parameters $d$, all of which
satisfy the assumptions in Theorems 2 and 4. The first matrix we test is

$P^{1}(ni)=\begin{bmatrix}0.9125&0.0875&0&0&0&0\\\
0&0.825&0.1125&0.045&0.0175&0\\\ 0&0&0.75&0.175&0.075&0\\\ 0&0&0&1&0&0\\\
0&0&0&0&1&0\\\ 0&0&0&0&0&1\end{bmatrix}.$

Table 1 presents the recommended inspection times for each inspection rule
with varying values for $d\in\\{14,18,22,26,30\\}$. These choices of $d$
provided a wide variety of optimal inspection times.

$d$ | $t_{V}$ | $t_{VC}$ | $t_{E}$
---|---|---|---
14 | 27 | 31 | 19
18 | 15 | 16 | 19
22 | 12 | 12 | 19
26 | 10 | 10 | 19
30 | 8 | 8 | 19
Table 1: Recommended Inspection Time Period for $P^{1}(ni)$

Table 2 presents the average value accumulated with each inspection rule, as
well as the percent of unexpected disruptive events caught. We use the results
of $t_{E}$ as a baseline, and, for each other inspection rule, we provide the
absolute difference for percent of unexpected disruptive events caught and the
relative difference for the average value accumulated. For each rule, we
present the value accumulated if we are not accounting for the outcome of the
inspection (no IC) and the value accumulated if we are accounting for the
outcome of the inspection (IC). We additionally present statistics around the
time until an unexpected disruptive event occurs in in Table 3.

Rule | $t_{E}$ | $t_{V}$ | $t_{VC}$ | 24 | 60 | 120 | NoIns
---|---|---|---|---|---|---|---
% Caught, $d=14$ | 44.297% | -21.084% | -27.857% | -14.459% | -43.103% | -44.265% | -
Value (no IC), $d=14$ | 8.3402 | +1.580% | - | +1.501% | +0.277% | +0.170% | +0.171%
Value (IC), $d=14$ | 8.2012 | - | +2.692% | +1.980% | +1.923% | +1.868% | -
% Caught, $d=18$ | 44.438% | +14.944% | +10.900% | -14.534% | -43.195% | -44.433% | -
Value (no IC), $d=18$ | 6.7869 | +1.798% | - | -4.114% | -17.156% | -17.734% | -17.739%
Value (IC), $d=18$ | 6.6488 | - | +1.558% | -3.621% | -15.464% | -16.022% | -
% Caught, $d=22$ | 44.351% | +27.165% | +27.165% | -14.587% | -43.177% | -44.344% | -
Value (no IC), $d=22$ | 5.1509 | +14.801% | - | -13.884% | -46.358% | -47.779% | -47.787%
Value (IC), $d=22$ | 5.0123 | - | +14.409% | -13.382% | -44.961% | -46.336% | -
% Caught, $d=26$ | 44.700% | +35.326% | +35.326% | -14.583% | -43.484% | -44.696% | -
Value (no IC), $d=26$ | 3.7237 | +43.271% | - | -30.341% | -98.751% | -101.641% | -101.646%
Value (IC), $d=26$ | 3.5834 | - | +43.955% | -30.404% | -98.822% | -101.705% | -
% Caught, $d=30$ | 44.029% | +43.628% | +43.628% | -14.329% | -42.828% | -44.026% | -
Value (no IC), $d=30$ | 1.9024 | +155.435% | - | -77.991% | -251.477% | -259.425% | -259.446%
Value (IC), $d=30$ | 1.7651 | - | +165.991% | -81.984% | -263.503% | -271.826% | -

Table 2: Percent of Unexpected Disruptive Events Caught and Average Value Accumulated with Each Inspection Rule for $P^{1}(ni)$ $d$ | 14 | 18 | 22 | 26 | 30
---|---|---|---|---|---
Average End Time | 20.7118 | 20.7540 | 20.6935 | 20.7876 | 20.6297
Std. Dev. End Time | 12.5409 | 12.6105 | 12.5441 | 12.5777 | 12.4737
Median End Time | 18 | 18 | 18 | 18 | 18
Min End Time | 3 | 3 | 3 | 3 | 3
Max End Time | 143 | 147 | 135 | 145 | 145
Table 3: Descriptive Statistics of Time of Unexpected Disruptive Event for
$P^{1}(ni)$

From these tables, we can see that, for $d=14$, while the ETD inspection rule
catches more unexpected disruptive events than the optimal inspection rules,
the value accumulated is less than that of other rules. This scenario is
analogous to the case where the facility experiencing an unexpected disruptive
event is not impactful enough to warrant more frequent inspections. As the
penalty for a manufacturing failure increases, the optimal inspection rules
inspect earlier and catch more unexpected disruptive events than ETD (and the
other inspection rules), which is fixed as it does not take into account the
penalty of unexpected disruptive events. The optimal inspection rules
additionally accumulated more value on average than the other inspection
rules, indicating that they are better able to account for the trade-offs in
the problem. Figure 2 graphically presents the relative differences in average
value accumulated between the ETD inspection rule and optimal inspection
rules. From this, we can see a dramatic trend in the increase in the relative
difference. For smaller values of $d$, the values accumulated between the
different rules are fairly similar, and the relative difference for both
optimal inspection rules grows super-linearly as $d$ increases. This
highlights how much more important it is for the FDA to prioritize inspecting
pharmaceutical facilities that manufacture products more critical to public
health.

Figure 2: Relative difference in value between the ETD inspection rule and
optimal inspection rules for $P^{1}(ni)$

Additionally, when accounting for the outcome of inspecting, the delay in
inspecting can result in a slight increase in value even though less
unexpected disruptive events are caught. In particular, when $d=14$, not
accounting for this closure possibility caught approximately $7\%$ more
unexpected disruptive events while accumulating less value, and when $d=18$,
not accounting for this closure caught approximately $4\%$ more unexpected
disruptive events while accumulating a similar value. These small differences
in the number of unexpected disruptive events caught, as compared to the
difference in average value accumulated, indicates that the FDA can take a
calculated risk in waiting to inspect longer if the penalties for unexpected
disruptive events are relatively small compared to the penalties of the
facility closing for mandatory maintenance.

We next want to understand the robustness of our inspection strategy. For the
next set of experiments, we randomly generate a probability matrix $\hat{P}$
where each entry $\hat{p}_{ij}$ is randomly sampled from a normal distribution
with mean $p_{ij}$ and a given standard deviation $s$. We then modify
$\hat{P}$ to be $\tilde{P}$ to ensure that each row in the matrix is indeed a
probability distribution and that the states $D$ and $C$ can be transitioned
to from at least state $O$. We set
$\tilde{p}_{SS}=\min\\{\hat{p}_{SS},0.995\\}$, and
$\tilde{p}_{SS^{\prime}}=\max\\{0,\min\\{\hat{p}_{SS^{\prime}},1-\sum_{S^{\prime\prime}\in\\{N,V,O\\}:\hat{p}_{SS^{\prime\prime}}>0}\hat{p}_{SS^{\prime\prime}}\\}\\}$.
To ensure that each row sums to one, we do not randomly generate the
transition probability for the last nonzero entry in the row
$\hat{p}_{SS_{F}}$ and instead use
$\tilde{p}_{SS_{F}}=1-\sum_{S^{\prime\prime}\in\\{N,V,O\\}:\tilde{p}_{SS^{\prime\prime}}>0}\tilde{p}_{SS^{\prime\prime}}$.
The matrix $\tilde{P}$ is the transition probability matrix used. The
inspection times considered are still the inspection times presented in Table
1, indicating that the FDA used the expected value for transition
probabilities when calculating the optimal inspection time.

We test two different standard deviations $s\in\\{0.01,0.02\\}$ to see how the
uncertainty around the transition probabilities impacts the number of
unexpected disruptive events caught and total value accumulated. Tables 2 and
5 present results and descriptive statistics for experiments with $s=0.01$,
and Tables 6 present results and descriptive statistics for experiments with
$s=0.02$.

Rule | $t_{E}$ | $t_{V}$ | $t_{VC}$ | 24 | 60 | 120 | NoIns
---|---|---|---|---|---|---|---
% Caught, $d=14$ | 44.940% | -21.091% | -27.832% | -14.546% | -43.533% | -44.933% | -
Value (no IC), $d=14$ | 8.4838 | +2.132% | - | +1.695% | +2.208% | +2.130% | +2.219%
Value (IC), $d=14$ | 8.3447 | - | +3.376% | +2.193% | +3.848% | +3.831% | -
% Caught, $d=18$ | 44.611% | +14.713% | +10.864% | -14.443% | -43.240% | -44.596% | -
Value (no IC), $d=18$ | 6.8494 | +1.216% | - | -3.811% | -15.794% | -16.283% | -16.292%
Value (IC), $d=18$ | 6.7123 | - | +1.110% | -3.331% | -14.143% | -14.575% | -
% Caught, $d=22$ | 44.598% | +26.999% | +26.999% | -14.461% | -43.200% | -44.581% | -
Value (no IC), $d=22$ | 5.3079 | +12.415% | - | -12.756% | -43.618% | -45.018% | -45.042%
Value (IC), $d=22$ | 5.1693 | - | +12.004% | -12.354% | -42.198% | -43.546% | -
% Caught, $d=26$ | 44.856% | +35.119% | +35.119% | -41.820% | -43.471% | -44.846% | -
Value (no IC), $d=26$ | 3.8274 | +39.460% | - | -28.562% | -91.605% | -94.686% | -94.717%
Value (IC), $d=26$ | 3.6879 | - | +39.969% | -28.599% | -91.415% | -94.485% | -
% Caught, $d=30$ | 44.372% | +43.165% | +43.165% | -14.241% | -43.001% | -44.364% | -
Value (no IC), $d=30$ | 2.1102 | +129.703% | - | -68.377% | -220.609% | -228.021% | -228.054%
Value (IC), $d=30$ | 1.9728 | - | +137.358% | -71.300% | -229.253% | -236.942% | -

Table 4: Percent of Unexpected Disruptive Events Caught and Average Value Accumulated with Each Inspection Rule for $\tilde{P}^{1}(ni)$ with $s=0.01$ $d$ | 14 | 18 | 22 | 26 | 30
---|---|---|---|---|---
Average End Time | 20.9702 | 20.8595 | 20.8659 | 20.9291 | 20.8380
Std. Dev. End Time | 12.8893 | 12.8529 | 12.8729 | 12.9012 | 12.8479
Median End Time | 18 | 18 | 18 | 18 | 18
Min End Time | 3 | 3 | 3 | 3 | 3
Max End Time | 151 | 173 | 158 | 149 | 166
Table 5: Descriptive Statistics of Time of Unexpected Disruptive Event for
$\tilde{P}^{1}(ni)$, $s=0.01$

Rule | $t_{E}$ | $t_{V}$ | $t_{VC}$ | 24 | 60 | 120 | NoIns
---|---|---|---|---|---|---|---
% Caught, $d=14$ | 45.704% | -20.928% | -27.574% | -14.213% | -43.617% | -45.636% | -
Value (no IC), $d=14$ | 8.7804 | +3.507% | - | +2.840% | +6.925% | +7.885% | +8.068%
Value (IC), $d=14$ | 8.6422 | - | +5.335% | +3.313% | +8.563% | +9.608% | -
% Caught, $d=18$ | 45.722% | +14.214% | +10.325% | -14.249% | -43.665% | -45.651% | -
Value (no IC), $d=18$ | 7.3038 | -0.998% | - | -1.941% | -7.977% | -7.733% | -7.614%
Value (IC), $d=18$ | 7.1650 | - | -0.844% | -1.439% | -6.282% | -5.947% | -
% Caught, $d=22$ | 45.807% | +26.309% | +26.309% | -14.141% | -43.674% | -45.728% | -
Value (no IC), $d=22$ | 5.8957 | +6.666% | - | -8.313% | -29.164% | -29.527% | -29.421%
Value (IC), $d=22$ | 5.7476 | - | +6.361% | -7.704% | -27.443% | -27.712% | -
% Caught, $d=26$ | 45.516% | +34.782% | +34.792% | -14.138% | -43.518% | -45.462% | -
Value (no IC), $d=26$ | 4.3086 | +30.226% | - | -21.090% | -67.614% | -69.352% | -69.134%
Value (IC), $d=26$ | 4.1710 | - | +30.283% | -20.894% | -66.684% | -68.343% | -
% Caught, $d=30$ | 45.722% | +42.033% | +42.033% | -14.252% | -43.641% | -45.654% | -
Value (no IC), $d=30$ | 2.9540 | +71.364% | - | -44.259% | -141.134% | -145.833% | -145.650%
Value (IC), $d=30$ | 2.8141 | - | +74.180% | -45.073% | -143.385% | -148.118% | -

Table 6: Percent of Unexpected Disruptive Events Caught and Average Value Accumulated with Each Inspection Rule for $\tilde{P}^{1}(ni)$ with $s=0.02$ $d$ | 14 | 18 | 22 | 36 | 30
---|---|---|---|---|---
Average End Time | 21.5801 | 21.5648 | 21.6401 | 21.5522 | 21.5749
Std. Dev. End Time | 14.3624 | 14.1660 | 14.3501 | 14.3273 | 14.2560
Median End Time | 18 | 18 | 18 | 18 | 18
Min End Time | 3 | 3 | 3 | 3 | 3
Max End Time | 511 | 261 | 335 | 706 | 431
Table 7: Descriptive Statistics of Time of Unexpected Disruptive Event for
$\tilde{P}^{1}(ni)$, $s=0.02$

When $s=0.01$, the results regarding unexpected disruptive events caught and
value accumulated are similar to the case when $P$ is known with certainty,
indicating that the optimal inspection rules are still able to accurately
determine if the facility is critical enough to monitor more frequently.
However, the value accumulated for fixed inspection times seems to increase
more than the increase in the optimal inspection rules, particularly more as
$T$ increases. This is likely due to instances where the randomly generated
matrix is more likely to transition to the state it is currently in as
compared to $P^{1}$, resulting in more instances with longer times to reach
either state $D$ or $C$, which increases the average value. When $s=0.02$ and
$d=14$, we see that the value for waiting to inspect until $T\geq 60$
accumulates more value than when implementing the optimal inspection rules.
This indicates that, for less critical facilities, being less certain about
the how the quality of the facility will decline over time proves to be a
challenge for risk-based tools, so using simpler metrics may be more
effective.

To help understand why the optimal inspection rules struggle as we become less
certain about the transition probabilities, we explore what the ETD inspection
rule would recommend if it were computed for each randomly generated matrix.
Figure 4 in Appendix D presents histograms of the time periods the ETD
inspection rule decides to inspect for $s=0.01$ (Figure 4(a)) and $s=0.02$
(Figure 4(b)), respectively, for the matrices generated when testing $d=18$.
From these figures, it is clear to see that, for $s=0.01$, the deviation in
when the ETD inspection rules would decide to inspect is relatively small,
suggesting that the randomly generated transition matrices are similar enough
that the optimal inspection rules still accurately capture the trade-off
between waiting to inspect and preventing an unexpected disruptive event.
However, when $s=0.02$, the number of matrices with a significantly different
recommendation from the ETD inspection rule increases dramatically, meaning
that the optimal inspection rules no longer accurately capture this trade-off.
Recall that the ETD inspection rule recommends inspecting in time period $19$
for $P^{1}(ni)$. For the randomly generated matrices, over $90\%$ of the ETD
inspection rule recommendations suggest inspecting in the range $[17,21]$ when
$s=0.01$, while less than $65\%$ of the recommendations are in this range when
$s=0.02$. The generated transition matrices with $s=0.02$ are also expected to
reach states $D$ or $C$ significantly later than the base matrix. When
$s=0.02$, over $10\%$ of the ETD inspection rule recommendations suggest
inspecting in time period $24$ or later, while only $1\%$ of the
recommendations suggest inspecting in time period $24$ or later when $s=0.01$.
This suggests there is significant value in the FDA trying to better
understand how the quality of facilities will degrade over time, as even a
small decrease in standard deviation will lead to much more accurate
recommendations from risk-based tools.

We additionally wish to see how varying the likelihood of transitioning to the
state the facility is currently in for the base transition probability matrix
we use impacts different aspects of the model. The second matrix we test is
more prone to unexpected disruptive events, as the Markov chain can transition
to the state $D$ from any operational state and it is less likely to
transition to the state it is currently in. Table 9 presents the recommended
inspection times for each inspection rule with the varying values for
$d\in\\{10,14,18\\}$. We test different values of $d$ for $P^{2}(ni)$ since
smaller values of $d$ resulted in catching more unexpected disruptive events
than $P^{1}(ni)$.

$d$ | $t_{V}$ | $t_{VC}$ | $t_{E}$
---|---|---|---
10 | 11 | 11 | 12
14 | 7 | 7 | 12
18 | 6 | 6 | 12
Table 8: Recommended Inspection Time Period for $P^{2}(ni)$

$d$ | $t_{V}$ | $t_{VC}$ | $t_{E}$
---|---|---|---
18 | 84 | Inf | 25
22 | 24 | 24 | 25
26 | 17 | 17 | 25
30 | 14 | 14 | 25
34 | 12 | 12 | 25
Table 9: Recommended Inspection Time Period for $P^{3}(ni)$

Table 10 presents the number of unexpected disruptive events caught by each
inspection rule and average value accumulated. The relative differences in the
average value accumulated by the optimal inspection rules follow similar
trends with $P^{1}(ni)$. Because the facility is more prone to experiencing an
unexpected disruptive event, waiting longer to inspect proves to be more
detrimental to the average value accumulated than with $P^{1}(ni)$, even with
smaller values of $d$. Additionally, when $d$ is large enough, waiting too
long can cause the average value to become increasingly negative, indicating
that the facility needs to be monitored more frequently to mitigate the
severely detrimental impact of an unexpected disruptive event.

Rule | $t_{E}$ | $t_{V}$ | $t_{VC}$ | 24 | 60 | 120 | NoIns
---|---|---|---|---|---|---|---
% Caught, $d=10$ | 46.193% | +5.629% | +5.629% | -37.572% | -46.177% | -46.193% | -
Value (no IC), $d=10$ | 4.6667 | +0.351% | - | -12.192% | -17.444% | -17.459% | -17.459%
Value (IC), $d=10$ | 4.4784 | - | +0.051% | -9.503% | -13.975% | -13.989% | -
% Caught, $d=14$ | 46.436% | +30.634% | +30.634% | -37.846% | -46.421% | -46.436% | -
Value (no IC), $d=14$ | 3.1659 | +16.618% | - | -51.435% | -66.252% | -66.271% | -66.271%
Value (IC), $d=14$ | 2.9766 | - | +16.428% | -49.845% | -64.110% | -64.126% | -
% Caught, $d=18$ | 46.309% | +36.809% | +36.809% | -37.738% | -46.297% | -46.309% | -
Value (no IC), $d=18$ | 1.6367 | +89.931% | - | -161.947% | -205.205% | -205.291% | -205.291%
Value (IC), $d=18$ | 1.4456 | - | +99.910% | -173.187% | -219.120% | -219.120% | -

Table 10: Percent of Unexpected Disruptive Events Caught and Average Value
Accumulated with Each Inspection Rule for $P^{2}(ni)$

We again randomly generate the transition probabilities using the same
generation procedure as before with mean values $P^{2}(ni)$. For brevity,
these results appear in Appendix D. In these experiments, the uncertainty
around the transition probabilities is less impactful than for $P^{1}(ni)$.
While the average values accumulated decreased as the standard deviation
increases, the patterns in average values accumulated and number of unexpected
disruptive events caught by each inspection rule follow the same trends as
when the transition matrix is known with certainty. This suggests that for
facilities thought to be more prone to quality degradation, the increased
frequency in monitoring suggested by the optimal inspection rules is robust to
the lack of certainty about the quality degradation of the facility.

The third transition probability matrix we test is more more likely to
transition to the state it is currently in than $P^{1}(ni)$. Table 9 presents
the recommended inspection times for each inspection rule with the varying
values for $d\in\\{18,22,26,30,34\\}$. Note that, with $d=18$, the model where
inspection can result in closure determines that it is never optimal to
inspect. As such, we exclude $d=14$, which will follow similar trends, and
instead include $d=34$.

Table 11 presents the results for $P^{3}(ni)$. The relative differences in the
average value accumulated by the optimal inspection rules again follows
similar trends as $P^{1}(ni)$ and $P^{2}(ni)$ for $d\geq 22$. The most
noteworthy difference from the other transition matrices tested is that, when
$d=18$ for $P^{3}(ni)$, it is never optimal to inspect when inspecting may
result in the facility needing to close for mandatory maintenance, but still
attains an average value accumulated larger than that of most other inspection
rules. This suggests the super-linear behavior observed with $P^{1}(ni)$ and
$P^{2}(ni)$ is actually quadratic. If the penalties for experiencing an
unexpected disruptive event are relatively small compared to how quickly we
expect the facility to experience an unexpected disruptive event, it is more
beneficial to let the facility produce as much as possible. As the penalties
increase, the trade-off between the value of letting the facility manufacture
their product and inspecting before an unexpected disruptive event occurs
becomes more difficult to determine, until the penalties outweigh the value of
letting the facility continue operations. At this point, the relative value of
inspecting earlier grows drastically as the penalties grow. The rate of this
growth in relative value is larger for facilities more likely to experience an
unexpected disruptive event. This suggests that the FDA can best allocate
inspection resources by dedicating more resources to facilities that produce
drugs that are of high importance to public health, primarily focusing on
high-risk facilities.

Rule | $t_{E}$ | $t_{V}$ | $t_{VC}$ | 24 | 60 | 120 | NoIns
---|---|---|---|---|---|---|---
% Caught, $d=18$ | 42.901% | -42.044% | -42.901% | +2.512% | -38.387% | -42.829% | -
Value (no IC), $d=18$ | 10.8196 | +3.490% | - | -0.644% | +3.6748% | +3.506% | +3.475%
Value (IC), $d=18$ | 10.7003 | - | +4.629% | -0.699% | +4.698% | +4.657% | -
% Caught, $d=22$ | 42.800% | +2.526% | +2.526% | +2.526% | -38.170% | -42.706% | -
Value (no IC), $d=22$ | 9.1597 | +0.128% | - | +0.128% | -6.731% | -8.249% | -8.273%
Value (IC), $d=22$ | 9.0403 | - | +0.063% | +0.063% | -5.929% | -7.041% | -
% Caught, $d=26$ | 42.665% | +22.864% | +22.864% | +2.595% | -38.101% | -42.592% | -
Value (no IC), $d=26$ | 7.4723 | +5.804% | - | +1.301% | -23.728% | -27.314% | -27.383%
Value (IC), $d=26$ | 7.3546 | - | +5.379% | +1.247% | -22.713% | -26.155% | -
% Caught, $d=30$ | 42.678% | +32.480% | +32.480% | +2.529% | -38.033% | -42.600% | -
Value (no IC), $d=30$ | 5.8242 | +21.050% | - | +2.725% | -48.632% | -55.469% | -55.757%
Value (IC), $d=30$ | 5.7064 | - | +20.689% | +2.686% | -47.825% | -54.443% | -
% Caught, $d=34$ | 43.005% | +38.272% | +38.272% | +2.478% | -38.390% | -42.915% | -
Value (no IC), $d=34$ | 4.3305 | +48.112% | - | +4.863% | -92.391% | -100.327% | -104.545%
Value (IC), $d=34$ | 4.2093 | - | +48.664% | +4.894% | -92.517% | -104.457% | -

Table 11: Percent of Unexpected Disruptive Events Caught and Average Value
Accumulated with Each Inspection Rule for $P^{3}(ni)$

We again randomly generate transition probabilities using a mean of
$P^{3}(ni)$ to test the robustness of the recommended inspection strategies.
For brevity, these results also appear in Appendix D. As the base transition
matrix is less prone to experiencing an unexpected disruptive event, the
uncertainty around the transition probabilities proves to be a challenge in
determining when is optimal to inspect. In particular, when $s=0.02$, the
value of waiting longer to inspect is often more than that of inspecting
earlier for the lower tested values of $d$. This suggests that, if the
facility is already expected to maintain higher quality for a longer time,
even when we are less certain of the true rate at which the facility’s quality
will degrade, it will still be acceptable to delay inspecting. However, it is
still important to better understand the rate at which facility’s quality will
degrade, as we may expect delaying inspecting will result in more value we
would actually gain.

###  6.2 Penalty Parameter Sensitivity Analysis

We now seek to understand how sensitive our inspection strategy is to changes
in the penalty parameters. Given a fixed transition probability matrix, we
test how $d$ needs to change such that the recommended inspection time
changes. Based on our experiments in Section 6.1, we explored what the
inspection time would need to be in order to catch between 50% and 90% of
disruptive events for each transition matrix tested. We wish to identify the
range of values that $d$ can take such that the inspection periods stay the
same, that is, how sensitive the inspection rules are to variations in $d$.
Figure 3 presents step plots of the ranges $d$ can lie in such that each
$t\in[4,22]$ is optimal for each transition matrix. The earliest inspection
time to catch at least 90% of unexpected disruptive events is $t=4$ with
$P^{2}(ni)$, while the latest to catch at least 50% is $t=22$ with
$P^{3}(ni)$.

Figure 3: Optimal inspection times with varying $d$

We observe that, for all three transition matrices, later inspection times are
more sensitive to small perturbations of $d$ than earlier inspection times.
This seems to be reasonable, as we believe the facility will be more likely to
experience an unexpected disruptive event the later we wait to inspect, making
changes in $d$ more impactful on later inspection times. Thus, if external
factors were to cause a facility to become more critical to the FDA - such as
if another facility producing a substitute-able product were to experience an
unexpected disruptive events - then it is important for the FDA to inspect the
operating facility if it has not been inspected in a long time. Additionally,
to be certain that a later inspection time is optimal, it is more critical
that the FDA has an accurate estimate of $d$. For earlier inspection times,
the FDA does not need to have as accurate an estimation of $d$.

##  7 Conclusion and Managerial Insights

In this work, we propose a novel POMDP approach to support the FDA in deciding
when to inspect manufacturing facilities. Our model utilizes the
classifications a facility can receive when inspected as the state space, and
weighs the trade-offs of the risk of an unexpected disruptive event with the
value gained from a facility staying in operation. Under realistic
assumptions, we are able to reduce the problem of determining when is the
optimal time to inspect to a series of simple computations comparing the value
of inspecting now against the value of inspecting after waiting a single time
period. Our numerical results show that the optimal inspection rules produced
by our models are able to achieve higher average values accumulated over the
time between inspections than other rules tested, especially when the penalty
for a manufacturing failure occurring is large. We found that our approach is
more robust to randomness in the transition probabilities for facilities that
are more likely to experience unexpected disruptive events.

Our work has some key insights for FDA decision makers. First, determining
when to inspect facilities independently from each other does not require
long-term planning; it is sufficient to analyze the trade-offs between
inspecting immediately and waiting until the next planning period to inspect.
Second, there is a quadratic relationship between the penalty for a
manufacturing failure occurring and the relative increase in the average value
accumulated when inspection time accounts for the impact a disruptive event
has on public health. The rate at which this quadratic grows is impacted by
the likelihood of experiencing an unexpected disruptive event. This highlights
the importance of dedicating more inspection resources to high-risk facilities
that produce drugs that are more critical to public health. It is also
important for the FDA to better understand how the quality of facilities
degrade over time. Third, uncertainty in the penalties on public health is
more impactful on determining the optimal inspection time when the FDA waits
longer to inspect. The FDA will need to ensure their estimation of the
penalties is accurate to verify that a later inspection time is optimal,
whereas the FDA can be less careful about this estimation in situations when
the optimal inspection time is shorter.

There are many avenues for future work based off of this initial model. One
such avenue is to consider multiple facilities simultaneously. The FDA may be
able to better mitigate for drug shortages if they are able to amend their
inspection policies to account for the operation statuses for multiple
facilities producing substitute-able products. Another avenue for future work
is to model how manufacturers react to FDA inspections. If the FDA can more
accurately anticipate whether or not manufacturers will comply with any
voluntary actions suggested by the FDA, they can better determine how strictly
a facility may need to be monitored and how harsh penalties for non-compliance
need to be to ensure compliance with the CGMP without forcing the facility to
close. Lastly, our theoretical results are a product of the dynamic structure
of the value functions and assumptions on the sequences of observation
probabilities, neither of which are exclusively tied to the update scheme for
belief probabilities in POMDPs. A third avenue of future research is to
investigate how inspections times would vary when observation probability
sequences that still satisfy our assumptions are generated by different
models.

## Acknowledgements

This material is based upon work partially supported by the National Science
Foundation under Grant No. 2228510.

## References

  * (1)
  * Ahmad & Kamaruddin (2012) Ahmad, R. & Kamaruddin, S. (2012), ‘An overview of time-based and condition-based maintenance in industrial application’, Computers and Industrial Engineering 63(1), 135–149.
  * Ahuja et al. (2021) Ahuja, V., Alvarez, C. A., Birge, J. R. & Syverson, C. (2021), ‘Enhancing regulatory decision making for postmarket drug safety’, Management Science 67(12), 7493–7510.
  * Anand et al. (2012) Anand, G., Gray, J. & Siemsen, E. (2012), ‘Decay, shock, and renewal: Operational routines and process entropy in the pharmaceutical industry’, Organization Science 23(6), 1700–1716.
  * Babich & Tang (2012) Babich, V. & Tang, C. S. (2012), ‘Managing opportunistic supplier product adulteration: Deferred payments, inspection, and combined mechanisms’, Manufacturing & Service Operations Management 14(2), 301–314.
  * Ball et al. (2017) Ball, G., Siemsen, E. & Shahc, R. (2017), ‘Do plant inspections predict future quality? the role of investigator experience’, Manufacturing and Service Operations Management 19(4), 534–550.
  * Brémaud (2020) Brémaud, P. (2020), Markov Chains, Vol. 31, Springer International Publishing, Cham, Switzerland.
  * Buckley et al. (2013) Buckley, G. J., Gostin, L. O. et al. (2013), Countering the problem of falsified and substandard drugs, National Academies Press, Washington, D.C., U.S.A.
  * Byon et al. (2010) Byon, E., Ntaimo, L. & Ding, Y. (2010), ‘Optimal maintenance strategies for wind turbine systems under stochastic weather conditions’, IEEE Transactions on Reliability 59(2), 393–404.
  * Center for Drug Evaluation and Research (2018) Center for Drug Evaluation and Research (2018), ‘Understanding CDER’s risk-based site selection model’.  Accessed March 31, 2023.
https://www.fda.gov/media/116004/download

  * Chang et al. (2017) Chang, Y., Erera, A. L. & White, C. C. (2017), ‘Risk assessment of deliberate contamination of food production facilities’, IEEE Transactions on Systems, Man, and Cybernetics: Systems 47(3), 381–393.
  * Cheng et al. (2023) Cheng, G., Li, L., Shangguan, C., Yang, N., Jiang, B. & Tao, N. (2023), ‘Optimal joint inspection and mission abort policy for a partially observable system’, Reliability Engineering and System Safety 229, 1 – 11.
  * Csardi & Nepusz (2006) Csardi, G. & Nepusz, T. (2006), ‘The iGraph software package for complex network research’, InterJournal, complex systems 1695(5), 1–9.
  * Edney (2019) Edney, A. (2019), ‘America’s love affair with cheap drugs has a hidden cost’.  Accessed March 31, 2023.
https://www.bloomberg.com/news/features/2019-01-29/america-s-love-affair-with-
cheap-drugs-has-a-hidden-cost

  * Ergun et al. (2022) Ergun, Ö., Griffin, J., Chicoine, N., Gong, M., Mohaddesi, O., Raziei, Z., Harteveld, C., Kaeli, D. & Marsella, S. (2022), Supply chain resilience: Impact of stakeholder behavior and trustworthy information sharing with a case study on pharmaceutical supply chains, in ‘Tutorials in Operations Research: Emerging and Impactful Topics in Operations’, INFORMS, Maryland, U.S.A., pp. 133–159.
  * Francas et al. (2023) Francas, D., Mohr, S. & Hoberg, K. (2023), ‘On the drivers of drug shortages: empirical evidence from germany’, International Journal of Operations & Production Management pp. 1–33.
  * Ge et al. (2007) Ge, H., Tomasevicz, C. L. & Asgarpoor, S. (2007), Optimum maintenance policy with inspection by semi-markov decision processes, in ‘2007 39th North American Power Symposium’, IEEE, Las Cruces, NM, U.S.A., pp. 541–546.
  * Ghandali et al. (2020) Ghandali, R., Abooie, M. H. & Nezhad, M. S. (2020), ‘A pomdp framework to find optimal policy in sustainable maintenance’, Scientia Iranica 27(3), 1544–1561.
  * Hein et al. (2019) Hein, N., Rantou, E. & Schuette, P. (2019), ‘Comparing methods for clinical investigator site inspection selection: a comparison of site selection methods of investigators in clinical trials’, Journal of Biopharmaceutical Statistics 29(5), 860–873.
  * Jensen et al. (2002) Jensen, V., Kimzey, L. M. & Goldberger, M. J. (2002), ‘FDA’s role in responding to drug shortages’, American journal of health-system pharmacy 59(15), 1423–1425.
  * Jin et al. (2021) Jin, C.-y., Retsef, L., Liang, Q., Renegar, N. & Zhou, J.-h. (2021), ‘Food safety inspection and the adoption of traceability in aquatic wholesale markets: A game-theoretic model and empirical evidence’, Journal of Integrative Agriculture 20(10), 2807–2819.
  * Kimball (2023) Kimball, S. (2023), ‘FDA allows temporary import of unapproved Chinese cancer drug to ease U.S. shortage’.  Accessed June 5, 2023.
https://www.cnbc.com/2023/06/02/cancer-drug-shortage-fda-allows-import-of-
unapproved-china-chemo-med.html

  * Klimberg et al. (1992) Klimberg, R., Revelle, C. & Cohon, J. (1992), ‘Improving the effectiveness of FDA drug inspection’, Operations Research 40(5), 845–855.
  * Krishnamurthy (2016) Krishnamurthy, V. (2016), Partially Observed Markov Decision Processes, Cambridge University Press, Cambridge, U.K.
  * Levi et al. (2019) Levi, R., Renegar, N., Springs, S. & Zaman, T. (2019), ‘Supply chain network analytics guiding food regulatory operational policy’, Available at SSRN 3374620 .
  * Levitin et al. (2019) Levitin, G., Xing, L. & Huang, H. Z. (2019), ‘Cost effective scheduling of imperfect inspections in systems with hidden failures and rescue possibility’, Applied Mathematical Modelling 68, 662–674.
  * Liu et al. (2023) Liu, Y., Gao, J., Jiang, T. & Zeng, Z. (2023), ‘Selective maintenance and inspection optimization for partially observable systems: An interactively sequential decision framework’, IISE Transactions 55(5), 463–479.
  * Macher et al. (2006) Macher, J. T., Mayo, J. W. & Nickerson, J. A. (2006), Exploring the information asymmetry gap: Evidence from FDA regulation, Technical report.
https://www.researchgate.net/publication/228664514

  * Morato et al. (2022) Morato, P. G., Papakonstantinou, K. G., Andriotis, C. P., Nielsen, J. S. & Rigo, P. (2022), ‘Optimal inspection and maintenance planning for deteriorating structural components through dynamic bayesian networks and markov decision processes’, Structural Safety 94, 1–16.
  * Myers (2009) Myers, A. (2009), ‘Probability of loss assessment of critical k-out-of-n:g systems having a mission abort policy’, IEEE Transactions on Reliability 58(4), 694–701.
  * National Academies of Sciences, Engineering, and Medicine (2022) National Academies of Sciences, Engineering, and Medicine (2022), Building Resilience into the Nation’s Medical Product Supply Chains, National Academies Press, Washington, D.C., U.S.A.
  * Papakonstantinou & Shinozuka (2014a) Papakonstantinou, K. G. & Shinozuka, M. (2014a), ‘Optimum inspection and maintenance policies for corroded structures using partially observable markov decision processes and stochastic, physically based models’, Probabilistic Engineering Mechanics 37, 93–108.
  * Papakonstantinou & Shinozuka (2014b) Papakonstantinou, K. G. & Shinozuka, M. (2014b), ‘Planning structural inspection and maintenance policies via dynamic programming and markov processes. part i: Theory’, Reliability Engineering and System Safety 130, 202–213.
  * Papakonstantinou & Shinozuka (2014c) Papakonstantinou, K. G. & Shinozuka, M. (2014c), ‘Planning structural inspection and maintenance policies via dynamic programming and markov processes. part ii: Pomdp implementation’, Reliability Engineering and System Safety 130, 214–224.
  * Pazhayattil et al. (2020) Pazhayattil, A. B., Ingram, M. & Sayeed, N. (2020), ‘A quantitative study of US FDA inspection data for drug manufacturing sites’, Therapeutic Innovation and Regulatory Science 54(4), 725–730.
  * Pew Health Group (2012) Pew Health Group (2012), ‘Heparin: A wake-up call on risks to the u.s. drug supply’.  Accessed June 1, 2023.
https://www.pewtrusts.org/en/research-and-analysis/issue-
briefs/2012/05/16/heparin-a-wakeup-call-on-risks-to-the-us-drug-supply

  * Qiu & Cui (2019) Qiu, Q. & Cui, L. (2019), ‘Gamma process based optimal mission abort policy’, Reliability Engineering and System Safety 190, 1–9.
  * Qiu et al. (2023) Qiu, Q., Maillart, L. M., Prokopyev, O. A. & Cui, L. (2023), ‘Optimal condition-based mission abort decisions’, IEEE Transactions on Reliability 72(1), 408–425.
  * Tucker et al. (2020) Tucker, E. L., Cao, Y., Fox, E. R. & Sweet, B. V. (2020), ‘The drug shortage era: a scoping review of the literature 2001–2019’, Clinical Pharmacology & Therapeutics 108(6), 1150–1155.
  * U.S. Food & Drug Administration (2018) U.S. Food & Drug Administration (2018), ‘What does FDA inspect?’.  Accessed March 31, 2023.
https://www.fda.gov/about-fda/fda-basics/what-does-fda-inspect

  * U.S. Food & Drug Administration (2019) U.S. Food & Drug Administration (2019), ‘Drug shortages: Root causes and potential solutions’.  Accessed June 1, 2023.
https://www.fda.gov/media/131130/download

  * U.S. Food & Drug Administration (2020a) U.S. Food & Drug Administration (2020a), ‘Eighth annual report on drug shortages for calendar year 2020’.  Accessed June 1, 2023.
https://www.fda.gov/media/150409/download

  * U.S. Food & Drug Administration (2020b) U.S. Food & Drug Administration (2020b), ‘Inspections database frequently asked questions’.  Accessed March 31, 2023.
https://www.fda.gov/inspections-compliance-enforcement-and-criminal-
investigations/inspection-references/inspections-database-frequently-asked-
questions

  * U.S. Food & Drug Administration (2020c) U.S. Food & Drug Administration (2020c), ‘Notifying FDA of a permanent discontinuance or interruption in manufacturing under section 506c of the FD&C Act’.  Accessed June 1, 2023.
https://www.fda.gov/media/136486/download

  * U.S. Food & Drug Administration (2023) U.S. Food & Drug Administration (2023), ‘Current good manufacturing practice (CGMP) regulations’.  Accessed May 31, 2023.
https://www.fda.gov/drugs/pharmaceutical-quality-resources/current-good-
manufacturing-practice-cgmp-regulations

  * U.S. Senate Committee on Homeland Security & Governmental Affairs (2023) U.S. Senate Committee on Homeland Security & Governmental Affairs (2023), ‘Short suply: The health and national security risks of drug shortages’.  Accessed June 1, 2023.
https://www.hsgac.senate.gov/wp-content/uploads/2023-06-06-HSGAC-Majority-
Draft-Drug-Shortages-Report.-FINAL-CORRECTED.pdf

  * Ventola (2011) Ventola, C. L. (2011), ‘The drug shortage crisis in the United States: causes, impact, and management strategies’, Pharmacy and Therapeutics 36(11), 740.
  * Wang et al. (2023) Wang, Y. I., Li, J. & Anupindi, R. (2023), ‘Manufacturing and regulatory barriers to generic drug competition: A structural model approach’, Management Science 69(3), 1449–1467.
  * White (2020) White, C. M. (2020), ‘Generic drugs not as safe as FDA wants you to believe’, Annals of Pharmacotherapy 54(3), 283–286.
  * Woodcock (2019) Woodcock, J. (2019), ‘Securing the U.S. drug supply chain: Oversight of FDA’s foreign inspection program’.  Accessed March 31, 2023.
https://www.fda.gov/news-events/congressional-testimony/securing-us-drug-
supply-chain-oversight-fdas-foreign-inspection-program-12102019

  * Woodcock & Kopcha (2020) Woodcock, J. & Kopcha, M. (2020), ‘Quality: The often overlooked critical element for assuring access to safe and effective drugs’, Health Affairs Forefront .
  * Wu & Wang (2020) Wu, A. & Wang, Y. I. (2020), ‘The more monitoring, the better quality? Empirical evidence from the generic drug industry’, Available at SSRN 3596559 .
  * Xu et al. (2022) Xu, J., Zhao, X. & Liu, B. (2022), ‘A risk-aware maintenance model based on a constrained markov decision process’, IISE Transactions 54(11), 1072–1083.
  * Zhang & Revie (2017) Zhang, M. & Revie, M. (2017), ‘Continuous-observation partially observable semi-markov decision processes for machine maintenance’, IEEE Transactions on Reliability 66(1), 202–218.
  * Zhang et al. (2019) Zhang, M., Zhang, J., Cheng, T. C., Hua, G., Yan, X. & Liu, Y. (2019), ‘The effect of unannounced inspection on prevention of drug fraud’, Journal of Systems Science and Systems Engineering 28(1), 63–90.
  * Zhao et al. (2021) Zhao, X., Sun, J., Qiu, Q. & Chen, K. (2021), ‘Optimal inspection and mission abort policies for systems subject to degradation’, European Journal of Operational Research 292(2), 610–621.
  * Zhou et al. (2022) Zhou, J., Jin, Y. & Liang, Q. (2022), ‘Effects of regulatory policy mixes on traceability adoption in wholesale markets: Food safety inspection and information disclosure’, Food Policy 107, 1–11.
  * Zhu & Xiang (2021) Zhu, Z. & Xiang, Y. (2021), ‘Condition-based maintenance for multi-component systems: Modeling, structural properties, and algorithms’, IISE Transactions 53(1), 88–100.
  * Zouch et al. (2011) Zouch, M., Yeung, T. G. & Castanier, B. (2011), Optimal resurfacing decisions for road maintenance: A pomdp perspective, in ‘2011 Proceedings-Annual Reliability and Maintainability Symposium’, IEEE, Lake Buena Vista, FL, U.S.A., pp. 1–6.

##  Appendix A Summary of Notation

Parameter | Description of Parameter
---|---
$T$ | Time Horizon
$d$ |  Penalty for being in the “Experienced a Manufacturing Failure” state $D$
$c$ |  Penalty for being in the “Closed For Non-mandatory Maintenance” state $C$
$\tilde{c}$ | Penalty for the manufacturer closing for mandatory maintenance after an inspection
$\alpha_{S}$ |  Factor by which $\tilde{c}$ is smaller than the penalty of being in state $S\in\\{D,C\\}$
$P$ | Transition probability matrix
$p_{SS^{\prime}}$ |  Probability of transitioning from state $S$ to state $S^{\prime}$
$p_{SIC}$ |  | Probability of the manufacturer closing after an inspection occurs
---
when inspected in state $S$
$b^{t}_{S}$ |  Belief probability that we are in state $S$ at time $t$
$P[(dr)]^{t}$ |  Probability of experiencing a disrupting based on belief probabilities $b^{t}$
$P[(cr)]^{t}$ |  | Probability of experiencing a closure without intervention
---
based on belief probabilities $b^{t}$
$P[(nr)]^{t}$ |  | Probability of no issues being reported by the manufacturer
---
based on belief probabilities $b^{t}$
$P[(cr)|(i)]^{t}$ |  Probability of the facility closing due to inspection based on belief probabilities $b^{t}$
$\sigma^{t}_{j}$ |  Conditional plan starting at $t$ and inspecting during time period $t+j$
$V^{t}(b,\sigma)$ |  |  Cumulative expected value accrued from time period $t$ to $T$,
---
starting with belief probability vector $b$ and implementing conditional plan
$\sigma$
$\Theta^{i}_{SS^{\prime}}$ |  Set of all paths from state $S$ to state $S^{\prime}$ in $i-1$ time periods
$\theta^{i}$ |  Path from state $\theta_{1}^{i}$ to state $\theta^{i}_{i}$
$k_{S}$ |  Expected reward after waiting one time period, starting from state $S$
$k_{SS^{\prime}}^{t}$ |  |  Expected reward after waiting $t-1$ time periods,
---
starting from state $S$ and ending at state $S^{\prime}$ at time $T-1$
$f_{\theta^{i}}$ |  Recursive parameter based on transition probabilities following the path $\theta^{i}$
$\tilde{c}_{SS^{\prime}IC}^{i}$ |  | Expected penalty from closing after an inspection
---
when inspecting after $i-1$ time periods, starting at state $S$ and inspecting
in state $S^{\prime}$
Table 12: Description of model parameters Parameter | Description of Parameter
---|---
$t^{*}$ | optimal inspection time
$\delta$ |  |  Minimum increase in $d$ such that inspecting one time period earlier
---
than $t^{*}$ becomes optimal
$\gamma$ |  |  Minimum increase in $c$ such that inspecting one time period earlier
---
than $t^{*}$ becomes optimal
Table 13: Description of sensitivity analysis parameters Parameter | Description of Parameter
---|---
$\mu_{S}$ |  Expected time until transitioning into an absorbing state starting at state $S$
$t_{V}$ | Inspection time recommended by base POMDP model
$t_{VC}$ | Inspection time recommended by POMDP model with closure due to inspection
$t_{E}$ | Inspection time determined by the expected time until reaching an absorbing state
$\tilde{P}$ | Randomly generated transition probability matrix using normal distribution
$\tilde{p}_{ij}$ |  Randomly generated probability of transition from state $i$ to state $j$
$s$ | standard deviation
$d_{L}$ |  minimum value for $d$ where inspecting in a given time period is optimal
$d_{U}$ |  maximum value for $d$ where inspecting in a given time period is optimal
Table 14: Description of sensitivity analysis parameters

##  Appendix B Proofs

Proof of Theorem 1:

###### Proof.

We prove Equations (1) and (2) are equivalent by induction. First, consider
$t=T-2$. For Equation (1), we have

$\displaystyle V^{T-2}(b^{T-2},\sigma_{2}^{T-2})$
$\displaystyle=1-P[(dr)]^{T-1}d-P[(cr)]^{T-1}c+P[(nr)]^{T-1}V^{T-1}(b^{T-1},\sigma_{1}^{T-1})$
$\displaystyle=1-P[(dr)]^{T-1}d-P[(cr)]^{T-1}c+P[(nr)]^{T-1}(1+k_{N}b^{T-1}_{N}+k_{V}b^{T-1}_{V}+k_{O}b^{T-1}_{O})$
$\displaystyle=1-P[(dr)]^{T-1}d-P[(cr)]^{T-1}c+P[(nr)]^{T-1}$
$\displaystyle+k_{N}(p_{NN}b_{N}^{T-2})+k_{V}(p_{NV}b_{N}^{T-2}+p_{VV}b_{V}^{T-2})+k_{O}(p_{NO}b_{N}^{T-2}+p_{VO}b_{V}^{T-2}+p_{OO}b_{O}^{T-2})$
$\displaystyle=1+(k_{N}b_{N}^{T-2}+k_{V}b_{V}^{T-2}+k_{O}b_{O}^{T-2})$
$\displaystyle+(k_{O}p_{NO}+k_{V}p_{NV}+k_{N}p_{NN})b_{N}^{T-2}+(k_{O}p_{VO}^{T-2}+k_{V}p_{VV})b_{V}^{T-2}+k_{O}p_{OO}b_{O}^{T-2}$
$\displaystyle=1+(k_{O}p_{NO}+k_{V}p_{NV}+k_{N}(p_{NN}+1))b_{N}^{T-2}$
$\displaystyle+(k_{O}p_{VO}+k_{V}(p_{VV}+1))b_{V}^{T-2}+k_{O}(p_{OO}+1)b_{O}^{T-2}.$

For Equation (2), we first note that
$\Theta^{2}=\\{(N,N),(N,V),(N,O),(V,V),(V,O),(O,O)\\}$. By Equation (2), we
have

$\displaystyle V^{T-2}(b^{T-2},\sigma_{2}^{T-2})$
$\displaystyle=1+k_{N}(p_{NN}+1)b_{N}^{T-2}+k_{V}p_{NV}b_{N}^{T-2}+k_{O}p_{NO}b_{N}^{T-2}$
$\displaystyle+k_{V}(p_{VV}+1)b_{V}^{T-2}+k_{O}p_{VO}b_{V}^{T-2}+k_{O}(p_{OO}+1)b_{O}^{T-2}$
$\displaystyle=1+(k_{O}p_{NO}+k_{V}p_{NV}+k_{N}(p_{NN}+1))b_{N}^{T-2}$
$\displaystyle+(k_{O}p_{VO}+k_{V}(p_{VV}+1))b_{V}^{T-2}+k_{O}(p_{OO}+1)b_{O}^{T-2}.$

Thus, for $t=T-2$, Equations (1) and (2) are equivalent. Next, suppose that
Equations (1) and (2) are equivalent for $t=T-j,\ldots,T-2$. We now show they
are also equivalent for $t=T-(j+1)$.

First, note that for any $\theta^{j+1}\in\Theta^{j+1}_{SS^{\prime}}$, there is
a $\bar{\theta}^{j}\in\Theta^{j}_{\theta^{j+1}_{2}S^{\prime}}$ such that
$\theta^{j+1}=(\theta^{j+1}_{1},\bar{\theta}^{j})$. Thus,

$\displaystyle V^{T-(j+1)}(b^{T-(j+1)},\sigma_{j+1}^{T-(j+1)})$
$\displaystyle=1-P[(dr)]^{T-j}d-P[(cr)]^{T-j}c+P[(nr)]^{T-j}V^{T-j}(b^{T-j},\sigma_{j}^{T-j})$
$\displaystyle=1-P[(dr)]^{T-j}d-P[(cr)]^{T-j}c+P[(nr)]^{T-j}(1+\sum_{S\in\\{N,V,O\\}}\sum_{\begin{subarray}{c}S^{\prime}\in\\{N,V,O\\}\\\
S^{\prime}\text{ is reachable from
}S\end{subarray}}k_{SS^{\prime}}^{j}b^{T-j}_{S})$
$\displaystyle=1+k_{N}b_{N}^{T-(j+1)}+k_{V}b_{V}^{T-(j+1)}+k_{O}b_{O}^{T-(j+1)}+\sum_{S^{\prime}\in\\{N,V,O\\}}k_{NS^{\prime}}^{j}(p_{NN}b_{N}^{T-(j+1)})$
$\displaystyle+\sum_{S\in\\{V,O\\}}k_{VS^{\prime}}^{j}(p_{NV}b_{N}^{T-(j+1)}+p_{VV}b_{V}^{T-(j+1)})+k_{OO}^{j}(p_{NO}b_{N}^{T-(j+1)}+p_{VO}b_{V}^{T-(j+1)}+p_{OO}b_{O}^{T-(j+1)})$
$\displaystyle=1+(k_{N}+\sum_{S^{\prime}\in\\{N,V,O\\}}k_{NS^{\prime}}^{j}p_{NN}+\sum_{S\in\\{V,O\\}}k_{VS^{\prime}}^{j}p_{NV}+k_{OO}^{j}p_{NO})b_{N}^{T-(j+1)}$
$\displaystyle+(k_{V}+\sum_{S\in\\{V,O\\}}k_{VS^{\prime}}^{j}p_{VV}+k_{OO}^{j}p_{VO})b_{V}^{T-(j+1)}+(k_{O}+k_{OO}^{j}p_{OO})b_{O}^{T-(j+1)}.$

For $S,S^{\prime}\in\\{N,V,O\\}$ where $S\neq S^{\prime}$, we have that

$\displaystyle k_{SS^{\prime}}^{j+1}$
$\displaystyle=k_{S^{\prime}}\sum_{\theta^{j+1}\in\Theta^{j+1}_{SS^{\prime}}}f_{\theta^{j+1}}$
$\displaystyle=k_{S^{\prime}}\sum_{\begin{subarray}{c}\theta^{j+1}_{2}\in\\{N,V,O\\}\\\
\theta^{j+1}_{2}\text{ is reachable from
}S\end{subarray}}p_{S\theta^{j+1}_{2}}\left(\sum_{\bar{\theta}^{j}\in\Theta^{j}_{\theta^{j+1}_{2}S^{\prime}}}f_{\bar{\theta}^{j}}\right)$
$\displaystyle=\sum_{\begin{subarray}{c}\theta^{j+1}_{2}\in\\{N,V,O\\}\\\
\theta^{j+1}_{2}\text{ is reachable from
}S\end{subarray}}p_{S\theta^{j+1}_{2}}\left(k_{S}^{\prime}\sum_{\bar{\theta}^{j}\in\Theta^{j}_{\theta^{j+1}_{2}S^{\prime}}}f_{\bar{\theta}^{j}}\right)$
$\displaystyle=\sum_{\begin{subarray}{c}\theta^{j+1}_{2}\in\\{N,V,O\\}\\\
\theta^{j+1}_{2}\text{ is reachable from
}S\end{subarray}}k_{\bar{\theta}^{j}_{2}S^{\prime}}p_{S\theta^{j+1}_{2}}.$

Likewise, when $S=S^{\prime}$, we have that

$\displaystyle k_{SS}^{j+1}$
$\displaystyle=k_{S}f_{(S,S,\ldots,S)}^{j+1}=k_{S}(p_{SS}f_{(S,\ldots,S)}^{j}+1)=k_{S}+k_{SS}^{j}p_{SS}$

Thus,

$\displaystyle
1+(k_{N}+\sum_{S^{\prime}\in\\{N,V,O\\}}k_{NS^{\prime}}^{i}p_{NN}+\sum_{S\in\\{V,O\\}}k_{VS^{\prime}}^{i}p_{NV}+k_{OO}^{i}p_{NO})b_{N}^{T-(j+1)}$
$\displaystyle+(k_{V}+\sum_{S\in\\{V,O\\}}k_{VS^{\prime}}^{j}p_{VV}+k_{OO}^{j}p_{VO})b_{V}^{T-(j+1)}+(k_{O}+k_{OO}^{j}p_{OO})b_{O}^{T-(j+1)}$
$\displaystyle=1+\sum_{S\in\\{N,V,O\\}}\sum_{\begin{subarray}{c}S^{\prime}\in\\{N,V,O\\}\\\
S^{\prime}\text{ is reachable from
}S\end{subarray}}k_{SS^{\prime}}^{j+1}b_{S}^{T-(j+1)}.$

Thus, for $t=T-(j+1)$, Equations (1) and (2) are equivalent. ∎

Proof of Lemma 1:

###### Proof.

Suppose that $(P[(dr)]^{t})$ and $(P[(cr)]^{t})$ are non-decreasing sequences
and that $(P[(nr)]^{t})$ is a non-increasing sequence. We prove that if
$V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$ implies
$V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$ by
induction. First consider $j=0$. Suppose that $V^{t}(b^{t},(i))\geq
V^{t}(b^{t},(ni,i))$, and recall that $V^{t}(b^{t},(i))=1$ for all $t$. We
have that

$\displaystyle V^{t+1}(b^{t+1},(i))=1$ $\displaystyle\geq V^{t}(b^{t},(ni,i))$
$\displaystyle=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}$
$\displaystyle\geq 1-P[(dr)]^{t+2}d-P[(cr)]^{t+2}c+P[(nr)]^{t+2}$
$\displaystyle=V^{t+1}(b^{t+1},(ni,i))$

Next, suppose $V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$
implies $V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq
V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$ for $0\leq j\leq l<T-t$. We now show that
if $V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$, then
$V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$
for $j=l+1$. Suppose $V^{t}(b^{t},\sigma_{l+1}^{t})\geq
V^{t}(b^{t},\sigma_{l+2}^{t})$, and note that

$\displaystyle V^{t}(b^{t},\sigma_{l+1}^{t})\geq
V^{t}(b^{t},\sigma_{l+2}^{t})$ $\displaystyle
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))$
$\displaystyle\geq
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}))$
$\displaystyle V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))\geq
V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}).$

Since $V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))\geq
V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})$, then
$V^{t+2}(b^{t+2},\sigma_{l}^{t+2}))\geq V^{t+2}(b^{t+2},\sigma_{l+1}^{t+2})$
by our inductive hypothesis. Thus,

$\displaystyle V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})$
$\displaystyle=1-P[(dr)]^{t+2}d-P[(cr)]^{t+2}c+P[(nr)]^{t+2}(V^{t+2}(b^{t+2},\sigma_{l}^{t+2}))$
$\displaystyle\geq
1-P[(dr)]^{t+2}d-P[(cr)]^{t+2}c+P[(nr)]^{t+2}(V^{t+2}(b^{t+2},\sigma_{l+1}^{t+2}))$
$\displaystyle=V^{t+1}(b^{t+1},\sigma_{l+2}^{t+1}).$

We thus have that if $V^{t}(b^{t},\sigma_{j}^{t})\geq
V^{t}(b^{t},\sigma_{j+1}^{t})$ implies that
$V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$,
then $V^{t}(b^{t},\sigma_{j+1}^{t})\geq V^{t}(b^{t},\sigma_{j+2}^{t})$ implies
that $V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})\geq
V^{t+1}(b^{t+1},\sigma_{j+2}^{t+1})$.

Similarly, we prove that if $V^{t}(b^{t},\sigma_{j}^{t})\leq
V^{t}(b^{t},\sigma_{j+1}^{t})$, then $V^{t-1}(b^{t-1},\sigma_{j}^{t-1})\leq
V^{t-1}(b^{t-1},\sigma_{j+1}^{t})$ by induction on $j$. First consider $j=0$.
Suppose that $V^{t}(b^{t},(i))\leq V^{t}(b^{t},(ni,i))$. We have that

$\displaystyle V^{t-1}(b^{t+1},(i))=1$ $\displaystyle\leq V^{t}(b^{t},(ni,i))$
$\displaystyle=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}$
$\displaystyle\leq 1-P[(dr)]^{t}d-P[(cr)]^{t}c+P[(nr)]^{t}$
$\displaystyle=V^{t-1}(b^{t-1},(ni,i))$

Next, suppose that $V^{t}(b^{t},\sigma_{j}^{t})\leq
V^{t}(b^{t},\sigma_{j+1}^{t})$ implies $V^{t-1}(b^{t-1},\sigma_{j}^{t-1})\geq
V^{t-1}(b^{t-1},\sigma_{j+1}^{t-1})$ for $0\leq j\leq l<T-t$. We know show
$V^{t}(b^{t},\sigma_{j}^{t})\leq V^{t}(b^{t},\sigma_{j+1}^{t})$ implies
$V^{t-1}(b^{t-1},\sigma_{j}^{t-1})\geq V^{t-1}(b^{t-1},\sigma_{j+1}^{t-1})$
for $j=l+1$. Suppose $V^{t}(b^{t},\sigma_{l+1}^{t})\leq
V^{t}(b^{t},\sigma_{l+2}^{t})$, and note that

$\displaystyle V^{t}(b^{t},\sigma_{l+1}^{t})\leq
V^{t}(b^{t},\sigma_{l+2}^{t})$ $\displaystyle
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))$
$\displaystyle\leq
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}))$
$\displaystyle V^{t}(b^{t+1},\sigma_{l}^{t+1}))\leq
V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}).$

Since $V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))\leq
V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})$, then $V^{t}(b^{t},\sigma_{l}^{t}))\leq
V^{t}(b^{t},\sigma_{l+1}^{t})$ by our inductive hypothesis. Thus,

$\displaystyle V^{t-1}(b^{t-1},\sigma_{l+1}^{t-1})$
$\displaystyle=1-P[(dr)]^{t}d-P[(cr)]^{t}c+P[(nr)]^{t}(V^{t}(b^{t},\sigma_{l}^{t}))$
$\displaystyle\leq
1-P[(dr)]^{t}d-P[(cr)]^{t}c+P[(nr)]^{t}(V^{t}(b^{t},\sigma_{l+1}^{t}))$
$\displaystyle=V^{t-1}(b^{t-1},\sigma_{l+2}^{t-1}).$

We thus have that if $V^{t}(b^{t},\sigma_{j}^{t})\leq
V^{t}(b^{t},\sigma_{j+1}^{t})$, then $V^{t-1}(b^{t-1},\sigma_{j}^{t-1})\leq
V^{t-1}(b^{t-1},\sigma_{j+1}^{t})$, then $V^{t}(b^{t},\sigma_{j+1}^{t})\leq
V^{t}(b^{t},\sigma_{j+2}^{t})$, then $V^{t-1}(b^{t-1},\sigma_{j+1}^{t-1})\leq
V^{t-1}(b^{t-1},\sigma_{j+2}^{t})$. ∎

Proof of Theorem 2:

###### Proof.

Suppose $(P[(dr)]^{t})$ and $(P[(cr)]^{t})$ are non-decreasing sequences,
$(P[(nr)]^{t})$ is a non-increasing sequence and there exists a $t$ such that
$V^{t}(b^{t},(i))\geq V^{t}(b^{t},(ni,i))$, and recall that
$V^{t}(b^{t},(i))=1$. We show that $V^{t}(b^{t},\sigma_{j}^{t})\geq
V^{t}(b^{t},\sigma_{j+1}^{t})$ for $1\leq j<T-t$ be induction. First consider
$j=1$. By Lemma 1, we have that $V^{t+1}(b^{t+1},(i))\geq
V^{t+1}(b^{t+1},(ni,i))$. We have that

$\displaystyle V^{t}(b^{t},(ni,ni,i))$
$\displaystyle=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},(ni,i)))$
$\displaystyle\leq
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},(i)))$
$\displaystyle\leq 1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}$
$\displaystyle=V^{t}(b^{t},(ni,i))$

Thus, $V^{t}(b^{t},(ni,i))\geq V^{t}(b^{t},(ni,ni,i))$.

Now suppose $V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$
for $1\leq j\leq l<T-t$. We now show $V^{t}(b^{t},\sigma_{j}^{t})\geq
V^{t}(b^{t},\sigma_{j+1}^{t})$ for $j=l+1$. Suppose
$V^{t}(b^{t},\sigma_{l}^{t})\geq V^{t}(b^{t},\sigma_{l+1}^{t})$. We know that
$V^{t+1}(b^{t+1},\sigma_{l}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})$ by
Lemma 1. We have that

$\displaystyle V^{t}(b^{t},\sigma_{l+1}^{t})$
$\displaystyle=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))$
$\displaystyle\geq
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}))$
$\displaystyle=V^{t}(b^{t},\sigma_{l+2}^{t})$

Thus, if $V^{t}(b^{t},\sigma_{l}^{t})\geq V^{t}(b^{t},\sigma_{l+1}^{t})$, then
$V^{t}(b^{t},\sigma_{l+1}^{t})\geq V^{t}(b^{t},\sigma_{l+2}^{t})$. ∎

Proof of Theorem 3:

###### Proof.

We prove Equations (1) and (3) are equivalent by induction. First, consider
$t=T-2$. For Equation 1,

$\displaystyle V^{T-2}(b^{T-2},\sigma_{2}^{T-2})$
$\displaystyle=1-P[(dr)]^{T-1}d-P[(cr)]^{T-1}c+P[(nr)]^{T-1}V^{T-1}(b^{T-1},\sigma_{1}^{T-1})$
$\displaystyle=1-P[(dr)]^{T-1}d-P[(cr)]^{T-1}c$
$\displaystyle+P[(nr)]^{T-1}(1+(k_{N}-\tilde{c}(p_{NN}p_{NIC}+p_{NV}p_{VIC}+p_{NO}p_{OIC}))b_{N}^{T-1}$
$\displaystyle+(k_{V}-\tilde{c}(p_{VV}p_{VIC}+p_{VO}p_{OIC}))b_{V}^{T-1}+(k_{O}-\tilde{c}p_{OO}p_{OIC})b_{O}^{T-1})$
$\displaystyle=1-P[(dr)]^{T-1}d-P[(cr)]^{T-1}c+P[(nr)]^{T-1}$
$\displaystyle+(k_{N}-\tilde{c}(p_{NN}p_{NIC}+p_{NV}p_{VIC}+p_{NO}p_{OIC}))(p_{NN}b_{N}^{T-2})$
$\displaystyle+(k_{V}-\tilde{c}(p_{VV}p_{VIC}+p_{VO}p_{OIC}))(p_{NV}b_{N}^{T-2}+p_{VV}b_{V}^{T-2})$
$\displaystyle+(k_{O}-\tilde{c}p_{OO}p_{OIC})(p_{NO}b_{N}^{T-2}+p_{VO}b_{V}^{T-2}+p_{OO}b_{O}^{T-2})$
$\displaystyle=1+(k_{N}b_{N}^{T-2}+k_{V}b_{V}^{T-2}+k_{O}b_{O}^{T-2})$
$\displaystyle+(k_{O}p_{NO}+k_{V}p_{NV}+k_{N}p_{NN}-\tilde{c}(p_{NO}p_{OO}p_{OIC}+p_{NV}p_{VO}p_{OIC}+p_{NV}p_{VV}p_{VIC}$
$\displaystyle+p_{NN}p_{NO}p_{OIC}+p_{NN}p_{NV}p_{VIC}+p_{NN}p_{NN}p_{NIC}))b_{N}^{T-2}$
$\displaystyle+(k_{O}p_{VO}+k_{V}p_{VV}-\tilde{c}(p_{VO}p_{OO}p_{OIC}+p_{VV}p_{VO}p_{OIC}+p_{VV}p_{VV}p_{VIC}))b_{V}^{T-2}$
$\displaystyle+(k_{O}p_{OO}-\tilde{c}p_{OO}p_{OO}p_{OIC})b_{O}^{T-2}$
$\displaystyle=1+(k_{O}p_{NO}+k_{V}p_{NV}+k_{N}(p_{NN}+1)-\tilde{c}(p_{NO}p_{OO}p_{OIC}+p_{NV}p_{VO}p_{OIC}$
$\displaystyle+p_{NV}p_{VV}p_{VIC}+p_{NN}p_{NO}p_{OIC}+p_{NN}p_{NV}p_{VIC}+p_{NN}p_{NN}p_{NIC}))b_{N}^{T-2}$
$\displaystyle+(k_{O}p_{VO}+k_{V}(p_{VV}+1)-\tilde{c}(p_{VO}p_{OO}p_{OIC}+p_{VV}p_{VO}p_{OIC}+p_{VV}p_{VV}p_{VIC}))b_{V}^{T-2}$
$\displaystyle+(k_{O}(p_{OO}+1)-\tilde{c}p_{OO}p_{OO}p_{OIC})b_{O}^{T-2}.$

For Equation (3), we first note that
$\Theta^{2}=\\{(N,N),(N,V),(N,O),(V,V),(V,O),(O,O)\\}$ and

$\displaystyle\Theta^{3}=\\{\,$
$\displaystyle(N,N,N),(N,N,V),(N,N,O),(N,V,V),(N,V,O),$
$\displaystyle(N,O,O),(V,V,V),(V,V,O),(V,O,O),(O,O,O)\,\\}\enspace.$

By Equation (3), we have

$\displaystyle V^{T-2}(b^{T-2},\sigma_{2}^{T-2})$
$\displaystyle=1+k_{N}(p_{NN}+1)b_{N}^{T-2}+k_{V}p_{NV}b_{N}^{T-2}+k_{O}p_{NO}b_{N}^{T-2}$
$\displaystyle+k_{V}(p_{VV}+1)b_{V}^{T-2}+k_{O}p_{VO}b_{V}^{T-2}+k_{O}(p_{OO}+1)b_{O}^{T-2}$
$\displaystyle-\tilde{c}(p_{NO}p_{OO}p_{OIC}+p_{NV}p_{VO}p_{OIC}+p_{NV}p_{VV}p_{VIC}$
$\displaystyle+p_{NN}p_{NO}p_{OIC}+p_{NN}p_{NV}p_{VIC}+p_{NN}p_{NN}p_{NIC}))b_{N}^{T-2}$
$\displaystyle-\tilde{c}(p_{VO}p_{OO}p_{OIC}+p_{VV}p_{VO}p_{OIC}+p_{VV}p_{VV}p_{VIC}))b_{V}^{T-2}-\tilde{c}p_{OO}p_{OO}p_{OIC}b_{O}^{T-2}$
$\displaystyle=1+(k_{O}p_{NO}+k_{V}p_{NV}+k_{N}(p_{NN}+1)-\tilde{c}(p_{NO}p_{OO}p_{OIC}+p_{NV}p_{VO}p_{OIC}$
$\displaystyle+p_{NV}p_{VV}p_{VIC}+p_{NN}p_{NO}p_{OIC}+p_{NN}p_{NV}p_{VIC}+p_{NN}p_{NN}p_{NIC}))b_{N}^{T-2}$
$\displaystyle+(k_{O}p_{VO}+k_{V}(p_{VV}+1)-\tilde{c}(p_{VO}p_{OO}p_{OIC}+p_{VV}p_{VO}p_{OIC}+p_{VV}p_{VV}p_{VIC}))b_{V}^{T-2}$
$\displaystyle+(k_{O}(p_{OO}+1)-\tilde{c}p_{OO}p_{OO}p_{OIC})b_{O}^{T-2}.$

Thus, for $t=T-2$, Equations (1) and (2) are equivalent.

Suppose that Equations (1) and (2) are equivalent for $t=T-j,\ldots,T-2$. We
now show they are also equivalent for $t=T-(j+1)$.

Recall that, for all $j$, for any $\theta^{j+1}\in\Theta^{j+1}_{SS^{\prime}}$,
there is a $\bar{\theta}^{j}\in\Theta^{j}_{\theta^{j+1}_{2}S^{\prime}}$ such
that $\theta^{j+1}=(\theta^{j+1}_{1},\bar{\theta}^{j})$. Thus,

$\displaystyle V^{T-(j+1)}(b^{T-(j+1)},\sigma_{j+1}^{T-(j+1)})$
$\displaystyle=1-P[(dr)]^{T-j}d-P[(cr)]^{T-j}c+P[(nr)]^{T-j}V^{T-j}(b^{T-j},\sigma_{j}^{T-j})$
$\displaystyle=1-P[(dr)]^{T-j}d-P[(cr)]^{T-j}c$
$\displaystyle+P[(nr)]^{T-j}(1+\sum_{S\in\\{N,V,O\\}}\sum_{S\in\\{N,V,O\\}}\sum_{\begin{subarray}{c}S^{\prime}\in\\{N,V,O\\}\\\
S^{\prime}\text{ is reachable from
}S\end{subarray}}(k_{SS^{\prime}}^{j}-\tilde{c}_{SS^{\prime}IC}^{j+1})b_{S}^{T-j})$
$\displaystyle=1+k_{N}b_{N}^{T-(j+1)}+k_{V}b_{V}^{T-(j+1)}+k_{O}b_{O}^{T-(j+1)}+\sum_{S^{\prime}\in\\{N,V,O\\}}(k_{NS^{\prime}}^{j}-\tilde{c}_{NS^{\prime}IC}^{j+1})(p_{NN}b_{N}^{T-(j+1)})$
$\displaystyle+\sum_{S\in\\{V,O\\}}(k_{VS^{\prime}}^{j}-\tilde{c}_{VS^{\prime}IC}^{j+1})(p_{NV}b_{N}^{T-(j+1)}+p_{VV}b_{V}^{T-(j+1)})$
$\displaystyle+(k_{OO}^{j}-\tilde{c}_{OOIC}^{j+1})(p_{NO}b_{N}^{T-(j+1)}+p_{VO}b_{V}^{T-(j+1)}+p_{OO}b_{O}^{T-(j+1)})$
$\displaystyle=1+(k_{N}+\sum_{S^{\prime}\in\\{N,V,O\\}}(k_{NS^{\prime}}^{j}-\tilde{c}_{NS^{\prime}IC}^{j+1})p_{NN}$
$\displaystyle+\sum_{S\in\\{V,O\\}}(k_{VS^{\prime}}^{j}-\tilde{c}_{VS^{\prime}IC}^{j+1})p_{NV}+(k_{OO}^{j}-\tilde{c}_{OOIC}^{j+1}))p_{NO})b_{N}^{T-(j+1)}$
$\displaystyle+(k_{V}+\sum_{S\in\\{V,O\\}}(k_{VS^{\prime}}^{j}-\tilde{c}_{VS^{\prime}IC}^{j+1}))p_{VV}+(k_{OO}^{j}-\tilde{c}_{OOIC}^{j+1})p_{VO})b_{V}^{T-(j+1)}$
$\displaystyle+(k_{O}+(k_{OO}^{j}-\tilde{c}_{OOIC}^{j+1}))p_{OO})b_{O}^{T-(j+1)}.$

For any $S,S^{\prime}\in\\{N,V,O\\}$, it is clear that

$\displaystyle\tilde{c}_{SS^{\prime}IC}^{j+2}$
$\displaystyle=\tilde{c}p_{S^{\prime}IC}\sum_{\theta^{j+2}\in\Theta^{j+2}_{SS^{\prime}}}\left(\prod_{l=1}^{j+1}p_{\theta_{l}^{j+1}\theta_{l+1}^{j+1}}\right)$
$\displaystyle=\tilde{c}p_{S^{\prime}IC}\sum_{\begin{subarray}{c}\theta^{j+1}_{2}\in\\{N,V,O\\}\\\
\theta^{j+1}_{2}\text{ is reachable from
}S\end{subarray}}\sum_{\bar{\theta}^{j+1}\in\Theta^{j+1}_{\theta^{j+1}_{2}S^{\prime}}}\left(p_{S\theta^{j+1}_{2}}\prod_{l=2}^{j+1}p_{\theta_{l}^{j+1}\theta_{l+1}^{j+1}}\right)$
$\displaystyle\sum_{\begin{subarray}{c}\theta^{j+1}_{2}\in\\{N,V,O\\}\\\
\theta^{j+1}_{2}\text{ is reachable from
}S\end{subarray}}p_{S\theta^{j+1}_{2}}\left(\tilde{c}p_{S^{\prime}IC}\sum_{\bar{\theta}^{j+1}\in\Theta^{j+1}_{\theta^{j+1}_{2}S^{\prime}}}\prod_{l=2}^{j+1}p_{\theta_{l}^{j+1}\theta_{l+1}^{j+1}}\right)$
$\displaystyle=\sum_{\begin{subarray}{c}\theta^{j+1}_{2}\in\\{N,V,O\\}\\\
\theta^{j+1}_{2}\text{ is reachable from
}S\end{subarray}}p_{S\theta^{j+1}_{2}}\tilde{c}_{\theta^{j+1}_{2}S^{\prime}IC}^{j+1}$

As in Theorem 1, we have that when $S\neq S^{\prime}$,
$k_{SS^{\prime}}^{j+1}=\sum_{\begin{subarray}{c}\theta^{j+1}_{2}\in\\{N,V,O\\}\\\
\theta^{j+1}_{2}\text{ is reachable from
}S\end{subarray}}k_{\bar{\theta}^{j}_{2}S^{\prime}}p_{S\theta^{j+1}_{2}}$, and
when $S=S^{\prime}$, $k_{SS}^{j+1}=k_{S}+k_{SS}^{j}p_{SS}$. Thus,

$\displaystyle
1+(k_{N}+\sum_{S^{\prime}\in\\{N,V,O\\}}(k_{NS^{\prime}}^{j}-\tilde{c}_{NS^{\prime}IC}^{j+1})p_{NN}+\sum_{S\in\\{V,O\\}}(k_{VS^{\prime}}^{j}-\tilde{c}_{VS^{\prime}IC}^{j+1})p_{NV}+(k_{OO}^{j}-\tilde{c}_{OOIC}^{j+1}))p_{NO})b_{N}^{T-(j+1)}$
$\displaystyle+(k_{V}+\sum_{S\in\\{V,O\\}}(k_{VS^{\prime}}^{j}-\tilde{c}_{VS^{\prime}IC}^{j+1}))p_{VV}+(k_{OO}^{j}-\tilde{c}_{OOIC}^{j+1})p_{VO})b_{V}^{T-(j+1)}$
$\displaystyle+(k_{O}+(k_{OO}^{j}-\tilde{c}_{OOIC}^{j+1}))p_{OO})b_{O}^{T-(j+1)}$
$\displaystyle=1+\sum_{S\in\\{N,V,O\\}}\sum_{\begin{subarray}{c}S^{\prime}\in\\{N,V,O\\}\\\
S^{\prime}\text{ is reachable from
}S\end{subarray}}(k_{SS^{\prime}}^{j+1}-\tilde{c}_{SS^{\prime}IC}^{j+2})b_{S}^{T-(j+1)}.$

Thus, for $t=T-(j+1)$, Equations (1) and (3) are equivalent. ∎

Proof of Lemma 2:

###### Proof.

Suppose $(P[(dr)]^{t})$, $(P[(cr)]^{t})$, $(P[(cr)|(i)]^{t})$ and
$(P[(nr)]^{t}P[(cr)|(i)]^{t})$ are non-decreasing sequences, $(P[(nr)]^{t})$
is a non-increasing sequence and that
$\alpha_{d}(P[(dr)]^{t+1}-P[(dr)]^{t})+\alpha_{c}(P[(cr)]^{t+1}-P[(cr)]^{t})\geq(P[(cr)|(i)]^{t+1}-P[(cr)|(i)]^{t})$
for all $t$. Suppose for given $j\leq t$, $V^{t}(b^{t},\sigma_{j}^{t})\geq
V^{t}(b^{t},\sigma_{j+1}^{t})$. We prove
$V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$ by
induction on $j$.

First consider the case where $j=0$, i.e., there exists a $t$ such that
$V^{t}(b^{t},(i))\geq V^{t}(b^{t},(ni,i))$. We show $V^{t+1}(b^{t+1},(i))\geq
V^{t+1}(b^{t+1},(ni,i))$ by showing
$V^{t}(b^{t},(ni,i))-V^{t+1}(b^{t+1},(ni,i))\geq
V^{t}(b^{t},(i))-V^{t+1}(b^{t+1},(i))$.

$\displaystyle V^{t}(b^{t},(ni,i))-V^{t+1}(b^{t+1},(ni,i))$
$\displaystyle=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}V^{t+1}(b^{t+1},(i))$
$\displaystyle-\left(1-P[(dr)]^{t+2}d-P[(cr)]^{t+2}c+P[(nr)]^{t+2}V^{t+2}(b^{t+2},(i))\right)$
$\displaystyle=(P[(dr)]^{t+2}-P[(dr)]^{t+1})d+(P[(cr)]^{t+2}-P[(cr)]^{t+1})c$
$\displaystyle+P[(nr)]^{t+1}-P[(nr)]^{t+2}$
$\displaystyle+\left(P[(nr)]^{t+2}P[(cr)|(i)]^{t+2}-P[(nr)]^{t+1}P[(cr)|(i)]^{t+1}\right)\tilde{c}$
$\displaystyle\geq\left((P[(dr)]^{t+1}-P[(dr)]^{t})\alpha_{d}+(P[(cr)]^{t+1}-P[(cr)]^{t})\alpha_{c}\right)\tilde{c}$
$\displaystyle\geq(P[(cr)|(i)]^{t+1}-P[(cr)|(i)]^{t})\tilde{c}$
$\displaystyle=(1-\tilde{c}P[(cr)|(i)]^{t})-(1-\tilde{c}P[(cr)|(i)]^{t+1})$
$\displaystyle=V^{t}(b^{t},(i))-V^{t+1}(b^{t+1},(i)).$

Thus,

$\displaystyle V^{t}(b^{t},(ni,i))-V^{t+1}(b^{t+1},(ni,i))$ $\displaystyle\geq
V^{t}(b^{t},(i))-V^{t+1}(b^{t+1},(i))$ $\displaystyle
V^{t+1}(b^{t+1},(i))-V^{t+1}(b^{t+1},(ni,i))$ $\displaystyle\geq
V^{t}(b^{t},(i))-V^{t}(b^{t},(ni,i))$ $\displaystyle
V^{t+1}(b^{t+1},(i))-V^{t+1}(b^{t+1},(ni,i))$ $\displaystyle\geq 0$
$\displaystyle V^{t+1}(b^{t+1},(i))$ $\displaystyle\geq
V^{t+1}(b^{t+1},(ni,i)).$

Next, suppose $V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$
implies $V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq
V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$ for $0\leq j\leq l<T-t$. We now show that
if $V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$, then
$V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$
for $j=l+1$. Suppose $V^{t}(b^{t},\sigma_{l+1}^{t})\geq
V^{t}(b^{t},\sigma_{l+2}^{t})$, and note that

$\displaystyle V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})\geq
V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$ $\displaystyle
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))$
$\displaystyle\geq
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}))$
$\displaystyle V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))\geq
V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}).$

Since $V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))\geq
V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})$, then
$V^{t+2}(b^{t+2},\sigma_{l}^{t+2}))\geq V^{t+2}(b^{t+2},\sigma_{l+1}^{t+2})$
by our inductive hypothesis.

$\displaystyle V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})$
$\displaystyle=1-P[(dr)]^{t+2}d-P[(cr)]^{t+2}c+P[(nr)]^{t+2}(V^{t+2}(b^{t+2},\sigma_{l}^{t+2}))$
$\displaystyle\geq
1-P[(dr)]^{t+2}d-P[(cr)]^{t+2}c+P[(nr)]^{t+2}(V^{t+2}(b^{t+2},\sigma_{l+1}^{t+2}))$
$\displaystyle=V^{t+1}(b^{t+1},\sigma_{l+2}^{t+1}).$

We thus have that if $V^{t}(b^{t},\sigma_{j}^{t})\geq
V^{t}(b^{t},\sigma_{j+1}^{t})$ implies that
$V^{t+1}(b^{t+1},\sigma_{j}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})$,
then $V^{t}(b^{t},\sigma_{j+1}^{t})\geq V^{t}(b^{t},\sigma_{j+2}^{t})$ implies
that $V^{t+1}(b^{t+1},\sigma_{j+1}^{t+1})\geq
V^{t+1}(b^{t+1},\sigma_{j+2}^{t+1})$. ∎

Proof of Theorem 4:

###### Proof.

Suppose $(P[(dr)]^{t})$, $(P[(cr)]^{t})$, $(P[(cr)|(i)]^{t})$ and
$(P[(nr)]^{t}P[(cr)|(i)]^{t})$ are non-decreasing sequences, $(P[(nr)]^{t})$
is a non-increasing sequence and that
$\alpha_{d}(P[(dr)]^{t+1}-P[(dr)]^{t})+\alpha_{c}(P[(cr)]^{t+1}-P[(cr)]^{t})\geq(P[(cr)|(i)]^{t+1}-P[(cr)|(i)]^{t})$
for all $t$. Additionally, suppose there exists a $t$ such that
$V^{t}(b^{t},(i))\geq V^{t}(b^{t},(ni,i))$. We show that
$V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$ for $1\leq
j\leq T-t$ be induction. First consider $j=1$. By Lemma 2, we also know that
$V^{t+1}(b^{t+1},(i))\geq V^{t+1}(b^{t+1},(ni,i))$. We have

$\displaystyle V^{t}(b^{t},(ni,ni,i))$
$\displaystyle=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},(ni,i)))$
$\displaystyle\leq
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},(i)))$
$\displaystyle=V^{t}(b^{t},(ni,i))$

Thus, $V^{t}(b^{t},(ni,i))\geq V^{t}(b^{t},(ni,ni,i))$.

Now suppose $V^{t}(b^{t},\sigma_{j}^{t})\geq V^{t}(b^{t},\sigma_{j+1}^{t})$
for $1\leq j\leq l<T-t$. We now show $V^{t}(b^{t},\sigma_{j}^{t})\geq
V^{t}(b^{t},\sigma_{j+1}^{t})$ for $j=l+1$. Suppose
$V^{t}(b^{t},\sigma_{l}^{t})\geq V^{t}(b^{t},\sigma_{l+1}^{t})$. We know that
$V^{t+1}(b^{t+1},\sigma_{l}^{t+1})\geq V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1})$ by
Lemma 2. We have that

$\displaystyle V^{t}(b^{t},\sigma_{l+1}^{t})$
$\displaystyle=1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l}^{t+1}))$
$\displaystyle\geq
1-P[(dr)]^{t+1}d-P[(cr)]^{t+1}c+P[(nr)]^{t+1}(V^{t+1}(b^{t+1},\sigma_{l+1}^{t+1}))$
$\displaystyle=V^{t}(b^{t},\sigma_{l+2}^{t})$

Thus, if $V^{t}(b^{t},\sigma_{l}^{t})\geq V^{t}(b^{t},\sigma_{l+1}^{t})$, then
$V^{t}(b^{t},\sigma_{l+1}^{t})\geq V^{t}(b^{t},\sigma_{l+2}^{t})$. ∎

##  Appendix C Sensitivity of Optimal Inspection Time on Penalty Parameters

We now explore how varying $d$ and $c$ impacts $t^{*}$. For the entirety of
this section, we suppose there exists a $t^{*}$ where the optimal decision is
to inspect. We perform this analysis on the model presented in Section 3, as
the same logic can be applied to the model presented in Section 5.
Additionally, we assume $b^{1}=(1,0,0,0,0,0)$, as the same logic can be
applied to any other starting $b^{1}$.

We first explore the impact of an increase in $d$ and $c$ on $t^{*}$, such
that we would want to inspect earlier. Let $\delta$ be the increase in the
manufacturing failure penalty and $\gamma$ be the increase in penalty for
closing for non-mandatory maintenance. Let
$\bar{k}_{S}=1-(d+\delta+1)p_{SD}-(c+\gamma+1)p_{SC}$. We want to explore what
values $\delta$ and $\gamma$ need to take such that
$V^{1}(b^{1},\sigma^{1}_{t^{*}-1})\geq V^{1}(b^{1},\sigma^{1}_{t^{*}})$. The
following is a simplified inequality on $\delta$ and $\gamma$.

$\displaystyle
V^{1}(b^{1},\sigma^{1}_{t^{*}-1})-V^{1}(b^{1},\sigma^{1}_{t^{*}})\geq 0$
$\displaystyle\sum_{S^{\prime}\in\\{N,V,O\\}}(k_{NS^{\prime}}^{t^{*}-1}-k_{NS^{\prime}}^{t^{*}})\geq
0$
$\displaystyle\sum_{S^{\prime}\in\\{N,V,O\\}}\bar{k}_{S^{\prime}}\left(\sum_{\theta^{t^{*}-1}\in\Theta_{NS^{\prime}}^{t^{*}-1}}f_{\theta^{t^{*}-1}}-\sum_{\theta^{t^{*}}\in\Theta_{NS^{\prime}}^{t^{*}}}f_{\theta^{t^{*}}}\right)\geq
0$
$\displaystyle\sum_{S^{\prime}\in\\{N,V,O\\}}(k_{S^{\prime}}-p_{S^{\prime}D}\delta-
p_{S^{\prime}C}\gamma)\left(\sum_{\theta^{t^{*}-1}\in\Theta_{NS^{\prime}}^{t^{*}-1}}f_{\theta^{t^{*}-1}}-\sum_{\theta^{t^{*}}\in\Theta_{NS^{\prime}}^{t^{*}}}f_{\theta^{t^{*}}}\right)\geq
0$
$\displaystyle\left(\sum_{S^{\prime}\in\\{N,V,O\\}}\left(\sum_{\theta^{t^{*}-1}\in\Theta_{NS^{\prime}}^{t^{*}-1}}f_{\theta^{t^{*}-1}}-\sum_{\theta^{t^{*}}\in\Theta_{NS^{\prime}}^{t^{*}}}f_{\theta^{t^{*}}}\right)p_{S^{\prime}D}\right)\delta$