# Selective Randomization Inference for Adaptive Experiments

Tobias Freidling1 Qingyuan Zhao1 Zijun Gao2

(1Statistical Laboratory, DPMMS, University of Cambridge, UK
2Marshall School of Business, University of Southern California, USA )

###### Abstract

Adaptive experiments use preliminary analyses of the data to inform further
course of action and are commonly used in many disciplines including medical
and social sciences. Because the null hypothesis and experimental design are
not pre-specified, it has long been recognized that statistical inference for
adaptive experiments is not straightforward. Most existing methods only apply
to specific adaptive designs and rely on strong assumptions. In this work, we
propose selective randomization inference as a general framework for analyzing
adaptive experiments. In a nutshell, our approach applies conditional post-
selection inference to randomization tests. By using directed acyclic graphs
to describe the data generating process, we derive a selective randomization
p-value that controls the selective type-I error without requiring independent
and identically distributed data or any other modelling assumptions. We show
how rejection sampling and Markov Chain Monte Carlo can be used to compute the
selective randomization p-values and construct confidence intervals for a
homogeneous treatment effect. To mitigate the risk of disconnected confidence
intervals, we propose the use of hold-out units. Lastly, we demonstrate our
method and compare it with other randomization tests using synthetic and real-
world data.

Keywords: Causal inference, enrichment design, randomization test, selection
bias, selective inference, response-adaptive randomization

##  1 Introduction

In a talk in 1956 titled “Iterative Experimentation”, the great statistician
George Box suggested that we should keep in mind that scientific research is
usually an iterative process. He wrote in the abstract:

> The cycle: conjecture-design-experiment-analysis leads to a new cycle of
> conjecture-design-experiment-analysis and so on. […] Although this cycle is
> repeated many times during an inves-tigation, the experimental environment
> in which it is employed and the techniques appropriate for design and
> analysis tend to change as the investigation proceeds. [8]

John Tukey, another great statistician and Box’s contemporary, acknowledged
that Box’s cycle “will serve excellently” if “an oversimple paradigm is to be
selected”, but he noted that “the point where the circle is to be broken can
be freely chosen” and “there are short cuts and repeated steps as well as
branchings and rejoinings” [70].

Thus, although the demarcation in Box’s cycle is very useful to understand the
role of statistical methodology in different phases of experimentation,
scientific research in the real world does not always follow this linear
process. Often, it is desirable to continuously revise the scientific
hypothesis and design of the study as more data are gathered. This has
motivated two long-standing themes of research in statistics: (i) _selective
inference_ , where valid inference is required on potential findings that are
discovered after viewing the data, and (ii) _adaptive experimentation_ , where
data-adaptive designs are used to allow us to collect more relevant data, save
time and resources, and/or assign more participants to a better treatment.

As the scale of datasets grows and increasingly precise conclusions are
sought, selective inference and adaptive experimentation have attracted much
attention in the recent literature. However, there is strikingly little work
on applying selective inference to adaptively collected data. In this article,
we will attempt to fill this gap and develop a general method termed
“selective randomization inference” that can be applied to a wide range of
adaptive experiments. For the rest of this section, we will first introduce
our proposal and then put it in the broader context set up by Box and Tukey.

###  1.1 Overview of the Selective Randomization Test

Figure 1: Schematic of a two-stage enrichment trial with two subgroups: low
genetic risk score (green) and high genetic risk score (red).

Let us first use a hypothetical two-stage enrichment trial as a running
example to motivate the use of adaptive designs, illustrate the potential
pitfalls, and introduce our proposal (see also Figure 1). This example is
motivated by a post hoc analysis by [44] of the FOURIER trial (Further
Cardiovascular Outcomes Researh With PCSK9 Inhibition in Subjects With
Elevated Risk). By partitioning patients into different groups according to
their genetic risk scores for cardiovascular diseases, [44] found that the new
evolocumab therapy (PCSK9 inhibition) appear to benefit patients with high
genetic risk much more than others. However, because of the post hoc nature of
this subgroup analysis, further randomized studies may be required to confirm
this conclusion.

Now consider a two-stage enrichment trial in which the patient population is
divided at the beginning of the study into a low genetic risk group and a high
genetic risk group. In the first stage of the trial, patients are recruited
from both groups and randomly assigned the new therapy ($Z_{1i}=1$) or the
placebo ($Z_{1i}=0$, where $i$ is the patient index). Based on a preliminary
analysis of data from the first stage, we may wish to select one subgroup or
both subgroups for further investigation in the second stage. For example, we
may choose to only recruit people with high genetic risk score if they show
more promising responses in the first stage and potentially assign the new
therapy with higher probabilities as well. Let the treatment assignment in the
second stage be denoted by $Z_{2}$. We are interested in testing whether there
is any treatment effect for the subgroup(s) selected at the end of the first
stage, which is a categorical variable denoted by
$S_{1}\in\mathcal{S}_{1}=\\{\text{only high},\text{only low},\text{both}\\}$.

It has been recognized very early on that a naive analysis of such a
sequential trial tends to overstate statistical significance [3, 2, 53]. To be
more concrete, consider the following randomization p-value that ignores
selection. Let $Z=(Z_{1},Z_{2})$ denote all the treatment assignments and $W$
denote the recruitment decisions, covariates and potential outcomes of the
patients. A randomization test conditions on $W$ and only uses the randomness
in the treatment assignment $Z$ to calculate the following p-value

$P_{\text{naive}}=\mathbb{P}(T(Z^{*},W)\leq T(Z,W)\mid Z,W),$

where $T(Z,W)$ is a test statistic that measures treatment efficacy and
$Z^{*}=(Z^{*}_{1},Z^{*}_{2})$ has the same distribution of $Z$ but is
conditionally independent of $Z$ given $W$. In this definition, conditioning
on $Z$ and $W$ means that the probability is evaluated over the distribution
of $Z^{*}$, an independent realization of the treatment assignment. Due to
subgroup selection, $P_{\text{naive}}$ is generally not a valid p-value in the
sense that $\mathbb{P}(P_{\text{naive}}\leq\alpha)$ may be larger than the
significance level $0<\alpha<1$. This is because even though the treatment
assignment $Z$ is randomized, the selection “changes” its distribution.
Heuristically, suppose the high genetic risk group is selected at the end of
stage 1 (so $S_{1}=\text{only high}$), then the first stage data must have
already shown promises for the new therapy in the high genetic risk group, and
it is no longer appropriate to use the randomization distribution of $Z_{1}$
to calculate the p-value. In other words, it is “unfair” to compare $Z_{1}$
with $Z_{1}^{*}$ generated from the original randomization distribution
because $Z_{1}$ has gone through selection but $Z_{1}^{*}$ does not.

A convenient solution to this problem is to discard the first stage, aka data
splitting [13]. That is, we use the data of the first stage to choose the
design and null hypothesis and the second stage for statistical inference.
Mathematically, this is equivalent to conditioning on $Z_{1}^{*}=Z_{1}$, and
in the context of randomization inference, this amounts to using

$P_{\text{split}}=\mathbb{P}(T(Z^{*},W)\leq T(Z,W)\mid Z,W,Z_{1}^{*}=Z_{1}).$

Obviously, this p-value is valid but uses the data in an inefficient way.

In a nutsell, our proposal is to condition on the exact amount of information
used in the selection. This is inspired by the recent literature on
conditional post-selection inference, as will be explained in Section 1.2. The
key idea is that the selection in the first stage $S_{1}=S_{1}(Z_{1},W)$ is a
function of the treatment assignment $Z_{1}$ and is thus a random variable. We
argue that the appropriate way to adjust for selection is to condition on it,
which leads to the definition of the _selective randomization p-value_ :

$P_{\text{sel}}=\mathbb{P}(T(Z^{*},W)\leq T(Z,W)\mid
Z,W,S_{1}(Z_{1}^{*},W)=S_{1}(Z_{1},W)).$

We will show that this p-value controls the selective type-I error in the
sense that

$\mathbb{P}(P_{\text{sel}}\leq\alpha\mid
W,S_{1}=s_{1})\leq\alpha,\quad\text{for
all}~{}s_{1}\in\mathcal{S}_{1}~{}\text{and}~{}0<\alpha<1.$ (1)

Thus, no matter which subgroup is selected at the end of the first stage, the
false positive rate of this selective randomization test is controlled.
Compared to data splitting, this test is expected to be more powerful because
it uses the “left out” information in the first stage that is not used by the
selection.

###  1.2 Related Work

Our proposal is related to a wide range of research that offers alternatives
to the linear cycle of scientific research described by Box and discussed by
Tukey. Next we will review the related literature and compare them with our
proposal.

Adaptive designs can be traced back to the work of [68] and are now frequently
used in many fields of empirical research including medicine [10], political
science [49], and economics [17, 36]. In many situations, the experiments have
several designated stages, and within each stage the participants may be
admitted on a rolling basis. Therefore, preliminary data from the initial
participants are often available before the study concludes. This forms the
basis for adapting the treatment allocation or recruitment in later stages of
the experiment. For instance, it may be desirable to increase the assignment
probability of more promising treatments or policies (this is called
_response-adaptive randomization_ , see e.g. [59] and [20]), discontinue
certain arms of the study early (this is called _multi-arm multi-stage design_
, see e.g. [42] and [67]), or focus on the subpopulation of participants that
exhibit the best reactions (this is called _enrichment design_ , see e.g. [43]
and [30]). [50] provided an excellent review of adaptive designs in clinical
trials from a practical perspective.

Logistical hurdles and statistical challenges have prevented a wider adoption
of adaptive designs in clinical trials despite their ethical advantages [55].
In particular, because Box’s linear process of “conjecture-design-experiment-
analysis” no longer applies to adaptive experiments, most standard statistical
methods exhibit selection bias [57]. For certain parametric models, the
distribution of a test statistic in an adaptive study can be derived [64, 41,
24], or one can conduct hypothesis tests for the different stages of the trial
separately and combine the resulting p-values [54, 47, 65]. However, these
approaches are limited to just a few trial designs and heavily rely on
modelling assumptions that are often difficult to justify. In the last decade,
applications in online experiments of web services sparked renewed interest in
adaptively collected data. A recent line of research uses adaptive weighting
techniques to recover asymptotic normality [15, 74, 28, 75]. In another novel
approach, [32] use martingale concentration inequalities to construct any-time
valid confidence intervals for the average treatment effect. These new methods
usually require weaker assumptions but are often technically involved and only
apply to simple response-adaptive designs.

Selective inference seeks valid inference on potential findings that are
discovered after viewing the data [4]. Several notions of “validity” (or
selective risks) have been considered. The notion with the longest history is
familywise error, also known as simultaneous inference, dating back to the
methods of [69] and [61]. Although simultaneous inference is often associated
with multiple hypothesis testing, it can also be applied to post-model
selection problems [7]. A more lenient notion of risk is the average error
rate over the selected; examples include the false discovery rate [5] and the
false coverage-statement rate [6]. A third notion, which is the one we use in
the proposed selective randomization test, is the conditional post-selection
error [22]. Interest in this approach has surged recently after successful
applications to the model selection problem [39]; see [37] for a recent
review.

conjecturedesignexperimentfinal analysis (a) Box’s linear process.

conjecturedesignprelim analysisexperimentfinal analysis (b) Response-adaptive
randomization.

conjecturedesignprelim analysisexperimentfinal analysis (c) Enrichment design.

conjecturedesignprelim analysisexperimentfinal analysis (d) Post-selection
inference.

conjecturedesignprelim analysisexperimentfinal analysis (e) Our problem.

Figure 2: A schematic illustration of different modes of scientific research.
Adaptive experimentation (Figures 2(b) and 2(c)) and post-selection inference
(Figure 2(d)) break away from traditional statistical inference by inserting
different “mini cycles” within each cycle of conjecture-design-experiment-
analysis described by [8].

Figure 2 illustrates how adaptive experiments and selective inference refine
Box’s linear process of conjecture-design-experiment-analysis (Figure 2(a)).
In adaptive experiments, preliminary data from the experiment are used to
modify the experimental design (Figure 2(b)) and conjecture (Figure 2(c)). In
post-selection inference, preliminary analyses (e.g. a model selection
procedure) are used to choose the conjecture (Figure 2(d)). As mentioned
previously, the selective randomization test proposed here applies conditional
post-selection inference to adaptive experiments (Figure 2(e)).

Two points merit a discussion at this point. First, among the different
approaches to selective inference, we believe conditional inference is the
most appropriate for adaptive clinical trials. This is because controlling the
selective type-I error as in (1) allows us to _treat the selected hypothesis
as given_ when interpreting the results. This is crucial in objectively
evaluating the evidence in a randomized experiment from a regulatory
perspective. The familywise error rate is unnecessarily strict as we are often
just interested in testing one hypothesis, and the more lenient notions such
as the false discovery rate are more suitable for discovering promising
hypotheses but not appropriate for confirmatory studies.

Second, compared to existing methods for inference after model selection, a
distinguishing feature of the selective randomization test is that it is
model-free. In particular, it does not require any parametric model on the
outcome, not even that the data points are independent or identically
distributed. Originally suggested by [21] and elaborated by [52],
randomization inference is exactly based on the randomness introduced by the
experimenter, or in Fisher’s words, the “physical act of randomization”
justifies the statistical inference. Although model-based inference is
predominant in the analysis of randomized experiments, interest in
randomization inference, especially for complex designs, has rekindled in
recent years [35, 58, 71, 49, 9]. Some previous works examined the use of
randomization inference in adaptive experiments [19, 63], but the selective
randomization test proposed here can be applied to a much wider class of
adaptive designs.

###  1.3 Organization of the Article

In Section 2, we introduce our notation for the variables in a general
adaptive study with multiple stages and describe their relationships using a
directed acyclic graph. In Section 3, we formally define the selective
randomization p-value and explain how it can be used to estimate and construct
confidence intervals for a homogeneous treatment effect. In Section 4, we
propose two Monte Carlo approximation strategies to compute the p-value:
rejection sampling and Markov Chain Monte Carlo. In Sections 5 and 6, we
illustrate the validity of our method using synthetic and real-world data. We
conclude with a discussion on future research questions in Section 7. Proofs
as well as further results and details about the simulation studies are
deferred to the appendix.

##  2 Adaptive Multi-stage Studies

We consider a adaptive study with $K$ stages and $L+1$ different treatments
(e.g. drugs, dosages, policies, etc.) that are enumerated from $0$ to $L\geq
1$. Denote $[K]:=\\{1,\ldots,K\\},\,[0]:=\emptyset$, and we will use $k\in[K]$
to denote an arbitrary stage and $l\in\\{0,\ldots,L\\}$ an arbitrary
treatment. At every stage, both the recruitment of participants and the
treatment assignment mechanism may depend on the data observed in previous
stages. Our assumptions can be visualized by the directed acyclic graph in
Figure 3 for $K=2$ stages. Next, we describe the setup more closely.

blablaStage 1 blablaStage 2
$Y_{R_{1}}$$R_{1}$$X_{R_{1}}$$Z_{1}$$S_{1}$$Y_{R_{1}}(\cdot)$$Y_{R_{2}}$$R_{2}$$X_{R_{2}}$$Z_{2}$$S_{2}$$Y_{R_{2}}(\cdot)$$Y(\cdot)$$X$
Figure 3: Directed acyclic graph (DAG) of a two-stage adaptive trial, where
Assumptions (A1*) and (2) hold. In addition, $S_{2}$ may directly depend on
the observed data from the first stage: $R_{1},X_{R_{1}},Z_{1}$ and
$Y_{R_{1}}$. For simplicity, these relationships are not depicted.

We assume participants are recruited from a pool of $n$ units. For each
candidate $i\in[n]$, we assume some covariate information $X_{i}$ would become
available if the candidate is recruited, and let us denote
$X:=(X_{i})_{i\in[n]}$. Following the Neyman-Rubin causal model, the causal
effect of a treatment $l$ on a unit $i$ is described using the potential
outcome $Y_{i}(l)$. Let $Y_{i}(\cdot):=(Y_{i}(l))_{l\in\\{0,\dots,L\\}}$
denote all the potential outcomes of participant $i$, and let
$Y(\cdot):=(Y_{i}(\cdot))_{i\in[n]}$. Here, we implicitly make the familiar
assumption of “no interference”, meaning the potential outcomes of one unit
does not depend on the treatments received by the other units. Finally, for a
subset $I\subseteq[n]$ of units, we use the notations $X_{I}:=(X_{i})_{i\in
I}$ and $Y_{I}(\cdot):=(Y_{i}(\cdot))_{i\in I}$ . We refer the reader to [33]
for a comprehensive introduction to the Neyman-Rubin potential outcomes model.

The first distinguishing feature of our setup is that recruitment is modelled
as a random event. In the clinical trials and causal inference literature, it
is usually assumed (implicitly) that the recruited units are fixed. However,
in adaptive experiments such as enrichment trials, recruitment decisions can
depend on previously observed data. To incorporate this, we model the indices
of units participating in each stage as the random sets
$R_{1},\ldots,R_{K}\subseteq[n]$. To simplify the problem, we do not consider
longitudinal studies with patient follow-up over multiple time points in this
work and assume that each unit can be included into the trial at most once, so
$R_{1},\ldots,R_{K}$ are disjoint. This distinguishes our set-up from
sequential multiple assignment randomized trials (SMART), where the same
participants pass through multiple stages and receive interventions/treatments
that can be adapted to their own past outcomes [1]. The covariates and
potential outcomes of participants at stage $k$ are denoted by $X_{R_{k}}$ and
$Y_{R_{k}}(\cdot)$, respectively. Denote ${R=\cup_{k}R_{k}}$,
$R_{[k]}=\cup_{j=1}^{k}R_{k}$ and $R^{C}=[n]\setminus R$.

At stage $k$, a total of $|R_{k}|$ units are recruited; let their administered
treatments be denoted as
$Z_{k}\in\mathcal{Z}_{k}\subseteq\\{0,\ldots,L\\}^{\lvert R_{k}\rvert}$. The
treatment assignment determines which of the potential outcomes is actually
realized: the observed values of the outcome variable are denoted by
$Y_{R_{k}}$. We assume consistency between the potential outcomes and factual
outcomes, that is, $Y_{R_{k}}=Y_{R_{k}}(Z_{k})$. In the following, we use the
short-hand notations $Z=(Z_{k})_{k\in[K]}$,
$\mathcal{Z}=\bigtimes_{k=1}^{K}\,\mathcal{Z}_{k}$ and
$Y_{R}=(Y_{R_{k}})_{k\in[K]}$. Further, we denote
$W_{k}=(R_{k},X_{R_{k}},Y_{R_{k}}(\cdot))$ and $W=(W_{k})_{k}\in\mathcal{W}$.

In adaptive experiments, $Z_{k}$ is randomized by the experimenter and may
depend on the covariates of the recruited participants $X_{R_{k}}$ and
_observed_ data from previous stages. We can formalize these conditions as the
following conditional independence relationship:

$Z_{k}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Y_{R_{[k]}}(\cdot)\mid
R_{[k]},X_{R_{[k]}},Y_{R_{[k-1]}},Z_{[k-1]},\qquad\forall\,k\in[K].$ (A1)

Assumption (A1) is similar to the ignorability or no unmeasured confounders
conditions in causal inference that says that the treatment assignment is
(conditionally) independent of the potential outcomes. It is satisfied if the
treatment assignment in stage $k$ is indeed randomized, and the randomization
distribution is allowed to depend on all the data observed before that time
point.

The second characteristic feature of our setting is the selection statistics
$S_{1},\dots,S_{K}$ which describe the adaptive nature of the experiment.
After administering the treatment and recording the response in stage $k$, the
experimenter is allowed to give a preliminary analysis of the data from stages
$1$ through $k$ and summarize their findings by a statistic $S_{k}$, which
then determines the design of the next stage of the experiment. In other
words, the recruitment and treatment assignment in stage $k+1$ are allowed to
depend on $S_{k}$; in Figure 3, this corresponds to the directed edges
$S_{1}\rightarrow R_{2}$ and $S_{1}\rightarrow Z_{2}$. Note that $S_{k}$ is a
deterministic function of the previously observed data
$R_{[k]},X_{R_{[k]}},Y_{R_{[k]}}$ and $Z_{[k]}$. After completing the trial,
the researchers choose a null hypothesis that may depend on the observed data
of all stages of the trial. Hence, we model it as the final selection
statistic $S_{K}$. We set
$S:=(S_{1},\ldots,S_{K})\in\mathcal{S}=\mathcal{S}_{-K}\times\mathcal{S}_{K}$.
To emphasize the dependence of $S$ on $Z$, we often write it as $S(Z)$.
Moreover, we use the convention $S_{0}:=\emptyset$.

For the methodological development below, it is important to assume that the
selection statistic $S_{k-1}$ captures _all the information_ in previously
observed data that impacts the design of the stage $k$. This can be formalized
as follows:

$R_{k},X_{R_{k}},Y_{R_{k}}(\cdot)\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z_{[k-1]}\mid
W_{[k-1]},S_{k-1},\qquad\forall\,k\in[K].$ (A2)

Assumption (A2) stipulates that the recruitment as well as the covariates and
potential outcomes of the recruited units in stage $k$ depend on the
treatments assigned in earlier stages $Z_{[k-1]}$ only through earlier
recruitment decisions, covariates and potential outcomes as well as the
statistic $S_{k-1}$.

Assumptions (A1) and (A2) are the minimal conditions which we require in this
article. They allow for highly adaptive recruitment decisions and treatment
assignment distributions. In practice, however, many designs fulfill stronger
conditional independence relationships that are easier to verify. For
instance, (A1) can be strengthened as follows

$Z_{k}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z_{[k-1]},W_{[k-1]},Y_{R_{k}}(\cdot)\mid
R_{k},X_{R_{k}},S_{k-1},\qquad\forall\,k\in[K].$ (A1*)

This assumption states that the treatments $Z_{k}$ depend on (observed and
counterfactual) data from previous stages only through the recruitment and
covariates in stage $k$ and the selection statistic $S_{k-1}$. One notable
feature of designs which satisfy (A1*) is that they allow to parallelize the
computation of the selective randomization, see Proposition 3.2. Moreover, we
can strengthen Assumption (A2) by additionally requiring that the recruitment
in stage $k$ depends on data from previous stages only through $S_{k-1}$. We
can express this stronger condition as the following two conditional
independence relationships

$\displaystyle R_{k}$ $\displaystyle\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z_{[k-1]},W_{[k-1]}\mid S_{k-1},\qquad$
$\displaystyle\forall\,k\in[K],$ (A2*-1) $\displaystyle
X_{R_{k}},Y_{R_{k}}(\cdot)$ $\displaystyle\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z_{[k-1]}\mid R_{k},W_{[k-1]},S_{k-1},\qquad$
$\displaystyle\forall\,k\in[K].$ (A2*-2)

Note that (A2) is implied by the two assumptions above via the rules of weak
union and contraction for conditional independence [14, 51]. From here on, we
collectively refer to Assumptions (A2*-1) and (A2*-2) as (2).

We visualize the relationships between the different variables via a directed
acyclic graph (DAG), see Figure 3 for a study with two stages. All the
assumptions (for $k=1,2$) can be read off from the DAG by applying the
d-separation criterion [51]. Note that the DAG in Figure 3 does not depict the
most general case and thus implies more conditional independence relationships
than needed.

##  3 Selective Randomization Inference

This section applies ideas from the literature on conditional post-selection
inference to randomization tests in adaptive experiments. We will formally
define the selective randomization p-value and discuss how to estimate and
construct confidence intervals for the treatment effect.

###  3.1 Selective Randomization Distribution

Randomization inference leverages the fact that the conditional distribution
of the treatment assignments $Z$ is specified by the experimenter and is thus
known. In order to avoid imposing any modelling assumptions on the potential
outcomes $Y(\cdot)$ and covariates $X$ that may be unrealistic, we condition
the distribution of $Z$ on these quantities. In most settings, the choice of
units to include into the study is fixed and therefore not explicitly pointed
out. In adaptive studies, however, the recruitment of participants $R$ may
depend on the treatment assignments $Z$. Since we do not observe any of the
potential outcomes of non-recruited units and may not even know their
covariates, we cannot conduct inference for them. To restrict our analysis to
recruited units, we additionally condition on $R$. This yields the
randomization distribution

$\mathbb{P}(Z=\cdot\mid W)=\mathbb{P}(Z=\cdot\mid R,X_{R},Y_{R}(\cdot)).$

As explained in Section 1.1, this distribution can be used to conduct naive
randomization inference. Yet, such an approach does generally not provide
statistical guarantees, like type-I error control, because the data-dependent
choices of recruitment, treatment assignment and null hypothesis carry
information about the validity of the null hypothesis. To remove any such
information, a general solution is to additionally condition on the selection
statistics $S=(S_{1},\dots,S_{K})$ [22]. This solves two problems. First,
there are two sources of randomness of $Z_{R_{k}}$: the recruitment $R_{k}$,
which is modelled as a random event, and the randomized treatment assignment
given $R_{k}$, which is the basis of randomization inference. Thus, it is
essential that we condition on the recruitment $R_{k}$, whose distribution
depends on $S_{k-1}$ due to the adaptive nature of the design. Second, the
null hypothesis, represented by the statistic $S_{K}$, is also generally data-
dependent in an adaptive experiment. It is critical to condition on $S_{K}$ to
ensure type-I error control for the _selected_ hypothesis. The resulting
_selective randomization distribution_ is then given by

$\mathbb{P}(Z=\cdot\mid W,S(Z))=\mathbb{P}(Z=\cdot\mid
R,X_{R},Y_{R}(\cdot),S(Z)).$

###  3.2 Hypothesis Testing

In order to analyse the effectiveness of the studied treatments, we compare
the corresponding potential outcomes. Since we observe only one potential
outcome for every recruited participant, the null hypothesis typically relates
(a subset of) the realized to the unobserved potential outcomes. Thus, we can
impute (a subset of) $Y_{R}(\cdot)$ from $Y_{R}$. A common hypothesis in
randomization inference is Fisher’s sharp null that states there is no
treatment effect whatsoever:

$\mathrm{H}_{0}:\quad Y_{i}(l)=Y_{i}(l^{\prime}),\quad\text{for all
}l,l^{\prime}\in\\{0,\ldots,L\\},\,i\in R.$

Under this hypothesis, all potential outcomes can be imputed from the observed
outcomes, i.e. $W$ is known, and we can directly use the selective
randomization distribution to conduct inference. To this end, we choose a test
statistic $T(Z,W)$ and compare its observed value to its selective
randomization distribution under the null.

Yet, in many adaptive studies the null hypothesis of interest only allows to
impute a subset of the potential outcomes $Y_{R}(\cdot)$ from the $Y_{R}$.
[76] call such a null hypothesis _partially sharp_. In this case, we usually
need to restrict the support of the selective randomization distribution by
conditioning on an additional statistic $G(Z):=G(Z,W)$.

######  Example 1.

We may want to test the effect of the treatments only in a specific subgroup,
e.g.

$\mathrm{H}_{0}:\quad Y_{i}(l)=Y_{i}(l^{\prime}),\quad\text{for all
}l,l^{\prime}\in\\{0,\ldots,L\\},\,i\in R:X_{i}\geq 0.$

Since the null hypothesis only applies to a certain subset of participants, we
discard the data from other units in the hypothesis test. Mathematically, this
can be achieved by conditioning the selective randomization distribution on
$G(Z)=(Z_{k,i}\colon X_{k,i}<0)_{k,i}$.

######  Example 2.

We may only be interested in comparing a subset of treatments, e.g. $L\geq 2$
and

$\mathrm{H}_{0}:\quad Y_{i}(0)=Y_{i}(1),\quad\text{for all }i\in R.$

In this case, we only consider units who received the treatments $0$ or $1$ in
the original study and discard data from other participants. Moreover, we need
to restrict the treatment assignment for the remaining units to $0$ and $1$ as
only the corresponding potential outcomes can be imputed. This is can be
accomplished by conditioning on
$G(Z)=(Z_{k,i}\cdot\mathbf{1}\\{Z_{k,i}\notin\\{0,1\\}\\})_{k,i}$.

In summary, conditioning on $S$ accounts for the selection of design and null
hypothesis, and conditioning on $G$ accommodates partially sharp null
hypothesis. Based on the selective randomization distribution of the test
statistic $T$ given by

$\mathbb{P}(T(Z,W)=\cdot\mid W,S(Z),G(Z)),$

we define the selective randomization p-value as follows.

######  Definition 1.

In the setting above, let $Z^{*}$ have the same distribution as $Z$ and
fulfill $Z\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z^{*}\mid W$. Then, the _selective
randomization p-value_ is given by

$P_{\text{sel}}:=\mathbb{P}\Big{(}T(Z^{*},W)\leq
T(Z,W)\,\Big{|}\,W,Z,S(Z^{*})=S(Z),G(Z^{*})=G(Z)\Big{)}.$ (3)

Note that the selective randomization p-value is a specific instance of the
conditional randomization p-value of [76]. Therefore, the selective type-I
error control follows immediately.

######  Proposition 3.1.

Let $\alpha\in[0,1]$. The selective randomization p-value as given in
Definition 1 stochastically dominates the uniform distribution on $[0,1]$:

$\mathbb{P}(\,P_{\textnormal{sel}}\leq\alpha\,|\,W,S(Z)=s,G(Z)=g)\leq\alpha,\quad\forall\,s\in\mathcal{S},g\in\mathcal{G}.$

Consequently, a test which rejects the selected null hypothesis when
$P_{\textnormal{sel}}\leq\alpha$ controls the selective type-I error, that is

$\mathbb{P}(\,P_{\textnormal{sel}}\leq\alpha\mid
W,S(Z)=s)\leq\alpha,\quad\forall\,s\in\mathcal{S}.$

###### Proof.

See [76, Thm. 1], where the partition of the treatment assignment space
$\mathcal{Z}$ is given by the preimages of $(S,G)$, cf. [76, Prop. 1]. ∎

######  Remark 1.

[63] consider experiments where the recruitment scheme and null hypothesis are
fixed, but the treatment assignment distribution may depend on data from
previous stages. They implicitly assume ignorability, cf. Assumption (A1), and
prove that – under a condition akin to Assumption (A2) – the usual/naive
randomization p-value is valid. Their result follows as a special case of
Proposition 3.1: The indices of the recruited people are deterministic
$R=[n]$; moreover, their assumptions imply that $S$ can only be function of
$W$ but not $Z$ and can consequently be dropped as $W$ is already conditioned
on.

When randomization inference is applied in practice, $S$ is stipulated by the
experimenters before executing the trial, whereas $G$ and the test statistic
$T$ are chosen by the statisticians analysing the trial after its completion.
If the selected null hypothesis is partially sharp, only a subset of the
potential outcomes $Y_{R}(\cdot)$ of the recruited participants can be imputed
from the observed outcomes $Y_{R}$. Since the selective randomization p-value
generally depends on all potential outcomes, this raises the question whether
it can actually be computed. To ensure this, statisticians need to choose the
test statistic $T$ and the function $G$ such that for all
$z^{*}\in\mathcal{Z}$ satisfying $S(z^{*})=S(Z)$ and $G(z^{*})=G(Z)$ the value
of the test statistic $T(z^{*},W)$ depends only on the potential outcomes that
can be imputed under the null hypothesis chosen by $S_{K}(Z)$. Accordingly,
such a test statistic is called _imputable_ ; for a more detailed discussion,
we refer to [76]. Likewise, the treatment assignment distribution in stage $k$
may depend on previously realized outcomes $Y_{R_{[k-1]}}(Z_{[k-1]})$.
Therefore, $G$ also has to restrict the support of $\mathcal{Z}$ such that
$Y_{R_{[k]}}(z^{*}_{[k]})$ can be imputed for all $z^{*}\in\mathcal{Z}$ that
fulfill $S(z^{*})=S(Z)$ and $G(z^{*})=G(Z)$. We then call the treatment
assignments _imputable_.

######  Proposition 3.2.

Let $z\in\mathcal{Z},w=(r,x,y)\in\mathcal{W},s\in\mathcal{S}$ and
$g\in\mathcal{G}$. Under Assumptions (A1) and (A2),

$\mathbb{P}(Z=z\mid W=w,S(Z)=s,G(Z)=g)\,=\,\frac{q(z\mid
w,s,g)}{\sum_{z^{\prime}\in\mathcal{Z}}q(z^{\prime}\mid w,s,g)},$ (4)

where $q$ is defined as

$q(z\mid w,s,g)=\,\,\mathbf{1}{\\{G(z)=g,S(z)=s\\}}\,\cdot\\\
\prod_{k=1}^{K}\mathbb{P}(Z_{k}=z_{k}\mid
R_{[k]}=r_{[k]},X_{R_{[k]}}=x_{R_{[k]}},Y_{R_{[k-1]}}(z_{k-1})=y_{R_{[k]}},Z_{[k-1]}=z_{[k-1]}).$

If additionally Assumption (A1*) holds, this simplifies to

$q(z\mid
w,s,g)=\mathbf{1}{\\{G(z)=g,S(z)=s\\}}\cdot\prod_{k=1}^{K}\mathbb{P}(Z_{k}=z_{k}\mid
R_{k}=r_{k},X_{R_{k}}=x_{R_{k}},S_{k-1}=s_{k-1}).$

Assume that the test statistic $T$ and the treatment assignments are
imputable. Then, the selective randomization p-value can be computed as
follows:

$P_{\textnormal{sel}}=\frac{\sum_{z^{*}\in\mathcal{Z}}\,\mathbf{1}\big{\\{}T(z^{*},W)\leq
T(Z,W)\big{\\}}\,\,q\big{(}z^{*}\mid
W,S(Z),G(Z)\big{)}}{\sum_{z^{*}\in\mathcal{Z}}\,q\big{(}z^{*}\mid
W,S(Z),G(Z)\big{)}}.$

###### Proof Sketch.

The left-hand side of (4) is proportional to
$\mathbb{P}(Z=z,W=w,S(Z)=s,G(Z)=g)$ for any given $(w,s,g)$. The key idea of
the proof is constructing a factorization of this probability according to the
topological ordering of the DAG (or equivalently the progress of time in the
experiment):

$\ldots\quad\rightarrow\quad R_{k}\quad\rightarrow\quad
X_{R_{k}},Y_{R_{k}}(\cdot)\quad\rightarrow\quad Z_{k}\quad\rightarrow\quad
S_{k}\quad\rightarrow\quad\ldots$

At each stage $k$, we obtain four terms: the conditional probability of
$R_{k}$ given all data from previous stages, then the conditional probability
of $X_{R_{k}},Y_{R_{k}}(\cdot)$ given previous data _and_ $R_{k}$ and so on.
Multiplying the terms from all stages yields the factorization of
$\mathbb{P}(Z=z,W,S(Z)=s,G(Z)=g)$. Assumptions (A1) and (A2) help us to
simplify the resulting expression. Thus, we derive (4).

Then, the formula for the selective randomization p-value directly follows
from its definition, the conditional independence
$Z^{*}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z\mid W$ and normalizing the probabilities
$q$. The details can be found in Appendix A. ∎

The formula above provides a straightforward way of computing (or
approximating) the selective randomization p-value: If the probabilities
$q(z^{*}\mid S(Z),G(Z))$ are known analytically, we can directly compute
$P_{\textnormal{sel}}$; otherwise, we can generate samples from the treatment
assignment distribution and use a Monte Carlo approximation, see Section 4. If
Assumption (A1*) holds, we can even parallelize the sampling for the different
stages as the distribution of each $Z_{k}^{*}$ depends on data from previous
stages only through $S_{k-1}$ which is conditioned on.

###  3.3 Confidence Intervals and Estimation

Thus far, we have focused on hypothesis tests. Yet, our proposed selective
randomization p-value can also be used to construct confidence intervals and
estimators for a homogeneous treatment effect $\tau$. This effect is usually
defined as the differences between the potential outcomes under two treatments
$l$ and $l^{\prime}$ in a subset of the recruited units $I$. The associated
null hypothesis is given by

$\mathrm{H}_{0}^{\tau}\colon\quad
Y_{i}(l)-Y_{i}(l^{\prime})=\tau\quad\forall\,i\in I\subseteq R.$

and the corresponding selective randomization p-value is denoted by
$P_{\text{sel}}(\tau)$. (Note that we can equally define the homogeneous
treatment effect $\tau$ across more than two treatments.) To emphasize the
dependence of the selection statistic on $\tau$, we use the notation
$S(Z;\tau)$.

Due to the duality of hypothesis testing and confidence sets, we can simply
invert the selective hypothesis tests for different values of $\tau$ to
construct the confidence set

$C_{1-\alpha}:=\\{\tau\in\mathbb{R}\colon P_{\text{sel}}(\tau)>\alpha\\}.$ (5)

######  Lemma 3.1.

Let $\alpha\in[0,1]$. If there exists a $\tau_{0}$ such that the hypothesis
$\mathrm{H}_{0}^{\tau_{0}}$ is true, then $C_{1-\alpha}$ is a selective
$1-\alpha$ confidence set, that is

$\mathbb{P}(\tau_{0}\in C_{1-\alpha}\mid W,S(Z;\tau_{0})=s)\geq
1-\alpha\quad\forall\,s\in\mathcal{S}.$

###### Proof.

By definition of the confidence set and Proposition 3.1, we obtain

$\mathbb{P}(\tau_{0}\notin C_{1-\alpha}\mid
W,S(Z;\tau_{0})=s)=\mathbb{P}(P_{\text{sel}}(\tau_{0})\leq\alpha\mid
W,S(Z;\tau_{0})=s)\leq\alpha\quad\forall\,s\in\mathcal{S}.\qed$

Applying a similar reasoning, we can estimate the homogeneous effect as the
$\tau$-value that achieves a selective randomization p-value of $1/2$, i.e.
$\hat{\tau}=\tau$ such that $P_{\text{sel}}(\tau)=1/2$. Since it may be
impossible to find a value of $\tau$ that fulfills this condition exactly, the
Hodges-Lehmann (HL) estimator [31, 56] is defined as follows

$\hat{\tau}=\frac{\sup\\{\tau\colon
P_{\text{sel}}(\tau)<1/2\\}+\inf\\{\tau\colon
P_{\text{sel}}(\tau)>1/2\\}}{2}.$

When the p-value $P_{\text{sel}}(\tau)$ is monotonically increasing in $\tau$,
the curve $\tau\mapsto P_{\text{sel}}(\tau)$ crosses every level exactly once.
Consequently, the proposed confidence set and estimator are “well-behaved”:
$C_{1-\alpha}$ as defined in (5) is an _interval_ and the HL estimator is the
most intuitively right definition of a point estimator. For unconditional
randomization inference, [11] provide conditions under which the corresponding
p-value is indeed monotone.

Yet, when we condition on the selection event, the p-value curve is generally
not monotone; even under [11]’s conditions. Heuristically, if we test
increasingly extreme values of $\tau$, there are usually fewer and fewer
alternative treatment assignments $Z^{*}$ that lead to the same selective
choices as in the original trial, i.e. $S(Z^{*};\tau)=S(Z;\tau)$. Therefore,
the p-value curve becomes increasingly jagged. Hence, the confidence set
defined via inversion of tests may be disconnected and the p-value curve may
cross the level $1/2$ multiple times complicating the definition of a sensible
point estimator. We demonstrate this phenomenon with the two-stage trial
example from the introduction. More details on this simulation study can be
found in Section 5 and Appendix B. When the selection of the null hypothesis
depends on data from the first _and_ second stage, the first panel of Figure 4
shows that the p-value curve may be far from monotone. For instance, it
crosses the level $1/2$ six times yielding six potential point estimates.

Figure 4: Smoothing effect of hold-out units. In the first panel, the p-value
curve (green) is plotted for a selection rule that uses stage I and II units;
in the second panel, only stage I units are used for selection; in the third
panel, only stage I units are used for selection but inference is based on the
second stage randomization p-value, cf. data splitting. The true treatment
effect equals 1 in this simulation.

To our knowledge, the problem of non-monotone p-value curves has not been
studied before and cannot be completely circumvented. However, we can mitigate
the risk of a jagged p-value curve and the resulting complications in
interpreting the inference results by using _hold-out units_. These units are
not taken into account when making selective choices; for example, the
experimenters may decide on the null hypothesis before gathering data from the
participants in the last stage of the study. Since $S$ does not depend on the
treatments assigned to hold-out units, the number of alternative treatments
$Z^{*}$ cannot sink below a certain threshold – even for extreme values of
$\tau$. In the second panel of Figure 4, we plot the p-value curve when only
the first stage data is used for selection of the null hypothesis and the
second stage participants serve as hold-out units. The curve is not monotone
but clearly smoother and yields a unique point estimate and $90\%$ confidence
interval.

##  4 Computation

The feasible treatment assignment space is denoted as
${\mathcal{Z}}=\bigtimes_{k}{\mathcal{Z}}_{k}$, and we use
${\mathcal{Z}}_{s,g}:=\\{Z\in{\mathcal{Z}}:S({Z})=s,G({Z})=g\\}$ to denote the
subset of treatments yielding $S({Z})=s$ and $G({Z})=g$. In Section 4.1, we
display the computational challenges, primarily stemming from conditioning on
$S({Z})$ and $G({Z})$. In Section 4.2, we describe two approaches for
computing a single conditional randomization test p-value: the standard
rejection sampling and a Markov Chain Monte Carlo algorithm. In Section 4.3,
we employ the inversion of a series of conditional randomization tests to
construct confidence intervals.

###  4.1 Computational Challenges

For tests based on randomization or permutation, it is typically infeasible to
enumerate all possible treatment configurations whose number grows
exponentially regarding the sample size. As an approximation, one could draw
$M$ Monte Carlo samples $Z^{(t)}$, $t=1$, $\ldots$, $M$, satisfying
$Z^{(t)}\mid{W}$ and $Z\mid{W}$ being equal in distribution, and compute

$\displaystyle\hat{P}_{M}:=\frac{1+\sum_{t=1}^{M}\,\mathbf{1}\\{T(Z^{(t)},W)\leq
T(Z,W)\\}}{1+M}.$

Here the additional one is added to the denominator and numerator to ensure
the validity of the p-value.

In this section, we focus on the additional challenges induced by the
conditioning event, namely, the process of generating random samples belonging
to ${\mathcal{Z}}_{s,g}$. First, the probability of the conditional event
might be exceedingly small. For example, when one conditions on a continuous
statistic attaining a specific value, or when the potential outcomes are
imputed under some pre-defined null that is at odds with the observed data, it
becomes difficult to generate samples that yield the observed selection, cf.
Section 3.3. Second, the feasible treatment space ${\mathcal{Z}}$ typically
has a simple topology and convenient to sample from, for instance, when each
${\mathcal{Z}}_{k}$ is associated with a completely randomized trial or a
Bernoulli trial. However, the additional constraint $S({Z})=s$, $G({Z})=g$
typically disrupts this helpful structure. Third, since ${Z}$ is a discrete
random vector, gradient-based sampling methods are typically not suitable.

###  4.2 Testing a Single Hypothesis

In Section 4.2.1, we discuss the rejection sampling method, which is
practically convenient but may consume exceedingly high computation time when
the probability of the event conditioned on is small. Markov chain Monte Carlo
(MCMC) algorithms could potentially be much more efficient but require the
construction of a suitable Markov Chain and careful tuning. Table 1 provides a
summary of the two computation methods.

Table 1: Summary of approaches for computing a single conditional randomization test p-value. We recommend rejection sampling when the success probability is not exceptionally low, and the Markov Chain Monte Carlo algorithm otherwise. | Rejection sampling | Markov Chain Monte Carlo (MCMC)
---|---|---
Algorithm | Algorithm 1 | Algorithm 2
Treatment assignment scheme | Arbitrary | Arbitrary
Conditioning event | Arbitrary | Arbitrary
Small prob. conditioning event | Slow | Efficient
Hyper-parameter tuning | Tuning-free | Window-size, burn-in etc.

####  4.2.1 Rejection Sampling

Rejection sampling starts with the random selection of a treatment vector
${Z}^{*}$ from ${\mathcal{Z}}$ following the actual randomization mechanism
without any conditioning. If ${Z}^{*}\in{\mathcal{Z}}_{s,g}$, the generated
treatment ${Z}^{*}$ is accepted; otherwise, ${Z}^{*}$ is rejected. This
process continues until the desired number of samples is reached. The
algorithm is summarized in Algorithm 1.

Suppose the success probability $\mathbb{P}({Z}^{*}\in{\mathcal{Z}}_{s,g})$
equals $q$, then the expected number of draws required to obtain one
acceptable treatment is $1/q$. When $q$ is extremely small, potentially
scaling exponentially with the number of randomized units, the expected number
of $Z^{*}$ attempted can be considerably high. This computational challenge is
illustrated through simulations in Section 5: In the trials where only one
subgroup is selected, the conditional randomization test with rejection
sampling takes around 150 times of the computation time compared to the
randomization test without conditioning; in other words, roughly only one out
of 150 proposals is accepted.

Algorithm 1 Rejection sampling

Input: data ${Z}$, ${W}$ (observed or imputed); selection statistic functions
$S(\cdot)$, $G(\cdot)$, selected test statistic $T(\cdot)$; realized treatment
assignment mechanism $\pi$; number of samples $M$.

Initialization: the current number of accepted treatments $t=0$.

while $t<M$ do

Generate a treatment assignment ${Z}^{*}$ following the distribution $\pi$

if ${Z}^{*}\in{\mathcal{Z}}_{S({Z}),G({Z})}$ then

Accept ${Z}^{*}$ and let ${Z}^{(t)}\leftarrow{Z}^{*}$, $t\leftarrow t+1$.

else

Reject ${Z}^{*}$ and continue.

end if

end while

Compute $\hat{P}_{M}=\left({1+\sum_{t=1}^{M}\mathbf{1}\\{T({Z}^{(t)},{W})\leq
T_{S,G}({Z},{W})\\}}\right)/({1+M})$.

Output: $\hat{P}_{M}$.

####  4.2.2 Random Walk Metropolis-Hastings Algorithm

We describe a general random walk Metropolis-Hastings algorithm [45] to
compute the p-value. The algorithm can be significantly simplified for
treatment assignments where at each stage a completely randomized design or a
Bernoulli trial is conducted given the selection in the previous stages, see
Example 3 and Example 4 below.

We initialize the random walk Metropolis-Hastings (RWM) algorithm at the
observed treatment assignment ${Z}$ and repeat the following iteration $M$
times. In the $t$-th iteration, we select with probability $\rho(\mathcal{A})$
a subset $\mathcal{A}\subseteq R$ of entries of $Z^{(t-1)}$ for updating their
values. Here $\rho$ can be any probability measure defined on the power set of
$R$. Let $\pi$ denote the joint distribution of the realized treatment
assignment mechanism. Provided with $\mathcal{A}$, we fix the treatment
assignment of units not in $\mathcal{A}$, and generate the treatment of units
in $\mathcal{A}$ following the conditional probability
$\pi({Z}_{\mathcal{A}}^{*}\mid{Z}_{\mathcal{A}^{c}}^{*}={Z}^{(t)}_{\mathcal{A}^{c}})$.
Upon obtaining a new proposal ${Z}^{*}$, we set ${Z}^{(t)}={Z}^{*}$ if
${Z}^{*}\in{\mathcal{Z}}_{s,g}$ and reject ${Z}^{*}$ and use
${Z}^{(t)}={Z}^{(t-1)}$, otherwise. The details are summarized in Algorithm 2.

######  Example 3 (Completely randomized design).

A completely randomized design randomly assigns _prefixed_ numbers of units to
the different arms of the study with uniform probability. Suppose that at each
stage a completely randomized design is conducted independently, and let
${Z}_{k}^{(t)}$ be the treatment assignment of stage $k$ in ${Z}^{(t)}$. Then,
for each stage $k$, we can choose a window-size $h_{k}\in\mathbb{N}$, and
randomly select $h_{k}$ entries from ${Z}_{k}^{(t)}$ and shuffle these $h_{k}$
values to generate a new ${Z}_{k}^{*}$. This is equivalent to setting $\rho$
as the uniform measure supported on all subsets consisting of exactly $h_{k}$
units from stage $k$.

######  Example 4 (Bernoulli trial).

In a Bernoulli trial, each unit $i$ is independently assigned to the treatment
with probability $p_{i}$, where the probability $p_{i}$ may depend on their
covariates. Suppose at each stage an independent Bernoulli trial is conducted,
where the probability parameter may depend on the selection made in prior
stages and a unit’s covariates. Similar to Example 3, in each iteration, we
randomly select $2\leq h_{k}\in\mathbb{N}$ entries from ${Z}_{k}^{(t)}$ and
replace them by independent Bernoulli random variables with the same
probability parameter.

Markov Chain Monte Carlo algorithms, like the RWM algorithm, generate samples
from the treatment assignment distribution by traversing the space
$\mathcal{Z}$. As the conditioning event of the selective randomization
p-value may be complex, the selective randomization distribution may have
disconnected support. Therefore, the RWM sampler may not be able “reach” every
feasible treatment assignment. This phenomenon is described by the notion of
_communication_ : For a Markov chain, we say that $z\in\mathcal{Z}$ and
$z^{\prime}\in\mathcal{Z}$ communicate if and only if $z$ can be reached from
$z^{\prime}$ in finitely many steps with positive probability and vice versa.
Hence, communication defines an equivalence relation and partitions
$\mathcal{Z}$ into communication classes. For the RWM algorithm, a larger
window-size $h$ is associated with larger and fewer communication classes,
typically more powerful tests, but also higher computational complexity.

Algorithm 2 Random walk Metropolis-Hastings algorithm

Input: data ${Z}$, ${W}$ (observed or imputed); selection statistic functions
$S(\cdot)$, $G(\cdot)$, selected test statistic $T(\cdot)$; realized treatment
assignment mechanism $\pi$; a probability measure $\rho$ on the power set of
all entries in ${Z}^{*}$; number of samples $M$; burn-in size $b$.

Initialization: Set ${Z}^{(0)}={Z}$ (observed value).

for t $=1:M$ do

Select a subset $\mathcal{A}$ of entries with probability $\rho(\mathcal{A})$,
and generate a new proposal ${Z}^{*}$ following the conditional distribution
$\pi(\cdot\mid{Z}_{\mathcal{A}^{c}}^{*}={Z}^{(t-1)}_{\mathcal{A}^{c}})$.

if ${Z}^{*}\in{\mathcal{Z}}_{S({Z}),G({Z})}$ then

${Z}^{(t)}\leftarrow{Z}^{*}$.

else

${Z}^{(t)}\leftarrow{Z}^{(t-1)}$.

end if

end for

Compute
$\hat{P}_{M}=\left({1+\sum_{t=b+1}^{M}\mathbf{1}\\{T({Z}^{(t)},{W})\leq
T({Z},{W})\\}}\right)/({1+(M-b)})$.

Output: $\hat{P}_{M}$.

Since a Markov chain is always confined to the states that communicate with
the starting value $Z^{(0)}$, we cannot always approximate the selective
randomization p-value as defined in (3). Instead, we additionally condition it
on the communication class and denote the resulting p-value
$\tilde{P}_{\text{sel}}$. Importantly, this p-value inherits the selective
type-I error control – more details in Appendix C.1 – and can be approximated
by the RWM sampler.

######  Proposition 4.1.

For the Monte Carlo approximation in Algorithm 2, there exists
$\sigma^{2}<\infty$ such that

$\displaystyle\sqrt{M}\big{(}\hat{P}_{M}-\tilde{P}_{\text{sel}}\big{)}\mid{Z},{W}\,\stackrel{{\scriptstyle
d}}{{\to}}\,\mathcal{N}\left(0,\sigma^{2}\right),\quad\text{as }M\to\infty.$

This result establishes the asymptotic distribution of the p-value calculated
with Algorithm 2 under general treatment designs including Bernoulli and
completely randomized trials. We can directly read off that the convergence
rate is of order $\mathcal{O}_{p}(M^{-1/2})$. The proof of Proposition 4.1
relies on the observation that the proposed chain in Algorithm 2 is self-
reflexive, i.e.
$\pi({Z}^{*}={Z}^{(t)}\mid{Z}_{\mathcal{A}^{c}}^{*}={Z}_{\mathcal{A}^{c}}^{(t)})>0$,
with the stationary distribution being the desired treatment assignment
mechanism. Details are summarized in Appendix C.2.

To conclude this subsection, we discuss hyper-parameter selection. We choose
the window size to maximize the empirical efficiency metric Mean Squared
Euclidean Jump Distance (MSEJD) defined as

$\displaystyle
S^{2}=\frac{1}{M-1}\sum_{t=1}^{M-1}\|{Z}^{(t+1)}-{Z}^{(t)}\|_{2}^{2}.$

MSEJD is equivalent to a weighted sum of the lag-$1$ auto-correlation for
stationary chains with finite posterior variance [62], and a higher MSEJD
value indicates greater efficiency. We remark that for binary treatments,
$S^{2}$ is equivalent to the average Hamming distance between adjacent samples
${Z}^{(t)}$ and ${Z}^{(t+1)}$, $t\in\\{1,\ldots,M-1\\}$. Compared to the
rejection probability of a new proposal, the significance of MSEJD lies in its
ability to not only account for the frequency of acceptance but also capture
the magnitude of the differences ${Z}^{*}-{Z}^{(t)}$ among accepted proposals
${Z}^{*}$.

###  4.3 Constructing Confidence Set

We discuss computation details of constructing confidence sets as outlined in
(5). Let $\mathcal{I}$ denote a user-specified set of potential constant
treatment and suppose $\mathcal{I}$ is of finite-length. For arbitrary
selective hypothesis test p-values, possibly non-monotone, one can discretize
$\mathcal{I}$ into bins of width $\varepsilon>0$, and evaluate the p-value at
the representative value of each bin. The confidence set shall comprise all
bins associated with p-values above the significant level. The testing of
different representative values of $\tau$ can be computed independently,
allowing the entire procedure to be accelerated through parallel computing. If
the p-values are monotone, the confidence set is an interval. In this case,
the Robbins-Monro algorithm and the bisection method can be employed to
efficiently compute the lower and upper limits of this interval [26, 71].

##  5 Simulation Study

To verify our proposed methods empirically, we use the two-stage enrichment
trial example introduced in Section 1. We use the following notations and
data-generating mechanism.

The population of interest is subdivided into two groups with low and high
genetic risk score, respectively; this is captured in the covariate
$X_{i}\in\\{\text{low},\text{high}\\}$, where $i$ is the patient index. Every
participant is assigned to the new drug ($Z_{i}=1$) or the placebo
($Z_{i}=0$). In both stages, of the trial we use a completely randomized
treatment assignment mechanism. That is, we assign half of the units to the
new drug and half of the units to the placebo with equal probability. The
outcome variable of interest $Y_{i}$ is continuous, e.g. LDL cholesterol
level. In this simulation study, we generate the potential outcomes
$Y_{i}(1)=Y_{i}(0)$ as i.i.d. standard normals. Hence, there is no treatment
effect in neither group and the sharp null hypothesis holds true. In the first
stage, we simulate $n_{1}=100$ recruited patients, 50 with low and high
genetic risk score, respectively. To select the group(s) to recruit from in
the second stage, we compare the standardized average treatment effects
$\Delta_{r}$ in the first stage, which are defined as

$\Delta_{r}=\frac{\bar{Y}_{r,1}-\bar{Y}_{r,0}}{\sqrt{\hat{\sigma}^{2}_{r,1}+\hat{\sigma}^{2}_{r,0}}},\qquad\hat{\sigma}^{2}_{r,t}=\frac{\sum\limits_{i:X_{i}=r,Z_{i}=t}(Y_{i}-\bar{Y}_{r,t})^{2}}{\sum\limits_{i=1}^{n_{1}}\mathbbm{1}_{\\{X_{i}=r,Z_{i}=t\\}}},\qquad
r\in\\{\text{low},\text{high}\\},\,t\in\\{0,1\\},$ (6)

where $\bar{Y}_{r,t}$ denotes the average outcome in group $r$ under treatment
$t$. We define the selection statistic $S$ in terms of the scaled difference
ATE difference $\Delta=(\Delta_{\text{high}}-\Delta_{\text{low}})/\sqrt{2}$ as
follows

$S=\begin{cases}\text{only
low},&\quad\Delta<\Phi^{-1}(0.2),\qquad\text{recruit 40 from group
$X_{i}=\text{low}$},\\\ \text{only
high},&\quad\Delta>\Phi^{-1}(0.8),\qquad\text{recruit 40 from group
$X_{i}=\text{high}$},\\\ \text{both},&\quad\text{otherwise},\hskip
17.64056pt\qquad\,\text{recruit 20 from group $X_{i}=\text{low}$ and
$X_{i}=\text{high}$ each}.\end{cases}$ (7)

Here $\Phi(\cdot)$ denotes the cdf of a standard normal distribution. This
selection rule is motivated by a traditional model: If the distribution of the
outcomes is the same under placebo and drug for both groups and the data
points are i.i.d., $\Delta$ approximately follows a standard normal
distribution. Note that these assumptions are only used to design the
experiment but not to analyse the resulting data.

In the following, we compare three types of randomization inference: the
standard randomization test (RT), the randomization test of the second stage
(RT 2nd) and the proposed selection randomization test (SRT). For the latter,
we experiment with two implementations: rejection sampling and the RWM sampler
with window size of 5. Appendix D contains additional results on the RWM
sampler.

For the test statistic $T$, we use the standardized ATE (6) applied to the
selected group(s) with data from both stages. A larger value of the test
statistic $T$ indicates a stronger treatment effect. We perform inference on
the null hypothesis across a range of treatment effects
$\tau\in\\{-1,-0.8,\ldots,0.8,1\\}$. Null hypotheses corresponding to smaller
treatment effects should have a higher probability of being rejected.
Additionally, the probability of rejecting the null hypothesis with $\tau=0$
should not exceed the target significance level. We repeat from the generation
of potential outcomes and covariates $400$ times and report the rejection
probabilities for each null hypothesis. We also display the rejection
probabilities among trials with a specific stage $1$ subgroup selection:
subgroup $X_{i}=\text{low}$, or subgroup $X_{i}=\text{high}$, or both
subgroups. In addition, we compare the computation time of RT, RT 2nd, SRT
with rejection sampling, and SRT with RWM sampling. The source code is
available on Github.

###  5.1 Hypothesis Testing

As demonstrated in Figure 5, RT does not control the type I error (the null
with zero effect, i.e., $\tau=0$), especially in cases where the selection
bias is severe: selecting subgroup $X_{i}=\text{high}$ or selecting subgroup
$X_{i}=\text{low}$). SRT and RT 2nd both control the unconditional and
unconditional type I errors. SRT exhibits greater power than RT 2nd when both
subgroups are selected due to the randomness recycled from the first stage.
The power curves do not significantly differ across rejection sampling and
MCMC.

In terms of the computation time, RT and RT 2nd are the most efficient,
followed by RWM. Rejection sampling is significantly slower, especially when
one of the subgroups is selected and the hypothesis being tested is extreme.
In this scenario the success probability of the rejection sampling is very
low, leading to a prohibitive computational burden.

(a) Rejection probability

(b) Computation time

Figure 5: Comparison of randomization tests regarding computation time and
rejection probabilities for testing a sequence of nulls of constant treatment
effect $\tau$. The potential outcomes satisfy $\tau=0$.

###  5.2 Confidence Intervals

Table 2:  Comparison of randomization tests regarding coverage probability (significance level $0.1$). We report the conditional coverage given the subgroup(s) selected, and the unconditional coverage marginalizing over the selection.  | SRT |  | RT 2nd | RT
---|---|---|---|---
| MCMC | RS |  |
Unconditional | 0.910 | 0.920 | 0.910 | 0.858
Subgroup $X_{i}=\text{high}$ | 0.861 | 0.873 | 0.873 | 0.722
Subgroup $X_{i}=\text{low}$ | 0.921 | 0.934 | 0.949 | 0.795
Both subgroups | 0.909 | 0.918 | 0.909 | 0.922

According to the discussion of the monotonicity of the selective randomization
p-value, we proceed to construct confidence intervals by inverting
randomization tests using the baseline discretization method. In particular,
we construct one-sided confidence intervals and use binary search to determine
the left boundary value $\underline{\tau}$. We evaluate the coverage of the
confidence intervals and also compute the average value of $\underline{\tau}$,
where a greater value of $\underline{\tau}$ indicates higher power.

According to Table 2, RT fails to produce the right coverage when only one of
the subgroups is selected. SRT and RT 2nd yield reasonably good coverage.
Figure 6 suggests that the lower boundary of SRT is in general larger than
that of RT 2nd, suggesting that SRT is more powerful. The results are
consistent with those in Section 5.1.

Figure 6: Comparison of randomization tests regarding lower boundary of the
confidence interval.

##  6 Real Data Analysis

We create hypothetical two-stage enrichment trials from the data of a one-
stage randomized experiment and compare the RT, SRT, and RT 2nd adopted in the
simulation section on the hypothetical data.

The hypothetical dataset is generated from the Systolic blood pressure
intervention trial (SPRINT) [48, 25] which aims to study whether a new
treatment program targeting reducing systolic blood pressure will reduce
cardiovascular disease (CVD) risk. The outcome is whether a major CVD event
happens to the subject. There are $20$ features: $3$ demographic features
including age, race; $6$ medical history features including daily Aspirin use,
history of CVD; $11$ lab measurements including body mass index (BMI), SBP. We
classify the subjects into four age subgroups: no older than $59$, $60$ to
$69$, $70$ to $79$, no younger than $80$. In the first stage, we randomly
sample $2000$ units without replacement, compute the relative risk
($\mathbb{P}(\text{CVD event}\mid\text{treatment arm})/\mathbb{P}(\text{CVD
event}\mid\text{control arm})$) for each age group, and select the age group
with the minimal relative risk, i.e. the most significant improvement. In the
second stage, we randomly sample an additional $200$ units without replacement
of the selected age group from the units not chosen by the first stage. The
hypothetical dataset is summarized in Table 3.

We test the sharp null that there is no treatment effect. We use the relative
risk as the test statistic and approximate the p-value using $1000$ Monte
Carlo samples. For the selective randomization test, we adopt the rejection
sampling algorithm. As shown in Table 3, SRT exhibits significantly higher
statistical power than RT 2nd due to the randomness in the first stage left
after the selection. In this scenario, SRT is comparable to RT regarding
power, but the RT is not guaranteed to control the type-I error.

Table 3: Comparison of randomization tests on a hypothetical dataset generated
from the SPRINT dataset.

Stage | Stage 1 ($N_{1}=2000$) | Stage 2 ($N_{2}=200$)
---|---|---
Age group | $\leq 60$ | $[60,69]$ | $[70,79]$ | $\geq 80$ | $\geq 80$
Control | $Y=1$ | 8 | 17 | 22 | 19 | 17
$Y=0$ | 187 | 349 | 275 | 123 | 87
Treated | $Y=1$ | 12 | 13 | 22 | 7 | 13
$Y=0$ | 201 | 331 | 289 | 125 | 83

| SRT | RT 2nd | RT
---|---|---|---
P-value | 0.048 | 0.314 | 0.027

In Appendix E, we repeat the above power comparison from constructing the
hypothetical real dataset via subsampling multiple times. The distribution of
the p-value from the SRT is stochastically dominated by that of the RT 2nd. We
also perform a placebo analysis evaluating the type-I error. In particular, we
subsample from the control group of the original SPRINT trial and generate
artificial treatment assignments following independent Bernoulli$(0.5)$
multiple times. All the units in the constructed dataset did not receive the
treatment, and the null hypothesis that the artificial treatment has no
treatment effect is true. In the placebo analysis, the RT fails to control the
type-I error while SRT does.

The source code for generating and analysing this hypothetical clinical trial
is available on Github.

##  7 Discussion and Outlook

To analyse data from adaptive studies, we use the paradigm of randomization
inference as it has two major benefits. First, randomization inference does
not require any modelling assumptions on the distribution of the covariates
and outcomes or the ubiquitous condition of i.i.d. data points. Therefore, it
is an ideal default tool for conducting inference that can stand on its own or
complement a model-based analysis. Second, randomization inference naturally
lends itself to handle selective choices. Recent model-based proposals
typically involve considerable technical difficulties and can only be used for
specific types of adaptive studies. On the other hand, our proposed selective
randomization p-value can accommodate all kinds of selective choices. The
reason for this flexibility stems from the following insight: In standard
randomization inference, we compare the realized treatment assignment $Z$ to
all alternative assignments $Z^{*}$; in adaptive studies, however, it is only
fair to compare to $Z^{*}$ that lead to the same choices as in the original
trial. Hence, conditioning on the information expressed in the selection, i.e.
conditioning on $S(Z^{*})=S(Z)$, suffices to provide valid inference. This
holds true regardless of the type of selective choice.

While this article is mainly concerned with introducing selective
randomization inference and exploring some immediate questions like
constructing confidence intervals or computing the p-value, there are many
conceivable extensions.

First, this work focuses on developing a general theoretical framework to
_analyse_ data from adaptive experiments. In turn, the developed methodology
also has implications on the design of such adaptive studies: The preliminary
data analyses between stages, the treatment assignments and the recruitment
decisions must obey Assumptions (A1) and (A2) or their stronger versions (A1*)
and (2). This touches on the subtle question at which point(s) in the study
the experimenters can use which type of data to make adaptive choices. We save
a detailed investigation of this issue for future, more application-oriented
work.

Second, we only consider the classical case of a (partially) sharp null
hypothesis in this work. This implies testing that the treatment effect is the
same for every (or a subgroup of) individual in the study. Since such null
hypotheses are sometimes criticized as overly restrictive and therefore of
little interest to practitioners, many recent articles focus on relaxing the
sharp null. [11] propose to test one-sided null hypotheses; another line of
research considers weak null hypotheses that concern the average treatment
effect among the participants [16, 73, 12]. Moreover, standard treatment
assignment mechanisms, e.g. completely randomized or Bernoulli trials, may
lead to covariate imbalance between different arms of the study. To remedy
this issue, rerandomization tests [46, 72] condition the distribution of $Z$
on a balance criterion. Integrating these recent developments into our
selective randomization inference framework, would greatly add to its
flexibility and appeal.

Third, this work is mainly targeted to adaptive studies that use a selection
statistic $S$ that takes discrete values. Conceptually, there is no obstacle
to apply our method to continuous selection rules. However, the selective
randomization p-value may reduce to the data splitting p-value. Consider our
running example of a two-stage enrichment trial: In the extreme case that each
first-stage treatment assignment leads to a different value of the selection
statistic $S(Z_{1})$, conditioning on $S(Z_{1}^{*})=S(Z_{1})$ is equivalent to
conditioning on $Z_{1}^{*}=Z_{1}$. Even when there are a few first-stage
treatment assignments yielding the same value of a continuous selection
statistic, it is computationally hard to identify them, especially when
rejection sampling is used. These characteristics seem to be unfavourable for
analysing a common type of enrichment trial where a continuous biomarker is
used as selection criterion. Researchers typically use the first stage data to
determine a threshold and recruit in the second stage only participants whose
biomarker passes this hurdle [23, 66, 60, 38, 65]. However, if we make
additional assumptions such as a monotone relationship between the biomarker
and treatment effect, some of the “first-stage randomness” can be retained and
used for conducting inference.

Fourth, drop-out units, which are commonly observed in clinical trials, pose
challenges for drawing valid inference. The particular difficulty of
conducting randomization tests lies in the inability to impute the missing
outcomes of drop-out units. Solutions to this problem include model-based or
non-informative outcome imputation and reweighting the treatment assignments
probabilities among the subjects with observed outcomes based on the outcome
missingness probabilities [18, 58, 34, 29]. If the missingness is independent
of the outcomes but possibly influenced by the treatment administered
(analogous to the missing at random mechanism), we can extend our proposal to
incorporate the missingness following the path of reweighting. The resulting
p-values will be valid for finite samples if the mechanism of missingness is
known, or valid asymptotically if the missing mechanism can be consistently
estimated.

## References

  * [1] Daniel Almirall, Inbal Nahum-Shani, Nancy E. Sherwood and Susan A. Murphy  “Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research”  In _Translational Behavioral Medicine_ 4.3, 2014, pp. 260–274
  * [2] F. J. Anscombe  “Sequential Medical Trials”  In _Journal of the American Statistical Association_ 58.302, 1963, pp. 365–383
  * [3] P.M.A. Armitage  “Sequential medical trials”  In _Sequential Medical Trials_ Blackwell Scientific Publications, 1960
  * [4] Yoav Benjamini  “Selective Inference: The Silent Killer of Replicability”  In _Harvard Data Science Review_ 2.4, 2020
  * [5] Yoav Benjamini and Yosef Hochberg  “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing”  In _Journal of the Royal Statistical Society. Series B (Methodological)_ 57.1 [Royal Statistical Society, Wiley], 1995, pp. 289–300
  * [6] Yoav Benjamini and Daniel Yekutieli  “False Discovery Rate-Adjusted Multiple Confidence Intervals for Selected Parameters”  In _Journal of the American Statistical Association_ 100.469 Taylor & Francis, 2005, pp. 71–81
  * [7] Richard Berk, Lawrence Brown, Andreas Buja, Kai Zhang and Linda Zhao  “Valid Post-Selection Inference”  In _The Annals of Statistics_ 41.2 Institute of Mathematical Statistics, 2013, pp. 802–837
  * [8] George E. P. Box  “Abstracts”  In _Biometrics_ 13.2 Wiley, International Biometric Society, 1957, pp. 238–246
  * [9] Federico A. Bugni, Ivan A. Canay and Azeem M. Shaikh  “Inference Under Covariate-Adaptive Randomization”  In _Journal of the American Statistical Association_ 113.524, 2018, pp. 1784–1796
  * [10] Thomas Burnett, Pavel Mozgunov, Philip Pallmann, Sofia S. Villar, Graham M. Wheeler and Thomas Jaki  “Adding flexibility to clinical trial designs: an example-based guide to the practical use of adaptive designs”  In _BMC Medicine_ 18.1, 2020, pp. 352
  * [11] Devin Caughey, Allan Dafoe, Xinran Li and Luke Miratrix  “Randomisation inference beyond the sharp null: bounded null hypotheses and quantiles of individual treatment effects”  In _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 85.5, 2023, pp. 1471–1491
  * [12] Peter L. Cohen and Colin B. Fogarty  “Gaussian Prepivoting for Finite Population Causal Inference”  In _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 84.2, 2022, pp. 295–320
  * [13] D. R. Cox  “A note on data-splitting for the evaluation of significance levels”  In _Biometrika_ 62.2, 1975, pp. 441–444
  * [14] A Philip Dawid  “Conditional independence in statistical theory”  In _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 41.1 Oxford University Press, 1979, pp. 1–15
  * [15] Yash Deshpande, Lester Mackey, Vasilis Syrgkanis and Matt Taddy  “Accurate Inference for Adaptive Linear Models”  In _Proceedings of the 35th International Conference on Machine Learning_ PMLR, 2018, pp. 1194–1203
  * [16] Peng Ding  “A Paradox from Randomization-Based Causal Inference”  In _Statistical Science_ 32.3, 2017, pp. 331–345
  * [17] Esther Duflo, Rachel Glennerster and Michael Kremer  “Chapter 61 Using Randomization in Development Economics Research: A Toolkit”  In _Handbook of Development Economics_ 4, 2007, pp. 3895–3962
  * [18] Eugene Edgington and Patrick Onghena  “Randomization tests”  ChapmanHall/CRC, 2007
  * [19] David Edwards  “On model prespecification in confirmatory randomized studies”  In _Statistics in Medicine_ 18.7, 1999, pp. 771–785
  * [20] Babalola Faseru, Edward F. Ellerbeck, Delwyn Catley, Byron J. Gajewski, Taneisha S. Scheuermann, Theresa I. Shireman, Laura M. Mussulman, Niaman Nazir, Terry Bush and Kimber P. Richter  “Changing the default for tobacco-cessation treatment in an inpatient setting: study protocol of a randomized controlled trial”  In _Trials_ 18.1, 2017, pp. 379
  * [21] R. A. Fisher  “The design of experiments”  Edinburgh: Oliver & Boyd, 1935
  * [22] William Fithian, Dennis Sun and Jonathan Taylor  “Optimal Inference After Model Selection”  In _arXiv: 1410.2597_ , 2017
  * [23] Tim Friede, N Parsons and Nigel Stallard  “A conditional error function approach for subgroup selection in adaptive clinical trials”  In _Statistics in medicine_ 31.30 Wiley Online Library, 2012, pp. 4309–4320
  * [24] Rosamarie Frieri, William Fisher Rosenberger, Nancy Flournoy and Zhantao Lin  “Design considerations for two-stage enrichment clinical trials”  In _Biometrics_ 79.3, 2023, pp. 2565–2576
  * [25] Zijun Gao, Trevor Hastie and Robert Tibshirani  “Assessment of heterogeneous treatment effect estimation accuracy via matching”  In _Statistics in Medicine_ 40.17, 2021, pp. 3990–4013
  * [26] Paul H Garthwaite  “Confidence intervals from randomization tests”  In _Biometrics_ 52 JSTOR, 1996, pp. 1387–1393
  * [27] Walter R Gilks, Sylvia Richardson and David Spiegelhalter  “Markov chain Monte Carlo in practice”  ChapmanHall, London, 1996
  * [28] Vitor Hadad, David A. Hirshberg, Ruohan Zhan, Stefan Wager and Susan Athey  “Confidence intervals for policy evaluation in adaptive experiments”  In _Proceedings of the National Academy of Sciences_ 118.15, 2021, pp. e2014602118
  * [29] Nicole Heussen, Ralf-Dieter Hilgers, William F Rosenberger, Xiao Tan and Diane Uschner  “Ran-domization-based inference for clinical trials with missing outcome data”  In _Statistics in Biopharmaceutical Research_ Taylor & Francis, 2023, pp. 1–17
  * [30] Tony W Ho, Eric Pearlman, Donald Lewis, Mirja Hämäläinen, Kathryn Connor, David Michelson, Ying Zhang, Christopher Assaid, Lyn Harper Mozley, Nancy Strickler, Robert Bachman, Erin Mahoney, Christopher Lines and David J Hewitt  “Efficacy and tolerability of rizatriptan in pediatric migraineurs: Results from a randomized, double-blind, placebo-controlled trial using a novel adaptive enrichment design”  In _Cephalalgia_ 32.10, 2012, pp. 750–765
  * [31] J. L. Hodges and E. L. Lehmann  “Estimates of Location Based on Rank Tests”  In _The Annals of Mathematical Statistics_ 34.2 Institute of Mathematical Statistics, 1963, pp. 598–611
  * [32] Steven R. Howard, Aaditya Ramdas, Jon McAuliffe and Jasjeet Sekhon  “Time-uniform, nonparametric, nonasymptotic confidence sequences”  In _The Annals of Statistics_ 49.2, 2021
  * [33] Guido W Imbens and Donald B Rubin  “Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction”  Cambridge University Press, 2015
  * [34] Anastasia Ivanova, Seth Lederman, Philip B Stark, Gregory Sullivan and Ben Vaughn  “Randomization tests in clinical trials with multiple imputation for handling missing data”  In _Journal of Biopharmaceutical Statistics_ 32.3 Taylor & Francis, 2022, pp. 441–449
  * [35] Xinyao Ji, Gunther Fink, Paul Jacob Robyn and Dylan S. Small  “Randomization inference for stepped-wedge cluster-randomized trials: An application to community-based health insurance”  In _The Annals of Applied Statistics_ 11.1, 2017, pp. 1–20
  * [36] Maximilian Kasy and Anja Sautmann  “Adaptive Treatment Assignment in Experiments for Policy Choice”  In _Econometrica_ 89.1, 2021, pp. 113–132
  * [37] Arun K. Kuchibhotla, John E. Kolassa and Todd A. Kuffner  “Post-Selection Inference”  In _Annual Review of Statistics and Its Application_ 9.1, 2022, pp. 505–527
  * [38] Tze Leung Lai, Philip W Lavori and Ka Wai Tsang  “Adaptive enrichment designs for confirmatory trials”  In _Statistics in medicine_ 38.4 Wiley Online Library, 2019, pp. 613–624
  * [39] Jason D. Lee, Dennis L. Sun, Yuekai Sun and Jonathan E. Taylor  “Exact post-selection inference, with application to the lasso”  In _The Annals of Statistics_ 44.3, 2016
  * [40] David A Levin and Yuval Peres  “Markov chains and mixing times”  American Mathematical Society, 2017
  * [41] Zhantao Lin, Nancy Flournoy and William F. Rosenberger  “Inference for a two-stage enrichment design”  In _The Annals of Statistics_ 49.5, 2021
  * [42] D. Magirr, T. Jaki and J. Whitehead  “A generalized Dunnett test for multi-arm multi-stage clinical studies with treatment selection”  In _Biometrika_ 99.2, 2012, pp. 494–501
  * [43] Baldur P. Magnusson and Bruce W. Turnbull  “Group sequential enrichment design incorporating subgroup selection”  In _Statistics in Medicine_ 32.16, 2013, pp. 2695–2714
  * [44] Nicholas A. Marston, Frederick K. Kamanu, Francesco Nordio, Yared Gurmu, Carolina Roselli, Peter S. Sever, Terje R. Pedersen, Anthony C. Keech, Huei Wang, Armando Lira Pineda, Robert P. Giugliano, Steven A. Lubitz, Patrick T. Ellinor, Marc S. Sabatine and Christian T. Ruff  “Predicting Benefit From Evolocumab Therapy in Patients With Atherosclerotic Disease Using a Genetic Risk Score”  In _Circulation_ 141.8, 2020, pp. 616–623
  * [45] Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller and Edward Teller  “Equation of state calculations by fast computing machines”  In _The journal of chemical physics_ 21.6 American Institute of Physics, 1953, pp. 1087–1092
  * [46] Kari Lock Morgan and Donald B. Rubin  “Rerandomization to improve covariate balance in experiments”  In _The Annals of Statistics_ 40.2, 2012, pp. 1263–1282
  * [47] Hans-Helge Müller and Helmut Schäfer  “Adaptive Group Sequential Designs for Clinical Trials: Combining the Advantages of Adaptive and of Classical Group Sequential Approaches”  In _Biometrics_ 57.3, 2001, pp. 886–891
  * [48] National Heart, Lung, and Blood Institute (NHLBI)  “Systolic Blood Pressure Intervention Trial (SPRINT)” Identifier: NCT01206062, https://www.clinicaltrials.gov/, 2016
  * [49] Molly Offer-Westort, Alexander Coppock and Donald P. Green  “Adaptive Experimental Design: Prospects and Applications in Political Science”  In _American Journal of Political Science_ 65.4, 2021, pp. 826–844
  * [50] Philip Pallmann, Alun W. Bedding, Babak Choodari-Oskooei, Munyaradzi Dimairo, Laura Flight, Lisa V. Hampson, Jane Holmes, Adrian P. Mander, Lang’o Odondi, Matthew R. Sydes, Sofía S. Villar, James M. S. Wason, Christopher J. Weir, Graham M. Wheeler, Christina Yap and Thomas Jaki  “Adaptive designs in clinical trials: why use them, and how to run and report them”  In _BMC Medicine_ 16.1, 2018, pp. 29
  * [51] Judea Pearl  “Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference”  Morgan Kaufmann Publishers Inc., 1988
  * [52] E. J. G. Pitman  “Significance Tests Which May be Applied to Samples From any Populations”  In _Supplement to the Journal of the Royal Statistical Society_ 4.1, 1937, pp. 119–130
  * [53] S. J. Pocock  “Group sequential methods in the design and analysis of clinical trials”  In _Biometrika_ 64.2, 1977, pp. 191–199
  * [54] Stuart J. Pocock  “Interim Analyses for Randomized Clinical Trials: The Group Sequential Approach”  In _Biometrics_ 38.1, 1982, pp. 153
  * [55] David S. Robertson, Babak Choodari-Oskooei, Munya Dimairo, Laura Flight, Philip Pallmann and Thomas Jaki  “Point estimation for adaptive trial designs I: A methodological review”  In _Statistics in Medicine_ 42.2, 2023, pp. 122–145
  * [56] Paul R. Rosenbaum  “Hodges-Lehmann Point Estimates of Treatment Effect in Observational Studies”  In _Journal of the American Statistical Association_ 88.424 Taylor & Francis, 1993, pp. 1250–1253
  * [57] William F. Rosenberger and John M. Lachin  “Randomization in Clinical Trials: Theory and Practice”  John Wiley & Sons, 2015
  * [58] William F. Rosenberger, Diane Uschner and Yanying Wang  “Randomization: The forgotten component of the randomized clinical trial”  In _Statistics in Medicine_ 38.1, 2019, pp. 1–12
  * [59] William F. Rosenberger, A. N. Vidyashankar and Deepak K. Agarwal  “Covariate-Adjusted Response-Adaptive Designs for Binary Response”  In _Journal of Biopharmaceutical Statistics_ 11.4, 2001, pp. 227–236
  * [60] Michael Rosenblum, Brandon Luber, Richard E Thompson and Daniel Hanley  “Group sequential designs with prospectively planned rules for subpopulation enrichment”  In _Statistics in medicine_ 35.21 Wiley Online Library, 2016, pp. 3776–3791
  * [61] Henry Scheffé  “The Analysis of Variance”  John WileySons, 1959
  * [62] Chris Sherlock, Paul Fearnhead and Gareth O Roberts  “The random walk Metropolis: linking theory and practice through a case study”  In _Statistical Science_ 25.2, 2010, pp. 172–190
  * [63] Richard Simon and Noah Robin Simon  “Using randomization tests to preserve type I error with response adaptive and covariate adaptive randomization”  In _Statistics & probability letters_ 81.7 Elsevier, 2011, pp. 767–772
  * [64] Amy V. Spencer, Chris Harbron, Adrian Mander, James Wason and Ian Peers  “An adaptive design for updating the threshold value of a continuous biomarker: An Adaptive Design for Updating the Threshold of a Biomarker”  In _Statistics in Medicine_ 35.27, 2016, pp. 4909–4923
  * [65] Nigel Stallard  “Adaptive enrichment designs with a continuous biomarker”  In _Biometrics_ 79.1, 2023, pp. 9–19
  * [66] Nigel Stallard, Thomas Hamborg, Nicholas Parsons and Tim Friede  “Adaptive designs for confirmatory clinical trials with subgroup selection”  In _Journal of biopharmaceutical statistics_ 24.1 Taylor & Francis, 2014, pp. 168–187
  * [67] Matthew R. Sydes, Mahesh KB Parmar, Nicholas D. James, Noel W. Clarke, David P. Dearnaley, Malcolm D. Mason, Rachel C. Morgan, Karen Sanders and Patrick Royston  “Issues in applying multi-arm multi-stage methodology to a clinical trial in prostate cancer: the MRC STAMPEDE trial”  In _Trials_ 10.1, 2009, pp. 39
  * [68] William R. Thompson  “On the likelihood that one unknown probability exceeds another in view of the evidence of two samples”  In _Biometrika_ 25.3-4, 1933, pp. 285–294
  * [69] John W. Tukey  “Comparing Individual Means in the Analysis of Variance”  In _Biometrics_ 5.2 [Wiley, International Biometric Society], 1949, pp. 99–114
  * [70] John W. Tukey  “Statistical and quantitative methodology”  In _Trends in Social Science_ New York: Philosophical Library, 1963, pp. 84–136
  * [71] Yanying Wang, William F. Rosenberger and Diane Uschner  “Randomization tests for multiarmed randomized clinical trials”  In _Statistics in Medicine_ 39.4, 2020, pp. 494–509
  * [72] Yuhao Wang and Xinran Li  “Rerandomization with diminishing covariate imbalance and diverging number of covariates”  In _The Annals of Statistics_ 50.6, 2022, pp. 3439–3465
  * [73] Jason Wu and Peng Ding  “Randomization Tests for Weak Null Hypotheses in Randomized Experiments”  In _Journal of the American Statistical Association_ 116.536, 2021, pp. 1898–1913
  * [74] Kelly Zhang, Lucas Janson and Susan Murphy  “Inference for Batched Bandits”  In _Advances in Neural Information Processing Systems_ 33 Curran Associates, Inc., 2020, pp. 9818–9829
  * [75] Kelly Zhang, Lucas Janson and Susan Murphy  “Statistical Inference with M-Estimators on Adaptively Collected Data”  In _Advances in Neural Information Processing Systems_ 34 Curran Associates, Inc., 2021, pp. 7460 –7471
  * [76] Yao Zhang and Qingyuan Zhao  “What is a Randomization Test?”  In _Journal of the American Statistical Association_ 0.0, 2023, pp. 1–15  DOI: 10.1080/01621459.2023.2199814

##  Appendix A Proof of Proposition 3.2

###### Proof.

First, we derive the factorization of the selective randomization
distribution. To this end, let
${z\in\mathcal{Z}},w\in\mathcal{W},s\in\mathcal{S}$ and $g\in\mathcal{G}$. We
repeatedly use the definition of conditional independence to construct a
factorization where each random variable/vector is only conditioned on
variables that predate it in the topological ordering of the DAG (and in
time):

$\displaystyle\mathbb{P}\big{(}$ $\displaystyle Z=z\mid
W=w,S=s,G=g\big{)}=\frac{\mathbb{P}\big{(}Z=z,W=w,S=s,G=g\big{)}}{\mathbb{P}\big{(}W=w,S=s,G=g\big{)}}$
$\displaystyle\propto\begin{aligned} \mathbb{P}\big{(}S_{K}=s_{K},&G=g\mid
Z=z,W=w,S_{[k-1]}=s_{[K-1]})\,\cdot\\\\[-2.15277pt]
\prod_{k=1}^{K}\,\,&\mathbb{P}(R_{k}=r_{k}\mid
W_{[k-1]}=w_{[k-1]},Z_{[k-1]}=z_{[k-1]},S_{[k-1]}=s_{[k-1]})\,\cdot\\\\[-6.45831pt]
&\mathbb{P}(X_{R_{k}}=x_{R_{k}},Y_{k}(\cdot)=y_{k}(\cdot)\mid
R_{k}=r_{k},W_{[k-1]}=w_{[k-1]},Z_{[k-1]}=z_{[k-1]},S_{[k-1]}=s_{[k-1]})\,\cdot\\\\[4.30554pt]
&\mathbb{P}(Z_{k}=z_{k}\mid
W_{[k]}=w_{[k]},Z_{[k-1]}=z_{[k-1]},S_{[k-1]}=s_{[k-1]})\,\cdot\\\\[4.30554pt]
&\mathbb{P}(S_{k}=s_{k}\mid
W_{[k]}=w_{[k]},Z_{[k]}=z_{[k]},S_{[k-1]}=s_{[k-1]})\end{aligned}$

To simplify this expression, we apply Assumptions (A1) and (A2). Moreover, we
use the fact that $G$ is a deterministic function of $Z$ and $W$ and $S_{k}$
is a deterministic function of $R_{[k]},X_{R_{[k]}},Y_{R_{[k]}}$ and
$Z_{[k]}$, where $k\in[K]$; this is denoted by (D). We obtain

$\displaystyle\mathrlap{\mathbb{P}\big{(}Z=z\mid W=w,S=s,G=g\big{)}}\qquad$
$\displaystyle\mathrlap{\propto\mathbf{1}\\{G(z,w)=\,g,S_{K}(z,w)=s_{K}\\}\,\cdot}\qquad$
(D) $\displaystyle\prod_{k=1}^{K}\,\,$
$\displaystyle\mathbb{P}(R_{k}=r_{k}\mid S_{k-1}=s_{k-1})\,\cdot$ (A2)
$\displaystyle\mathbb{P}(X_{R_{k}}=x_{R_{k}},Y_{k}(\cdot)=y_{k}(\cdot)\mid
R_{k}=r_{k},W_{[k-1]}=w_{[k-1]},S_{k-1}=s_{k-1})\,\cdot$ (A2)
$\displaystyle\mathbb{P}(Z_{k}=z_{k}\mid
R_{[k]}=r_{[k]},X_{R_{[k]}}=x_{R_{[k]}},Y_{R_{[k-1]}}(z_{[k-1]})=y_{R_{[k-1]}},Z_{[k-1]}=z_{[k-1]})\,\cdot$
(A1) $\displaystyle\mathbf{1}{\\{S_{k}(w_{[k]},z_{[k]})=s_{k}\\}}$ (D)
$\displaystyle\mathrlap{\propto\mathbf{1}\\{G(z,w)=\,g,S(z,w)=s\\}}\qquad$
$\displaystyle\mathbb{P}(Z_{k}=z_{k}\mid
R_{[k]}=r_{[k]},X_{R_{[k]}}=x_{R_{[k]}},Y_{R_{[k-1]}}(z_{[k-1]})=y_{R_{[k-1]}},Z_{[k-1]}=z_{[k-1]})$
$\displaystyle\mathrlap{\propto q(z\mid s,g)}\qquad$

If additionally Assumption (A1*) holds, the expression above simplifies
further:

$\begin{multlined}\mathbb{P}\big{(}Z=z\mid W=w,S=s,G=g\big{)}\\\\[-4.30554pt]
\propto\mathbf{1}{\\{G(z,w)=g,S(z,w)=s\\}}\prod_{k=1}^{K}\mathbb{P}(Z_{k}=z_{k}\mid
R_{k}=r_{k},X_{R_{k}}=x_{R_{k}},S_{k-1}=s_{k-1}).\end{multlined}\mathbb{P}\big{(}Z=z\mid
W=w,S=s,G=g\big{)}\\\\[-4.30554pt]
\propto\mathbf{1}{\\{G(z,w)=g,S(z,w)=s\\}}\prod_{k=1}^{K}\mathbb{P}(Z_{k}=z_{k}\mid
R_{k}=r_{k},X_{R_{k}}=x_{R_{k}},S_{k-1}=s_{k-1}).$

Next, we consider the selective randomization distribution of $Z^{*}$
conditional on $Z$; let $z^{*}\in\mathcal{Z}$. Using the fact that $S$ and $G$
are deterministic functions of $W$ and $Z^{*}$ or $W$ and $Z$, respectively,
and applying the conditional independence $Z^{*}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z\mid W$, we find

$\displaystyle\mathbb{P}(Z^{*}=z^{*}\mid W$
$\displaystyle=w,Z=z,S(z^{*})=S(z),G(z^{*})=G(z))$ $\displaystyle=\,\,$
$\displaystyle\mathbf{1}\\{S(z^{*})=S(z),G(z^{*})=G(z)\\}\cdot\mathbb{P}(Z^{*}=z^{*}\mid
W=w,Z=z)$ $\displaystyle=\,\,$ $\displaystyle\mathbb{P}(Z^{*}=z^{*}\mid
W=w,S(z^{*})=S(z),G(z^{*})=G(z)).$

The result now follows from expanding the formula of the selective
randomization p-value, applying the equation above as well as (4):

$\displaystyle P_{\textnormal{sel}}$
$\displaystyle=\sum\limits_{z^{*}\in\mathcal{Z}}\mathbf{1}\big{\\{}T(z^{*},W)\leq
T(Z,W)\big{\\}}\cdot\mathbb{P}(Z^{*}=z^{*}\mid W,S(z^{*})=S(Z),G(z^{*})=G(Z))$
$\displaystyle=\frac{\sum_{z^{*}\in\mathcal{Z}}\,\mathbf{1}\big{\\{}T(z^{*},W)\leq
T(Z,W)\big{\\}}\,\,q\big{(}z^{*}\mid
S(Z),G(Z)\big{)}}{\sum_{z^{*}\in\mathcal{Z}}\,q\big{(}z^{*}\mid
S(Z),G(Z)\big{)}}$

This expression can be computed because the test statistic and the treatment
assignment are assumed to be imputable. Under the stronger condition (A1*), we
can simplify the formula of the conditional probabilities $q(z^{*}\mid
S(Z),G(Z))$. ∎

##  Appendix B Details of the Simulation Study on Hold-out Units

We use the data-generating mechanism of the two-stage enrichment trial as
described in Section 5 and modify it as follows.

First, we consider the common scenario that there is treatment effect
heterogeneity between groups, cf. [44]. To this end, we generate the potential
outcomes as $Y_{i}(1)=Y_{i}(0)+1$ for participants with high genetic risk and
as $Y_{i}(1)=Y_{i}(0)$ for participants with low genetic risk; $Y_{i}(0)$ are
i.i.d. draws from a standard normal distribution. Hence, the treatment effect
in the former group is $1$ and $0$ in the latter.

Second, we lower the number of recruited patients to 16 in the first and
second stage each. This allows us to precisely compute the p-values for
different treatment effects $\tau$ without the need for a Monte Carlo
approximation. Therefore, we can exclude numerical errors as a cause of our
findings.

Third, we employ two different selection rules. For the study with hold-out
units, we decide after the first stage for which group(s) we test the
(partially) sharp null hypothesis. Hence, we use $S$ as defined in (7) with
data from the first stage. Thus, the selection depends on $Z_{1}$ but not
$Z_{2}$. For the study without hold-out units, we choose the null hypothesis
only after completing the second stage. Hence, we use $S$ (7) with data from
both stages. Consequently, $S$ depends on $Z_{1}$ and $Z_{2}$ in this
scenario.

We simulate 10 different datasets. Since the true treatment effect in the
group with high genetic risk is larger, we observe that the selection rules
choose this group in the majority of cases. Since we employ two different
rules, they may disagree, however. In order to make a fair comparison between
the two adaptive designs, we only consider scenarios where both rules choose
to test the null hypothesis for the group with high genetic risk. Among the
remaining datasets, we compute the p-value curve via discretizing the interval
$[-2,3]$. Figure 4 depicts the most jagged p-value curve we found.

The source code of this simulation study is available on Github.

##  Appendix C Details on Random Walk Metropolis-Hastings Algorithm

###  C.1 Communication Class

Recall that for a Markov chain we say that $z\in\mathcal{Z}$ and
$z^{\prime}\in\mathcal{Z}$ communicate if and only if $z\in\mathcal{Z}$ can be
reached from $z^{\prime}$ in finitely many steps of the Markov chain with
positive probability and vice versa. We denote the resulting equivalence
relation $\equiv_{C}$ and define the communication class of a state
$z\in\mathcal{Z}$ as follows

$C_{s,g}(z):=\\{z^{\prime}\in\mathcal{Z}\colon
S(z^{\prime})=s,G(z^{\prime})=g,z\equiv_{C}z^{\prime}\\},\quad\text{for
}s\in\mathcal{S},g\in\mathcal{G}.$

Analogously to Definition 1, we can define the selective randomization p-value
that additionally conditions on the communication class of the starting point
of a Markov chain

######  Definition 2.

In the setting above, let $Z^{*}$ have the same distribution as $Z$ and
fulfill $Z^{*}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}Z\,|\,W$. Then, the selective randomization
p-value conditional on the communication class is given by

$\tilde{P}_{\text{sel}}:=\mathbb{P}\Big{(}T_{S,G}(Z^{*},W)\leq
T_{S,G}(Z,W)\,\Big{|}\,W,Z,S(Z^{*})=S(Z),G(Z^{*})=G(Z),Z^{*}\in
C_{S,G}(Z)\Big{)}.$

Moreover, the modified selective randomization p-value preserves the type-I
error control.

######  Proposition C.1.

Let $\alpha\in[0,1]$. The selective randomization p-value as given in
Definition 2 stochastically dominates the uniform distribution on $[0,1]$:

$\mathbb{P}(\,\tilde{P}_{\text{sel}}\leq\alpha\,|\,W,S(Z)=s,G(Z)=g,Z^{*}\in
C_{S,G}(Z))\leq\alpha\quad\forall\,s\in\mathcal{S},g\in\mathcal{G}.$

Consequently, a test which rejects the selected null hypothesis when
$\tilde{P}_{\text{sel}}\leq\alpha$ controls the selective type-I error, that
is

$\mathbb{P}(\,\tilde{P}_{\text{sel}}\leq\alpha\mid
W,S(Z)=s)\leq\alpha\quad\forall\,s\in\mathcal{S}.$

###### Proof.

See [76, Thm. 1], where the partition of the treatment assignment space
$\mathcal{Z}$ is given by the preimages of $(S,G)$ and the equivalence
relation $\equiv_{C}$, cf. [76, Prop. 1]. ∎

###  C.2 Approximation of P-value

Here, we state and prove a more rigorous version of Proposition 4.1. We
consider the Markov chain defined by the random walk Metropolis-Hastings
Algorithm 2.

######  Proposition C.2.

For fixed $Z$, the Markov Chain restricted to the communicating class
$C_{s,g}(Z)$ is aperiodic, irreducible, and reversible at equilibrium with the
stationary distribution $\pi(\cdot)$ restricted to $C_{s,g}(Z)$. In addition,
suppose ${Z}^{(0)}=Z$, then there exists $\sigma^{2}({Z},{W})<\infty$ such
that

$\displaystyle\sqrt{M}\left(\hat{P}_{M}-\tilde{P}_{\text{sel}}({Z},{W})\right)\mid{Z},{W}\stackrel{{\scriptstyle
d}}{{\rightarrow}}\mathcal{N}\left(0,\sigma^{2}({Z},{W})\right),\quad
M\to\infty.$

###### Proof.

For any acceptable ${z}$, the probability of reaching itself is positive, and
thus the Markov Chain is aperiodic. Moreover, it is irreducible since we
restrict it to one communication class.

If the transition probability is reversible at equilibrium with the stationary
distribution $\pi(\cdot)$ restricted to $C_{s,g}(Z)$, then $\pi(\cdot)$
restricted to $C_{s,g}(Z)$ is the stationary distribution. By [40, Thm. 4.9],
the Markov Chain is then geometrically ergodic on $C_{s,g}(Z)$. Since any
geometrically ergodic reversible Markov chain satisfies a central limit
theorem for all functions with a finite second moment with respect to the
stationary distribution [27] and that the indicator function
$\mathbf{1}{\\{T({z},{W})\leq T({Z},{W})\\}}$ is bounded, we obtain the
desired central limit theorem like convergence result.

Hence, it is left to show that the transition probability is reversible at
equilibrium with the stationary distribution $\pi(\cdot)$ restricted to
$C_{s,g}(Z)$. Let $\Sigma({z})$ be the set of operations $\sigma=\kappa\circ
s$, where $s$ denotes the operation of randomly picking $h_{k}$ entries of
$z_{k}$, $k\in\\{1,\ldots,K\\}$, and $\kappa$ denotes randomly shuffling the
$h_{k}$ entries. Note that $|\Sigma({z})|$ does not depend on ${z}$. Let
$\Sigma({z}\to{z}^{*}):=\\{\sigma\in\Sigma({z}):\sigma({z})={z}^{*}\\}$ be the
set of operations in $\Sigma({z})$ transitioning ${z}$ to ${z}^{*}$. Note that
if $\kappa\circ s({z})={z}^{*}$, then $\kappa^{-1}\circ s({z}^{*})={z}$. This
implies there is an injection mapping from $\Sigma({z}\to{z}^{*})$ to
$\Sigma({z}^{*}\to{z})$. By symmetry, the mapping is a bijection, and we
obtain $|\Sigma({z}\to{z}^{*})|=|\Sigma({z}^{*}\to{z})|$.

Note that the proposal distribution of the Markov chain is given by

$\displaystyle
p({z}^{*}\mid{z})=\frac{|\Sigma({z}\to{z}^{*})|}{|\Sigma({z})|},\quad{z}^{*}\in{\mathcal{Z}},$

which satisfies

$\displaystyle
p({z}\mid{z}^{*})=\frac{|\Sigma({z}\to{z}^{*})|}{|\Sigma({z})|}=\frac{|\Sigma({z}^{*}\to{z})|}{|\Sigma({z}^{*})|}=p({z}^{*}\mid{z}).$

Recall the acceptance probability is
$\alpha({z}^{*}|{z})=\mathbf{1}\\{{z}^{*}\in C_{s,g}(z)\\}$, then the
probability of the new proposal being accepted is given by

$\displaystyle\alpha({z})=\sum_{{z}^{*}\in
C_{s,g}(z)}\frac{|\Sigma({z}\to{z}^{*})|}{|\Sigma({z})|}.$

The transition probability matrix takes the form

$\displaystyle\mathrm{P}({z},{z}^{*})=p({z}^{*}\mid{z})\,\mathbf{1}\\{{z}^{*}\in
C_{s,g}({z})\\}+(1-\alpha({z}))\,\mathbf{1}\\{{z}^{*}={z}\\}.$

Note that $\pi(\cdot)$ restricted to $C_{s,g}({z})$ in this case is the
uniform distribution $\mathcal{U}(C_{s,g}({z}))$. For ${z}^{*}\in
C_{s,g}({z})$, ${z}^{*}\neq{z}$,

$\displaystyle\pi({z})\,{\mathrm{P}}({z},{z}^{*})=\pi({z})\,p({z}^{*}\mid{z})=\pi({z}^{*})\,p({z}\mid{z}^{*})=\pi({z}^{*})\,{\mathrm{P}}({z}^{*},{z}),$

which yields the desired reversibility. ∎

##  Appendix D Additional Results on the RWM Sampler

We vary the window size of the RWM sampler in $\\{2,5,10,15\\}$ and
demonstrate the computation time as well as the rejection probability at
different null hypotheses, see Figure 7.

As the window size increases, the running time largely remains the same (or
increases only mildly). This is because, regardless if the proposal is
accepted or not, the RWM sampler will proceed to the next iteration, in
contrast to the rejection sampling algorithm. The RWM sampler is slower than
RT 2nd and RT due to the extra step verifying whether the proposal is
acceptable.

The rejection probability remains relatively consistent across window sizes,
possibly because the communicating classes associated with a small window size
$h=2$ is already sufficiently large. The power comparison of SRT, RT 2nd, and
RT is similar to that in Section 5.

Figure 7: Comparison of computation time and rejection probability of the RWM
sampler with different window sizes. We follow the simulation setting in
Section 5 and vary the window size in $\\{2,5,10,15\\}$.

##  Appendix E Additional Real Data Analysis

We perform a placebo analysis evaluating the type-I error. To this end, we
generate artificial treatment assignments following independent
Bernoulli$(0.5)$ multiple times for the control group. As in Section 6, in the
first stage, we randomly sample $2000$ units without replacement but only from
the original control group, compute the relative risk for each age group based
on the artificial treatment assignment, and select the age group with the
minimal relative risk (the most significant improvement). In the second stage,
we randomly sample an additional $200$ units without replacement of the
selected age group, originally under control, and not recruited in the first
stage. We repeat from generating artificial treatments multiple times and, as
in Section 6, focus on the trials where the age group “no younger than $80$”
is selected. All the units in the constructed dataset did not receive the
treatment, and the null hypothesis that the artificial treatment has no
treatment effect is true. As indicated by the CDF curves of the p-values in
Figure 8(a), which are based on $1000$ trials with the target selection, the
RT fails to control the type I error while SRT and RT 2nd do.

We repeat the power comparison in Section 6 from constructing the hypothetical
real dataset via subsampling multiple times. We focus on the trials where the
age group “no younger than $80$” is selected. As indicated by the CDF curves
of the p-values in Figure 8(b), the p-value from the SRT is stochastically
dominated by that of the RT 2nd. An analysis of the treatment effect in the
age group “no younger than $80$” based on the original SPRINT trial rejects
the null of no treatment effect, and the p-value CDFs indicate that the SRT is
more powerful than RT 2nd. In particular, the rejection probability at the
significance level $0.05$ is $35.3\%$ for SRT but only $25.8\%$ for RT 2nd.

(a) Type-I error

(b) Power

Figure 8: Hypothetical two-stage enrichment trials generated from the SPRINT
dataset.