# The Cost of Sequential Adaptation and the Lower Bound for Mean Squared Error

Sergey Tarima
Institute for Health and Equity, Medical College of Wisconsin
and
Nancy Flournoy
Department of Statistics, University of Missouri

###### Abstract

Informative interim adaptations lead to random sample sizes. The random sample
size becomes a component of the sufficient statistic and estimation based
solely on observed samples or on the likelihood function does not use all
available statistical evidence. The total Fisher Information (FI) is
decomposed into the design FI and a conditional-on-design FI. The FI unspent
by the interim adaptation is used to determine the lower mean squared error in
post-adaptation estimation. Theoretical results are illustrated with simple
normal samples collected according to a two-stage design with a possibility of
early stopping.

Keywords: Adaptive designs, adapted support, Cramer-Rao lower bound, group
sequential designs, Fisher information, interim hypothesis testing

##  1 Introduction

Study designs with data-dependent sample sizes provide an attractive
alternative to fixed sample size designs because interim decisions may reduce
cost and decrease the duration of the study. Two groups of such sequential
procedures, namely, groups sequential designs (GSD) and sample size re-
estimation (SSR), are popular examples of studies with random sample sizes
[see, for example, Jennison and Turnbull (1999); Proschan et al. (2006);
Chuang-Stein et al. (2006); Friede and Kieser (2006)]. Our manuscript mostly
deals with GSDs and SSR, but other informative adaptive designs may also
benefit from the theoretical findings reported here.

The benefits of interim adaptations come with certain costs as the
distributions of sample-based statistics change [Armitage et al. (1969); Efron
et al. (1975); Siegmund (1985); Liu and Hall (1999); Liu et al. (2006); Tarima
and Flournoy (2022)]. Some recognized consequences are that Bayesian
procedures (that typically are not adjusted for multiple testing) do not
control Type 1 error [see Table 18.1 in Jennison and Turnbull (1999)], maximum
likelihood estimators become biased [Whitehead (1986)], and Wald confidence
intervals do not provide the desired coverage after GSDs or SSRs. In this
paper we use FI to quantify the information loss associated with interim
decisions using interim data that depends on the parameter of interest.

For simplicity of exposition, we focus on two-stage adaptive designs with a
single interim analysis. Extension to multiple interim analyses follows from
recursive application of the presented methods. For illustrative purposes we
use a normal example with arbitrary sample sizes $n_{1}$ and $n_{2}$ for stage
1 and 2, respectively.

Section 2 introduces notation in the context of two-stage designs without
possibility of early stopping. Section 3 shows the impact of the possibility
of early stopping on support, sample space, and data distributions. In Section
4, FI is decomposed into the portion consumed by the interim decision
[analogous to the Design Information in Marschner (2021) that strictly
pertained to the Observed Information] and the remainder which is represented
as a weighted average of FI conditional on the possible interim decisions.
Section 6 illustrates results with numerical calculations for a simple normal
example with $X_{i}\sim N(\theta,1)$, $i=1,2$ (that is, $n_{1}=n_{2}=1$) under
a design that has a possibility of early stopping after observing $X_{1}$ if
$X_{1}>1.96$. Section 7 concludes the manuscript with a short summary.

New results reported in Sections 3, 4 and 5 are illustrated in Section 6.
Section 2 is necessary for understanding.

##  2 Notation and Review of Basics for Independent Observations

Feature | Description
---|---
r.v. | random variable
$\theta$ | parameter of interest
$\mathbf{X}_{d}$ | stage-specific sample of r.v.s $\left(X_{n_{(d-1)}+1},\ldots,X_{n_{(d)}}\right)$
$\mathbf{X}_{(d)}$ | cumulative sample of r.v.s $\left(X_{1},\ldots,X_{n_{(d)}}\right)$
$\mathbf{x}_{d}$ | a realization of $\mathbf{X}_{d}$
$\mathbf{x}_{(d)}$ | a realization of $\mathbf{X}_{(d)}$
$n_{d}$ | stage-specific sample size
$n_{(d)}$ | $n_{(d)}=\sum_{k=1}^{d}n_{d}$
$t_{d}$ | $=\sum_{n_{(d-1)}+1}^{n_{(d)}}X_{i}$
$t_{(d)}$ | $=\sum_{1}^{n_{d}}X_{i}$
$Z_{d}$ | $=t_{d}/\sqrt{n_{d}}$
$Z_{(d)}$ | $=t_{(d)}/\sqrt{n_{(d)}}$
$\hat{\theta}_{d}$ | stage-specific MLE $=t_{d}/n_{d}$
$\hat{\theta}_{(d)}$ | cumulative MLE $=t_{(d)}/n_{(d)}$
$\tilde{\theta}_{d}$ | a stage-specific statistic $\tilde{\theta}_{d}=\tilde{\theta}_{d}\left(\mathbf{X}_{d}\right)$
$\tilde{\theta}_{(d)}$ | a cumulative statistic $\tilde{\theta}_{(d)}=\tilde{\theta}_{(d)}\left(\mathbf{X}_{(d)}\right)$
${\cal{T}}$ | adapted-to-design support
$\mathbf{X}_{\cal{T}}$ |  $\mathbf{X}=\left(\mathbf{X}_{1},\mathbf{X}_{2}\right)$ defined on ${\cal{T}}$
${\cal{I}}_{\mathbf{X}_{d}}(\theta)$ | state-specific expected Fisher information
${\cal{I}}_{\mathbf{X}_{d}=\mathbf{x}_{d}}^{obs}(\theta)$ | state-specific observed Fisher information
${\cal{I}}_{\mathbf{X}_{\cal{T}}}(\theta)$ | cumulative expected Fisher information, or total if $d=2$
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D}(\theta)$ | conditional-on-design cumulative expected Fisher information
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D=d}(\theta)$ | conditional-on-a-realized-design expected Fisher information
Table 1: Notation: decisions $d=1,2$; $n_{0}=0$; $\prod_{i=1}^{0}[\cdot]=1$.

In this section, we assume that there is no impact of interim analyses and the
data are aggregated into two samples only for convenience. A summary of our
notation is given in Table 1.

Let $\mathbf{X}_{1}=\left(X_{1},\ldots,X_{n_{1}}\right)$ and
$\mathbf{X}_{2}=\left(X_{n_{1}+1},\ldots,X_{n_{1}+n_{2}}\right)$ be two
samples of independent random variables with $X_{i}\sim
f_{X}\left(x|\theta\right)$. Joint $d$-stage-specific densities
$f_{\mathbf{X}_{d}}\left(\mathbf{X}_{d}|\theta\right)$ are used to define the
FI in $\mathbf{X}_{d}$, $d\in\\{1,2\\}$:

${\cal{I}}_{\mathbf{X}_{d}}(\theta)=Var\left[\frac{\partial}{\partial\theta}\log
f_{\mathbf{X}_{d}}(\mathbf{x}|\theta)\right].$

If $\mathbf{X}_{d}=\mathbf{x}_{d}$ alone is observed, we assume that the log-
likelihood function $l(\theta|\mathbf{X}_{d}=\mathbf{x}_{d})$ has a unique
maximum at $\widehat{\theta}_{d}$. If both $\mathbf{X}_{1}=\mathbf{x}_{1}$ and
$\mathbf{X}_{2}=\mathbf{x}_{2}$ are observed independently,

$l(\theta|\mathbf{X}_{1}=\mathbf{x}_{1},\mathbf{X}_{2}=\mathbf{x}_{2})=l(\theta|\mathbf{X}_{1}=\mathbf{x}_{1})+l(\theta|\mathbf{X}_{2}=\mathbf{x}_{2})$

denotes their log-likelihood function and its maximum is denoted
$\widehat{\theta}_{(2)}$. Note that we use parentheses to distinguish
estimators based on cumulative data (e.g., $\hat{\theta}_{(2)}$ uses data from
both stages 1 and 2) from stage-specific estimators (e.g., $\hat{\theta}_{2}$
is based on stage 2 data only).

###  2.1 Cramer-Rao Lower Bound (CRLB) if no early stopping is possible

Let $\tilde{\theta}_{d}=\tilde{\theta}_{d}\left(\mathbf{X}_{d}\right)$ be a
stage-specific estimator for which
$\textrm{E}[\tilde{\theta}_{d}]=\theta+b_{d}(\theta)$ and
$\left(\partial/\partial\theta\right)f_{\mathbf{X}_{d}}(t|\theta)$ exist and
$\partial/\partial\theta$ can be passed under the integral sign in $\int
f_{\mathbf{X}_{d}}(t|\theta)dt=1$ and $\int t\
f_{\mathbf{X}_{d}}(t|\theta)dt$. Then

$\textrm{Var}[\tilde{\theta}_{d}]\geq\frac{\left[1+\frac{\partial}{\partial\theta}b_{d}(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{d}}(\theta)}$
(1)

and

$\textrm{E}\left[(\tilde{\theta}_{d}-\theta)^{2}\right]\geq\frac{\left[1+\frac{\partial}{\partial\theta}b_{d}(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{d}}(\theta)}+b_{d}^{2}(\theta).$
(2)

In a similar manner, for an estimator $\tilde{\theta}_{(2)}$ defined
$\mathbf{X}_{(2)}=\left(\mathbf{X}_{1},\mathbf{X}_{2}\right)$:

$\textrm{Var}[\tilde{\theta}_{(2)}]\geq\frac{\left[1+\frac{\partial}{\partial\theta}b_{(2)}(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{(2)}}(\theta)}$
(3)

and

$\textrm{E}\left[(\tilde{\theta}_{(2)}-\theta)^{2}\right]\geq\frac{\left[1+\frac{\partial}{\partial\theta}b_{(2)}(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{(d)}}(\theta)}+b^{2}_{(2)}(\theta),$
(4)

where $\textrm{E}[\tilde{\theta}_{(2)}]=\theta+b_{(2)}(\theta)$. If
$\mathbf{X}_{1}$ and $\mathbf{X}_{2}$ are independent
${\cal{I}}_{\mathbf{X}_{(2)}}={\cal{I}}_{\mathbf{X}_{1}}+{\cal{I}}_{\mathbf{X}_{2}}$.

The Cramer-Rao lower bound (CRLB) was suggested independently by Rao (1945)
and Cramér (1946) under the assumption of fixed sample sizes.

###  2.2 Normal Example

Let $t_{d}=\sum_{i=n_{(d-1)}+1}^{n_{(d)}}x_{i}$. Now if $X_{i}\sim
N(\theta,1)$, the stage-specific MLE is $\hat{\theta}_{d}=n_{d}^{-1}t_{d}\sim
N(\theta,n_{d}^{-1})$ with FI

${\cal{I}}_{\mathbf{X}_{d}}(\theta)={\cal{I}}^{obs}_{\mathbf{X}_{d}=\mathbf{x}_{d}}(\theta)=-\left(\partial^{2}/\partial\theta^{2}\right)l(\theta|\mathbf{X}_{d}=\mathbf{x}_{d})=n_{d}.$

The random variable $(\hat{\theta}_{1},\hat{\theta}_{2})$ is defined on the
probability space $\left(R^{2},{\cal{B}},P\right)$, where $R^{2}$ is the
sample space [$R=\left(-\infty,\infty\right)$]; ${\cal{B}}$ is the Borel
$\sigma$-algebra on $R^{2}$; and if both $\mathbf{X}_{1}=\mathbf{x}_{1}$ and
$\mathbf{X}_{2}=\mathbf{x}_{2}$ are observed independently, the probability
measure $P$ is a bivariate normal distribution with mean $(\theta,\theta)$,
variances $n_{1}^{-1}$ and $n_{2}^{-1}$, and zero correlation. Then, the
observed and expected Fisher informations are equal and

${\cal{I}}^{obs}_{\mathbf{X}_{1}=\mathbf{x}_{1},\mathbf{X}_{2}=\mathbf{x}_{2}}(\theta)=-\frac{\partial^{2}}{\partial\theta^{2}}l(\theta|\mathbf{X}_{1}=\mathbf{x}_{1})=-\frac{\partial^{2}}{\partial\theta^{2}}l(\theta|\mathbf{X}_{2}=\mathbf{x}_{2})=n_{1}+n_{2}.$

##  3 Two-Stage Experiments with the Possibility of Early Stopping

Let $Z_{1}=Z_{1}\left(\mathbf{X}_{1}\right)$ be an interim test statistic
based on stage one data alone. What is the distribution of two stage-specific
statistics $\tilde{\theta}_{1}=\tilde{\theta}_{1}\left(\mathbf{X}_{1}\right)$
and $\tilde{\theta}_{2}=\tilde{\theta}_{2}\left(\mathbf{X}_{2}\right)$ in
sequential settings when $\tilde{\theta}_{2}$ is only observed if
$z_{1}<c_{1}$, where $c_{1}$ is some pre-determined critical value?

Let $D$ denote an interim rule for choosing a decision from a decision space
${\cal{D}}$. In GSDs with one interim analysis, the interim decision rule is

${\cal{D}}=\begin{cases}1=\text{`stop'}&\text{if }z_{1}\geq c_{1};\\\
2=\text{`collect }n_{2}\text{ extra observations'}&\text{if
}z_{1}<c_{1}.\end{cases}$ (5)

###  3.1 The Probability Distribution of the Observable Random Variables

In such sequential settings, the joint support of
$(\tilde{\theta}_{1},\tilde{\theta}_{2}$) changes from that of independent
statistics as $\tilde{\theta}_{2}$ is not just missing but impossible when
$Z_{1}\geq c_{1}$. Tarima and Flournoy (2022) observe that because some
sampling combinations of $\tilde{\theta}_{1}$ and $\tilde{\theta}_{2}$ are not
observable, the support of $\tilde{\theta}_{1}$ and $\tilde{\theta}_{2}$
becomes

${\cal{T}}=\left\\{\tilde{\theta}_{1}|z_{1}\geq
c_{1}\right\\}\cup\left\\{\\{\tilde{\theta}_{1}|z_{1}<c_{1}\\}\cap\\{\tilde{\theta}_{2}\in
R\\}\right\\}.$

Note that ${\cal{T}}\subset R^{2}$. If $A\in{\cal{T}}$, then the probability
measure on the $\sigma$-algebra, $\sigma({\cal{T}})$ is

$\displaystyle\mu_{{\cal{T}}}=\text{Pr}[(\tilde{\theta}_{1},\tilde{\theta}_{2})\in
A]$ $\displaystyle=\text{Pr}(Z_{1}\geq c_{1})\text{Pr}(\tilde{\theta}_{1}\in
A|Z_{1}\geq c_{1})$
$\displaystyle\qquad\qquad+\text{Pr}(Z_{1}<c_{1})\text{Pr}[(\tilde{\theta}_{1},\tilde{\theta}_{2})\in
A|Z_{1}<c_{1}].$

On the new probability space
$\\{{\cal{T}},\sigma({\cal{T}}),\mu_{{\cal{T}}}\\}$, we have an observable
random variable, namely,

$\displaystyle\tilde{\theta}=\begin{cases}\tilde{\theta}_{1}&\textrm{ if
}$D=1$\\\ (\tilde{\theta}_{1},\tilde{\theta}_{2})&\textrm{ if
}$D=2$.\end{cases}$

Factor the density of ${\tilde{\theta}_{1}}$ as

$f_{\tilde{\theta}_{1}}(y_{1})=\sum_{d=1}^{2}f_{D}(d)[f_{\tilde{\theta}_{1}|D}(y_{1}|d)]^{I(D=d)}=f_{D}(d)[f_{\tilde{\theta}_{1}|D}(y_{1}|d)]^{I(D=d)},$
(6)

where $f_{D}(d)=\prod_{d=1}^{2}[\text{Pr}(D=d)]^{I(D=d)}$ is the probability
distribution of the Bernoulli random variable $D$ and
$f_{\tilde{\theta}_{1}|D=d}$ is a random probability distribution for
different realizations of $D$:

$\displaystyle
f_{\tilde{\theta}_{1}|D=d}(y_{1}|d)=f_{\tilde{\theta}_{1}}(y_{1})\frac{I(D=d)}{\text{Pr}(D=d)}=\frac{\left[f_{\tilde{\theta}_{1}}(y_{1})\right]^{I(D=d)}}{\text{Pr}(D=d)}.$
(7)

Alternatively, $\tilde{\theta}_{1}$ can be viewed as a mixture of
$\tilde{\theta}_{1}|D=1$ and $\tilde{\theta}_{1}|D=2$ with

$f_{\tilde{\theta}_{1}}(y_{1})=\text{Pr}(D=1)f_{\tilde{\theta}_{1}|D=1}(y_{1}|D=1)+\text{Pr}(D=2)f_{\tilde{\theta}_{1}|D=2}(y_{1}|D=2).$
(8)

The same decomposition applied to $f_{\tilde{\theta}}(\mathbf{y})$ with
$\mathbf{y}=(y_{1},y_{2})$ provides

$f_{\tilde{\theta}}(\mathbf{y})=\text{Pr}(D=1)f_{\tilde{\theta}_{1}|D=1}(y_{1}|D=1)+\text{Pr}(D=2)f_{\left(\tilde{\theta}_{1},\tilde{\theta}_{2}\right)|D=2}(\mathbf{y}|D=2).$
(9)

Thus, $f_{\tilde{\theta}_{1}}(y_{1})$ and $f_{\tilde{\theta}}(\mathbf{y})$
both can be described as mixtures of distributions.

Using $f_{\tilde{\theta}_{1}|D=1}(y_{1}|D=1)$ from Equation (7) and the
analogous expression

$\displaystyle
f_{\tilde{\theta}|D=2}(\mathbf{y}|D=2)=f_{\left(\tilde{\theta}_{1},\tilde{\theta}_{2}\right)}(\mathbf{y})\frac{I(D=2)}{\text{Pr}(D=2)}=\frac{\left[f_{\left(\tilde{\theta}_{1},\tilde{\theta}_{2}\right)}(\mathbf{y})\right]^{I(D=2)}}{\text{Pr}(D=2)}$
(10)

in Equation (9), another useful representation for the density
$f_{\tilde{\theta}}(\mathbf{y})$ is

$\displaystyle f_{\tilde{\theta}}(\mathbf{y})$
$\displaystyle=\left[f_{\tilde{\theta}_{1}}(y_{1})\right]^{I(D=1)}+\left[f_{\left(\tilde{\theta}_{1},\tilde{\theta}_{2}\right)}(\mathbf{y})\right]^{I(D=2)}$
$\displaystyle=\left[f_{\tilde{\theta}_{1}}(y_{1})\right]^{I(D=1)}\left[f_{\left(\tilde{\theta}_{1},\tilde{\theta}_{2}\right)}(\mathbf{y})\right]^{I(D=2)},$
(11)

where the right-hand side of the mixture density can be represented as a sum
or as a product of two subdensities.

###  3.2 The Likelihood Function’s Insensitivity to Early Stopping
Possibilities

The log-likelihood function is conditional on $\mathbf{X}_{1}=\mathbf{x}_{1}$,
and on $\mathbf{X}_{2}=\mathbf{x}_{2}$ if stage-two observations are made, and
consequently on the stopping stage $d$:

$\displaystyle
l(\theta|\mathbf{X}_{1}=\mathbf{x}_{1},\mathbf{X}_{2}=\mathbf{x}_{2},D=d)=\begin{cases}l(\theta|\mathbf{X}_{1}=\mathbf{x}_{1})&\textrm{
if }D=1,\\\
l(\theta|\mathbf{X}_{1}=\mathbf{x}_{1},\mathbf{X}_{2}=\mathbf{x}_{2})&\textrm{
if }D=2.\end{cases}$

Importantly, the log-likelihood function is the same as in the case of
independent observations without the possibility of early stopping. More
generally, the likelihood is insensitive to the effect of early stopping
mechanisms. This insensitivity is inherited by the score function, the MLE,
and the observed Fisher information.

The expected Fisher information, however, changes with early stopping options
[see Section 4]. Consider three experiments with $X_{i}\sim N(\theta,1)$:

$E_{1}:$

An experiment with no early stopping having a fixed sample size $n_{1}+n_{2}$
and a joint sampling density
$\prod_{i=1}^{n_{1}+n_{2}}f_{X_{i}}(x_{i}|\theta)$.

$E_{2}:$

A sequential experiment with early stopping if $\sqrt{n_{1}}\bar{X}_{1}>1.96$
[motivated by a one-stage experiment with $\alpha=0.025$ and no planned
interim test] has joint sampling density
$\prod_{i=1}^{n_{1}}f_{X_{i}}(x_{i}|\theta)\left[\prod_{i=n_{1}+1}^{n_{1}+n_{2}}f_{X_{i}}(x_{i}|\theta)\right]^{I\left(\sqrt{n_{1}}\bar{X}_{1}{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}<}1.96\right)}$.

$E_{3}:$

A sequential experiment with early stopping if $\sqrt{n_{1}}\bar{X}_{1}>2.78$
[motivated by the O’Brien and Fleming (1979) stopping boundary] has joint
sampling density
$\prod_{i=1}^{n_{1}}f_{X_{i}}(x_{i}|\theta)\left[\prod_{i=n_{1}+1}^{n_{1}+n_{2}}f_{X_{i}}(x_{i}|\theta)\right]^{I\left(\sqrt{n_{1}}\bar{X}_{1}{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}<}2.78\right)}$.

The sampling density in $E_{1}$ is defined on ${R}^{2}$, whereas the densities
under $E_{2}$ and $E_{3}$ have support $\mathbf{\cal{T}}$ as described in
Section 3.1.

Suppose each of the three experiments has been run, and $E_{2}$ and $E_{3}$
did not stop early because $\sqrt{n_{1}}\bar{X}_{1}>1.96$ and
$\sqrt{n_{1}}\bar{X}_{1}>2.78$ are observed, respectively. Suppose further
that same data (outcomes) are observed in all three experiments:
$\mathbf{X}_{1}=\mathbf{x}_{1}$ and $\mathbf{X}_{2}=\mathbf{x}_{2}$. Then
despite different underlying designs and densities, all experiments ($E_{1}$,
$E_{2}$ and $E_{3}$) share the same likelihood function:

$L(\theta|X_{1}=x_{1},\ldots,X_{n_{1}+n_{2}}=x_{n_{1}+n_{2}})=\prod_{i=1}^{n_{1}+n_{2}}f_{X_{i}}(x_{i}|\theta).$
(12)

The likelihood $L(\theta|X_{1},\ldots,X_{n_{1}+n_{2}})$ is a random variable,
but its generating distributions differ across the three experiments, which
illustrates that $L(\theta|X_{1},\ldots,X_{n_{1}+n_{2}})$ is insensitive to
the study design. This is the same phenomenon as the well-known
proportionality of the binomial and negative binomial likelihoods [see Example
9 in Berger and Wolpert (1988) and Example 6.3.7 on page 295 in Casella and
Berger (2002)].

###  3.3 Normal Example (Two-Stage Design, Arbitrary $n_{1}$ and $n_{2}$)

The log-likelihood function is

$\displaystyle
l\left(\theta\Big{|}\hat{\theta}_{(d)}=\frac{t_{(d)}}{n_{(d)}},D=d\right)=\frac{1}{2\pi}\exp\left(-\frac{1}{2}n_{(d)}\left(t_{(d)}-\theta\right)^{2}\right),$

where $t_{(d)}=\sum_{i=1}^{n_{(d)}}x_{i}$, and the value of
$\hat{\theta}_{(2)}$ is not observed when $D=2$. The interim test statistic is
$Z_{1}=t_{1}/\sqrt{n_{1}}$ and the MLE

$\displaystyle\widehat{\theta}$ $\displaystyle=I(z_{1}\geq
c_{1})\frac{t_{1}}{n_{1}}+I(z_{1}<c_{1})\left(\frac{t_{(2)}}{n_{(2)}}\right)$
$\displaystyle=I(z_{1}\geq
c_{1})\hat{\theta}_{1}+I(z_{1}<c_{1})\left(\frac{n_{1}}{n_{(2)}}\hat{\theta}_{1}+\frac{n_{2}}{n_{(2)}}\hat{\theta}_{2}\right)$
$\displaystyle=I(z_{1}\geq
c_{1})\hat{\theta}_{1}+I(z_{1}<c_{1})\hat{\theta}_{(2)},$ (13)

is a random variable defined on
$\left({\cal{T}},\sigma({\cal{T}}),\mu_{{\cal{T}}}\right)$. Significantly,
$\widehat{\theta}$ is a mixture of a left truncated normal random variable
$\\{\hat{\theta}_{1}|D=1\\}$ and the weighted average of a right truncated
normal $\\{\hat{\theta}_{1}|D=2\\}$ and a normal $\hat{\theta}_{2}$. Thus, in
this sequential setting, $\widehat{\theta}$ is no longer a normal random
variable.

##  4 Fisher Information with Informative Interim Decisions

To this point we have focused on simple binary interim decisions (Section
4.1), but to accommodate SSR procedures, Section 4.2 introduces a wider
decision space and more complex decision functions; FI is decomposed to expose
the proportion consumed by the interim decision options for both GSD and SSR
procedures.

###  4.1 Fisher Information with an Interim Test

Assuming standard regularity conditions [e.g. Ferguson (1996)], FI in
$\mathbf{X}_{1}$ conditional on the interim decision variable $D$ is the
variance of the score function for $f_{\mathbf{X}_{1}|D}(y_{1}|D)$:

$\displaystyle{\cal{I}}_{\mathbf{X}_{1}|D}(\theta)$
$\displaystyle=\textrm{Var}\left[\frac{\partial}{\partial\theta}\log
f_{\mathbf{X}_{1}|D}(y_{1}|D)\right]$
$\displaystyle=\int_{-\infty}^{\infty}-\frac{\partial^{2}}{\partial\theta^{2}}[\log
f_{\mathbf{X}_{1}|D}(y_{1}|D)]f_{\mathbf{X}_{1}}(\mathbf{x}_{1})d\mathbf{x}_{1}.$
(14)

In contrast, FI in $\mathbf{X}_{1}$ conditional on a specific realization $d$
of an interim decision is

$\displaystyle{\cal{I}}_{\mathbf{X}_{1}|D=d}(\theta)=\int_{\infty}^{\infty}I(D=d)\frac{\partial^{2}}{\partial\theta^{2}}\left[-\log
f_{\mathbf{X}_{1}|D}(\mathbf{x}_{1}|d)\right]f_{\mathbf{X}_{1}|D}(\mathbf{x}_{1}|d)d\mathbf{x}_{1}$
$\displaystyle=\begin{cases}\displaystyle\int_{-\infty}^{\infty}\frac{\partial^{2}}{\partial\theta^{2}}\left[-\log\frac{f_{\mathbf{X}_{1}}(\mathbf{x}_{1})}{\text{Pr}(z_{1}\geq
c_{1})}\right]\frac{f_{\mathbf{X}_{1}}(\mathbf{x}_{1})I(z_{1}\geq
c_{1})}{\text{Pr}(z_{1}\geq c_{1})}d\mathbf{x}_{1}&\text{ if }z_{1}\geq
c_{1};\\\\[10.0pt]
\displaystyle\int_{-\infty}^{\infty}\frac{\partial^{2}}{\partial\theta^{2}}\left[-\log\frac{f_{\mathbf{X}_{1}}(\mathbf{x}_{1})}{\text{Pr}(z_{1}<c_{1})}\right]\frac{f_{\mathbf{X}_{1}}(\mathbf{x}_{1})I(z_{1}<c_{1})}{\text{Pr}(z_{1}<c_{1})}d\mathbf{x}_{1}&\text{
if }z_{1}<c_{1}.\end{cases}$ (15)

We note that the integration is multiple to highlight the fact that
$\mathbf{x}_{1}$ has $n_{1}$ components and each is integrated out.

The information measures defined by Equations (14) and (4.1) differ by the
type of conditioning: ${\cal{I}}_{\mathbf{X}_{1}|D}(\theta)$ averages over
distributions of $\mathbf{X}_{1}$ (and $D$ is fully determined by
$\mathbf{X}_{1}$), whereas ${\cal{I}}_{\mathbf{X}_{1}|D=d}(\theta)$ averages
over the distribution of the conditional random variable $\mathbf{X}_{1}|D=d$.
Conditioning on a random variable, as in (14), is used in Zegers (2015) for
example, while conditioning on an observed constant, as in (4.1), is used in
Mihoc and Ioana Fatu (2003).

While in this manuscript, we condition is on $D=d$ unless stated otherwise.
However, ${\cal{I}}_{\mathbf{X}_{1}|D}(\theta)$ provides an important measure
of the information left in $\mathbf{X}_{1}$ after informative adaptation.
Taking the $\log$ of the joint density
$f_{\mathbf{X}_{1}}(\mathbf{x}_{1})=f_{D}(d)f_{\mathbf{X}_{1}|D}(\mathbf{x}_{1}|d)^{I(D=d)}$
and using the law of total variance on the score function, FI in
$\mathbf{X}_{1}$ and $\mathbf{X}_{\cal{T}}$, respectively, can be written as

$\displaystyle{\cal{I}}_{\mathbf{X}_{1}}(\theta)$ $\displaystyle=$
$\displaystyle{\cal{I}}_{D}(\theta)+{\cal{I}}_{\mathbf{X}_{1}|D}(\theta)$ (16)
$\displaystyle=$
$\displaystyle{\cal{I}}_{D}(\theta)+\text{Pr}(D=1|\theta){\cal{I}}_{\mathbf{X}_{1}|D=1}(\theta)+\text{Pr}(D=2|\theta){\cal{I}}_{\mathbf{X}_{1}|D=2}(\theta)$

and

$\displaystyle{\cal{I}}_{\mathbf{X}_{\cal{T}}}(\theta)=$
$\displaystyle{\cal{I}}_{D}(\theta)+\text{Pr}\left(D=1|\theta\right){\cal{I}}_{\mathbf{X}_{1}|D=1}(\theta)+\text{Pr}\left(D=2|\theta\right)\left[{\cal{I}}_{\mathbf{X}_{1}|D=2}(\theta)+{\cal{I}}_{\mathbf{X}_{2}}(\theta)\right]$
$\displaystyle=$
$\displaystyle{\cal{I}}_{D}(\theta)+{\cal{I}}_{\mathbf{X}|D}(\theta)$
$\displaystyle=$
$\displaystyle{\cal{I}}_{D}(\theta)+{\cal{I}}_{\mathbf{X}_{1}}(\theta)+\text{Pr}\left(D=2|\theta\right){\cal{I}}_{\mathbf{X}_{2}}(\theta),$
(17)

where ${\cal{I}}_{D}(\theta)$ is the portion of the total FI that is consumed
by permitting one interim test in a two-stage design (the design information):

${\cal{I}}_{D}(\theta)=\textrm{Var}\left[\frac{\partial}{\partial\theta}\log
f_{D}(d)\right]=\sum_{d=1}^{2}\frac{\partial^{2}}{\partial\theta^{2}}\left[-\log\text{Pr}(D=d|\theta)\right]\text{Pr}(D=d|\theta).$
(18)

###  4.2 Fisher Information with Sample Size Re-estimation

Given a sample size re-estimation formula, there are still many ways to
construct the procedure to be used in an application. For example, if the
calculated estimate is below the current sample size $n_{1}$, the procedure
may call for the study to be stopped; if the estimate exceeds a given limit it
will be truncated. To accommodate such practical considerations, we define the
decision rule as a function of the available data $\mathbf{x}_{1}$ rather than
the summary $Z$-statistic.

Let $\left\\{{\cal{C}}_{1},\ldots,{\cal{C}}_{K}\right\\}$ be a partition of
all possible realizations of stage 1 data. Then the SSR can be formalized by
the following decision function

${\cal{D}}=\begin{cases}1=\text{`stop'}&\text{if
}\mathbf{x}_{1}\in{\cal{C}}_{1}\\\ 2=\text{`collect }n_{2}^{(2)}\text{ extra
observations'}&\text{if }\mathbf{x}_{1}\in{\cal{C}}_{2}\\\ \vdots\
=\,\vdots&\vdots\\\ K=\text{`collect }n_{2}^{(K)}\text{ extra
observations'}&\text{if }\mathbf{x}_{1}\in{\cal{C}}_{K}\end{cases}$ (19)

The decision space specified by (19) accommodates all possible sample size
recalculation procedures (e.g., procedures based on the observed variability
of the treatment effect, the observed treatment effect itself, etc.). The
resulting support $\left({\cal{T}}\right)$ and the probability measure of the
observable random variables are defined as generalizations of those detailed
in Section 3.1.

Note that different “labels” may be assigned to the $\\{1,\ldots,K\\}$
decisions in Equation (19) to describe other types of interim decisions (e.g.,
joint futility and efficacy stopping, enrichment procedures).

The sequential decomposition of FI with an interim test shown in Equation
(4.1) is extended to designs that have multiple decision options at the time
of the interim analysis in Theorem 1:

######  Theorem 1.

If a decision $d\in\\{1,\ldots,K\\}$ can be made at an interim analysis, the
following decomposition of Fisher Information applies:

$\displaystyle{\cal{I}}_{\mathbf{X}_{\cal{T}}}(\theta)=$
$\displaystyle{\cal{I}}_{D}(\theta)+\sum_{d=1}^{K}\text{Pr}\left(D=d|\theta\right)\left[{\cal{I}}_{\mathbf{X}_{1}|D=d}(\theta)+{\cal{I}}_{\mathbf{X}_{2}|D=d}(\theta)\right].$
(20)

When the interim decision $D=d$ is made, the amount of FI available for
further inferential (e.g., further hypothesis testing and estimation) consists
of the previous information conditional on $D=d$
[${\cal{I}}_{\mathbf{X}_{1}|D=d}(\theta)$] and the new information to be
collected before next analysis [${\cal{I}}_{\mathbf{X}_{2}|D=d}(\theta)$].

If there is more than one interim analysis, Theorem 1 can be applied
sequentially with $\mathbf{X}_{1}$ referring to the data collected before an
interim analysis and $\mathbf{X}_{2}$ referring to the data that might be
collected after the interim analysis. Thus, Theorem 1 describes how Fisher
Information is allocated across the whole decision tree branching overtime
with interim analyses.

Section 6 presents a simple normal example to illustrate the theoretical
findings.

##  5 Lower Bound for Mean Squared Error in Sequential Experiments

Section 2.1 reviews how Fisher information appears in the CRLB inequality that
determines the minimum variance, and more generally the minimum MSE among all
regular estimators. Moreover, this bound is reached in one-parameter
exponential family models where the MLE absorbs all statistical evidence about
the canonical parameter $\theta$. Section 4 shows that the situation changes
with informative interim adaptations: not all Fisher information is available
for post-adaptation inference. Thus, CRLB inequality needs to be extended to
account for interim adaptations.

######  Theorem 2.

(Lower Bounds for Mean Squared Error in Sequential Experiments) If, for any
interim decision $d\in\\{1,\ldots,K\\}$,
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D=d}(\theta)$ exists for any $\theta$ and
$\frac{\partial}{\partial\theta}E(\tilde{\theta}\mid
D=d)=E(\frac{\partial}{\partial\theta}\tilde{\theta}\mid D=d)$, then

  1. 1.

conditionally on $D=d$, the lower boundary for the MSE of $\tilde{\theta}$ is

$\textrm{E}\left([\tilde{\theta}_{(d)}-\theta]^{2}\right)\geq\frac{\left[1+\frac{\partial}{\partial\theta}b(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{\cal{T}}|D=d}(\theta)}+b_{(d)}^{2}(\theta)$
(21)

  2. 2.

and, unconditionally, the lower bound is

$\textrm{E}\left([\tilde{\theta}-\theta]^{2}\right)\geq\sum_{d=1}^{K}P_{d}(\theta)\left[\frac{\left[1+\frac{\partial}{\partial\theta}b_{(d)}(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{\cal{T}}|D=d}(\theta)}+b_{(d)}^{2}(\theta)\right],$
(22)

where $E(\tilde{\theta}_{(d)})=\theta+b_{(d)}(\theta)$.

Proof of Theorem 2: Existence of Fisher information and interchangeability of
integration and differentiation ensures existence of the Cramer-Rao lower
boundary at each $D=d$, which proves inequality (21). From
$\textrm{E}([\hat{\theta}-\theta]^{2})=\sum_{d=1}^{K}P_{d}(\theta)\textrm{E}([\hat{\theta}_{(d)}-\theta]^{2})$,
inequality (22) immediately follows. Q.E.D.

######  Corollary 3.

The MLE for a canonical parameter in the one-parameter exponential family
attain the CRLB. The MLE $\hat{\theta}$ in the one-parameter exponential
family reaches the CRLB with minimum MSE:

$\textrm{E}\left([\hat{\theta}-\theta]^{2}\right)=\sum_{d=1}^{K}P_{d}(\theta)\left(\frac{\left[1+\frac{\partial}{\partial\theta}b(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{\cal{T}}|D=d}(\theta)}+b_{(d)}^{2}(\theta)\right).$
(23)

Proof of Corollary 3: Tarima and Flournoy (2022) showed that the possibility
of early stopping after an MLE-based interim hypothesis test of the form
$\hat{\theta}_{1}>c_{1}$ changes the distribution of the test statistic, but
conditionally on $D=d$, $\hat{\theta}_{(d)}$ continues being sufficient and to
belong to the exponential family. Same argument applies for interim decision
specific $\hat{\theta}_{(d)}$, where interim decision are defined by (19).
Thus, under Theorem 2 assumptions, the Cramer-Rao lower boundary is reached
for any $D=d$:

$\textrm{E}\left([\hat{\theta}_{(d)}-\theta]^{2}\right)=\frac{\left[1+\frac{\partial}{\partial\theta}b(\theta)\right]^{2}}{{\cal{I}}_{\mathbf{X}_{\cal{T}}|D=d}(\theta)}+b_{(d)}^{2}(\theta)$
(24)

and
${\cal{I}}_{\hat{\theta}_{(d)}}(\theta)={\cal{I}}_{\mathbf{X}_{\cal{T}}|D=d}(\theta)$.
Further, from

$\textrm{E}\left([\hat{\theta}-\theta]^{2}\right)=\sum_{d=1}^{K}P_{d}(\theta)\textrm{E}\left([\hat{\theta}_{(d)}-\theta]^{2}\right)$

the result follows. Q.E.D.

Section 6 presents a simple normal example to illustrate the theoretical
findings.

##  6 Normal Example, $n_{1}=n_{2}=1$, $X_{i}\sim N(\theta,1)$

If $n_{1}=n_{2}=1$, $\mathbf{X}_{1}=X_{1}$, $\mathbf{X}_{2}=X_{2}$, and the
MLE $\hat{\theta}=X_{1}$ when $D=1$ and $\hat{\theta}=(X_{1}+X_{2})/2$ when
$D=2$. The density of $\hat{\theta}$ if stopped early $(D=1)$ is

$f_{\hat{\theta}|D=1}(t_{1})=\frac{\phi(t_{1}-\theta)}{\Phi(c_{1}-\theta)};$

if the experiment proceeds to stage 2 ($D=2$), the density is the convolution

$\displaystyle f_{\hat{\theta}|D=2}\left(t_{(2)}\right)$
$\displaystyle=\frac{1}{1-\Phi(c_{1}-\theta)}\int_{-\infty}^{c_{1}}\left(\frac{1}{\sqrt{2\pi
0.5}}\right)^{2}$
$\displaystyle\times\exp\left[-\frac{1}{2}\left(\frac{t_{(2)}-t_{1}/2-\theta/2}{1/2}\right)^{2}\right]\exp\left[-\frac{1}{2}\left(\frac{t_{1}/2-\theta/2}{1/2}\right)^{2}\right]dt_{1}$
$\displaystyle=\frac{1}{1-\Phi(c_{1}-\theta)}\int_{-\infty}^{c_{1}}\frac{1}{\pi}\phi\left(2t_{(2)}-t_{1}-\theta\right)\phi\left(t_{1}-\theta\right)dt_{1}.$
(25)

Minor algebra shows that Fisher information in a normal random variable
$X_{1}\sim N(\theta,1)$ truncated to the interval $[a,b]$ is equal to its
variance

$1-A^{-1}\left[b_{st}\phi(b_{st})-a_{at}\phi(a_{st})\right]-A^{-2}\left[\phi(b_{st})-\phi(a_{st})\right]^{2},$
(26)

where $b_{st}=b-\theta$, $a_{st}=a-\theta$ and $A=\Phi(b_{st})-\Phi(a_{st})$.

Information | Value | Comment
---|---|---
Informative Stopping, $c_{1}=1.96,\;\text{Pr}(D=1\mid\theta)=0.5,\
\theta=1.96$
${\cal{I}}_{X_{1}|D=1}$ | 0.3634 |
${\cal{I}}_{X_{1}|D=2}$ | 0.3634 |
${\cal{I}}_{X_{1}|D}$ | 0.3634 | $=0.5\cdot 0.3634-0.5\cdot 0.3634$
${\cal{I}}_{D}$ | 0.6366 | $=1-0.3634$
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D=2}$ | 1.3634 | =1+0.3634
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D}$ | 0.8634 | $=0.5\cdot 0.3634+0.5(0.3634+1)$
${\cal{I}}_{\mathbf{X}_{\cal{T}}}$ | 1.5 | $=0.6366+0.5\cdot 0.3634+0.5(0.3634+1)$
Informative Stopping, $c_{1}=1.96,\;\text{Pr}(D=1\mid\theta)=0.025,\;\theta=0$
${\cal{I}}_{X_{1}|D=1}$ | 0.8789 |
${\cal{I}}_{X_{1}|D=2}$ | 0.3634 |
${\cal{I}}_{X_{1}|D}$ | 0.8598 | $=0.025\cdot 0.1167+0.975\cdot 0.8789$
${\cal{I}}_{D}$ | 0.1402 | $=1-0.8598$
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D=2}$ | 1.8789 | =1+0.8789
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D}$ | 1.8349 | $=0.025\cdot 0.1167+0.975(0.8789+1)$
${\cal{I}}_{\mathbf{X}_{\cal{T}}}$ | 1.975 | $=0.1402+0.025\cdot 0.1167+0.975(0.8789+1)$
Non-Informative Stopping with $\text{Pr}(D=1|\theta)=0.5$, $\forall\theta$
${\cal{I}}_{X_{1}|D=1}$ | 1 |
${\cal{I}}_{X_{1}|D=2}$ | 1 |
${\cal{I}}_{X_{1}|D}$ | 1 |
${\cal{I}}_{D}$ | 0 |
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D=2}$ | 2 | $=1+1$
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D}$ | 1.5 | $=1+0.5(1+1)-0$
${\cal{I}}_{\mathbf{X}_{\cal{T}}}$ | 1.5 | $=1+0.5(1+1)$
Non-Informative Stopping with $\text{Pr}(D=1|\theta)=0.025$, $\forall\theta$
${\cal{I}}_{X_{1}|D=1}$ | 1 |
${\cal{I}}_{X_{1}|D=2}$ | 1 |
${\cal{I}}_{X_{1}|D}$ | 1 |
${\cal{I}}_{D}$ | 0 |
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D=2}$ | 2 | $=1+1$
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D}$ | 1.975 | $=1+0.975(1+1)-0$
${\cal{I}}_{\mathbf{X}_{\cal{T}}}$ | 1.975 | $=1+0.975(1+1)$
Table 2: Numerically calculated values of various information measures given
$n_{1}=n_{2}=1$, $X_{i}\sim N(\theta,1)$

Formula (26) permits calculation of ${\cal{I}}_{\mathbf{X}_{1}|D=1}(\theta)$
and ${\cal{I}}_{\mathbf{X}_{1}|D=2}(\theta)$ in the Fisher information
partitioning equation (4.1). Other components (4.1) are easily calculable
using untruncated normal formulas:
${\cal{I}}_{\mathbf{X}_{1}}(\theta)={\cal{I}}_{\mathbf{X}_{2}}(\theta)=1$; and
the design information simplifies as

$\displaystyle{\cal{I}}_{D}$
$\displaystyle=E_{X_{1}}\left(-\frac{\partial^{2}}{\partial\theta^{2}}\log\text{Pr}(X_{1}>c_{1})\right)=E_{X_{1}}\left(-\frac{\partial^{2}}{\partial\theta^{2}}\log\Phi(c_{1}-\theta)\right)$
$\displaystyle=1-\text{Pr}(D=1){\cal{I}}_{\mathbf{X}_{1}|D=1}(\theta)-\text{Pr}(D=2){\cal{I}}_{\mathbf{X}_{1}|D=2}(\theta).$

Various measures of information are plotted in Figure 1 as a function of
$\theta$. The two subfigures shown are constructed under experiments E2 (an
interim test that stops the experiment if $z_{1}\geq c_{1}=1.96$) and E3 (an
interim test that stops the experiment if $z_{1}\geq c_{1}=2.78$). The total
FI in an experiment with two independent normal random variables without an
interim stopping option is 2; this is the maximum information available with
stopping options. The total Fisher information (FI) with the interim stopping
options (the blue curves) is $1+1\cdot\text{Pr}\left(z_{1}\geq
c_{1}\mid\theta\right)$; it decreases from the maximum 2.0 for small values of
$\theta$ to 1.0 for large $\theta$, reflecting the changing probability of
observing one or two observations. If $\theta$ equals the critical value, then
the chance of continuing is equal to the chance of stopping and the total FI
is 1.5.

The total FI is seen to decompose into the information conditional on the
design (solid black) and the information in the design (solid red). The design
possibility of an interim test consumes the greatest amount of information
when $\theta$ equals the critical value [$\theta=1.96$ in panel (a) and
$\theta=2.78$ in panel (b)] , and tends to zero as $\theta\to\pm\infty$.
Panels (a) and (b) illustrate that different stopping rules cause the same
likelihood to be associated with different information measures.

The information curves conditional on stopping at $D=1$ (dashed black line)
increases from zero to 1.0, while the information conditional on stopping at
$D=2$ (dotted black line) decreases from 2.0 to 1.0.

For more concrete comparison, Table 2 reports the actual numerical
calculations of ${\cal{I}}_{X_{1}|D=1}$, ${\cal{I}}_{X_{1}|D=2}$,
${\cal{I}}_{X_{1}|D}$, ${\cal{I}}_{D}$,
${\cal{I}}_{\mathbf{X}_{\cal{T}}|D=2}$, ${\cal{I}}_{\mathbf{X}_{\cal{T}}|D}$,
and ${\cal{I}}_{\mathbf{X}_{\cal{T}}}$ for two interim decision rules
(informative stopping with $c_{1}=1.96$ and non-informative stopping) under
$\theta=0$ and $\theta=1.96$.

Table 3 provides numerical calculations of the lowest possible mean squared
error for the design with early stopping defined by $c_{1}=1.96$ under
$\theta=0$ and 1.96. Per Corollary 3, this lower bound for MSE is reached by
the MLE. The first and third number in each set were verified by simulation.
Note that if one makes the “right” interim decision, bias and mean squared
error are reduced. That is if the experiment stops earlier and the true
$\theta$ is higher, the bias and mean squared error for post-testing
estimation are reduced; if the experiment proceeds to the second stage and
$\theta$ is lower, the bias and mean squared error are reduced as well.

Information | Value | Comment
---|---|---
Informative Stopping,
$c_{1}=1.96,\;\theta=1.96,\;\phi(0)=0.3989,\;\Phi(0)=0.5$
$E\left(\hat{\theta}-\theta|D=1\right)$ | 0.7979 | $=0.3989\cdot 2$
$\frac{\partial}{\partial\theta}E\left(\hat{\theta}-\theta|D=1\right)$ | -0.6366 | $=-4\cdot 0.3989^{2}$
$E\left[\left(\hat{\theta}-\theta\right)^{2}|D=1\right]$ | 1 | $=(1-0.6366)^{2}/0.3634+0.7979^{2}$
$E\left(\hat{\theta}-\theta|D=2\right)$ | -0.3989 |
$\frac{\partial}{\partial\theta}E\left(\hat{\theta}-\theta|D=2\right)$ | -0.3183 | $=-4\cdot 0.3989^{2}/2$
$E\left[\left(\hat{\theta}-\theta\right)^{2}|D=2\right]$ | 0.5 | $=(1-0.3183)^{2}/1.3634+(-0.3989)^{2}$
Informative Stopping,
$c_{1}=1.96,\;\theta=0,\;\phi(1.96)=0.0584,\;\Phi(0)=0.975$
$E\left(\hat{\theta}-\theta|D=1\right)$ | 2.3378 | $=0.0584/(1-0.9750)$
$\frac{\partial}{\partial\theta}E\left(\hat{\theta}-\theta|D=1\right)$ | -0.8833 | $=(1.96\cdot 0.0584\cdot 0.0250-0.0584^{2})/(0.0250^{2})$
$E\left[\left(\hat{\theta}-\theta\right)^{2}|D=1\right]$ | 5.5821 | $=(1-0.8833)^{2}/0.1167+2.3378^{2}$
$E\left(\hat{\theta}-\theta|D=2\right)$ | -0.0300 | $=-0.0584/(2\cdot 0.9750)$
$\frac{\partial}{\partial\theta}E\left(\hat{\theta}-\theta|D=2\right)$ | -0.0605 | $=-0.5\cdot(1.96\cdot 0.0584\cdot 0.9750+0.0584^{2})/(0.9750^{2})$
$E\left[\left(\hat{\theta}-\theta\right)^{2}|D=2\right]$ | 0.4706 | $=(1-0.0605)^{2}/1.8789+(-0.0300)^{2}$
Table 3: Bias conditional on interim decision $D$ for $\theta=1.96$ and
$\theta=0$, its derivative, and the mean squared error based on analytic
formulas at $n_{1}=n_{2}=1$, $X_{i}\sim N(\theta,1)$

##  7 Summary

The cost of adaptation is quantified. We show how the cost of an interim test
for early stopping varies with the parameter of interest and how it changes
with the specific stopping rule even though the likelihood is insensitive to
the study design and the data collection procedure.

Theorem 1 decomposes Fisher Information into a component resulting from the
inclusion of informative decisions in the design, and separate components for
before and after interim decision points. Formula (4.1) details this
decomposition for an experiment having a single interim test. This
decomposition of the total Fisher information allows us to make the following
useful observations:

  * •

With non-informative adaptations, ${\cal{I}}_{D}(\theta)=0$; no information is
lost for likelihood-based inference. For example, because the variance
estimate of a normal random variable is independent of the estimated effect
size, there is no loss of information when sample size re-estimation is based
only on an estimate of the standard deviation at the time of the interim
analysis.

  * •

It is possible that all the information in stage 1 is used for sample size re-
estimation, in which case none is left for final estimation.

  * •

The higher ${\cal{I}}_{D}(\theta)$ is the less Fisher information is left for
estimation after stopping.

  * •

The Fisher Information available in design, ${\cal{I}}_{D}(\theta)$, is a
measure of the cost of informative stopping. This information is not absorbed
by the likelihood function and is lost for likelihood-based inference. More
generally, any estimators based solely on the observed data do not use the
design information.

  * •

With one interim test for early stopping, the greatest information loss occurs
when the critical value $c_{1}$ is equal to the parameter of interest
$\theta$. At this value of $\theta$ post-testing estimation is affected the
most.

  * •

Fisher information available for estimation of $\theta$ or further testing
after stopping at stages 1 or proceeding to stage 2 is
${\cal{I}}_{\mathbf{X}_{1}|D=1}(\theta)$ and
${\cal{I}}_{\mathbf{X}_{1}|D=2}(\theta)+{\cal{I}}_{\mathbf{X}_{2}}(\theta)$,
respectively.

Since some Fisher Information is spent for interim adaptation, the amount
available for post-adaptation inference is smaller than the total. This is one
reason why Cramer-Rao lower bound needs to be adjusted for informative interim
adaptations. Another reason is the change in support of observable random
variables. A sequential version of the Cramer-Rao lower bound suggested in
Wolfowitz (1947) does not apply to designs with informative interim
adaptations: see Simons (1980) for examples illustrating when Wolfowitz’s
bound does not work.

Theorem 2 gives a new bound for the smallest mean squared error, and as shown
in Corollary 3 this lower bound is reached by the MLE in one-parameter
exponential family with canonical parameterization.

We anticipate that our results will be useful for deriving and justifying
optimal sequential designs with a dual goal of sequential testing and post-
test estimation.

## Conflict of interest

The authors declare that they have no conflict of interest.

## Data Availability

Data sharing is not applicable to this article as no new data were created or
analyzed in this study.

## Funding

This research received no specific grant from any funding agency in the
public, commercial, or not-for-profit sectors.

## References

  * Armitage et al. (1969) Armitage, P., C. McPherson, and B. Rowe (1969).  Repeated significance tests on accumulating data.  Journal of the Royal Statistical Society: Series A 132, 235–244.
  * Berger and Wolpert (1988) Berger, J. and R. Wolpert (1988).  The Likelihood Principle (second edition).  Institute of Mathematical Statistics. Lecture notes : monographs series. Institute of Mathematical Statistics.
  * Casella and Berger (2002) Casella, G. and R. Berger (2002).  Statistical Inference.  Duxberry Advanced Series, edition=2.
  * Chuang-Stein et al. (2006) Chuang-Stein, C., K. Anderson, P. Gallo, and S. Collins (2006).  Sample size reestimation: a review and recommendations.  Drug Information Journal: DIJ/Drug Information Association 40(4), 475–484.
  * Cramér (1946) Cramér, H. (1946).  A contribution to the theory of statistical estimation.  Scandinavian Actuarial Journal 1946(1), 85–94.
  * Efron et al. (1975) Efron, B. et al. (1975).  Defining the curvature of a statistical problem (with applications to second order efficiency).  The Annals of Statistics 3(6), 1189–1242.
  * Ferguson (1996) Ferguson, T. (1996).  A Course in Large Sample Theory.  New York: Routledge.
  * Friede and Kieser (2006) Friede, T. and M. Kieser (2006).  Sample size recalculation in internal pilot study designs: a review.  Biometrical Journal: Journal of Mathematical Methods in Biosciences 48(4), 537–555.
  * Jennison and Turnbull (1999) Jennison, C. and B. Turnbull (1999).  Group Sequential Methods with Applications to Clinical Trials.  Chapman & Hall/CRC Interdisciplinary Statistics. CRC Press.
  * Liu and Hall (1999) Liu, A. and W. Hall (1999).  Unbiased estimation following a group sequential test.  Biometrika 86(1), 71–78.
  * Liu et al. (2006) Liu, A., W. Hall, K. F. Yu, and C. Wu (2006).  Estimation following a group sequential test for distributions in the one-parameter exponential family.  Statistica Sinica 16(1), 165–181.
  * Marschner (2021) Marschner, I. C. (2021).  A general framework for the analysis of adaptive experiments.  Statistical Science 36(3), 465–492.
  * Mihoc and Ioana Fatu (2003) Mihoc, I. and C. Ioana Fatu (2003).  Fisher’s information measures and truncated normal distributions (ii).  J. Numer. Anal. Approx. Theory 32(2), 177–186.
  * O’Brien and Fleming (1979) O’Brien, P. C. and T. R. Fleming (1979).  A multiple testing procedure for clinical trials.  Biometrics, 549–556.
  * Proschan et al. (2006) Proschan, M. A., K. K. G. Lan, and J. T. Wittes (2006).  Statistical Monitoring of Clinical Trials: A Unified Approach.  Springer.
  * Rao (1945) Rao, C. R. (1945).  Information and the accuracy attainable in the estimation of statistical parameters.  Reson. J. Sci. Educ 20, 78–90.
  * Siegmund (1985) Siegmund, D. (1985).  Sequential Analysis: Tests and Confidence Intervals.  Springer Science & Business Media.
  * Simons (1980) Simons, G. (1980).  Sequential estimators and the cramér-rao lower bound.  Journal of Statistical Planning and Inference 4(1), 67–74.
  * Tarima and Flournoy (2022) Tarima, S. and N. Flournoy (2022).  Most powerful test sequences with early stopping options.  Metrika 85(4), 491–513.
  * Whitehead (1986) Whitehead, J. (1986).  On the bias of maximum likelihood estimation following a sequential test.  Biometrika 73(3), 573–581.
  * Wolfowitz (1947) Wolfowitz, J. (1947).  The efficiency of sequential estimates and wald’s equation for sequential processes.  The Annals of Mathematical Statistics 18(2), 215–230.
  * Zegers (2015) Zegers, P. (2015).  Fisher information properties.  Entropy 17(7), 4918–4939.

(a) $c_{1}=1.96$

(b) $c_{1}=2.78$

Figure 1: Fisher Information (FI) with $n_{1}=n_{2}=1$ versus $\theta$; total
FI in the sequential experiment is shown by a solid blue line, FI conditional
on design - solid black, FI in design - solid red, FI conditional on $D=1$ \-
dashed black and FI conditional on $D=2$ \- dotted black; is shown by a thin
dashed vertical line