# A likelihood based sensitivity analysis for publication bias on summary ROC
in meta-analysis of diagnostic test accuracy

Yi<EMAIL_ADDRESS>Ao Huang
Department of Biomedical Statistics, Graduate School of Medicine, Osaka
University
Satoshi Hattori†††To whom correspondence should be addressed.
<EMAIL_ADDRESS>
Department of Biomedical Statistics, Graduate School of Medicine, Osaka
University
and
Integrated Frontier Research for Medical Science Division,
Institute for Open and Transdisciplinary Research Initiatives (OTRI),
Osaka University, Osaka, Japan

###### Abstract

In meta-analysis of diagnostic test accuracy, summary receiver operating
characteristic (SROC) is a recommended method to summarize the discriminant
capacity of a diagnostic test in the presence of study-specific cutoff values
and the area under the SROC (SAUC) gives the aggregate measure of test
accuracy. SROC or SAUC can be estimated by bivariate modeling of pairs of
sensitivity and specificity over the primary diagnostic studies. However,
publication bias is a major threat to the validity of estimates in meta-
analysis. To address this issue, we propose to adopt sensitivity analysis to
make an objective inference for the impact of publication bias on SROC or
SAUC. We extend Copas likelihood based sensitivity analysis to the bivariate
normal model to evaluate how much SROC or SAUC would change under several
selective publication mechanisms dependent on sensitivity and/or specificity.
The selection probability is modeled by a selection function on $t$-type
statistic for the linear combination of logit-transformed sensitivity and
specificity, allowing the selective publication of each study to be influenced
by the cutoff-dependent $p$-value for sensitivity, specificity, or diagnostic
odds ratio. By embedding the selection function into the bivariate normal
model, the conditional likelihood is proposed and the bias-corrected SROC or
SAUC can be estimated by maximizing the likelihood. We illustrate the proposed
sensitivity analysis by reanalyzing a meta-analysis of test accuracy for
intravascular device related infection. Simulation studies are conducted to
investigate the performance of proposed methods. Keywords SROC; Publication
bias; Sensitivity analysis; Meta-analysis; Diagnostic test accuracy

##  1 Introduction

Meta-analysis of diagnostic test accuracy is an important statistical tool for
summarizing test accuracy from multiple primary diagnostic studies and is
useful to provide evidence on how well the test discriminates people with the
target condition or disease from people without. The test accuracy from
primary studies are usually reported in pairs of sensitivity and specificity,
dependent on the study-specific cutoff values defining test positives and test
negatives. These study-specific cutoff values induce heterogeneity in test
accuracy across multiple studies; thus, it is inappropriate to pool
sensitivity or specificity separately regardless of the cutoff values in the
meta-analysis (Macaskill et al., 2010; Hattori and Zhou, 2018). Alternatively,
summary receiver operating characteristic curve (SROC) is recommended to
synthesize the reported pairs of sensitivity and specificity (Moses et al.,
1993; Rutter and Gatsonis, 2001; Macaskill, 2004; Reitsma et al., 2005; Chu et
al., 2010; Hattori and Zhou, 2018). SROC presents the relationship between
sensitivity (true positive rate, TPR) and one minus specificity (false
positive rate, FPR) across studies, depicting test accuracy at all possible
cutoff values by a monotonic curve. A natural candidate summary is the area
under SROC (SAUC), giving the average of TPR over all possible values of FPR
(Walter, 2002). Two bivariate models are often applied to estimate SROC or
SAUC. One is bivariate binomial model (hierarchical SROC or HSROC model) for
pairs of TPR and FPR, based on the ordinal logistic regression with Bayesian
inference (Rutter and Gatsonis, 2001) or maximum likelihood estimation
(Macaskill, 2004). The other is bivariate normal model for pairs of logit-
transformed sensitivity and specificity, based on asymptotic bivariate
normality of empirical sensitivity and specificity (Reitsma et al., 2005).

The greatest threat to the validity of meta-analysis is publication bias,
induced by the fact that studies reporting a statistically significant result
are more likely to be published (Sutton et al., 2000). Models dealing with
publication bias have been extensively studied in meta-analysis of
intervention studies and can be generally grouped into graphical methods or
selection model based methods (Li et al., 2021). Graphical methods, such as
the funnel plot and the trim-and-fill method, are widely used to detect and
adjust publication bias (Macaskill et al., 2001). Various kinds of sensitivity
analysis methods are also available based on Heckman-type (Heckman, 1976,
1979) selection functions (Copas, 1999; Copas and Shi, 2000, 2001), an
alternative parametric selection function (Copas, 2013), or a worst-case
nonparametric bound (Copas and Jackson, 2004).

In contrast, methods for publication bias in meta-analysis of diagnostic test
accuracy have been understudied. Some graphical methods have been proposed
based on the commonly used univariate measure, diagnostic odds ratio (DOR).
Deeks et al. (2005) conducted the simulation studies to compare the the Begg,
Egger, and Macaskill tests of funnel plot asymmetry on the log-transformed DOR
(lnDOR) for diagnostic studies. Their results showed that those funnel plot
investigations based on standard error of lnDOR might be seriously misleading.
Bürkner and Doebler (2014) compared the performance of the tests combined with
four different univariate measures of diagnostic accuracy. Their simulation
studies showed that the combination of lnDOR and trim-and-fill was the best to
detect publication bias, but it lacked power when the number of studies in
meta-analysis was small. These graphical methods may be useful to detect
publication bias, but cannot address the relationship between TPR and FPR like
SROC. To evaluate the impact of publication bias on SROC, Hattori and Zhou
(2018) extended the sensitivity analysis method of Copas and Shi (2000, 2001)
to the meta-analysis of diagnostic test accuracy based on the bivariate
binomial model. Piao et al. (2019) proposed the conditional likelihood to
correct publication bias in sensitivity and specificity based on the bivariate
normal model. Li et al. (2021) proposed an empirical (full) likelihood based
approach under a similar setting to Piao et al. (2019) and increased the
correction accuracy. All the methods considered the Heckman-type selection
function on the sample size of a study. As explained in Hattori and Zhou
(2018), this Heckman-type selection function gives a certain framework to
model selective publication mechanisms in the diagnostic studies; a drawback
is that it is completely free from the cutoff values. Given different cutoff
values, the measures representing the test accuracy will also change. If some
statistical hypothesis testing was used in the study, the statistical
significance, $p$-value, conditional on the cutoff value would be influential
in the selective publication process; accordingly, selection functions
allowing study-specific cutoff values would be appealing.

For meta-analysis of intervention studies, Copas (2013) designed a likelihood
based sensitivity analysis, in which the selection probability of each study
was modeled as a function of $t$-type statistic or $p$-value for the primary
outcome. Extending Copas (2013) into meta-analysis of diagnostic test
accuracy, we embed a cutoff-dependent selection function to the bivariate
normal model of Reitsma et al. (2005) (hereinafter referred to as Reistma
model) and propose a sensitivity analysis model to evaluate the impact of
publication bias on SROC and SAUC. Our selection function allows the selection
probability of each study to be determined by $t$-type statistic for a linear
combination of logit-transformed sensitivity and specificity. It can handle
situations where the selective publication process is influenced by $t$-type
statistic for sensitivity, specificity, or diagnostic odds ratio (DOR), and
would be an appealing alternative to the existing Heckman-type selection
functions.

The rest of this paper is organized as follows. In Section 2.1, we review
Reistma model with corresponding SROC and SAUC. In Section 2.2, we propose
$t$-type statistic based selection function with sensitivity analysis on SROC
and SAUC in Section 2.3. In Section 3, we illustrate the proposed sensitivity
analysis methods with a meta-analysis. In Section 4, we conduct simulation
studies to assess the finite sample performance of the proposed method by
comparing it with Reistma model without accounting for publication bias, and
use SROC to graphically show the trend of publication bias. Finally, we
provide a discussion in Section 5.

##  2 Methods

###  2.1 Bivariate normal model

Suppose that $N$ studies are selected into a meta-analysis for review from a
population of diagnostic studies on the test of interest. Each study
$i~{}(i=1,2,\dots,N$) reports the observed number of true positives, false
negatives, true negatives, and false positives, denoted by
$n_{11}^{i},n_{01}^{i},n_{00}^{i}$, and $n_{10}^{i}$, respectively, as
formulated in Table 1. Let $n_{+1}^{i}=n_{11}^{i}+n_{01}^{i}$ be the number of
people with the target condition and $n_{+0}^{i}=n_{00}^{i}+n_{10}^{i}$ the
number of people without, the observed sensitivity and specificity are
estimated by $\mathrm{\hat{se}}_{i}={n_{11}^{i}}/{n_{+1}^{i}}$ and
$\mathrm{\hat{sp}}_{i}={n_{00}^{i}}/{n_{+0}^{i}}$, respectively. Following
Reitsma et al. (2005), we define $\mu_{1i}$ and $\mu_{2i}$ be the logit-
transformed true sensitivity and specificity of the $i$th study. Reitsma model
assumes that $(\mu_{1i},\mu_{2i})^{T}$ is normally distributed across $S$
studies in the population:

$\binom{\mu_{1i}}{\mu_{2i}}\sim
N\left(\binom{\mu_{1}}{\mu_{2}},\boldsymbol{\Omega}\right)\mathrm{~{}with~{}}\boldsymbol{\Omega}=\begin{pmatrix}\tau_{1}^{2}&\tau_{12}\\\
\tau_{12}&\tau_{2}^{2}\end{pmatrix},$ (1)

where $\mu_{1}$ and $\mu_{2}$ are the common means of logit-transformed
sensitivity and specificity, $\tau_{1}^{2}~{}(\tau_{1}>0)$ and
$\tau_{2}^{2}~{}(\tau_{2}>0)$ are their between-study variances,
$\tau_{12}=\rho\tau_{1}\tau_{2}$ is the covariance between $\mu_{1i}$ and
$\mu_{2i}$, and $\rho~{}(-1\leq\rho\leq 1)$ is the correlation coefficient.
Let $y_{1i}$ and $y_{2i}$ be the logit-transformed observed sensitivity
($\mathrm{\hat{se}}_{i}$) and specificity ($\mathrm{\hat{sp}}_{i}$). Given
$(\mu_{1i},\mu_{2i})$, it is assumed that,

$\binom{y_{1i}}{y_{2i}}\sim
N\left(\binom{\mu_{1i}}{\mu_{2i}},\boldsymbol{\Sigma}_{i}\right)\mathrm{~{}with~{}}\boldsymbol{\Sigma}_{i}=\begin{pmatrix}s_{1i}^{2}&0\\\
0&s_{2i}^{2}\end{pmatrix},$ (2)

where $s_{1i}^{2}$ and $s_{2i}^{2}$ are the observed variances of $y_{1i}$ and
$y_{2i}$ within each study. When $n_{+1}^{i}$ and $n_{+0}^{i}$ are large and
$0<\mathrm{\hat{se}}_{i};~{}\mathrm{\hat{sp}}_{i}<1$, the variances can be
estimated as $s_{1i}^{2}=1/n_{11}^{i}+1/n_{01}^{i}$ and
$s_{2i}^{2}=1/n_{00}^{i}+1/n_{10}^{i}$, respectively. For studies with
frequencies of zero in Table 1, the continuity correction is made by adding
0.5 to all the cells. Following the convention of literature-based meta-
analysis, we regard $\boldsymbol{\Sigma}_{i}$ as known. Then, combining the
distributions (1) and (2) leads to the marginal Reitsma model:
$\boldsymbol{y}_{i}|\boldsymbol{\Sigma}_{i}\sim
N_{2}\left(\boldsymbol{\mu},\boldsymbol{\Omega}+\boldsymbol{\Sigma}_{i}\right),$
where $\boldsymbol{y}_{i}=(y_{1i},y_{2i})^{T}$,
$\boldsymbol{\mu}=(\mu_{1},\mu_{2})^{T}$, and $N_{2}$ denotes the bivariate
normal distribution.

The conditional expectation of $\mu_{1i}$ given $\mu_{2i}$ can be derived as
$E(\mu_{1i}|\mu_{2i}=y)=\mu_{1}+\dfrac{\tau_{12}}{\tau_{2}^{2}}(y-\mu_{2}),$
which motivates the definition of SROC (Reitsma et al., 2005; Doebler et al.,
2012) as follows. Let $x$ be FPR ($1-\mathrm{specificity}$), SROC is defined
by

$SROC(x;\boldsymbol{\mu},\boldsymbol{\Omega})=\mathrm{logit}^{-1}\left[\mu_{1}-\dfrac{\tau_{12}}{\tau_{2}^{2}}\\{\mathrm{logit}(x)+\mu_{2}\\}\right].$

Accordingly, SAUC is defined by

$SAUC(\boldsymbol{\mu},\boldsymbol{\Omega})=\int_{0}^{1}SROC(x;\boldsymbol{\mu},\boldsymbol{\Omega})dx.$

###  2.2 Modeling selective publication process using $t$-type statistic

Differently from the Heckman-type selection functions (Hattori and Zhou, 2018;
Piao et al., 2019; Li et al., 2021), we propose a selection function relying
on a $t$-type statistic or its $p$-value for some measure of diagnostic test
accuracy to allow for a cutoff-dependent selective publication process. DOR is
a widely used to present the test accuracy in the diagnostic studies (Honest
and Khan, 2002; Glas et al., 2003). Its log transformation, lnDOR, defined by

$\displaystyle\mathrm{lnDOR}=\log\dfrac{\mathrm{se}\times\mathrm{sp}}{(1-\mathrm{se})\times(1-\mathrm{sp})}=\mathrm{logit(se)}+\mathrm{logit(sp)}=y_{1}+y_{2},$

is approximately normal distributed. Extending lnDOR, we consider a linear
combination of $\boldsymbol{c}^{T}\boldsymbol{y}_{i}$, by introducing a
contrast vector $\boldsymbol{c}=(c_{1},c_{2})^{T}$, to allow for different
weights on sensitivity and specificity in hypothesis testing. If sensitivity
(or specificity) is more of interest in a diagnostic test, we can consider
$(c_{1},c_{2})=(1,0)$ (or $(c_{1},c_{2})=(0,1)$). Since $t$-type statistic or
its $p$-value is scale-invariant, without loss of generality, we assume
$\boldsymbol{c}^{T}\boldsymbol{c}=1~{}(0\leq c_{1};c_{2}\leq 1)$; and then
lnDOR corresponds to $(c_{1},c_{2})=(1/\sqrt{2},1/\sqrt{2})$. Assuming that
studies with significant results are more likely to be selected from the
population, we consider the $t$-test statistic for testing the hypothesis
$H_{0}:\boldsymbol{c}^{T}\boldsymbol{\mu}=0$

$\displaystyle
t_{i}=\dfrac{\boldsymbol{c}^{T}\boldsymbol{y}_{i}}{\sqrt{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}}\sim
N\left(\dfrac{\boldsymbol{c}^{T}\boldsymbol{\mu}}{\sqrt{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}},1+\dfrac{\boldsymbol{c}^{T}\boldsymbol{\Omega}\boldsymbol{c}}{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}\right).$
(3)

To model the probability of selecting study $i$ from the population, we define
some function on $t_{i}$, denoted by $a(t_{i})$:

$P(\mathrm{select}|\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})=a(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})=a(t_{i}).$
(4)

Copas (2013) discussed three plausible parametric functions for $a(.)$, among
which the probit selection function was most tractable. We adopt the probit
function in this study and the selection probability in (4) can be written as

$P(\mathrm{select}|\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})=a(t_{i})=\Phi(\beta
t_{i}+\alpha),$ (5)

where $\beta$ and $\alpha$ are the parameters controlling the selection
probability. This monotonic parametric model bridges two extreme cases of
random selection. One case is where random selection with
$P(\mathrm{select})=p_{0}$ happens when $\beta=0$ and
$\alpha=\Phi^{-1}(p_{0})$; the other is where all the studies are selected
with $P(\mathrm{select})=1$ when $\beta$ approaches infinity.

The definition of the probit function allow us to represent (5) into

$P(\mathrm{select}|\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})=a(t_{i})=P(z_{i}<\beta
t_{i}+\alpha|\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})=\Phi\left({\beta\dfrac{\boldsymbol{c}^{T}\boldsymbol{y}_{i}}{\sqrt{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}}}+\alpha\right),$

where $z_{i}\sim N(0,1)$ independent of $t_{i}$. Utilizing the distribution of
$t_{i}$ in (3), selection function $a(t_{i})$ induces a dependence of
selection on $\boldsymbol{\Sigma}_{i}$, denoted by
$b(\boldsymbol{\Sigma}_{i})$:

$\displaystyle
P(\mathrm{select}|\boldsymbol{\Sigma}_{i})=b(\boldsymbol{\Sigma}_{i})=P\left(z_{i}-\beta
t_{i}<\alpha|\boldsymbol{\Sigma}_{i}\right)=\Phi\left\\{\dfrac{\beta\dfrac{\boldsymbol{c}^{T}\boldsymbol{\mu}}{\sqrt{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}}+\alpha}{\sqrt{1+\beta^{2}\left(1+\dfrac{\boldsymbol{c}^{T}\boldsymbol{\Omega}\boldsymbol{c}}{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}\right)}}\right\\}.$

###  2.3 Conditional likelihood

Following Copas (2013), we derive the distribution of $\boldsymbol{y}_{i}$ and
$\boldsymbol{\Sigma}_{i}$ across the selected studies, given the selection
functions $a(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})$,
$b(\boldsymbol{\Sigma}_{i})$, the bivariate normal distribution of
$\boldsymbol{y}_{i}|\boldsymbol{\Sigma}_{i}$, and a given value of marginal
selection probability $P(\mathrm{select})=p$. Hereinafter, we use $f_{O}$ to
denote the distribution across $N$ selected (observed) studies, and $f_{P}$
the distribution across $S$ population studies. The distribution of the
observed $\boldsymbol{\Sigma}_{i}$ can be obtained by

$f_{O}(\boldsymbol{\Sigma}_{i})=P(\boldsymbol{\Sigma}_{i}|\mathrm{select})=\dfrac{P(\mathrm{select}|\boldsymbol{\Sigma}_{i})P(\boldsymbol{\Sigma}_{i})}{P(\mathrm{select})}=\dfrac{b(\boldsymbol{\Sigma}_{i})f_{P}(\boldsymbol{\Sigma}_{i})}{p},$

which gives

$f_{P}(\boldsymbol{\Sigma}_{i})=p\dfrac{1}{b(\boldsymbol{\Sigma}_{i})}f_{O}(\boldsymbol{\Sigma}_{i}).$
(6)

When integrating both sides of equation (6) over $\boldsymbol{\Sigma}_{i}$,
with $f_{P}(\boldsymbol{\Sigma}_{i})$ integrate into 1, we can get
$p=[E_{O}\\{b(\boldsymbol{\Sigma}_{i})^{-1}\\}]^{-1}$ and it approximately
holds ${p}\approx N/\sum_{i=1}^{N}\\{b(\boldsymbol{\Sigma}_{i})^{-1}\\}$.
Finally, the joint distribution of the observed
$(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})$ can be written as:

$\displaystyle f_{O}(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})$
$\displaystyle=P(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i}|\mathrm{select})$
$\displaystyle=\dfrac{f_{P}(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})a(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})}{p}$
$\displaystyle=\dfrac{f_{P}(\boldsymbol{y}_{i}|\boldsymbol{\Sigma}_{i})f_{P}(\boldsymbol{\Sigma}_{i})a(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})f_{O}(\boldsymbol{\Sigma}_{i})}{b(\boldsymbol{\Sigma}_{i})f_{P}(\boldsymbol{\Sigma}_{i})}$
$\displaystyle=\dfrac{f_{P}(\boldsymbol{y}_{i}|\boldsymbol{\Sigma}_{i})a(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})f_{O}(\boldsymbol{\Sigma}_{i})}{b(\boldsymbol{\Sigma}_{i})},$

which gives the log-likelihood based on the selected studies:

$\displaystyle\ell_{O}(\boldsymbol{\mu},\boldsymbol{\Omega},\boldsymbol{c},\beta,\alpha)$
$\displaystyle=\log\prod_{i=1}^{N}f_{O}(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})$
$\displaystyle=\sum_{i=1}^{N}\log
f_{P}(\boldsymbol{y}_{i}|\boldsymbol{\Sigma}_{i})+\sum_{i=1}^{N}\log
a(\boldsymbol{y}_{i},\boldsymbol{\Sigma}_{i})-\sum_{i=1}^{N}\log
b(\boldsymbol{\Sigma}_{i})+\sum_{i=1}^{N}\log f_{O}(\boldsymbol{\Sigma}_{i}).$
(7)

The first term in (7) is the log-likelihood of Reitsma model. The second and
third terms imply the bias correction. If there is no bias in the observed
studies, $\ell_{O}$ will degenerate to the log-likelihood of Reitsma model.
The last term is a constant only dependent on the observed variances. The
first three terms make up the reduced likelihood for the unknown parameters if
we take each observed $\boldsymbol{\Sigma}_{i}$ as fixed. Recall that
${p}\approx N/\sum_{s=1}^{N}\\{b(\boldsymbol{\Sigma}_{i})^{-1}\\}$; then by
solving this equation, $\alpha$ can be written as a function of
$(\boldsymbol{\mu},\boldsymbol{\Omega},\boldsymbol{c},\beta)$ with a given
value of $p$, denoted by
$\alpha_{p}=\alpha_{p}(\boldsymbol{\mu},\boldsymbol{\Omega},\boldsymbol{c},\beta)$.
We represent the log-likelihood as a function of the parameters
$(\boldsymbol{\mu},\boldsymbol{\Omega},\boldsymbol{c},\beta)$, given a
specified value of selection probability $p$:

$\displaystyle\ell_{O}(\boldsymbol{\mu},\boldsymbol{\Omega},\boldsymbol{c},\beta)$
$\displaystyle=\ell_{O}(\boldsymbol{\mu},\boldsymbol{\Omega},\boldsymbol{c},\beta,\alpha_{p})$
$\displaystyle=\sum_{i=1}^{N}\left\\{-\frac{1}{2}(\boldsymbol{y}_{i}-\boldsymbol{\mu})^{T}(\boldsymbol{\Sigma}_{i}+\boldsymbol{\Omega})^{-1}(\boldsymbol{y}_{i}-\boldsymbol{\mu}))-\frac{1}{2}\log|\boldsymbol{\Sigma}_{i}+\boldsymbol{\Omega}|\right\\}$
$\displaystyle+\sum_{i=1}^{N}\log\Phi\left({\beta\dfrac{\boldsymbol{c}^{T}\boldsymbol{y}_{i}}{\sqrt{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}}}+\alpha_{p}\right)-\sum_{i=1}^{N}\log\Phi\left\\{\dfrac{\beta\dfrac{\boldsymbol{c}^{T}\boldsymbol{\mu}}{\sqrt{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}}+\alpha_{p}}{\sqrt{1+\beta^{2}\left(1+\dfrac{\boldsymbol{c}^{T}\boldsymbol{\Omega}\boldsymbol{c}}{\boldsymbol{c}^{T}\boldsymbol{\Sigma}_{i}\boldsymbol{c}}\right)}}\right\\}.$
(8)

The parameters can be estimated by maximizing the conditional log-likelihood
(8). The contrast vector $\boldsymbol{c}$ can be regarded as unknown
parameters and can be estimated. On the other hand, if one is interested in
some specific selective publication process, $\boldsymbol{c}$ can be specified
with certain values. Let
$(\hat{\boldsymbol{\mu}},\hat{\boldsymbol{\Omega}},\hat{\boldsymbol{c}},\hat{\beta})$
or $(\hat{\boldsymbol{\mu}},\hat{\boldsymbol{\Omega}},\hat{\beta})$ be the
maximum likelihood estimates, we denote SROC and SAUC estimates as
$SR\hat{O}C=SROC(x;\hat{\boldsymbol{\mu}},\hat{\boldsymbol{\Omega}})$ and
$SA\hat{U}C=SAUC(\hat{\boldsymbol{\mu}},\hat{\boldsymbol{\Omega}})$. The two-
tailed confidence interval of $SA\hat{U}C$ with significance level $\alpha$
can be estimated by using the delta method and logit-transformation of
$SA\hat{U}C$:

$\mathrm{logit}^{-1}\left\\{\mathrm{logit}(SA\hat{U}C)\pm
z_{1-\alpha/2}\dfrac{\hat{s}}{SA\hat{U}C(1-SA\hat{U}C)}\right\\},$

where $\hat{s}$ is the estimated variance of logit-transformed $SA\hat{U}C$
following the standard likelihood theory.

##  3 Example

We illustrated the proposed model by reanalyzing the published meta-analysis
(Safdar et al., 2005), which was also used in Hattori and Zhou (2018). This
meta-analysis enrolled 33 diagnostic studies to examine the diagnostic test
accuracy if the semi-quantitative or quantitative catheter segment culture for
intravascular device (IVD) related bloodstream infection. We first applied our
proposed method assuming $(c_{1},c_{2})$ is unknown. In the left upper panel
of Figure 1, SROC estimates were shown given $p$ = 1, 0.8, 0.6, 0.4. Recall
that $p=P(\mathrm{select})$ is the marginal selection probability; thus $p=1$
implies that there were no unselected studies and the inference agreed with
Reitsma model. If $p=0.8$ for example, there were $(1-0.8)\times 33/0.8\approx
8$ studies supposed to be unselected. SAUC estimates decreased from 0.859 by
Reitema model (with $p=1$) to 0.773 with $p=0.4$. We also applied our proposed
model with $(c_{1},c_{2})$ specified as $(1/\sqrt{2},1/\sqrt{2})$, $(1,0)$, or
$(0,1)$ and SROC estimates were also shown in Figure 1. As given in the
caption of Figure 1, the estimates of $(\hat{c}_{1},\hat{c}_{2})$ suggested
that $c_{1}=c_{2}$ hold approximately. Correspondingly, SROC estimates under
$(\hat{c}_{1},\hat{c}_{2})$ were very similar with those under $c_{1}=c_{2}$
assumed. On the other hand, SROC estimates seem different under
$(c_{1},c_{2})=(1,0)$ or $(0,1)$, indicating that specification of
$\boldsymbol{c}$ in the selection function may certainly influence the
estimates. In each panel of Figure 1, the trajectories of summary operating
points were different under different settings of $\boldsymbol{c}$. Under
$(c_{1},c_{2})=(1,0)$, the selective publication process is determined only by
$t$-type statistic for the sensitivity. Correspondingly, as seen in the left
lower panel of Figure 1, the estimated FPR ($x$-axis) of the summary operating
points did not change much, whereas the estimated TPR ($y$-axis) changed as
$p$ decreased. Under $c_{1}=c_{2}$, DOR was responsible for the selective
publication process. In the right upper panel of Figure 1, the summary
operating points changed similarly in TPR or FPR. It may suggest that, under
the selection function with $c_{1}=c_{2}$, studies in the right lower part of
SROC space are likely to be unselected. From these observations, one may
expect that tracing the summary operating points would be helpful to
understand and interpret the selection function assumed. To examine the
influence of estimation of $(c_{1},c_{2})$ on the precision of SROC estimate,
in Figure 2, we presented SAUC estimates with the 95% confidence intervals
with unknown $(c_{1},c_{2})$ and those with $c_{1}=c_{2}$ over
$p=1,0.9,\dots,0.1$. The confidence interval with unknown $(c_{1},c_{2})$ were
slightly wider than those assuming $c_{1}=c_{2}$, indicating that estimating
$(c_{1},c_{2})$ seemed not to lead serious loss in precision. With
$\hat{\beta}$ and $\hat{\alpha}$ given different $p$, we presented the
estimated probit selection function and $t$-scores ($x$-axis) of the selected
studies under different settings of $(c_{1},c_{2})$ in Figure 3. Under
$(\hat{c}_{1},\hat{c}_{2})$ and $c_{1}=c_{2}$, the selected studies almost had
high selection probability ($y$-axis), indicating that the estimated selection
function was reasonable to describe the probability of selecting study. In the
bottom left and right panels of Figure 3, with $(c_{1},c_{2})=(1,0)$ or
$(0,1)$, that is, $t$-scores were only dependent on sensitivity or
specificity, the estimated selection functions became much flatter. In
summary, regardless of the selective publication mechanisms, the statistically
significant diagnostic capacity of the test was maintained even with $p=0.3$,
which corresponded to the presence of 77 unpublished studies. This
strengthened the conclusion that quantitative and semi-quantitative catheter
segment culture tests were accurate for the diagnosis of IVD related
bloodstream infection (Safdar et al., 2005).

##  4 Simulation study

###  4.1 Performance of estimation on SAUC

Simulation studies were conducted to assess the performance of estimation of
the proposed model. We considered three different sizes of population studies
for meta-analysis: two moderate sizes ($S=25$ or $50$) and one large size
($S=200$). The large sample size might be impractical, but was considered to
check the performance of the proposed model in an ideal situation. True
sensitivity and specificity were set as $(0.5,0.85)$, $(0.8,0.8)$, or
$(0.9,0.4)$, yielding $(\mu_{1},\mu_{2})$ to be $(0,1.735),~{}(1.386,1.386)$,
or $(2.197,-0.405)$, respectively. The between-study variances
$(\tau_{1}^{2},\tau_{2}^{2})$ were set as $(1,4)$, and their correlation
$\rho$ was set as $-0.3$ or $-0.6$. The within-study variances $s_{1}^{2}$ or
$s_{2}^{2}$ were randomly generated from the square of $N(0.5,0.5^{2})$ (Piao
et al., 2019). In the selection function, we considered $\beta$ to be $0.5$
and three pairs of weight for the contrast vector:
$(c_{1},c_{2})=(1/\sqrt{2},1/\sqrt{2})$, $(1,0)$, or $(0,1)$, to simulate the
true selective publication process influenced by the $t$-type statistic on
DOR, only sensitivity, or only specificity, respectively. Under these
settings, $\alpha$ was set by leaving the marginal selection probability
$P(\mathrm{select})=p\approx 0.7$, yielding about 70% studies ($N\approx
18,35,140$) selected in the meta-analysis. The scenarios was summarized in
Table 2. In each scenario, we generated 1000 datasets of meta-analysis, and
estimated SAUC by the proposed model as well as Reitsma model that does not
account for the selection model. In the proposed model, we considered the
three settings: when $(c_{1},c_{2})$ was estimated as
$(\hat{c}_{1},\hat{c}_{2})$, when $(c_{1},c_{2})$ was correctly specified by
the true values, or when $(c_{1},c_{2})$ was misspecified. The
misspecification of $(c_{1},c_{2})$ was used to verify that misspecifying
contrast vector would cause biased estimates. All statistical computing was
conducted by R (R Development Core Team, Version 4.0.5). Reitsma model was
conducted by R package mvmeta (version 1.0.3, Gasparrini et al., 2012). The
proposed likelihood was numerically optimized by using R funciton nlminb().
One potential concern in the optimization is that the initial values or the
constrained bounds for $\beta$ may have some impact on the validity of
estimates. To check these issues, we compared the results from several
reasonable initial values and some reasonable constrained bounds for $\beta$
and the results did not differ much. The simulated selective publication
process was described in Appendix A of the Supplementary Materials.

Empirical SAUC estimates from the simulation datasets generated under true
selective publication process with $c_{1}=c_{2}$ were summarized in Table 3;
with $(c_{1},c_{2})=(1,0)$ summarized in Table 4, or with
$(c_{1},c_{2})=(0,1)$ summarized in Table 5. ReistmaP and ReistmaO refer the
estimates by Reitsma model applied to all the $S$ studies (selected and
unselected) and to only the selected $N$ studies, respectively. The
differences between ReistmaP and ReistmaO revealed the impact of publication
bias on SAUC. However, under $(c_{1},c_{2})=(0,1)$ in the selection function,
only specificity was responsible for the selective publication process,
leaving random selection on sensitivity. Since SAUC represents the average
measure of sensitivity given specificity, the publication bias mainly on
specificity was not obviously shown by the differences between ReistmaP and
ReistmaO. The overall results of SAUC estimates showed that the proposed model
with $(\hat{c}_{1},\hat{c}_{2})$ or with $(c_{1},c_{2})$ correctly specified
considerably removed the publication bias from ReistmaO and had less than 5%
bias when DOR or only sensitivity was responsible for the publication bias
(Table 3-4). When only specificity was responsible for the publication bias,
the proposed model with $(c_{1},c_{2})$ correctly specified had less than 0.5%
bias (Table 5), while the proposed model with $(\hat{c}_{1},\hat{c}_{2})$ had
some moderate bias (less than 10%) when $S=25,50$ in the low sensitivity
scenarios (Scenario No. 1 and No. 2 of Table 5). The moderate bias was mainly
resulted from the bias on $(\hat{c}_{1},\hat{c}_{2})$ given limited numbers of
selected studies with low sensitivity. In the other scenarios, the proposed
model with $(\hat{c}_{1},\hat{c}_{2})$ had less than 5% bias. When
$(c_{1},c_{2})$ was misspecified in the proposed model, biased estimates were
presented in all the scenarios. The scenarios of small between-study variances
$(\tau_{1}^{2},\tau_{2}^{2})=(0.5,0.5)$ were additionally considered in the
simulation studies, and the results were in agreement with those of
$(\tau_{1}^{2},\tau_{2}^{2})=(1,4)$ with less bias on SAUC estimates, as shown
in Table S3-S5 of the Supplementary Materials.

###  4.2 Tracking summary operating points to speculate the underlying
publication mechanism

As suggested in Section 3, tracking the trajectory of the summary operating
points over $p$ might give us some insights on the underlying selective
publication process. We randomly generated one dataset of meta-analysis with
$S=50$ under Scenario No. 3 (Table 2) to illustrate how the summary operating
points estimates could track the unselected studies. We considered the
selective publication process under $c_{1}=c_{2}$, $(c_{1},c_{2})=(1,0)$, and
$(c_{1},c_{2})=(0,1)$ with the selection probability of $p=0.5$. Based on the
selected studies, SROC and summary operating points were estimated using the
proposed model correctly specifying $(c_{1},c_{2})$, given $p=1,0.9,0.7,0.5$.
In the upper panels of Figure 4, the scatter plots of the selected (open
circles) and unselected (cross) studies were shown in the SROC space. In the
lower panels, SROC estimates over $p=1,0.9,0.7,0.5$ and the trajectory of the
summary operating points were shown. The regions of the selected studies were
shown in solid outline and the unselected studies in dashed outline,
respectively. The trajectories of the summary operating points seem to
approach the region of unselected studies.

##  5 Discussion

In meta-analysis of intervention studies, sensitivity analysis methods based
on selection models provide more objective and stable ways to address
publication bias issues than the widely used graphical methods, such as the
funnel plot and the trim-and-fill method (Carpenter et al., 2009; Schwarzer et
al., 2010), and then should play important roles in practice. Recently,
sensitivity analysis methods have been extended to more complicated situations
including network meta-analysis (Mavridis et al., 2013) and diagnostic studies
(Hattori and Zhou, 2018; Piao et al., 2019; Li et al., 2021). The importance
of the sensitivity analysis approach should be more emphasized in meta-
analysis of diagnostic test accuracy on SROC, since the graphical methods used
in meta-analysis of intervention studies are not reasonably applicable due to
the bivariate nature of the problem. Sensitivity analysis in multivariate or
network meta-analyses have been proposed based on the extensions of Heckman-
type (Copas, 1999; Copas and Shi, 2000, 2001) selection functions. In this
study, we took a different approach by extending the likelihood based
sensitivity analysis by Copas (2013), which enabled us to use a selection
function dependent on the study-specific cutoff values. This feature is
appealing in practice since in many diagnostic studies, main results are often
reported with study-specific cutoff values so that the selective publication
process should be influenced by them.

Even with these advantages, we must mention several limitations in our
methods. Our methods allow incorporating $t$-type statistic for the linear
combination of logit-transformed sensitivity and specificity to represent the
selective publication process, for example, determined by sensitivity,
specificity, or DOR. However, it is not easy to clarify what kind of selective
publication mechanism there is, and it is unclear whether a common measure
like DOR is responsible for publication in all studies. An intuitive idea is
to allow choosing different types of statistic in each study, and define a
selection function through its $p$-value, although it might be hard to make
study-specific determinations on which statistic is responsible for selective
publication. Accumulating practical experience would be warrant. Since it may
be impossible to identify the underlying selective publication process that
all the studies can accept, one approach is to conduct several sensitivity
analyses from different viewpoints. As demonstrated in Section 3 and 4.2,
tracing the summary operating points might be useful to make careful
interpretations of sensitivity analyses. To reduce the uncertainty on the
underlying selection process, we proposed to estimate the contrast vector. The
simulation studies revealed that the contrast vector can be stably estimated
in the proposed methods. However, even if we could estimate the contrast
vector successfully, those estimated results should be considered as a
suggestion for the most likely selective publication mechanism. If the
unpublished studies employed different cutoff values, the locations of the
unpublished studies on SROC space can be changed. Thus, it would be still
useful to obtain bias-adjusted estimates with several specified contrast
vectors as a part of sensitivity analysis to draw robust conclusions.

##  6 Software

Software in the form of R code, together with a sample input data set and
complete documentation is available at
https://github.com/meta2020/dtametasa-r.

## Acknowledgments

The first author’s research was partly supported by Grant-in-Aid for
Challenging Exploratory Research (16K12403) and for Scientific Research
(16H06299, 18H03208) from the Ministry of Education, Science, Sports and
Technology of Japan. Conflict of Interest: None declared.

## References

  * Bürkner and Doebler (2014) P. C. Bürkner and P. Doebler.  Testing for publication bias in diagnostic meta-analysis: a simulation study.  _Statistics in Medicine_ , 33:3061–3077, 2014.
  * Carpenter et al. (2009) J. R. Carpenter, G. Schwarzer, G. Rücker, and R. Künstler.  Empirical evaluation showed that the Copas selection model provided a useful summary in 80% of meta-analyses.  _Journal of Clinical Epidemiology_ , 62:624–631, 2009.
  * Chu et al. (2010) H. Chu, H. Guo, and Y. Zhou.  Bivariate random effects meta-analysis of diagnostic studies using generalized linear mixed models.  _Medical Decision Making_ , 30:499–508, 2010.
  * Copas (1999) J. B. Copas.  What works?: selectivity models and meta-analysis.  _Journal of the Royal Statistical Society. Series A: Statistics in Society_ , 162:95–109, 1999.
  * Copas (2013) J. B. Copas.  A likelihood-based sensitivity analysis for publication bias in meta-analysis.  _Journal of the Royal Statistical Society: Series C (Applied Statistics)_ , 62:47–66, 2013.
  * Copas and Jackson (2004) J. B. Copas and D. Jackson.  A bound for publication bias based on the fraction of unpublished studies.  _Biometrics_ , 60:146–153, 2004.
  * Copas and Shi (2000) J. B. Copas and J. Q. Shi.  Meta-analysis, funnel plots and sensitivity analysis.  _Biostatistics_ , 1:247–262, 2000.
  * Copas and Shi (2001) J. B. Copas and J. Q. Shi.  A sensitivity analysis for publication bias in systematic reviews.  _Statistical Methods in Medical Research_ , 10:251–265, 2001.
  * Deeks et al. (2005) J. J. Deeks, P. Macaskill, and L. Irwig.  The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed.  _Journal of Clinical Epidemiology_ , 58:882–893, 2005.
  * Doebler et al. (2012) P. Doebler, H. Holling, and D. Böhning.  A mixed model approach to meta-analysis of diagnostic studies with binary test outcome.  _Psychological Methods_ , 17:418–436, 2012.
  * Gasparrini et al. (2012) A. Gasparrini, B. Armstrong, and M. G. Kenward.  Multivariate meta-analysis for non-linear and other multi-parameter associations.  _Statistics in Medicine_ , 31:3821–3839, 2012.
  * Glas et al. (2003) A. S. Glas, J. G. Lijmer, M. H. Prins, Gouke J. Bonsel, and P. M. M. Bossuyt.  The diagnostic odds ratio: a single indicator of test performance.  _Journal of Clinical Epidemiology_ , 56:1129–1135, 2003\.
  * Hattori and Zhou (2018) S. Hattori and X. H. Zhou.  Sensitivity analysis for publication bias in meta-analysis of diagnostic studies for a continuous biomarker.  _Statistics in Medicine_ , 37:327–342, 2018.
  * Heckman (1976) J. J. Heckman.  The common structure of statistical models of truncation, sample selection and limited dependent variables and a simple estimator for such models.  _Annals of Economic and Social Measurement_ , 5:475–492, 1976.
  * Heckman (1979) J. J. Heckman.  Sample selection bias as a specification error.  _Econometrica_ , 47:153–161, 1979.
  * Honest and Khan (2002) H. Honest and K. S. Khan.  Reporting of measures of accuracy in systematic reviews of diagnostic literature.  _BMC Health Services Research_ , 2:4, 2002.
  * Li et al. (2021) M. Li, Y. Fan, Y. Liu, and Y Liu.  Diagnostic test meta-analysis by empirical likelihood under a Copas-like selection model.  _Metrika_ , pages 1–21, 2021.
  * Macaskill (2004) P. Macaskill.  Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis.  _Journal of Clinical Epidemiology_ , 57:925–932, 2004.
  * Macaskill et al. (2001) P. Macaskill, S. D. Walter, and L. Irwig.  A comparison of methods to detect publication bias in meta-analysis.  _Statistics in Medicine_ , 20:641–654, 2001.
  * Macaskill et al. (2010) P. Macaskill, C. Gatsonis, J. Deeks, R. Harbord, and Y. Takwoingi.  Chapter 10: Analysing and Presenting Results.  In _Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0_. The Cochrane Collaboration, 2010, 2010.
  * Mavridis et al. (2013) D. Mavridis, A. Sutton, A. Cipriani, and G. Salanti.  A fully Bayesian application of the Copas selection model for publication bias extended to network meta-analysis.  _Statistics in Medicine_ , 32:51–66, 2013.
  * Moses et al. (1993) L. E. Moses, D. Shapiro, and B. Littenberg.  Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations.  _Statistics in Medicine_ , 12:1293–1316, 1993.
  * Piao et al. (2019) J. Piao, Y. Liu, Y. Chen, and J. Ning.  Copas-like selection model to correct publication bias in systematic review of diagnostic test studies.  _Statistical Methods in Medical Research_ , 28:2912–2923, 2019.
  * Reitsma et al. (2005) J. B. Reitsma, A. S. Glas, A. W.S. Rutjes, R.J.P.M. Scholten, P. M. Bossuyt, and A H. Zwinderman.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews.  _Journal of Clinical Epidemiology_ , 58:982–990, 2005.
  * Rutter and Gatsonis (2001) C. M. Rutter and C. A. Gatsonis.  A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations.  _Statistics in Medicine_ , 20:2865–2884, 2001.
  * Safdar et al. (2005) N. Safdar, J. P. Fine, and D. G. Maki.  Meta-analysis: methods for diagnosing intravascular device-related bloodstream infection.  _Annals of Internal Medicine_ , 142:451–466, 2005.
  * Schwarzer et al. (2010) G. Schwarzer, J. Carpenter, and G. Rücker.  Empirical evaluation suggests Copas selection model preferable to trim-and-fill method for selection bias in meta-analysis.  _Journal of Clinical Epidemiology_ , 63:282–288, 2010.
  * Sutton et al. (2000) A. J. Sutton, F. Song, S. M. Gilbody, and K. R. Abrams.  Modelling publication bias in meta-analysis: a review.  _Statistical Methods in Medical Research_ , 9:421–445, 2000\.
  * Walter (2002) S. D. Walter.  Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data.  _Statistics in Medicine_ , 21:1237–1256, 2002.

Figure 1:  IVD example: SROC estimates given estimated contrast vector
$(\hat{c}_{1},\hat{c}_{2})$ or specified $({c}_{1},{c}_{2})$. The circle
points are the estimated 1-specificity and sensitivity
$(1-\hat{\mathrm{sp}_{i}},\hat{\mathrm{se}_{i}})$ of each study $i$; diamond
points are the estimated summary operating points
$(1-\mathrm{logit}^{-1}\hat{\mu}_{2},\mathrm{logit}^{-1}\hat{\mu}_{1})$. In
the top left panel, $(\hat{c}_{1},\hat{c}_{2})=(0.746,0.666)$ given $p=0.8$,
$(\hat{c}_{1},\hat{c}_{2})=(0.691,0.723)$ given $p=0.6$, and
$(\hat{c}_{1},\hat{c}_{2})=(0.657,0.754)$ given $p=0.4$.  Figure 2:  IVD
example: SAUC estimates (black) with 95% confidence interval (red) over $p$.
The gray dashed horizontal lines are SAUC = 0.5.  Figure 3:  IVD example: the
probit selection function based on $\hat{\beta}$ and $\hat{\alpha}$ given
different $p$. In the left panel, the vertical lines at the bottom are the
$t$-type statistic of each study, calculated by
$\hat{t}=\hat{\boldsymbol{c}}^{T}\boldsymbol{y}/\sqrt{\hat{\boldsymbol{c}}^{T}\boldsymbol{\Sigma}\hat{\boldsymbol{c}}}$,
where $\hat{\boldsymbol{c}}^{T}=(\hat{c}_{1},\hat{c}_{2})$; red lines are
$\hat{t}$ given $p=0.8,(\hat{c}_{1},\hat{c}_{2})=(0.746,0.666)$; green lines
are $\hat{t}$ given $p=0.6,(\hat{c}_{1},\hat{c}_{2})=(0.691,0.723)$; blue
lines are $\hat{t}$ given $p=0.4,(\hat{c}_{1},\hat{c}_{2})=(0.657,0.754)$. In
the other panels, the black vertical lines at the bottom are the $t$-type
statistic of each study, calculated by
$\hat{t}={\boldsymbol{c}}^{T}\boldsymbol{y}/\sqrt{{\boldsymbol{c}}^{T}\boldsymbol{\Sigma}{\boldsymbol{c}}}$,
where $\boldsymbol{c}$ is specified by fixed values.  Figure 4:  Changes of
estimates of SROC and summary operating points when $(c_{1},c_{2})$ are
correctly specified with the true values. The selected or unselected studies
are simulated by using Scenario No. 3. The regions of the selected studies (in
solid gray outline) and the unselected studies (in dashed gray outline) were
given by the logistic-transformed ellipse, of which the centers are the
average of logit-transformed pairs of FPR and TPR $(-y_{2i},y_{1i})$ and
shapes are based on the covariance matrix of $(y_{2i},y_{1i})$.

Table 1: Notations for cell frequencies in $2\times 2$ confusion matrix for diagnostic study $i$.  | Truth
---|---
Diseased | Non-diseased
Diagnostic test | Positive | $n_{11}^{i}$ | $n_{10}^{i}$
| Negative | $n_{01}^{i}$ | $n_{00}^{i}$
Total | $n_{+1}^{i}$ | $n_{+0}^{i}$
Table 2: Scenarios used in the simulation studies with $p\approx 0.7$ |  |  |  |  |  |  |  |  | $c_{1}=c_{2}$ | $c_{1}=1$ | $c_{1}=0$
---|---|---|---|---|---|---|---|---|---|---|---
No. | SAUC | $\mu_{1}$ | $\mu_{2}$ | $\tau_{1}$ | $\tau_{2}$ | $\tau_{12}$ | $\rho$ | $\beta$ | $\alpha$ | $\alpha$ | $\alpha$
1 | 0.564 | 0.000 | 1.735 | 1 | 2 | -0.6 | -0.3 | 0.5 | -0.165 | 0.891 | -0.429
2 | 0.620 | 0.000 | 1.735 | 1 | 2 | -1.2 | -0.6 | 0.5 | -0.251 | 0.894 | -0.433
3 | 0.828 | 1.386 | 1.386 | 1 | 2 | -0.6 | -0.3 | 0.5 | -0.766 | -0.570 | -0.111
4 | 0.846 | 1.386 | 1.386 | 1 | 2 | -1.2 | -0.6 | 0.5 | -0.848 | -0.573 | -0.118
5 | 0.892 | 2.197 | -0.405 | 1 | 2 | -0.6 | -0.3 | 0.5 | -0.198 | -1.269 | 1.744
6 | 0.877 | 2.197 | -0.405 | 1 | 2 | -1.2 | -0.6 | 0.5 | -0.284 | -1.269 | 1.733
Table 3:  Summary of SAUC estimates when $c_{1}=c_{2}$ in the true publication
mechanism

|  |  | $S=25$ | $S=50$ | $S=200$
---|---|---|---|---|---
No. |  | SAUC | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR
1 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.564 | 0.591 (0.484, 0.670) | 99.5 | 0.567 (0.493, 0.631) | 99.9 | 0.563 (0.523, 0.607) | 99.8
| Proposed $(c_{1}=c_{2})$ |  | 0.596 (0.503, 0.672) | 99.9 | 0.582 (0.512, 0.639) | 99.8 | 0.574 (0.537, 0.610) | 98.9
| Proposed $(c_{1}=1)$ |  | 0.660 (0.593, 0.717) | 83.7 | 0.658 (0.608, 0.699) | 71.6 | 0.663 (0.641, 0.686) | 24.9
| ReistmaO |  | 0.676 (0.615, 0.727) | 99.9 | 0.670 (0.629, 0.708) | 99.9 | 0.673 (0.653, 0.692) | 100.0
| ReistmaP |  | 0.567 (0.517, 0.615) | 99.9 | 0.563 (0.529, 0.595) | 100.0 | 0.566 (0.546, 0.583) | 100.0
2 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.620 | 0.634 (0.558, 0.702) | 99.7 | 0.624 (0.566, 0.669) | 99.5 | 0.619 (0.583, 0.651) | 99.5
| Proposed $(c_{1}=c_{2})$ |  | 0.637 (0.566, 0.699) | 99.5 | 0.631 (0.581, 0.679) | 99.6 | 0.624 (0.595, 0.649) | 98.9
| Proposed $(c_{1}=1)$ |  | 0.690 (0.633, 0.735) | 83.9 | 0.692 (0.658, 0.721) | 66.1 | 0.692 (0.675, 0.704) | 21.2
| ReistmaO |  | 0.699 (0.649, 0.739) | 99.8 | 0.700 (0.664, 0.729) | 99.9 | 0.697 (0.681, 0.711) | 100.0
| ReistmaP |  | 0.621 (0.579, 0.658) | 99.8 | 0.619 (0.591, 0.648) | 100.0 | 0.619 (0.605, 0.633) | 99.9
3 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.828 | 0.840 (0.796, 0.873) | 99.9 | 0.837 (0.799, 0.863) | 99.7 | 0.830 (0.811, 0.846) | 99.8
| Proposed $(c_{1}=c_{2})$ |  | 0.835 (0.795, 0.871) | 99.8 | 0.836 (0.804, 0.863) | 99.7 | 0.829 (0.813, 0.845) | 99.9
| Proposed $(c_{1}=1)$ |  | 0.853 (0.819, 0.883) | 98.0 | 0.858 (0.832, 0.877) | 96.7 | 0.863 (0.850, 0.873) | 87.6
| ReistmaO |  | 0.872 (0.842, 0.892) | 100.0 | 0.873 (0.852, 0.888) | 99.7 | 0.872 (0.863, 0.880) | 99.9
| ReistmaP |  | 0.829 (0.799, 0.850) | 100.0 | 0.826 (0.807, 0.845) | 99.9 | 0.827 (0.819, 0.836) | 100.0
4 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.846 | 0.852 (0.818, 0.877) | 99.5 | 0.850 (0.827, 0.868) | 99.5 | 0.848 (0.836, 0.859) | 99.5
| Proposed $(c_{1}=c_{2})$ |  | 0.850 (0.819, 0.876) | 99.4 | 0.850 (0.829, 0.868) | 99.7 | 0.848 (0.837, 0.858) | 100.0
| Proposed $(c_{1}=1)$ |  | 0.864 (0.838, 0.885) | 97.5 | 0.866 (0.850, 0.880) | 95.8 | 0.870 (0.861, 0.877) | 82.5
| ReistmaO |  | 0.873 (0.853, 0.892) | 99.9 | 0.874 (0.861, 0.887) | 99.8 | 0.876 (0.869, 0.882) | 100.0
| ReistmaP |  | 0.846 (0.824, 0.864) | 100.0 | 0.845 (0.833, 0.858) | 99.9 | 0.846 (0.839, 0.852) | 100.0
5 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.892 | 0.891 (0.869, 0.909) | 99.6 | 0.891 (0.875, 0.905) | 99.6 | 0.892 (0.884, 0.899) | 99.7
| Proposed $(c_{1}=c_{2})$ |  | 0.892 (0.872, 0.909) | 99.6 | 0.893 (0.879, 0.905) | 99.8 | 0.892 (0.885, 0.899) | 99.2
| Proposed $(c_{1}=1)$ |  | 0.889 (0.862, 0.909) | 99.4 | 0.893 (0.873, 0.907) | 99.3 | 0.897 (0.888, 0.904) | 98.3
| ReistmaO |  | 0.904 (0.888, 0.919) | 99.9 | 0.907 (0.897, 0.916) | 100.0 | 0.908 (0.903, 0.913) | 99.9
| ReistmaP |  | 0.891 (0.873, 0.905) | 99.8 | 0.891 (0.879, 0.901) | 100.0 | 0.892 (0.886, 0.897) | 99.9
6 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.877 | 0.877 (0.852, 0.896) | 99.7 | 0.876 (0.861, 0.890) | 99.8 | 0.878 (0.870, 0.885) | 99.1
| Proposed $(c_{1}=c_{2})$ |  | 0.878 (0.858, 0.896) | 99.5 | 0.879 (0.864, 0.892) | 99.9 | 0.878 (0.872, 0.885) | 99.6
| Proposed $(c_{1}=1)$ |  | 0.873 (0.848, 0.893) | 99.0 | 0.877 (0.859, 0.891) | 99.1 | 0.881 (0.871, 0.888) | 97.3
| ReistmaO |  | 0.888 (0.870, 0.903) | 99.9 | 0.890 (0.877, 0.901) | 99.8 | 0.891 (0.886, 0.896) | 100.0
| ReistmaP |  | 0.875 (0.858, 0.891) | 100.0 | 0.877 (0.865, 0.889) | 99.8 | 0.877 (0.872, 0.883) | 100.0

  * •

SAUC is the true value of SAUC; Q1 is the first quartile; Q3 is the third
quartile; CR is convergence rate; Proposed$({\hat{c}_{1},\hat{c}_{2}})$ is the
proposed model estimating $(c_{1},c_{2})$; Proposed $(c_{1}=c_{2})$ is the
proposed model correctly specifying that $c_{1}=c_{2}$; Proposed $(c_{1}=1)$
is the proposed model misspecifying that $(c_{1},c_{2})=(1,0)$; ReitsmaO is
Reitsma model based on the selected studies; and ReitsmaP is Reitsma model
based on the population studies.

Table 4:  Summary of SAUC estimates when $(c_{1},c_{2})=(1,0)$ in the true
publication mechanism

|  |  | $S=25$ | $S=50$ | $S=200$
---|---|---|---|---|---
No. |  | SAUC | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR
1 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.564 | 0.596 (0.523, 0.657) | 99.6 | 0.570 (0.514, 0.624) | 99.3 | 0.562 (0.532, 0.588) | 98.0
| Proposed $(c_{1}=1)$ |  | 0.585 (0.520, 0.651) | 99.8 | 0.573 (0.522, 0.622) | 99.8 | 0.568 (0.541, 0.591) | 99.0
| Proposed $(c_{1}=c_{2})$ |  | 0.621 (0.575, 0.672) | 99.8 | 0.625 (0.588, 0.660) | 99.5 | 0.644 (0.627, 0.661) | 98.4
| ReistmaO |  | 0.648 (0.606, 0.693) | 100.0 | 0.648 (0.616, 0.677) | 100.0 | 0.651 (0.635, 0.667) | 100.0
| ReistmaP |  | 0.567 (0.517, 0.615) | 99.9 | 0.563 (0.529, 0.595) | 100.0 | 0.566 (0.546, 0.583) | 100.0
2 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.620 | 0.635 (0.569, 0.685) | 99.9 | 0.626 (0.579, 0.667) | 99.5 | 0.620 (0.600, 0.642) | 98.6
| Proposed $(c_{1}=1)$ |  | 0.637 (0.579, 0.687) | 99.7 | 0.630 (0.586, 0.665) | 99.9 | 0.621 (0.604, 0.639) | 99.7
| Proposed $(c_{1}=c_{2})$ |  | 0.653 (0.607, 0.695) | 100.0 | 0.656 (0.622, 0.683) | 99.7 | 0.666 (0.652, 0.681) | 98.9
| ReistmaO |  | 0.675 (0.637, 0.713) | 100.0 | 0.674 (0.647, 0.700) | 100.0 | 0.674 (0.662, 0.687) | 100.0
| ReistmaP |  | 0.621 (0.579, 0.658) | 99.8 | 0.619 (0.591, 0.648) | 100.0 | 0.619 (0.605, 0.633) | 99.9
3 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.828 | 0.836 (0.800, 0.866) | 99.6 | 0.834 (0.804, 0.857) | 99.6 | 0.829 (0.814, 0.843) | 99.1
| Proposed $(c_{1}=1)$ |  | 0.833 (0.798, 0.864) | 99.9 | 0.832 (0.804, 0.856) | 99.7 | 0.829 (0.816, 0.841) | 99.9
| Proposed $(c_{1}=c_{2})$ |  | 0.845 (0.818, 0.870) | 99.4 | 0.848 (0.829, 0.867) | 99.3 | 0.852 (0.841, 0.861) | 99.0
| ReistmaO |  | 0.861 (0.838, 0.884) | 100.0 | 0.862 (0.847, 0.878) | 99.8 | 0.864 (0.856, 0.871) | 99.8
| ReistmaP |  | 0.829 (0.799, 0.850) | 100.0 | 0.826 (0.807, 0.845) | 99.9 | 0.827 (0.819, 0.836) | 100.0
4 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.846 | 0.852 (0.828, 0.872) | 99.6 | 0.851 (0.832, 0.867) | 99.4 | 0.848 (0.838, 0.858) | 99.4
| Proposed $(c_{1}=1)$ |  | 0.850 (0.823, 0.871) | 99.5 | 0.851 (0.831, 0.865) | 99.5 | 0.846 (0.837, 0.855) | 99.5
| Proposed $(c_{1}=c_{2})$ |  | 0.857 (0.834, 0.874) | 99.7 | 0.858 (0.842, 0.871) | 99.8 | 0.860 (0.851, 0.868) | 99.3
| ReistmaO |  | 0.869 (0.849, 0.884) | 100.0 | 0.870 (0.858, 0.882) | 99.8 | 0.870 (0.864, 0.876) | 99.8
| ReistmaP |  | 0.846 (0.824, 0.864) | 100.0 | 0.845 (0.833, 0.858) | 99.9 | 0.846 (0.839, 0.852) | 100.0
5 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.892 | 0.901 (0.883, 0.918) | 99.1 | 0.899 (0.886, 0.912) | 99.0 | 0.895 (0.886, 0.903) | 95.4
| Proposed $(c_{1}=1)$ |  | 0.895 (0.870, 0.914) | 99.4 | 0.893 (0.877, 0.908) | 99.4 | 0.892 (0.884, 0.900) | 99.9
| Proposed $(c_{1}=c_{2})$ |  | 0.908 (0.892, 0.923) | 99.5 | 0.910 (0.900, 0.919) | 99.3 | 0.912 (0.907, 0.917) | 95.5
| ReistmaO |  | 0.913 (0.897, 0.927) | 100.0 | 0.915 (0.905, 0.923) | 100.0 | 0.915 (0.911, 0.920) | 99.9
| ReistmaP |  | 0.891 (0.873, 0.905) | 99.8 | 0.891 (0.879, 0.901) | 100.0 | 0.892 (0.886, 0.897) | 99.9
6 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.877 | 0.889 (0.867, 0.907) | 99.2 | 0.887 (0.873, 0.900) | 99.5 | 0.883 (0.873, 0.892) | 97.9
| Proposed $(c_{1}=1)$ |  | 0.882 (0.855, 0.901) | 99.3 | 0.879 (0.863, 0.895) | 99.4 | 0.878 (0.870, 0.887) | 99.8
| Proposed $(c_{1}=c_{2})$ |  | 0.895 (0.877, 0.911) | 99.9 | 0.895 (0.884, 0.907) | 99.3 | 0.896 (0.891, 0.902) | 98.4
| ReistmaO |  | 0.898 (0.880, 0.913) | 99.9 | 0.899 (0.887, 0.910) | 99.8 | 0.900 (0.895, 0.905) | 99.7
| ReistmaP |  | 0.875 (0.858, 0.891) | 100.0 | 0.877 (0.865, 0.889) | 99.8 | 0.877 (0.872, 0.883) | 100.0

  * •

SAUC is the true value of SAUC; Q1 is the first quartile; Q3 is the third
quartile; CR is convergence rate; Proposed $(\hat{c}_{1},\hat{c}_{2})$ is the
proposed model estimating $(c_{1},c_{2})$; Proposed $(c_{1}=1)$ is the
proposed model correctly specifying that $(c_{1},c_{2})=(1,0)$; Proposed
$(c_{1}=c_{2})$ is the proposed model misspecifying that $c_{1}=c_{2}$;
ReitsmaO is Reitsma model based on the selected studies; and ReitsmaP is
Reitsma model based on the population studies.

Table 5:  Summary of SAUC estimates when $(c_{1},c_{2})=(0,1)$ in the true
publication mechanism

|  |  | $S=25$ | $S=50$ | $S=200$
---|---|---|---|---|---
No. |  | SAUC | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR
1 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.564 | 0.475 (0.373, 0.596) | 99.9 | 0.488 (0.398, 0.568) | 99.1 | 0.533 (0.483, 0.571) | 98.9
| Proposed $(c_{1}=0)$ |  | 0.563 (0.467, 0.656) | 99.7 | 0.561 (0.499, 0.625) | 99.7 | 0.562 (0.533, 0.593) | 99.6
| Proposed $(c_{1}=c_{2})$ |  | 0.474 (0.362, 0.577) | 99.7 | 0.476 (0.385, 0.560) | 99.4 | 0.487 (0.439, 0.543) | 99.1
| ReistmaO |  | 0.568 (0.470, 0.663) | 100.0 | 0.569 (0.502, 0.634) | 99.9 | 0.569 (0.539, 0.601) | 100.0
| ReistmaP |  | 0.567 (0.517, 0.615) | 99.9 | 0.563 (0.529, 0.595) | 100.0 | 0.566 (0.546, 0.583) | 100.0
2 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.620 | 0.562 (0.444, 0.647) | 99.9 | 0.560 (0.484, 0.629) | 100.0 | 0.593 (0.556, 0.624) | 99.3
| Proposed $(c_{1}=0)$ |  | 0.621 (0.550, 0.687) | 99.9 | 0.621 (0.568, 0.670) | 99.8 | 0.619 (0.596, 0.640) | 99.3
| Proposed $(c_{1}=c_{2})$ |  | 0.554 (0.447, 0.643) | 99.9 | 0.554 (0.471, 0.620) | 99.8 | 0.564 (0.507, 0.613) | 98.8
| ReistmaO |  | 0.631 (0.558, 0.696) | 99.9 | 0.631 (0.578, 0.680) | 100.0 | 0.632 (0.607, 0.654) | 99.9
| ReistmaP |  | 0.621 (0.579, 0.658) | 99.8 | 0.619 (0.591, 0.648) | 100.0 | 0.619 (0.605, 0.633) | 99.9
3 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.828 | 0.797 (0.707, 0.848) | 99.8 | 0.806 (0.742, 0.845) | 99.3 | 0.818 (0.795, 0.836) | 99.4
| Proposed $(c_{1}=0)$ |  | 0.828 (0.771, 0.864) | 99.8 | 0.831 (0.796, 0.859) | 99.0 | 0.827 (0.813, 0.842) | 99.3
| Proposed $(c_{1}=c_{2})$ |  | 0.784 (0.690, 0.845) | 99.6 | 0.789 (0.723, 0.838) | 99.2 | 0.798 (0.763, 0.824) | 99.7
| ReistmaO |  | 0.831 (0.770, 0.867) | 100.0 | 0.835 (0.796, 0.863) | 99.7 | 0.831 (0.816, 0.846) | 99.9
| ReistmaP |  | 0.829 (0.799, 0.850) | 100.0 | 0.826 (0.807, 0.845) | 99.9 | 0.827 (0.819, 0.836) | 100.0
4 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.846 | 0.825 (0.777, 0.864) | 99.5 | 0.834 (0.799, 0.856) | 99.9 | 0.841 (0.827, 0.853) | 99.9
| Proposed $(c_{1}=0)$ |  | 0.844 (0.811, 0.873) | 99.7 | 0.848 (0.824, 0.865) | 99.9 | 0.846 (0.834, 0.856) | 99.7
| Proposed $(c_{1}=c_{2})$ |  | 0.822 (0.771, 0.863) | 99.9 | 0.826 (0.787, 0.854) | 99.9 | 0.835 (0.813, 0.850) | 99.5
| ReistmaO |  | 0.850 (0.816, 0.878) | 99.9 | 0.852 (0.829, 0.871) | 100.0 | 0.852 (0.840, 0.862) | 100.0
| ReistmaP |  | 0.846 (0.824, 0.864) | 100.0 | 0.845 (0.833, 0.858) | 99.9 | 0.846 (0.839, 0.852) | 100.0
5 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.892 | 0.878 (0.852, 0.897) | 99.2 | 0.883 (0.863, 0.897) | 99.6 | 0.890 (0.881, 0.896) | 99.9
| Proposed $(c_{1}=0)$ |  | 0.888 (0.869, 0.904) | 99.7 | 0.890 (0.878, 0.902) | 99.6 | 0.892 (0.885, 0.898) | 99.2
| Proposed $(c_{1}=c_{2})$ |  | 0.876 (0.849, 0.895) | 99.6 | 0.880 (0.861, 0.895) | 99.9 | 0.886 (0.876, 0.893) | 99.5
| ReistmaO |  | 0.888 (0.869, 0.905) | 99.9 | 0.892 (0.879, 0.903) | 100.0 | 0.893 (0.887, 0.899) | 100.0
| ReistmaP |  | 0.891 (0.873, 0.905) | 99.8 | 0.891 (0.879, 0.901) | 100.0 | 0.892 (0.886, 0.897) | 99.9
6 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.877 | 0.865 (0.838, 0.885) | 99.8 | 0.869 (0.851, 0.884) | 99.8 | 0.875 (0.868, 0.882) | 99.6
| Proposed $(c_{1}=0)$ |  | 0.874 (0.854, 0.892) | 100.0 | 0.876 (0.862, 0.889) | 99.8 | 0.877 (0.871, 0.883) | 99.5
| Proposed $(c_{1}=c_{2})$ |  | 0.865 (0.839, 0.886) | 99.9 | 0.869 (0.852, 0.884) | 99.7 | 0.875 (0.867, 0.882) | 99.7
| ReistmaO |  | 0.877 (0.857, 0.894) | 99.9 | 0.879 (0.865, 0.891) | 100.0 | 0.880 (0.874, 0.886) | 99.9
| ReistmaP |  | 0.875 (0.858, 0.891) | 100.0 | 0.877 (0.865, 0.889) | 99.8 | 0.877 (0.872, 0.883) | 100.0

  * •

SAUC is the true value of SAUC; Q1 is the first quartile; Q3 is the third
quartile; CR is convergence rate; Proposed $(\hat{c}_{1},\hat{c}_{2})$ is the
proposed model estimating $(c_{1},c_{2})$; Proposed $(c_{1}=1)$ is the
proposed model correctly specifying that $(c_{1},c_{2})=(0,1)$; Proposed
$(c_{1}=c_{2})$ is the proposed model misspecifying that $c_{1}=c_{2}$;
ReitsmaO is Reitsma model based on the selected studies; and ReitsmaP is
Reitsma model based on the population studies.

## Supplementary Material

## Appendix A Publication process and additional scenarios in the simulation
study

The publication process was simulated as follows:

  1. Step 1:

generate $S$ population studies from $\boldsymbol{y}_{i}\sim
N_{2}\left(\boldsymbol{\mu},\boldsymbol{\Omega}+\boldsymbol{\Sigma}_{i}\right)$,
with $s_{1i}$ or $s_{2i}\sim N(0.5,0.5^{2}),~{}i=1,2,\dots,S$;

  2. Step 2:

calculate the selection probability for each study $i$,
$p_{i}=\Phi(\beta{t}_{i}+\alpha)$, and generate the random indicator
$z_{i}\sim\mathrm{Bernoulli}(p_{i})$;

  3. Step 3:

select studies with indicator $z_{i}=1$ as one set of observed studies.

The selection probability $p$ was estimated from the population studies as
$\hat{p}=(\sum_{s=1}^{S}p_{i})/S$ in Step 2, and then $\hat{p}$ was used in
the proposed likelihood for the parameters estimation.

The additional scenarios are the combination of $(\mu_{1},\mu_{2})=(0,1.735)$,
$(1.386,1.386)$, or
$(2.917,-0.405)$; $(\tau_{1}^{2},\tau_{2}^{2})=(0.5,0.5)$, $\rho=-0.3$ or
$-0.6$, $\beta=0.5$, and $(c_{1},c_{2})=(1/\sqrt{2},1/\sqrt{2})$, $(1,0)$, or
$(0,1)$, given the selection probability $p\approx 0.7$. All the scenarios of
are shown in Table SS2. The population sample size $S=25$, $50$, or $200$ was
considered in each scenario. Three conditions of estimation were conducted by
the proposed model: (1) the contrast vector $\boldsymbol{c}$ is unknown and to
be estimated together with the other parameters; (2) $\boldsymbol{c}$ is
correctly specified by their true values; and (3) $\boldsymbol{c}$ is
misspecified.

All statistical computing was conducted by R (R Development Core Team, Version
4.0.5). The estimates from Reitsma model were computed by mvmeta() function in
R package mvmeta (version 1.0.3, Gasparrini et al., 2012). The estimates from
the proposed likelihood were numerically optimized by nlminb() function, where
the initial values of $\boldsymbol{\mu}$ and $\boldsymbol{\Omega}$ were
assigned as the estimates from the mvmeta() function. The initial values of
$\beta$ and $c_{1}$ were assigned as $1$ and $0.5$, respectively. The
optimization constrained boundaries of each parameters were the intervals of
$[-5,5]$ for $\mu_{1}$ and $\mu_{2}$, $(0,3]$ for $\tau_{1}$ and $\tau_{2}$,
$[-1,1]$ for $\rho$, $[0,2]$ for $\beta$, and $[0,1]$ for $c_{1}$. Parameter
$\alpha$ was solved by uniroot() function embedded in the optimization
process.

The estimates of SAUC for the additional Scenarios No.7$-$12 are presented in
Table SS3$-$SS5. The results are summarized by median with 25th empirical
quartile (Q1) and 75th empirical quartile (Q3). The following notations are
used in the tables: No. is the scenario numbers corresponding to that in Table
SS2; SAUC is true SAUC values; CR is the convergence rate in 1000 simulations.
Proposed ($\hat{c}_{1},\hat{c}_{2}$) is the proposed model when
$\boldsymbol{c}$ is estimated; Proposed ($c_{1}=c_{2}$) is the proposed model
when $(c_{1},c_{2})=(1/\sqrt{2},1/\sqrt{2})$ is specified; Proposed
($c_{1}=1$) is the proposed model when $(c_{1},c_{2})=(1,0)$ is specified; and
Proposed ($c_{1}=0$) is the proposed model when $(c_{1},c_{2})=(0,1)$ is
specified. ReitsmaO is Reitsma model based on the selected studies, and
ReitsmaP is Reitsma model based on the population studies.

Table S1:  IVD: estimates from different settings of $(c_{1},c_{2})$ $(c_{1},c_{2})$ | $p$ | SAUC (95%CI) | $\mu_{1}$ | $\mu_{2}$ | $\tau_{1}$ | $\tau_{2}$ | $\rho$ | $c_{1}$ | $c_{2}$ |  $\beta$ (95%CI) | $\alpha_{p}$ | se | sp
---|---|---|---|---|---|---|---|---|---|---|---|---|---
$(\hat{c}_{1},\hat{c}_{2})$ | 1 | 0.859 (0.790, 0.908) | 1.388 | 1.804 | 0.545 | 0.819 | -0.423 |  |  |  |  | 0.8 | 0.859
| 0.8 | 0.849 (0.772, 0.904) | 1.349 | 1.773 | 0.552 | 0.828 | -0.382 | 0.746 | 0.666 | 2.000 (-1.596, 5.596) | -3.449 | 0.794 | 0.855
| 0.6 | 0.819 (0.719, 0.889) | 1.236 | 1.64 | 0.551 | 0.859 | -0.297 | 0.691 | 0.723 | 2.000 (-0.536, 4.536) | -4.871 | 0.775 | 0.838
| 0.4 | 0.773 (0.629, 0.872) | 1.086 | 1.396 | 0.558 | 0.919 | -0.174 | 0.657 | 0.754 | 1.993 (0.418, 3.568) | -5.719 | 0.748 | 0.802
$(c_{1}=c_{2})$ | 1 | 0.859 (0.790, 0.908) | 1.388 | 1.804 | 0.545 | 0.819 | -0.423 |  |  |  |  | 0.8 | 0.859
| 0.8 | 0.849 (0.774, 0.902) | 1.352 | 1.769 | 0.55 | 0.831 | -0.382 | 0.707 | 0.707 | 2.000 (-0.194, 4.194) | -3.647 | 0.794 | 0.854
| 0.6 | 0.819 (0.723, 0.886) | 1.23 | 1.645 | 0.554 | 0.857 | -0.294 | 0.707 | 0.707 | 2.000 (0.097, 3.903) | -4.781 | 0.774 | 0.838
| 0.4 | 0.763 (0.617, 0.865) | 1.04 | 1.43 | 0.578 | 0.903 | -0.151 | 0.707 | 0.707 | 2.000 (0.410, 3.590) | -5.456 | 0.739 | 0.807
$(c_{1}=1)$ | 1 | 0.859 (0.790, 0.908) | 1.388 | 1.804 | 0.545 | 0.819 | -0.423 |  |  |  |  | 0.8 | 0.859
| 0.8 | 0.844 (0.762, 0.901) | 1.215 | 1.85 | 0.631 | 0.826 | -0.407 | 1 | 0 | 1.150 (-0.010, 2.310) | -0.204 | 0.771 | 0.864
| 0.6 | 0.820 (0.718, 0.891) | 0.99 | 1.904 | 0.691 | 0.832 | -0.404 | 1 | 0 | 1.046 (0.289, 1.804) | -0.842 | 0.729 | 0.87
| 0.4 | 0.786 (0.649, 0.879) | 0.701 | 1.972 | 0.749 | 0.837 | -0.402 | 1 | 0 | 0.987 (0.297, 1.676) | -1.29 | 0.668 | 0.878
$(c_{1}=0)$ | 1 | 0.859 (0.790, 0.908) | 1.388 | 1.804 | 0.545 | 0.819 | -0.423 |  |  |  |  | 0.8 | 0.859
| 0.8 | 0.857 (0.788, 0.906) | 1.447 | 1.567 | 0.56 | 0.968 | -0.468 | 0 | 1 | 0.663 (-0.051, 1.377) | -1.198 | 0.81 | 0.827
| 0.6 | 0.855 (0.787, 0.905) | 1.525 | 1.253 | 0.575 | 1.098 | -0.506 | 0 | 1 | 0.617 (0.079, 1.154) | -1.72 | 0.821 | 0.778
| 0.4 | 0.854 (0.786, 0.904) | 1.633 | 0.812 | 0.592 | 1.245 | -0.547 | 0 | 1 | 0.581 (0.073, 1.090) | -2.034 | 0.837 | 0.692

Table S2: All the scenarios used in simulation studies given $p=0.7$.  |  |  |  |  |  |  |  |  | $c_{1}=c_{2}$ | $c_{1}=1$ | $c_{1}=0$
---|---|---|---|---|---|---|---|---|---|---|---
No. | SAUC | $\mu_{1}$ | $\mu_{2}$ | $\tau_{1}$ | $\tau_{2}$ | $\tau_{12}$ | $\rho$ | $\beta$ | $\alpha$ | $\alpha$ | $\alpha$
1 | 0.564 | 0.000 | 1.735 | 1 | 2 | -0.6 | -0.3 | 0.5 | -0.165 | 0.891 | -0.429
2 | 0.620 | 0.000 | 1.735 | 1 | 2 | -1.2 | -0.6 | 0.5 | -0.251 | 0.894 | -0.433
3 | 0.828 | 1.386 | 1.386 | 1 | 2 | -0.6 | -0.3 | 0.5 | -0.766 | -0.570 | -0.111
4 | 0.846 | 1.386 | 1.386 | 1 | 2 | -1.2 | -0.6 | 0.5 | -0.848 | -0.573 | -0.118
5 | 0.892 | 2.197 | -0.405 | 1 | 2 | -0.6 | -0.3 | 0.5 | -0.198 | -1.269 | 1.744
6 | 0.877 | 2.197 | -0.405 | 1 | 2 | -1.2 | -0.6 | 0.5 | -0.284 | -1.269 | 1.733
7 | 0.620 | 0.000 | 1.735 | 0.707 | 0.707 | -0.15 | -0.3 | 0.5 | -0.423 | 0.794 | -0.993
8 | 0.702 | 0.000 | 1.735 | 0.707 | 0.707 | -0.30 | -0.6 | 0.5 | -0.461 | 0.795 | -0.996
9 | 0.846 | 1.386 | 1.386 | 0.707 | 0.707 | -0.15 | -0.3 | 0.5 | -1.003 | -0.698 | -0.697
10 | 0.864 | 1.386 | 1.386 | 0.707 | 0.707 | -0.30 | -0.6 | 0.5 | -1.032 | -0.701 | -0.698
11 | 0.877 | 2.197 | -0.405 | 0.707 | 0.707 | -0.15 | -0.3 | 0.5 | -0.457 | -1.362 | 1.342
12 | 0.835 | 2.197 | -0.405 | 0.707 | 0.707 | -0.30 | -0.6 | 0.5 | -0.492 | -1.362 | 1.335
Table S3:  Summary of SAUC estimates when $c_{1}=c_{2}$ in the true
publication mechanism

|  |  | $S=25$ | $S=50$ | $S=200$
---|---|---|---|---|---
No. |  | SAUC | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR
7 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.620 | 0.644 (0.496, 0.734) | 99.5 | 0.619 (0.514, 0.694) | 99.4 | 0.623 (0.573, 0.665) | 99.7
| Proposed $(c_{1}=c_{2})$ |  | 0.641 (0.492, 0.733) | 99.7 | 0.627 (0.524, 0.697) | 99.6 | 0.628 (0.580, 0.669) | 99.7
| Proposed $(c_{1}=1)$ |  | 0.695 (0.595, 0.756) | 87.5 | 0.684 (0.626, 0.732) | 78.8 | 0.690 (0.662, 0.712) | 36.4
| ReistmaO |  | 0.697 (0.611, 0.760) | 100.0 | 0.691 (0.640, 0.737) | 100.0 | 0.695 (0.667, 0.716) | 100.0
| ReistmaP |  | 0.630 (0.537, 0.701) | 100.0 | 0.618 (0.561, 0.668) | 100.0 | 0.621 (0.594, 0.649) | 100.0
8 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.702 | 0.711 (0.627, 0.769) | 99.2 | 0.710 (0.656, 0.751) | 99.5 | 0.703 (0.680, 0.727) | 99.4
| Proposed $(c_{1}=c_{2})$ |  | 0.714 (0.630, 0.770) | 99.6 | 0.709 (0.651, 0.753) | 99.6 | 0.706 (0.679, 0.726) | 99.9
| Proposed $(c_{1}=1)$ |  | 0.737 (0.683, 0.780) | 87.6 | 0.737 (0.707, 0.769) | 74.5 | 0.736 (0.719, 0.749) | 35.9
| ReistmaO |  | 0.741 (0.686, 0.782) | 99.9 | 0.742 (0.710, 0.772) | 100.0 | 0.740 (0.722, 0.753) | 100.0
| ReistmaP |  | 0.708 (0.651, 0.751) | 100.0 | 0.705 (0.669, 0.735) | 100.0 | 0.701 (0.685, 0.716) | 100.0
9 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.846 | 0.847 (0.804, 0.873) | 99.6 | 0.849 (0.817, 0.868) | 98.9 | 0.847 (0.835, 0.859) | 99.6
| Proposed $(c_{1}=c_{2})$ |  | 0.845 (0.796, 0.873) | 99.1 | 0.847 (0.813, 0.869) | 99.1 | 0.847 (0.834, 0.858) | 99.1
| Proposed $(c_{1}=1)$ |  | 0.852 (0.815, 0.876) | 99.1 | 0.855 (0.831, 0.873) | 98.9 | 0.859 (0.849, 0.867) | 96.7
| ReistmaO |  | 0.865 (0.833, 0.884) | 100.0 | 0.868 (0.848, 0.881) | 99.9 | 0.870 (0.861, 0.876) | 100.0
| ReistmaP |  | 0.844 (0.810, 0.864) | 100.0 | 0.844 (0.823, 0.860) | 100.0 | 0.846 (0.836, 0.854) | 100.0
10 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.864 | 0.863 (0.841, 0.877) | 99.2 | 0.865 (0.851, 0.875) | 98.3 | 0.865 (0.857, 0.870) | 99.4
| Proposed $(c_{1}=c_{2})$ |  | 0.863 (0.841, 0.878) | 98.9 | 0.865 (0.850, 0.875) | 99.3 | 0.864 (0.857, 0.870) | 99.5
| Proposed $(c_{1}=1)$ |  | 0.866 (0.846, 0.879) | 99.2 | 0.868 (0.856, 0.877) | 99.1 | 0.869 (0.864, 0.875) | 96.2
| ReistmaO |  | 0.873 (0.856, 0.885) | 100.0 | 0.876 (0.866, 0.884) | 100.0 | 0.877 (0.872, 0.881) | 100.0
| ReistmaP |  | 0.861 (0.845, 0.874) | 100.0 | 0.863 (0.853, 0.872) | 100.0 | 0.864 (0.859, 0.868) | 100.0
11 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.877 | 0.869 (0.830, 0.894) | 99.0 | 0.877 (0.853, 0.893) | 99.5 | 0.878 (0.866, 0.889) | 99.6
| Proposed $(c_{1}=c_{2})$ |  | 0.868 (0.831, 0.893) | 99.3 | 0.876 (0.853, 0.892) | 99.5 | 0.877 (0.864, 0.887) | 99.5
| Proposed $(c_{1}=1)$ |  | 0.858 (0.813, 0.890) | 98.9 | 0.867 (0.837, 0.887) | 99.5 | 0.867 (0.853, 0.880) | 99.6
| ReistmaO |  | 0.873 (0.835, 0.900) | 100.0 | 0.880 (0.856, 0.897) | 100.0 | 0.881 (0.869, 0.891) | 100.0
| ReistmaP |  | 0.870 (0.836, 0.893) | 100.0 | 0.877 (0.856, 0.892) | 100.0 | 0.877 (0.866, 0.886) | 100.0
12 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.835 | 0.833 (0.788, 0.869) | 99.2 | 0.835 (0.805, 0.864) | 99.0 | 0.838 (0.823, 0.853) | 99.0
| Proposed $(c_{1}=c_{2})$ |  | 0.829 (0.785, 0.864) | 99.7 | 0.834 (0.803, 0.860) | 99.8 | 0.836 (0.822, 0.849) | 99.6
| Proposed $(c_{1}=1)$ |  | 0.818 (0.775, 0.856) | 98.9 | 0.823 (0.794, 0.851) | 98.9 | 0.829 (0.815, 0.842) | 99.4
| ReistmaO |  | 0.832 (0.792, 0.867) | 100.0 | 0.838 (0.811, 0.862) | 100.0 | 0.843 (0.830, 0.855) | 100.0
| ReistmaP |  | 0.826 (0.786, 0.860) | 100.0 | 0.832 (0.805, 0.855) | 99.9 | 0.836 (0.822, 0.846) | 100.0

  * •

Proposed $(\hat{c}_{1},\hat{c}_{2})$ is the proposed model estimating
$(c_{1},c_{2})$; Proposed $(c_{1}=c_{2})$ is the proposed model correctly
specifying that $c_{1}=c_{2}$; Proposed $(c_{1}=1)$ is the proposed model
misspecifying that $(c_{1},c_{2})=(1,0)$; ReitsmaO is Reitsma model based on
the selected studies; and ReitsmaP is Reitsma model based on the population
studies.

Table S4:  Summary of SAUC estimates when $(c_{1},c_{2})=(1,0)$ in the true
publication mechanism

|  |  | $S=25$ | $S=50$ | $S=200$
---|---|---|---|---|---
No. |  | SAUC | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR
1 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.620 | 0.627 (0.508, 0.719) | 99.9 | 0.624 (0.543, 0.687) | 99.9 | 0.640 (0.605, 0.675) | 99.7
| Proposed $(c_{1}=1)$ |  | 0.642 (0.520, 0.732) | 99.8 | 0.627 (0.541, 0.693) | 99.4 | 0.626 (0.586, 0.660) | 99.9
| Proposed $(c_{1}=c_{2})$ |  | 0.629 (0.512, 0.721) | 99.7 | 0.634 (0.561, 0.691) | 99.7 | 0.656 (0.628, 0.684) | 99.8
| ReistmaO |  | 0.665 (0.560, 0.738) | 100.0 | 0.658 (0.598, 0.707) | 100.0 | 0.663 (0.637, 0.688) | 100.0
| ReistmaP |  | 0.630 (0.537, 0.701) | 100.0 | 0.618 (0.561, 0.668) | 100.0 | 0.621 (0.594, 0.649) | 100.0
2 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.702 | 0.704 (0.620, 0.758) | 99.6 | 0.705 (0.653, 0.743) | 99.5 | 0.708 (0.687, 0.726) | 99.7
| Proposed $(c_{1}=1)$ |  | 0.721 (0.643, 0.766) | 99.4 | 0.711 (0.662, 0.747) | 99.8 | 0.703 (0.683, 0.723) | 99.3
| Proposed $(c_{1}=c_{2})$ |  | 0.706 (0.614, 0.761) | 99.7 | 0.707 (0.657, 0.745) | 99.6 | 0.712 (0.693, 0.729) | 99.8
| ReistmaO |  | 0.725 (0.657, 0.770) | 100.0 | 0.721 (0.683, 0.754) | 100.0 | 0.718 (0.700, 0.733) | 100.0
| ReistmaP |  | 0.708 (0.651, 0.751) | 100.0 | 0.705 (0.669, 0.735) | 100.0 | 0.701 (0.685, 0.716) | 100.0
3 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.846 | 0.838 (0.788, 0.868) | 99.0 | 0.843 (0.808, 0.865) | 99.1 | 0.845 (0.831, 0.855) | 99.4
| Proposed $(c_{1}=1)$ |  | 0.841 (0.796, 0.871) | 99.4 | 0.846 (0.816, 0.864) | 99.6 | 0.847 (0.834, 0.856) | 99.6
| Proposed $(c_{1}=c_{2})$ |  | 0.840 (0.788, 0.870) | 99.2 | 0.846 (0.808, 0.867) | 99.4 | 0.849 (0.835, 0.860) | 99.4
| ReistmaO |  | 0.857 (0.820, 0.882) | 100.0 | 0.861 (0.837, 0.876) | 100.0 | 0.863 (0.853, 0.872) | 100.0
| ReistmaP |  | 0.844 (0.810, 0.864) | 100.0 | 0.844 (0.823, 0.860) | 100.0 | 0.846 (0.836, 0.854) | 100.0
4 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.864 | 0.860 (0.837, 0.876) | 99.1 | 0.862 (0.848, 0.874) | 99.4 | 0.863 (0.858, 0.869) | 99.6
| Proposed $(c_{1}=1)$ |  | 0.860 (0.839, 0.876) | 99.7 | 0.864 (0.850, 0.874) | 99.7 | 0.864 (0.858, 0.869) | 99.7
| Proposed $(c_{1}=c_{2})$ |  | 0.861 (0.837, 0.876) | 99.1 | 0.866 (0.851, 0.877) | 99.3 | 0.867 (0.860, 0.873) | 99.3
| ReistmaO |  | 0.869 (0.850, 0.883) | 100.0 | 0.874 (0.862, 0.883) | 100.0 | 0.875 (0.871, 0.880) | 100.0
| ReistmaP |  | 0.861 (0.845, 0.874) | 100.0 | 0.863 (0.853, 0.872) | 100.0 | 0.864 (0.859, 0.868) | 100.0
5 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.877 | 0.880 (0.840, 0.902) | 99.1 | 0.885 (0.861, 0.901) | 98.6 | 0.882 (0.869, 0.892) | 99.1
| Proposed $(c_{1}=1)$ |  | 0.869 (0.823, 0.896) | 98.8 | 0.878 (0.849, 0.896) | 98.6 | 0.877 (0.864, 0.888) | 99.0
| Proposed $(c_{1}=c_{2})$ |  | 0.881 (0.847, 0.903) | 99.4 | 0.891 (0.868, 0.904) | 99.2 | 0.892 (0.883, 0.900) | 98.8
| ReistmaO |  | 0.884 (0.845, 0.907) | 100.0 | 0.892 (0.867, 0.907) | 100.0 | 0.893 (0.884, 0.902) | 100.0
| ReistmaP |  | 0.870 (0.836, 0.893) | 100.0 | 0.877 (0.856, 0.892) | 100.0 | 0.877 (0.866, 0.886) | 100.0
6 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.835 | 0.839 (0.785, 0.878) | 98.6 | 0.843 (0.809, 0.874) | 98.5 | 0.840 (0.824, 0.855) | 99.5
| Proposed $(c_{1}=1)$ |  | 0.826 (0.775, 0.867) | 98.7 | 0.833 (0.800, 0.861) | 97.9 | 0.836 (0.821, 0.849) | 99.2
| Proposed $(c_{1}=c_{2})$ |  | 0.844 (0.792, 0.881) | 99.4 | 0.852 (0.817, 0.877) | 99.7 | 0.853 (0.838, 0.865) | 99.6
| ReistmaO |  | 0.842 (0.791, 0.879) | 100.0 | 0.849 (0.819, 0.875) | 100.0 | 0.854 (0.841, 0.866) | 100.0
| ReistmaP |  | 0.826 (0.786, 0.860) | 100.0 | 0.832 (0.805, 0.855) | 99.9 | 0.836 (0.822, 0.846) | 100.0

  * •

Proposed $(\hat{c}_{1},\hat{c}_{2})$ is the proposed model estimating
$(c_{1},c_{2})$; Proposed $(c_{1}=1)$ is the proposed model correctly
specifying that $(c_{1},c_{2})=(1,0)$; Proposed $(c_{1}=c_{2})$ is the
proposed model misspecifying that $c_{1}=c_{2}$; ReitsmaO is Reitsma model
based on the selected studies; and ReitsmaP is Reitsma model based on the
population studies.

Table S5:  Summary of SAUC estimates when $(c_{1},c_{2})=(0,1)$ in the true
publication mechanism

|  |  | $S=25$ | $S=50$ | $S=200$
---|---|---|---|---|---
No. |  | SAUC | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR | Median (Q1, Q3) | CR
7 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.620 | 0.559 (0.379, 0.701) | 99.1 | 0.558 (0.435, 0.651) | 99.0 | 0.595 (0.536, 0.640) | 99.9
| Proposed $(c_{1}=0)$ |  | 0.632 (0.511, 0.727) | 99.2 | 0.619 (0.540, 0.686) | 99.5 | 0.624 (0.586, 0.657) | 99.4
| Proposed $(c_{1}=c_{2})$ |  | 0.558 (0.362, 0.696) | 99.7 | 0.536 (0.415, 0.643) | 99.7 | 0.560 (0.496, 0.612) | 99.2
| ReistmaO |  | 0.637 (0.509, 0.734) | 100.0 | 0.626 (0.542, 0.695) | 100.0 | 0.629 (0.590, 0.664) | 100.0
| ReistmaP |  | 0.630 (0.537, 0.701) | 100.0 | 0.618 (0.561, 0.668) | 100.0 | 0.621 (0.594, 0.649) | 100.0
8 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.702 | 0.683 (0.567, 0.755) | 99.3 | 0.683 (0.607, 0.735) | 99.0 | 0.688 (0.663, 0.712) | 99.4
| Proposed $(c_{1}=0)$ |  | 0.709 (0.634, 0.762) | 99.2 | 0.707 (0.658, 0.748) | 99.2 | 0.702 (0.681, 0.721) | 99.6
| Proposed $(c_{1}=c_{2})$ |  | 0.680 (0.535, 0.756) | 99.7 | 0.677 (0.597, 0.736) | 99.3 | 0.674 (0.644, 0.705) | 99.4
| ReistmaO |  | 0.715 (0.640, 0.770) | 100.0 | 0.715 (0.666, 0.756) | 100.0 | 0.709 (0.689, 0.728) | 100.0
| ReistmaP |  | 0.708 (0.651, 0.751) | 100.0 | 0.705 (0.669, 0.735) | 100.0 | 0.701 (0.685, 0.716) | 100.0
9 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.846 | 0.828 (0.747, 0.862) | 99.3 | 0.831 (0.780, 0.860) | 99.5 | 0.839 (0.823, 0.854) | 99.7
| Proposed $(c_{1}=0)$ |  | 0.842 (0.785, 0.868) | 99.4 | 0.843 (0.810, 0.866) | 99.0 | 0.846 (0.831, 0.858) | 99.8
| Proposed $(c_{1}=c_{2})$ |  | 0.824 (0.733, 0.862) | 99.6 | 0.824 (0.770, 0.857) | 99.6 | 0.830 (0.807, 0.846) | 99.8
| ReistmaO |  | 0.845 (0.788, 0.873) | 100.0 | 0.846 (0.811, 0.869) | 99.8 | 0.850 (0.834, 0.861) | 100.0
| ReistmaP |  | 0.844 (0.810, 0.864) | 100.0 | 0.844 (0.823, 0.860) | 100.0 | 0.846 (0.836, 0.854) | 100.0
10 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.864 | 0.854 (0.829, 0.872) | 99.3 | 0.859 (0.841, 0.872) | 99.2 | 0.861 (0.853, 0.868) | 99.5
| Proposed $(c_{1}=0)$ |  | 0.861 (0.837, 0.876) | 99.7 | 0.863 (0.849, 0.874) | 99.5 | 0.864 (0.857, 0.869) | 99.6
| Proposed $(c_{1}=c_{2})$ |  | 0.854 (0.823, 0.872) | 99.0 | 0.857 (0.837, 0.871) | 98.4 | 0.857 (0.849, 0.865) | 99.2
| ReistmaO |  | 0.864 (0.841, 0.879) | 100.0 | 0.867 (0.853, 0.878) | 100.0 | 0.868 (0.861, 0.873) | 100.0
| ReistmaP |  | 0.861 (0.845, 0.874) | 100.0 | 0.863 (0.853, 0.872) | 100.0 | 0.864 (0.859, 0.868) | 100.0
11 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.877 | 0.839 (0.763, 0.877) | 99.8 | 0.861 (0.825, 0.883) | 99.7 | 0.871 (0.856, 0.883) | 99.3
| Proposed $(c_{1}=0)$ |  | 0.860 (0.812, 0.888) | 99.7 | 0.872 (0.845, 0.890) | 99.5 | 0.877 (0.864, 0.886) | 99.3
| Proposed $(c_{1}=c_{2})$ |  | 0.840 (0.772, 0.876) | 99.9 | 0.863 (0.830, 0.884) | 99.8 | 0.875 (0.860, 0.885) | 100.0
| ReistmaO |  | 0.846 (0.774, 0.882) | 100.0 | 0.869 (0.835, 0.889) | 100.0 | 0.876 (0.861, 0.886) | 100.0
| ReistmaP |  | 0.870 (0.836, 0.893) | 100.0 | 0.877 (0.856, 0.892) | 100.0 | 0.877 (0.866, 0.886) | 100.0
12 | Proposed $(\hat{c}_{1},\hat{c}_{2})$ | 0.835 | 0.807 (0.740, 0.850) | 99.2 | 0.819 (0.778, 0.852) | 99.7 | 0.825 (0.807, 0.842) | 99.4
| Proposed $(c_{1}=0)$ |  | 0.823 (0.772, 0.860) | 99.7 | 0.829 (0.795, 0.858) | 99.5 | 0.835 (0.819, 0.849) | 99.4
| Proposed $(c_{1}=c_{2})$ |  | 0.809 (0.746, 0.851) | 99.9 | 0.821 (0.780, 0.853) | 99.8 | 0.829 (0.811, 0.845) | 99.7
| ReistmaO |  | 0.809 (0.751, 0.853) | 100.0 | 0.821 (0.782, 0.854) | 100.0 | 0.830 (0.813, 0.845) | 100.0
| ReistmaP |  | 0.826 (0.786, 0.860) | 100.0 | 0.832 (0.805, 0.855) | 99.9 | 0.836 (0.822, 0.846) | 100.0

  * •

Proposed $(\hat{c}_{1},\hat{c}_{2})$ is the proposed model estimating
$(c_{1},c_{2})$; Proposed $(c_{1}=0)$ is the proposed model correctly
specifying that $(c_{1},c_{2})=(0,1)$; Proposed $(c_{1}=c_{2})$ is the
proposed model misspecifying that $c_{1}=c_{2}$; ReitsmaO is Reitsma model
based on the selected studies; and ReitsmaP is Reitsma model based on the
population studies.