# Dual Control of Testing Errors in High-Dimensional Data Analysis

X. Jessie Jeng and Yifei Hu
Department of Statistics, North Carolina State University  X. Jessie Jeng and
Yifei Hu
Department of Statistics, North Carolina State University

# High-Dimensional Inference for Unidentifiable Signals with False Negative
Control ††thanks: Address for correspondence: X. Jessie Jeng, Department of
Statistics, North Carolina State University, SAS Hall, 2311 Stinson Dr.,
Raleigh, NC 27695-8203, USA. E-mail<EMAIL_ADDRESS>
X. Jessie Jeng and Yifei Hu
Department of Statistics, North Carolina State University  X. Jessie Jeng and
Yifei Hu
Department of Statistics, North Carolina State University

###### Abstract

False negative errors are of major concern in applications where missing a
high proportion of true signals may cause serious consequences. False negative
control, however, raises a bottleneck challenge in high-dimensional inference
when signals are not identifiable at individual levels. We develop a new
analytic framework to regulate false negative errors under measures tailored
towards modern applications with high-dimensional data. A new method is
proposed in realistic settings with arbitrary covariance dependence between
variables. We explicate the joint effects of covariance dependence and signal
sparsity on the new method and interpret the results using a phase diagram. It
shows that signals that are not individually identifiable can be effectively
retained by the proposed method without incurring excessive false positives.
Simulation studies are conducted to compare the new method with several
existing methods. The new method outperforms the others in adapting to a user-
specified false negative control level. We apply the new method to analyze an
fMRI dataset to locate voxels that are functionally relevant to saccadic eye
movements. The new method exhibits a nice balance in retaining signal voxels
and avoiding excessive noise voxels.

Keywords: False negative proportion; Signal proportion estimation; Sparse
signal; Unnecessary false positives

##  1 Introduction

False negative errors are of major concern in applications where missing true
signals may cause serious consequences. For example, failures to detect active
cases of highly contagious diseases may cause delayed treatment and large-
scale epidemic spreading. More examples in medical, psychological, economic,
and legal studies can be found in Petticrew et al. (2000). In modern
applications with high-dimensional data, false negative control is a pressing
challenge as finding true signals are often as difficult as finding needles in
a haystack.

In the past decade, significant contributions have been made towards two
foundational challenges in high-dimensional sparse inference. One is the
detection of mixture models, which addresses the problem of “detecting” the
existence of sparse signals without specifying their exact locations. The
second is the identification of sparse signals, which considers the problem of
“identifying” signals at individual levels. Figure 1 modified from Donoho and
Jin (2015) illustrates the theoretical demarcation on the difficulty levels of
these two problems in the setting where all variables are independent and
signal variables are relatively sparse compared to the number of noise
variables. Given a sparsity level, signal intensity needs to be large enough
(above the undetectable region) for the signals to be detectable by a global
testing procedure. For the signals to be identifiable at individual levels
with multiple testing, signal intensity needs to be even larger (entering the
identifiable region). It can be seen that signals in the middle region of
Figure 1 can only be detected for their existence by global testing but are
not identifiable at individual levels using multiple testing. Such signals
commonly exist in high-dimensional data analysis and play vital roles in
modern applications. Providing the capacity for retaining such signals can
alleviate critical issues such as missing important signals in underpowered
studies and lack of replicability across multiple samples. Despite its
importance, how to effectively retain detectable yet unidentifiable signals
raises a bottleneck challenge. New developments are needed to expand the
frontier of high-dimensional sparse inference and bridge the methodological
gap.

Figure 1: Phase diagram for signal detection and identification under
independence. Signals in the middle region are only detectable for their
existence but not identifiable at individual levels.

In this paper, we propose a novel analytic framework to regulate false
negative errors under measures tailored towards modern applications with high-
dimensional data. More specifically, the method controls the false negative
proportion (FNP $=$ number of false negatives/number of alternative cases) at
a user-specified level and, at the same time, regulates the amount of
unnecessary false positives to achieve the FNP level. We have found that, in
real data analyses, researchers often have different tolerance levels for
false negative errors depending on their scientific inquiries. Our procedure
estimates and controls FNP at a user-specified level under conditions weak
enough to allow for unidentifiable signals. Moreover, its adaptivity to a
given FNP level, such as $0.1$, allows the method to exclude a certain amount
of the weakest signals to reduce possibly a large number of false positives.

The new method is developed in the realistic setting with arbitrary covariance
dependence between variables. We propose to calibrate the overall dependence
strength through a parameter whose scale is compatible with the existing
parameter space in high-dimensional sparse inference. Utilizing the new
calibration, we are able to explicate the joint effect of covariance
dependence and signal sparsity on the new method and interpret the results
using a new phase diagram, which extends the phase diagram in Figure 1 to
include dependence effect. Our theoretical findings are supported in
simulation examples where the data are generated under various dependence
structures.

Extensive simulation studies are conducted to compare the new method with
existing methods in both one-dimensional and two-dimensional examples. The new
method outperforms the others in adapting to a user-specified FNP control
level. We apply the new method to analyze fMRI data from the saccade
experiment in the Individual Brain Charting project (Pinho et al., 2018).
Compared to the other methods, results of the new method exhibit a nice
balance in identifying brain regions that are functionally relevant to
saccadic eye movement and avoiding the scattered or isolated voxels that are
not functionally relevant to saccades.

The rest of the paper is organized as follows. Section 2 introduces the new
method and justify its FNP control property in theory. Section 3 provides
several illustrative examples under different dependence structures. The
proposed method with the estimated number of signals is studied in Section 4,
and additional simulation examples are presented in Section 5. Section 6
applies the new method to fMRI data analysis. Section 7 provides conclusion
and further discussion. All the technical proofs are presented in the
Appendix.

##  2 Method and Theory

We consider a model with continuous null distribution $F_{0}$. Suppose that
the test statistics

$X_{j}\sim F_{0}\cdot 1\\{j\in I_{0}\\}+F_{j}\cdot 1\\{j\in I_{1}\\},\qquad
j=1,...,p,$

where $I_{0}$ is the set of indices for noise variables, $I_{1}$ is the set of
indices for signal variables, and $F_{j}$ are the signal distributions. All
$I_{0}$, $I_{1}$, and $F_{j}$ are unknown. One can perform inverse normal
transformation as $Z_{j}=\Phi^{-1}(F_{0}(X_{j}))$, where $\Phi^{-1}$ is the
inverse of the standard normal cumulative distribution. Then, we have

$Z_{j}\sim\Phi\cdot 1\\{j\in I_{0}\\}+G_{j}\cdot 1\\{j\in I_{1}\\},\qquad
j=1,...,p,$ (1)

where $G_{j}$ are the unknown signal distributions after the transformation.
For presentation simplicity, we assume that $G_{j}(t)<\Phi(t)$ for all
$t\in\mathbb{R}$. i.e., signal variables tend to show larger values than noise
variables. This assumption can be easily generalized to signals with two-sided
effects.

Consider a selection rule with threshold $t$. Define the numbers of selected
cases, false positives, and false negatives as

$\text{R}(t)=\sum_{j=1}^{p}1_{\\{z_{j}>t\\}},\quad\text{FP}(t)=\sum_{j\in
I_{0}}1_{\\{z_{j}>t\\}},\quad\text{FN}(t)=\sum_{j\in I_{1}}1_{\\{z_{j}\leq
t\\}}.$

Note that R$(t)$ can be directly observed from the data. FN$(t)$ and FP$(t)$
are unknown because $I_{0}$ and $I_{1}$ are unknown. A generalization to two-
sided signal effects can be accommodated by replacing $z_{j}$ with $|z_{j}|$
and only allowing $t>0$. Next, define FNP with respect to $t$ as

$\text{FNP}(t)=\text{FN}(t)/s,$ (2)

where $s$ denotes the number of signal variables, i.e. $s=|I_{1}|$. FNP$(t)$
may be regarded as the empirical type II error that is non-decreasing with
respect to $t$.

###  2.1 Estimation of False Negative Proportion

The new method is based on consistent estimation of FNP. By the fact that
$s=\text{FN}(t)+\text{TP}(t)$ and R$(t)=\text{FP}(t)+\text{TP}(t)$, we have

$\text{FNP}(t)=\text{FN}(t)/s=1-(\text{R}(t)-\text{FP}(t))/s,$ (3)

where R$(t)$ is directly observable from the data. Because the noise
distribution of $Z_{j}$ is $N(0,1)$ and there are $p-s$ noise variables,
FP$(t)$ can be approximated by its mean value
$E(\text{FP}(t))=(p-s)\bar{\Phi}(t)$, where $\bar{\Phi}(t)=1-\Phi(t)$. For
illustration purpose, we first assume that the true value of $s$ is known and
construct an estimator for FNP$(t)$ as

$\widehat{\text{FNP}}(t)=\max\\{1-\text{R}(t)/s+(p-s)\bar{\Phi}(t)/s,~{}0\\}.$
(4)

$\widehat{\text{FNP}}(t)$ with an estimated $s$ will be discussed in Section
4. Note that $\widehat{\text{FNP}}(t)$ is not the mean value of
$\text{FNP}(t)$, and its construction does not require information of the
signal distribution $G$. If two-sided signal effects are under consideration,
we can simply modify $\widehat{\text{FNP}}(t)$ by replacing $(p-s)$ with
$2(p-s)$.

Next, we study the estimation consistency of $\widehat{\text{FNP}}(t)$. It is
a challenging problem because the denominator $s$ is often much smaller than
$p$, which can explode the scale of the ratio and hence the approximation
error. This causes fundamental differences in analyzing the consistency of FNP
estimation and FDP estimation. Specifically, we adopt the existing calibration
in sparse inference for $s$ as

$s=s_{p}=p^{1-\gamma},\qquad\gamma\in(0,1].$ (5)

It can be seen that when $\gamma>0$, $s$ is of a smaller order than the data
dimension $p$, and when $\gamma=1$, the number of signals does not change with
$p$.

We are particularly interested in FNP estimation under general covariance
dependence. To this end, we assume that

$(Z_{1},\ldots,Z_{p})\sim N_{p}(\mu,\mathbf{\Sigma}),$ (6)

where $\mu$ is a p-dimensional vector with $\mu_{j}=A_{j}\cdot 1\\{j\in
I_{1}\\}$, $A_{j}>0$, and $\mathbf{\Sigma}$ is an arbitrary correlation
matrix. Define

$\bar{\rho}=\lVert\mathbf{\Sigma}\rVert_{1}/p^{2},$

where $\lVert\mathbf{\Sigma}\rVert_{1}=\sum_{ij}|\sigma_{ij}|$. In high-
dimensional data analysis with large $p$, $\bar{\rho}$ is often very close to
zero because not every variable is correlated to all the other variables. For
example, $\bar{\rho}$ of the $\mathbf{\Sigma}_{p\times p}$ from an
autoregressive model has the order of $p^{-1}$. In order to calibrate the
dependence effect in a scale that is compatible with the existing parameter
space in sparse inference, we introduce parameter $\eta$ through the following
re-parameterization:

$\bar{\rho}=\bar{\rho}_{p}=p^{-\eta},\qquad\eta\in[0,1].$ (7)

The parameter $\eta$ is in a constant scale and decreases with $\bar{\rho}$.
$\eta=0$ corresponds to the extremely dependent case where every variable is
correlated to all the other variables, and, at the opposite end, $\eta=1$
corresponds to the independent case.

With $\gamma$ and $\eta$ representing signal sparsity and overall strength of
covariance dependence, respectively, we discover a lower bound of $t$ for the
estimation consistency of $\widehat{\text{FNP}}(t)$, which is presented as
follows:

$\mu_{min}=\min\\{\mu_{1},~{}\mu_{2}\\},$ (8)

where

$\mu_{1}=\sqrt{2\gamma\log{p}}\qquad\text{and}\qquad\mu_{2}=\sqrt{(4\gamma-2\eta)_{+}\log{p}+4\log\log
p}.$

The lower bound $\mu_{min}$ takes the value of either $\mu_{1}$ or $\mu_{2}$,
depending on whichever is lower. It can be seen that $\mu_{2}<\mu_{1}$ when
$\eta$ is large enough or, in other words, when covariance dependence is weak
enough. As dependence gets stronger and $\eta$ gets smaller, $\mu_{1}<\mu_{2}$
and the lower bound equals to $\mu_{1}$ and stops to change with $\eta$. The
term $\log\log p$ is a technical term for asymptotic analysis. We have the
following theorem to summarize the estimation result.

######  Theorem 2.1.

Consider model (6). When $t\geq\mu_{min}$, where $\mu_{min}$ is defined in
(8), $\widehat{\text{FNP}}(t)$ constructed in (4) consistently estimates
$\text{FNP}(t)$, i.e.

$|\widehat{\text{FNP}}(t)-\text{FNP}(t)|=o_{P}(1).$ (9)

One can see that the consistency of $\widehat{\text{FNP}}(t)$ is achieved with
$t$ increasing with $p$ in a $\sqrt{\log p}$ scale. This is substantially
different from the analysis in FDR studies, where the consistency of FDR/FDP
estimation is studied with $t$ as a constant. The result explicates the
interesting relationship between covariance dependence ($\eta$) and signal
sparsity ($\gamma$) in determining the consistency of
$\widehat{\text{FNP}}(t)$ for varying $t$.

###  2.2 Controlling FNP at a User-Specified Level

In real studies, researchers may have different tolerance levels for false
negative errors. The proposed method allows a user-specified control level on
FNP and efficiently selects a subset of candidates to achieve the control
level. Moreover, its adaptivity to a pre-specified low FNP level, such as
$0.1$, allows the method to exclude a small percent of the weakest signals to
reduce possibly a large number of false positives.

Our method is constructed based on the estimator $\widehat{\text{FNP}}$
developed in Section 2.1. Given a user-specified constant $\beta(>0)$, we
determine the selection threshold as

$\hat{t}(\beta)=\sup{\\{t:\widehat{\text{FNP}}(t)<\beta\\}},$ (10)

and select all the candidates with $z_{j}>\hat{t}(\beta)$. We refer to this
procedure as a dual control method. It can been seen that when all variables
are ranked such that $z_{(1)}\geq z_{(2)}\geq\ldots\geq z_{(p)}$, the dual
control method selects the smallest subsets from the top whose estimated FNP
is less than $\beta$. If two-sided signal effects are considered, all
candidates with $|z_{j}|>\hat{t}(\beta)$ will be selected.

The following theorem presents the dual control property of the proposed
method which is to regulate both the FNP at a pre-specified $\beta$ level and
the associated false positives.

######  Theorem 2.2.

Consider model (6) and a user-specified control level $\beta$ of FNP. Assume
$A_{j}-\mu_{min}\to\infty,\forall j\in I_{1}$, where $\mu_{min}$ is defined in
(8), then the dual control method with threshold $\hat{t}(\beta)$ efficiently
controls the true FNP at the level of $\beta$, i.e.,

$P(\text{FNP}(\hat{t}(\beta))\leq\beta)\rightarrow 1,$ (11)

and, for any smaller set of variables associated with threshold
$\tilde{t}>\hat{t}(\beta)$,

$P\\{\text{FNP}(\tilde{t})>\beta-\delta)\rightarrow 1$ (12)

for arbitrarily small constant $\delta>0$.

Theorem 2.2 shows efficiency of the dual control method in selecting the
smallest subset of variables to achieve FNP control at the $\beta$ level. In
other words, the method does not incur unnecessary false positives to achieve
the FNP control level. The condition on signal intensity ($A_{j}$) specifies
the joint effect of covariance dependence and signal sparsity on the dual
control method. Recall $\mu_{min}=\min\\{\mu_{1},\mu_{2}\\}$, where $\mu_{1}$
only relies on signal sparsity ($\gamma$), and $\mu_{2}$ replies on both
signal sparsity ($\gamma$) and covariance dependence ($\eta$). The condition
says that the difficulty of dual control increases as dependence gets
stronger, but only to a certain degree that depends on the signal sparsity.

The parameterization of covariance dependence through $\eta$ allows us to
present the signal intensity condition using a phase diagram similar to Figure
1. More specifically, let $\min_{j\in I_{1}}A_{j}=\sqrt{2r\log p}$, $r>0$. The
condition on $A_{j}$ in Theorem 2.2 can be transformed to
$r>\min\\{\gamma,~{}2\gamma-\eta\\}$ and demonstrated in a two-dimensional
phase diagram with $\gamma$ as the x axis and $r$ as the y axis. The lower
bound $\min\\{\gamma,~{}2\gamma-\eta\\}$ is illustrated in Figure 2 by the red
solid line that moves with the dependence parameter $\eta$. Signals in the
area above the red solid line can be efficiently retained by the dual control
method at a pre-specified level. Recall Figure 1, which implies that under
independence, signals in the region between the two dashed lines are only
detectable for their existence but not identifiable at individual levels. The
special case of our results under independence ($\eta=1$) shows that many of
the unidentifiable signals can be efficiently retained by the new method.

Figure 2: Dual control region under covariance dependence. Signals in the area
above a solid red line can be retained by the dual control method at a pre-
specified FNP level. The area increases with $\eta$ as the dependence get
weaker.

##  3 Examples

In this section, we present simulation examples with different dependence
structures to illustrate the performance of the new method. We first evaluate
the estimator $\widehat{\text{FNP}}(t)$ from (4), then demonstrate finite
sample results of the dual control method. These examples help understand the
foundation of the new method. Additional simulation studies are provided in
Section 5.

We generate a series of test statistics

$(Z_{1},\ldots,Z_{p})^{T}\sim
N((\mu_{1},\ldots,\mu_{p})^{T},\mathbf{\Sigma}),$ (13)

where $p=2000$, $\mu_{j}=A\cdot 1\\{j\in I_{1}\\}$, and $I_{1}$ is a set of
indices randomly sampled from $\\{1,...,p\\}$ with cardinality
$s=|I_{1}|=p^{1-\gamma}$. We set $\gamma=0.3$, which corresponds to $s=205$,
i.e. there are 205 signal variables with elevated mean values randomly located
among 1795 noise variables. The signal intensity varies with $A=3$ and 2. . We
consider three different dependence structures:

  * •

Model 1 [Autoregressive]: $\mathbf{\Sigma}=(\sigma^{(1)}_{ij})$, where
$\sigma^{(1)}_{ij}=\lambda^{|i-j|}$ for $1\leq i,j\leq p$.

  * •

Model 2 [Block dependence]:
$\mathbf{\Sigma}=\mathbf{I}_{p/k}\otimes\mathbf{D}$, where $\mathbf{D}$ is a
$k\times k$ matrix with diagonal entries 1 and off-diagonal entries $r$.

  * •

Model 3 [Factor model]: $\mathbf{\Sigma}=(\sigma^{(3)}_{ij})$, where
$\sigma^{(3)}_{ij}=V_{ij}/\sqrt{V_{ii}V_{jj}}$ for $1\leq i,j\leq p$,
$\mathbf{V}=\tau\mathbf{h}\mathbf{h}^{T}+\mathbf{I}_{p}$ with $\tau\in(0,1)$
and $\mathbf{h}\sim N(\mathbf{0},\mathbf{I}_{p})$.

In this section, we set $\lambda=0.2,k=40,r=0.5$, and $\tau=0.5$, so that the
dependence parameter $\eta$ decreases from 0.95 in Model 1 to 0.57 in Model 2
to 0.23 in Model 3. Correspondingly, dependence among test statistics
increases from very weak in Model 1 to moderately strong in Model 2 to very
strong in Model 3.

It has been derived in Section 2.1 that $\mu_{min}$, as the boundary value for
estimation consistency, is the minimum of $\mu_{1}$ and $\mu_{2}$, where
$\mu_{1}$ depends on signal sparsity through $\gamma$ and $\mu_{2}$ depends on
both signal sparsity and dependence through $\gamma$ and $\eta$. In these
examples, $(\mu_{1}=2.14,\mu_{2}=1.69)$ for Model 1,
$(\mu_{1}=2.14,\mu_{2}=2.92)$ for Model 2, and $(\mu_{1}=2.14,\mu_{2}=3.69)$
for Model 3. Consequently, $\mu_{min}=1.69$ for Model 1 but remains the same
at $2.14$ for Model 2 and 3. These values of $\mu_{min}$ are illustrated as
the solid vertical lines in Figure 3. The dotted vertical lines represent
$\mu_{1}$ or $\mu_{2}$ whichever is larger. The dotted curves represent the
absolute difference between the estimated $\widehat{\text{FNP}}(t)$ and the
true FNP$(t)$ from 100 replications. It can be seen that the estimation
accuracy of $\widehat{\text{FNP}}(t)$ increases with $t$, and the majority of
the replicated differences are close to $0$ after passing $\mu_{min}$, which
support $\mu_{min}$ as the boundary value for estimation consistency under
very different dependence structures.

Figure 3: Estimation accuracy of $\widehat{\text{FNP}}(t)$ based on 100
replications. Plots from left to right are generated under Model 1 - 3. The
solid vertical lines represent the boundary value
$\mu_{min}=\min\\{\mu_{1},\mu_{2}\\}$. Model 1 has very weak dependence with
$\eta=0.95$ and $\mu_{min}=\mu_{2}=1.69$. Model 2 has moderately strong
dependence with $\eta=0.57$ and $\mu_{min}=\mu_{1}=2.14$. Model 3 has very
strong dependence with $\eta=0.23$ and $\mu_{min}=\mu_{1}=2.14$.

Next, we demonstrate the finite sample performance of the dual control method,
which is denoted as DCOE (acronym for Dual Control Of Errors) in Table 1. We
present the mean values and standard deviations of the realized FNP of DCOE
with different $\beta$ levels from 100 replications. The results of DCOE are
compared with those of the classical BH-FDR procedure (Benjamini and Hochberg,
1995) with different nominal levels. For illustrative purposes, we also
present the mean values of the realized false discovery proportion (FDP) of
the two methods.

Table 1: Mean values and standard deviations (in brackets) of the realized FNP and FDP of the proposed DCOE method with different nominal levels and the existing BH-FDR procedure with different nominal levels.  |  |  | FNP | FDP
---|---|---|---|---
A=3 | Autoregressive | DCOE(${\bf\beta=0.2}$) |  0.198 (0.023) | 0.149 (0.035)
|  | DCOE($\beta={\bf 0.1}$) |  0.101 (0.037) | 0.307 (0.084)
|  | BH-FDR($\alpha={\bf 0.05}$) | 0.378 (0.040) |  0.044 (0.018)
|  | BH-FDR($\alpha={\bf 0.2}$) | 0.166 (0.028) |  0.183 (0.028)
| Block dependence | DCOE($\beta={\bf 0.2}$) |  0.170 (0.086) | 0.259 (0.247)
|  | DCOE($\beta={\bf 0.1}$) |  0.089 (0.074) | 0.444 (0.268)
|  | BH-FDR($\alpha={\bf 0.05}$) | 0.387 (0.071) |  0.047 (0.036)
|  | BH-FDR($\alpha={\bf 0.2}$) | 0.169 (0.049) |  0.180 (0.070)
| Factor model | DCOE($\beta={\bf 0.2}$) |  0.160 (0.099) | 0.262 (0.215)
|  | DCOE($\beta={\bf 0.1}$) |  0.084 (0.104) | 0.533 (0.227)
|  | BH-FDR($\alpha={\bf 0.05}$) | 0.400 (0.043) |  0.041 (0.049)
|  | BH-FDR($\alpha={\bf 0.2}$) | 0.176 (0.049) |  0.170 (0.113)
A=2 | Autoregressive | DCOE(${\bf\beta=0.2}$) |  0.201 (0.069) | 0.576 (0.073)
|  | DCOE($\beta={\bf 0.1}$) |  0.114 (0.069) | 0.688 (0.085)
|  | BH-FDR($\alpha={\bf 0.05}$) | 0.889 (0.035) |  0.040 (0.043)
|  | BH-FDR($\alpha={\bf 0.2}$) | 0.630 (0.050) |  0.181 (0.044)
| Block dependence | DCOE($\beta={\bf 0.2}$) |  0.193 (0.132) | 0.607 (0.161)
|  | DCOE($\beta={\bf 0.1}$) |  0.132 (0.116) | 0.685 (0.152)
|  | BH-FDR($\alpha={\bf 0.05}$) | 0.894 (0.058) |  0.055 (0.070)
|  | BH-FDR($\alpha={\bf 0.2}$) | 0.647 (0.097) |  0.182 (0.093)
| Factor model | DCOE($\beta={\bf 0.2}$) |  0.188 (0.169) | 0.642 (0.107)
|  | DCOE($\beta={\bf 0.1}$) |  0.156 (0.159) | 0.693 (0.098)
|  | BH-FDR($\alpha={\bf 0.05}$) | 0.886 (0.088) |  0.032 (0.061)
|  | BH-FDR($\alpha={\bf 0.2}$) | 0.659 (0.088) |  0.160 (0.123)

It can be seen that for Model 1 [Autoregressive], the mean values of FNP for
DCOE with $\beta=0.2$ and $0.1$ are fairly close to their corresponding
nominal levels. Specifically, when $A=3$ we have 0.198 and 0.101 respectively,
and when $A=2$, we have 0.201 and 0.114 respectively.

In the more challenging cases generated by Model 2 [Block dependence] and 3
[Factor model], the boundary value $\mu_{min}$ required for estimation
consistency increases from $1.69$ to $2.14$; and the differences between
realized FNPs of DCOE and the nominal levels increase slightly. These
observations agree with the theoretical results in Sections 2.2.

For illustrative purposes, we also present the results of the classical BH-FDR
procedure. The nominal levels of BH-FDR are set at $\alpha=0.05$ and $0.2$.
Table 1 shows that the mean values of the realized FDP of BH-FDR are fairly
close to their corresponding nominal levels of $\alpha$. However, the mean
values of FNP for BH-FDR are generally larger than those of DCOE, especially
when signal intensity is relatively weak. It is true that one can always
increase the nominal level of BH-FDR to reduce FNP, but it is unclear by how
much we should increase $\alpha$ to achieve a target level of FNP. For
example, when $A=3$, BH-FDR with $\alpha=0.2$ has the mean values of FNP in
the range of 0.166-0.176 for the three dependence models, which seem to be
comparable to the FNP range of DCOE with $\beta=0.2$. However, when $A=2$, BH-
FDR with $\alpha=0.2$ has much bigger mean values of FNPs in the range of 0.63
- 0.66.

These results demonstrate the fundamental differences between multiple testing
procedures and the proposed dual control method as they serve for very
different purposes. The proposed method aims to retain more signals with a
target FNP level, for which the multiple testing methods cannot achieve. On
the other hand, the new method pays the price of having a higher FDP.
Additional simulation examples illustrating the efficiency of the new method
compared to other false negative control methods are presented in Section 5

##  4 FNP control with estimated number of signals

Implementation of the new method requires information of the number of signals
($s$) which, in real applications, is often unknown. Existing studies for the
estimation of $s$ usually assume independence among variables (Genovese and
Wasserman, 2004; Meinshausen and Rice, 2006; Cai et al., 2007; Cai and Jin,
2010), and most of them are for relatively dense signals. We are interested in
finding an estimator of $s$ that works in our setting with arbitrary
covariance dependence. Recent study in Jeng (2021) introduces an estimator for
the signal proportion $\pi(=s/p)$ with the form
$\hat{\pi}=\max\\{\hat{\pi}_{\delta},\delta\in\Delta\\}$, where $\Delta$ is a
set of functions that render the most powerful consistent estimators in a
family of estimators under different dependence scenarios. More specifically,
the family of estimators are defined as

$\hat{\pi}_{\delta}=\sup_{t>0}{\frac{R(t)/p-2\bar{\Phi}(t)-c_{p,\delta}\delta\left(t\right)}{1-2\bar{\Phi}(t)}},$
(14)

where $\delta(t)$ is a strictly positive function on $(0,\infty)$ and
$c_{p,\delta}$ is its corresponding bounding sequence. It is discovered that
$\delta(t)=[\bar{\Phi}(t)]^{1/2}$ and $\delta(t)=\bar{\Phi}(t)$ result in
powerful $\hat{\pi}_{\delta}$ under different dependence scenarios, and a new
estimator is developed as

$\hat{\pi}=\max\\{\hat{\pi}_{0.5},\hat{\pi}_{1}\\},$ (15)

where $\hat{\pi}_{0.5}$ denotes $\hat{\pi}_{\delta}$ with
$\delta=[\bar{\Phi}(t)]^{1/2}$, and $\hat{\pi}_{1}$ denotes
$\hat{\pi}_{\delta}$ with $\delta=\bar{\Phi}(t)$. Details about the selection
of $\delta(t)$, the construction of $c_{p,\delta}$, and the properties of
$\hat{\pi}_{\delta}$ and $\hat{\pi}$ can be found in Jeng (2021).

We extend the results in Jeng (2021) to our setting with $\gamma$ and $\eta$
representing signal sparsity and overall covariance strength, respectively, so
that the conditions for the consistency of $\hat{\pi}$ and the conditions for
the FNP control of our proposed method can be unified. Because the theoretical
analysis in Jeng (2021) is conducted on a discretized version
($\hat{\pi}_{\delta}^{*}$) of $\hat{\pi}_{\delta}$, which is defined by
replacing $\sup_{t>0}$ in (14) with $\max_{t\in\mathbb{T}}$, where
$\mathbb{T}=[1,\sqrt{5\log p}]\cap\mathbb{N}$, we present the unified
conditions for the discretized version of $\hat{\pi}$ as well and define it as
$\hat{\pi}^{*}=\max\\{\hat{\pi}^{*}_{0.5},\hat{\pi}^{*}_{1}\\}$.

######  Proposition 4.1.

Assume the same conditions as in Theorem 2.2. Then, for any constant
$\epsilon>0$,

$P((1-\epsilon)<\hat{\pi}^{*}/\pi<1)\rightarrow 1.$ (16)

Replace $s$ in (10) with the estimator $\hat{s}=p\cdot\hat{\pi}^{*}$ and
denote the selection threshold as $\hat{t}_{\hat{s}}(\beta)$. We have

$P(\text{FNP}(\hat{t}_{\hat{s}}(\beta))\leq\beta)\rightarrow 1$ (17)

and, for any threshold $\tilde{t}>\hat{t}_{\hat{s}}(\beta)$,

$P\\{\text{FNP}(\tilde{t})>\beta-\delta)\rightarrow 1$ (18)

for arbitrarily small constant $\delta>0$.

Proposition 4.1 shows that the same set of conditions are sufficient for both
the consistency of $\hat{\pi}^{*}$ and the efficient FNP control of the
proposed method implemented with $\hat{\pi}^{*}$. This result can also be
interpreted by the phase diagram in Figure 2, where the solid red line that
moves with the dependence level $\eta$ now indicates the required signal
intensity for both $s$ estimation and FNP control. In other words, for signals
above the solid red line, although they may not be individually identifiable,
we can still perform meaningful inference by not only estimating their total
numbers but also effectively retaining them at a target FNP level.

Generally speaking, if we want to derive information of sparse signals from a
dataset, a condition on the signal intensity is unfortunately unavoidable.
When signal intensity is too low, no methods can even detect their existence
(Donoho and Jin, 2004; Arias-Castro et al., 2011; Cai et al., 2011), let along
the more challenging tasks of FNP estimation and control. When the condition
on signal intensity does not hold, the estimator $\hat{\pi}$ tends to
underestimate because of its lower bound property (Jeng, 2021). Consequently,
the dual control method implemented with $\hat{s}=p\cdot\hat{\pi}$ tends to
have a threshold higher than the ideal threshold that achieves the target
level of $\beta$, which results in conservative variable selection with
inflated FNP and correspondingly less false positives. This tendency is
observed in simulation examples with low signal-to-noise ratio in Section 5.1.

We conclude the section with numerical algorithms to calculate the bounding
sequence $c_{p,\delta}$ and the proportion estimator $\hat{\pi}$ in (14), and
to select variables by the dual control method.

Algorithm 1 Bounding Sequence $c_{p,\delta}$

1:Input: N sets of $\\{w_{1},\ldots,w_{p}\\}$ generated by the joint null
distribution of $\\{z_{1},\ldots z_{p}\\}$

2:Output: bounding sequences $c_{p,0.5}$ and $c_{p,1}$

3:for $a=1,2\ldots,N$ do

4:Rank the $a$-th set of $\\{w_{1},\ldots,w_{p}\\}$ such that
$w_{(1)}>w_{(2)}>\ldots>w_{(p)}$

5:Compute  $V_{0.5,a}=\max_{1\leq j\leq
p}{|j/p-\bar{\Phi}(w_{(j)})|\over\sqrt{\bar{\Phi}(w_{(j)})}}\qquad\text{and}\qquad
V_{1,a}=\max_{1\leq j\leq
p}{|j/p-\bar{\Phi}(w_{(j)})|\over\bar{\Phi}(w_{(j)})}$

6:end for

7:Compute $c_{p,0.5}$ and $c_{p,1}$ as the $(1-1/\sqrt{\log p})$-th quantiles
of the empirical distributions of $V_{0.5,a}$ and $V_{1,a}$, respectively

Note that the computation of $c_{p,\delta}$ requires input of statistics
generated by the joint null distribution. When the joint null distribution is
unknown in real applications, we often can simulate such statistics non-
parametrically. For example, when$\\{z_{1},\ldots z_{p}\\}$ are a set of test
statistics for associations between a set of explanatory variables and a
response variable, we may randomly shuffling only the sample of the response
variable to remove the potential associations and then calculate the test
statistics.

Algorithm 2 Signal Proportion Estimator

1:Input: test statistics $\\{z_{1},\ldots z_{p}\\}$ and bounding sequences
$c_{p,0.5}$ and $c_{p,1}$

2:Output: a proportion estimate $\hat{\pi}$

3:Rank the variables by their test statistics so that
$z_{(1)}>z_{(2)}>\ldots>z_{(p)}$

4:Compute  $\hat{\pi}_{0.5}=\max_{1\leq j\leq
p}{\frac{j/p-\bar{\Phi}(z_{(j)})-c_{p,0.5}\sqrt{\bar{\Phi}(z_{(j)})}}{1-\bar{\Phi}(z_{(j)})}}\qquad\text{and}\qquad\hat{\pi}_{1}=\max_{1\leq
j\leq
p}{\frac{j/p-\bar{\Phi}(z_{(j)})-c_{p,1}\bar{\Phi}(z_{(j)})}{1-\bar{\Phi}(z_{(j)})}}$

5:Obtain $\hat{\pi}=\max\\{\hat{\pi}_{0.5},\hat{\pi}_{1}\\}$

Algorithm 3 Dual Control Procedure

1:Input: test statistics $z_{1},\ldots z_{p}$, a user-specified $\beta$, and a
proportion estimate $\hat{\pi}$

2:Output: a set of selected variables

3:Rank the variables by their test statistics so that
$z_{(1)}>z_{(2)}>\ldots>z_{(p)}$

4:Compute $\hat{s}=\hat{\pi}p$

5:For $j=1,2,\ldots,p$, compute
$\widehat{FNP}(z_{(j)})=\max\\{1-j/\hat{s}+(p-\hat{s})\bar{\Phi}(z_{(j)})/\hat{s},~{}0\\}$

6:Let $k=1$

7:while $\widehat{FNP}(z_{(k)})\geq\beta$ do $k=k+1$

8:end while

9:Obtain variables ranked at $1,\ldots,k-1$

##  5 Simulation

In this section, We compare the empirical performances of the new method
(DCOE) with existing false negative control methods. Such methods are
relatively limited and have only appeared recently. For example, the AFNC
method in Jeng et al. (2016) was proposed to control the so-called signal
missing rate to detect rare variants in genome-wide association study (Jeng et
al., 2016); the MDR method in Cai and Sun (2017) was proposed to control the
so-called missed discovery rate using an empirical Bayesian approach (Cai and
Sun, 2017); the AdSMR method in Jeng et al. (2019) focuses on controlling a
FNP-based exceedance probability; and the FNC-Reg approach in Jeng and Chen
(2019b) considers FNP control in linear regression. Among these methods, AFCN,
MDR, and FNC-Reg are more comparable to DCOE because they all require the
input of a user-specified control level. However, AFNC and MDR were developed
under the independent assumption, and FNC-Reg considered specific dependence
and signal sparsity conditions to facilitate accurate precision matrix
estimation and bias mitigation. All of these methods require estimates of the
number of signals. For a fair comparison, we implement the same estimator
$\hat{s}=p\cdot\hat{\pi}$ to all the methods.

###  5.1 One-dimensional problems

In this section, we generate the test statistics by (13) with a covariance
matrix that has 20 diagonal blocks with block sizes randomly generated from 10
to 100. The non-zero off-diagonal correlations are set at 0.5. The dependence
parameter $\eta$ varies from sample to sample due to the random block size. In
the first set of examples, signal sparsity is fixed with $\gamma=0.3$, and
signal intensity (A) increases from $3$ to $5$. In the second set of examples,
signal intensity is fixed and signal sparsity varies as $\gamma=0.3$ and
$0.5$.

The performances of the methods are evaluated by three measures. The first two
measures, FNP and FDP are the same as in Table 1. The last measure is the
Fowlkes-Mallows index (Fowlkes and Mallows, 1983; Halkidi et al., 2001), which
summarizes the measures of FNP and FDP by calculating the geometric mean of
($1-$FNP) and ($1-$FDP), i.e.,

$\text{FM-index}=\sqrt{(1-\text{FNP})\times(1-\text{FDP})}.$

Higher values of the FM-index indicate better classification results. The FM-
index is a sensible summary measure in high-dimensional settings with sparse
alternative cases because the scale of its FDP component is more comparable to
that of its FNP component than the classical false positive proportion (FPP =
number of false positives/number of null cases). Because the methods compared
in this section all focus on false negative control, it is appropriate to use
FM-index to evaluate their relative efficiency.

Results of the first set of examples are summarized in Table 2. First, because
the estimation boundary $\mu_{min}$ remains at $2.14$ in these examples, the
condition $A_{j}-\mu_{min}\to\infty$ in Theorem 2.2 is not well supported for
smaller $A_{j}$ ($=A$ in these examples). Also, because the estimator
$\hat{s}$ underestimates the true $s$ when signal intensity is not strong
enough, DCOE implemented with $\hat{s}$ selects less variables than actually
needed to reach the nominal level of $\beta$. These result in inflated
realized FNP as seen in Table 2. Further, as $A$ increases, the mean value of
FNP of DCOE gets closer to the nominal level of $\beta$, which agrees with the
claims in (17) and (18) in Proposition 4.1. Among the three methods presented
in Table 2, DCOE shows a clear tendency to adapt to the nominal level of
$\beta$ as $A$ increases, which is not observed for the other two methods.
DCOE also has the lowest FNP values. In terms of the FM-index, DCOE generally
outperforms the other two methods especially when signal intensity gets
stronger.

Table 2: Effect of signal intensity on different FN control methods with estimated s. Mean values and standard deviations (in brackets) of the realized FNP, FDP, and the FM-index are presented for the proposed DCOE method and two existing methods, AFNC and MDR from 100 replications.  |  | FNP | FDP | FM-index
---|---|---|---|---
$A=3$ | DCOE($\beta={\bf 0.1}$) |  0.28 (0.12) | 0.13 (0.14) |  0.78 (0.04)
| AFNC($\beta={\bf 0.1}$) | 0.19 (0.11) | 0.22 (0.18) | 0.78 (0.07)
| MDR($\beta={\bf 0.1}$) | 0.09 (0.08) | 0.49 (0.26) | 0.65 (0.15)
$A=4$ | DCOE($\beta={\bf 0.1}$) |  0.16 (0.07) | 0.06 (0.12) |  0.88 (0.05)
| AFNC($\beta={\bf 0.1}$) | 0.07 (0.04) | 0.14 (0.18) | 0.89 (0.10)
| MDR($\beta={\bf 0.1}$) | 0.06 (0.05) | 0.37 (0.32) | 0.74 (0.18)
$A=5$ | DCOE($\beta={\bf 0.1}$) |  0.10 (0.04) | 0.04 (0.12) |  0.93 (0.06)
| AFNC($\beta={\bf 0.1}$) | 0.01 (0.01) | 0.36 (0.35) | 0.75 (0.26)
| MDR($\beta={\bf 0.1}$) | 0.04 (0.04) | 0.36 (0.33) | 0.75 (0.19)

The second set of examples have $\gamma$ increased from 0.3 to 0.5, so that
the number of signals deceases from 205 to 45. The nominal level of $\beta$
also varies. The signal intensity $A$ is fixed at $5$. Results summarized in
Table 3 show that when signals get sparser with larger $\gamma$, the
performances of all three methods deteriorate by including more noise
variables. However, DCOE continues to outperform the other two methods in
adapting to different nominal levels and incurring less false positives. Its
advantage seems to be more prominent when signals get more sparse.

Table 3: Effect of nominal level and signal sparsity on different FN control methods with estimated s. Same notations as in Table 2 are used.  |  | FNP | FDP | FM-index
---|---|---|---|---
$\gamma=0.3$ | DCOE($\beta={\bf 0.1}$) |  0.10 (0.04) | 0.04 (0.12) |  0.93 (0.06)
| AFNC($\beta={\bf 0.1}$) | 0.01 (0.01) | 0.36 (0.35) | 0.75 (0.26)
| MDR($\beta={\bf 0.1}$) | 0.04 (0.04) | 0.36 (0.33) | 0.75 (0.19)
| DCOE($\beta={\bf 0.2}$) |  0.19 (0.06) | 0.02 (0.08) |  0.89 (0.03)
| AFNC($\beta={\bf 0.2}$) | 0.01 (0.01) | 0.21 (0.29) | 0.86 (0.20)
| MDR($\beta={\bf 0.2}$) | 0.08 (0.09) | 0.26 (0.28) | 0.80 (0.14)
$\gamma=0.5$ | DCOE($\beta={\bf 0.1}$) |  0.09 (0.05) | 0.15 (0.29) |  0.85 (0.18)
| AFNC($\beta={\bf 0.1}$) | 0.00 (0.01) | 0.73 (0.28) | 0.44 (0.27)
| MDR($\beta={\bf 0.1}$) | 0.03 (0.05) | 0.56 (0.44) | 0.54 (0.34)
| DCOE($\beta={\bf 0.2}$) |  0.16 (0.09) | 0.12 (0.27) |  0.83 (0.15)
| AFNC($\beta={\bf 0.2}$) | 0.01 (0.02) | 0.60 (0.36) | 0.55 (0.31)
| MDR($\beta={\bf 0.2}$) | 0.07 (0.10) | 0.54 (0.45) | 0.54 (0.32)

###  5.2 A Two-dimensional example

In this example, we simulate a $100\times 100$ two-dimensional grid graph with
the signal region demonstrated in 4(a). The signal region covers $994$ nodes,
i.e. $s=994$. At each node, test statistic $Z_{ij}$ is generated independently
from $N(A_{ij},1)$, where $A_{ij}\sim$ Uniform$[1,2.5]$ if $(i,j)$ is in the
signal region, and $A_{ij}=0$ otherwise.

We apply the proposed DCOE method with different nominal levels and compare
the results with those of BH-FDR, AFNC, and MDR. 4(b) shows the selected nodes
of BH-FDR with $\alpha=0.05$ in a single trial, 4(c) and 4(d) show the results
of AFNC and MDR with $\beta=0.1$, respectively, and 4(e) and 4(f) show the
results of DCOE with $\beta=0.1$ and $0.5$, respectively. It can be seen that
DCOE identifies more true signals and delineates the signal region better than
BH-FDR even when $\beta$ is as large as $0.5$. On the other hand, it pays the
price with some more false positives. Compared to AFNC and MDR, DCOE has less
false positives and less true positives, which does not seem to impair its
ability to delineate the signal region much.

(a) Pure Signals

(b) BH-FDR($\alpha=0.05$)

(c) AFNC($\beta=0.1$)

(d) MDR($\beta=0.1$)

(e) DCOE($\beta=0.1$)

(f) DCOE($\beta=0.5$)

Figure 4: Comparison results from a single trial. The clustered signals are
shown in plot (a). Plots (b) - (f) demonstrate the results of different
methods.

##  6 Application

We obtained the fMRI data from the Individual Brain Charting (IBC) Project,
which is a publicly available high-resolution fMRI dataset for cognitive
mapping (Pinho et al., 2018). The dataset refers to a cohort of 12
participants performing different tasks, addressing both low- and high- level
cognitive functions. We focus on the data from the saccade experiment for
spatial cognition, in which ocular movements were performed according to the
displacement of a fixation cross from the center toward peripheral locations
in the image displayed.

The data were collected using a Gradient-Echo (GE) pulse, whole-brain Multi-
Band (MB) accelerated Echo-Planar Imaging (EPI) T2⋆-weighted sequence with
Blood-Oxygenation-Level-Dependent (BOLD) contrasts, and preprocessed using
PyPreprocess, a collection of python tools for preprocessing fMRI data. In
order to assess the statistical significance of the differences among evoked
BOLD responses, test statistics are computed at every voxel for each contrast
using General Linear Model (GLM). All images are confined to an average mask
of the gray matter across subjects, which yields 371,817 voxels at the chosen
resolution. More details about the dataset can be found in Pinho et al.
(2018).

We apply the proposed method (DCOE), the popular BH-FDR procedure, and the
existing AFNC and MDR methods to the statistical maps of 12 participants. The
numbers of selected voxels for each participant are reported in Table 4. Among
all the methods, BH-FDR selects the least voxels and MDR selects the most
voxels. DCOE selects more than BH-FDR and less than AFNC and MDR.

Table 4: Numbers of selected voxels by different methods. Subject | BH-FDR($\alpha=0.05$) | DCOE($\beta=0.1$) | AFNC($\beta=0.1$) | MDR($\beta=0.1$)
---|---|---|---|---
sub-1 | 18500 | 19379 | 24837 | 27793
sub-2 | 44902 | 52614 | 70981 | 78288
sub-4 | 15291 | 31852 | 44893 | 60356
sub-5 | 8623 | 23236 | 37333 | 62976
sub-6 | 17778 | 29298 | 40537 | 47812
sub-7 | 25011 | 51405 | 76931 | 88407
sub-8 | 35463 | 47467 | 65238 | 74915
sub-9 | 20846 | 35662 | 50516 | 63989
sub-11 | 28586 | 32989 | 43147 | 48892
sub-12 | 20469 | 27388 | 36256 | 38353
sub-13 | 21365 | 44094 | 63192 | 74508
sub-14 | 21067 | 58494 | 85889 | 96651

Figure 5 illustrates the selected voxels of each method in the image of a
single participant (sub-5) from posterior, superior, and left views. The
figure is generated using the Multi-image Analysis GUI
(http://ric.uthscsa.edu/mango/). We look into several regions that are known
to be associated with saccadic eye movements. First, the visual cortex (VC) on
occipital lobe in the posterior region of the brain, as indicated in 5(d), is
the primary cortical region that receives, integrates, and processes visual
information (Bodis-Wollner et al., 1997). We can see that all four methods
have identified voxels in VC. However, the results of DCOE, AFNC and MDR seem
to match the VC region much better than that of BH-FDR. From the superior and
left views, it shows that BH-FDR has only a few or no discoveries in the
frontal eye fields (FEF) located in Brodmann area 8, the supplementary eye
fields (SEF) located in Broadmann area 6, and the posterior parietal cortex
(PPC). The locations of FEF, SEF, and PPC are indicated in 5(e) and 5(f). FEF
and SEF are believed to play important roles in visual attention and eye
movements as electrical stimulation of these areas evokes eye movements (Bruce
and Goldberg, 1985; Bruce et al., 1985). PPC, on the other hand, is related to
decision making and saccades (Goldberg et al., 2002; Schluppeck et al., 2005).
It can be seen that DCOE, AFNC, and MDR have better power for identifying
signal voxels in FEF, SEF, and PPC. Among these three methods, DCOE selects
the least scattered or isolated voxels that are not functionally relevant to
saccades.

(a) BH-FDR (posterior)

(b) BH-FDR (superior)

(c) BH-FDR (left)

(d) DCOE (posterior)

(e) DCOE (superior)

(f) DCOE (left)

(g) AFNC (posterior)

(h) AFNC (superior)

(i) AFNC (left)

(j) MDR (posterior)

(k) MDR (superior)

(l) MDR (left)

Figure 5: The selected voxels from BH-FDR, DCOE, AFNC, and MDR for a single
participant (sub-05).

##  7 Conclusion and Discussion

In this paper, we develop a new analytic framework to perform inference on
signals that are detectable yet unidentifiable. A new dual control method is
proposed to control FNP at a user-specified level and, at the same time,
regulate the unnecessary false positives to achieve the target FNP level. The
new method is developed under arbitrary covariance dependence calibrated
through a dependence parameter whose scale is compatible with the existing
formulation in sparse inference. Utilizing the new calibration, we are able to
explicate the joint effects of covariance dependence and signal sparsity and
interpret the results using a phase digram to gain important insight.

We demonstrate the finite sample performance of the dual control method in
simulation and apply the method to identify functionally relevant regions for
saccadic eye movement using fMRI data. Its results are compared with those of
other methods and with regions that are known to be associated with saccade.
The new method seems to benefit from its dual control property and exhibits a
good balance in identifying signal voxels in the functionally relevant regions
and avoiding the scattered noise voxels.

This new method will be useful in, for example, pre-surgical fMRI data
analysis, where efficient false negative control is of vital importance
because neurosurgical patients are likely to experience significant harm from
mistakenly deeming a region to be functionally uninvolved and subsequently
resecting critical tissues. In a broad sense, our study expands the frontier
of high-dimensional sparse inference by bridging an important methodological
gap.

##  8 Appendix

This section provides the proofs of Theorem 2.1, Theorem 2.2, and Proposition
4.1, as well as additional simulation results. We will frequently use the
following result on Mill’s ratio:

$\bar{\Phi}(x)\leq x^{-1}\phi(x)\qquad\text{for any}\quad x>0.$

The symbol $C$ denotes a genetic, finite constant whose value can be different
at different occurrences.

###  8.1 Proof of Theorem 2.1

For notation simplicity, let

$B(t)=1-\text{R}(t)/s+(p-s)\bar{\Phi}(t)/s.$ (19)

Then $\widehat{\text{FNP}}(t)=\max\\{B(t),~{}0\\}$, and it is sufficient to
show that

$|B(t)-\text{FNP}(t)|=o_{p}(1)\qquad\text{when}\quad B(t)\geq 0$ (20)

and

$\text{FNP}(t)=o_{p}(1)\qquad\text{when}\quad B(t)<0.$ (21)

Consider (20) first. By the definitions of $B(t)$ and $\text{FNP}(t)$,

$|B(t)-\text{FNP}(t)|=|s^{-1}(\text{R}(t)-(p-s)\bar{\Phi}(t))-s^{-1}(\text{R}(t)-\text{FP}(t))|=s^{-1}|\text{FP}(t)-(p-s)\bar{\Phi}(t)|.$

Therefore, it is sufficient to show

$s^{-1}|\text{FP}(t)-(p-s)\bar{\Phi}(t)|=o_{P}(1).$ (22)

Recall the definition of $\mu_{1}$ and $\mu_{2}$ in (8). The following proof
is composed of two parts. The first part assumes $t\geq\mu_{1}$ and the second
part assumes $\mu_{2}\leq t<\mu_{1}$.

Consider the first part. It’s sufficient to show $s^{-1}\text{FP}(t)=o_{P}(1)$
and $s^{-1}(p-s)\bar{\Phi}(t)=o(1)$ with $t\geq\mu_{1}$. By Mill’s ratio and
the re-parameterization of $s$ in $\gamma$,

$s^{-1}(p-s)\bar{\Phi}(t)\leq\frac{Cpe^{-t^{2}/2}}{ts}\leq\frac{C}{\sqrt{\log{p}}}=o(1).$

On the other hand, for a fixed constant $a>0$,

$P(s^{-1}\text{FP}(t)>a)\leq\frac{E(\text{FP}(t))}{as}\leq\frac{p\max_{j\in
I_{0}}P(z_{j}>t)}{as}\leq\frac{Cp\bar{\Phi}(t)}{s}=o(1).$

Therefore, the claim in (22) is justified for
$t\geq\mu_{1}=\sqrt{2\gamma\log{p}}$.

Next we present the second part of the proof with $\mu_{2}\leq t<\mu_{1}$.
Define

$D_{p}=s^{-2}e^{-t^{2}/2}\lVert\mathbf{\Sigma}\rVert_{1}\log{p}.$

By the condition $t\geq\mu_{2}$ and the re-parameterizations of
$\lVert\mathbf{\Sigma}\rVert_{1}$ in $\eta$ and $s$ in $\gamma$, it can be
shown that

$\displaystyle D_{p}=p^{2\gamma-\eta}e^{-t^{2}/2}\log{p}\leq\log^{-1}p=o(1).$

Then, (22) is implied by

$s^{-1}|\text{FP}({t})-(p-s)\bar{\Phi}(t)|=o_{P}(\sqrt{D_{p}}).$ (23)

Apply Chebyshev’s inequality,

$P(s^{-1}|\text{FP}(t)-(p-s)\bar{\Phi}(t)|>\sqrt{D_{p}})\leq\frac{Var(\text{FP}(t))}{s^{2}D_{p}}.$
(24)

We have the following lemma for the order of $Var(\text{FP}(t))$ under
dependence. The proof is essentially the same as that of Lemma 5.2 in Jeng
(2021). We omit the details to save space.

######  Lemma 8.1.

Consider model (6). Denote $\mathbf{\Sigma}_{0}$ as the correlation matrix of
$Z_{j},j\in I_{0}$. We have

$Var(\sum_{j\in
I_{0}}1_{\\{z_{j}>t\\}})=O(e^{-t^{2}/2}\lVert\mathbf{\Sigma_{0}}\rVert_{1}).$

Therefore, $Var(\text{FP}(t))=Var(\sum_{j\in I_{0}}1_{\\{z_{j}>t\\}})\leq
Ce^{-t^{2}/2}\lVert\mathbf{\Sigma}_{0}\rVert_{1}\leq
Ce^{-t^{2}/2}\lVert\mathbf{\Sigma}\rVert_{1}$, and it follows that

$\frac{Var(\text{FP}(t))}{s^{2}D_{p}}=o(1)$ (25)

by the definition of $D_{p}$. Combining (24) and (25) gives (23), which
justifies the claim in (22) for $\mu_{2}\leq t<\mu_{1}$.

Finally, consider (21). It can be shown that $B(t)<0$ implies
$1-\text{FP}(t)/s-\text{TP}(t)/s+(p-s)\bar{\Phi}(t)/s<0$, which, combined with
FNP$(t)=1-\text{TP}(t)/s$, further implies

$\text{FNP}(t)<s^{-1}(\text{FP}(t)-(p-s)\bar{\Phi}(t)).$

Since the order of the right hand side has been derived in (22), (21) follows.
∎

###  8.2 Proof of Theorem 2.2

First, by Markov’s inequality, for a fixed constant $a>0$,

$\displaystyle P(\text{FNP}(\mu_{min})>a)$ $\displaystyle=$ $\displaystyle
P(s^{-1}\sum_{j\in
I_{1}}1_{\\{z_{j}\leq\mu_{min}\\}}>a)\leq\frac{1}{as}\sum_{j\in
I_{1}}P(Z_{j}\leq\mu_{min})$ $\displaystyle\leq$
$\displaystyle\frac{1}{a}\max_{j\in I_{1}}P(N(0,1)\leq\mu_{min}-A_{j})=o(1),$

where the last step is by the condition on $A_{j}$. Therefore,
$\text{FNP}(\mu_{min})=o_{P}(1)$. By Theorem 2.1, we have
$|\widehat{\text{FNP}}(\mu_{min})-\text{FNP}(\mu_{min})|=o_{P}(1)$, which
implies $\widehat{\text{FNP}}(\mu_{min})=o_{P}(1)$. Then, by the construction
of $\hat{t}(\beta)$, we have $P(\hat{t}(\beta)\geq\mu_{min})\rightarrow 1$.
Consequently, since $\hat{t}(\beta)$ is random,

$\displaystyle\lim_{p\to\infty}P(|\widehat{\text{FNP}}(\hat{t}(\beta))-\text{FNP}(\hat{t}(\beta))|>a)$
$\displaystyle=$
$\displaystyle\lim_{p\to\infty}E\left[E\left(1\\{|\widehat{\text{FNP}}(\hat{t}(\beta))-\text{FNP}(\hat{t}(\beta))|>a\\}|\hat{t}(\beta)\right)\right]$
$\displaystyle=$ $\displaystyle
E\left[\lim_{p\to\infty}E\left(1\\{|\widehat{\text{FNP}}(\hat{t}(\beta))-\text{FNP}(\hat{t}(\beta))|>a\\}|\hat{t}(\beta)\right)\right]=0,$

where the second step above is by dominated convergence theorem and the last
step is by Theorem 2.1. Therefore,
$|\widehat{\text{FNP}}(\hat{t}(\beta))-\text{FNP}(\hat{t}(\beta))|=o_{P}(1)$.
Now, since $\widehat{\text{FNP}}(\hat{t}(\beta))<\beta$ almost surely by the
construction of $\hat{t}(\beta)$, claim in (11) follows.

Next, let’s consider the claim in (12). Because $\tilde{t}>\hat{t}(\beta)$,
the construction of $\hat{t}(\beta)$ implies that
$\widehat{\text{FNP}}(\tilde{t})\geq\beta$ almost surely. On the other hand,
because $\tilde{t}>\hat{t}(\beta)\geq\mu_{min}$ with probability tending to 1,
similar arguments as the above lead to
$|\widehat{\text{FNP}}(\tilde{t})-\text{FNP}(\tilde{t})|=o_{P}(1)$. Therefore,
the claim in (12) follows. ∎

###  8.3 Proof of Proposition 4.1

The consistency of $\hat{\pi}^{*}$ is based on the following two lemmas, which
are implied by Theorem 2.2 and 2.3 in Jeng (2021).

######  Lemma 8.2.

Consider model (6). Let $\delta(t)=[\bar{\Phi}(t)]^{1/2}$. Then, there exists
a bounding sequence $c_{p,\delta}^{*}=O\left(\sqrt{\bar{\rho}\cdot\log
p}\right)$, and the corresponding estimator $\hat{\pi}^{*}_{\delta}$ satisfies
$P(\hat{\pi}^{*}_{\delta}<\pi)\to 1$. Moreover, for $\pi$ satisfying $0<\pi\ll
1$, if $\tau=\min_{j\in I_{1}}A_{j}\gg 1$ and

$\tau-\bar{\Phi}^{-1}\left({\pi^{2}\over\bar{\rho}\cdot\log
p}\right)\to\infty,$ (26)

then $P(\hat{\pi}^{*}_{\delta}>(1-\epsilon)\pi)\to 1$ for any constant
$\epsilon>0$.

######  Lemma 8.3.

Consider model (6). Let $\delta(t)=\bar{\Phi}(t)$. Then, there exists a
bounding sequence $c_{p,\delta}^{*}=O(\sqrt{\log p})$, and the corresponding
estimator $\hat{\pi}^{*}_{\delta}$ satisfies $P(\hat{\pi}^{*}_{\delta}<\pi)\to
1$. Moreover, for $\pi$ satisfying $0<\pi\ll 1$, if $\tau=\min_{j\in
I_{1}}A_{j}\gg 1$ and

$\tau-\bar{\Phi}^{-1}\left({\pi\over\sqrt{\log p}}\right)\to\infty.$ (27)

then $P(\hat{\pi}^{*}_{\delta}>(1-\epsilon)\pi)\to 1$ for any constant
$\epsilon>0$.

Recall that $\pi=p^{-\gamma}$, $\gamma\in(0,1]$, and
$\bar{\rho}=p^{-\eta},\eta\in[0,1]$. Recall $\mu_{1}$ and $\mu_{2}$ in (8).
Direct calculation shows that when $\tau-\mu_{2}\to\infty$, the condition in
(26) is satisfied, and when $\tau-\mu_{1}\to\infty$, the condition in (27) is
satisfied. Therefore, given $\tau=\min_{j\in I_{1}}A_{j}-\mu_{min}\to\infty$,
the consistency of the estimator
$\hat{\pi}^{*}=\max\\{\hat{\pi}_{0.5},\hat{\pi}_{1}\\}$ holds as in (16).

Next consider the claims in (17) and (18). Similar arguments as in the proof
of Theorem 2.2 can be applied, and we only need to show that Theorem 2.1
continues to hold with $\widehat{\text{FNP}}(t)$ replaced by
$\widehat{\text{FNP}}_{\hat{s}}(t)$. Given the result in (9), it is sufficient
to show $|\widehat{\text{FNP}}_{\hat{s}}(t)-\widehat{\text{FNP}}(t)|=o_{P}(1)$
for $t\geq\mu_{min}$, which is implied by

$|B_{\hat{s}}(t)-B(t)|=o_{P}(1)\quad\text{for}\quad t\geq\mu_{min},$ (28)

given the definition of $B(t)$ in (19). By direct calculation,

$\displaystyle|B_{\hat{s}}(t)-B(t)|$ $\displaystyle=$
$\displaystyle|(\hat{s}^{-1}-s^{-1})(\text{R}(t)-p\bar{\Phi}(t))|$ (29)
$\displaystyle\leq$
$\displaystyle|\hat{s}^{-1}-s^{-1}|\cdot\text{TP}(t)+|\hat{s}^{-1}-s^{-1}|\cdot|\text{FP}(t)-p\bar{\Phi}(t)|.$

Given $P((1-\delta)<\hat{s}/s<1)\rightarrow 1$ for any $\delta>0$, it can be
shown that

$P(|\hat{s}^{-1}-s^{-1}|<\frac{\delta}{1-\delta}s^{-1})\to 1.$

On the other hand, $TP(t)\leq s$ almost surely. Then it follows that the first
term in (29),

$|\hat{s}^{-1}-s^{-1}|\cdot\text{TP}(t)=o_{p}(1).$

For the second term in (29),

$|\hat{s}^{-1}-s^{-1}|\cdot|\text{FP}(t)-p\bar{\Phi}(t)|<\frac{\delta}{1-\delta}s^{-1}|\text{FP}(t)-p\bar{\Phi}(t)|)\leq\frac{\delta}{1-\delta}\left(s^{-1}|\text{FP}(t)-(p-s)\bar{\Phi}(t)|)+\bar{\Phi}(t)\right)$

with probability tending 1, where $\bar{\Phi}(t)=o(1)$ for $t\geq\mu_{min}$,
and it has been shown as for (22) that
$s^{-1}|\text{FP}(t)-(p-s)\bar{\Phi}(t)|=o_{P}(1)$ for $t\geq\mu_{min}$. Then
it follows that

$|\hat{s}^{-1}-s^{-1}|\cdot|\text{FP}(t)-p\bar{\Phi}(t)|=o_{P}(1).$

Summing up the above gives (28). ∎

## References

  * Arias-Castro et al. (2011) Arias-Castro, E., E. J. Candès, and Y. Plan (2011).  Global testing under sparse alternatives: Anova, multiple comparisons and the higher criticism.  The Annals of Statistics 39(5), 2533–2556.
  * Barber and Candès (2015) Barber, R. F. and E. J. Candès (2015).  Controlling the false discovery rate via knockoffs.  Ann. Statist. 43(5), 2055–2085.
  * Benjamini and Hochberg (1995) Benjamini, Y. and Y. Hochberg (1995).  Controlling the false discovery rate: a practical and powerful approach to multiple testing.  J. R. Statist. Soc. Ser. B 57(1), 289–300.
  * Bodis-Wollner et al. (1997) Bodis-Wollner, I., S. Bucher, K. Seelos, W. Paulus, M. Reiser, and W. Oertel (1997).  Functional mri mapping of occipital and frontal cortical activity during voluntary and imagined saccades.  Neurology 49(2), 416–420.
  * Bruce and Goldberg (1985) Bruce, C. J. and M. E. Goldberg (1985).  Primate frontal eye fields. i. single neurons discharging before saccades.  Journal of neurophysiology 53(3), 603–635.
  * Bruce et al. (1985) Bruce, C. J., M. E. Goldberg, M. C. Bushnell, and G. B. Stanton (1985).  Primate frontal eye fields. ii. physiological and anatomical correlates of electrically evoked eye movements.  Journal of neurophysiology 54(3), 714–734.
  * Cai et al. (2007) Cai, T., J. Jin, and M. Low (2007).  Estimation and confidence sets for sparse normal mixtures.  The Annals of Statistics 35(6), 2421–2449.
  * Cai et al. (2011) Cai, T. T., X. J. Jeng, and J. Jin (2011).  Optimal detection of heterogeneous and heteroscedastic mixtures.  Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73(5), 629–662.
  * Cai and Jin (2010) Cai, T. T. and J. Jin (2010).  Optimal rates of convergence for estimating the null density and proportion of nonnull effects in large-scale multiple testing.  The Annals of Statistics, 100–145.
  * Cai and Sun (2017) Cai, T. T. and W. Sun (2017).  Optimal screening and discovery of sparse signals with applications to multistage high-throughput studies.  Journal of the Royal Statistical Society: Series B 79(1), 197–223.
  * Candes et al. (2018) Candes, E., Y. Fan, L. Janson, and J. Lv (2018).  Panning for gold: Model-x knockoffs for high dimensional controlled variable selection.  J. R. Statist. Soc. Ser. B 80(3), 551–577.
  * Donoho and Jin (2004) Donoho, D. and J. Jin (2004).  Higher criticism for detecting sparse heterogeneous mixtures.  The Annals of Statistics 32(3), 962–994.
  * Donoho and Jin (2015) Donoho, D. and J. Jin (2015).  Special invited paper: Higher criticism for large-scale inference, especially for rare and weak effects.  Statistical Science, 1–25.
  * Dudoit and Van Der Laan (2007) Dudoit, S. and M. J. Van Der Laan (2007).  Multiple testing procedures with applications to genomics.  Springer Science & Business Media.
  * Fan et al. (2012) Fan, J., X. Han, and W. Gu (2012).  Estimating false discovery proportion under arbitrary covariance dependence.  J. Amer. Statist. Assoc. 107(499), 1019–1035.
  * Fowlkes and Mallows (1983) Fowlkes, E. B. and C. L. Mallows (1983).  A method for comparing two hierarchical clusterings.  Journal of the American statistical association 78(383), 553–569.
  * Genovese and Wasserman (2004) Genovese, C. and L. Wasserman (2004).  A stochastic process approach to false discovery control.  Annals of Statistics 32, 1035–1061.
  * Goldberg et al. (2002) Goldberg, M. E., J. Bisley, K. D. Powell, J. Gottlieb, and M. Kusunoki (2002).  The role of the lateral intraparietal area of the monkey in the generation of saccades and visuospatial attention.  Annals of the New York Academy of Sciences 956(1), 205–215.
  * Halkidi et al. (2001) Halkidi, M., Y. Batistakis, and M. Vazirgiannis (2001).  On clustering validation techniques.  Journal of intelligent information systems 17(2-3), 107–145.
  * Jeng (2021) Jeng, X. J. (2021).  Estimating the proportion of signal variables under arbitrary covariance dependence.  arXiv preprint arXiv:2102.09053.
  * Jeng and Chen (2019a) Jeng, X. J. and X. Chen (2019a).  Predictor ranking and false discovery proportion control in high-dimensional regression.  J. Multivariate Anal. 171, 163–175.
  * Jeng and Chen (2019b) Jeng, X. J. and X. Chen (2019b).  Variable selection via adaptive false negative control in linear regression.  Electron. J. Statist. 13(2), 5306–5333.
  * Jeng et al. (2016) Jeng, X. J., Z. J. Daye, W. Lu, and J.-Y. Tzeng (2016).  Rare variants association analysis in large-scale sequencing studies at the single locus level.  PLoS computational biology 12(6).
  * Jeng et al. (2019) Jeng, X. J., T. Zhang, and J.-Y. Tzeng (2019).  Efficient signal inclusion with genomic applications.  Journal of the American Statistical Association 114, 1787–1799.
  * Meinshausen and Bühlmann (2005) Meinshausen, N. and P. Bühlmann (2005).  Lower bounds for the number of false null hypotheses for multiple testing of associations under general dependence structures.  Biometrika 92(4), 893–907.
  * Meinshausen and Rice (2006) Meinshausen, N. and J. Rice (2006).  Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses.  Ann. Statist. 34(1), 373–393.
  * Petticrew et al. (2000) Petticrew, M., A. Sowden, D. Lister-Sharp, and K. Wright (2000).  False-negative results in screening programmes: systematic review of impact and implications.  Health technology assessment (Winchester, England) 4(5), 1–120.
  * Pinho et al. (2018) Pinho, A. L., A. Amadon, T. Ruest, M. Fabre, E. Dohmatob, I. Denghien, C. Ginisty, S. Becuwe-Desmidt, S. Roger, L. Laurier, et al. (2018).  Individual brain charting, a high-resolution fmri dataset for cognitive mapping.  Scientific data 5, 180105.
  * Schluppeck et al. (2005) Schluppeck, D., P. Glimcher, and D. J. Heeger (2005).  Topographic organization for delayed saccades in human posterior parietal cortex.  Journal of neurophysiology 94(2), 1372–1384.


cohomology groups. One should however keep in mind that in our approach we are
starting with a set of objects that contain the information of singular
homology groups, namely the bump delta forms $\delta_{n-p}(\Pi_{p})$, and
replacing them by a countable set of smooth forms $\delta_{n-p}^{\rm
sm}(\Pi_{p})$ that should remember part of the torsion data. As a possible
analogy, one may consider a finite good cover of $X_{n}$ and its nerve $N$,
which is a triangulation of $X_{n}$ [21]. We then consider the delta forms
$\delta_{n-p}(\sigma_{p,\alpha})$ of the $p$-simplexes $\sigma_{p,\alpha}$ of
$N$, with a small smearing (such that $1/\lambda_{\rm max}$ is below the
spacings in $N$). This produces a lattice of smooth $p$-forms from which one
can compute $H^{p}(X_{n},\mathbb{Z})$ via singular cohomology. Our proposal
can be thought of as a limit of this construction, in the sense that we
perform a much more dramatic smearing, namely at wavelenghts above ${\rm
Vol}(X_{n})^{1/n}$. This more drastic coarse-graining is allowed geometrically
because, if the assumptions behind Conjecture 1 are true, then there should be
a $G$-structure manifold that one can construct by fibering $X_{n}$ over flat
space, which is the EFT solution of a D-brane wrapped on $\Pi_{p}$. One could
then use this non-compact, higher-dimensional manifold to compute topological
information of $X_{n}$ via smeared data.

It is also instructive to compare our approach with some of the discussion in
the string theory literature, like the one carried in [48] based on the
classification theorems of Wall [71] and Žubr [72]. As pointed out in [48]
these theorems classify six-manifolds up to diffeomorphisms and the
classification data match the content of the massless sector of the
compactification, discarding exact and co-exact $p$-forms. Our results are not
in tension with this classification, because we need to endow $X_{n}$ with a
$G$-structure metric in order to extract the torsion cohomology data. This
choice of metric also specifies the set of calibrated cycles and the light
spectrum of the Laplacian, so it is crucial in order to select those exact and
co-exact forms that contain the torsion information. In this light, it would
be interesting to see if the presence of torsion in cohomology restricts the
choice of $G$-structure metrics on a manifold, or if one can always choose a
$G$-structure metric where all the smeared delta forms of calibrated torsion
cycles vanish.

An important part of our analysis is based on constructing explicit examples
of SU(3)-structure manifolds with calibrated torsion cycles. This allowed us
to perform a direct comparison of the torsion linking number and its smeared
version, where we observed a remarkable cancellation between terms in the
eigenmode expansion of the delta form, that is reminiscent of the computations
of topological indices. It would be very interesting to understand the meaning
of this feature and if it is also realised in more involved setups, providing
further evidence of Conjecture 1. Our explicit constructions also provided
concrete EFT descriptions of BPS configurations of branes ending on branes,
like the 4d Aharanov-Bohm strings stretching between a domain wall and a
monopole in ${\cal N}=2$ gauged supergravities. Moreover, the properties of
such objects resulted in a physical interpretation of the Hitchin flow
equations, which could be useful to further understand the properties of these
subtle objects. It is likely that this connection sheds light into the physics
of ${\cal N}=2$ gauged supergravities, like for instance when applied to the
black hole supergravity solutions recently revisited in [73], and which share
many properties with AB strings and particles.

As a direct application of our proposal, we have revisited the dimensional
reduction framework developed in [29, 30, 31, 32] to furnish it with one of
its main missing elements. That is, a geometric prescription to define the
basis of $p$-forms in which the RR potentials and the calibration forms must
be expanded. We have verified that our definition fits perfectly with the
physical properties that these forms should have, and which define the
periodicity properties of massive axions and gauge bosons of the 4d ${\cal
N}=2$ EFT. Such periodicities are crucial to define the global gauge
transformations for massive $p$-form in more general setups. This applies in
particular to 4d ${\cal N}=1$ compactifications, where the Stückelberg-like
couplings related to our method involve the gauging of three- and four-forms
in 4d.

To sum up, our findings seem to point out that torsion in cohomology could
lead to specific, measurable physics in the massive sector of 4d EFTs obtained
from string theory. It could be that exploiting this new link between geometry
and physics could give us a new, more approachable understanding of the subtle
objects that are torsion $p$-cycles.

Acknowledgments

We would like to thank Iñaki García-Etxebarria, Luca Martucci, Luca Melotti,
Miguel Montero, David Prieto and Ángel Uranga for discussions. This work is
supported through the grants CEX2020-001007-S and PID2021-123017NB-I00, funded
by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe. G.F.C.
is supported by the grant PRE2021-097279 funded by MCIN/AEI/
10.13039/501100011033 and by ESF+. M.Z. is supported by the fellowship
LCF/BQ/DI20/11780035 from “La Caixa” Foundation (ID 100010434).

##  Appendix A NS5-branes and domain wall solutions

In [29] a 4d domain-wall solution based on a backreacted N5-brane is
considered, in order to arrive at a background of the form (4.1) via mirror
symmetry. In this appendix we review the NS5-brane setup, and describe a set
of type IIB D-branes that are mutually BPS with the NS5-brane. Such D-branes
have a well-defined interpretation from the 4d domain-wall viewpoint, and are
in one-to-one correspondence with the type IIA D-brane objects discussed in
section 4.

Following [29], let us consider a toroidal compactification of type IIB string
theory to 4d. Let us call the three complex coordinates of ${\bf T}^{6}=({\bf
T}^{2})_{1}\times({\bf T}^{2})_{2}\times({\bf T}^{2})_{3}$ as
$dz^{j}=dx^{j}+idx^{j+3}$, $j=1,2,3$. Then one places $M$ NS5-branes along the
three-cycle $(1,0)_{1}(1,0)_{2}(0,1)_{3}$ (that is, the coordinates
$\\{x^{1},x^{2},x^{6}\\}$) and spanning two spatial dimensions in
$\mathbb{R}^{1,3}$. One then backreacts such NS5-branes and smears the
solution down to 4d. In this approximation the harmonic function $V$ that
describes an NS5-brane backreaction becomes a linear function of its
transverse coordinate in 4d, therefore $V=1-\zeta\xi$ with $\zeta$ a real
constant.131313See footnote 5 for an explanation of different choice of $V$
compared to [29]. One obtains the background

$\displaystyle ds^{2}$ $\displaystyle=$ $\displaystyle
ds^{2}_{\mathbb{R}^{1,2}}+\ell_{s}^{2}V(d\xi)^{2}+\ell_{s}^{2}ds^{2}_{{\bf
T}^{6}},$ (A.1) $\displaystyle ds^{2}_{{\bf T}^{6}}$ $\displaystyle=$
$\displaystyle(2\pi)^{2}\left[(r_{1}dx^{1})^{2}+(r_{2}dx^{2})^{2}+V(r_{3}dx^{3})^{2}+V(r_{4}dx^{4})^{2}+V(r_{5}dx^{5})^{2}+(r_{6}dx^{6})^{2}\right],$
(A.2) $\displaystyle H$ $\displaystyle=$ $\displaystyle-Mdx^{4}\wedge
dx^{5}\wedge dx^{3},$ (A.3) $\displaystyle e^{2\phi}$ $\displaystyle=$
$\displaystyle e^{2\phi_{0}}V.$ (A.4)

with $M\in\mathbb{N}$. Upon three T-dualities along the coordinates
$\\{x^{1},x^{2},x^{3}\\}$ one is led to a type IIA background with constant
dilaton, no $H$-flux, and an internal metric of the form (4.2), more precisely
with $M_{3}=M$ and $R_{j}=1/r_{j}$ for $j=1,2,3$.

This smeared solution is interpreted as the long-wavelength description of the
$M$ NS5-branes’ backreaction, more precisely as the 4d domain-wall solution
that is perceived at wavelengths much larger than the compactification radii.
As such, a D-brane that is BPS in the microscopic background should also be so
in its long-wavelength approximation. In practice, this means that if we
consider D-branes that are mutually BPS with the NS5-branes sourcing the
solution, they should correspond to BPS objects in the 4d domain-wall
background. To match out discussion with that of section 4, we will consider
type IIB D-branes whose embedding survives the
$\mathbb{Z}_{2}\times\mathbb{Z}_{2}$ orbifold projection with generators
$\theta_{1}:(x^{1},x^{2},x^{3},x^{4},x^{5},x^{6})\mapsto(x^{1},-x^{2},-x^{3},x^{4},-x^{5},-x^{6})$
and
$\theta_{2}:(x^{1},x^{2},x^{3},x^{4},x^{5},x^{6})\mapsto(-x^{1},-x^{2},-x^{3},-x^{4},-x^{5},x^{6})$.

For instance, D1 and D5-branes are mutually supersymmetric with respect to an
NS5-brane if the system has $2+4k$ ND directions.141414In an abuse of
language, we are borrowing the nomenclature used for configurations of pairs
of D-branes. We may then consider

  * -

A D5-brane wrapping $(T^{2})_{i}\times(T^{2})_{j}$ and extended along $\xi$
$\Longrightarrow$ 6 ND directions.

  * -

A D1-brane extended along $\xi$ $\Longrightarrow$ 6 ND directions.

  * -

An Euclidean D5-brane wrapped on ${\bf T}^{6}$ $\Longrightarrow$ 6 ND
directions.

Additionally, a D3-brane is mutually BPS with an NS5-brane if the system has
$4k$ ND directions. So for instance we can have:

  * -

A D3-brane wrapping $(0,1)_{i}(0,1)_{j}(1,0)_{k}$ and point-like along $\xi$
$\Longrightarrow$ 4 or 8 ND directions.

  * -

An Euclidean D3-brane on $(0,1)_{i}(1,0)_{j}(1,0)_{k}$ and along $\xi$
$\Longrightarrow$ 4 or 8 ND directions.

All these objects are mutually BPS with the NS5-brane sources. If their
orientation is reversed, they will still be BPS in the NS5-brane background,
but preserving a disjoint set of supercharges.

Upon three T-dualities along $\\{x^{1},x^{2},x^{3}\\}$, these D-branes are
mapped to some of the type IIA D-branes discussed in section 4. More
precisely, the D1-brane becomes a D4-brane wrapped on $\Pi_{3}^{\rm tor}$ and
the Euclidean D5-brane is mapped to the Euclidean D2-brane wrapping the three-
chain $\Sigma_{3}$. Similarly, the D3-branes become the D4-branes on the four-
chains $\Sigma_{4}^{i}$ and the Euclidean D3-branes become the Euclidean
D2-branes wrapped on $\Pi_{2}^{i}$. The configurations of AB strings and
particles ending on monopoles and instantons described in section 4 have a
clear microscopic origin in this dual type IIB setup. For instance, a D3-brane
wrapped on $(0,1)_{1}(0,1)_{2}(1,0)_{3}$ intersects the NS5-brane at a point
in ${\bf T}^{6}$ and is pointlike in 4d, by a Hanany-Witten effect (see e.g.
[12, Appendix B]) it must have $M$ D1-branes stretched along $\xi$ between the
NS5-branes and its 4d location. The same occurs in the smeared description
(A.1), where the Freed-Witten anomaly induced by the $H$-flux on such a
D3-brane is cured by the same stack of $M$ D1-branes.

##  Appendix B Massive $p$-from spectra in twisted tori

The aim of this appendix is to provide the explicit $p$-form eigenforms and
eigenvalues of the twisted tori $\tilde{\bf T}^{3}$, whose $p$-form spectrum
was discussed in section 5 based on the method of [61]. Furthermore, we
compare the results with the analyses carried out in [62], to show that the
said spectrum is complete.

Consider the three-dimensional twisted torus $\tilde{\bf T}^{3}$ with metric

$ds^{2}_{\tilde{\bf
T}^{3}}=(2\pi)^{2}\left[(R_{i}\eta^{i})^{2}+(R_{j+3}\eta^{j+3})^{2}+(R_{k+3}\eta^{k+3})^{2}\right]\,,$
(B.1)

and twist $d\eta^{i}=-N\eta^{j+3}\wedge\eta^{k+3}$. We choose angular
coordinates $x^{a}\in[0,1]$ such that the $\eta^{a}$ are parameterized as

$\eta^{i}=dx^{i}+Nx^{k+3}dx^{j+3},\hskip 10.00002pt\eta^{j+3}=dx^{j+3},\hskip
10.00002pt\eta^{{k+3}}=dx^{{k+3}},$ (B.2)

with $N=2\pi\lambda_{\chi}R_{k+3}R_{j+3}/R_{i}$ and
$\lambda_{\chi}\in\mathbb{R}$.

Then, we take the one-form Killing vector $\chi=2\pi R_{i}\eta^{i}/\sqrt{V}$,
which satisfies the assumed properties presented in section 5:

$\star
d\chi=\lambda_{\chi}\,\chi\,,\qquad\Delta_{3}\chi=\lambda_{\chi}^{2}\,\chi\,,\qquad\chi^{2}=1\,,\qquad\lambda_{\chi},\in\mathbb{R}\,,$
(B.3)

where $V=8\pi^{3}R_{i}R_{j+3}R_{k+3}$ is the volume of $\tilde{\bf T}^{3}$.

Let $\\{\phi_{i}\\}$ represent the basis of complex scalar eigenforms of the
Laplacian. The explicit shape of these eigenforms is [62]

$\displaystyle\phi_{p,q}=\frac{e^{2\pi iqx^{j+3}}e^{2\pi
ipx^{k+3}}}{\sqrt{V}},$ (B.4) $\displaystyle\phi_{k,l,n}=\sqrt{\frac{2\pi
R_{j+3}}{|N|V}}\frac{1}{\sqrt{2^{n}n!\sqrt{\pi}}}e^{2\pi
ik(x^{i}+Nx^{k+3}x^{j+3})}e^{2\pi lx^{k+3}}\sum_{m\in\mathbb{Z}}e^{2\pi
ikmx^{k+3}}\Phi_{n}^{\lambda}(\omega_{m}),$ (B.5)

with eigenvalues

$\displaystyle\sigma^{2}_{p,q}=\frac{q^{2}}{R^{2}_{j+3}}+\frac{p^{2}}{R^{2}_{k+3}},$
(B.6)
$\displaystyle\sigma^{2}_{k,l,n}=\frac{k^{2}}{R_{i}^{2}}+(2n+1)k\,\frac{|\lambda_{\chi}|}{R_{i}},$
(B.7)

where we have defined $\lambda=k\lambda_{\chi}/R_{i}$, $\omega_{m}=2\pi
R_{j+3}(x^{j+3}+\frac{m}{N}+\frac{l}{kN})$ and
$\Phi_{n}^{\lambda}(z)=|\lambda|^{1/4}\Phi_{n}(|\lambda|^{1/2}z)$ with
$\Phi_{n}$ being the Hermite polynomial. The integers have the following
ranges

$p,q\in\mathbb{Z}^{2},\,\,\,k\in\mathbb{Z}/\\{0\\},\,\,\,n\in\mathbb{N},\,\,\,l=0,1,\dots,|k|-1.$
(B.8)

Moreover, the exact one-forms are given by the complete set $\\{d\phi_{i}\\}$.
Based on the approach presented in section 5, we observe that the sets of co-
exact one-forms, denoted by $S_{i}$ and $T_{i}$ and defined in equation (5.5),
exhibit closure properties when subjected to the $\star d$ operator.
Substituting the scalar eigenforms (B.4) and (B.5) into the sets $S_{i}$ and
$T_{i}$, we obtain the eigenforms $U_{i}^{\pm}$ defined in (5.7). Also, the
corresponding eigenvalues $\lambda^{\pm}_{i}$ can be obtained from (5.8).

Using the first tower of scalar eigenforms, $\phi_{p,q}$, we acquire the
following results:

$\text{For}\,\,p,q\in\mathbb{Z}^{2}/\\{0,0\\}:$

$\displaystyle
U^{\pm}_{p,q}=\frac{2\pi\,\phi_{p,q}}{\sqrt{(\lambda^{\pm}_{p,q})^{2}+\sigma_{p,q}^{2}}}\left(\frac{(\lambda^{\pm}_{p,q})^{2}-\sigma_{p,q}^{2}}{i\lambda_{\chi}}\,\,R_{i}\,\eta^{i}-\frac{R_{j+3}}{R_{k+3}}\,p\,\eta^{j+3}+\frac{R_{k+3}}{R_{j+3}}\,q\,\eta^{k+3}\right),$
$\displaystyle(\lambda^{\pm}_{p,q})^{2}=\frac{q^{2}}{R^{2}_{j+3}}+\frac{p^{2}}{R^{2}_{k+3}}+\frac{\lambda_{\chi}^{2}}{2}\pm\frac{\lambda_{\chi}}{2}\sqrt{\lambda_{\chi}^{2}+\frac{4\,q^{2}}{R^{2}_{j+3}}+\frac{4\,p^{2}}{R^{2}_{k+3}}}\,.$
(B.9)

For $p=q=0$:

$\displaystyle U^{-}=0,\hskip 10.00002ptU^{+}_{0,0}=\frac{2\pi
R_{i}}{\sqrt{V}}\,\eta^{i},$ (B.10)
$\displaystyle\lambda^{+}_{0,0}=\lambda_{\chi}^{2}.$ (B.11)

Comparing with the analysis carried out in [62] one can see that the above
results reproduce exactly half of the co-exact one-form spectrum (see [62],
eq. (2.34) and Table 1).

Finally, for the second type of eigen-scalars, $\phi_{k,l,n}$, due to its very
intricate expression (see eq. (2.59) from [62] for details), let us focus
exclusively on the resulting eigenvalues and their degeneracy. Substituting
again in (5.8) we obtain

$(\lambda_{k,l,n}^{\pm})^{2}=\frac{k^{2}}{R_{i}^{2}}+\frac{(2n+1)k|\lambda_{\chi}|}{R_{i}}+\frac{\lambda_{\chi}^{2}}{2}\pm\frac{\lambda_{\chi}}{2}\sqrt{\lambda_{\chi}^{2}+\frac{4k^{2}}{R_{i}^{2}}+\frac{4(2n+1)k|\lambda_{\chi}|}{R_{i}}},\,\,\,n\in\mathbb{N}^{*}.$
(B.12)

Therefore, we observe an exact correspondence with the remaining spectrum. It
is important to note that we have shifted $n$ to $n+1$ in (B.12). This is
because the solution (B.12) with $n=0$ is in fact trivial using our method,
and the matching with [62] occurs at our $n\geq 1$.

## References

  * [1] K. Becker, M. Becker and J. H. Schwarz, _String theory and M-theory: A modern introduction_. Cambridge University Press, 12, 2006, 10.1017/CBO9780511816086.
  * [2] L. E. Ibanez and A. M. Uranga, _String theory and particle physics: An introduction to string phenomenology_. Cambridge University Press, 2, 2012.
  * [3] R. Blumenhagen, D. Lüst and S. Theisen, _Basic concepts of string theory_ , Theoretical and Mathematical Physics. Springer, Heidelberg, Germany, 2013, 10.1007/978-3-642-29497-6.
  * [4] D. Baumann and L. McAllister, _Inflation and String Theory_ , Cambridge Monographs on Mathematical Physics. Cambridge University Press, 5, 2015, 10.1017/CBO9781316105733, [1404.2601].
  * [5] A. Tomasiello, _Geometry of String Theory Compactifications_. Cambridge University Press, 1, 2022, 10.1017/9781108635745.
  * [6] R. Harvey and H. B. Lawson, Jr., _Calibrated geometries_ , _Acta Math._ 148 (1982) 47.
  * [7] L. Martucci and P. Smyth, _Supersymmetric D-branes and calibrations on general N=1 backgrounds_ , _JHEP_ 11 (2005) 048 [hep-th/0507099].
  * [8] P. Koerber, _Stable D-branes, calibrations and generalized Calabi-Yau geometry_ , _JHEP_ 08 (2005) 099 [hep-th/0506154].
  * [9] P. Koerber, _Lectures on Generalized Complex Geometry for Physicists_ , _Fortsch. Phys._ 59 (2011) 169 [1006.1536].
  * [10] T. Banks and N. Seiberg, _Symmetries and Strings in Field Theory and Gravity_ , _Phys. Rev. D_ 83 (2011) 084019 [1011.5120].
  * [11] M. Berasaluce-Gonzalez, L. E. Ibanez, P. Soler and A. M. Uranga, _Discrete gauge symmetries in D-brane models_ , _JHEP_ 12 (2011) 113 [1106.4169].
  * [12] M. Berasaluce-Gonzalez, P. G. Camara, F. Marchesano and A. M. Uranga, _Zp charged branes in flux compactifications_ , _JHEP_ 04 (2013) 138 [1211.5317].
  * [13] S. Gukov, M. Rangamani and E. Witten, _Dibaryons, strings and branes in AdS orbifold models_ , _JHEP_ 12 (1998) 025 [hep-th/9811048].
  * [14] E. Witten, _AdS / CFT correspondence and topological field theory_ , _JHEP_ 12 (1998) 012 [hep-th/9812012].
  * [15] P. G. Camara, L. E. Ibanez and F. Marchesano, _RR photons_ , _JHEP_ 09 (2011) 110 [1106.0060].
  * [16] T. W. Grimm, M. Kerstan, E. Palti and T. Weigand, _Massive Abelian Gauge Symmetries and Fluxes in F-theory_ , _JHEP_ 12 (2011) 004 [1107.3842].
  * [17] M. Berasaluce-Gonzalez, P. G. Camara, F. Marchesano, D. Regalado and A. M. Uranga, _Non-Abelian discrete gauge symmetries in 4d string models_ , _JHEP_ 09 (2012) 059 [1206.2383].
  * [18] C. Mayrhofer, E. Palti, O. Till and T. Weigand, _On Discrete Symmetries and Torsion Homology in F-Theory_ , _JHEP_ 06 (2015) 029 [1410.7814].
  * [19] T. W. Grimm, T. G. Pugh and D. Regalado, _Non-Abelian discrete gauge symmetries in F-theory_ , _JHEP_ 02 (2016) 066 [1504.06272].
  * [20] V. Braun, M. Cvetic, R. Donagi and M. Poretschkin, _Type II String Theory on Calabi-Yau Manifolds with Torsion and Non-Abelian Discrete Gauge Symmetries_ , _JHEP_ 07 (2017) 129 [1702.08071].
  * [21] R. Bott and L. W. Tu, _Differential forms in algebraic topology_ , in _Graduate texts in mathematics_ , 1982.
  * [22] F. Apruzzi, F. Bonetti, I. n. G. Etxebarria, S. S. Hosseini and S. Schafer-Nameki, _Symmetry TFTs from String Theory_ , 2112.02092.
  * [23] W. D. Goldberger and M. B. Wise, _Renormalization group flows for brane couplings_ , _Phys. Rev. D_ 65 (2002) 025011 [hep-th/0104170].
  * [24] B. Michel, E. Mintun, J. Polchinski, A. Puhm and P. Saad, _Remarks on brane and antibrane dynamics_ , _JHEP_ 09 (2015) 021 [1412.5702].
  * [25] J. Polchinski, _Brane/antibrane dynamics and KKLT stability_ , 1509.05710.
  * [26] S. Lanza, F. Marchesano, L. Martucci and I. Valenzuela, _Swampland Conjectures for Strings and Membranes_ , _JHEP_ 02 (2021) 006 [2006.15154].
  * [27] S. Lanza, F. Marchesano, L. Martucci and I. Valenzuela, _The EFT stringy viewpoint on large distances_ , _JHEP_ 09 (2021) 197 [2104.05726].
  * [28] S. Lanza, F. Marchesano, L. Martucci and I. Valenzuela, _Large Field Distances from EFT strings_ , _PoS_ CORFU2021 (2022) 169 [2205.04532].
  * [29] S. Gurrieri, J. Louis, A. Micu and D. Waldram, _Mirror symmetry in generalized Calabi-Yau compactifications_ , _Nucl. Phys. B_ 654 (2003) 61 [hep-th/0211102].
  * [30] R. D’Auria, S. Ferrara, M. Trigiante and S. Vaula, _Gauging the Heisenberg algebra of special quaternionic manifolds_ , _Phys. Lett. B_ 610 (2005) 147 [hep-th/0410290].
  * [31] M. Grana, J. Louis and D. Waldram, _Hitchin functionals in N=2 supergravity_ , _JHEP_ 01 (2006) 008 [hep-th/0505264].
  * [32] A.-K. Kashani-Poor and R. Minasian, _Towards reduction of type II theories on SU(3) structure manifolds_ , _JHEP_ 03 (2007) 109 [hep-th/0611106].
  * [33] A. Gray and L. M. Hervella, _The sixteen classes of almost hermitian manifolds and their linear invariants_ , _Annali di Matematica Pura ed Applicata_ 123 (1980) 35.
  * [34] S. Chiossi and S. Salamon, _The intrinsic torsion of su(3) and $g_{2}$ structures_, 2002.
  * [35] C. M. Hull, _SUPERSTRING COMPACTIFICATIONS WITH TORSION AND SPACE-TIME SUPERSYMMETRY_ , in _First Torino Meeting on Superunification and Extra Dimensions_ , 3, 1986.
  * [36] A. Strominger, _Superstrings with Torsion_ , _Nucl. Phys. B_ 274 (1986) 253.
  * [37] G. Lopes Cardoso, G. Curio, G. Dall’Agata, D. Lust, P. Manousselis and G. Zoupanos, _NonKahler string backgrounds and their five torsion classes_ , _Nucl. Phys. B_ 652 (2003) 5 [hep-th/0211118].
  * [38] J. P. Gauntlett, D. Martelli and D. Waldram, _Superstrings with intrinsic torsion_ , _Phys. Rev. D_ 69 (2004) 086002 [hep-th/0302158].
  * [39] M. Grana, _Flux compactifications in string theory: A Comprehensive review_ , _Phys. Rept._ 423 (2006) 91 [hep-th/0509003].
  * [40] G. T. Horowitz and M. Srednicki, _A Quantum Field Theoretic Description of Linking Numbers and Their Generalization_ , _Commun. Math. Phys._ 130 (1990) 83.
  * [41] N. J. Hitchin, _Lectures on special Lagrangian submanifolds_ , _AMS/IP Stud. Adv. Math._ 23 (2001) 151 [math/9907034].
  * [42] F. Marchesano, D. Regalado and G. Zoccarato, _On D-brane moduli stabilisation_ , _JHEP_ 11 (2014) 097 [1410.0209].
  * [43] F. Marchesano, D. Regalado and G. Zoccarato, _U(1) mixing and D-brane linear equivalence_ , _JHEP_ 08 (2014) 157 [1406.2729].
  * [44] J. Blaback, U. H. Danielsson, D. Junghans, T. Van Riet, T. Wrase and M. Zagermann, _Smeared versus localised sources in flux compactifications_ , _JHEP_ 12 (2010) 043 [1009.1877].
  * [45] F. Marchesano, G. Shiu and A. M. Uranga, _F-term Axion Monodromy Inflation_ , _JHEP_ 09 (2014) 184 [1404.3040].
  * [46] S. Lanza, F. Marchesano, L. Martucci and D. Sorokin, _How many fluxes fit in an EFT?_ , _JHEP_ 10 (2019) 110 [1907.11256].
  * [47] J. Evslin and L. Martucci, _D-brane networks in flux vacua, generalized cycles and calibrations_ , _JHEP_ 07 (2007) 040 [hep-th/0703129].
  * [48] A. Tomasiello, _Topological mirror symmetry with fluxes_ , _JHEP_ 06 (2005) 067 [hep-th/0502148].
  * [49] F. Marchesano, _D6-branes and torsion_ , _JHEP_ 05 (2006) 019 [hep-th/0603210].
  * [50] A. Hebecker, P. Henkenjohann and L. T. Witkowski, _What is the Magnetic Weak Gravity Conjecture for Axions?_ , _Fortsch. Phys._ 65 (2017) 1700011 [1701.06553].
  * [51] B. Heidenreich, J. McNamara, M. Montero, M. Reece, T. Rudelius and I. Valenzuela, _Chern-Weil global symmetries and how quantum gravity avoids them_ , _JHEP_ 11 (2021) 053 [2012.00009].
  * [52] S. Kachru, M. B. Schulz, P. K. Tripathy and S. P. Trivedi, _New supersymmetric string compactifications_ , _JHEP_ 03 (2003) 061 [hep-th/0211182].
  * [53] G. Curio, A. Klemm, D. Lust and S. Theisen, _On the vacuum structure of type II string compactifications on Calabi-Yau spaces with H fluxes_ , _Nucl. Phys. B_ 609 (2001) 3 [hep-th/0012213].
  * [54] K. Behrndt, S. Gukov and M. Shmakova, _Domain walls, black holes, and supersymmetric quantum mechanics_ , _Nucl. Phys. B_ 601 (2001) 49 [hep-th/0101119].
  * [55] K. Behrndt, G. Lopes Cardoso and D. Lust, _Curved BPS domain wall solutions in four-dimensional N=2 supergravity_ , _Nucl. Phys. B_ 607 (2001) 391 [hep-th/0102128].
  * [56] K. Nomizu, _On the cohomology of compact homogeneous spaces of nilpotent lie groups_ , _Annals of Mathematics_ 59 (1954) 531.
  * [57] B. Cenkl and R. D. Porter, _Nilmanifolds and associated lie algebras over the integers_ , _Pacific Journal of Mathematics_ 193 (2000) 5\.
  * [58] A. Hanany and E. Witten, _Type IIB superstrings, BPS monopoles, and three-dimensional gauge dynamics_ , _Nucl. Phys. B_ 492 (1997) 152 [hep-th/9611230].
  * [59] C. Mayer and T. Mohaupt, _Domain walls, Hitchin’s flow equations and G(2)-manifolds_ , _Class. Quant. Grav._ 22 (2005) 379 [hep-th/0407198].
  * [60] N. Hitchin, _Stable forms and special metrics_ , 2001.
  * [61] J. B. Achour, E. Huguet, J. Queva and J. Renaud, _Explicit vector spherical harmonics on the 3-sphere_ , _Journal of Mathematical Physics_ 57 (2016) 023504.
  * [62] D. Andriot and D. Tsimpis, _Laplacian spectrum on a nilmanifold, truncations and effective theories_ , _JHEP_ 09 (2018) 096 [1806.05156].
  * [63] M. Grana, J. Louis and D. Waldram, _SU(3) x SU(3) compactification and mirror duals of magnetic fluxes_ , _JHEP_ 04 (2007) 101 [hep-th/0612237].
  * [64] J. Polchinski, _Monopoles, duality, and string theory_ , _Int. J. Mod. Phys. A_ 19S1 (2004) 145 [hep-th/0304042].
  * [65] D. Lust, F. Marchesano, L. Martucci and D. Tsimpis, _Generalized non-supersymmetric flux vacua_ , _JHEP_ 11 (2008) 021 [0807.4540].
  * [66] A. Tomasiello, _Reformulating supersymmetry with a generalized Dolbeault operator_ , _JHEP_ 02 (2008) 010 [0704.2613].
  * [67] G. Villadoro and F. Zwirner, _N=1 effective potential from dual type-IIA D6/O6 orientifolds with general fluxes_ , _JHEP_ 06 (2005) 047 [hep-th/0503169].
  * [68] P. G. Camara, A. Font and L. E. Ibanez, _Fluxes, moduli fixing and MSSM-like vacua in a simple IIA orientifold_ , _JHEP_ 09 (2005) 013 [hep-th/0506066].
  * [69] G. F. Casas, F. Marchesano and D. Prieto, _Membranes in AdS 4 orientifold vacua and their Weak Gravity Conjecture_, _JHEP_ 09 (2022) 034 [2204.11892].
  * [70] W. Buchmuller, C. Wieck and M. W. Winkler, _Supersymmetric Moduli Stabilization and High-Scale Inflation_ , _Phys. Lett. B_ 736 (2014) 237 [1404.2275].
  * [71] C. WALL, _Classification problems in differential topology. v. on certain 6-manifolds._ , _Inventiones mathematicae_ 1 (1966) 355.
  * [72] A. V. Zhubr, _Classification of simply connected six-dimensional spin manifolds_ , _Journal of Soviet Mathematics_ 10 (1978) 451.
  * [73] R. Angius, J. Huertas and A. M. Uranga, _Small black hole explosions_ , _JHEP_ 06 (2023) 070 [2303.15903].