David<EMAIL_ADDRESS>Menelaos
<EMAIL_ADDRESS>Stamatios
<EMAIL_ADDRESS>Luc Van<EMAIL_ADDRESS>Computer Vision Lab
ETH Zurich
Switzerland Branched Multi-Task Networks

# Automated Search for Resource-Efficient Branched Multi-Task Networks

###### Abstract

The multi-modal nature of many vision problems calls for neural network
architectures that can perform multiple tasks concurrently. Typically, such
architectures have been handcrafted in the literature. However, given the size
and complexity of the problem, this manual architecture exploration likely
exceeds human design abilities. In this paper, we propose a principled
approach, rooted in differentiable neural architecture search, to
automatically define branching (tree-like) structures in the encoding stage of
a multi-task neural network. To allow flexibility within resource-constrained
environments, we introduce a proxyless, resource-aware loss that dynamically
controls the model size. Evaluations across a variety of dense prediction
tasks show that our approach consistently finds high-performing branching
structures within limited resource budgets.

##  1 Introduction

Over the last decade neural networks have shown impressive results for a
multitude of tasks. This is typically achieved by designing network
architectures that satisfy the specific needs of the task at hand, and
training them in a fully supervised fashion [Girshick et al.(2014)Girshick,
Donahue, Darrell, and Malik, Long et al.(2015)Long, Shelhamer, and Darrell,
Xie and Tu(2015)]. This well-established strategy assumes that each task is
tackled in isolation, and consequently, a dedicated network should be learned
for every task. However, real-world problems are inherently multi-modal (e.g,
an autonomous car should be able to detect road lanes, recognize pedestrians
in its vicinity, semantically understand its surroundings, estimate its
distance from other objects, etc.), which calls for architectures that can
perform multiple tasks simultaneously.

Motivated by these observations researchers started designing multi-task
architectures that, given an input image, can produce predictions for all
desired tasks [Ruder(2017)]. Arguably the most characteristic example is
branched multi-task networks [Kokkinos(2017), Neven et al.(2017)Neven, De
Brabandere, Georgoulis, Proesmans, and Van Gool, Long et al.(2017)Long, Cao,
Wang, and Philip, Kendall et al.(2018)Kendall, Gal, and Cipolla, Chen et
al.(2018b)Chen, Badrinarayanan, Lee, and Rabinovich], where a shared encoding
stage branches out to a set of task-specific heads that decode the shared
features to yield task predictions. Follow-up works [Misra et al.(2016)Misra,
Shrivastava, Gupta, and Hebert, Xu et al.(2018)Xu, Ouyang, Wang, and Sebe, Liu
et al.(2019c)Liu, Johns, and Davison, Gao et al.(2019)Gao, Ma, Zhao, Liu, and
Yuille, Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos, Vandenhende et
al.(2020b)Vandenhende, Georgoulis, and Van Gool] proposed more advanced
mechanisms for multi-task learning. However, despite their advantages, they
similarly assume that the multi-task architecture can be handcrafted prior to
training. In a different vein, some works [Lu et al.(2017)Lu, Kumar, Zhai,
Cheng, Javidi, and Feris, Vandenhende et al.(2020a)Vandenhende, Georgoulis, De
Brabandere, and Van Gool] opted for a semi-automated architecture design where
the branching occurs at finer locations in the encoding stage. Task groupings
at each branching location are determined based on a measure of ‘task
relatedness’ in pre-trained networks. However, defining branching points based
on such offline criteria disregards potential optimization benefits resulting
from jointly learning particular groups of tasks [Bingel and Søgaard(2017)].

Generally, finding an architecture suitable for multi-task learning poses
great challenges, arising from conflicting objectives. On the one hand, a
multi-task network should perform comparably to its single-task counterparts.
This is not trivial as task interference111Task interference is a well-
documented problem in multi-task networks [Kokkinos(2017), Kendall et
al.(2018)Kendall, Gal, and Cipolla, Sener and Koltun(2018), Maninis et
al.(2019)Maninis, Radosavovic, and Kokkinos] where important information for
one task might be a nuisance for another, leading to conflicts in the
optimization of shared parameters. can significantly affect individual task
performance. On the other hand, a multi-task network should remain within a
low computational budget during inference with respect to the single-task
case. The relative importance of task performance vs. computational efficiency
depends on the application, highlighting the necessity of being able to adapt
the architecture design flexibly. This calls for automatic architecture search
techniques to mitigate the effort accompanying architecture handcrafting.

In this paper, we address the aforementioned needs and propose Branched Multi-
Task Architecture Search (BMTAS222Reference code at
https://github.com/brdav/bmtas), a principled approach to automatically
determine encoder branching in multi-task architectures. To avoid a brute-
force or greedy search as in [Vandenhende et al.(2020a)Vandenhende,
Georgoulis, De Brabandere, and Van Gool] and [Lu et al.(2017)Lu, Kumar, Zhai,
Cheng, Javidi, and Feris] respectively, we build a differentiable neural
architecture search algorithm with a search space directly encompassing all
possible branching structures. Our approach is end-to-end trainable, and
allows flexibility in the model construction through the introduction of a
proxyless, resource-aware loss. Experimental analysis on multiple dense
prediction tasks shows that models generated by our method effectively balance
the trade-off between overall performance and computational cost.

##  2 Related Work

Multi-Task Learning (MTL) in deep neural networks addresses the problem of
training a single model that can perform multiple tasks concurrently. To this
end, a first group of works [Misra et al.(2016)Misra, Shrivastava, Gupta, and
Hebert, Ruder et al.(2019)Ruder, Bingel, Augenstein, and Søgaard, Liu et
al.(2019c)Liu, Johns, and Davison, Gao et al.(2019)Gao, Ma, Zhao, Liu, and
Yuille] incorporated feature sharing mechanisms on top of a set of task-
specific networks. Typically, these soft parameter sharing approaches scale
poorly to an increasing number of tasks, as the parameter count usually
exceeds the single-task case. A second group of works [Kokkinos(2017), Neven
et al.(2017)Neven, De Brabandere, Georgoulis, Proesmans, and Van Gool, Long et
al.(2017)Long, Cao, Wang, and Philip, Kendall et al.(2018)Kendall, Gal, and
Cipolla, Chen et al.(2018b)Chen, Badrinarayanan, Lee, and Rabinovich] shared
the majority of network operations, before branching out to a set of task-
specific heads that produce the task predictions. In these hard parameter
sharing approaches the branching point is manually determined prior to
training, which can lead to task interference if ‘unrelated’ (groups of) tasks
are selected.

Follow-up works employed different techniques to address the MTL problem. PAD-
Net [Xu et al.(2018)Xu, Ouyang, Wang, and Sebe] and MTI-Net [Vandenhende et
al.(2020b)Vandenhende, Georgoulis, and Van Gool] proposed to refine the
initial task predictions by distilling cross-task information. Dynamic task
prioritization [Guo et al.(2018)Guo, Haque, Huang, Yeung, and Fei-Fei]
constructed a hierarchical network that dynamically prioritizes the learning
of ‘easy’ tasks at earlier layers. ASTMT [Maninis et al.(2019)Maninis,
Radosavovic, and Kokkinos] and RCM [Kanakis et al.(2020)Kanakis, Bruggemann,
Saha, Georgoulis, Obukhov, and Van Gool] adopted a task-conditional approach,
where only one task is forward-propagated at a time, to sequentially generate
all task predictions. All the above mentioned approaches assume that the
multi-task architecture can be handcrafted. However, given the size and
complexity of the problem, this manual exploration likely exceeds human design
abilities. In contrast, stochastic filter groups [Bragman et al.(2019)Bragman,
Tanno, Ourselin, Alexander, and Cardoso] re-purposed the convolution kernels
in each layer to support shared or task-specific behavior, but their method
only operates at the channel level of the network.

Our work is closer to [Lu et al.(2017)Lu, Kumar, Zhai, Cheng, Javidi, and
Feris, Vandenhende et al.(2020a)Vandenhende, Georgoulis, De Brabandere, and
Van Gool] where branching structures are generated in the encoding stage of
the network. However, instead of determining the task groupings at each
branching location in pre-trained networks via a brute-force or greedy
algorithm, we propose a principled solution rooted in differentiable neural
architecture search that is trainable end-to-end.

Neural Architecture Search (NAS) aims to automate the procedure of designing
neural network architectures in contrast to the established protocol of
manually drafting them based on prior knowledge. To achieve this, a first
group of works [Zoph and Le(2017), Zoph et al.(2018)Zoph, Vasudevan, Shlens,
and Le, Pham et al.(2018)Pham, Guan, Zoph, Le, and Dean] used a recurrent
neural network to sample architectures from a pre-defined search space and
trained it with reinforcement learning to maximize the expected accuracy on a
validation set. A second group of works [Real et al.(2017)Real, Moore, Selle,
Saxena, Suematsu, Tan, Le, and Kurakin, Real et al.(2019)Real, Aggarwal,
Huang, and Le] employed evolutionary algorithms to gradually evolve a
population of models through mutations. Despite their great success, these
works are computationally demanding during the search phase, which motivated
researchers to explore differentiable NAS [Liu et al.(2019b)Liu, Simonyan, and
Yang] through continuous relaxation of the architecture representation. SNAS
[Xie et al.(2019)Xie, Zheng, Liu, and Lin] built upon Gumbel-Softmax [Jang et
al.(2017)Jang, Gu, and Poole, Maddison et al.(2017)Maddison, Mnih, and Teh] to
propose an effective search gradient. Follow-up works [Cai et al.(2019)Cai,
Zhu, and Han, Wu et al.(2019)Wu, Dai, Zhang, Wang, Sun, Wu, Tian, Vajda, Jia,
and Keutzer] introduced resource constraints into the optimization pipeline to
control the model size.

In general, NAS works have mainly focused on the image classification task,
with a few exceptions that addressed other tasks too (e.g, semantic
segmentation [Liu et al.(2019a)Liu, Chen, Schroff, Adam, Hua, Yuille, and Fei-
Fei]). Moreover, each task is tackled in isolation, i.e., a dedicated
architecture is generated for each task. When it comes to MTL however, there
is a need to design architectures that perform well across a variety of tasks.
Routing networks [Rosenbaum et al.(2018)Rosenbaum, Klinger, and Riemer] made a
first attempt in this direction by determining the connectivity of a network’s
function blocks through routing, but they focus on classification tasks, as
opposed to the more challenging dense prediction considered in this paper.
MTL-NAS [Gao et al.(2020)Gao, Bai, Jie, Ma, Jia, and Liu] extended NDDR-CNN
[Gao et al.(2019)Gao, Ma, Zhao, Liu, and Yuille] to automatically find the
feature fusion mechanisms on top of the task-specific networks, but their
search is solely limited to the feature sharing mechanisms. In contrast, we
automatically generate branching (or tree-like) structures in the encoding
stage of the network, which allows more explicit control over the size of the
final model.

##  3 Method

Given a set of dense prediction tasks and an arbitrary neural architecture,
our goal is to find resource-efficient branching structures in the encoder,
that promote the sharing of general-purpose features between tasks and the
decoupling of task-specific features across tasks. In this section, we
elaborate on three key components of the proposed BMTAS, which builds upon
differentiable NAS to tackle the aforementioned objective: the structure of
the search space (Sec. 3.1), the algorithm to traverse that search space (Sec.
3.2), and a novel objective function to enforce resource efficiency (Sec.
3.3).

###  3.1 Search Space

In contrast to most established NAS works [Zoph and Le(2017), Zoph et
al.(2018)Zoph, Vasudevan, Shlens, and Le, Pham et al.(2018)Pham, Guan, Zoph,
Le, and Dean, Liu et al.(2019b)Liu, Simonyan, and Yang] which only search for
subcomponents (e.g, a cell) of the target architecture, our search space
directly encompasses all possible branching structures for a given number of
$T$ tasks in an encoder network. As shown in Fig. 1, we can describe our
search space with a directed acyclic graph (cyan box), where the vertices
represent intermediate feature tensors and the edges operations (e.g,
bottleneck blocks for a ResNet-50 backbone). For an encoder with $L$ layers
the graph has length $L$ in total, and width $T$ between consecutive vertices.
Parallel edges denote candidate operations, which are (non-parameter-sharing)
duplicates of the original operation in the respective layer. The operation
parameters of each duplicate block are ‘warmed up’ for a few iterations on the
corresponding task before the architecture search. Through this procedure,
each block is softly assigned to a task. For any one task $t$, a specific
routing (subgraph) through the supergraph can be obtained by sampling
operations with a mask $z^{(t)}\in\\{0,1\\}^{L\times T}$ with one-hot rows
(Fig. 1 center). Any possible branching structure for a set of tasks can then
be produced by combining the task-specific routings. Importantly, computation
sharing in layer $l$ for any two tasks only occurs if their sampled edges
coincide in all layers 1 to $l$. We present a loss function which incentivizes
such computation sharing in Sec. 3.3.

Figure 1: Schematic showing the construction of a branched architecture
(right) from a supergraph (left), in a setting with four tasks. For each of
the tasks A to D, a subgraph is sampled from the supergraph, using the
learnable masks $z^{(\cdot)}$. To form a branching structure, the subgraphs
are combined according to the sampling consensus, yielding task groupings at
each layer. During the architecture search, the masks $z^{(\cdot)}$ are
learned by minimizing a resource loss $\mathcal{L}_{resource}$ (computed using
a look-up table) and task performance losses $\mathcal{L}_{A}$ to
$\mathcal{L}_{D}$ simultaneously.

###  3.2 Search Algorithm

The goal of BMTAS is to find a set of masks $Z$ =
$\\{z^{(t)}|t\in\\{1,...,T\\}\\}$ to sample $T$ subgraphs from the supergraph
described in Sec. 3.1. We model this set with a parameterized distribution
$p_{\alpha}(Z)$, where $\alpha^{(t)}\in\mathbb{R}^{L\times T}$ represents the
unnormalized log probabilities for sampling individual operations for task
$t$. Therefore, the overall optimization problem is:

$\min_{\alpha,\theta}\mathbb{E}_{Z\sim
p_{\alpha}(Z)}\left[\mathcal{L}\left(Z,\theta\right)\right]$ (1)

$\theta$ denotes the neural operation parameters of the supergraph and
$\mathcal{L}$ an appropriate loss function (see Sec. 3.3). By solving this
optimization problem, we are maximizing the expected performance (in
accordance with the chosen $\mathcal{L}$) of branching structures sampled from
$p_{\alpha}(Z)$.

Following [Xie et al.(2019)Xie, Zheng, Liu, and Lin], we can relax the
discrete architecture distribution $p_{\alpha}(Z)$ to be continuous and
differentiable using the gradient estimator proposed in [Jang et
al.(2017)Jang, Gu, and Poole, Maddison et al.(2017)Maddison, Mnih, and Teh]:

$z_{l,j}^{(t)}=\frac{\exp{\left(\left(\alpha_{l,j}^{(t)}+g_{l,j}^{(t)}\right)/\tau\right)}}{\sum_{i=1}^{T}\exp{\left(\left(\alpha_{l,i}^{(t)}+g_{l,i}^{(t)}\right)/\tau\right)}},\quad
j=1,...,T$ (2)

$g_{l,j}^{(t)}\sim\text{{\tt Gumbel}}(0,1)$ is random noise and $\tau>0$ a
temperature parameter. If $\tau$ is very large, the sampling distribution is
nearly uniform, regardless of the $\alpha^{(t)}$ values. This has the
advantage that the resulting gradients in the backpropagation are smooth. For
$\tau$ close to 0, we approach sampling from the categorical distribution
$p_{\alpha}(Z)$. In practice, we gradually anneal $\tau\rightarrow 0$ during
training. Samples from the supergraph are obtained by multiplying the edges in
layer $l$ with the softened one-hot vector $z_{l}^{(t)}$ and summing the
output.

The above function, known as Gumbel-Softmax, enables us to directly learn the
unnormalized log probability masks $\alpha^{(t)}$ through gradient descent for
each task $t$. During training, we alternate between updating the architecture
parameters $\alpha$ and the operation parameters $\theta$, as this empirically
leads to more stable convergence in our case. Afterwards, we discretize
$\alpha^{(t)}$ using argmax to obtain one-hot masks that determine the final
routing. As is common practice [Liu et al.(2019b)Liu, Simonyan, and Yang, Xie
et al.(2019)Xie, Zheng, Liu, and Lin], we retrain the searched architectures
from scratch.

###  3.3 Resource-Aware Objective Function

In the simplest case, the loss function in the overall optimization problem
(Eq. 1) of BMTAS consists of a weighted sum of the task-specific losses:

$\mathcal{L}_{tasks}(Z,\theta)=\sum_{t=1}^{T}\omega_{t}\mathcal{L}_{t}(Z,\theta)$
(3)

Simply searching encoder structures via this objective function yields
performance-oriented branching structures, irrespective of the efficiency of
the resulting model. We show in Sec. 4.3 that the outcome resembles separate
single-task models, i.e., the tasks stop sharing computations and branch out
in early layers, increasing the capacity of the resulting network. To obtain
more compact encoder structures and to actively navigate the efficiency vs.
performance trade-off, we introduce a resource-aware term
$\mathcal{L}_{resource}$ in the objective function:

$\mathcal{L}_{search}=\mathcal{L}_{tasks}+\lambda\mathcal{L}_{resource}$ (4)

The emphasis shifts to resource efficiency in the architecture search as
$\lambda$ is increased. We follow related work [Gordon et al.(2018)Gordon,
Eban, Nachum, Chen, Wu, Yang, and Choi, Molchanov et al.(2017)Molchanov,
Tyree, Karras, Aila, and Kautz, Veniat and Denoyer(2018)] in choosing the
number of multiply-add operations (MAdds) during inference as a surrogate for
resource efficiency. The MAdds $C(Z)$ of a sampled branched architecture $Z$
depends on the present task groupings in each network layer. We define a task
grouping as a partition of the set $\\{1,...,T\\}$, where the parts indicate
computation sharing. For an encoder with $L$ layers and $K$ possible groupings
per layer, the resource objective can be formulated as:

$\mathbb{E}_{Z\sim
p_{\alpha}(Z)}\left[C\left(Z\right)\right]=\sum_{l=1}^{L}\sum_{k=1}^{K}p_{\alpha}(\kappa_{l}=k)c(k,l)$
(5)

where $\kappa_{l}$ is the task grouping at layer $l$, and $c(k,l)$ the MAdds
of grouping $k$ at layer $l$. $c(k,l)$ can be simply determined by a look-up
table, as the computational cost only depends on the layer index and the
number of required operations given the grouping.

In contrast to differentiable NAS works [Xie et al.(2019)Xie, Zheng, Liu, and
Lin, Wu et al.(2019)Wu, Dai, Zhang, Wang, Sun, Wu, Tian, Vajda, Jia, and
Keutzer, Cai et al.(2019)Cai, Zhu, and Han], we cannot calculate
$p_{\alpha}(\kappa_{l}=k)$ for each layer independently, since task groupings
depend on previous layers. As mentioned in Sec. 3.1, two tasks only share
computation in layer $l$ provided that they also do so in all preceding
layers. Thus, for each task grouping $k$, we need to map out the set of valid
ancestor groupings $\mathcal{A}_{k}$. In mathematical terms, $\mathcal{A}_{k}$
contains every partition (i.e., task grouping) of which $k$ is a refinement.
Computation sharing with grouping $k$ in layer $l$ only occurs if the
groupings in every layer 1 to $(l-1)$ are in $\mathcal{A}_{k}$. Considering
this dependency structure, we can decompose $p_{\alpha}(\kappa_{l}=k)$ as a
recursive formulation of conditional probabilities:

$p_{\alpha}(\kappa_{l}=k)=p_{\alpha}(\kappa_{l}=k|\kappa_{l-1},...,\kappa_{1}\in\mathcal{A}_{k})\sum_{m\in\mathcal{A}_{k}}p_{\alpha}(\kappa_{l-1}=m),\quad
l=2,...,L$ (6)

The conditional probabilities
$p_{\alpha}(\kappa_{l}=k|\kappa_{l-1},...,\kappa_{1}\in\mathcal{A}_{k})$ are
independent for each layer and can be easily constructed from the unnormalized
log sampling probabilities $\alpha$.

The derivations presented in this section yield a proxyless resource loss
function, i.e., it encourages solutions which directly minimize the expected
resource cost of the final model. By design, this resource objective function
allows us to find tree-like structures without resorting to curriculum
learning techniques during the architecture search, which would be infeasible
using simpler, indirect constraints (e.g, $\mathcal{L}_{2}$ regularization).

##  4 Experiments

In this section, we first describe the experimental setup (Sec. 4.1), and
consequently evaluate the proposed method on several datasets using different
backbones (Sec. 4.2). We also present ablation studies to validate our search
algorithm (Sec. 4.3), and analyze the resulting task groupings in a case study
(Sec. 4.4).

###  4.1 Experimental Setup

Datasets. We carry out experiments on PASCAL [Everingham et
al.(2010)Everingham, Van Gool, Williams, Winn, and Zisserman] and NYUD-v2
[Silberman et al.(2012)Silberman, Hoiem, Kohli, and Fergus], two popular
datasets for dense prediction MTL. For PASCAL we use the PASCAL-Context [Chen
et al.(2014)Chen, Mottaghi, Liu, Fidler, Urtasun, and Yuille] data split,
which includes 4998 training and 5105 testing images, densely labeled for
semantic segmentation (SemSeg), human parts segmentation (PartSeg), saliency
estimation (Sal), surface normal estimation (Norm), and edge detection (Edge).
We adopt the distilled saliency and surface normal labels from [Maninis et
al.(2019)Maninis, Radosavovic, and Kokkinos]. The NYUD-v2 dataset comprises
795 training and 654 testing images of indoor scenes, fully labeled for
semantic segmentation (SemSeg), depth estimation (Depth), surface normal
estimation (Norm), and edge detection (Edge). All training and evaluation of
the baselines and final branched models was conducted on the full-resolution
images. Yet, to accelerate the architecture search, we optimize BMTAS using
resized input images (1/2 and 2/3 resolution for PASCAL-Context and NYUD-v2
respectively).

Architectures. For all experiments we use a DeepLabv3+ base architecture [Chen
et al.(2018a)Chen, Zhu, Papandreou, Schroff, and Adam], which was designed for
semantic segmentation, but was shown to perform well for various dense
prediction tasks [Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos,
Kanakis et al.(2020)Kanakis, Bruggemann, Saha, Georgoulis, Obukhov, and Van
Gool]. To demonstrate generalization of our method across encoder types, we
report results for both MobileNetV2 [Sandler et al.(2018)Sandler, Howard, Zhu,
Zhmoginov, and Chen] and ResNet-50 [He et al.(2016)He, Zhang, Ren, and Sun]
backbones. For MobileNetV2 we employ a reduced design of the ASPP module
(R-ASPP), proposed in [Sandler et al.(2018)Sandler, Howard, Zhu, Zhmoginov,
and Chen].

Metrics. We evaluate semantic segmentation, human parts segmentation and
saliency estimation using mean intersection over union. For surface normal
estimation we use mean angular error, for edge detection optimal dataset
F-measure [Martin et al.(2004)Martin, Fowlkes, and Malik], and for depth
estimation root mean square error. To obtain a single-number performance
metric for a multi-task model, we additionally report the average per-task
performance drop ($\Delta_{m}$) with respect to single-task baselines b for
model m, as proposed in [Maninis et al.(2019)Maninis, Radosavovic, and
Kokkinos]. It is defined as

$\Delta_{m}=\frac{1}{T}\sum_{i=1}^{T}(-1)^{l_{i}}\left(M_{m,i}-M_{b,i}\right)/M_{b,i}$
(7)

where $l_{i}=1$ if a lower value for metric $M_{i}$ indicates better
performance, and $l_{i}=0$ otherwise.

Baselines. To eliminate the influence of differences in training setup (data
augmentation, hyperparameters, etc.), we compare BMTAS with our own
implementations of the baselines. As a reference, we first report the
performance of standard single-task models (‘Single’) and a multi-task model
consisting of a fully shared encoder and task-specific heads (‘Shared’). We
consider the ASPP (resp. R-ASPP) module part of the encoder, and as such,
share it between all tasks for this model. Furthermore, we create branched
encoder structures by two competing methods: i) In FAFS [Lu et al.(2017)Lu,
Kumar, Zhai, Cheng, Javidi, and Feris], a fully shared encoder is first pre-
trained and then greedily split layer by layer, starting from the final layer.
Branches are separated based on task similarity in sample space, taking into
account the complexity of the resulting model. ii) In BMN [Vandenhende et
al.(2020a)Vandenhende, Georgoulis, De Brabandere, and Van Gool], branching
structures are found by measuring task affinity with feature map correlations
between pre-trained single-task networks. The final models are the result of
an exhaustive search over all possible branching configurations, capped at
maximum resource cost. Finally, we evaluate the performances of three state-
of-the-art MTL approaches: Cross-Stitch Networks (C.-S.) [Misra et
al.(2016)Misra, Shrivastava, Gupta, and Hebert], NDDR-CNN [Gao et
al.(2019)Gao, Ma, Zhao, Liu, and Yuille] and MTAN [Liu et al.(2019c)Liu,
Johns, and Davison].

###  4.2 Comparison with the State-of-the-Art

Figure 2: Comparison of multi-task performance $\Delta_{m}$ as a function of
multiply-add operations (MAdds) of encoder branching methods on PASCAL-
Context, for a MobileNetV2 backbone (left) and a ResNet-50 backbone (right).
For all methods three models with adjustable complexity penalty were
evaluated—except ours (BMTAS), for which we additionally report the model
obtained by an architecture search without resource loss (no complexity
penalty). BMTAS models favorably balance the efficiency vs. performance trade-
off.

Model |  MAdds $\downarrow$ |  SemSeg $\uparrow$ |  PartSeg $\uparrow$ |  Sal $\uparrow$ |  Norm $\downarrow$ |  Edge $\uparrow$ |  $\Delta_{m}$ [%] $\uparrow$
---|---|---|---|---|---|---|---
Single |  15.7B |  65.11 |  57.54 |  65.41 |  13.98 |  69.50 | 0.00
Shared |  06.0B |  59.69 |  55.96 |  63.03 |  16.02 |  67.80 | -6.35
C.-S. [Misra et al.(2016)Misra, Shrivastava, Gupta, and Hebert] |  15.7B |  63.28 |  60.21 |  65.13 |  14.17 |  71.10 | 0.47
NDDR-CNN [Gao et al.(2019)Gao, Ma, Zhao, Liu, and Yuille] |  21.8B |  63.22 |  56.12 |  65.16 |  14.47 |  68.90 | -2.02
MTAN [Liu et al.(2019c)Liu, Johns, and Davison] |  09.5B |  61.55 |  58.89 |  64.96 |  14.74 |  69.90 | -1.73
BMTAS-1 |  07.5B |  61.43 |  56.77 |  63.64 |  14.77 |  68.20 | -3.44
BMTAS-2 |  09.1B |  62.80 |  57.72 |  64.92 |  14.48 |  68.70 | -1.74
BMTAS-3 |  12.5B |  64.07 |  58.60 |  64.72 |  14.27 |  69.40 | -0.60

Table 1: Comparison of BMTAS with MTL baselines on PASCAL-Context using a
MobileNetV2 backbone. The resource loss weights $\lambda$ for BMTAS-{1, 2, 3}
are chosen on a logarithmic scale: {0.1, 0.05, 0.02} respectively.

Since the proposed method is targeted towards resource-constrained
applications, we optimize two objectives concurrently: multi-task performance
and resource cost, and we ultimately seek pareto-efficient solutions. Fig. 2
depicts a comparison of the principal encoder branching methods. For each
method, three models are generated by varying the complexity penalties in the
algorithms. Our models compare favorably to random structures and FAFS, and
perform on par with BMN. Unlike BMN however, our approach is end-to-end
trainable, and does not rely on an offline brute-force search over the entire
space of configurations.

Table 4.2 and Table 4.2 show a breakdown of individual task metrics for our
method and state-of-the-art MTL approaches on PASCAL-Context and NYUD-v2 using
a MobileNetV2 and ResNet-50 backbone, respectively. On PASCAL-Context, the
BMTAS multi-task performances lie between the ‘Shared’ and ‘Single’ baselines.
Although the C.-S. network performs best in this setting, it also requires
considerably more MAdds than all the BMTAS models. On the smaller NYUD-v2
datasets, MTL approaches generally perform better compared to the ‘Single’
baseline. Owing to the large number of channels in the feature maps, both
NDDR-CNN and MTAN scale inefficiently to a ResNet-50 backbone. The performance
boost for these methods therefore comes at the cost of a sizeable increase in
resource cost.

Model |  MAdds $\downarrow$ |  SemSeg $\uparrow$ |  Depth $\downarrow$ |  Norm $\downarrow$ |  Edge $\uparrow$ |  $\Delta_{m}$ [%] $\uparrow$
---|---|---|---|---|---|---
Single |  254.4B |  40.08 |  0.5479 |  21.67 |  70.10 | 0.00
Shared |  122.2B |  38.37 |  0.5766 |  22.66 |  70.90 | -3.23
C.-S. [Misra et al.(2016)Misra, Shrivastava, Gupta, and Hebert] |  254.4B |  41.01 |  0.5380 |  22.00 |  70.40 | 0.77
NDDR-CNN [Gao et al.(2019)Gao, Ma, Zhao, Liu, and Yuille] |  366.2B |  40.88 |  0.5358 |  21.86 |  70.30 | 0.91
MTAN [Liu et al.(2019c)Liu, Johns, and Davison] |  600.1B |  42.03 |  0.5191 |  21.89 |  70.40 | 2.38
BMTAS-1 |  155.8B |  40.66 |  0.5691 |  21.84 |  70.70 | -0.58
BMTAS-2 |  223.7B |  40.37 |  0.5413 |  21.74 |  69.80 | 0.30
BMTAS-3 |  248.3B |  41.10 |  0.5431 |  21.56 |  70.10 | 0.98

Table 2: Comparison of BMTAS with MTL baselines on NYUD-v2 using a ResNet-50
backbone. The resource loss weights $\lambda$ for BMTAS are {0.005, 0.001,
0.0002} respectively.

Overall, BMTAS models gain the advantage in applications where both resource
efficiency and task performance are essential. The method is universally
applicable to any backbone and thus able to produce compact multi-task models
in all tested scenarios. Furthermore, BMTAS is freely adaptable to the
application-specific resource budget, a flexibility which is not provided by
C.-S., NDDR-CNN or MTAN without changing the backbone. However, the proposed
approach also shows some limitations. Notably, the number of possible task
groupings grows quickly with increasing tasks333The total number of task
groupings for $T$ tasks is given by the Bell number $B_{T}$., raising the
complexity of computing the resource loss. Combined with the enlarged space of
possible branching configurations, this can slow down the architecture search
considerably for many-task learning.

For a medium number of tasks however, BMTAS is reasonably efficient.
Concretely, the search time was around 1 day on NYUD-v2 (4 tasks, 25000
iterations) and 3.5 days on PASCAL-Context (5 tasks, 50000 iterations) for
either backbone using a single V100 GPU.

###  4.3 Ablation Studies

In Table 4.3 we present two sets of ablation studies on our search algorithm.
As a reference model, we use ‘BMTAS-2’ from Table 4.2.

First, we validate the efficacy of the obtained branching by randomly
permuting task-to-branch pairings in the reference structure, before training
it (‘Permutations’). We repeat this procedure five times and report the mean
scores and standard deviations in Table 4.3. We also train a simple branching
structure consisting of some fully shared layers and simultaneous branching
for all tasks (‘Vanilla’). The branching point is chosen such that the number
of MAdds coincides with our reference model. Although these two baselines
outperform our reference model on SemSeg and Edge, the overall multi-task
performance is lower in both cases, suggesting that our method learns to
disentangle tasks more efficiently.

Model |  SemSeg $\uparrow$ |  PartSeg $\uparrow$ |  Sal $\uparrow$ |  Norm $\downarrow$ |  Edge $\uparrow$ |  $\Delta_{m}$ [%] $\uparrow$
---|---|---|---|---|---|---
BMTAS |  62.80 |  57.72 |  64.92 |  14.48 |  68.70 | -1.74
Permutations |  63.07$\pm$1.09 |  57.46$\pm$0.39 |  64.25$\pm$0.20 |  14.94$\pm$0.25 |  69.20$\pm$0.73 |  -2.47$\pm$0.39
Vanilla |  64.29 |  57.15 |  64.43 |  15.06 |  69.10 | -2.34
w/o res. loss |  64.70 |  58.82 |  65.32 |  14.14 |  70.00 | 0.21
w/o warm-up |  62.70 |  57.93 |  63.66 |  14.74 |  68.60 | -2.49
w/o resizing |  63.29 |  57.50 |  64.06 |  14.37 |  68.70 | -1.77

Table 3: Ablation studies on PASCAL-Context using a MobileNetV2 backbone. The
numbers for ‘Permutations’ reflect the mean and standard deviation of five
independent runs.

Second, we ablate three components of our search algorithm and report the
corresponding results in Table 4.3: i) We discard the resource term in the
loss (‘w/o res. loss’), i.e., we set $\lambda=0$ in the total search loss of
Eq. 4. Both the performance and resource cost (see also Fig. 2 left) of the
found model are close to the single-task case, as suggested in Sec. 3.3. ii)
We disable the task-specific ‘warm-up’ of the parallel candidate operations
(‘w/o warm-up’, see Sec. 3.1) and directly search architectures with ImageNet
initialization for $\theta$. The resulting model underperforms our reference
model, indicating the importance of softly assigning each candidate operation
to a particular task before starting the search: This helps counteract the
bias of the algorithm toward full operation sharing early in the training, as
well as breaks parameter initialization symmetry. Only the neural operation
parameters $\theta$ are ‘warmed up’ before the search, while the architecture
distribution parameters $\alpha$ are simply initialized with zeros. iii) We
conduct the architecture search on the full resolution input images (‘w/o
resizing’), instead of resizing them as mentioned in Sec. 4.1. The performance
remains approximately equal to the reference model, justifying the use of
resized images during architecture search.

###  4.4 Task Pairing Case Study

Figure 3: Graph of a ResNet-50-DeepLabv3+ branching structure obtained with
our method on the PASCAL-Context dataset (‘BMTAS-2’ in Table D). Edges of the
graph indicate operations and vertices indicate feature tensors. On the left,
a Representational Similarity Analysis (RSA) matrix determined from
correlating encoder output feature maps of the single-task networks is shown.
Grouped tasks in the branched architecture exhibit high RSA correlations,
validating our searched configuration.

Fig. 3 shows a sample ResNet-50 encoder branching structure determined with
our method. We compare the resulting task groupings with the Representational
Similarity Analysis (RSA) matrix obtained from correlating encoded feature
maps of the individual single-task networks, as proposed in [Vandenhende et
al.(2020a)Vandenhende, Georgoulis, De Brabandere, and Van Gool]. The RSA
suggests task groupings similar to ours, with the pairs SemSeg-PartSeg and
Norm-Edge having high task affinities in both cases.

##  5 Conclusion

We presented BMTAS, an end-to-end approach for automatically finding efficient
encoder branching structures for MTL. Individual task routings through a
supergraph are determined by jointly learning architecture distribution
parameters and neural operation parameters through backpropagation, using a
novel resource-aware loss function. Combined, the routings form branching
structures which exhibit high overall performance while being computationally
efficient, as we demonstrate across several datasets (PASCAL-Context, NYUD-v2)
and network backbones (ResNet-50, MobileNetV2). The proposed method is highly
flexible and can serve as a basis for further exploration in MTL NAS.

##  Appendix A Training Settings

In this section, we describe the training setup used for experiments on
PASCAL-Context. On NYUD-v2, the exact same setup was used, except that the
number of training iterations was halved. Training and evaluation code for
this project was written using the PyTorch library [Paszke et al.(2017)Paszke,
Gross, Chintala, Chanan, Yang, DeVito, Lin, Desmaison, Antiga, and Lerer].

Data augmentation. We augment input images during training by random scaling
with values between 0.5 and 2.0 (in increments of 0.25), random cropping to
input size (which was fixed to $512\times 512$ for full-resolution PASCAL-
Context) and random horizontal flipping. Image intensities are rescaled to the
[-1, 1] range.

Task losses. For semantic segmentation and human parts segmentation we use a
cross-entropy loss (loss weights $\omega_{t}=1$ and $\omega_{t}=2$ in Eq. 3,
respectively), for saliency estimation a balanced cross-entropy loss
($\omega_{t}=5$), for depth estimation a $\mathcal{L}_{1}$ loss
($\omega_{t}=1$), for surface normal estimation a $\mathcal{L}_{1}$ loss with
unit vector normalization ($\omega_{t}=10$) and for edge detection a weighted
cross-entropy loss ($\omega_{t}=50$). For edge detection, the positive pixels
are weighted with 0.95 and the negative pixels with 0.05 on PASCAL-Context,
while on NYUD-v2 the weights are 0.8 and 0.2. $\omega_{t}$ for each task was
found by conducting a logarithmic grid search over candidate values with
single-task networks.

Optimization hyperparameters. Model weights $\theta$ are updated using
Stochastic Gradient Descent (SGD) with momentum of 0.9 and weight decay
0.0001. The initial learning rate is set to 0.005 and decayed during training
according to a ‘poly’ learning rate policy [Chen et al.(2017)Chen, Papandreou,
Kokkinos, Murphy, and Yuille]. For the architecture distribution parameters
$\alpha$, we use an Adam optimizer [Kingma and Ba(2015)] with learning rate
0.01 and weight decay 0.00005. We use a batchsize of 8 and 16 for ResNet-50
and MobileNetV2, respectively.

Architecture search. We update the supergraph sequentially for each task.
Before the architecture search, we ‘warm up’ the supergraph by training each
operation’s model weights $\theta$ (initialized with ImageNet weights) on the
corresponding task only for 2000 iterations. The architecture distribution
parameters $\alpha$ are initialized with zeros. During the search, we
alternatively train $\alpha$ on 20% of the data and $\theta$ on the other 80%.
This cycle is repeated until $\theta$ has received 40000 updates. Over the
course of training, the Gumbel-Softmax temperature $\tau$ is annealed linearly
from 5.0 to 0.1. Importantly, we use the batch-specific statistics for batch
normalization during the $\alpha$ update phase and reset the batch statistics
before training $\theta$ after every architecture change. Furthermore, to
equalize the scale of candidate operations for the search, we disable
learnable affine parameters in the last batch normalization of every
operation. Finally, the momentum of the $\theta$-optimizer is reset after
every change to the architecture.

Branched network training. After the architecture search, the resulting
branched network is retrained from scratch for 40000 iterations. The encoder
network weights are initialized with ImageNet weights. For all operations that
are shared between several tasks, we divide the learning rate by the number of
tasks sharing, since those operations receive more updates.

##  Appendix B Implementation of Baselines

For FAFS [Lu et al.(2017)Lu, Kumar, Zhai, Cheng, Javidi, and Feris], we pre-
train a fully shared network on all the tasks and then calculate the task
groupings in all layers greedily (starting from the last) according to the
task affinity measure described in [Lu et al.(2017)Lu, Kumar, Zhai, Cheng,
Javidi, and Feris]. For edge detection, we use the loss instead of the optimal
dataset F-measure to determine sample difficulty. Finding branching structures
via the BMN approach [Vandenhende et al.(2020a)Vandenhende, Georgoulis, De
Brabandere, and Van Gool] involved training separate single-task networks and
computing the Representational Similarity Analysis matrix from the resulting
task-specific feature maps, exactly as described in the paper. Various
branching structures can be found by exhaustively searching candidates among a
reduced pool, containing all possible structures below a specified MAdds
value. To keep the comparison fair, we trained the branched structures
resulting from BMN and FAFS in exactly the same setting as ours.

We implemented Cross-Stitch Networks [Misra et al.(2016)Misra, Shrivastava,
Gupta, and Hebert], NDDR-CNN [Gao et al.(2019)Gao, Ma, Zhao, Liu, and Yuille]
and MTAN [Liu et al.(2019c)Liu, Johns, and Davison] based on the code provided
by the authors and information given in the papers. We use a similar training
setup as the one described for our method, however we conducted a logarithmic
grid search over learning rates for each baseline individually. For Cross-
Stitch Networks, applying one unit per feature tensor (as opposed to channel-
wise) yielded more stable results. The weights of Cross-Stitch- and NDDR-CNN-
units are initialized with $\alpha=0.8$ and $\beta=\frac{0.2}{T-1}$, where $T$
is the number of tasks. Both methods are applied on the fully pre-trained
single-task networks. For MTAN with the MobileNetV2 backbone, we change the
$3\times 3$ convolutions in the attention modules to depthwise separable
convolutions. In general, all ReLU activations are replaced with ReLU6 for
MobileNetV2.

##  Appendix C Implementation Verification

Backbone |  SemSeg $\uparrow$ |  PartSeg $\uparrow$ |  Sal $\uparrow$ |  Norm $\downarrow$ |  Edge $\uparrow$
---|---|---|---|---|---
MobileNetV2, [Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos] |  62.10 |  54.88 |  66.30 |  14.88 | 69.50
MobileNetV2, ours |  65.11 |  57.54 |  65.41 |  13.98 | 69.50
ResNet-50, [Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos] |  68.30 |  60.70 |  65.40 |  14.61 | 72.70
ResNet-50, ours |  70.43 |  63.93 |  67.34 |  13.39 | 74.10

Table C-1: DeepLabv3+ performance in a single-task setting on PASCAL-Context
using either MobileNetV2 or ResNet-50 as a backbone. We compare the
performance obtained using our implementation with the results published in
[Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos].

To show that our implementations of DeepLabv3+ with the above mentioned
backbones perform competitively for the tasks of interest, we compare in Table
C our single-task performances on PASCAL-Context with published results in
[Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos]. A direct comparison
is inconclusive even though the architectures are analogous, as the results in
[Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos] are obtained with
different training setups. Nevertheless, the numbers demonstrate that our
single-task networks represent a strong baseline for comparison.

##  Appendix D Complementary Results

In Table D, we present the performances of our method and simple baselines for
a ResNet-50 backbone on PASCAL-Context (plotted on the right in Fig. 2). For
this setting, we choose not to report scores for Cross-Stitch Networks [Misra
et al.(2016)Misra, Shrivastava, Gupta, and Hebert], NDDR-CNN [Gao et
al.(2019)Gao, Ma, Zhao, Liu, and Yuille] and MTAN [Liu et al.(2019c)Liu,
Johns, and Davison] since we were unable to obtain competitive performances
for those approaches, despite the extensive learning rate grid-search.

Model |  MAdds $\downarrow$ |  SemSeg $\uparrow$ |  PartSeg $\uparrow$ |  Sal $\uparrow$ |  Norm $\downarrow$ |  Edge $\uparrow$ |  $\Delta_{m}$ [%] $\uparrow$
---|---|---|---|---|---|---|---
Single |  345.7B |  70.43 |  63.93 |  67.34 |  13.39 |  74.10 | 0.00
Shared |  154.6B |  68.24 |  62.18 |  65.16 |  14.98 |  71.90 | -4.80
BMTAS-1 |  199.0B |  68.17 |  62.36 |  65.64 |  14.09 |  72.20 | -3.20
BMTAS-2 |  225.8B |  66.92 |  62.93 |  65.82 |  13.70 |  72.90 | -2.56
BMTAS-3 |  298.2B |  69.58 |  64.36 |  66.68 |  13.65 |  73.00 | -1.00

Table D-1: Comparison of our method with simple baselines on PASCAL-Context
using a ResNet-50 backbone. The resource loss weights $\lambda$ for BMTAS are
{0.02, 0.005, 0.001} respectively.

## References

  * [Bingel and Søgaard(2017)] Joachim Bingel and Anders Søgaard.  Identifying beneficial task relations for multi-task learning in deep neural networks.  In _European Chapter of the Association for Computational Linguistics_ , 2017.
  * [Bragman et al.(2019)Bragman, Tanno, Ourselin, Alexander, and Cardoso] Felix JS Bragman, Ryutaro Tanno, Sebastien Ourselin, Daniel C Alexander, and Jorge Cardoso.  Stochastic filter groups for multi-task cnns: Learning specialist and generalist convolution kernels.  In _ICCV_ , 2019.
  * [Cai et al.(2019)Cai, Zhu, and Han] Han Cai, Ligeng Zhu, and Song Han.  Proxylessnas: Direct neural architecture search on target task and hardware.  In _ICLR_ , 2019.
  * [Chen et al.(2017)Chen, Papandreou, Kokkinos, Murphy, and Yuille] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille.  Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.  _TPAMI_ , 40(4):834–848, 2017.
  * [Chen et al.(2018a)Chen, Zhu, Papandreou, Schroff, and Adam] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam.  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In _ECCV_ , 2018a.
  * [Chen et al.(2014)Chen, Mottaghi, Liu, Fidler, Urtasun, and Yuille] Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fidler, Raquel Urtasun, and Alan Yuille.  Detect what you can: Detecting and representing objects using holistic models and body parts.  In _CVPR_ , 2014.
  * [Chen et al.(2018b)Chen, Badrinarayanan, Lee, and Rabinovich] Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich.  Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks.  In _ICML_ , 2018b.
  * [Everingham et al.(2010)Everingham, Van Gool, Williams, Winn, and Zisserman] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman.  The pascal visual object classes (voc) challenge.  _IJCV_ , 88(2):303–338, 2010.
  * [Gao et al.(2019)Gao, Ma, Zhao, Liu, and Yuille] Yuan Gao, Jiayi Ma, Mingbo Zhao, Wei Liu, and Alan L Yuille.  Nddr-cnn: Layerwise feature fusing in multi-task cnns by neural discriminative dimensionality reduction.  In _CVPR_ , 2019.
  * [Gao et al.(2020)Gao, Bai, Jie, Ma, Jia, and Liu] Yuan Gao, Haoping Bai, Zequn Jie, Jiayi Ma, Kui Jia, and Wei Liu.  Mtl-nas: Task-agnostic neural architecture search towards general-purpose multi-task learning.  In _CVPR_ , 2020.
  * [Girshick et al.(2014)Girshick, Donahue, Darrell, and Malik] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  In _CVPR_ , 2014.
  * [Gordon et al.(2018)Gordon, Eban, Nachum, Chen, Wu, Yang, and Choi] Ariel Gordon, Elad Eban, Ofir Nachum, Bo Chen, Hao Wu, Tien-Ju Yang, and Edward Choi.  Morphnet: Fast & simple resource-constrained structure learning of deep networks.  In _CVPR_ , 2018.
  * [Guo et al.(2018)Guo, Haque, Huang, Yeung, and Fei-Fei] Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Li Fei-Fei.  Dynamic task prioritization for multitask learning.  In _ECCV_ , 2018.
  * [He et al.(2016)He, Zhang, Ren, and Sun] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _CVPR_ , 2016.
  * [Jang et al.(2017)Jang, Gu, and Poole] Eric Jang, Shixiang Gu, and Ben Poole.  Categorical reparameterization with gumbel-softmax.  In _ICLR_ , 2017.
  * [Kanakis et al.(2020)Kanakis, Bruggemann, Saha, Georgoulis, Obukhov, and Van Gool] Menelaos Kanakis, David Bruggemann, Suman Saha, Stamatios Georgoulis, Anton Obukhov, and Luc Van Gool.  Reparameterizing convolutions for incremental multi-task learning without task interference.  In _ECCV_ , 2020.
  * [Kendall et al.(2018)Kendall, Gal, and Cipolla] Alex Kendall, Yarin Gal, and Roberto Cipolla.  Multi-task learning using uncertainty to weigh losses for scene geometry and semantics.  In _CVPR_ , 2018.
  * [Kingma and Ba(2015)] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _ICLR_ , 2015.
  * [Kokkinos(2017)] Iasonas Kokkinos.  Ubernet: Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory.  In _CVPR_ , 2017.
  * [Liu et al.(2019a)Liu, Chen, Schroff, Adam, Hua, Yuille, and Fei-Fei] Chenxi Liu, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, Wei Hua, Alan L Yuille, and Li Fei-Fei.  Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation.  In _CVPR_ , 2019a.
  * [Liu et al.(2019b)Liu, Simonyan, and Yang] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  Darts: Differentiable architecture search.  In _ICLR_ , 2019b.
  * [Liu et al.(2019c)Liu, Johns, and Davison] Shikun Liu, Edward Johns, and Andrew J Davison.  End-to-end multi-task learning with attention.  In _CVPR_ , 2019c.
  * [Long et al.(2015)Long, Shelhamer, and Darrell] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully convolutional networks for semantic segmentation.  In _CVPR_ , 2015.
  * [Long et al.(2017)Long, Cao, Wang, and Philip] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and S Yu Philip.  Learning multiple tasks with multilinear relationship networks.  In _NeurIPS_ , 2017.
  * [Lu et al.(2017)Lu, Kumar, Zhai, Cheng, Javidi, and Feris] Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris.  Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification.  In _CVPR_ , 2017.
  * [Maddison et al.(2017)Maddison, Mnih, and Teh] Chris J Maddison, Andriy Mnih, and Yee Whye Teh.  The concrete distribution: A continuous relaxation of discrete random variables.  In _ICLR_ , 2017.
  * [Maninis et al.(2019)Maninis, Radosavovic, and Kokkinos] Kevis-Kokitsi Maninis, Ilija Radosavovic, and Iasonas Kokkinos.  Attentive single-tasking of multiple tasks.  In _CVPR_ , 2019.
  * [Martin et al.(2004)Martin, Fowlkes, and Malik] David R Martin, Charless C Fowlkes, and Jitendra Malik.  Learning to detect natural image boundaries using local brightness, color, and texture cues.  _TPAMI_ , 26(5):530–549, 2004.
  * [Misra et al.(2016)Misra, Shrivastava, Gupta, and Hebert] Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert.  Cross-stitch networks for multi-task learning.  In _CVPR_ , 2016.
  * [Molchanov et al.(2017)Molchanov, Tyree, Karras, Aila, and Kautz] Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz.  Pruning convolutional neural networks for resource efficient inference.  In _ICLR_ , 2017.
  * [Neven et al.(2017)Neven, De Brabandere, Georgoulis, Proesmans, and Van Gool] Davy Neven, Bert De Brabandere, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool.  Fast scene understanding for autonomous driving.  In _IEEE Symposium on Intelligent Vehicles Workshop_ , 2017.
  * [Paszke et al.(2017)Paszke, Gross, Chintala, Chanan, Yang, DeVito, Lin, Desmaison, Antiga, and Lerer] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in pytorch.  In _NeurIPS Workshop_ , 2017.
  * [Pham et al.(2018)Pham, Guan, Zoph, Le, and Dean] Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, and Jeff Dean.  Efficient neural architecture search via parameter sharing.  In _ICML_ , 2018.
  * [Real et al.(2017)Real, Moore, Selle, Saxena, Suematsu, Tan, Le, and Kurakin] Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc V Le, and Alexey Kurakin.  Large-scale evolution of image classifiers.  In _ICML_ , 2017.
  * [Real et al.(2019)Real, Aggarwal, Huang, and Le] Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le.  Regularized evolution for image classifier architecture search.  In _AAAI_ , 2019.
  * [Rosenbaum et al.(2018)Rosenbaum, Klinger, and Riemer] Clemens Rosenbaum, Tim Klinger, and Matthew Riemer.  Routing networks: Adaptive selection of non-linear functions for multi-task learning.  In _ICLR_ , 2018.
  * [Ruder(2017)] Sebastian Ruder.  An overview of multi-task learning in deep neural networks.  _arXiv preprint arXiv:1706.05098_ , 2017.
  * [Ruder et al.(2019)Ruder, Bingel, Augenstein, and Søgaard] Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, and Anders Søgaard.  Latent multi-task architecture learning.  In _AAAI_ , 2019.
  * [Sandler et al.(2018)Sandler, Howard, Zhu, Zhmoginov, and Chen] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In _CVPR_ , 2018.
  * [Sener and Koltun(2018)] Ozan Sener and Vladlen Koltun.  Multi-task learning as multi-objective optimization.  In _NeurIPS_ , 2018.
  * [Silberman et al.(2012)Silberman, Hoiem, Kohli, and Fergus] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus.  Indoor segmentation and support inference from rgbd images.  In _ECCV_ , 2012.
  * [Vandenhende et al.(2020a)Vandenhende, Georgoulis, De Brabandere, and Van Gool] Simon Vandenhende, Stamatios Georgoulis, Bert De Brabandere, and Luc Van Gool.  Branched multi-task networks: deciding what layers to share.  In _BMVC_ , 2020a.
  * [Vandenhende et al.(2020b)Vandenhende, Georgoulis, and Van Gool] Simon Vandenhende, Stamatios Georgoulis, and Luc Van Gool.  Mti-net: Multi-scale task interaction networks for multi-task learning.  In _ECCV_ , 2020b.
  * [Veniat and Denoyer(2018)] Tom Veniat and Ludovic Denoyer.  Learning time/memory-efficient deep architectures with budgeted super networks.  In _CVPR_ , 2018.
  * [Wu et al.(2019)Wu, Dai, Zhang, Wang, Sun, Wu, Tian, Vajda, Jia, and Keutzer] Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer.  Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search.  In _CVPR_ , 2019.
  * [Xie and Tu(2015)] Saining Xie and Zhuowen Tu.  Holistically-nested edge detection.  In _ICCV_ , 2015.
  * [Xie et al.(2019)Xie, Zheng, Liu, and Lin] Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin.  Snas: stochastic neural architecture search.  In _ICLR_ , 2019.
  * [Xu et al.(2018)Xu, Ouyang, Wang, and Sebe] Dan Xu, Wanli Ouyang, Xiaogang Wang, and Nicu Sebe.  Pad-net: Multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing.  In _CVPR_ , 2018.
  * [Zoph and Le(2017)] Barret Zoph and Quoc V Le.  Neural architecture search with reinforcement learning.  In _ICLR_ , 2017.
  * [Zoph et al.(2018)Zoph, Vasudevan, Shlens, and Le] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le.  Learning transferable architectures for scalable image recognition.  In _CVPR_ , 2018.


nucleation, in line with the AdS Instability Conjecture [1, 2]. We have
analysed those instabilities that correspond to 4d membranes made up from
D8-branes wrapping the compact manifold $X_{6}$, building on the previous work
[27]. As pointed out therein, one should be able to determine whether $Q>T$ or
not for this class of membranes with our current, approximate description of a
family of $\mathcal{N}=0$ that are closely related to supersymmetric ones. In
this work we have expanded on this observation by analysing such D8-brane
charge and tension in several orientifold backgrounds with different space-
time filling D6-brane configurations. We have considered D6-branes that lie on
top of O6-planes, which always solve the vacua conditions.

As pointed out in [27] at leading order $Q_{\rm D8}=T_{\rm D8}$, and then
there are three corrections that can tip the scales to one side or the other,
represented in (2.30). Out of these three corrections two of them are
unavoidable, namely the curvature correction $K_{a}^{(2)}T_{\rm D4}^{a}$ and
the BIon correction $\Delta_{\rm D8}^{\rm Bion}=-T_{\rm D8}^{\rm BIon}$. It
turns out that $K_{a}^{(2)}T_{\rm D4}^{a}$ always favours $Q_{\rm D8}^{\rm
total}>T_{\rm D8}^{\rm total}$, while $\Delta_{\rm D8}^{\rm Bion}$ can have
both signs. Therefore in a setup in which $K_{a}^{(2)}T_{\rm
D4}^{a}\ll|\Delta_{\rm D8}^{\rm Bion}|$ it will be the sign of the latter that
will determine whether $Q_{\rm D8}^{\rm total}>T_{\rm D8}^{\rm total}$ or
$Q_{\rm D8}^{\rm total}<T_{\rm D8}^{\rm total}$. This will for instance occur
near orbifold limits of Calabi–Yau geometries. We have computed $\Delta_{\rm
D8}^{\rm Bion}$ in such orbifold limits, finding that the simple expression
(3.8) that indeed shows that this correction can be either positive or
negative. A negative value is favoured when we have pairs of D6-branes that do
not intersect in the internal dimensions, so that open strings stretched
between them lead to a spectrum with masses above the compactification scale.
By choosing the D6-brane positions one can build configurations where
$\Delta_{\rm D8}^{\rm Bion}<0$, which are in tension with the WGC for 4d
membranes. They are however not necessarily in tension with the AdS
Instability Conjecture, since there could be other channels, in particular
D4-brane nucleation, that could mediate a non-perturbative decay to an
$\mathcal{N}=0$ vacuum of lower energy.

We have pointed out some caveats that could reconcile our results with our
expectations from the WGC for 4d membranes. From these, perhaps the most
promising one is the one-loop threshold corrections to the vacuum energy,
which as $\Delta_{\rm D8}^{\rm Bion}$ depends on the D6-brane positions, and
could decrease the vacuum energy such that the controversial decay channels
are no longer energetically favoured. If this was the explanation, computing
$\Delta_{\rm D8}^{\rm Bion}$ could give very valuable information about one-
loop corrections in $\mathcal{N}=0$ vacua. If not, one should perhaps
reconsider what the WGC statement should be for this particular class of 4d
membranes. After all, they have a very special nature even from the 4d
viewpoint: their 4d backreaction does not lead to a thin wall, they have
space-time filling D-branes attached to them and their charges are bounded by
the tadpole condition. This last point is particularly important, as it could
modify the usual convex hull condition, that is typically formulated for an
unbounded lattice of charges. In this respect the formalism of [59] to
implement tadpole constraints in 4d EFTs could be of use.

Taken at face value, our results suggest that $\mathcal{N}=0$ AdS4 vacua with
a gauge sector without zero/light modes charged under it are more stable than
those that contain charged light modes. Showing whether or not this is true is
an interesting challenge, as well as to unveil the would-be implications for
our understanding of the string Landscape.

Acknowledgments

We would like to thank L. E. Ibáñez, A. Uranga and I. Valenzuela for useful
discussions. This work is supported through the grants CEX2020-001007-S and
PGC2018-095976-B-C21, funded by MCIN/AEI/10.13039/501100011033 and by ERDF A
way of making Europe. G. F. C. is supported through the JAE Intro ICU grant
JAEIntro-IFT-2021-SWAMP-02. D. P. is supported through the grant FPU19/04298
funded by MCIN/AEI/10.13039/501100011033 and by ESF Investing in your future.

##  Appendix A Moduli stabilisation in
$T^{6}/(\mathbb{Z}_{2}\times\mathbb{Z}_{2})$

In this appendix we consider thee moduli stabilisation of the Kähler sector in
the $T^{6}/(\mathbb{Z}_{2}\times\mathbb{Z}_{2})$ orientifold background with
$(h^{1,1},h^{2,1})_{\rm orb}=(51,3)$. As in [14, 22], we look for vacua where
the twisted two- and four-cycles are blown up due to the presence of
background four-form fluxes. As pointed out in [21], for the class of type IIA
flux vacua analysed in the main text the Kähler moduli stabilisation
conditions amount to

${\cal K}_{a}=\epsilon\frac{10}{3m}\hat{e}_{a}\,,\qquad\hat{e}_{a}\coloneqq
e_{a}-\frac{1}{2}\frac{{\cal
K}_{abc}m^{b}m^{c}}{m}-\frac{1}{2}\mathcal{K}_{aab}m^{b}+mK_{a}^{(2)}\,,$
(A.1)

where

${\cal K}_{a}=\int_{X_{6}}J_{\rm CY}\wedge J_{\rm CY}\wedge\omega_{a}\,,\qquad
e_{a}=\frac{1}{\ell_{s}^{5}}\int_{X_{6}}\bar{G}_{4}\wedge\omega_{a}\in\mathbb{Z}\,,\qquad
m^{a}=\frac{1}{\ell_{s}^{5}}\int_{X_{6}}\bar{G}_{2}\wedge\tilde{\omega}^{a}\in\mathbb{Z}\,,$
(A.2)

and ${\epsilon}=\pm 1$ distinguishes between supersymmetric and non-
supersymmetric vacua, as in eq.(2.5). The connection with this set of
equations can be made by taking into account the dependence of $G_{4}$ on
$\bar{G}_{4}$, $\bar{G}_{2}$, $G_{0}$ and the B-field axions, something that
it is usually done in the smeared approximation. In any event, in the
following we will consider compactifications where $m^{a}=0$, so that these
subtleties disappear and (A.1) simplifies.

To look for solutions to this equation we need to compute the quantity
$-\frac{1}{2}{\cal K}_{a}$, that in our conventions measures the volume of
holomorphic four-cycles or divisors. For this we need to parametrise the
Kähler form in terms of such divisors, including the exceptional ones, and
compute their triple intersection numbers. This exercise was done in [66] for
the above orbifold background $T^{6}/(\mathbb{Z}_{2}\times\mathbb{Z}_{2})$
with a type IIB orientifold projection that leads to O3- and O7-planes. Notice
that the orientifold projection that we are interested in is different, as it
leads to type IIA O6-planes. Therefore, we will take the approach of [21] and
solve (A.1) for the unorientifolded orbifold geometry
$T^{6}/(\mathbb{Z}_{2}\times\mathbb{Z}_{2})$. Then, following the remarks in
section 3.2, we will demand that $e_{a}\in 2\mathbb{Z}$ for the four-form flux
quanta defined in the covering space
$T^{6}/(\mathbb{Z}_{2}\times\mathbb{Z}_{2})$. The necessary topological data
for this case can be extracted from the results of [67, 68].

The Kähler form for the blown-up orbifold
$T^{6}/(\mathbb{Z}_{2}\times\mathbb{Z}_{2})$ reads

$J=r_{i}R_{i}-t_{1\alpha,2\beta}E_{1\alpha,2\beta}-t_{2\beta,3{\gamma}}E_{2\beta,3{\gamma}}-t_{3{\gamma},1\alpha}E_{3{\gamma},1\alpha}\,,$
(A.3)

where $\alpha,\beta,{\gamma}$ run over the four fixed points of a given
$T^{2}$. Also, $R_{i}$ and $E_{A}\equiv E_{i\alpha,j\beta}$ correspond to
divisors that satisfy the linear equivalence relation [67, 68]

$R_{1}\simeq
2D_{1\alpha}+\sum_{\beta}E_{1\alpha,2\beta}+\sum_{\gamma}E_{3{\gamma},1\alpha}\qquad\forall\alpha\,,$
(A.4)

that differs by a factor of 2 compared to [66, eq.(6.2)], due to the lack of
orientifold action. With these conventions and assuming the symmetric
resolution of [66] one finds that the intersection form is given by

$\displaystyle-{\cal I}=$
$\displaystyle\,2R_{1}R_{2}R_{3}-2\Big{(}\sum_{\alpha\beta}E^{2}_{1\alpha,2\beta}R_{3}+\ldots\Big{)}+4\Big{(}\sum_{\alpha\beta}E^{3}_{1\alpha,2\beta}+\ldots\Big{)}$
(A.5)
$\displaystyle-\Big{[}\sum_{\alpha\beta\gamma}E_{1\alpha,2\beta}(E^{2}_{2\beta,3\gamma}+E^{2}_{3\gamma,1\alpha})+\ldots\Big{]}+\sum_{\alpha\beta\gamma}E_{1\alpha,2\beta}E_{2\beta,3\gamma}E_{3\gamma,1\alpha}$

where $\ldots$ are $(1,2,3)$ cyclically permuted terms. This matches the
results of [68, section B.19.4] up to a global minus sign, which reflects the
fact that in our conventions the volume form is $-J^{3}/6$ instead of
$J^{3}/6$.

With this intersection form one can compute the quantity ${\cal K}_{a}$ for
each divisor $R_{i}$ and $E_{i\alpha,j\beta}$. For simplicity we assume that
all twisted moduli and untwisted moduli are equal among them:

$r_{i}=r\,,\qquad t_{A}\equiv t_{i\alpha,j\beta}=t\,.$ (A.6)

One then obtains that

$-{\cal K}_{i}=4r^{2}-32t^{2}\,\qquad-{\cal K}_{A}=4rt-12t^{2}\,.$ (A.7)

A sensible flux Ansatz to solve (A.1) is $e_{i}=e$ and $e_{A}\equiv
e_{i\alpha,j\beta}=f$, with $e,f\in 2\mathbb{Z}$. Equation (A.1) then reduces
to

$4r^{2}-32t^{2}=-\epsilon\frac{10}{3m}e\,,\qquad
4rt-12t^{2}=-\epsilon\frac{10}{3m}f\,.$ (A.8)

Since four-form flux quanta are not constrained by tadpoles, it is easy to
choose values for $e,f$ such that $-{\cal K}_{i}$ and $-{\cal K}_{A}$ are
positive and $r\gg t$. Let us parametrise a solution as $r=xt$, with $x\gg 1$.
For supersymmetric vacua $(\epsilon=-1)$ we obtain

$10e=3m(4x^{2}-32)t^{2}\,,\qquad 10f=3m(4x-12)t^{2}\,,$ (A.9)

and so

$e=\frac{x^{2}-8}{x-3}f\,.$ (A.10)

It is thus simple to find reasonable solutions by taking $x\in\mathbb{N}$,
like for instance $x=10$, $f=126m$ and $t=\sqrt{15}$. For non-supersymmetric
vacua one should only flip the sign of the fluxes.

What is important, though, is that the values for $r$ and $t$ correspond to
the interior of the Kähler cone. From [66, eq.(6.11)] this amounts to require
that $r>4t>0$. This is satisfied as long as $t>0$ and $x>4$, which is in
general quite easy to achieve.

## References

  * [1] H. Ooguri and C. Vafa, _Non-supersymmetric AdS and the Swampland_ , _Adv. Theor. Math. Phys._ 21 (2017) 1787 [1610.01533].
  * [2] B. Freivogel and M. Kleban, _Vacua Morghulis_ , 1610.04564.
  * [3] J. M. Maldacena, J. Michelson and A. Strominger, _Anti-de Sitter fragmentation_ , _JHEP_ 02 (1999) 011 [hep-th/9812073].
  * [4] D. Gaiotto and A. Tomasiello, _The gauge dual of Romans mass_ , _JHEP_ 01 (2010) 015 [0901.0969].
  * [5] R. Antonelli and I. Basile, _Brane annihilation in non-supersymmetric strings_ , _JHEP_ 11 (2019) 021 [1908.04352].
  * [6] F. Apruzzi, G. Bruno De Luca, A. Gnecchi, G. Lo Monaco and A. Tomasiello, _On AdS 7 stability_, _JHEP_ 07 (2020) 033 [1912.13491].
  * [7] I. Bena, K. Pilch and N. P. Warner, _Brane-Jet Instabilities_ , _JHEP_ 10 (2020) 091 [2003.02851].
  * [8] M. Suh, _The non-SUSY AdS 6 and AdS7 fixed points are brane-jet unstable_, _JHEP_ 10 (2020) 010 [2004.06823].
  * [9] A. Guarino, J. Tarrio and O. Varela, _Brane-jet stability of non-supersymmetric AdS vacua_ , _JHEP_ 09 (2020) 110 [2005.07072].
  * [10] A. Guarino, E. Malek and H. Samtleben, _Stable Nonsupersymmetric Anti–de Sitter Vacua of Massive IIA Supergravity_ , _Phys. Rev. Lett._ 126 (2021) 061601 [2011.06600].
  * [11] I. Basile, _Supersymmetry Breaking and Stability in String Vacua: brane dynamics, bubbles and the swampland_ , _Riv. Nuovo Cim._ 1 (2021) 98 [2107.02814].
  * [12] F. Apruzzi, G. Bruno De Luca, G. Lo Monaco and C. F. Uhlemann, _Non-supersymmetric AdS 6 and the swampland_, 2110.03003.
  * [13] P. Bomans, D. Cassani, G. Dibitetto and N. Petri, _Bubble instability of mIIA on $\mathrm{AdS}_{4}\times S^{6}$_, 2110.08276.
  * [14] O. DeWolfe, A. Giryavets, S. Kachru and W. Taylor, _Type IIA moduli stabilization_ , _JHEP_ 07 (2005) 066 [hep-th/0505160].
  * [15] P. G. Camara, A. Font and L. E. Ibanez, _Fluxes, moduli fixing and MSSM-like vacua in a simple IIA orientifold_ , _JHEP_ 09 (2005) 013 [hep-th/0506066].
  * [16] S. Bielleman, L. E. Ibanez and I. Valenzuela, _Minkowski 3-forms, Flux String Vacua, Axion Stability and Naturalness_ , _JHEP_ 12 (2015) 119 [1507.06793].
  * [17] F. Carta, F. Marchesano, W. Staessens and G. Zoccarato, _Open string multi-branched and Kähler potentials_ , _JHEP_ 09 (2016) 062 [1606.00508].
  * [18] A. Herraez, L. E. Ibanez, F. Marchesano and G. Zoccarato, _The Type IIA Flux Potential, 4-forms and Freed-Witten anomalies_ , _JHEP_ 09 (2018) 018 [1802.05771].
  * [19] D. Escobar, F. Marchesano and W. Staessens, _Type IIA Flux Vacua with Mobile D6-branes_ , _JHEP_ 01 (2019) 096 [1811.09282].
  * [20] D. Escobar, F. Marchesano and W. Staessens, _Type IIA flux vacua and $\alpha^{\prime}$-corrections_, _JHEP_ 06 (2019) 129 [1812.08735].
  * [21] F. Marchesano and J. Quirant, _A Landscape of AdS Flux Vacua_ , _JHEP_ 12 (2019) 110 [1908.11386].
  * [22] M. Ihl and T. Wrase, _Towards a Realistic Type IIA T**6/Z(4) Orientifold Model with Background Fluxes. Part 1. Moduli Stabilization_ , _JHEP_ 07 (2006) 027 [hep-th/0604087].
  * [23] O. Aharony, Y. E. Antebi and M. Berkooz, _On the Conformal Field Theory Duals of type IIA AdS(4) Flux Compactifications_ , _JHEP_ 02 (2008) 093 [0801.3326].
  * [24] P. Narayan and S. P. Trivedi, _On The Stability Of Non-Supersymmetric AdS Vacua_ , _JHEP_ 07 (2010) 089 [1002.4498].
  * [25] D. Junghans, _O-Plane Backreaction and Scale Separation in Type IIA Flux Vacua_ , _Fortsch. Phys._ 68 (2020) 2000040 [2003.06274].
  * [26] F. Marchesano, E. Palti, J. Quirant and A. Tomasiello, _On supersymmetric AdS 4 orientifold vacua_, _JHEP_ 08 (2020) 087 [2003.13578].
  * [27] F. Marchesano, D. Prieto and J. Quirant, _BIonic membranes and AdS instabilities_ , 2110.11370.
  * [28] S. Giri, L. Martucci and A. Tomasiello, _On the Stability of String Theory Vacua_ , 2112.10795.
  * [29] R. Blumenhagen, M. Cvetic, P. Langacker and G. Shiu, _Toward realistic intersecting D-brane models_ , _Ann. Rev. Nucl. Part. Sci._ 55 (2005) 71 [hep-th/0502005].
  * [30] R. Blumenhagen, B. Kors, D. Lust and S. Stieberger, _Four-dimensional String Compactifications with D-Branes, Orientifolds and Fluxes_ , _Phys. Rept._ 445 (2007) 1 [hep-th/0610327].
  * [31] F. Marchesano, _Progress in D-brane model building_ , _Fortsch. Phys._ 55 (2007) 491 [hep-th/0702094].
  * [32] L. E. Ibanez and A. M. Uranga, _String theory and particle physics: An introduction to string phenomenology_. Cambridge University Press, 2, 2012.
  * [33] A. Font, L. E. Ibanez and F. Marchesano, _Coisotropic D8-branes and model-building_ , _JHEP_ 09 (2006) 080 [hep-th/0607219].
  * [34] F. Saracco and A. Tomasiello, _Localized O6-plane solutions with Romans mass_ , _JHEP_ 07 (2012) 077 [1201.5378].
  * [35] G. B. De Luca and A. Tomasiello, _Leaps and bounds towards scale separation_ , 2104.12773.
  * [36] E. Bergshoeff, R. Kallosh, T. Ortin, D. Roest and A. Van Proeyen, _New formulations of D = 10 supersymmetry and D8 - O8 domain walls_ , _Class. Quant. Grav._ 18 (2001) 3359 [hep-th/0103233].
  * [37] N. Hitchin, _Lectures on generalized geometry_ , 1008.0973.
  * [38] J. Gomis, F. Marchesano and D. Mateos, _An Open string landscape_ , _JHEP_ 11 (2005) 021 [hep-th/0506179].
  * [39] A. Mininno and A. M. Uranga, _Dynamical tadpoles and Weak Gravity Constraints_ , _JHEP_ 05 (2021) 177 [2011.00051].
  * [40] J. M. Maldacena, G. W. Moore and N. Seiberg, _D-brane instantons and K theory charges_ , _JHEP_ 11 (2001) 062 [hep-th/0108100].
  * [41] G. W. Gibbons, _Born-Infeld particles and Dirichlet p-branes_ , _Nucl. Phys. B_ 514 (1998) 603 [hep-th/9709027].
  * [42] J. Evslin and L. Martucci, _D-brane networks in flux vacua, generalized cycles and calibrations_ , _JHEP_ 07 (2007) 040 [hep-th/0703129].
  * [43] F. Marchesano, _D6-branes and torsion_ , _JHEP_ 05 (2006) 019 [hep-th/0603210].
  * [44] D. Marolf, _Chern-Simons terms and the three notions of charge_ , in _Quantization, gauge theory, and strings. Proceedings, International Conference dedicated to the memory of Professor Efim Fradkin, Moscow, Russia, June 5-10, 2000. Vol. 1+2_ , pp. 312–320, 2000, hep-th/0006117.
  * [45] A. R. Frey and J. Polchinski, _N=3 warped compactifications_ , _Phys. Rev. D_ 65 (2002) 126009 [hep-th/0201029].
  * [46] R. Blumenhagen, D. Lust and T. R. Taylor, _Moduli stabilization in chiral type IIB orientifold models with fluxes_ , _Nucl. Phys. B_ 663 (2003) 319 [hep-th/0303016].
  * [47] J. F. G. Cascales and A. M. Uranga, _Chiral 4d string vacua with D branes and NSNS and RR fluxes_ , _JHEP_ 05 (2003) 011 [hep-th/0303024].
  * [48] A. Font, L. E. Ibanez and F. Quevedo, _$Z(N$) X $Z$(m) Orbifolds and Discrete Torsion_, _Phys. Lett. B_ 217 (1989) 272.
  * [49] C. Vafa and E. Witten, _On orbifolds with discrete torsion_ , _J. Geom. Phys._ 15 (1995) 189 [hep-th/9409188].
  * [50] C. Angelantonj, I. Antoniadis, G. D’Appollonio, E. Dudas and A. Sagnotti, _Type I vacua with brane supersymmetry breaking_ , _Nucl. Phys. B_ 572 (2000) 36 [hep-th/9911081].
  * [51] F. Marchesano and G. Shiu, _Building MSSM flux vacua_ , _JHEP_ 11 (2004) 041 [hep-th/0409132].
  * [52] R. Blumenhagen, M. Cvetic, F. Marchesano and G. Shiu, _Chiral D-brane models with frozen open string moduli_ , _JHEP_ 03 (2005) 050 [hep-th/0502095].
  * [53] M. R. Douglas, _D-branes and discrete torsion_ , hep-th/9807235.
  * [54] J. Gomis, _D-branes on orbifolds with discrete torsion and topological obstruction_ , _JHEP_ 05 (2000) 006 [hep-th/0001200].
  * [55] M. Cvetic, G. Shiu and A. M. Uranga, _Chiral four-dimensional N=1 supersymmetric type 2A orientifolds from intersecting D6 branes_ , _Nucl. Phys. B_ 615 (2001) 3 [hep-th/0107166].
  * [56] A. Herraez, _A Note on Membrane Interactions and the Scalar potential_ , _JHEP_ 10 (2020) 009 [2006.01160].
  * [57] S. Lanza, F. Marchesano, L. Martucci and I. Valenzuela, _Swampland Conjectures for Strings and Membranes_ , 2006.15154.
  * [58] C. Cheung and G. N. Remmen, _Naturalness and the Weak Gravity Conjecture_ , _Phys. Rev. Lett._ 113 (2014) 051601 [1402.2287].
  * [59] S. Lanza, F. Marchesano, L. Martucci and D. Sorokin, _How many fluxes fit in an EFT?_ , _JHEP_ 10 (2019) 110 [1907.11256].
  * [60] R. Blumenhagen, L. Görlich and B. Kors, _Supersymmetric 4D orientifolds of type IIA with D6-branes at angles_ , _JHEP_ 01 (2000) 040 [hep-th/9912204].
  * [61] R. Blumenhagen, L. Görlich and T. Ott, _Supersymmetric Intersecting Branes on the Type IIA T6/Z4 Orientifold_ , _JHEP_ 08 (2003) 021 [hep-th/0211059].
  * [62] S. Förste, G. Honecker and R. Schreyer, _Orientifolds with branes at angles_ , _JHEP_ 06 (2001) 004 [hep-th/0105208].
  * [63] A. Strominger, _Topology of superstring compactification_ , in _Santa Barbara Workshop on Unified String Theories, Santa Barbara, California_ , 1985\.
  * [64] D. Lüst, S. Reffert, W. Schulgin and S. Stieberger, _Moduli stabilization in type IIB orientifolds (I)_ , _Nuclear Physics B_ 766 (2007) 68 [hep-th/0506090].
  * [65] S. Förste, G. Honecker and R. Schreyer, _Supersymmetric ZN $\times$ ZM orientifolds in 4D with D-branes at angles_, _Nuclear Physics B_ 593 (2001) 127 [hep-th/0008250].
  * [66] F. Denef, M. R. Douglas, B. Florea, A. Grassi and S. Kachru, _Fixing all moduli in a simple f-theory compactification_ , _Adv. Theor. Math. Phys._ 9 (2005) 861 [hep-th/0503124].
  * [67] D. Lust, S. Reffert, E. Scheidegger and S. Stieberger, _Resolved Toroidal Orbifolds and their Orientifolds_ , _Adv. Theor. Math. Phys._ 12 (2008) 67 [hep-th/0609014].
  * [68] S. Reffert, _Toroidal Orbifolds: Resolutions, Orientifolds and Applications in String Phenomenology_ , Ph.D. thesis, Munich U., 2006.  hep-th/0609040.