# Cohomology of knotted semimetals in three dimensions

Joshua Celeste

(December 2023)

###### Abstract

We extend the topological classification scheme of Weyl semimetals via
cohomology and the Mayer-Vietoris sequence to account for nodal line
semimetals with space-time inversion symmetry. These are semimetals where
bands meet generally in 1-dimensional submanifolds, which can generally be
knots in $\mathbb{T}^{3}$. These nodal loops have two charges, the quantized
Berry phase and the $\mathbb{Z}_{2}$-monopole charge, the second related to
linking numbers of nodal knots between bands. We provide a manifestly
topological proof of the Weyl charge cancellation condition for the
$\mathbb{Z}_{2}$ monopole charge, which is known to be the second Stiefel-
Whitney class of a tubular neighbourhood surrounding a Weyl submanifold via
the Mayer-Vietoris sequence.

## Acknowledgements

This work is part of ongoing work towards a PhD thesis, as such I would like
to acknowledge the financial support of the University of Adelaide, and thank
my principal supervisor Mathai Varghese for helpful discussions throughout the
production of this work.

##  1 Introduction

Weyl semimetals are systems that realise Weyl fermions in condensed matter
systems, these were predicted in [Wan+11] and have gained much interest since
their experimental discovery, such as in [Xu+15]. Much of the initial focus
has been on semimetals with band crossings at Weyl points, these systems
exhibit interesting experimental signatures when a boundary is added, such as
the presence of Fermi arcs joining projections of Weyl points on the boundary.
However, there is an increasing focus on systems with band crossings at higher
dimensional submanifolds, such as nodal semimetals, these systems also exhibit
interesting surface states on the boundary, namely drumhead states and hinge
states, and are expected to have interesting transport properties [BHB11,
Mat+13, Fan+15, ZL17].

Here we extend the previous work of [MT17, MT17a] to further account for
certain classes of nodal semimetals, in particular nodal semimetals with
space-time inversion symmetry $I_{ST}$ present. These were first described in
[Fan+15], these systems possess nodal lines with two charges, the quantized
Berry phase around each nodal loop, and a second charge known in the physics
literature as the $\mathbb{Z}_{2}$-monopole charge. However, it was shown in
[Ahn+19] that these simply correspond to the first and second Stiefel-Whitney
numbers of the valence bundle for certain submanifolds related to the nodal
lines. In this work, the topological properties of such nodal semimetals are
analysed via the Mayer-Vietoris sequence. Consequently, we obtain a
topologically manifest proof of the charge cancellation condition of the
$\mathbb{Z}_{2}$ monopole charge for nodal semimetals via the Mayer-Vietoris
sequence.

##  2 Topology and semimetal phases

###  2.1 Topological invariants of insulating phases

For any system amenable to study via the Bloch-Floquet transform, with some
sort of a band gap present, the usual approach is to view the system as a
family of Hamiltonians acting on a family of finite dimensional (usually
complex) vector spaces that live over each point in the Brillouin zone. We
denote the Brillouin zone generally by $T$, although it is often an
$n$-dimensional torus, in which case we shall simply write it as
$\mathbb{T}^{n}$. When there is a band gap present everywhere at the Fermi
level, we say the phase is insulating, otherwise it is semimetallic. We are
interested in the behaviour of the system only up to the Fermi energy, the
projection of the Hamiltonian up to the Fermi level, then defines a family of
projections of which act on the aforementioned family of vector spaces, giving
rise to another, possibly topologically non-trivial family of vector spaces.
This is precisely the notion of a vector bundle, and a known tool to assist in
distinguishing such objects are characteristic classes. Such objects are
elements of cohomology groups associated to the Brillouin zone and provide
topological obstructions to continuously deforming one family of Hamiltonians
to another.

Many invariants of topological insulators and semimetals are cohomological in
nature, and are typically derived from some expression of the Berry curvature.
For instance, 2D Chern insulators are classified by the first Chern number
which is simply the integral of the Berry curvature over the Brillouin zone. A
more topological picture occurs when we view this instead as the pairing
between the first Chern class $c_{1}(V)$ of the complex line bundle determined
by the valence band, and the fundamental class of the Brillouin zone $[T]$. In
this view, the Chern class lives in the cohomology group $H^{2}(T;\mathbb{Z})$
while the fundamental class $[T]$ is an element of the homology group
$H_{2}(T;\mathbb{Z})$.

Generally, there is a pairing between cohomology and homology groups
$\braket{\cdot,\cdot}:H^{n}(T;\mathbb{Z})\times
H_{n}(T;\mathbb{Z})\rightarrow\mathbb{Z}$, in particular for cohomology
classes that are representable in deRham cohomology, such as the first Chern
class, this pairing is simply given by integrating an appropriate
representative differential form (for instance the Berry curvature) over an
oriented submanifold that represents the desired homology class in
$H^{n}(T;\mathbb{Z})$. Hence for a 2D Chern insulator, the Brillouin zone
itself has a fundamental class that lives in $H^{2}(T;\mathbb{Z})$, and its
pairing with the first Chern class of the valence bundle gives rise to the
first Chern number.

Since the valence line bundle is a complex line bundle bundle and such objects
are known to be classified up to isomorphism by their first Chern class, it
follows that the first Chern number is the only such cohomological invariant
for the Chern insulator in 2 dimensions. However, in more complicated systems,
higher dimensional invariants can be present, and if additional symmetries are
allowed, the relevant cohomology groups may not be the ordinary integral
cohomology groups, and equivariant cohomology is required instead [FM13,
Thi16].

It is of note that often one does not simply wish to classify vector bundles
up to isomorphism, but instead by the weaker notion of stable isomorphism. In
such a case there are other classification schemes, such as those involving
K-theory, which gives rise to the known classification of topological
insulators [KLF09].

Not every invariant of a band system lives in cohomology or K-theory however.
For instance, 2D semimetals, such as graphene have the Berry phase around a
closed loop in the Brillouin zone being a topological invariant [Van18].
However, this has no cohomological interpretation, instead it is better viewed
as the holonomy of the Berry connection, more refined invariants such as these
are elements of more exotic cohomology theories such as differential
cohomology [SS23] which we shall not describe here.

###  2.2 Invariants of phases with a band intersection

One can consider more general systems where a band gap is not present
everywhere over the Brillouin zone and hence two bands intersect, usually at
the Fermi energy. We shall refer to these points as a band intersection or
band crossing, which determines some submanifold $W$ of the Brillouin zone $T$
called the Weyl submanifold of the system. Due to the presence of a band
intersection at the Fermi level, the projection operators obtained from the
family of Hamiltonians are not defined everywhere, only away from $W$, thus we
only obtain a vector bundle over $T\setminus W$. As a consequence, for a
cohomological study of these phases of matter, the appropriate groups to study
are the cohomology groups $H^{n}(T\setminus W)$. Even when the Brillouin zone
is the torus $T=\mathbb{T}^{n}$, determining such cohomology groups depends on
$W$ and can be quite non-trivial.

####  2.2.1 Topology of Weyl Semimetals

In this section we briefly discuss the topological classification of Weyl
semimetals as seen in [MT17, MT17a] These are semimetallic phases where band
crossings occur at points in momentum space. The case of most relevant
interest is a simple two-band model in three dimensions. Consider a family of
Hamiltonians of the form

$H(k)=\mathbf{h}(k)\cdot\sigma$

where $\mathbf{h}(k)$ is some smooth function
$\mathbf{h}:\mathbb{T}^{3}\rightarrow\mathbb{R}^{3}$ and
$\sigma=(\sigma_{x},\sigma_{y},\sigma_{z})$ is a vector containing the three
Pauli matrices. The spectrum is given by $\pm|\mathbf{h}|$ and so if one sets
the Fermi level to be zero, band crossings occur at the zeroes of
$\mathbf{h}$, which are generically points, such points are called Weyl
points. We assume for this section therefore that $\mathbf{h}$ has finitely
many isolated zeroes $k_{i}$ with the Weyl submanifold being their union
$W=\bigcup_{i}k_{i}$.

Although a band gap does not occur everywhere, the system is gapped along
submanifolds of the Brillouin zone which avoid $W$ and thus the Berry
curvature is well defined away from $W$. For each Weyl point, one defines a
local charge by integrating the Berry curvature on a sphere around a point.
Alternatively, one can normalise the map $\mathbf{h}$ to give a map
$\widetilde{\mathbf{h}}:\mathbb{T}^{3}\rightarrow S^{2}$. Restricting
$\widetilde{\mathbf{h}}$ to a small sphere around each Weyl point $k_{i}$
gives a collection of maps $\widetilde{\mathbf{h}}_{i}:S^{2}\rightarrow
S^{2}$, each map naturally having a degree which can be viewed as the local
charge of the $i$th Weyl point. However, it can be shown this is related to a
more cohomological picture. Namely, that the local charge of $k_{i}$ can be
viewed as the pullback under $\widetilde{\mathbf{h}}_{i}$ of the first Chern
class of the hyperplane line bundle over the surrounding sphere $S^{2}$.

These local charges provide obstructions to being able to continuously deform
the Hamiltonian so that it becomes a gapped system, and thus a topological
insulator. Furthermore, we can consider the restriction of the valence bundle
to subtori of $\mathbb{T}^{3}$, typically coordinate subtori defined by fixing
the value of one of the coordinates $k_{x},k_{y}$ or $k_{z}$ of the Brillouin
zone. These have three independent first Chern classes $c_{1}(V)_{x}$,
$c_{1}(V)_{y}$ and $c_{1}(V)_{z}$ , or equivalently, Chern numbers
$c_{1x},c_{1y},c_{1z}$ by pairing the cohomology classes with the coordinate
subtori, which remain constant due to continuity, until a Weyl point is
passed. Due to Stokes’ theorem, there is a jump in the Chern numbers by the
value of the local charge passed. It is also this behaviour which gives rise
Fermi arcs when a boundary is introduced that join the Weyl points. The
topological nature of these were also studied in [MT17, MT17a] via Euler
chains.

However, as their name suggests, local charges are only local in nature, and
do not provide the full picture. For the local charges to be Chern classes of
the valence bundle restricted to their surrounding spheres, a global
consistency condition is required which can be derived from the Mayer-Vietoris
sequence in cohomology [BT08]. The Mayer-Vietoris sequence relates the spaces
$\mathbb{T}^{3}\setminus W$, a tubular neighbourhood of $W$, which we shall
denote by $D_{W}$ and the their intersection, which is a space homotopic to
the spheres surrounding the Weyl points described above, denoted $S_{W}$. We
obtain an exact sequence of maps, and of particular interest is the end of the
sequence, namely

$\dots\rightarrow
H^{2}(\mathbb{T}^{3})\xrightarrow{\iota_{\mathbb{T}^{3}\setminus
W}^{*}}H^{2}(\mathbb{T}^{3}\setminus
W)\xrightarrow{\iota_{S_{W}}^{*}}H^{2}(S_{W})\xrightarrow{\Sigma}H^{3}(\mathbb{T}^{3})\rightarrow
0$

where $\iota_{\mathbb{T}^{3}\setminus W}$ and $\iota_{S_{W}}$ are the maps on
cohomology induced by inclusion. Meanwhile $H^{2}(S_{W})$ decomposes as a
direct sum
$H^{2}(S_{W})\cong\bigoplus_{i}H^{2}(S_{(i)})\cong\bigoplus_{i}\mathbb{Z}_{(i)}$
and $H^{3}(\mathbb{T}^{3})\cong\mathbb{Z}$, under these isomorphisms the map
$\Sigma$ is simply the sum of the integers in $H^{2}(S_{W})$, thus we view it
as a ’sum of charges’ map.

Exactness means that the kernel of one map is the image of the proceeding map,
since complex line bundles are classified by the integral second cohomology
groups of the base space, it follows that a complex line bundle over $S_{W}$
(which corresponds to a collection of local charges) is in the image of
$\iota_{S_{W}}^{*}$ i.e. the restriction of of a complex line bundle over
$\mathbb{T}^{3}\setminus W$ and hence is the line bundle of a Weyl semimetal,
if and only if the local charges are in the kernel of $\Sigma$. Since $\Sigma$
can be viewed as the sum of charges, this implies that local charges in a Weyl
semimetal must satisfy a charge cancellation condition, that is, the local
charges must all sum to zero.

Figure 1: Representation of a Weyl semimetal with two Weyl points, one of
charge $+1$ and the other with $-1$. This is required by the charge
cancellation condition. Above is a depiction of a Fermi Arc which occurs when
a boundary is a continuous curve of states on the boundary joining the two
Weyl points.

###  2.3 The Mayer-Vietoris sequence in the general case

Here we investigate the Mayer-Vietoris sequence for Weyl submanifolds of
general dimension in $\mathbb{T}^{n}$. A general two band model in dimension
$n$ has the form

$H(k)=\sum_{i=1}^{N}h^{i}(k)\sigma_{i}$

where $\sigma_{n}$ are representations of the generators of the complex
Clifford Algebra. Such a system is essentially specified by the real-valued
function $\mathbf{h}$ on $\mathbb{T}^{n}$ with components $h^{i}(k)$.

One could consider more complicated systems, such as one with multiple bands,
nonetheless, the following argument is purely topological and still applies.
Assume that $\mathbf{h}$ has zeroes on some submanifold $W$ which decomposes
into disjoint connected submanifolds $W_{i}$. Furthermore, we may surround
each $W_{i}$ with a tubular neighbourhood $D_{i}$ with boundary $S_{i}$,
importantly, these boundaries $S_{i}$ are of dimension $n-1$. Denote the
disjoint union of these submanifolds by $W$, $D_{W}$ and $S_{W}$ respectively,
then set $U=\mathbb{T}^{n}\setminus W$ and $V=D_{W}$. Applying the Mayer-
Vietoris sequence then gives the following result

######  Proposition 2.1.

Let $W=\sqcup_{i}W_{i}$ and $D_{i}$ be as above, then the following sequence
is exact

$\dots\rightarrow H^{p}(\mathbb{T}^{n})\rightarrow
H^{p}(\mathbb{T}^{n}\setminus
W)\oplus\bigoplus_{i}H^{p}(W_{i})\rightarrow\bigoplus_{i}H^{p}(S_{i})\xrightarrow{\Sigma}H^{p+1}(\mathbb{T}^{n})\rightarrow
0$

where $\Sigma$ is the boundary map of the Mayer-Vietoris sequence.

in particular for $n>2$ we have

$\dots\rightarrow
H^{n-1}(\mathbb{T}^{n})\xrightarrow{\iota}H^{n-1}(\mathbb{T}^{n}\setminus
W)\xrightarrow{\iota}\bigoplus_{i}H^{n-1}(D_{i})\xrightarrow{\Sigma}H^{n}(\mathbb{T}^{n})\rightarrow
0.$

Since the $D_{i}$ are $n-1$ dimensional compact connected submanifolds of
$\mathbb{T}^{n}$, it follows that $H^{n-1}(D_{i})\cong\mathbb{Z}_{(i)}$ which
we label with $i$ to keep track of the index $i$, we also have
$H^{n}(\mathbb{T}^{n})\cong\mathbb{Z}$. Under these isomorphisms, the map
$\Sigma$ is equivalent to a map
$\Sigma:\bigoplus_{i}\mathbb{Z}_{(i)}\rightarrow\mathbb{Z}$ defined by

$\Sigma(x_{1},\dots x_{k})=\sum_{i=1}^{k}x_{i}.$

To prove this it is easiest to see with deRham cohomology, which we may use
since the last two remaining terms in the integral cohomology groups are
torsion-free.

###### Proof.

Suppose that $\omega$ is an $n-1$ form representing a class in $H^{n-1}(U\cap
V)$. Under the isomorphism $H^{n-1}(U\cap V)\cong H^{n-1}(S_{W})$ it suffices
to consider an $n-1$ form defined on $S_{W}$ which is the restriction of
$\omega$ to the boundary $S_{W}=U\cap V$. Since $S_{W}$ decomposes as a
disjoint union, the cohomology group can be written as a direct sum

$H^{n-1}(S_{W})\cong\bigoplus_{i}H^{n-1}(S_{i})$

so we may write $\omega=\sum_{i}\omega_{i}$ where each $\omega_{i}$ has
support in $S_{i}$, the corresponding cohomology class is
$([\omega_{1}],\dots[\omega_{\ell}])$. We shall further take the restriction
of $\omega_{i}$ to the boundary $H^{n-1}(S_{i})$ and assume its integral is 1
so that it is a generator for the $i$-th component of $H^{n-1}(S_{W})$. Let
$\rho_{U},\rho_{V}$ be a partition of unity subordinate to ${U,V}$. The
boundary map of the Mayer-Vietoris sequence in deRham cohomology is defined by

$\displaystyle\Sigma(\omega)=\begin{cases}&[-d(\rho_{V}\omega)]\text{ on U}\\\
&[d(\rho_{U}\omega)]\text{ on V}\end{cases}$

this is a globally defined form on $\mathbb{T}^{n}$ supported in $U\cap V$,
since $H_{\text{dR}}^{n}(\mathbb{T}^{n})\cong\mathbb{R}$ with the isomorphism
given by integration over $\mathbb{T}^{n}$, we compute

$\displaystyle\int_{\mathbb{T}^{n}}\Sigma(\omega)$ $\displaystyle=\int_{U\cap
V}d(\rho_{U}\omega)$ $\displaystyle=\int_{\partial(U\cap V)}\rho_{U}\omega$
$\displaystyle=\int_{S_{W}}\rho_{U}\omega.$

However, $S_{W}$ is contained entirely in $V$ where $\rho_{U}=1$, hence

$\displaystyle\int_{S_{W}}\rho_{U}\omega$ $\displaystyle=\int_{S_{W}}\omega$
$\displaystyle=\sum_{i}\int_{S_{i}}\omega_{i}$

and so under the identification
$[\omega]=([\omega_{1}],\dots,[\omega_{\ell}])$, we see that

$\Sigma(\omega_{1},\dots,\omega_{\ell})=\sum_{i}\omega_{i}.$

∎

######  Remark 2.2.

Note that this result technically holds in deRham, not integral cohomology.
Provided all the singular cohomology groups in the Mayer-Vietoris sequence are
free, there is an injection $H^{q}(X)\hookrightarrow H^{q}(X)\otimes R\cong
H_{dR}^{q}(X)$. Consequently, we may choose the generators in the above proof
to have integral periods, and thus represent elements in singular cohomology.

######  Corollary 2.3.

As a consequence of exactness of the above sequence, cohomological local
charges in degree $n-1$ (elements of $H^{n-1}(D_{i})$) arise from degree $n-1$
characteristic classes of vector bundles over $\\\ T^{n}\setminus W$ (elements
of $H^{n-1}(\mathbb{T}^{n}\setminus W)$), if and only if the local
cohomological charges are in the kernel of the boundary map. That is, the
charges sum to zero.

If nodal lines had their own cohomological charge in degree $d-1$, then it
would follow from the above proposition that they must satisfy a charge
cancellation condition identical to that of Weyl semimetals. However, in the
key application of interest, namely $d=3$, this is not the case. To obtain
nodal lines in $3D$ generically requires one of the components of $\mathbf{h}$
to be zero, and so the corresponding map into the 2-sphere after normalisation
cannot be surjective, and thus has degree zero. So the standard procedure to
obtain a cohomological invariant does not work for nodal lines. On the other
hand, the standard local invariant for nodal lines is the berry phase around a
loop containing the nodal line, which is only well-defined modulo $2\pi$, and
thus cannot be interpreted as an element of integral cohomology.

Figure 2: Depiction of tubular neighbourhoods of a nodal link in the Brillouin
torus.

##  3 Semimetallic phases with a real symmetry

###  3.1 Examples of Hamiltonians with a real symmetry

One means to force the Weyl submanifold to have dimension 1 when the Brillouin
zone is 3 dimensional, is to force the Hamiltonian to be real. As long as
spin-orbit coupling is not considered, this can be achieved with space-time
inversion symmetry $I_{ST}$. Further details on this symmetry and its
connection to the Stiefel-Whitney classes is further discussed in [Avr+88].
This an anti-unitary symmetry satisfying $I_{ST}^{2}=1$, and can be obtained
in a 3D semimetal as $I_{ST}=P\circ T$ where $P$ spatial inversion and $T$ is
time-reversal. Hence in position space this is simply the map
$(t,\vec{r})\mapsto(-t,-\vec{r})$.

In the presence of space-time inversion symmetry, one can choose a basis in
which it is represented as complex conjugation[Fan+15, Kim+15]. In such a
case, the Hamiltonian and the states of the system are forced to be real.
Consequently, one is left to classify real vector bundles over
$\mathbb{T}^{n}\setminus W$ as opposed to complex vector bundles. Moreover,
since a two-band semimetal Hamiltonian generally has the form

$H(k)=\mathbf{h}(k)\cdot\vec{\sigma}$

in the presence of such a symmetry, coefficients of $\mathbf{h}$ which
correspond to $\sigma_{i}$ which do not have purely real entries are forced to
be zero. In the case of 3 dimensions, this forces $h^{2}=0$. Giving rise to
the Hamiltonian

$H(k)=h^{1}(k)\sigma_{1}+h^{3}(k)\sigma_{3}.$

which generically has a 1 dimensional zero set, which in the most general case
can be a knot or a link. As such we refer to these as knotted semimetals,
although they are most often referred to as nodal loop or nodal line
semimetals in the physics literature.

Since the corresponding vector bundle over $\mathbb{T}^{3}\setminus W$ is
real, the appropriate characteristic classes for their classification are the
Stiefel-Whitney classes. These live in cohomology with $\mathbb{Z}_{2}$
coefficients and thus give rise to numbers which are $\mathbb{Z}_{2}$ valued
when paired with homology classes. Just as with the pairing of Chern classes
with distinguished homology classes to give rise to the local charges of Weyl
points, it was shown in[Ahn+19] that the first and second Stiefel-Whitney
classes give rise to two charges of nodal loops in momentum space, namely the
quantized Berry phase and the $\mathbb{Z}_{2}$ monopole charge respectively.
The first Stiefel-Whitney class can be paired with a loop that encloses a
nodal line, and measures the non-orientability of the valence band restricted
to this loop.

One should keep in mind that generically in a two-band system, the only
invariant is the first Stiefel-Whitey class. To obtain a nodal semimetal with
non-zero $\mathbb{Z}_{2}$-monopole charges, one must consider systems with
more than two bands, for instance one described by a Hamiltonian of the form

$H(k)=k_{x}\mathbf{1}\otimes\sigma_{x}+k_{y}\tau_{y}\otimes\sigma_{y}+k_{z}\mathbf{1}\otimes\sigma_{z}+m\tau_{z}\otimes\sigma_{z}$

where the $\sigma_{i},\tau_{i}$ are Pauli matrices. The spectrum is given by
$E=\pm\sqrt{k_{x}^{2}+(\rho\pm|m|)^{2}}$ with
$\rho=\sqrt{k_{y}^{2}+k_{z}^{2}}$. This yields a system with two linked nodal
loops at $k_{x}=0$ and $\rho=0$ obtained from the crossing of two distinct
sets of occupied bands.

The $\mathbb{Z}_{2}$ monopole charge is an obstruction to deforming the nodal
loop to a point, which could then be further made to vanish, while
mathematically, the second Stiefel-Whitney class of a vector bundle is an
obstruction to defining a spin structure on said vector bundle, and is a
cohomology class on the base space on the bundle.

###  3.2 A proof of charge cancellation

It is claimed that the $\mathbb{Z}_{2}$ monopole charge in particular has a
mod 2 charge cancellation condition. However, in this section we provide a
manifestly topological proof of this fact in dimension 3 by extending the
Mayer-Vietoris sequence in integer cohomology to the $\mathbb{Z}_{2}$ case.

The idea is generally simple, namely to apply the well-known universal
coefficient theorem to the integral cohomology sequence. However, there is
subtelty required in justifying this application and using it to ensure an
exact sequence of the appropriate cohomology groups, and that the resulting
final boundary map is indeed the sum of charges modulo 2.

The following is a general statement of the universal coefficient theorem for
free chain complexes [Spa95, Ch. 5, Thm. 10]

######  Theorem 3.1.

Let $C$ be a free chain complex such that $H(C)$ is of finite type or $G$ is
finitely generated, then there is a functorial short exact sequence

$0\rightarrow H^{q}(C;R)\otimes
G\xrightarrow{\mu}H^{q}(C;G)\rightarrow\operatorname{Tor}(H^{q+1}(C;R),G)\rightarrow
0$

and this sequence also splits.

Note that the map is induced by the map $\mu:\operatorname{Hom}(A,G)\otimes
G^{\prime}\rightarrow\operatorname{Hom}(A,G\otimes G^{\prime})$ acting on
singular cochain complexes and is given by

$\mu(f\otimes g^{\prime})(a):=f(a)\otimes g^{\prime}$

where $f\in\operatorname{Hom}(A,G),g\in G^{\prime}$ and $a\in A$.

We apply this proposition in the case when $R=\mathbb{Z}$ and
$G=\mathbb{Z}_{2}$. Under the condition that all the cohomology groups are
free, the torsion term vanishes in the exact sequence. It follows that
$H^{q}(C;\mathbb{Z})\otimes\mathbb{Z}_{2}\cong H^{q}(C;\mathbb{Z}_{2})$.
Furthermore, taking the tensor product is functorial. It follows that
tensoring a sequence of $\mathbb{Z}$-modules by $\mathbb{Z}_{2}$ produces
another sequence of $\mathbb{Z}$-modules.with the maps induced by taking the
tensor product with the identity on the $\mathbb{Z}_{2}$ factor.

It is of note, that the tensor product is not always an exact functor. The
stereotypical example of this behaviour being the fact that the sequence

$0\rightarrow Z\xrightarrow{\cdot
2x}\mathbb{Z}\rightarrow\mathbb{Z}_{2}\rightarrow 0$

is exact, but the tensoring this sequence by $\mathbb{Z}_{2}$ only gives that
the sequence

$\mathbb{Z}_{2}\xrightarrow{0}\mathbb{Z}_{2}\xrightarrow{\text{id}}\mathbb{Z}_{2}\rightarrow
0$

with the leftmost map no longer injective.

In order to justify exactness we shall use a fairly standard of result of
commutative algebra, consider Proposition 1.2.6 of [Liu10] from which a fairly
elementary fact of commutative algebra follows, namely the following
proposition

######  Proposition 3.2.

Let $0\rightarrow M_{1}\rightarrow M_{2}\rightarrow\dots\rightarrow
M_{k}\rightarrow 0$ be a short exact sequence of free $\mathbb{Z}$-modules and
$N$ an arbitrary $\mathbb{Z}$-module. Then the induced sequence given by
tensoring with $N$

$0\rightarrow M_{1}\otimes N\rightarrow\dots\rightarrow M_{k}\otimes
N\rightarrow 0$

is also exact.

###### Proof.

Since the sequence is finite, one may break up the original long exact
sequence into a series of short exact sequences, starting from the right.
Since the right-most two modules are free, the left most entry is also free.
Proceeding by induction yields the result. ∎

######  Remark 3.3.

This result holds more generally over for modules over any ring if one
replaces ’free’ with ’flat’, although since we are only considering
$\mathbb{Z}$-modules, the two notions are equivalent and we choose to work
with the former for simplicity.

It remains to show that each term in the Mayer-Vietoris sequence is free.
Assume that $X$ is any compact, oriented topological space. It is known that
$H^{0}(X)$ is simply a free $\mathbb{Z}$-module with rank counting the number
of connected components of $X$ and is thus free [Hat02]. The universal
coefficient theorem for finitely generated abelian groups [Spa95] gives a non-
canonical isomorphism

$H^{p}(X)\cong\operatorname{Hom}(H_{p}(X),\mathbb{Z})\oplus\operatorname{Ext}(H_{p-1}(X),\mathbb{Z})$

from which it follows that $H^{1}(X)$ is free, since the Ext term is zero and
the Hom term is always free, Poincaré duality gives $H^{2}(X)\cong H_{1}(X)$.

Now $H_{1}(X)$ is not generally torsion free. However, for the spaces we are
interested in this is the case. This fact essentially follows since $H_{1}$ is
the abelianisation of the fundamental group. In the case of
$\mathbb{T}^{3}\setminus W$, one picks up extra $\mathbb{Z}$ factors from each
nodal loop in the fundamental group, still producing a free first homology
group, and $U\cap V$ for nodal loops is homotopic to a disjoint union of
2-dimensional tori, each having $\mathbb{Z}^{2}$ as their first homology, and
the union being their direct sum.

Finally, the only group of degree $3$ in the Mayer-Veitoris sequence in 3
dimensions is $H^{3}(\mathbb{T}^{3})$, which is isomorphic to
$H_{0}(\mathbb{T}^{3})$ by Poincaré duality, hence torsion free also.

It follows from the above argument that the following sequence, obtained by
taking the tensor product of the Mayer-Vietoris sequence in integral
cohomology with $\mathbb{Z}_{2}$

$\dots\xrightarrow{\partial}H^{2}(\mathbb{T}^{3};\mathbb{Z}_{2})\rightarrow
H^{2}(\mathbb{T}^{3}\setminus
W;\mathbb{Z}_{2})\rightarrow\bigoplus_{i}H^{2}(\mathbb{T}^{2}_{(i)};\mathbb{Z}_{2})\xrightarrow{\Sigma\text{
(mod 2)}}H^{3}(\mathbb{T}^{3};\mathbb{Z})\rightarrow 0$

is exact.

Furthermore, under the isomorphisms
$\mathbb{Z}^{k}\otimes\mathbb{Z}_{2}\cong(\mathbb{Z}\otimes\mathbb{Z}_{2})^{k}\cong\mathbb{Z}_{2}^{k}$
given by $(x_{1},\dots,x_{k})\otimes a\mapsto(x_{1}\otimes
a,\dots,x_{k}\otimes a)\mapsto(x_{1}(\text{mod}2),\dots x_{k}(\text{mod}2))$.

######  Corollary 3.4.

Due to exactness elements of
$\bigoplus_{i}H^{2}(\mathbb{T}^{2}_{(i)};\mathbb{Z}_{2})$ come from the
restriction of an element of $H^{2}(\mathbb{T}^{3}\setminus W;\mathbb{Z})$,
i.e. non-zero $\mathbb{Z}_{2}$ monopole charges arise from a vector bundle
over $\mathbb{T}^{3}\setminus W$, if and only if they are in the kernel of the
boundary map in the Mayer-Vietoris sequence. That is, if and only if the
charges sum to zero modulo 2.

## References

  * [Ahn+19] Junyeong Ahn et al.  “Stiefel–Whitney Classes and Topological Phases in Band Theory”  In _Chinese Physics B_ 28.11 Chinese Physical Society and IOP Publishing Ltd, 2019, pp. 117101  DOI: 10.1088/1674-1056/ab4d3b
  * [Avr+88] J. E. Avron, L. Sadun, J. Segert and B. Simon  “Topological Invariants in Fermi Systems with Time-Reversal Invariance”  In _Physical Review Letters_ 61.12, 1988, pp. 1329–1332  DOI: 10.1103/PhysRevLett.61.1329
  * [BHB11] A. A. Burkov, M. D. Hook and Leon Balents  “Topological Nodal Semimetals”  In _Physical Review B_ 84.23, 2011, pp. 235126  DOI: 10.1103/PhysRevB.84.235126
  * [BT08] Raoul Bott and Loring Wuliang Tu  “Differential Forms in Algebraic Topology”, Graduate Texts in Mathematics 82  New York: Springer, 2008
  * [Fan+15] Chen Fang, Yige Chen, Hae-Young Kee and Liang Fu  “Topological Nodal Line Semimetals with and without Spin-Orbital Coupling”  In _Physical Review B_ 92.8, 2015, pp. 081201  DOI: 10.1103/PhysRevB.92.081201
  * [FM13] Daniel S. Freed and Gregory W. Moore  “Twisted Equivariant Matter”  In _Annales Henri Poincaré_ 14.8, 2013, pp. 1927–2023  DOI: 10.1007/s00023-013-0236-x
  * [Hat02] Allen Hatcher  “Algebraic Topology”  Cambridge ; New York: Cambridge University Press, 2002
  * [Kim+15] Youngkuk Kim, Benjamin J. Wieder, C. L. Kane and Andrew M. Rappe  “Dirac Line Nodes in Inversion-Symmetric Crystals”  In _Physical Review Letters_ 115.3, 2015, pp. 036806  DOI: 10.1103/PhysRevLett.115.036806
  * [KLF09] Alexei Kitaev, Vladimir Lebedev and Mikhail Feigel’man  “Periodic Table for Topological Insulators and Superconductors”  In _AIP Conference Proceedings_ Chernogolokova (Russia): AIP, 2009, pp. 22–30  DOI: 10.1063/1.3149495
  * [Liu10] Qing Liu  “Algebraic Geometry and Arithmetic Curves”, Oxford Graduate Texts in Mathematics 6  Oxford: Oxford Univ. Press, 2010
  * [Mat+13] Shunji Matsuura, Po-Yao Chang, Andreas P Schnyder and Shinsei Ryu  “Protected Boundary States in Gapless Topological Phases”  In _New Journal of Physics_ 15.6, 2013, pp. 065001  DOI: 10.1088/1367-2630/15/6/065001
  * [MT17] Varghese Mathai and Guo Chuan Thiang  “Differential Topology of Semimetals”  In _Communications in Mathematical Physics_ 355.2, 2017, pp. 561–602  DOI: 10.1007/s00220-017-2965-z
  * [MT17a] Varghese Mathai and Guo Chuan Thiang  “Global Topology of Weyl Semimetals and Fermi Arcs”  In _Journal of Physics A: Mathematical and Theoretical_ 50.11, 2017, pp. 11LT01  DOI: 10.1088/1751-8121/aa59b2
  * [Spa95] Edwin Henry Spanier  “Algebraic Topology”  New York Berlin Barcelona Budapest: Springer, 1995
  * [SS23] Hisham Sati and Urs Schreiber  “Anyonic Topological Order in Twisted Equivariant Differential (TED) K-theory”  In _Reviews in Mathematical Physics_ 35.03, 2023, pp. 2350001  DOI: 10.1142/S0129055X23500010
  * [Thi16] Guo Chuan Thiang  “On the K-Theoretic Classification of Topological Phases of Matter”  In _Annales Henri Poincaré_ 17.4, 2016, pp. 757–794  DOI: 10.1007/s00023-015-0418-9
  * [Van18] David Vanderbilt  “Berry Phases in Electronic Structure Theory: Electric Polarization, Orbital Magnetization and Topological Insulators”  Cambridge: Cambridge University Press, 2018
  * [Wan+11] Xiangang Wan, Ari M. Turner, Ashvin Vishwanath and Sergey Y. Savrasov  “Topological Semimetal and Fermi-arc Surface States in the Electronic Structure of Pyrochlore Iridates”  In _Physical Review B_ 83.20, 2011, pp. 205101  DOI: 10.1103/PhysRevB.83.205101
  * [Xu+15] Su-Yang Xu et al.  “Discovery of a Weyl Fermion Semimetal and Topological Fermi Arcs”  In _Science_ 349.6248, 2015, pp. 613–617  DOI: 10.1126/science.aaa9297
  * [ZL17] Y. X. Zhao and Y. Lu  “P T -Symmetric Real Dirac Fermions and Semimetals”  In _Physical Review Letters_ 118.5, 2017, pp. 056401  DOI: 10.1103/PhysRevLett.118.056401


$\displaystyle\frac{||h_{S}-h_{S_{j}^{i}}||_{2}^{2}}{\sigma_{\text{max}}}\leq\text{tr}\left((H_{S}-H_{S_{j}^{i}})\Omega^{-1}(H_{S}-H_{S_{j}^{i}})^{\top}\right)\leq\frac{\rho
R}{NK}||h_{S}-h_{S_{j}^{i}}||_{2}\Rightarrow||h_{S}-h_{S_{j}^{i}}||_{2}\leq\frac{\sigma_{\text{max}}\rho
R}{NK},$

which gives

$\displaystyle\beta\leq\frac{\sigma_{\text{max}}\rho^{2}R^{2}}{NK}.$

Proof [Proof of Lemma 36] For any task $j$, by the definition of $h_{j}^{*}$
and $\bar{h}_{j}$, we have
$\mathcal{L}_{S_{j}}(\bar{h}_{j})+{\frac{K}{\sigma_{\text{max}}}}||\bar{h}_{j}||_{2}^{2}\leq\mathcal{L}_{S_{j}}(h_{j}^{*})+{\frac{K}{\sigma_{\text{max}}}}||h_{j}^{*}||_{2}^{2}$.
On the other hand, we also have

$\displaystyle\frac{1}{K}\sum_{k=1}^{K}\mathcal{L}_{S_{k}}(h_{k}^{*})+\frac{1}{\sigma_{\text{max}}}\text{tr}({H^{*}}^{\top}H^{*})$
$\displaystyle\leq\frac{1}{K}\sum_{k=1}^{K}\mathcal{L}_{S_{k}}(h_{k}^{*})+\text{tr}(H^{*}\Omega^{-1}{H^{*}}^{\top})$
$\displaystyle\leq\frac{1}{K}\sum_{k=1}^{K}\mathcal{L}_{S_{k}}(\bar{h}_{0})+\text{tr}(\bar{H}_{0}\Omega^{-1}\bar{H}_{0}^{\top}),$

where $\bar{H}_{0}=[\bar{h}_{0},\dots,\bar{h}_{0}]$. Since
$\text{tr}(H_{0}\Omega^{-1}H_{0}^{\top})=\omega||h_{0}||_{2}^{2}$, where
$\omega$ is the sum of the elements of $\Omega^{-1}$, for any task $j$, we
have

$\displaystyle\frac{K}{\sigma_{\text{max}}(\Omega)}||h_{j}^{*}||_{2}^{2}$
$\displaystyle\leq\sum_{k=1}^{K}\mathcal{L}_{S_{k}}(\bar{h}_{0})+K\omega||\bar{h}_{0}||_{2}^{2}-\sum_{k\neq
j}\left[\mathcal{L}_{S_{k}}(h_{k}^{*})+\frac{K}{\sigma_{\text{max}}(\Omega)}||h_{k}^{*}||_{2}^{2}\right]-\mathcal{L}_{S_{j}}(h_{j}^{*})$
$\displaystyle\leq\sum_{k=1}^{K}\mathcal{L}_{S_{k}}(\bar{h}_{0})+K\omega||\bar{h}_{0}||_{2}^{2}-\sum_{k\neq
j}\left[\mathcal{L}_{S_{k}}(\bar{h}_{k})+\frac{K}{\sigma_{\text{max}}(\Omega)}||\bar{h}_{k}||_{2}^{2}\right]-\mathcal{L}_{S_{j}}(h_{j}^{*})$
$\displaystyle\leq\nabla+\mathcal{V}_{j}(\bar{h}_{j})-\mathcal{L}_{S_{j}}(h_{j}^{*})$
$\displaystyle\leq\nabla+\frac{K}{\sigma_{\text{max}}(\Omega)}||\bar{h}_{j}||_{2}^{2},$

which gives

$\displaystyle||h^{*}_{j}||_{2}\leq\sqrt{\frac{\sigma_{\text{max}}\nabla}{K}+||\bar{h}_{j}||_{2}^{2}}\enspace.$

###  A.7 $\operatorname{\mathsf{gapBoost}}$

In order to prove Proposition 38, we need the following auxiliary result.

###### Lemma 45

Let $\mathcal{H}$ be a hypothesis class of real-valued functions returned by
the transfer learning algorithm (35)

$\displaystyle\min_{h\in\mathcal{H}}\mathcal{L}_{S}^{\Gamma}(h)+\lambda\mathcal{R}(h)\enspace,$
(35)

with a $\rho$-Lipschitz continuous loss function. The convex hull of
$\mathcal{H}$ is defined as

$\displaystyle\mathcal{F}=\left\\{\sum_{k=1}^{K}\mu_{k}h_{k}(x):\sum_{k=1}^{K}\mu_{k}=1,\mu_{k}\geq
0,h_{k}\in\mathcal{H},\forall k=\\{1,\dots,K\\}\right\\}.$

Define $\Gamma=[\Gamma_{1},\dots,\Gamma_{K}]\in\mathbb{R}^{N\times K}$, where
for any $k\in 1,\dots,K$,
$\Gamma_{k}=[\Gamma_{k}^{\mathcal{T}};\Gamma^{\mathcal{S}}_{k}]=[\gamma_{1,k}^{\mathcal{T}},\dots,\gamma_{N_{\mathcal{S}},k}^{\mathcal{S}}]^{\top}\in\mathbb{R}^{N}$
are the weights for the $k$-th base learner. Then, for any $\delta\in(0,1)$,
we probability at least $1-\delta$, we have

$\displaystyle\mathfrak{R}_{\mathcal{D}_{\mathcal{T}}}(\mathcal{F})\leq\frac{\gamma_{\infty}^{\mathcal{T}}\rho
R^{2}}{\lambda}\sqrt{2N\log\frac{2}{\delta}}$

where
$\gamma_{\infty}^{\mathcal{T}}=\max_{k}\\{\|\Gamma_{k}^{\mathcal{T}}\|_{\infty}\\}_{k=1}^{K}$
is the largest weight of the target sample over all the boosting iterations.

Proof  We derive the generalization bound from the unweighted target training
sample, treating source domain sample can be treated as a regularizer (Liu et
al., 2017c, b). Then, follow the similar proof schema as in Lemma 7.4 of
(Mohri et al., 2018), we have

$\displaystyle\mathfrak{R}_{\mathcal{D}_{\mathcal{T}}}(\mathcal{F})$
$\displaystyle=\frac{1}{N_{\mathcal{T}}}\mathbb{E}\sup_{\begin{subarray}{c}h_{1}\in\mathcal{H},\dots,h_{K}\in\mathcal{H}\\\
\mu_{k}\geq 0,\sum_{k=1}^{K}\mu_{k}\leq
1\end{subarray}}\sum_{i=1}^{N_{\mathcal{T}}}\sigma_{i}\sum_{k=1}^{K}\mu_{k}h_{k}(x_{i})$
$\displaystyle=\frac{1}{N_{\mathcal{T}}}\mathbb{E}\sup_{\begin{subarray}{c}h_{1}\in\mathcal{H},\dots,h_{K}\in\mathcal{H}\\\
\mu_{k}\geq 0,\sum_{k=1}^{K}\mu_{k}\leq
1\end{subarray}}\sum_{k=1}^{K}\mu_{k}\left(\sum_{i=1}^{N_{\mathcal{T}}}\sigma_{i}h_{k}(x_{i})\right)$
$\displaystyle=\frac{1}{N_{\mathcal{T}}}\mathbb{E}\sup_{h_{1}\in\mathcal{H},\dots,h_{K}\in\mathcal{H}}\max_{k\in\\{1,\dots,K\\}}\sum_{i=1}^{N_{\mathcal{T}}}\sigma_{i}\langle
h_{k}-\mathbb{E}h_{k,S},x_{i}\rangle$
$\displaystyle\leq\frac{1}{N_{\mathcal{T}}}\mathbb{E}\max_{k\in\\{1,\dots,K\\}}\sup_{h_{1}\in\mathcal{H},\dots,h_{K}\in\mathcal{H}}\|h_{k}-\mathbb{E}h_{k,S}\|\left|\left|\sum_{i=1}^{N_{\mathcal{T}}}\sigma_{i}x_{i}\right|\right|_{2}$
$\displaystyle\leq\frac{1}{N_{\mathcal{T}}}\max_{k\in\\{1,\dots,K\\}}\left\\{\frac{\|\Gamma_{k}^{\mathcal{T}}\|_{\infty}\rho
R}{\lambda}\sqrt{2N_{\mathcal{T}}\log\frac{2}{\delta}}\right\\}\mathbb{E}\left|\left|\sum_{i=1}^{N_{\mathcal{T}}}\sigma_{i}\gamma_{i}^{k}x_{i}\right|\right|_{2}$
$\displaystyle\leq\frac{\gamma_{\infty}^{\mathcal{T}}\rho
R}{N_{\mathcal{T}}\lambda}\sqrt{2N_{\mathcal{T}}\log\frac{2}{\delta}}R\sqrt{N_{\mathcal{T}}}=\frac{\gamma_{\infty}^{\mathcal{T}}\rho
R^{2}}{\lambda}\sqrt{2\log\frac{2}{\delta}}$

Note that compared with Lemma 7.4 of (Mohri et al., 2018), the main difference
in our proof is that for each base learner, its hypothesis class defined by
learning algorithm (35) is different from others.

Proof [Proof of Proposition 38] Given Lemma 45, by following the standard
proof schema as in (Mohri et al., 2018), we immediately obtain Proposition 38.

###  A.8 $\operatorname{\mathsf{gapMTNN}}$

Proof [Proof of Proposition 40] The prediction loss for each task
$\mathcal{D}_{t}$ can be expressed as:

$\displaystyle\mathcal{L}_{\mathcal{D}_{t}}(h\circ g)$
$\displaystyle=\frac{1}{|\mathcal{Y}|}\sum_{y}\mathcal{L}_{\mathcal{D}_{t}(z|Y=y)}(h)$

According to (Shui et al., 2021), as for another task $k$, we have the
following inequality:

$\displaystyle\mathcal{L}_{\mathcal{D}_{j}}(h_{j}^{*})=\frac{1}{|\mathcal{Y}|}\sum_{y}\mathcal{L}_{\mathcal{D}_{j}(z|Y=y)}(h_{j}^{*})\leq\frac{1}{|\mathcal{Y}|}\sum_{y}\mathcal{L}_{\mathcal{D}_{k}(z|Y=y)}(h_{j}^{*})+\frac{\nu\rho}{|\mathcal{Y}|}\sum_{y}W_{1}(\mathcal{D}_{j}(z|y)\|\mathcal{D}_{k}(z|y))$

For any task $k\neq j$, we assign $\gamma^{j}_{k}$ for each task, then we
have:

$\displaystyle\mathcal{L}_{\mathcal{D}_{j}}(h_{j}^{*})=\sum_{k}\Gamma_{t}[k]\mathcal{L}_{\mathcal{D}_{j}}(h_{j}^{*})$
$\displaystyle\leq\sum_{k=1}^{K}\gamma^{j}_{k}\mathcal{L}_{\mathcal{D}_{k}}(h^{*}_{j})+\frac{\nu\rho}{|\mathcal{Y}|}\sum_{k}\gamma^{j}_{k}\sum_{y}W_{1}(\mathcal{D}_{t}(z|y)\|\mathcal{D}_{k}(z|y))$
$\displaystyle=\mathcal{L}^{\Gamma^{j}}_{\mathcal{D}}(h_{j}^{*})+\frac{\nu\rho}{|\mathcal{Y}|}\sum_{k\neq
j}\gamma^{j}_{k}\sum_{y}W_{1}(\mathcal{D}_{t}(z|y)\|\mathcal{D}_{k}(z|y))$

##  B Implementation Details for the Multitask Learning Experiments

We evaluate the algorithm on PACS, Office-31 and Office-home data sets. In
this part we show the implementation details about these experiments.

###  B.1 Network architecture

For experiments on PACS, we adopt the pre-trained AlexNet provided in Pytorch
while removing the last FC layer as feature extractor, which leads to the
output feature size as 4096. Then, we implement several three-MLPs for the
task-specific classification (4096-256-RELU-# classes), the architecture of
the classification network is,

  * •

(0): Linear(in_features=4096, out_features=256, bias=True)

  * •

(1): Linear(in_features=256, out_features= # classes, bias=True)

  * •

(2): Softmax(dim=-1)

For the Office-31 and Office-home data sets, we adopt pre-trained ResNet-18 in
Pytorch while removing the last FC layers as feature extractor. The output
size of the feature extractor is 512. Then, we also implement a MLP for the
classification (512-256-RELU-# classes))

  * •

(0): Linear(in_features=512, out_features=256, bias=True)

  * •

(1): Linear(in_features=256, out_features=# classes, bias=True)

  * •

(2): Softmax(dim=-1)

# classes of PACS data sets is 7, # classes of Office-31 is 31 and # classes
of Office-home is 65. When computing the semantic loss, we extract the
features with size 256.

###  B.2 data set processing and hyper-parameter setup

For PACS data set, we adopt the same original split of train / validation /
test splits of (Li et al., 2017). Then, for training the set, we randomly
select $10\%,15\%,20\%$ of the total instances in the original training set
and use the original validation set for developing the model and report the
empirical results on the testing set. We train the model with mini-batch size
$64$. We didn’t specify random seed for the random selection process. We
follow the pre-processing protocol of (Carlucci et al., 2019) to process the
data set. For the office-31 and office-home data sets, we adopt the same train
/ validation / test splits strategy of (Long et al., 2017; Zhou et al.,
2021a). We randomly select $5\%,10\%,20\%$ of the instances from the training
set to train the model. The validation set is used for developing the model
while we test on the rest test data set. As mentioned in the manuscript, the
Office-Caltech data set is relatively small, we only train the model using
$20\%$ of the total instances. For Office-31 and Office-Caltech we use
training batch size $16$ and for Office-home data set we use training batch
size $24$.

We adopt the Adam optimizer to train the model for the three data sets. The
initial learning rate was set to $2\times 10^{-4}$. During the training
process, we decay the learning rate $5\%$ for every 5 epochs. To enforce a
$L_{2}$ regularization, we also enable the weight decay option in Adam
provided by Pytorch. The relation coefficients $\mathbf{\alpha}$ is
initialized by $1/\\#\text{tasks}$ for each class, then it is optimized
dynamically as the training goes on. We set a weight $0.1$ to regularize the
semantic loss and the marginal alignment objective.

All the experiments were repeated for 10 times on Intel Gold 6148 Skylake 2.4
GHz CPU and 2 x NVidia V100SXM2 (16G memory) GPU. The main software packages
using for the experiments are: Pytorch version 1.0, Torchvision Version 0.2,
Python version 3.6, Python CVXPY package 1.0.

## References

  * Acuna et al. (2020) David Acuna, Guojun Zhang, Marc T Law, and Sanja Fidler.  $f$-domain-adversarial learning: Theory and algorithms for unsupervised domain adaptation with neural networks.  2020\.
  * Argyriou et al. (2007) Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil.  Multi-task feature learning.  In _NIPS_ , pages 41–48, 2007.
  * Arjovsky et al. (2017) Martin Arjovsky, Soumith Chintala, and Léon Bottou.  Wasserstein GAN.  _arXiv preprint arXiv:1701.07875_ , 2017.
  * Bard (2013) Jonathan F Bard.  _Practical bilevel optimization: algorithms and applications_ , volume 30.  Springer Science & Business Media, 2013.
  * Baxter (2000) Jonathan Baxter.  A model of inductive bias learning.  _Journal of Artificial Intelligence Research_ , 12(149-198):3, 2000.
  * Ben-David et al. (2007) Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira.  Analysis of representations for domain adaptation.  In _Advances in Neural Information Processing Systems_ , pages 137–144, 2007.
  * Ben-David et al. (2010) Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan.  A theory of learning from different domains.  _Machine Learning_ , 79(1-2):151–175, 2010.
  * Bingel and Søgaard (2017) Joachim Bingel and Anders Søgaard.  Identifying beneficial task relations for multi-task learning in deep neural networks.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 164–169, 2017.
  * Blitzer et al. (2008) John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman.  Learning bounds for domain adaptation.  In _Advances in Neural Information Processing Systems_ , pages 129–136, 2008.
  * Bousquet and Elisseeff (2002) Olivier Bousquet and André Elisseeff.  Stability and generalization.  _Journal of Machine Learning Research_ , 2:499–526, 2002\.
  * Boyd and Vandenberghe (2004) Stephen Boyd and Lieven Vandenberghe.  _Convex Optimization_.  Cambridge university press, 2004.
  * Carlucci et al. (2019) Fabio Maria Carlucci, Antonio D’Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi.  Domain generalization by solving jigsaw puzzles.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019.
  * Chen et al. (2018) Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich.  Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks.  In _International Conference on Machine Learning_ , pages 794–803. PMLR, 2018.
  * Cortes et al. (2008) Corinna Cortes, Mehryar Mohri, Michael Riley, and Afshin Rostamizadeh.  Sample selection bias correction theory.  In _Proceedings of the International Conference on Algorithmic Learning Theory_ , pages 38–53. Springer, 2008.
  * Dai et al. (2007) Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu.  Boosting for transfer learning.  In _Proceedings of the International Conference on Machine Learning_ , pages 193–200, 2007.
  * Dou et al. (2019) Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker.  Domain generalization via model-agnostic learning of semantic features.  _Advances in Neural Information Processing Systems_ , 32:6450–6461, 2019.
  * Drucker (1997) Harris Drucker.  Improving regressors using boosting techniques.  In _Proceedings of the International Conference on Machine Learning_ , pages 107–115, 1997.
  * Eaton and desJardins (2011) Eric Eaton and Marie desJardins.  Selective transfer between learning tasks using task-based boosting.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 337–342, 2011.
  * Edmonds et al. (2020) Mark Edmonds, Xiaojian Ma, Siyuan Qi, Yixin Zhu, Hongjing Lu, and Song-Chun Zhu.  Theory-based causal transfer: Integrating instance-level induction and abstract-level structure learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 1283–1291, 2020.
  * Efron et al. (2004) Bradley Efron, Trevor Hastie, Iain Johnstone, and Robert Tibshirani.  Least angle regression.  _The Annals of statistics_ , 32(2):407–499, 2004\.
  * Evgeniou and Pontil (2004) Theodoros Evgeniou and Massimiliano Pontil.  Regularized multi-task learning.  In _SIGKDD_ , pages 109–117, 2004.
  * Franceschi et al. (2018) Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil.  Bilevel programming for hyperparameter optimization and meta-learning.  In _International Conference on Machine Learning_ , pages 1568–1577, 2018.
  * Freund and Schapire (1997) Yoav Freund and Robert E Schapire.  A decision-theoretic generalization of on-line learning and an application to boosting.  _Journal of Computer and System Sciences_ , 55(1):119–139, 1997.
  * Friedman (1991) Jerome H Friedman.  Multivariate adaptive regression splines.  _The annals of statistics_ , pages 1–67, 1991.
  * Ganin et al. (2016) Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  _The Journal of Machine Learning Research_ , 17(1):2096–2030, 2016.
  * Gong et al. (2012) Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman.  Geodesic flow kernel for unsupervised domain adaptation.  In _2012 IEEE conference on computer vision and pattern recognition_ , pages 2066–2073. IEEE, 2012.
  * Gretton et al. (2007) Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alex J Smola.  A kernel method for the two-sample-problem.  In _Advances in Neural Information Processing Systems_ , pages 513–520, 2007.
  * Herbster et al. (2020) Mark Herbster, Stephen Pasteris, and Lisa Tse.  Online multitask learning with long-term memory.  _arXiv preprint arXiv:2008.07055_ , 2020.
  * Huang et al. (2007) Jiayuan Huang, Arthur Gretton, Karsten M Borgwardt, Bernhard Schölkopf, and Alex J Smola.  Correcting sample selection bias by unlabeled data.  In _Advances in Neural Information Processing Systems_ , pages 601–608, 2007.
  * Kalan et al. (2020) Seyed Mohammadreza Mousavi Kalan, Zalan Fabian, A Salman Avestimehr, and Mahdi Soltanolkotabi.  Minimax lower bounds for transfer learning with linear and one-hidden layer neural networks.  _arXiv preprint arXiv:2006.10581_ , 2020.
  * Kumar and Daumé III (2012) Abhishek Kumar and Hal Daumé III.  Learning task grouping and overlap in multi-task learning.  In _Proceedings of the International Conference on Machine Learning_ , pages 1723–1730. Omnipress, 2012.
  * Kuzborskij and Orabona (2013) Ilja Kuzborskij and Francesco Orabona.  Stability and hypothesis transfer learning.  In _Proceedings of the International Conference on Machine Learning_ , pages 942–950, 2013.
  * Kuzborskij and Orabona (2017) Ilja Kuzborskij and Francesco Orabona.  Fast rates by transferring from auxiliary hypotheses.  _Machine Learning_ , 106(2):171–195, 2017.
  * Li et al. (2021) Bo Li, Yezhen Wang, Shanghang Zhang, Dongsheng Li, Kurt Keutzer, Trevor Darrell, and Han Zhao.  Learning invariant representations and risks for semi-supervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1104–1113, 2021.
  * Li et al. (2017) Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales.  Deeper, broader and artier domain generalization.  In _Proceedings of the IEEE international conference on computer vision_ , pages 5542–5550, 2017.
  * Liang et al. (2020) Kaizhao Liang, Jacky Y Zhang, Oluwasanmi Koyejo, and Bo Li.  Does adversarial transferability indicate knowledge transferability?  _arXiv preprint arXiv:2006.14512_ , 2020.
  * Liu et al. (2017a) Pengfei Liu, Xipeng Qiu, and Xuanjing Huang.  Adversarial multi-task learning for text classification.  _arXiv preprint arXiv:1704.05742_ , 2017a.
  * Liu et al. (2017b) Tongliang Liu, Dacheng Tao, Mingli Song, and Stephen J Maybank.  Algorithm-dependent generalization bounds for multi-task learning.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 39(2):227–241, 2017b.
  * Liu et al. (2017c) Tongliang Liu, Qiang Yang, and Dacheng Tao.  Understanding how feature structure transfers in transfer learning.  In _Proceedings of the International Joint Conferences on Artificial Intelligence_ , pages 2365–2371, 2017c.
  * Long et al. (2015) Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan.  Learning transferable features with deep adaptation networks.  In _Proceedings of the International Conference on Machine Learning_ , pages 97–105, 2015.
  * Long et al. (2017) Mingsheng Long, Zhangjie Cao, Jianmin Wang, and S Yu Philip.  Learning multiple tasks with multilinear relationship networks.  In _Advances in Neural Information Processing Systems_ , pages 1594–1603, 2017.
  * Luo et al. (2017a) Yong Luo, Dacheng Tao, and Yonggang Wen.  Exploiting high-order information in heterogeneous multi-task feature learning.  In _IJCAI_ , pages 2443–2449, 2017a.
  * Luo et al. (2017b) Zelun Luo, Yuliang Zou, Judy Hoffman, and Li Fei-Fei.  Label efficient learning of transferable representations across domains and tasks.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , pages 164–176, 2017b.
  * Mansour et al. (2009) Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh.  Domain adaptation: Learning bounds and algorithms.  In _Proceedings of the Conference on Learning Theory_ , 2009.
  * Mansour et al. (2021) Yishay Mansour, Mehryar Mohri, Jae Ro, Ananda Theertha Suresh, and Ke Wu.  A theory of multiple-source adaptation with limited target labeled data.  In _International Conference on Artificial Intelligence and Statistics_ , pages 2332–2340. PMLR, 2021.
  * Maurer et al. (2013) Andreas Maurer, Massimiliano Pontil, and Bernardino Romera-Paredes.  Sparse coding for multitask and transfer learning.  In _Proceedings of the International Conference on Machine Learning_ , pages 343–351, 2013.
  * Maurer et al. (2014) Andreas Maurer, Massimiliano Pontil, and Bernardino Romera-Paredes.  An inequality with applications to structured sparsity and multitask dictionary learning.  In _Proceedings of the Conference on Learning Theory_ , pages 440–460, 2014.
  * Maurer et al. (2016) Andreas Maurer, Massimiliano Pontil, and Bernardino Romera-Paredes.  The benefit of multitask representation learning.  _The Journal of Machine Learning Research_ , 17(1):2853–2884, 2016.
  * Mohri and Medina (2012) Mehryar Mohri and Andres Munoz Medina.  New analysis and algorithm for learning with drifting distributions.  In _International Conference on Algorithmic Learning Theory_ , pages 124–138. Springer, 2012.
  * Mohri et al. (2018) Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.  _Foundations of Machine Learning_.  MIT press, 2018.
  * Murugesan et al. (2016) Keerthiram Murugesan, Hanxiao Liu, Jaime Carbonell, and Yiming Yang.  Adaptive smoothed online multi-task learning.  In _Advances in Neural Information Processing Systems_ , pages 4296–4304, 2016.
  * Pan and Yang (2010) Sinno Jialin Pan and Qiang Yang.  A survey on transfer learning.  _IEEE Transactions Knowledge and Data Engineering_ , 22(10):1345–1359, 2010.
  * Pan et al. (2011) Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang.  Domain adaptation via transfer component analysis.  _IEEE Transactions on Neural Networks_ , 22(2):199–210, 2011.
  * Parameswaran and Weinberger (2010) Shibin Parameswaran and Kilian Q Weinberger.  Large margin multi-task metric learning.  In _Advances in Neural Information Processing Systems_ , pages 1867–1875, 2010.
  * Pardoe and Stone (2010) David Pardoe and Peter Stone.  Boosting for regression transfer.  In _Proceedings of the International Conference on Machine Learning_ , pages 863–870, 2010.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  In _Advances in Neural Information Processing Systems_ , pages 8024–8035, 2019.
  * Perrot and Habrard (2015) Michaël Perrot and Amaury Habrard.  A theoretical analysis of metric hypothesis transfer learning.  In _Proceedings of the International Conference on Machine Learning_ , pages 1708–1717, 2015.
  * Pong et al. (2010) Ting Kei Pong, Paul Tseng, Shuiwang Ji, and Jieping Ye.  Trace norm regularization: Reformulations, algorithms, and multi-task learning.  _SIAM Journal on Optimization_ , 20(6):3465–3489, 2010.
  * Redko et al. (2017) Ievgen Redko, Amaury Habrard, and Marc Sebban.  Theoretical analysis of domain adaptation with optimal transport.  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ , pages 737–753. Springer, 2017.
  * Saenko et al. (2010) Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell.  Adapting visual category models to new domains.  In _European conference on computer vision_ , pages 213–226. Springer, 2010.
  * Saito et al. (2019) Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko.  Semi-supervised domain adaptation via minimax entropy.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 8050–8058, 2019.
  * Sener and Koltun (2018) Ozan Sener and Vladlen Koltun.  Multi-task learning as multi-objective optimization.  In _Advances in Neural Information Processing Systems_ , pages 527–538, 2018.
  * Shen et al. (2018) Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu.  Wasserstein distance guided representation learning for domain adaptation.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018\.
  * Shui et al. (2019) Changjian Shui, Mahdieh Abbasi, Louis-Émile Robitaille, Boyu Wang, and Christian Gagné.  A principled approach for learning task similarity in multitask learning.  In _Proceedings of the 28th International Joint Conference on Artificial Intelligence_ , pages 3446–3452. AAAI Press, 2019.
  * Shui et al. (2020) Changjian Shui, Qi Chen, Jun Wen, Fan Zhou, Christian Gagné, and Boyu Wang.  Beyond $\mathcal{H}$-divergence: Domain adaptation theory with jensen-shannon divergence.  _arXiv preprint arXiv:2007.15567_ , 2020.
  * Shui et al. (2021) Changjian Shui, Zijian Li, Jiaqi Li, Christian Gagné, Charles X Ling, and Boyu Wang.  Aggregating from multiple target-shifted sources.  In _Proceedings of the International Conference on Machine Learning_ , pages 9638–9648, 2021.
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , pages 4080–4090, 2017.
  * Standley et al. (2020) Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese.  Which tasks should be learned together in multi-task learning?  In _International Conference on Machine Learning_ , pages 9120–9132. PMLR, 2020.
  * Sugiyama and Kawanabe (2012) Masashi Sugiyama and Motoaki Kawanabe.  _Machine Learning in Non-stationary Environments: Introduction to Covariate Shift Adaptation_.  MIT Press, 2012.
  * Sun et al. (2011) Qian Sun, Rita Chattopadhyay, Sethuraman Panchanathan, and Jieping Ye.  A two-stage weighting framework for multi-source domain adaptation.  In _Advances in Neural Information Processing Systems_ , pages 505–513, 2011.
  * Tiomoko et al. (2020) Malik Tiomoko, Hafiz Tiomoko Ali, and Romain Couillet.  Deciphering and optimizing multi-task learning: a random matrix approach.  In _International Conference on Learning Representations_ , 2020.
  * Tripuraneni et al. (2020) Nilesh Tripuraneni, Michael I Jordan, and Chi Jin.  On the theory of transfer learning: The importance of task diversity.  _arXiv preprint arXiv:2006.11650_ , 2020.
  * Venkateswara et al. (2017) Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan.  Deep hashing network for unsupervised domain adaptation.  In _(IEEE) Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017.
  * Wang et al. (2019) Boyu Wang, Jorge Mendez, Mingbo Cai, and Eric Eaton.  Transfer learning via minimizing the performance gap between domains.  In _Advances in Neural Information Processing Systems_ , pages 10645–10655, 2019.
  * Wang and Schneider (2015) Xuezhi Wang and Jeff Schneider.  Generalization bounds for transfer learning under model shift.  In _Proceedings of the Conference on Uncertainty in Artificial Intelligence_ , pages 922–931, 2015.
  * Weiss et al. (2016) Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang.  A survey of transfer learning.  _Journal of Big Data_ , 3(1):9, 2016.
  * Wilson and Cook (2020) Garrett Wilson and Diane J. Cook.  A survey of unsupervised deep domain adaptation.  _ACM Trans. Intell. Syst. Technol._ , 11(5):51:1–51:46, 2020.
  * Wu et al. (2020) Sen Wu, Hongyang R Zhang, and Christopher Ré.  Understanding and improving information transfer in multi-task learning.  _arXiv preprint arXiv:2005.00944_ , 2020.
  * Xie et al. (2018) Shaoan Xie, Zibin Zheng, Liang Chen, and Chuan Chen.  Learning semantic representations for unsupervised domain adaptation.  In _International conference on machine learning_ , pages 5423–5432. PMLR, 2018.
  * Yao and Doretto (2010) Yi Yao and Gianfranco Doretto.  Boosting for transfer learning with multiple sources.  In _Proceedings of the IEEE Conference Computer Vision and Pattern Recognition_ , pages 1855–1862, 2010.
  * Yu et al. (2020) Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn.  Gradient surgery for multi-task learning.  _arXiv preprint arXiv:2001.06782_ , 2020.
  * Zhang et al. (2020) Chao Zhang, Dacheng Tao, Tao Hu, and Bingchen Liu.  Generalization bounds of multitask learning from perspective of vector-valued function learning.  _IEEE Transactions on Neural Networks and Learning Systems_ , 32(5):1906–1919, 2020.
  * Zhang (2015) Yu Zhang.  Multi-task learning and algorithmic stability.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 3181–3187, 2015.
  * Zhang and Yeung (2010) Yu Zhang and Dit-Yan Yeung.  A convex formulation for learning task relationships in multi-task learning.  In _UAI_ , page 733–742, 2010.
  * Zhang and Yeung (2012) Yu Zhang and Dit-Yan Yeung.  A convex formulation for learning task relationships in multi-task learning.  _arXiv preprint arXiv:1203.3536_ , 2012.
  * Zhang et al. (2019) Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan.  Bridging theory and algorithm for domain adaptation.  In _International Conference on Machine Learning_ , pages 7404–7413. PMLR, 2019.
  * Zhao et al. (2019) Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon.  On learning invariant representations for domain adaptation.  In _International Conference on Machine Learning_ , pages 7523–7532, 2019.
  * Zhou et al. (2021a) Fan Zhou, Brahim Chaib-draa, and Boyu Wang.  Multi-task learning by leveraging the semantic information.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 11088–11096, 2021a.
  * Zhou et al. (2021b) Fan Zhou, Changjian Shui, Mahdieh Abbasi, Louis-Émile Robitaille, Boyu Wang, and Christian Gagné.  Task similarity estimation through adversarial multitask neural network.  _IEEE Transactions on Neural Networks and Learning Systems_ , 32(2):466–480, 2021b.  doi: 10.1109/TNNLS.2020.3028022.