by $N$. Then, as $q_1\circ i=\id_{\bbA^1\times Y}$ and $i^*_\sP j_!^\sP=0$,
we have $i^*_\sP N=M[2]$. But $i$ is the complement of an open affine
embedding, so $i^*_\sP$ is of perverse cohomological amplitude
$[-1,0]$ by <cit.>, hence $M=0$.

Finally, we get an exact sequence of perverse sheaves on $\bbA^1\times Y$ :

\[0\ra \pHc^0 q_{1*}^\sP q_{2\sP}^* K\ra K\ra
q_{1*}^\sP j_!^\sP j^*_\sP q_{2\sP}^*K[1]\ra \pHc^1 q_{1*}^\sP q_{2\sP}^* K\ra 0.\]

Consider the functors $\mathsf{F}_\sP,\mathsf{G}_\sP:
\sP(\bbA^1\times Y)\ra\Db(\sP(\bbA^1\times Y))$ defined by

\[\mathsf{F}_\sP(K):=K\]

\[\mathsf{G}_\sP(K):=q_{1*}^\sP j_!^\sP j^*_\sP q_{2\sP}^*K[1].\]

We have just proved that these functors are t-exact (of course, this is
obvious for the first one) and that there is a functorial exact

\[q_{1*}^\sP q_{2\sP}^*\ra\mathsf{F}_\sP\ra\mathsf{G}_\sP\xrightarrow{+1}.\]

The functors $\mathsf{F}_\sP$ and $\mathsf{G}_\sP$ are defined in terms of the four operations. The existence of these operations in the categories
$\DA_\ct(-)$ and the compatibility of the Betti realization with the four operations (see <cit.>), imply by the universal property of the categories of perverse motives that there exist :

* two exact functors

\[\mathsf{F}_\sM,\mathsf{G}_\sM:\sM(\bbA^1\times Y)\ra\sM(\bbA^1\times Y),\]

* a  natural transformation $\mathsf{F}_\sM\ra\mathsf{G}_\sM$, and

* two invertible natural transformations $\rat^\sM_{\bbA^1\times
Y}\circ \mathsf{F}_\sM\ra \mathsf{F}_\sP\circ \rat^\sM_{\bbA^1\times Y}$
and $\rat^\sM_{\bbA^1\times Y}\circ \mathsf{G}_\sM\ra \mathsf{G}_\sP\circ
\rat^\sM_{\bbA^1\times Y} $

such that the diagram

\[\xymatrix{{\rat^\sM_{\bbA^1\times Y}\circ \mathsf{F}_\sM}\ar[r]\ar[d] &
{\rat^\sM_{\bbA^1\times Y}\circ \mathsf{G}_\sM}\ar[d]\\
{\mathsf{F}_\sP\circ\rat^\sM_{\bbA^1\times Y}}\ar[r] &
{\mathsf{G}_\sP\circ\rat^\sM_{\bbA^1\times Y}}}\]

is commutative.

Given a complex $M^\bullet$ of perverse motives on $X$, let
$\mathsf{H}_\sM(M^\bullet)$ be the mapping fiber of the morphism
$\mathsf{F}_\sM(M^\bullet)\ra\mathsf{G}_\sM(M^\bullet)$ of complexes
of perverse motives on $\bbA^1\times Y$. We get a triangulated functor

\[\mathsf{H}_\sM:\Db(\sM(\bbA^1\times Y))\ra\Db(\sM(\bbA^1\times Y)),\]

the Betti realization of $\mathsf{H}_\sM$ is isomorphic to
$q_{1*}^\sP q_2^{*\sP}$.

We now define a functor

\[p^\sM_\bullet:=s^*_\sM\mathsf{H}_\sM(-):\Db(\sM(\bbA^1\times Y))
\ra\Db(\sM(Y)).\]

By construction of $p^\sM_\bullet$, we have an invertible natural
transformation $\rat^\sM_{\bbA^1\times Y} p_\bullet^\sM\ra p_*^\sP
\rat^\sM_{\bbA^1\times Y}$.

Note also the following useful fact. We denote by $f:\bbA^1\times Y\ra
\bbA^1$ the first projection and by $a:\bbG_m\times Y\ra
\bbA^1\times Y$ the inclusion. Then applying $s^*_\sM$ to the
connecting map in the third exact sequence of
subsec:AppPerverseMotive, we get a natural transformation

\[s^*_\sM\ra{}^p\mathsf{Log}^{\sM}_f a^*_\sM[1],\]

whose composition with the functor $\mathsf{H}_\sM$ is invertible. Indeed,
we can check this last statement after applying the functors
$\rat^\sM_Y$, and then this follows from the exact triangle
$q_{1*}^\sP q_2^{*\sP}\ra\mathsf{F}_\sP\ra\mathsf{G}_\sP\xrightarrow{+1}$
and the fact that the composition of the natural transformation
$s^*_\sP\ra{}^p\mathsf{Log}^{\mathsf{B}}_f a^*_\sP[1]$ and of the functor
$q_{1*}^\sP q_2^{*\sP}\simeq p^*_\sP p_*^\sP\simeq\bbQ_{\bbA^1}
\boxtimes p_*^\sP$ is invertible. As the functor
${}^p\mathsf{Log}^{\sM}_f a^*_\sM$ is exact, we get an
isomorphism from $p_\bullet^\sM$
to the mapping cone of the morphism of exact
functors ${}^p\mathsf{Log}^{\sM}_f a^*_\sM\circ\mathsf{F}_\sM\ra
{}^p\mathsf{Log}^{\sM}_f a^*_\sM\circ\mathsf{G}_\sM$.

Let us prove that the functor $p_\bullet^\sM$
is right adjoint to the functor $p^*_\sM$.
Let $\eta_\sP:\id\ra p^\sP_*p_\sP^*$ and $\delta_\sP: p_\sP^* p_*^\sP\ra
\id$
be the unit and the counit
of the adjunction between $p^*_\sP$ and $p_\sP^*$.
It suffices to lift $\eta_\sP$ and $\delta_\sP$ to natural
$\eta_\sM:\id\ra p^\sM_\bullet p_\sM^*$ and $\delta_\sM: p_\sM^* p_\bullet
^\sM\ra\id$ such that the two natural transformations

\[p^*_\sM\flnom{p^*_\sM\eta_\sM}p^*_\sM p_\bullet^\sM p^*_\sM
\flnom{\delta_\sM p^*_\sM}p^*_\sM\]

\[p_\bullet^\sM\flnom{\eta_\sM p_\bullet^\sM} p_\bullet^\sM p^*_\sM p_\bullet^\sM
\flnom{p_\bullet^\sM\delta_\sM}

are isomorphisms and the first one is the identity (see <cit.>).
Note that the fact that these natural transformations are isomorphisms
will follow automatically from the conservativity of the functors

We first construct $\eta_\sM$. Let us first show that $\mathsf{G}_\sM\circ p^*_\sM=0$.
As the functors $\rat_X^\sM$ are conservative, it suffices to prove that
$\mathsf{G}_\sP\circ p^*_\sP=0$. Let $k:U\ra\bbA^1\times\bbA^1$ be the open immersion
(remember that $U$ is the complement of the diagonal in $\bbA^1\times
\bbA^1$), so that $j=k\times\id_Y$, and let $\pi:\bbA^1\times\bbA^1\ra
\bbA^1$ be the second projection. Then

\[\mathsf{G}_\sP\circ p^*_\sP=q_{1*}^\sP j_!^\sP j^*_\sP q_{2\sP}^*p^*_\sM[1]\simeq
q_{1*}^\sP((k_!^\sP\bbQ_{U})\boxtimes -)[1]
\simeq(\pi^\sP_*k^\sP_!\bbQ_{U})\boxtimes (-)[1],\]

so it suffices to show that

\[\pi_*^\sP k_!^\sP\bbQ_{U}=0.\]

Let $\Delta:\bbA^1\ra\bbA^1\times\bbA^1$ be the diagonal embedding. Then we
have an exact triangle

\[k_!\bbQ_U\ra\bbQ_{\bbA^1\times\bbA^1}\ra
\Delta_*\bbQ_{\bbA^1}\xrightarrow{+1},\]

so, applying $\pi_*^\sP$, we get an exact triangle

\[\pi_*^\sP k_!^\sP\bbQ_{U}\ra\bbQ_{\bbA^1}\xrightarrow
\bbQ_{\bbA^1}\xrightarrow{+1},\]

and this implies the desired result.

Now that we know that $\mathsf{G}_\sP\circ p^*_\sM=0$, we get $\mathsf{H}_\sM\circ p^*_\sM=
p^*_\sM$, hence $p_\bullet^\sM p^*_\sM=s^*_\sM\circ \mathsf{H}_\sM\circ p^*_\sM=
s^*_\sM p^*_\sM$, and we take for $\eta_\sM:\id\ra p_\bullet^\sM p^*_\sM$ the
inverse of the
connection isomorphism $s^*_\sM p^*_\sM\xra{\sim}\id$.

Next we construct $\delta_\sM$. First we define a functor
$q_{1\bullet}^\sM:\Db(\sM(\bbA^1\times Y))\ra\Db(\sM(\bbA^1\times\bbA^1
\times Y))$ in the same way as $p_\bullet^\sM$. That is, we consider the
commutative diagram

\[\xymatrix{\bbA^1\times\bbA^1\times Y\ar[d]_{q_1} & \bbA^1\times\bbA^1
\times \bbA^1\times Y
\ar[l]_-{r_2}\ar[d]^(.3){r_1}& \bbA^1\times U\times Y\ar[l]^-J \\
\bbA^1\times Y & \bbA^1\times\bbA^1\times Y\ar[l]^-{q_1} & \bbA^1\times
\bbA^1\times Y\ar[l]^-{\id}\ar[lu]_-{I}\ar[llu]^

where $r_1=\id_{\bbA^1}\times q_1$, $r_2=\id_{\bbA^1}\times q_2$,
$I=\id_{\bbA^1}\times i$ and $J=\id_{\bbA^1}\times j$, and
we set $t=\id_{\bbA^1}\times s:\bbA^1\times Y\ra\bbA^1\times\bbA^1\times Y$.
Then the functors $\mathsf{F}'_\sP,\mathsf{G}'_\sP$ from $\Db(\sP(\bbA^1\times\bbA^1\times Y))$ to
itself defined by $\mathsf{F}'_\sP=\id$ and $\mathsf{G}'_\sP=r_{1*}^\sP J_!^\sP J^*_\sP
r_{2\sP}^*[1]$ are $t$-exact and
we have a natural transformation $\mathsf{F}'_\sP\ra \mathsf{G}'_\sP$. As before, we can lift
these functors and transformation to endofunctors $\mathsf{F}'_\sM\ra \mathsf{G}'_\sM$
of $\Db(\sM(\bbA^1\times\bbA^1\times Y))$. We denote by $\mathsf{H}'_\sM$ the
mapping fiber of $\mathsf{F}'_\sM\ra \mathsf{G}'_\sM$, and we set $q_{1\bullet}^\sM=t^*_\sM\circ
Also, if we denote by $f':\bbA^1\times\bbA^1\times Y\ra\bbA^1$ the second
projection and by $a'$ the injection of
$\bbA^1\times\bbG_m\times Y$ into $\bbA^1\times\bbA^1\times Y$, we get as
above an invertible natural transformation from $q_{1\bullet}^\sM$
to the mapping cone of the morphism of exact functors
${}^p\mathsf{Log}^{\sM}_{f'} (a')^*_\sM\circ\mathsf{F}'_\sM\ra
{}^p\mathsf{Log}^{\sM}_{f'} (a')^*_\sM\circ\mathsf{G}'_\sM$.

Let's show that the base change isomorphism $p^*_\sP p_*^\sP\xra{\sim} q_{1*}
^\sP q_{2\sP}^*$ lifts to a morphism
$p^*_\sM p_\bullet^\sM\ra q_{1\bullet}
^\sM q_{2\sM}^*$ (which will automatically be an isomorphism).
We have invertible natural transformations
$\mathsf{F}'_\sP\circ q_{2\sP}^*\simeq q_{2\sP}^*\circ \mathsf{F}_\sP$ and
$\mathsf{G}'_\sP\circ q_{2\sP}^*\simeq q_{2\sP}^*\circ \mathsf{G}_\sP$. As all the functors
involved are t-exact up to the same shift, the transformations lift to
natural transformations $\mathsf{F}'_\sM\circ q_{2\sM}^*\simeq q_{2\sM}^*\circ\mathsf{F_\sM}$ and
$\mathsf{G}'_\sM\circ q_{2\sM}^*\simeq q_{2\sM}^*\circ\mathsf{G}_\sM$, and induce an
invertible natural transformation $\mathsf{H}'_\sM\circ q_{2\sM}^*\simeq q_{2\sM}^*\circ
\mathsf{H}_\sM$. Composing on the left with $t_\sM^*$ and using the connection
isomorphism $t_\sM^* q_{2\sM}^*\simeq p_\sM^* s_\sM^*$, we get
the desired isomorphism $p^*_\sM p_\bullet^\sM\xra{\sim} q_{1\bullet}
^\sM q_{2\sM}^*$.

Composing this isomorphism with the unit of the adjunction
$(i^*_\sM,i_*^\sM)$ and using the connection isomorphism $i^*_\sM q_{2\sM}^*
\simeq\id$ gives a natural transformation $p^*_\sM p_\bullet^\sM\ra
q_{1\bullet}^\sM i_*^\sM$. It remains to show that the isomorphism
$q_{1*}^\sP i_*^\sP\simeq\id$ lifts to a natural transformation
$q_{1\bullet}^\sM i_*^\sM\ra\id$.
First we note that the functors

\begin{equation}\label{eq:funcH''}
{}^p\mathsf{Log}^{\mathsf{B}}_{f'}(a')^*_\sP r_{1*}^\sP r_2^{*\sP}i_*^\sP[1]
\end{equation}

\[{}^p\mathsf{Log}^{\mathsf{B}}_{f'}(a')^*_\sP r_{1*}^\sP J_!^\sP J^*_\sP

are $t$-exact and the counit of the adjunction $(J_!^\sP,J^*_\sP)$ induces a natural transformation from the second one to the first one. Hence, the functor (<ref>)
induces an exact
endofunctor $\mathsf{H}''_\sM$of $\sM(\bbA^1\times Y)$,
together with a natural transformation
${}^p\mathsf{Log}^\sM (a')^*_\sM
\circ \mathsf{G}'_\sM\circ i_*^\sM\ra \mathsf{H}''_\sM$.
But we also have an invertible natural transformation
of $t$-exact functors

\begin{align*}
\id_{\Db(\sP(\bbA^1\times Y))} & \simeq t^*_\sP q_1^{*\sP}q_{1*}^\sP i_*^\sP
\mbox{ (connection isomorphisms)} \\
&\xra{\sim} t^*_\sP r_{1*}^\sP r_{2\sP}^* i_*^\sM\mbox{ (base change)}\\
& \xra{\sim} {}^p\mathsf{Log}^{\mathsf{B}}_{f'}(a')^*_\sP r_{1*}^\sP r_{2\sP}^*
i_*^\sM[1]\mbox{ (by \subsectionref{subsec:AppPerverseMotive} as above)}
\end{align*}

and all the maps in it are defined in the categories
$\DA_\ct(-)$, so it induces an invertible natural transformation
$\id_{\sM(\bbA^1\times Y)}\xra{\sim}\mathsf{H}''_\sM$. Composing it with
${}^p\mathsf{Log}^\sM (a')^*_\sM
\circ\mathsf{G}'_\sM\circ i_*^\sM\ra\mathsf{H}''_\sM$
and using the isomorphism from $q_{1\bullet}^\sM$ to the mapping cone of
${}^p\mathsf{Log}^\sM (a')^*_\sM\circ\mathsf{F}'_\sM\ra
{}^p\mathsf{Log}^\sM (a')^*_\sM\circ\mathsf{G}'_\sM$,
we finally get the desired natural
transformation $\delta_\sM$ :

\[p^*_\sM p_\bullet^\sM\xra{\sim}q_{1\bullet}^\sM q_2^{*\sM}
\ra q_{1\bullet}^\sM i_*^\sM\ra {}^p\mathsf{Log}^\sM (a')^*_\sM
\circ\mathsf{G}'_\sM\circ i_*^\sM\ra\mathsf{H}''_\sM\simeq\id_{D^b(\sM(\bbA^1\times Y))}.\]

Finally, we check that the natural transformation

\[p^*_\sM\flnom{p^*_\sM\eta_\sM}p^*_\sM p_\bullet^\sM p^*_\sM
\flnom{\delta_\sM p^*_\sM}p^*_\sM\]

is the identity.
The two functors $p^*_\sM p_\bullet^\sM p^*_\sM[1]$ and $p^*_\sM[1]$ are
exact and equal to the derived functor of their $\mathbf{H}^0$, and the
natural transformations $(p^*_\sM\eta_\sM)^{-1},\delta_\sM p^*_\sM:
p^*_\sM p_\bullet^\sM p^*_\sM[1]\ra p^*_\sM[1]$ are also defined by
extending their action on the $\mathbf{H}^0$'s, so it suffices to
check that they are equal on these $\mathbf{H}^0$'s. But this follows
from the analogous result for the category of perverse sheaves.

Step 3 : By steps 1 and 2, the proposition
is true if $f$ is an affine
morphism. Indeed, if $f$ is affine then we can write $f=p\circ i$, where $i$ is
a closed immersion and $p$ is the composition of a finite number of maps
of the form $\bbA^1_Z\ra Z$.

Step 4 : So far, we have constructed the $2$-functor $f\mapsto
on the category of quasi-projective
$k$-schemes and affine morphisms, and we have shown that is
right adjoint to the $2$-functor $f\mapsto f^*_\sM$. As in <cit.>, we can extend this $2$-functor to the category
of quasi-projective
$k$-schemes and all morphisms using Beĭlinson's construction (see
<cit.> and

It remains to show that this extended $2$-functor is still right
adjoint to the $2$-functor $f\mapsto f^*_\sM$. Every morphism
$f:X\ra Y$ between quasi-projective $k$-schemes can be written as $f=pji$,
where $i$ is a closed immersion, $j$ is an open immersion, and $p$ is a
second projection
$Z\times Y\ra Y$ with $Z$ smooth equidimensional. We already know the
result for closed immersions, so we just need to prove it for
$j$ and $p$. But the proof in these two cases is exactly the same
as in <cit.>.

For every smooth morphism $f:X\ra Y$ in $(\Sch/k)$ the functor

\[f^*_\sM:\Db(\sM(Y))\ra\Db(\sM(X))\]

admits a left adjoint $f_\sharp$. Moreover

* the natural transformation

\[f^\sP_\sharp\rat^\sM_X \ra \rat^\sM_Yf^\sM_\sharp\]

obtained from $\theta^\sM_f$ by adjunction, is invertible;

* for every cartesian square

\[\xymatrix{{X'}\ar[r]^-{g'}\ar[d]^-{f'} & {X}\ar[d]^-{f}\\
{Y'}\ar[r]^-{g} & {Y}}\]

with $f$ smooth, the exchange $2$-morphism $f'^\sM_\sharp g'^*_\sM\ra g^*_\sM f^\sM_\sharp$ is invertible.

The assertion (2) is an immediate consequence of (1) since the functor $\rat^\sM_Y$ is conservative.

We deduce the proposition from prop:rightadjoint using Verdier duality. Let $f:X\ra Y$ be a smooth morphisms of relative dimension $d$.  Note that $f^*_\sP$ has a left adjoint  given by $f_\sharp^\sP:=\bbD^\sP_Y f^\sP_*(-d)[-2d]\bbD^\sP_X$.
Therefore we similarly set $f_\sharp^\sM:=\bbD^\sM_Y f^\sM_* (-d)[-2d]\bbD^\sM_X$.

Let $A$ be an object in $\Db(\sM(X))$ and $B$ be an object in $\Db(\sM(Y))$. Then, prop:rightadjoint and prop:duality provide isomorphisms

\begin{align*}
\Hom(f_\sharp^\sM A,B)&\simeq\Hom(\bbD^\sM_Y B,f^\sM_*(-d)[-2d]\bbD^\sM_X A)\simeq\Hom(f^*_\sM(d)[2d]\bbD^\sM_YB,\bbD_X^\sM A)\\
& \simeq\Hom(A,\bbD_X^\sM f^*_\sM(d)[2d] \bbD^\sM_YB)\simeq\Hom(A,f^*_\sM B).
\end{align*}

This shows that $(f_\sharp^\sM,f_\sM^*)$ form a pair of adjoint functors.
Note that the counit ${}^\sM\delta^*_\sharp$ of the adjunction is given by the composition

\begin{equation}\label{eq:counitstarsharp}
f^\sM_\sharp f^*_\sM\xra{\varepsilon_f^\sM} \bbD_Y^\sM f^\sM_*f^*_\sM\bbD_Y^\sM\xra{{}^\sM\eta^*_*}(\bbD^\sM_Y)^2\xra{(\varepsilon_Y^\sM)^{-1}} \Id
\end{equation}

and the unit ${}^\sM\eta^*_\sharp$ by the composition

\begin{equation}\label{eq:unitstarsharp}
\Id\xra{\varepsilon^\sM_X}(\bbD^\sM_X)^2\xra{{}^\sM\delta^*_*} \bbD^\sM_X f^*_\sM f^\sM_*\bbD^\sM_X \xra{\varepsilon_f^\sM}f^*_\sM f_\sharp^\sM.
\end{equation}

To show that the morphism

\[f^\sP_\sharp\rat^\sM_X \xra{{}^\sM\eta^*_\sharp}f^\sP_\sharp\rat^\sM_Xf^*_\sM f^\sM_\sharp\xra{(\theta^\sM_f)^{-1}}f^\sP_\sharp f^*_\sP\rat^\sM_Y f^\sM_\sharp\xra{{}^\sP\delta^*_\sharp} \rat^\sM_Yf^\sM_\sharp\]

is invertible, it is enough to check that it is equal to the morphism

\[\xymatrix{{f^\sP_\sharp\rat^\sM_X}\ar[r]^-{(\nu^\sM_X)^{-1}} & {\bbD^\sP_Yf^\sP_*(-d)[-2d]\rat^\sM_X \bbD^\sM_X}\ar[d]^-{\gamma_f^\sM} & {}\\
{} &{\bbD^\sP_Y\rat^\sM_Y f^\sM_*(-d)[-2d] \bbD^\sM_X}\ar[r]^-{\nu_Y^\sM} &{\rat^\sM_Yf^\sM_\sharp}}\]

where $\gamma_f^\sM$ is the invertible natural transformation of prop:rightadjoint. Using the expressions of ${}^\sM\delta^*_\sharp $ and ${}^\sM\eta^*_\sharp $ given in (<ref>) and (<ref>), this follows directly from prop:duality (2) and prop:duality (1), which ensure that the diagram

\[\xymatrix{{\bbD_X^\sP f^*_\sP[d]\bbD_Y^\sP\rat^\sM_Y}\ar[rr]^-{\varepsilon_f^\sP}\ar[rd]^-{\varepsilon_f^\sM} & {} &{(\bbD^\sP_X)^2 f_\sP^*(d)[d]\rat^\sM_Y}\ar[d]^-{(\varepsilon^\sP_X)^{-1}}\\
{\bbD^\sP_X f_\sP^*[d]\rat^\sM_Y\bbD^\sM_Y}\ar[d]^-{\varepsilon_f^\sP}\ar[u]_-{\nu_Y^\sM} & {f^*_\sP(d)[d](\bbD^\sP_Y)^2\rat^\sM_Y}\ar[ld]_-{\nu^\sM_Y}\ar[r]^-{(\varepsilon^\sP_Y)^{-1}} &{f_\sP^*(d)[d]\rat^\sM_Y}\\
{f_\sP^*(d)[d]\bbD^\sP_Y\rat^\sM_Y\bbD^\sM_Y}\ar[rr]^-{\nu^\sM_Y} &{} & {f_\sP^*(d)[d]\rat^\sM_Y(\bbD^\sM_Y)^2}\ar[u]_-{(\varepsilon^\sM_Y)^{-1}}}\]

is commutative.

The pair $(j^*_\sM, i^*_\sM)$ is conservative, since so is the pair $(j^*_\sP, i^*_\sP)$. This follows from the existence of the isomorphisms $\theta^\sM_j$, $\theta^\sM_i$ and the fact that $\rat^\sM_X$ is a conservative functor.

To finish the proof of theo:maintheo, it remains to check that, if $s$ the zero section of  $p:\bbA^1_X\ra X$, then $p^\sM_\sharp s^\sM_*$ is an equivalence of categories. By construction

\[p^\sM_\sharp s^\sM_*=\bbD_{X}^\sM p_*^\sM(-1)[-2]\bbD^\sM_{\bbA^1_X} s^\sM_*.\]

Note that the isomorphism $\bbD^\sP_{\bbA^1_X}s^\sP_*\simeq s^\sP_*\bbD^\sP_X$ exists in the category of (constructible) étale motives, hence
the compatibility of the Betti realization with the four operations (see <cit.>), implies by the universal property of the categories of perverse motives that it lifts to an isomorphism $\bbD^\sM_{\bbA^1_X}s^\sM_*\simeq s^\sM_*\bbD^\sM_X$. Therefore, we have an isomorphism

\[p^\sM_\sharp s^\sM_*\simeq\bbD_{X}^\sM p_*^\sM s^\sM_*(-1)[-2]\bbD^\sM_{\bbA^1_X}\simeq \Id(-1)[-2].\]

This shows that $p^\sM_\sharp s^\sM_*$ is an equivalence of categories and concludes the proof of theo:maintheo.

Let $f:X\ra Y$ be a morphism of $k$-schemes. Then, the natural transformations

\[\xi^\sM_f:\rat^\sM_X f^!_\sM\ra f^!_\sP\rat^\sM_Y\qquad \rho^\sM_f:f^\sP_!\rat^\sM_X\ra \rat^\sM_Y f^\sM_!\]

are invertible.

By <cit.>, it just remains to check that $\xi^\sM_i$ is invertible if $i:Z\hookrightarrow X$ is a closed immersion. Let $j:U\hookrightarrow X$ be the open immersion of the complement of $Z$ in $X$. Then, we have a commutative diagram

\[
\xymatrix{{i^\sP_*i^!_\sP\rat^\sM_X}\ar[r] &{\rat^\sM_X}\ar[r]\ar@{=}[d] &{j^\sP_*j^*_\sP\rat^\sM_X}\ar[r]^-{+1} &{}\\
{i^\sP_*\rat^\sM_Z i^!_\sM}\ar[r]\ar[u]_-{\xi^\sM_i} &{\rat^\sM_X}\ar[r]\ar@{=}[d] &{j^\sP_*\rat^\sM_U j^*_\sM}\ar[r]^-{+1}\ar[u]_-{(\theta_j^\sM)^{-1}} &{}\\
{\rat^\sM_Xi^\sM_*i^!_\sM}\ar[r]\ar[u]_-{\gamma^\sM_i} &{\rat^\sM_X}\ar[r] &{\rat^\sM_X j^\sM_*j^*_\sM}\ar[r]^-{+1}\ar[u]_-{\gamma_j^\sM} &{}}
\]

which implies that the image of $\xi^\sM_i$ under $i^\sM_*$ is invertible since all the other morphisms are. Therefore $\xi^\sM_i$ is also invertible.

§.§ Some consequences

In this subsection, we draw some immediate consequences of the main theorem (theo:maintheo). A $\bbQ$-local system $\scr L$ on a $k$-variety $X$ will be called geometric if there exists a smooth proper morphism $g:Z\ra X$ such that $\scr L=R^ig_*\bbQ$ for some integer $i\in\bbZ$.

§.§.§ Intersection cohomology

The four operations formalism allows the definition of a motivic avatar of intersection complexes. In particular, intersection cohomology groups with coefficients in geometric systems are motivic. More precisely:

Let $X$ be an irreducible quasi-projective $k$-variety and $\scr L$ be a local system on a smooth dense open subscheme of $X$. If $\scr L$ is geometric, then the intersection cohomology group $IH^i(X,\scr L)$, for $i\in\bbZ$, is canonically the Betti realization of a Nori motive over $k$.

Let us denote by $\mathrm{H}^i$ the standard cohomology functors on $\Db(\sM(X))$.
If $\scr L$ is geometric, then $\scr L[d]=R^jg_*\bbQ_Z[d]=\pH^{j+d}g_*\bbQ_Z$ for some smooth proper morphism $g:Z\ra U$ and some integer $j\in\bbZ$.  Hence $\scr L[d]$ is the image under $\rat^\sM_U$ of the perverse motivic sheaf $A:=\mathrm{H}^{j+d}(g_*^\sM\bbQ^\sM_Z)$. Then the intersection complex

\[\mathrm{IC}_X(\scr L):=\Img(\pH^0j^\sP_!\scr L[d]\ra\pH^0j_*^\sP\scr L[d])\]

is canonically isomorphic to the image under $\rat^\sM_X$ of the perverse motive $B:=\Img(\mathrm{H}^0(j^\sM_! A)\ra \mathrm{H}^0(j^\sM_*A))$. (Note that $B:=j^\sM_{!*}A$ with the notation of defi:IC).) This implies that
$IH^i(X,\scr L):={\bf H}^{i-d}(X,\mathrm{IC}_X(\scr L))$ is the Betti realization of the Nori motive $\mathrm{H}^{i-d}(\pi^\sM_*B)$ where $\pi:X\ra\Spec k$ is the structural morphism.

This shows, in particular, that intersection cohomology groups carry a natural Hodge structure. If $X$ is a smooth projective curve, and $\scr L$ underlies a polarizable variation of Hodge structure, then the Hodge structure on the intersection cohomology groups was constructed by Zucker in <cit.>. In general, it follows from Saito's work on mixed Hodge modules [59] and a different proof has been given in [25]. We consider the weights in the next section (see theo:weightIC and coro:weightIC).

§.§.§ Leray spectral sequences

Let $f:X\ra Y$ be a morphism of quasi-projective $k$-varieties and $\scr L$ be a $\bbQ$-local system on $X$. Then, we can associate with it two Leray spectral sequences in Betti cohomology: the classical one

\[{\bf{H}}^r(Y, R^sf_*\scr L)\Longrightarrow H^{r+s}(X,\scr L)\]

and the perverse one

\[{\bf{H}}^r(Y, \pH^sf_*\scr L)\Longrightarrow H^{r+s}(X,\scr L).\]

The main theorem of [4] shows that, if $\scr L=\bbQ_X$ is the constant local system on $X$ and the morphism $f$ is projective, then  the classical Leray sequence is motivic, that is, it is the realization of a spectral sequence in the abelian category of Nori motives over $k$ (see precisely <cit.>).

This property is still true without the projectivity assumption and also more generally if the local system $\scr L$ is geometric:

If the local system $\scr L$ is geometric, then the classical Leray spectral sequence and the perverse Leray spectral sequence are spectral sequences of Nori motives over $k$.

If the local system $\scr L$ is geometric, there exists a smooth proper morphism $g:Z\ra X$ such that $\scr L=R^ig_*\bbQ_Z$ for some integer $i\in\bbZ$. Then $\scr L$ is the image under the functor $\rat^\sM_X$ of the perverse motive ${}^\mathrm{c}\mathrm{H}^i(g^\sM_*\bbQ^\sM_Z)$, where ${}^{\mathrm{c}}\mathrm{H}^i$ is the cohomological functor associated with the constructible $t$-structure (see below). So the result follows from the functoriality of the direct image functors.

In particular, the Leray spectral sequences are  spectral sequences of (polarizable) mixed Hodge structures. The compatibility of the classical Leray spectral sequence result in Hodge theory was already proved by Zucker in [68] when $X$ is a curve and more generally, for both spectral sequences, by  Saito if $\scr L$ underlies an admissible variation of mixed Hodge structures (see [59]). This result has been recovered by de Cataldo and Migliorini with different technics in [26].

§.§.§ Nearby cycles
The theory developed here also shows that nearby cycles functors applied to perverse motives produce Nori motives.

Let $X$ be a $k$-variety, $f:X\ra \bbA^1_k$ a flat morphism with smooth generic fiber $X_\eta$ and $\scr L$ be a $\bbQ$-local system on $X_\eta$. If
$\scr L$ is geometric, then, for every point $x\in X_\sigma(k)$ and every integer $i\in\bbZ$, the Betti cohomology $H^i(\Psi_f(\scr L)_x)$ of the nearby fiber is canonically a Nori motive over $k$.

The nearby cycles functor $\psi_f:=\Psi_f[-1]$ is $t$-exact for the perverse $t$-structure. Since it exists in the triangulated category of constructible étale motives (see [7]) and the Betti realization is compatible with the nearby cycles functor by <cit.>, the universal property ensures the existence of  an exact functor $\psi_f^\sM:\sM(X_\eta)\ra\sM(X_\sigma)$ and an invertible natural transformation $\rat^\sM_{X_\sigma}\psi_f^\sM\simeq \psi_f\rat^\sM_{X_\eta}$.

Let $d$ be the dimension of the generic fiber. If $\scr L$ is geometric, then $\scr L[d]=R^jg_*\bbQ_Z[d]=\pH^{j+d}g_*\bbQ_Z$ for some smooth proper morphism $g:Z\ra X_\eta$ and some integer $j\in\bbZ$.  Hence $\scr L[d]$ is the image under $\rat^\sM_{X_\eta}$ of the perverse motivic sheaf $A:=\mathrm{H}^{j+d}(g_*^\sM\bbQ^\sM_Z)$. So we conclude that $H^i(\Psi_f(\scr L)_x)$ is the Betti realization of the Nori motive $\mathrm{H}^{i+1-d}(x^*\psi_f^\sM A)$.

§.§.§ Exponential motives

The perverse motives introduced in the present paper and their stability under the four operations can be used also in the study of exponential motives as introduced in [31]. Indeed, recall that Kontsevich and Soibelman define an exponential mixed Hodge structure as a mixed Hodge module $A$ on the complex affine line $\bbA^1_\bbC$ such that $p_*A=0$, where $p:\bbA^1_\bbC\ra\Spec(\bbC)$ is the projection (see [49]). Their definition can be mimicked in the motivic context and the abelian category of exponential Nori motives can be defined as the full subcategory of $\sM(\bbA^1_k)$ formed by the objects which have no global cohomology.

§.§.§ Constructible $t$-structure

Let us conclude by a possible comparison with Arapura's construction from [5]. Let $X$ be a $k$-variety and consider the following full subcategories of $\Db(\sM(X))$

\begin{align*}
{}^c\mathrm{D}^{\leqslant 0}&:=\{A\in\Db(\sM(X)):H^k(x^*_\sM A)=0,\; \forall\; x\in X, \forall\; k>0\},\\
{}^c\mathrm{D}^{\geqslant 0}&:=\{A\in\Db(\sM(X)):H^k(x^*_\sM A)=0,\; \forall\; x\in X, \forall\; k<0\}.
\end{align*}

As in <cit.> (see also <cit.>), we can check that these categories define a $t$-structure on $\Db(\sM(X))$.

Let $\sMct(X)$ be the heart of this $t$-structure. Then, the functor $\rat^\sM_X$ induces a faithful exact functor from $\sMct(X)$ into the abelian category of constructible sheaves of $\bbQ$-vector spaces on $X$. Then, using the universal property of the category of constructible motives $\mathcal{M}(X,\bbQ)$ constructed by Arapura in [5], we get a faithful exact functor $\mathcal{M}(X,\bbQ)\ra\sMct(X)$. Is this functor an equivalence? If $X=\Spec k$, then
both categories are equivalent to the abelian category of Nori motives, so this functor is an equivalence.

§ WEIGHTS

In this section, we will use results on motives and weight structures from [21, 38]. To apply these references directly in our context, we will make use of the fact that, if
$S$ is a Noetherian scheme of finite dimension, then Ayoub's
category $\DA_\ct(S)$ is canonically isomorphic to the
category of Beĭlinson motives defined in Cisinski and
Déglise's book [23]. This is <cit.> and will
henceforth be used without further comment. (Note also that, though the authors of [21, 38] have chosen to use Beĭlinson's motives, étale motives could have been used.)

§.§ Continuity of the abelian hull

Remember that, in chapter 5 of Neeman's book [54], there are
four constructions of the abelian hull of a triangulated category.
The first one gives a lax 2-functor from the 2-category of triangulated
categories to that of abelian categories, but the other three constructions
give strict 2-functors. If we use the fourth
construction, which Neeman calls $D(\matheusm S)$ (see <cit.>), then the following proposition is immediate.

Let $\matheusm S$ be a triangulated category, and
suppose that we have an equivalence of triangulated categories
$\matheusm S\iso\varinjlim_{i\in I}
\Scat_i$, where $I$ is a small filtered category.

Then the canonical functor $\Ab^\tr(\Scat)\ra\varinjlim_{i\in I}\Ab^\tr
(\Scat_i)$ is an equivalence of abelian categories.

§.§ Étale realization and $\ell$-adic
perverse Nori motives

Let $S$ be a Noetherian excellent scheme finite-dimensional scheme,
let $\ell$ be a prime
number invertible over $S$; we assume that $\ell$ is odd and that
$S$ is a $\bbQ$-scheme.
(By Exposé XVIII-A of [42], the hypotheses above imply
Hypothesis 5.1 in Ayoub's paper [9].)
Under this hypothesis,
Ayoub has constructed an étale $\ell$-adic realization functor
on $\DA_\ct(S)$, compatible with pullbacks.

(See [9] sections 5 and 10.)
Denote by $\Dbc(S,\bbQ_\ell)$ the category of constructible
$\ell$-adic complexes on $S$.
Then we have a triangulated functor $\Ret_S:\DA_\ct(S)\ra
\Dbc(S,\bbQ_\ell)$ for every $S$
and, for every morphism $f:S\ra S'$, with $S'$ satisfying the same
hypotheses as $S$,
we have an
invertible natural transformation

\[\theta_f:f^*\circ\Ret_Y\ra \Ret_X\circ f^*.\]

Using results of Gabber (see [33], and also sections 4 and 5
of Fargues's article [30]), we can construct an abelian category
$\sP(S,\bbQ_\ell)$ of $\ell$-adic
perverse sheaves on $S$, satisfying
all the usual properties. In particular, we get a perverse cohomology
functor $\pH^0_\ell:=\pH^0\circ\Ret_S:\DA_\ct(S)\ra\sP(S,\bbQ_\ell)$.

Let $S$ be as above.
The Abelian category of $\ell$-adic perverse motives on $S$
is the Abelian category

\[\sM(S)_\ell:=\Ab^\tr(\DA_\ct(S),\pH^0_\ell).\]

By construction, the functor $\pH^0_\ell$ has a factorization

\[\DA_\ct(S)\xra{\pH^0_\sM}\sM(S)_\ell\xra{\rat^\sM_{S,\ell}}\sP(S,\bbQ_\ell)\]

where $\rat^\sM_{S,\ell}$ is a faithful exact functor
and $\pH^0_\sM$ is a homological functor.

By the universal property of $\sM(S)_\ell$, we also get pullback
functors between these categories as soon as the pullback functor
between the categories of $\ell$-adic complexes preserves the category
of perverse sheaves.

We will use the following important fact :
If we fix a base field $k$ of characteristic $0$
and only consider schemes
that are quasi-projective over $k$, then the main theorem (stated in
subsec:statement_main_theorem) stays true for the
categories $\sM(S)_\ell$, for any odd prime number $\ell$.
Of course,
we have to replace $\Db_\ct(S)$ and the Betti realization functor by
$\Dbc(S,\bbQ_\ell)$ and the étale realization functor
in all the statements. Indeed, the proof of the main theorem, of of the statements that
it uses, still work if we use the $\ell$-adic étale realization instead of the Betti realization.
The only result that requires a slightly different proof is lemm:compNCBetti : we have to
show that the $\ell$-adic realization of Ayoub's logarithmic motive $\Log^\vee_n$ is the local system
used in Beilison's construction of the unipotent nearby cycle functor (see 1.1 and 1.2 of
[20] or Definition 5.2.1 of [53]). As in the proof of
lemm:compNCBetti, it suffices to check this for $n=1$, and then it follows from
Lemma 11.22 of [9].

§.§ Mixed horizontal perverse sheaves

Let $k$ be a field and $S$ be a $k$-scheme of finite type. We also fix
a prime number $\ell$ invertible over $S$.
If $k$ is
finitely generated over its prime field, then the category
$\Db_m(S,\bbQ_\ell)$ of mixed horizontal
and its perverse t-structure
with heart $\sP_m(S,\bbQ_\ell)$ (the category of mixed horizontal
$\ell$-adic perverse sheaves on $S$)
were constructed
in Huber's article [39] (see also  <cit.>).
We can actually use exactly the same process to define
$\Db_m(S,\bbQ_\ell)$ and $\sP_m(S,\bbQ_\ell)$ for arbitrary $k$. The only
result in <cit.> that used the fact that $k$ is
finitely generated is the fact that
the obvious exact functor
from $\sP_m(S,\bbQ_\ell)$ to the usual category $\sP(S,\bbQ_\ell)$
of $\ell$-adic perverse
sheaves on $S$ is fully faithful. (This is not true for a general
field $k$, for example it is obviously false if $S=\Spec k$ and $k$ is
algebraically closed.)
All the other results of [39] about the 6 operations and
their interaction with weights stay true for the same reasons as in that

We could also have defined $\Db_m(S,\bbQ_\ell)$ and
$\sP_m(S,\bbQ_\ell)$ as direct 2-limits of the similar categories
over models of $S$ over finitely generated subfields of $k$. This
gives the same result because, for any field extension
$k\subset k'$, the pullbacks
functor $\Db_m(S,\bbQ_\ell)\ra\Db_m(S_{k'},\bbQ_\ell)$ is exact and

Now we want to show that the realization functor $\rat^\sM_{S,\ell}:\sM(S)_\ell\ra\sP(S,\bbQ_\ell)$ factors
through the subcategory $\sP_m(S,\bbQ_\ell)$.

We have a continuity theorem for the categories of étale motives, proved
in <cit.> and <cit.> (see also <cit.>).

Let $S$ be a Noetherian scheme of finite dimension.
Suppose that we have $S=\varprojlim_{i\in I} S_i$, where all the $S_i$
are finite-dimensional Noetherian schemes and all the transition maps
$S_i\ra S_j$ are affine. Then the canonical functor
$2-\varinjlim_{i\in I}\DA_\ct(S_i)\ra\DA_\ct(S)$ is an equivalence of
monoidal triangulated categories.

Using the definition of mixed horizontal $\ell$-adic complexes, we immediately
get the following corollary.

Let $S$ and $\ell$ be as in the beginning of this subsection, with $\ell$ odd.
Then the étale realization functor $\DA_\ct(S)\ra\Dbc(S,\bbQ_\ell)$ factors
through a functor $\DA_\ct(S)\ra\Db_m(S,\bbQ_\ell)$.
In particular, the perverse cohomology functor
$\pH^0_\ell:\DA_\ct(S)\ra\sP(S,\bbQ_\ell)$ factors through the subcategory $\sP_m(S,\bbQ_\ell)$.

The essential image of the
realization functor $\rat^\sM_{S,\ell}:\sM(S)_\ell\ra\sP(S,\bbQ_\ell)$ is contained in the
subcategory  $\sP_m(S,\bbQ_\ell)$.

We will also denote the resulting faithful exact functor $\sM(S)\ra\sP_m(S,\bbQ_\ell)$ by

§.§ Continuity for perverse Nori motives

Like the triangulated category of motives, the category of
perverse Nori motives satisfies a continuity property.

Let $S$ be a scheme and $\ell$ be a prime number
satisfying the conditions of
subsec:l-adic_perverse_motives. We assume that
$S=\varprojlim_{i\in I}S_i$, where
$(S_i)_{i\in I}$ is a directed projective system of schemes
satisfying the same conditions as $S$, and in which the transition maps
are affine. We also assume that the pullback by any transition map
$S_i\ra S_j$ preserves the category of perverse sheaves,
and that there exists $a\in\bbZ$ such that, if
$f_i:S\ra S_i$ is the canonical map, then $f^*_i[a]$
preserves the category of perverse sheaves for every $i\in I$.
Under these hypotheses, the functors $f_i^*[a]$ induce a functor

\[2-\varinjlim_{i\in I}\sM(S_i)_\ell\ra\sM(S)_\ell,\]

and this functor is full and essentially surjective.

If moreover
the canonical exact functor $2-\varinjlim_{i\in I}
\sP(S_i,\bbQ_\ell)\ra\sP(S,\bbQ_\ell)$
induced by the $f_i^*[a]$ is faithful,
then the canonical functor

\[2-\varinjlim_{i\in I}\sM(S_i)_\ell\ra\sM(S)_\ell\]

is an equivalence of abelian categories.

This follows from prop_continuity_abelian_hull and theo:cont.

Let $S$ and $\ell$
be as above, and suppose also that $S$ is integral.
Then, if
$\eta$ is the generic point of $S$, the canonical
exact functor

\[2-\varinjlim_U \sM(U)_\ell\ra\sM(\eta)_\ell,\]

where the limit is taken over all nonempty affine open subschemes of $S$,
is full and essentially surjective.

Note that the functor of the corollary sends a perverse motive
$K$ in $\sM(U)_\ell$ to $K_{\eta}[-\dim S]$. (The shift is necessary
to preserve perversity.)

§.§ Comparison of the different categories
of perverse Nori motives

In the next proposition, we compare the $\ell$-adic definition of perverse motives with the one used previously and obtained via the Betti realization.

Suppose that $k$ is a field
of characteristic $0$ and that $S$ is quasi-projective over $k$.

We write
$\rho_\ell$ for the canonical exact functor
$\Ab^\tr(\DA_\ct(S))\ra\sP(S,\bbQ_\ell)$ induced by $\pH^0_\ell$.
$\sigma$ is an embedding of $k$ into $\bbC$, then
we also have an exact functor
$\rho_\sigma:\Ab^\tr(\DA_\ct(S))\ra\sP(S)$ induced by $\pH^0$.
Then :

(i) If $\ell,\ell'$ are two odd prime numbers,
then $\Ker\rho_\ell=\Ker\rho_{\ell'}$.
In particular, we get
a canonical equivalence of abelian categories $\sM(S)_\ell=\sM(S)_{\ell'}$.

(ii) If $\sigma:k\ra\bbC$ is
an embedding, then $\Ker\rho_\ell=\Ker\rho_\sigma$.
In particular, we get
a canonical equivalence of abelian categories $\sM(S)_\ell=\sM(S)$.

We first treat the case $S=\Spec k$.
If $k$ can be embedded in $\bbC$, then (ii) follows from Huber's
construction of mixed realizations in [40], and (i) follows
from (ii). In the general case,
(i) follows from the case where
$k$ can be embedded in $\bbC$ and from prop_continuity_sM,
applied to the family of
subfields of $k$ that can be embedded in $\bbC$.

Now we treat the case of a general $k$-scheme $S$.
As in the first case, (i) follows from (ii)
and from prop_continuity_sM. So suppose that we have
an embedding $\sigma:k\ra\bbC$.
We prove the desired result by induction on the dimension of $S$.
The case $\dim S=0$ has already been treated, so we may assume that $\dim S>0$
and that the result is known for all the schemes of lower dimension.
We denote by $M\mapsto[M]$ the canonical functor $\DA_\ct(S)\ra\Ab^\tr(\DA_\ct(S))$;
as $\DA_\ct(S)$ is a triangulated category, this is a fully faithful
functor. Let $X$ be an object of $\Ab^\tr(\DA_\ct(S))$.
By construction of $\Ab^\tr(\DA_\ct(S))$,
exists a morphism $N\ra M$ in $\DA_\ct(S)$ such that $X$ is the
cokernel of $[N]\ra[M]$. Then $\rho_\ell(X)$ is the cokernel
of $\pH^0_\ell(N)\ra\pH^0_\ell(M)$, so $\rho_\ell(X)=0$ if and only
$\pH^0_\ell(N)\ra\pH^0_\ell(M)$ is surjective. Similary, $\rho_\sigma(X)=0$
if and only if $\pH^0(N)\ra\pH^0(M)$ is surjective. We can check these
conditions on a Zariski open covering of $S$, so we may assume that
$S$ is affine.
Choose a nonempty
smooth open subset $U$ of $S$ such that the restrictions to $U$ of
$\rho_\ell(M)$, $\rho_\ell(N)$,
$\rho_\sigma(M)$ and
$\rho_\sigma(N)$ are all locally constant sheaves placed in degree
$-\dim S$.
As $S$ is affine,
after shrinking $U$, we may assume that $U$ is the complement
of the vanishing set of a nonzero function $f\in\mathcal{O}(S)$.
By <cit.>, we have
that $\rho_\ell(N)\ra\rho_\ell(M)$ is surjective
if and only if both
and ${}^p\Phi_f^\sM\rho_\ell(N)\ra{}^p\Phi_f^\sM\rho_\ell(M)$ are,
which is equivalent to the surjectivity of
and $\rho_\ell(\Phi_f N)\ra\rho_\ell(\Phi_f M)$. We have a similar statement
for $\rho_\sigma$.
As $\dim(S-U)<\dim(S)$, we can use the induction hypothesis
to reduce to the case $S=U$.
It suffices to check the result on an étale cover of $S$, so we
may assume that $S$ has a rational point $x$.
Let $i:x\ra S$ be the obvious inclusion.
As $\rho_\ell(N)[-\dim S]$ and
$\rho_\ell(M)[-\dim S]$ are locally constant sheaves on $S$, the morphism
$\rho_\ell(N)\ra\rho_\ell(M)$ is surjective if and only if
$\rho_\ell(i^* N[-\dim S])\ra\rho_\ell(i^* M[-\dim S])$ is, and similarly
for $\rho_\sigma$. So we are reduced to the result on
the scheme $x$, which we have
already treated.

Let $k$ be a field of characteristic $0$ and $S$ a quasi-projective
scheme over $k$. We have a canonical $\bbQ$-linear
abelian category of perverse Nori motives
$\sM(S)$, together with a cohomological functor $\pH^0_\sM:\DA_\ct(S)\ra
\sM(S)$, with a $\ell$-adic realization functor
$\rat^\sM_{S,\ell}:\sM(S)\ra\sP(S,\bbQ_\ell)$ for every prime number
$\ell$, with a Betti realization functor
$\rat^\sM_{S,\sigma}:\sM(S)\ra\sP(S)$ for every embedding
$\sigma:k\ra\bbC$, and it has a formalism of the 4 operations, duality,
unipotent nearby and vanishing cycles compatible with all these operations.

§.§ Definition of weights

We fix a field $k$ of characteristic zero and a quasi-projective
scheme $S$ over $k$. We first define weights via the $\ell$-adic realizations.

Let $w\in\bbZ$.
Let $K$ be an object of $\sM(S)$. We say that $K$ is of weight $\leq w$
(resp. $\geq w$) if $\rat^\sM_{S,\ell}(K)\in\Ob(\sP_m(S,\bbQ_\ell))$
is of weight $\leq w$ (resp.
$\geq w$) for every odd prime number $\ell$. We say that $K$ is pure of
weight $w$ if it is both of weight $\leq w$ and of weight $\geq w$.

In prop:weightsaremotivic, we will a more intrisic definition of weights that does not use
the realization functors.

A weight filtration on an object $K$ of $\sM(S)$ is
an increasing filtration $W_\bullet K$ on $K$ such that $W_i K=0$ for
$i$ small enough, $W_i K=K$ for $i$ big enough, and $W_{i}K/W_{i-1}K$ is
pure of weight $i$ for every $i\in\bbZ$.

The next result follows immediately from the similar result in
the categories of mixed horizontal perverse sheaves (see Proposition 3.4 and Lemma 3.8 of

Let $K,L$ be objects of $\sM(S)$, and let $w\in\bbZ$.

(i) If $K$ is of weight $\leq w$ (resp. $\geq w$), so is every
subquotient of $K$.

(ii) If $K$ is of weight $\leq w$ and $L$ is of weight $\geq w+1$,
then $\Hom_{\sM(S)}(K,L)=0$.

A weight filtration on an object of $\sM(S)$ is unique
if it exists, and morphisms of $\sM(S)$ are strictly compatible with
weights filtrations.
In particular, if an object of $\sM(S)$ has a weight filtration, then so do
all its subquotients.

§.§ Application of Bondarko's weight structures

Let $S$ be as in the previous subsection. We will now make use of Bondarko's
Chow weight structure on $\DA_\ct(S)$.
Let $\Chow(S)$ be the full subcategory of $\DA_\ct(S)$ whose objects
are direct factors of finite direct sums of objects of the form
$f_!\bbQ_X(d)[2d]$, with $f:X\ra S$ a proper morphism from a smooth $k$-scheme
$X$ to $S$ and $d\in\bbZ$. Then, as shown in <cit.>, see also <cit.>, there exists a unique weight structure
on $\DA_\ct(S)$ with heart $\Chow(S)$
(see <cit.> or <cit.> for the definition of a weight structure).

In particular, for every object $K$ of $\DA_\ct(S)$, there exists an
exact triangle $A\ra K\ra B\xra{+1}$ (not unique) such that
$A$ (resp. $B$) is a direct factor of a successive extension of objects of
$\Chow(S)[i]$ with $i\leq 0$ (resp. $i\geq 1$).

Every object of $\sM(S)$ has a weight filtration. Moreover,
if $S=\Spec k$ and $\sigma$ is an embedding of $k$ in $\bbC$, then the
notion of weights of Definition <ref>
coincides with that of

We first prove that, if $M$ is an object of $\Chow(S)$,
then $\pH^0_\sM(M)$ is pure of weight $0$ in our sense, and also in the sense
of <cit.> if $S=\Spec k$ with $k$ embeddable in
$\bbC$. The second statement is actually an immediate consequence of
<cit.> and of the motivic Chow's lemma
(see for example <cit.>). To prove the first statement,
by definition of the weights on $\sP_m(S,\bbQ_\ell)$,
we may assume that $k$ is finitely generated
over $\bbQ$; then the statement follows immediately from <cit.>
(see the remark on page 116 of [39]).

Then we note that every object of $\sM(S)$ is a quotient of an
object of the form $\pH^0_\sM(M)$, for $M\in\Ob(\DA_\ct(S))$ (because
this is true for objects of $\Ab^\tr(\DA_\ct(S))$). So it suffices
to prove the result for objects in the essential image of
$\pH^0_\sM$. Let $M\in\Ob(\DA_\ct(S))$, and let $K=\pH^0_\sM(M)$.
Let $w\in\bbZ$.
By the first part of the proof,
if $M$
is a direct factor of a successive extension of objects
of $\Chow(S)[i]$ with $i\leq w$ (resp $i\geq w+1$), then $\pH^0_\sM(M)$
is of weight $\leq w$ (resp. $\geq w+1$) in our sense, and also in the
sense of [41] when this applies. In general, using the Chow weight structure of Bondarko, we can find an exact triangle
$A\ra M\ra B\xra{+1}$ such that $A$ (resp. $B$)
is a direct factor of a successive extension of objects
of $\Chow(S)[i]$ with $i\leq w$ (resp $i\geq w+1$). Applying $\pH^0_\sM$,
we get an exact sequence $\pH^0_\sM(A)\ra K\ra\pH^0_\sM(B)$,
with $\pH^0_\sM(A)$ of weight $\leq w$ and $\pH^0_\sM(B)$ of weight $\geq w+1$.
If we set $W_w K=\Img(\pH^0_\sM(A)\ra K)$, then $W_w K$ is of weight $\leq w$
and $K/W_w K$ is of weight $\geq w+1$. This defines a weight filtration
on $K$.

Weights and the related weight filtration so far have been defined and constructed for perverse motives via
the $\ell$-adic realizations. As we shall see now, we can also define weights more directly.
Let $\DA_\ct(S)_{w\leq i}$ be the full subcategory of $\DA_\ct(S)$ whose objects are direct factors of successive extensions of objects of
$\Chow(S)[w]$ with $w\leq i$ and consider the Abelian category

\[\sM(S)_{w\leq i}:=\Ab^\ad(\DA_\ct(S)_{w\leq i},\pH^0_\ell),\]

for some odd prime number $\ell$.
It follows from prop:independancel that this category, up to an equivalence, does not depend on $\ell$. Indeed, the universal property provides a commutative diagram (up to isomorphisms of functors)

\[\xymatrix{{\DA_\ct(S)_{w\leq i}}\ar[d]^-I\ar[r] &{\Ab^\ad(\DA_\ct(S)_{w\leq i})}\ar[d]^-{J}\ar[rd]^-{\varrho_\ell} & {}\\
{\DA_\ct(S)}\ar[r]\ar@/_2em/[rr]_-{\pH^0_\ell} &{\Ab^\tr(\DA_\ct(S))}\ar[r]^-{\rho_\ell} & {\sP_m(S,\bbQ_\ell)}}\]

where $I$ is the inclusion and $J,\varrho_\ell$ are exact functors. As by construction $\sM(S)_{w\leq i}:=\Ab^\ad(\DA_\ct(S)_{w\leq i})/\Ker\varrho_\ell$ it suffices to show that $\Ker\varrho_\ell$ is independant on $\ell$. Let $A$ be an object in $\Ab^\ad(\DA_\ct(S)_{w\leq i})$. Since $A$ belongs to $\Ker\varrho_\ell$ if and only if $J(A)$ belongs to $\Ker\rho_\ell$ our claim follows from prop:independancel.

The inclusion $\DA_\ct(S)_{w\leq i}\subseteq \DA_\ct(S)$ induces a faithful exact functor

\[u_i:\sM(S)_{w\leq i}\ra \sM(S).\]

Let $K$ be an object in $\sM(S)$. Given an object $(L,\alpha:u_i(L)\ra K)$ in the slice category $\sM(S)_{w\leq i}/K $ we can consider the subobject $\Img\alpha$ of $K$ and define $\eusm W_iK$ to be the union of all such subobjects in $K$, that is, we set

\[\eusm W_iK:=\colim_{(L,\alpha)\in\sM(S)_{w\leq i}/K}\Img\alpha.\]

This construction is functorial in $K$ (and moreover using the inclusion of $\DA_\ct(S)_{w\leq i} $ in $\DA_\ct(S)_{w\leq i+1} $ it is easy to see that it defines a filtration on $K$).

Let $K\in\sM(S)$. Then, $W_iK=\eusm W_i K$, for every integer $i\in\bbZ$.

As observed in the proof of cor_weight_filtration, if $M$ belongs to $\DA_\ct(S)_{w\leq i}$, then $\pH^0_\sM(M)$ is of weight $\leq i$. Hence, the functor $u_i$ takes its values in the Abelian subcategory of $\sM(S)$ formed by the objects of weight $\leq i$. As a consequence, for $(L,\alpha)$ in the slice category $\sM(S)_{w\leq i}/K$, the subobject $\Img\alpha$ of $K$ is of weight $\leq i$ and therefore $\eusm W_i K\subseteq W_iK$.

Conversely, there exists an epimorphism $e:\pH^0_\sM(M)\twoheadrightarrow K$ where $M$ belongs to $\DA_\ct(S)$. By construction

\[W_i\pH^0_\sM(M):=\Img\left(\pH^0_\sM(A)\ra\pH^0_\sM(M)\right)\subseteq \eusm W_i\pH^0_\sM(M)\]

where $A$ is an object of $\DA_\ct(S)_{w\leq i}$ that fits in an exact triangle $A\ra M\ra B\xra{+1}$ such that $B$
is a direct factor of a successive extension of objects
of $\Chow(S)[w]$ with $w\geq i+1$.
Therefore, since the weight filtration on $K$ is the induced filtration (see coro:strictness), we get

\[W_iK=e(W_i\pH^0_\sM(M))\subseteq e(\eusm W_i\pH^0_\sM(M))\subseteq \eusm W_i K.\]

This concludes the proof.

§.§ The intermediate extension functor

Recall the definition of the intermediate extension functor, that already appeared in the proof of coro:IC.
Let $j:S\ra T$ be a quasi-finite morphism between
quasi-projective $k$-schemes. We define a functor $j_{!*}^\sM:\sM(S)\ra\sM(T)$
by $j_{!*}^\sM(K)=\Img(\mathrm{H}^0(j_!^\sM K)\ra\mathrm{H}^0(j_*^\sM K))$.

Note that, as $j$ is quasi-finite, the functor $j_!^\sM$ is right exact
and the functor $j_*^\sM$ is left exact. In particular, the functor
$j_{!*}^\sM$ preserves injective and surjective morphisms, but it is not
exact in general.

Let $j:S\ra T$ be an open immersion, and let
Then, if $K\in
\Ob\sM(S)$ is of weight $\leq w$ (resp. of weight $\geq w$, resp.
pure of weight $w$), so is $j_{!*}^\sM K$.

Also, the functor $j_{!*}^\sM$ is exact on the full abelian subcategory
of objects that are pure of weight $w$.

It suffices to show these statement for mixed $\ell$-adic
perverse sheaves. The first statement follows from <cit.>
(more precisely, if $j$ is not affine,
it follows from <cit.> ). The second statement follows from <cit.>.

§.§ Pure objects

Let us start with the definition of objects with strict support on a given closed subscheme.
Let $Z$ be a closed integral subscheme of $S$.
We say that an object $K$ of $\sM(S)$ has strict support $Z$ if
$K_{|S-Z}=0$ and if, for every nonempty open subset $j:U\ra Z$, the adjunction
morphism $K\ra j_*^\sM j_\sM^* K$ is injective and induces an
isomorphism between $K$ and
$j_{!*}^\sM j_\sM^* K$.

For example, if $K_{|S-Z}=0$ and
if there exists a smooth dense open subset $j:U\ra Z$ such that
$\rat^\sM_U(K_{U})[-\dim U]$ (or any $\rat^\sM_{U,\ell}(K_{U})[-\dim U]$ for
some prime number $\ell$) is locally constant and $K_Z=j_{!*}^\sM(K_{|U})$,
then $K$ has strict support $Z$.
Indeed, this follows from the similar result for perverse sheaves, which
follows from <cit.>  (note that the proof of this result does
not use the hypothesis that $\mathrm{L}$ is irreducible).

(Compare with <cit.>.) Let $K$ be an object of
$\sM(S)$, and suppose that $K$ is pure of some weight. Then we can write
$K=\bigoplus_Z K_Z$, where the sum is over all integral closed subschemes
$Z$ of $S$, each $K_Z$ is an object of $\sM(S)$ with strict support $Z$, and
$K_Z=0$ for all but finitely many $Z$.

We prove the result by Noetherian induction on $S$. If
$\dim S=0$, there is nothing to prove. Suppose that $\dim S\geq 1$,
and let $j:U\ra S$ be a nonempty open affine subset of $S$.
After shrinking $U$, we may assume that $U$ is smooth and that $\rat^\sM_S
(K)[-\dim U]$ is a locally constant sheaf on $U$.
Let $w$ be the
weight of $K$. Then  <cit.> implies that
$j_*^\sM j^*_\sM K/j_{!*}^\sM j^*_\sM K$ is of weight $\geq w+1$, so
the adjunction morphism
$K\ra j_*^\sM j^*_\sM K$ factors through a morphism
$K\ra j_{!*}^\sM j^*_\sM K$. Similarly, the adjunction morphism
$j_!^\sM j^*_\sM K\ra K$ factors through a morphism
$j_{!*}^\sM j^*_\sM\ra K$. By definition of $j_{!*}^\sM$, the composition
$j_{!*}^\sM j^*_\sM K\ra K\ra j_{!*}^\sM j^*_\sM K$ is the identity of
$j_{!*}^\sM j^*_\sM K$. So we have $K=j_{!*}^\sM j^*_\sM K\oplus L$, with
$j^*_\sM L=0$. The first summand has support $\overline{U}$ by the remark
above, and $L_{|U}=0$, so the conclusion follows from the induction
hypothesis applied to $L_{|S-U}$.

Let $S$ be as before, and let $w\in\bbZ$. Let $\sM(S)_{\leq w}$ be
the full abelian subcategory of $\sM(S)$ whose objects
are motives that are pure of weight $w$.

Then $\sM(S)_{\leq w}$ is semisimple.

By prop_dec_support,
we may assume that $S$ is integral,
and it suffices to prove the result for the full subcategory $\sM(S)_w^0$ of
objects in $\sM(S)_w$ with strict support $S$ itself.

Let $\eta$ be the generic point of $S$. By
we have a full and essentially surjective exact functor
(given by the restriction morphisms) $2-\varinjlim_U\sM(U)\ra\sM(\eta)$,
where the limit is over the projective system of nonempty affine open
subsets $U$ of $S$. For such a $U$, we denote by
$\sM(U)_w^0$ the full subcategory of $\sM(U)$ whose objects are
motives that are pure of weight $w$ and have strict support $U$.
By Proposition <ref>, the functor above induces a
full and essentially surjective functor
$2-\varinjlim_U\sM(U)_w\ra\sM(\eta)_w$, and,
by prop_dec_support, this is turn gives a
full and essentially surjective functor
Moreover, if $j:U\ra S$ is a nonempty open subset, then the exact functor
$j^*_\sM:\sM(S)^0_w\ra\sM(U)_w^0$ is an equivalence of categories, because
it has a quasi-inverse, given by $j_{!*}^\sM$. So we deduce that the
restriction functor $\sM(S)_w^0\ra\sM(\eta)_w$ is full and essentially
surjective. But this functor is also faithful, because the analogous functor
on categories of $\ell$-adic perverse sheaves is faithful. So
$\sM(S)^0_w\ra\sM(\eta)_w$ is an equivalence of categories, which means that
we just need to show the theorem in the case $S=\eta$, i.e. if $S$ is the
spectrum of a field.

Now suppose that $S=\Spec k$. Then, by prop_continuity_sM,
where the limit is over all the subfields $k'$ of $k$ that are
finitely generated over $\bbQ$. So it suffices to show the theorem for
$k$ finitely generated over $\bbQ$. But then we can embed $k$ into $\bbC$,
and the conclusion follows from <cit.>.

Let $f:X\ra S$ be a proper morphism of quasi-projective $k$-varieties with $X$ irreductible.
Let $j:U\ra X$ be an open immersion, and
$K$ be a motivic perverse sheaf on $U$. If $K$ is pure of weight $w$, then
$\mathrm{H}^i(f_*^\sM j^\sM_{!*}K)$ is a motivic perverse sheaf that is pure of weight $w+i$.

Let us say that $L\in\Db(\sM(S))$ is pure of weight $w$ if $\mathrm{H}^iL$ is pure of weight $w+i$ for every $i\in\bbZ$. For such an $L$, by coro:allfourop, it follows from the Weil conjectures proved by Deligne in [27] that $f^\sM_*L$ is pure of weight $w$ (see the remark after <cit.>). Hence, prop:purityintermediate ensures that $f_*^\sM j^\sM_{!*}K$ is pure of weight $w$. This gives the conclusion.

In particular, this provides (for geometric variations of Hodge structures) an arithmetic proof of Zucker's theorem <cit.> via reduction to positive characteristic and to the Weil conjectures <cit.>. More generally, in higher dimension:

Let $k$ be a field embedded into $\bbC$.
Let $X$ be an irreducible proper $k$-variety and $\scr L$ be a $\bbQ$-local system on a smooth dense open subscheme $U$ of $X$ of the form $\scr L=R^wg_*\bbQ_V$ where $g:V\ra U$ is a  smooth proper morphism  and $w\in\bbZ$ is an integer. Then, the intersection cohomology group $IH^i(X,\scr L)$, for $i\in\bbZ$, is canonically the Betti realization of a Nori motive over $k$ which is pure of weight $i+w$. In particular, $IH^i(X,\scr L)$ carries a canonical pure Hodge structure of weight $i+w$.

Let $d$ be the dimension of $X$, $\pi:X\ra\Spec(k)$ be the structural morphism and $j$ be the inclusion
of $U$ in $X$.
As in coro:IC, $IH^i(X,\scr L)$ is the Betti realization of the Nori motive $\mathrm{H}^{i-d}(\pi_*^\sM j^\sM_{!*}\mathrm{H}^{w+d}(g^\sM_*\bbQ_V))$, which is pure of weight $w+i$ by theo:weightIC.

[1]
Groupes de monodromie en géométrie algébrique. II, Lecture Notes
in Mathematics, Vol. 340, Springer-Verlag, Berlin-New York, 1973,
Séminaire de Géométrie Algébrique du Bois-Marie 1967–1969
(SGA 7 II), Dirigé par P. Deligne et N. Katz. 0354657

[2]
Théorie des topos et cohomologie étale des schémas. Tome 3,
Lecture Notes in Mathematics, Vol. 305, Springer-Verlag, Berlin-New York,
1973, Séminaire de Géométrie Algébrique du Bois-Marie
1963–1964 (SGA 4), Dirigé par M. Artin, A. Grothendieck et J. L.
Verdier. Avec la collaboration de P. Deligne et B. Saint-Donat. 0354654

[3]
Yves André, Pour une théorie inconditionnelle des motifs, Inst.
Hautes Études Sci. Publ. Math. (1996), no. 83, 5–49. 1423019

[4]
Donu Arapura, The Leray spectral sequence is motivic, Invent. Math.
160 (2005), no. 3, 567–589. 2178703 (2006m:14025)

[5]
, An abelian category of motivic sheaves, Adv. Math. 233
(2013), 135–195. 2995668

[6]
Joseph Ayoub, Les six opérations de Grothendieck et le formalisme des
cycles évanescents dans le monde motivique. I, Astérisque (2007),
no. 314, x+466 pp. (2008).

[7]
, Les six opérations de Grothendieck et le formalisme des
cycles évanescents dans le monde motivique. II, Astérisque (2007),
no. 315, vi+364 pp. (2008).

[8]
, Note sur les opérations de Grothendieck et la réalisation
de Betti, J. Inst. Math. Jussieu 9 (2010), no. 2, 225–263.

[9]
, La réalisation étale et les opérations de Grothendieck,
Ann. Sci. École Norm. Sup. 47 (2014), no. 1, 1–141.

[10]
, L'algèbre de Hopf et le groupe de Galois motiviques d'un
corps de caractéristique nulle, I, J. Reine Angew. Math. 693
(2014), 1–149.

[11]
, L'algèbre de Hopf et le groupe de Galois motiviques d'un
corps de caractéristique nulle, II, J. Reine Angew. Math. 693
(2014), 151–226.

[12]
, Periods and the conjectures of Grothendieck and
Kontsevich-Zagier, Eur. Math. Soc. Newsl. (2014), no. 91, 12–18.

[13]
, Motifs des variétés analytiques rigides, Mém. Soc.
Math. Fr. (N.S.) (2015), no. 140-141, vi+386. 3381140

[14]
Luca Barbieri-Viale, Annette Huber, and Mike Prest, Tensor structure for
nori motives, <https://arxiv.org/abs/1803.00809v1>, 2018.

[15]
Luca Barbieri-Viale and Mike Prest, Definable categories and
$\mathbb{T}$-motives, to appear in Rend. Sem. Mat. Univ. Padova, 2018.

[16]
A. Beilinson, Remarks on Grothendieck's standard conjectures,
Regulators, Contemp. Math., vol. 571, Amer. Math. Soc., Providence, RI, 2012,
pp. 25–32. 2953406

[17]
A. A. Beĭlinson, How to glue perverse sheaves, $K$-theory,
arithmetic and geometry (Moscow, 1984–1986), Lecture Notes in Math., vol.
1289, Springer, Berlin, 1987, pp. 42–51. 923134 (89b:14028)

[18]
, On the derived category of perverse sheaves, $K$-theory,
arithmetic and geometry (Moscow, 1984–1986), Lecture Notes in Math., vol.
1289, Springer, Berlin, 1987, pp. 27–41.

[19]
A. A. Beĭlinson, J. Bernstein, and P. Deligne, Faisceaux pervers,
Analysis and topology on singular spaces, I (Luminy, 1981), Astérisque,
vol. 100, Soc. Math. France, Paris, 1982, pp. 5–171.

[20]
A. A. Beĭlinson, How to glue perverse sheaves, $K$-theory,
arithmetic and geometry (Moscow, 1984–1986), Lecture Notes in Math., vol.
1289, Springer, Berlin, 1987, pp. 42–51. 923134

[21]
Mikhail V. Bondarko, Weights for relative motives: relation with mixed
complexes of sheaves, Int. Math. Res. Not. IMRN (2014), no. 17, 4715–4767.

[22]
Utsav Choudhury and Martin Gallauer Alves de Souza, An isomorphism of
motivic Galois groups, Adv. Math. 313 (2017), 470–536.

[23]
Denis-Charles Cisinski and Frédéric Déglise, Triangulated
categories of mixed motives,
<http://deglise.perso.math.cnrs.fr/docs/2012/DM.pdf>, 2013.

[24]
Denis-Charles Cisinski and Amnon Neeman, Additivity for derivator
$K$-theory, Adv. Math. 217 (2008), no. 4, 1381–1475.

[25]
Mark Andrea A. de Cataldo, The perverse filtration and the Lefschetz
hyperplane theorem, II, J. Algebraic Geom. 21 (2012), no. 2,
305–345. 2877437

[26]
Mark Andrea A. de Cataldo and Luca Migliorini, The perverse filtration
and the Lefschetz hyperplane theorem, Ann. of Math. (2) 171
(2010), no. 3, 2089–2113. 2680404

[27]
Pierre Deligne, La conjecture de Weil. II, Inst. Hautes Études
Sci. Publ. Math. (1980), no. 52, 137–252.

[28]
, À quoi servent les motifs?, Motives (Seattle, WA,
1991), Proc. Sympos. Pure Math., vol. 55, Amer. Math. Soc., Providence, RI,
1994, pp. 143–161. 1265528 (95c:14013)

[29]
Najmuddin Fakhruddin, Notes of Nori's Lectures on Mixed Motives,
TIFR, Mumbai, 2000.

[30]
Laurent Fargues, Filtration de monodromie et cycles évanescents
formels, Invent. Math. 177 (2009), no. 2, 281–305. 2511743

[31]
Javier Fresán and Peter Jossen, Exponential motives,
<http://javier.fresan.perso.math.cnrs.fr/expmot.pdf>, 2018.

[32]
Peter Freyd, Representations in abelian categories, Proc. Conf.
Categorical Algebra (La Jolla, Calif., 1965), Springer, New York,
1966, pp. 95–120. 0209333

[33]
Ofer Gabber, Notes on some $t$-structures, Geometric aspects of
Dwork theory. Vol. I, II, Walter de Gruyter, Berlin, 2004,
pp. 711–734. 2099084

[34]
Moritz Groth, Derivators, pointed derivators and stable derivators,
Algebr. Geom. Topol. 13 (2013), no. 1, 313–374. 3031644

[35]
, Introduction to the theory of derivators, Project of book
available on the author webpage, 334 p., 2018.

[36]
A. Grothendieck, Éléments de géométrie algébrique. IV.
étude locale des schémas et des morphismes de schémas IV, Inst.
Hautes Études Sci. Publ. Math. (1967), no. 32, 361. 0238860

[37]
Alexandre Grothendieck, Les dérivateurs, Edited by M. Künzer, J.
Malgoire, G. Maltsiniotis, 1991.

[38]
David Hébert, Structure de poids à la Bondarko sur les motifs de
Beilinson, Compos. Math. 147 (2011), no. 5, 1447–1462.

[39]
Annette Huber, Mixed perverse sheaves for schemes over number fields,
Compositio Math. 108 (1997), no. 1, 107–121. 1458759

[40]
, Realization of Voevodsky's motives, J. Algebraic Geom.
9 (2000), no. 4, 755–799. 1775312

[41]
Annette Huber and Stefan Müller-Stach, Periods and Nori motives,
Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of
Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd
Series. A Series of Modern Surveys in Mathematics], vol. 65, Springer, Cham,
2017, With contributions by Benjamin Friedrich and Jonas von Wangenheim.

[42]
Luc Illusie, Yves Laszlo, and Fabrice Orgogozo (eds.), Travaux de
Gabber sur l'uniformisation locale et la cohomologie étale des
schémas quasi-excellents, Société Mathématique de France,
Paris, 2014, Séminaire à l'École Polytechnique 2006–2008. [Seminar
of the Polytechnic School 2006–2008], With the collaboration of
Frédéric Déglise, Alban Moreau, Vincent Pilloni, Michel Raynaud,
Joël Riou, Benoît Stroh, Michael Temkin and Weizhe Zheng,
Astérisque No. 363-364 (2014) (2014). 3309086

[43]
Florian Ivorra, Perverse, Hodge and motivic realizations of étale
motives, Compos. Math. 152 (2016), no. 6, 1237–1285.

[44]
, Perverse Nori motives, Math. Res. Lett. 24 (2017),
no. 4, 1097–1131. 3723805

[45]
Florian Ivorra and Takao Yamazaki, Nori motives of curves with modulus
and Laumon 1-motives, Canad. J. Math. 70 (2018), no. 4, 868–897.

[46]
Uwe Jannsen, Motivic sheaves and filtrations on Chow groups, Motives
(Seattle, WA, 1991), Proc. Sympos. Pure Math., vol. 55, Amer. Math. Soc.,
Providence, RI, 1994, pp. 245–302. 1265533 (95c:14006)

[47]
J. F. Jardine, Motivic symmetric spectra, Doc. Math. 5 (2000),
445–553 (electronic).

[48]
Masaki Kashiwara and Pierre Schapira, Sheaves on manifolds, Grundlehren
der Mathematischen Wissenschaften [Fundamental Principles of Mathematical
Sciences], vol. 292, Springer-Verlag, Berlin, 1994, With a chapter in French
by Christian Houzel, Corrected reprint of the 1990 original.

[49]
Maxim Kontsevich and Yan Soibelman, Cohomological Hall algebra,
exponential Hodge structures and motivic Donaldson-Thomas invariants,
Commun. Number Theory Phys. 5 (2011), no. 2, 231–352. 2851153

[50]
Henning Krause, Functors on locally finitely presented additive
categories, Colloq. Math. 75 (1998), no. 1, 105–132. 1487973

[51]
Georges Maltsiniotis, Carrés exacts homotopiques et dérivateurs,
Cah. Topol. Géom. Différ. Catég. 53 (2012), no. 1, 3–63.

[52]
Fabien Morel and Vladimir Voevodsky, ${\bf A}^1$-homotopy theory of
schemes, Inst. Hautes Études Sci. Publ. Math. (1999), no. 90, 45–143

[53]
Sophie Morel, Mixed $\ell$-adic complexes for schemes over number
fields, <https://arxiv.org/pdf/1806.03096.pdf>, 2018.

[54]
Amnon Neeman, Triangulated categories, Annals of Mathematics Studies,
vol. 148, Princeton University Press, Princeton, NJ, 2001. 1812507

[55]
Mike Prest, Definable additive categories: purity and model theory, Mem.
Amer. Math. Soc. 210 (2011), no. 987, vi+109. 2791358

[56]
Ryan Reich, Notes on Beilinson's “How to glue perverse sheaves”,
J. Singul. 1 (2010), 94–115. 2671769

[57]
Morihiko Saito, Duality for vanishing cycle functors, Publ. Res. Inst.
Math. Sci. 25 (1989), no. 6, 889–921. 1045997

[58]
, Extension of mixed Hodge modules, Compositio Math.
74 (1990), no. 2, 209–234. 1047741

[59]
, Mixed Hodge modules, Publ. Res. Inst. Math. Sci. 26
(1990), no. 2, 221–333. 1047415 (91m:14014)

[60]
, On the formalism of mixed sheaves, ,

[61]
Michael Shulman, Comparing composites of left and right derived
functors, New York J. Math. 17 (2011), 75–125. 2781909

[62]
The Stacks Project Authors, Stacks Project,
<http://stacks.math.columbia.edu>, 2018.

[63]
Jean-Louis Verdier, Dualité dans la cohomologie des espaces localement
compacts, Séminaire Bourbaki, Vol. 9, Soc. Math. France, Paris, 1995,
pp. Exp. No. 300, 337–349.

[64]
, Des catégories dérivées des catégories abéliennes,
Astérisque (1996), no. 239, xii+253 pp. (1997), With a preface by Luc
Illusie, Edited and with a note by Georges Maltsiniotis. 1453167

[65]
Vladimir Voevodsky, $\bold A^1$-homotopy theory, Proceedings of the
International Congress of Mathematicians, Vol. I (Berlin, 1998),
no. Extra Vol. I, 1998, pp. 579–604 (electronic).

[66]
, Motivic cohomology groups are isomorphic to higher Chow groups
in any characteristic, Int. Math. Res. Not. (2002), no. 7, 351–355.
1883180 (2003c:14021)

[67]
Vadim Vologodsky, On the derived DG functors, Math. Res. Lett.
17 (2010), no. 6, 1155–1170. 2729639

[68]
Steven Zucker, Hodge theory with degenerating coefficients. $L_{2}$
cohomology in the Poincaré metric, Ann. of Math. (2) 109
(1979), no. 3, 415–476. 534758


# Learnable Prompt for Few-Shot Semantic Segmentation
in Remote Sensing Domain

Steve Andreas Immanuel
TelePIX
Seoul, South Korea
<EMAIL_ADDRESS>Hagai Raja Sinulingga
TelePIX
Seoul, South Korea
<EMAIL_ADDRESS>
###### Abstract

Few-shot segmentation is a task to segment objects or regions of novel classes
within an image given only a few annotated examples. In the generalized
setting, the task extends to segment both the base and the novel classes. The
main challenge is how to train the model such that the addition of novel
classes does not hurt the base classes performance, also known as catastrophic
forgetting. To mitigate this issue, we use SegGPT as our base model and train
it on the base classes. Then, we use separate learnable prompts to handle
predictions for each novel class. To handle various object sizes which
typically present in remote sensing domain, we perform patch-based prediction.
To address the discontinuities along patch boundaries, we propose a patch-and-
stitch technique by re-framing the problem as an image inpainting task. During
inference, we also utilize image similarity search over image embeddings for
prompt selection and novel class filtering to reduce false positive
predictions. Based on our experiments, our proposed method boosts the weighted
mIoU of a simple fine-tuned SegGPT from 15.96 to 35.08 on the validation set
of few-shot OpenEarthMap dataset given in the challenge.

##  1 Introduction

Generalized Few-Shot Segmentation (GFSS) is a computer vision task wherein the
model must effectively segment novel classes with limited examples alongside
the base classes it has been trained on. This task holds significant relevance
in remote sensing applications, where annotation costs are high and diverse
user interests necessitate adaptable segmentation models. For instance,
agricultural societies prioritize distinguishing between cultivated and fallow
land, whereas civil registration departments require accurate house mapping
for population estimation. Recognizing the critical importance of addressing
these challenges, the OpenEarthMap Land Cover Mapping Few-Shot Challenge [33],
jointly organized with the L3D-IVU CVPR 2024 Workshop, was convened to drive
advancements in solving the problem.

The strategy for addressing the GFSS problem currently revolves around two
approaches: 1) individually predicting each novel class and then merging the
results using fusion techniques [22, 13], and 2) relearning the classifier so
it can predict both base and novel classes simultaneously [15, 10, 28]. Our
method follows the first approach but diverges from existing methods in that
we train the model only once using data from the base class. For the novel
classes segmentation, we only derive a prompt for each class obtained by
training solely on the support set. The reason we chose this is due to the
emergence of new foundation models with strong generalization capabilities
[32, 12, 23]. The prompt for each novel class serves as an adaptation layer to
handle a specific novel class characteristics. Therefore, our method is able
to handle any number of novel classes without performance degradation on the
base classes. This approach is both simple and straightforward, yet highly
extensible to real-life scenarios.

Furthermore, this challenge presents characteristics commonly encountered in
remote sensing, particularly varying object sizes. In remote sensing imagery,
there are both large-scale objects like industrial complexes, roads, and
lakes, as well as smaller objects such as trees, boats, and houses. Common
strategies include employing multi-scale features [6] to capture both large
and small objects, or segmenting the image into smaller patches [30]. Our
approach incorporates elements of both strategies, utilizing a comprehensive
foundational model comprising multi-scale layers and leveraging detailed
inference results from smaller patches. Although segmenting images into
patches is not widely favored due to the risk of information loss and
discontinuous results along patch boundaries, our method addresses this issue
by introducing a patch-and-stitch technique.

The contribution and novelty of our approach can be summarized as:

  * •

We introduce a simple, yet effective method to handle novel classes prediction
in few-shot setting using learnable prompts. Initial training is only done on
the base classes while the learnable prompts are optimized using the frozen
model.

  * •

We propose a patch-and-stitch technique to smooth out the results in patch-
based predictions, especially along the patch boundaries. We also incorporate
similar prompt searching based on similarity and novel class filtering to
further boost the performance.

##  2 Related Work

###  2.1 Semantic Segmentation

The basis of GFSS is semantic segmentation where models assign labels to
individual pixels in an image. While methods like FCN [16] and encoder-decoder
architectures [24, 2] have improved per-pixel predictions, incorporating
context information through techniques like dilated convolutions and attention
mechanisms has been proven to further enhance segmentation accuracy. Recent
advancements, including pyramid pooling [11], parallel dilated convolutions
[5], and the adoption of vision transformers [26], have led to significant
improvements in segmentation quality. However, challenges persist in adapting
these models to handle unseen classes without extensive fine-tuning using
sufficiently annotated data.

###  2.2 Few-Shot Semantic Segmentation

Few-shot semantic segmentation (FSS) is formulated to specifically answer the
challenge wherein pixel-wise labeling is required for novel classes with
limited support examples, mainly focusing on the novel class prediction only
without the base class. Methods like PL [8] and PANet [29] adapt prototype
learning, while ASR [14] learns orthogonal prototypes. Other method like OSLSM
[25] assigns weights to the final classifier and PFENet [27] leverages pre-
trained backbone knowledge and addresses spatial inconsistency with a Feature
Enrichment Module (FEM). Despite FSS models’ effectiveness with support
samples, the practical scenarios involving both base and novel classes on the
target image has not been covered by these methods.

###  2.3 Generalized Few-Shot Semantic Segmentation

In the generalized setting, the task is to predict not only the novel classes
but also the base classes. Several approaches that have been proposed can be
seperated into two categories based on the prediction design setting.
Directly predict all novel and base classes

  * •

POP [15], employs orthogonal loss to separate base and novel classes from the
background. The training process involves two stages: initially training a
PSPNet to obtain the base model, and then training the novel model using
combined data from both base and novel classes, with the base data chosen
through random sampling. Notably, the POP model updates labels from the base
class at every epoch.

  * •

DIaM [10], focuses on finetuning the classifier using a modified InfoMax
framework and knowledge distillation techniques to retain base class
knowledge.

  * •

CAPL [28], pioneers GFSS with three classifiers: one for base class together
with novel class, one for base class only, and a classifier to combine both
result.

Predict classes individually

  * •

BAM [13], trains a ResNet50 encoder and decoder for each novel class,
utilizing weights to select support images. However, this approach is tailored
for the one-class novel scenario and the performance is reported to degrade as
the number of novel classes increased [10].

  * •

HSNet [22], utilizes a 4D sparse correlation tensor over feature pyramids
generated by a CNN-based backbone network. Although HSNet is not designed for
GFSS, but adaptation of the model as reported in [15] show a competitive
result.

###  2.4 Foundation Model

The concept of training large-scale models through semi-supervised learning on
extensive datasets, referred to as an upstream task, and subsequently
employing them as foundation models for fine-tuning on downstream tasks, has
gained significant prominence recently [31, 23]. These large models exhibit
strong generalization capabilities due to their training on diverse datasets.
Some of these models have been explored for addressing general remote sensing
tasks such as segmentation, object detection, change detection and super
resolution [7, 20, 21, 3, 1, 18]. Notably, recent research [32] demonstrates
that even without fine-tuning on specialized remote-sensing data, this
approach can rival SOTA FSS methods, particularly with an increasing number of
shots. This observation motivate our direction in exploring the potential of
the foundation model, as no prior attempts have been made to tackle the GFSS
problem using such approach.

##  3 Method

###  3.1 Preliminaries

Given an RGB image $\textbf{X}\in\mathbb{R}^{H\times W\times 3}$, where $H$
and $W$ denote the height and width of the image, the goal of semantic
segmentation task is to predict semantic map $\textbf{Y}\in\mathbb{R}^{H\times
W}$. Each pixel $\textbf{Y}_{ij}$ corresponds to a class label from a
predefined set $\mathcal{C}=\\{c_{1},...,c_{P}\\}$, where $P$ is the number of
classes, reflecting the semantic class of the corresponding pixel
$\textbf{X}_{ij}$. In few-shot setting, the training dataset contains only
base classes, while for each novel class, we are given $k$ samples of images
and their semantic maps which only contain the novel class.

###  3.2 Model Architecture

We use SegGPT [32] as the foundation model due to its strong generalizability.
Internally, SegGPT uses ViT [9] as the backbone and is trained with
smooth-$l_{1}$ loss.

###  3.3 Training

We follow the masked image modeling (MIM) approach where the objective is to
reconstruct the masked regions of the input image. To this end, a pair of
images is fed to the model instead of only one image. Prompt image
$\textbf{X}^{p}$ and target image $\textbf{X}^{t}$, and their corresponding
semantic maps $\textbf{Y}^{p}$ and $\textbf{Y}^{t}$, are provided, where
certain patches of the semantic maps are masked as shown in Fig. 1. All
$\textbf{X}^{p}$, $\textbf{X}^{t}$, $\textbf{Y}^{p}$, and $\textbf{Y}^{t}$
need to be of the same dimension $H$ x $W$ x 3. Therefore, the semantic maps
are transformed into image space by mapping each class label into a color
using color map $\mathcal{M}:\mathbb{R}\rightarrow\mathbb{R}^{3}$. This color
is randomized for each data sample. The idea is to force the model to learn
the contextual information in order to reconstruct the masked region, rather
than exploiting the color [32]. This is particularly useful in few-shot
setting, because it prevents overfitting to the base classes.

(a) Random masking strategy

(b) Half masking strategy

Figure 1: Masked image modeling approach

####  3.3.1 Base Classes

The base classes are trained following the standard MIM approach. Each data
sample consists of $\textbf{X}^{p}$, $\textbf{X}^{t}$, $\textbf{Y}^{p}$, and
$\textbf{Y}^{t}$. In order to select $\textbf{X}^{p}$ and $\textbf{X}^{t}$, we
initially generate all possible pair combinations of images in the training
set. Then, we adopt different masking strategy for each pair depending on the
classes present in each image. If $\textbf{X}^{p}$ and $\textbf{X}^{t}$
contain at least one different class, we randomly mask $\alpha$ portion of the
patches of $\textbf{Y}^{p}$ and $\textbf{Y}^{t}$ as shown in Fig. 1(a).
Alternatively, we mask the whole $\textbf{Y}^{t}$ as shown in Fig. 1(b) if and
only if $\textbf{X}^{p}$ and $\textbf{X}^{t}$ contain the exact same classes.

The idea is that if $\textbf{X}^{p}$ and $\textbf{X}^{t}$ contain the same
classes, then given $\textbf{Y}^{p}$, the model should be able to predict the
whole $\textbf{Y}^{t}$. In contrast, if their classes differ, the model should
reconstruct the masked patches by leveraging contextual information from the
unmasked regions.

####  3.3.2 Novel Classes

Due to the limited number of samples, the novel classes cannot be trained with
the same approach as base classes. SegGPT inherently has strong few-shot
capabilities by feeding the $k$ samples as the segmentation context. However,
as we show in Tab. 1, it still does not suffice for this challenge.

The primary obstacle in few-shot setting is how to make the model able to
predict the novel classes given few samples, while simultaneously retaining
the performance on the base classes. To this end, we use a learnable prompt Z
which acts as $\textbf{X}^{p}$ and $\textbf{Y}^{p}$. After we train the model
on the base classes, we freeze the whole model and optimize only Z. Given
there are $N$ novel classes, we create $\\{\textbf{Z}^{i}\\}_{i=1}^{N}$, each
tailored to the characteristics of the corresponding $i$-th novel class, and
train them independently using samples from each class. The main strength of
this approach is that the introduction of novel classes does not compromise
the performance of the base classes. Moreover, the optimization for Z using
the $k$ samples is much faster than the initial training on the base classes.
Every learnable prompt for each novel class only amounts to about 5MB of model
parameters, which is highly practical. To predict the novel classes, we simply
utilize the corresponding learned prompt Z in a plug-and-play manner.

For novel classes training, we only use the half masking strategy. Since the
semantic maps for the novel classes only contains the novel classes, this task
reduces into a binary classification between the novel class and background.
As the novel class is typically confined to small regions within the image,
direct prediction of the whole image would make the model trivially predict
everything as background. Therefore, the learnable prompt Z is optimized in
two phases. In phase 1, we adopt a sliding window approach to crop the image
into smaller patches, utilizing only those containing the novel class to
optimize Z, thus facilitating the model’s comprehension of the novel class.
Subsequently, in phase 2, we incorporate all patches, including those
featuring only background, to mitigate false positive predictions of the novel
class. Additionally, we only use white color to represent the novel class and
black color to represent background, as opposed to random color as in base
classes training.

###  3.4 Inference

Inference works similarly as half masking strategy in training. The image
$\textbf{X}^{p}$ and its semantic map $\textbf{Y}^{p}$ acts as the prompt to
give contextual information. Then, given the target image $\textbf{X}^{t}$,
the model predicts $\hat{\textbf{Y}}^{t}$. We generate a fixed color map
$\mathcal{M}$ and use it to transform $\textbf{Y}^{p}$ into image space, and
its inverse $\mathcal{M}^{-1}$ to transform the prediction
$\hat{\textbf{Y}}^{t}$ back to class label space. The class of the $ij$-th
pixel in $\hat{\textbf{Y}}^{t}$ can be determined as follows,

$class(\hat{\textbf{Y}}^{t}_{ij})=\arg\underset{c\in\mathcal{C}}{\min}\
d(\hat{\textbf{Y}}^{t}_{ij},\mathcal{M}^{-1}(c)),$ (1)

where $c$ iterates over the set of class labels $\mathcal{C}$, and $d$ is
cosine similarity distance.

Image Similarity Search. The quality of the prompt $\textbf{X}^{p}$ and
$\textbf{Y}^{p}$ greatly affects the prediction result $\hat{\textbf{Y}}^{t}$.
In general, the more similar $\hat{\textbf{X}}^{p}$ and
$\hat{\textbf{X}}^{t}$, the better the result. Additionally, SegGPT can
incorporate multiple prompts in order to generate more accurate results. We
leverage CLIP-ViT [23] to extract the embeddings of each images in the
training set. Then, we retrieve the top-$l$ most similar images to
$\textbf{X}^{t}$ using cosine similarity and use them as the prompt.

Patch-and-Stitch. In remote sensing domain, the objects are typically small
and scattered across the image. Processing the whole image directly often
leads to objects not being detected. Therefore, we partition the image into
$2$x$2$ equal non-overlapping patches and perform the prediction on those
patches independently. To get the result for the whole image, we can simply
combine the prediction result on those patches directly. However, there might
be some artefacts along the edges of the patches shown by the discontinuity of
color (see Fig. 4 column 3). To mitigate this, we perform additional
predictions on the middle regions that overlap adjacent patches as illustrated
in Fig. 2. Instead of predicting the entire overlapping region, we focus
solely on predicting the middle portion, while the remaining areas are filled
using the previous predictions from the non-overlapping patches to give more
context. This process effectively frames the task as an image inpainting task,
enabling seamless integration of non-overlapping patch predictions.

Figure 2: Seamless stitching between non-overlapping patches

To get the final prediction containing both base and novel classes, we first
perform prediction for the base classes. For each of the novel class, we do
not utilize the image similarity search to get similar images as the prompt is
essentially replaced with Z. Instead, we calculate the similarity between the
target image $\textbf{X}^{t}$ and the $k$ given samples, analogously using
CLIP-ViT and cosine similarity distance. If the similarity does not exceed a
certain threshold we skip processing the corresponding novel class for the
target image altogether. The idea is if $\textbf{X}^{t}$ is not similar with
the $k$ given samples, then it is unlikely to contain the novel class. This
approach helps to further reduce false positive prediction of novel classes.
Subsequently, we simply overlay the novel classes predictions on top of the
base classes prediction.

##  4 Experiment

Figure 3: Semantic map prediction results on the training set Figure 4:
Seamless stitching produces more detailed and continuous result

###  4.1 Dataset

The dataset used in the challenge is a few-shot dataset consists of 408
samples of the original OpenEarthMap (OEM) benchmark dataset [33]. The
challenge dataset extends the original 7 semantic classes (excluding
background class) of the OEM to 15 classes, which is split into 7:4:4 for base
classes, validation novel classes, and test novel classes, respectively. All
base, validation novel, and test novel classes are disjointed.

The 408 samples are split into 258 as training set, 50 as validation set, and
100 as test set. Validation and test set only contains novel classes. For each
novel class, 5 example images are given with their corresponding semantic
maps. Therefore, 20 images are given as the support set, while 30 and 80
images are used as the query set for the validation and test set,
respectively.

###  4.2 Implementation Details

We initialize SegGPT with the pretrained checkpoint provided by the original
authors [32]. Other initial foundation model checkpoints that are specialized
on remote sensing domain [19, 7, 21, 3, 1, 18] can also be used, which might
improve the performance. We leave this for future work. During training, we
use image augmentations including, random cropping, random horizontal and
vertical flip, and color jittering. We set the $\alpha$ value for the random
masking to 0.75 empirically. During inference, we select the top-5 most
similar images as the prompt. We use AdamW [17] optimizer and cosine annealing
rate scheduler [4] with learning rate of 1e-4 and linear warmup for 500 steps
by default. To optimize the learnable prompt, we simply treat Z as model
parameters of size $\mathbb{R}^{H\times W\times 3}$ times two (to represent
$\textbf{X}^{p}$ and $\textbf{Y}^{p}$). All codes are implemented in Python
using PyTorch. Training and experiments are conducted using 4 Nvidia RTX
A6000.

###  4.3 Evaluation Metric

The evaluation metric is weighted mean intersection-over-union (mIoU) over all
classes excluding the background. As the focus of the challenge is in the
novel class, the final evaluation metric is calculated as $0.4*\text{base
classes mIoU}+0.6*\text{novel classes mIoU}$.

###  4.4 Quantitative Results

The mIoU results111The IoU for each class is not available because the
detailed evaluation for some of our submissions cannot be viewed in the
submission portal. on the validation set of our proposed method are presented
in Tab. 1. We can see incremental improvements for each of the method that we
employ.

Simply using SegGPT that is finetuned on the OEM dataset only gives out mIoU
of 15.96, mainly because the model is unable to detect any of the novel
classes. Utilizing similar images as the prompt offers small improvement of
+1.86 because it only improves the mIoU on the base classes. It is only after
the integration of the learnable prompt that the model becomes capable of
predicting the novel classes, leading to a substantial increase of mIoU by
+7.44.

Incorporating the patch and stitch approach further enhances the mIoU to
29.41, demonstrating a significant improvement in capturing finer details and
increasing overall accuracy. Filtering the novel classes based on the image
similarity to the given $k$ samples also proves to be very effective, shown by
another +5.67 increase in mIoU. By filtering the novel classes, we reduce the
false positive prediction. Considering that we overlay the predictions of
novel classes on top of the base classes, false positive predictions
negatively impact not only the mIoU of the novel classes but also that of the
base classes. As for the test set, we obtain a weighted mIoU of 36.52222The
breakdown of each method is not available for the test set as in Tab. 1 due to
the limited number of submissions during the competition and the submission
portal is closed once the competition ends.

Table 1: The impact of each method used on the mIoU of the validation set  Method | mIoU
---|---
SegGPT Baseline [32] | 15.96
\+ Similar image prompts | 17.82
\+ Learnable prompt on novel classes | 25.26
\+ Patch and stitch | 29.41
\+ Filter on novel classes | 35.08

###  4.5 Qualitative Results

Fig. 3 shows the results of our model on the training set. Our model ables to
predict well on most classes. One particular class that our model struggle
with is bareland, as shown in the top-left image in Fig. 3. This is mainly due
to the very limited number of samples in the training set, as well as the
inconsistency of the class definitions [33].

In Fig. 4, we compare the results when directly predicting the whole image and
using the patch and stitch method that we proposed. By using non-overlapping
patch prediction, the model can capture finer details in the segmentation
between buildings. Finally, the stitching mechanism allows aggregation of
predictions from each patch seamlessly.

###  4.6 Impact of Color Map on Inference

While we can use any random color to fill $\mathcal{M}$, we observe
empirically that some colors lead to higher performance. Specifically, we want
objects that are typically located close together to have as different color
as possible, e.g. bareland and sea, tree and road. Colors that are far apart
from each other in RGB color space make it easier for the model to separate
the close pixels between objects, reducing ambiguity. This is also similar to
humans who can easily distinguish two adjacent objects that have contrasting
colors.

###  4.7 Limitations

Due to how inference works in SegGPT, the results highly depend on the prompt
that is given, therefore finding the most suitable prompt given an image is a
crucial aspect to consider. The patch-and-stitch approach that we use in the
inference also introduce additional computational cost as a trade-off for
details and accuracy on the results.

##  5 Conclusion

In this work, we proposed a method to handle novel classes prediction in few-
shot setting using learnable prompt. We also introduced some additional
techniques including prompt based on image similarity, patch-and-stitch, and
novel class filtering which led to substantial performance improvements.

## References

  * Akiva et al. [2022] Peri Akiva, Matthew Purri, and Matthew Leotta.  Self-supervised material and texture representation learning for remote sensing tasks.  In _CVPR_ , pages 8203–8215, 2022.
  * Badrinarayanan et al. [2017] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla.  Segnet: A deep convolutional encoder-decoder architecture for image segmentation.  _PAMI_ , pages 2481–2495, 2017.
  * Bastani et al. [2023] Favyen Bastani, Piper Wolters, Ritwik Gupta, Joe Ferdinando, and Aniruddha Kembhavi.  Satlaspretrain: A large-scale dataset for remote sensing image understanding.  In _ICCV_ , pages 16772–16782, 2023.
  * Bridle [1989] John Bridle.  Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters.  In _ICLR_ , pages 211–217, 1989.
  * Chen et al. [2018] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam.  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In _ECCV_ , pages 801–818, 2018.
  * Cheng and Lei [2023] Xiang Cheng and Hong Lei.  Semantic segmentation of remote sensing imagery based on multiscale deformable cnn and densecrf.  _Remote Sensing_ , page 1229, 2023.
  * Cong et al. [2022] Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David Lobell, and Stefano Ermon.  Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery.  _NEURIPS_ , pages 197–211, 2022.
  * Dong and Xing [2018] Nanqing Dong and Eric P Xing.  Few-shot semantic segmentation with prototype learning.  In _BMVC_ , page 4, 2018.
  * Dosovitskiy et al. [2022] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In _CVPR_ , 2022.
  * Hajimiri et al. [2023] Sina Hajimiri, Malik Boudiaf, Ismail Ben Ayed, and Jose Dolz.  A strong baseline for generalized few-shot semantic segmentation.  In _CVPR_ , pages 11269–11278, 2023.
  * Hou et al. [2020] Qibin Hou, Li Zhang, Ming-Ming Cheng, and Jiashi Feng.  Strip pooling: Rethinking spatial pooling for scene parsing.  In _CVPR_ , pages 4003–4012, 2020.
  * Kirillov et al. [2023] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.  Segment anything.  In _ICCV_ , pages 4015–4026, 2023.
  * Lang et al. [2022] Chunbo Lang, Gong Cheng, Binfei Tu, and Junwei Han.  Learning what not to segment: A new perspective on few-shot segmentation.  In _CVPR_ , pages 8057–8067, 2022.
  * Liu et al. [2021] Binghao Liu, Yao Ding, Jianbin Jiao, Xiangyang Ji, and Qixiang Ye.  Anti-aliasing semantic reconstruction for few-shot semantic segmentation.  In _CVPR_ , pages 9747–9756, 2021.
  * Liu et al. [2023] Sun-Ao Liu, Yiheng Zhang, Zhaofan Qiu, Hongtao Xie, Yongdong Zhang, and Ting Yao.  Learning orthogonal prototypes for generalized few-shot semantic segmentation.  In _CVPR_ , 2023.
  * Long et al. [2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully convolutional networks for semantic segmentation.  In _CVPR_ , pages 3431–3440, 2015.
  * Loshchilov and Hutter [2019] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  _arXiv preprint, arXiv:1711.05101_ , 2019.
  * Mall et al. [2023] Utkarsh Mall, Bharath Hariharan, and Kavita Bala.  Change-aware sampling and contrastive learning for satellite images.  In _CVPR_ , pages 5261–5270, 2023.
  * Mall et al. [2024] Utkarsh Mall, Cheng Perng Phoo, Meilin Kelsey Liu, Carl Vondrick, Bharath Hariharan, and Kavita Bala.  Remote sensing vision-language foundation models without annotations via ground remote alignment.  In _ICLR_ , 2024.
  * Manas et al. [2021] Oscar Manas, Alexandre Lacoste, Xavier Giró-i Nieto, David Vazquez, and Pau Rodriguez.  Seasonal contrast: Unsupervised pre-training from uncurated remote sensing data.  In _ICCV_ , pages 9414–9423, 2021.
  * Mendieta et al. [2023] Matías Mendieta, Boran Han, Xingjian Shi, Yi Zhu, and Chen Chen.  Towards geospatial foundation models via continual pretraining.  In _ICCV_ , pages 16806–16816, 2023.
  * Min et al. [2021] Juhong Min, Dahyun Kang, and Minsu Cho.  Hypercorrelation squeeze for few-shot segmentation.  In _ICCV_ , 2021.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision.  In _ICML_ , 2021.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _MICCAI_ , pages 234–241. Springer, 2015.
  * Shaban et al. [2017] Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, and Byron Boots.  One-shot learning for semantic segmentation.  _arXiv preprint arXiv:1709.03410_ , 2017.
  * Strudel et al. [2021] Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid.  Segmenter: Transformer for semantic segmentation.  In _ICCV_ , pages 7262–7272, 2021.
  * Tian et al. [2020] Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Zhicheng Yang, Ruiyu Li, and Jiaya Jia.  Prior guided feature enrichment network for few-shot segmentation.  _PAMI_ , pages 1050–1065, 2020.
  * Tian et al. [2022] Zhuotao Tian, Xin Lai, Li Jiang, Shu Liu, Michelle Shu, Hengshuang Zhao, and Jiaya Jia.  Generalized few-shot semantic segmentation.  In _CVPR_ , 2022.
  * Wang et al. [2019] Kaixin Wang, Jun Hao Liew, Yingtian Zou, Daquan Zhou, and Jiashi Feng.  Panet: Few-shot image semantic segmentation with prototype alignment.  In _ICCV_ , pages 9197–9206, 2019.
  * Wang et al. [2023a] Xiaolei Wang, Zirong Hu, Shouhai Shi, Mei Hou, Lei Xu, and Xiang Zhang.  A deep learning method for optimizing semantic segmentation accuracy of remote sensing images based on improved unet.  _Scientific reports_ , page 7600, 2023a.
  * Wang et al. [2023b] Xinlong Wang, Wen Wang, Yue Cao, Chunhua Shen, and Tiejun Huang.  Images speak in images: A generalist painter for in-context visual learning.  In _CVPR_ , 2023b.
  * Wang et al. [2023c] Xinlong Wang, Xiaosong Zhang, Yue Cao, Wen Wang, Chunhua Shen, and Tiejun Huang.  SegGPT: Segmenting everything in context.  In _ICCV_ , 2023c.
  * Xia et al. [2023] Junshi Xia, Naoto Yokoya, Bruno Adriano, and Clifford Broni-Bediako.  Openearthmap: A benchmark dataset for global high-resolution land cover mapping.  In _WACV_ , pages 6254–6264, 2023.