#  $\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ Operator Connects Slow-Fast
Networks for Full Context Interaction

Harvie Zhang
HyperEvol AI Lab
<EMAIL_ADDRESS>
###### Abstract

The self-attention mechanism utilizes large implicit weight matrices,
programmed through dot product-based activations with very few trainable
parameters, to enable long sequence modeling. In this paper, we investigate
the possibility of discarding residual learning by employing large implicit
kernels to achieve full context interaction at each layer of the network. To
accomplish it, we introduce coordinate-based implicit MLPs as a slow network
to generate hyper-kernels111The definition of Hyper-Kernel comes from
HyperNEAT [39]. for another fast convolutional network222The slow network
serves as a hyper-kernel generator, while the fast network either interacts
with the context or has context-dependent fast weights. Our definitions of
slow and fast networks differ from [36, 35, 16].. To get context-varying
weights for fast dynamic encoding, we propose a
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator that connects hyper-
kernels ($\mathcal{W}$) and hidden activations ($\mathcal{Z}$) through simple
elementwise multiplication, followed by convolution of $\mathcal{Z}$ using the
context-dependent $\mathcal{W}$. Based on this design, we present a novel
Terminator architecture that integrates hyper-kernels of different sizes to
produce multi-branch hidden representations for enhancing the feature
extraction capability of each layer. Additionally, a bottleneck layer is
employed to compress the concatenated channels, allowing only valuable
information to propagate to the subsequent layers. Notably, our model
incorporates several innovative components and exhibits excellent properties,
such as introducing local feedback error for updating the slow network, stable
zero-mean features, faster training convergence, and fewer model parameters.
Extensive experimental results on pixel-level 1D and 2D image classification
benchmarks demonstrate the superior performance of our architecture. Our code
will be publicly available at https://github.com/hyperevolnet/Terminator.

##  1 Introduction

Figure 1: Comparison between the residual network and our Terminator
architecture. 1) Our Slow-Fast Neural Encoding (SFNE) block employs a multi-
branch structure, eliminating the need for residual learning (Figure 3). 2)
Our hidden layers do not utilize pooling layers for downsampling feature
resolution. 3) We introduce a novel local feedback error for updating the slow
network.

In the past decade, significant advancements have been made in neural
networks, particularly with the emergence of ResNet [15] and Transformer [44].
However, the additive connections between layers in residual learning, while
addressing the performance degeneration, pose challenges for the exploration
of interpretable architectures [45, 33] and the adoption of greedy layerwise
training [27]. Mainstream residual networks [15, 18, 49, 41, 24] typically
rely on single-branch architectures with small convolution kernels, which
significantly diminish the feature extraction capabilities of individual
layers. To compensate information loss, previous layer outputs are added to
subsequent layers. However, increasing the size of convolution kernels leads
to an exponential growth in model parameters. Consequently, it is crucial to
address the following questions in order to eliminate the reliance on residual
learning:

How can we generate large convolution kernels with fewer parameters and build
a multi-branch network?

In the Transformer models [44, 8, 23], the self-attention weights can be
viewed as context-dependent weight matrices or neural network-programmed fast
weights [36, 35, 1]. These models employ large implicit weight matrices,
derived from key-query activations with few trainable parameters, to encode
long sequence inputs. However, the utilization of dot product-based attention
matrices introduces quadratic time and space complexity, making it challenging
to scale Transformers to inputs with large context sizes. Therefore, this
paper aims to identify a simpler and more effective method than dot product-
based attention to obtain context-varying weights through network activations.

How can we obtain context-dependent fast weights through simpler elementwise
multiplication?

This paper not only investigates methods to leverage network activations for
generating context-dependent large implicit convolution kernels but also
proposes a novel multi-branch architecture to explore the possibility of
abandoning residual learning. Specifically, we employ coordinate-based
implicit MLPs [9] with few model parameters as a slow network to generate
hyper-kernels for another fast convolutional network. The weights of slow
network is data-independent and remain fixed after training. To make the
generated hyper-kernels to be context-dependent, we propose a
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator, connecting hyper-kernels
($\mathcal{W}$) and hidden activations ($\mathcal{Z}$) through simple
elementwise multiplication, a more efficient alternative to the dot product
attention [44]. The fast network utilizes context-varying weights to perform
convolution operations on $\mathcal{Z}$. Consequently, the weights of fast
network are context-dependent and can change at test time. Based on above
design, we present a new architecture called Terminator, which integrates
context-dependent hyper-kernels of different sizes to produce multi-branch
hidden representations, enhancing the feature extraction capability of each
layer. Additionally, a bottleneck layer is employed to compress the
concatenated channels, allowing only valuable information to propagate to the
subsequent layers.

Furthermore, several novel components are proposed to build the multi-branch
architecture, including Recursive Gated Unit (RGU), hyper-channel interaction
and hyper interaction (see Figure 5). We also introduce a local feedback loss
for updating the slow network and replace the normalization layer that
discards affine parameters and momentum argument with a standardization layer.
Our architecture exhibits excellent properties such as stable zero-mean
features, faster training convergence, and fewer model parameters. Extensive
experimental results on pixel-level 1D and 2D image classification benchmarks
demonstrate the superior performance of our architecture.

Notation: Let $\mathbf{x}\in\mathbb{R}^{C\times H\times W}$ be the 2D input,
where $H$, $W$ and $C$ are height, width, and channel dimension. The
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ can be abbreviated as HyperZZW.

Figure 2: Visualization of the feature maps in each block. For the convenience
of comparison, we enlarge the output of the 2$\sim$4 blocks of ResNet-152.

##  2 Method

In this section, we first provide an overview of the key properties of our
overall architecture. Subsequently, we present the specifics of the
coordinate-based implicit MLPs served as a slow network for generating hyper-
kernels, and elucidate the role played by the
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator. Following that, we
elaborate on the process of constructing a multi-branch SFNE block using our
proposed novel components. Lastly, we introduce the slow neural loss.

Figure 3: The overall framework of our Slow-Fast Neural Encoding (SFNE) block,
which utilizes channel mixers and multi-scale hyper-kernels (e.g. $N$ is the
input size) to construct a nine-branch structure. The ovals represent slow
networks used to generate hyper-kernels, and the rectangles represent fast
networks that interact directly with the input. The
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator is formed by combining
$\odot$, $\otimes$, and $\circledast$, enabling context-dependent fast
weights. RGU and Si-GLU represent the recursive gated unit and the simplified
gated linear unit.

###  2.1 Key Properties

  * $\square$ ✗

[No Residual Learning.] Residual learning has been widely employed to address
the performance degeneration problem in very deep neural networks [15, 49, 48,
41, 24]. In this paper, we content that the crucial motivation behind the
adoption of residual connections is to compensate for the limited
representation learning capacity of individual layers, particularly in the
case of residual networks with its single-branch architecture and small
convolution kernels. To overcome these limitations, we incorporate large
implicit convolution kernels of varying sizes to construct a multi-branch
network, enabling full context interaction at each layer and obviating the
necessity for residual learning. Based on above analysis, we introduce a novel
Terminator architecture, which is composed of a stack of SFNE blocks shown in
Figure 3.

  * $\square$ ✗

[No Dot Product Attention.] The quadratic increase in computational cost,
based on the number of pixels, necessitates the division of images into
patches in order to facilitate the successful implementation of vision
transformers [8]. However, the conversion of patch embeddings into fixed-
length representations not only limits the inclusion of local context
information but also hinders the estimation of attention scores for individual
pixels. In contrast, our proposed $\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$
operator enables the interaction between generated hyper-kernels and image
pixels through a simple elementwise multiplication, facilitating the
acquisition of precise pixel-level (i.e. token-level) scores (see Figure 4).
Notably, this approach is more efficient than the attention-free transformer
[51] as it eliminates the reliance on the Sigmoid function.

  * $\square$ ✗

[No Intermediate Pooling Layer.] Intermediate pooling layers enhance the
receptive field by reducing feature map size, enabling better capture of
global information. However, our visualizations in Figure 2 demonstrate that
the intermediate pooling layers can distort the overall structure of
objects.333To obtain 2D heat map, we sum the values of all channels in the
feature map. The visualization in Figure 4 follows the same process.
Therefore, this hampers final average pooling layer to generate accurate image
descriptors. In our architecture, the utilization of large implicit
convolutional kernels enables global contextual interaction at each layer,
obviating the need for pooling layers to expand the receptive field. Moreover,
the feature maps produced by our architecture exhibit a notable continuous
denoising process (see Figure 2), which not only alleviates the model’s
learning complexity but also facilitates the development of interpretable
neural networks.

  * $\square$ ✗

[No Normalization.] Traditionally, normalization layers such as BN [19], IN
[43], LN [2], GN [47] incorporate learnable affine parameters, including
scaling factor $\gamma$ and shift factor $\beta$. Previous work [19] asserted
that these parameters restore the network’s representation power. However, our
experimental findings, as illustrated in Figure 6, reveal that their primary
function is to correct batch statistics (mean and variance) to ensure stable
activations. BN [19] and IN [43] also employ a momentum argument to reduce
volatility between batches, but it can hinder the in-context learning in each
batch. In this paper, we remove the affine and momentum parameters,
transforming normalization into a standardized operation called z-score
standardization. The new standardization can be written as:

$\hat{x}=\frac{x-\mathrm{E}[x]}{\sqrt{Var[x]+\epsilon}},$ (1)

where $\epsilon$ is a constant added to the mini-batch variance for numerical
stability. Consequently, we replace the original BN and IN layers with Batch
Standardization (BS) and Instance Standardization (IS) respectively.

  * $\square$ ✓

[Slow-Fast Networks.] In our architecture, we employ coordinate-based implicit
MLPs [9] as a slow network to generate large convolution kernels (hyper-
kernels) for a fast network. The slow network has minimal trainable parameters
(see Table 1) but can generate large hyper-kernels to achieve full context
interaction at each layer. Notably, the weights of the slow network remain
fixed after training and do not directly interact with the context [35]. In
contrast, as depicted in Figure 3, the fast network consists of two
components. The first component is the channel mixers and bottleneck layer,
which directly interact with the feature maps. The second component involves
the context-dependent fast weights generated by the
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator, enabling convolution and
dot product operations. These fast weights can adapt in real-time during
testing based on the input.

###  2.2 Generation of Hyper-kernels

To address the first question presented in the introduction section, we
generate two distinct hyper-kernels using slow networks to construct multi-
branch networks. The first one, denoted as global hyper-kernel
$\mathbf{K}_{g}\in\mathbb{R}^{1\times C\times H\times W}$, has the same size
as the input $\mathbf{x}$. The second one, referred to as local hyper-kernel
is $\mathbf{K}_{l}\in\mathbb{R}^{C\times 1\times k\times k}$ for depth-wise
convolution, has small kernel size $k$. By combining these hyper-kernels, our
architecture can extract global features while preserving the input’s fine
details. Below we elaborate on the generation process of $\mathbf{K}_{g}$.

The global hyper-kernel $\mathbf{K}_{g}\in\mathbb{R}^{1\times C\times H\times
W}$ is generated using coordinate-based implicit MLPs, which is also known as
multiplicative filter networks [9]. The generation process is described by the
following equations:

$\displaystyle\mathbf{h}^{(1)}$
$\displaystyle=\mathit{g}\left(\mathbf{c};\mathbf{\theta}^{(1)}\right)$
$\displaystyle\mathbf{h}^{(i+1)}$
$\displaystyle=\left(\mathbf{W}^{(i)}\mathbf{h}^{(i)}+b^{(i)}\right)\odot\mathit{g}\left(\mathbf{c};\mathbf{\theta}^{(i+1)}\right),i=1,...,l$
$\displaystyle\mathbf{h}^{(o)}$
$\displaystyle=\mathbf{W}^{(o)}\mathbf{h}^{(i+1)}+b^{(o)},$ (2)

where $\odot$ denotes elementwise multiplication,
$\mathbf{c}\in\mathbb{R}^{1\times 2\times H\times W}$ represents the
normalized coordinates on the $H$ and $W$ dimensions, which lie in a uniform
linear space of $[-1,1]^{H}$ and $[-1,1]^{W}$. The weight matrix for the
$i$-th layer is denoted by $\mathbf{W}^{(i)}\in\mathbb{R}^{d\times d}$, and
the hidden unit of the $i$-th layer is $\mathbf{h}^{i}\in\mathbb{R}^{1\times
d\times H\times W}$. The weight $W^{(o)}\in\mathbb{R}^{C\times d}$ in the
output layer transforms $\mathbf{h}^{i+1}$ to
$\mathbf{h}^{o}\in\mathbb{R}^{1\times C\times H\times W}$. The function
$\mathit{g}\left(\mathbf{c};\mathbf{\theta}^{(i)}\right)$ is a simple
sinusoidal filter defined as:

$\mathit{g}\left(\mathbf{c};\mathbf{\theta}^{(i)}\right)=\text{sin}(\mathbf{\mathit{w}}^{(i)}\mathbf{c}+\mathbf{\phi}^{(i)}),$
(3)

where $\theta^{(i)}={\mathbf{\mathit{w}}^{(i)}\in\mathbb{R}^{d\times
2},\mathbf{\phi}^{(i)}\in\mathbb{R}^{d}}$.

In addition, we use a simple s-renormalization444The s-renormalization is
called Weight Normalization in [34], and we use it to renormalize the hidden
output of the slow network. operation on the hidden output
$\mathbf{h}^{(i+1)}$ to stabilize training of the slow network. This operation
is defined as follows:

$\hat{\mathbf{h}}^{(i+1)}=\text{diag}(\mathbf{a})\mathbf{I}+\text{diag}(\mathbf{b})\mathbf{h}^{(i+1)}_{\text{norm}},$
(4)

where $\mathbf{a}\in\mathbb{R}^{d}$ and $\mathbf{b}\in\mathbb{R}^{d}$ are
scaling vectors learned during training. $\mathbf{h}^{(i+1)}_{\text{norm}}$
denotes the Euclidean norm of $\mathbf{h}^{(i+1)}$. We initialize $\mathbf{a}$
and $\mathbf{b}$ to 1.0 and 0.1, respectively, following the setup described
in [50].

Figure 4: Visualization of global hyper-kernels in each block. By performing
elementwise multiplication between the sample activations and the global
hyper-kernels $\mathbf{K}_{g}$, the model can effectively leverage context-
dependent hyper-kernels $\hat{\mathbf{K}}_{g}$ to obtain pixel-level scores,
especially when trained with the slow neural loss.

###  2.3 HyperZZW Operator

Our proposed $\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator enables the
generated global and local hyper-kernels to interact with the context,
resulting in context-dependent fast weights. The utilization of
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ involves two steps. First, hidden
activation $\mathcal{Z}$ and hyper-kernel $\mathcal{W}$ are multiplied
elementwise to obtain context-varying weights $\hat{\mathcal{W}}$.
Subsequently, $\hat{\mathcal{W}}$ can be used to perform dot product or
convolution operations on $\mathcal{Z}$ for feature extraction. Unlike dot
product-based attention [44], our $\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$
operator achieves context-dependent fast weights with linear time and space
complexity. The following steps outline the specific usage
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ on hyper-kernels $\mathbf{K}_{g}$
and $\mathbf{K}_{l}$.

[Global HyperZZW.] For the global hyper-kernel $\mathbf{K}_{g}$, we employ a
simple elementwise multiplication to get context-dependent
$\hat{\mathbf{K}}_{g}$:

$\hat{\mathbf{K}}_{g}=\mathbf{Z}\odot\mathbf{K}_{g},$ (5)

where $\mathbf{Z}\in\mathbb{R}^{B\times C\times H\times W}$ represents $B$
samples in a batch, and $\hat{\mathbf{K}}_{g}\in\mathbb{R}^{B\times C\times
H\times W}$ due to broadcasting $\mathbf{K}_{g}$ along the batch dimension.
The visualization results, shown in Figure 4, demonstrate the effectiveness of
$\hat{\mathbf{K}}_{g}$ in capturing pixel-level weighting scores for each
batch sample, especially when the model is trained with slow neural loss (as
described in Section 2.5). In contrast, the context-independent
$\mathbf{K}_{g}$ generated by the slow neural network appears to resemble
random noise. Subsequently, instead of using sliding window-based convolution,
we employ the dot product $\otimes$ operation with $\hat{\mathbf{K}}_{g}$ for
global feature extraction:

$\mathbf{Z}_{g}=\mathbf{Z}\otimes\hat{\mathbf{K}}_{g},$ (6)

where $\mathbf{Z}_{g}\in\mathbb{R}^{B\times C\times H\times W}$. For 1D
samples $\mathbf{Z}\in\mathbb{R}^{B\times C\times L}$ (e.g. $L$ is the
sequence length), we can utilize the following Fast Fourier Transform (FFT) to
extract global features:

$\mathbf{Z}_{g}=\text{iFFT}(\text{FFT}(\mathbf{Z})\odot\text{FFT}(\hat{\mathbf{K}}_{g})),\vspace{-3pt}$
(7)

[Local HyperZZW.] To introduce context-dependency to the local hyper-kernel
$\mathbf{K}_{l}\in\mathbb{R}^{C\times 1\times k\times k}$, we incorporate the
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator into its generation
process. The process is described by the following equations:

$\displaystyle\hat{\mathbf{Z}}^{(1)}$
$\displaystyle=\mathit{T}\left(\mathbf{Z}\right)$
$\displaystyle\mathbf{h}^{(1)}$
$\displaystyle=\mathit{g}\left(\mathbf{c};\mathbf{\theta}^{(1)}\right)$
$\displaystyle\mathbf{h}^{(i+1)}$
$\displaystyle=\left(\mathbf{W}^{(i)}\mathbf{h}^{(i)}+b^{(i)}\right)\odot\mathit{g}\left(\mathbf{c};\mathbf{\theta}^{(i+1)}\right),i=1,...,l$
$\displaystyle\mathbf{h}^{(i+1)}$
$\displaystyle=\left(\mathbf{W}_{z}^{(i)}\hat{\mathbf{Z}}^{(i)}+b_{z}^{(i)}\right)\odot\mathbf{h}^{(i+1)}$
(8) $\displaystyle\mathbf{h}^{(o)}$
$\displaystyle=\mathbf{W}^{(o)}\mathbf{h}^{(l)}+b^{(o)}\vspace{-3pt}$

where $\mathbf{c}\in\mathbb{R}^{1\times 2\times k\times k}$ represents the
normalized coordinates on the $k$ dimension, which range from $-1$ to $1$ in a
uniform linear space. The hidden unit of the $i$-th layer is
$\mathbf{h}^{i}\in\mathbb{R}^{1\times d\times k\times k}$, and the weight
$W^{(o)}\in\mathbb{R}^{C\times d}$ in the output layer transforms
$\mathbf{h}^{i+1}$ to $\mathbf{h}^{o}\in\mathbb{R}^{1\times C\times k\times
k}$. The transformation $\mathit{T}$ converts
$\mathbf{Z}\in\mathbb{R}^{B\times C\times H\times W}$ to
$\hat{\mathbf{Z}}\in\mathbb{R}^{1\times d\times k\times k}$ through average
pooling ($B\times C\times k\times k$), channel reduction ($B\times d\times
k\times k$) and batch mean ($1\times d\times k\times k$). By performing
elementwise multiplication between $\hat{\mathbf{Z}}^{(i)}$ and
$\mathbf{h}^{(i+1)}$, the context-dependent local hyper-kernels are generated.
We can then utilize $\hat{\mathbf{K}}_{l}\in\mathbb{R}^{C\times 1\times
k\times k}$ to extract local features through sliding window-based convolution
operations:

$\mathbf{Z}_{l}=\mathbf{Z}\circledast\hat{\mathbf{K}}_{l},\vspace{-3pt}$ (9)

where $\circledast$ represents depth-wise convolution.

Our visualizations in the appendix show that the global features
$\mathbf{Z}_{g}$ primarily capture the overall outline of object, providing a
rough representation. On the other hand, local features $\mathbf{Z}_{l}$ excel
at preserving finer details, particularly when smaller convolution kernels are
utilized. To address the challenge of information loss at each layer, the
combination between global and local features through channel stacking is used
to enhance the comprehensive representation of the network. As a result, the
necessity for residual connections to compensate for this loss becomes
obsolete.

###  2.4 Slow-Fast Neural Encoding Block

Our SFNE block, as illustrated in Figure 3, plays a crucial role in
simultaneously achieving full context interaction in channel and spatial
dimensions, so that the network does not lose information at each layer. The
Figure 3 presents a nine-branch architecture, composed of three branches from
global $\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$, three branches from local
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$, one branch from the Si-GLU 555The
parameters-free Si-GLU can be written as
$Si$-$GLU(\mathbf{x})=\sigma(\mathbf{x})\odot\mathbf{x}$, where $\sigma$
denotes the Sigmoid function., one branch from the middle layer, and the final
one from hyper interaction. Notably, the global branches share a common hyper-
kernel. Moreover, the nine-branch architecture is flexible. To handle higher
resolution images, we can adjust the number and kernel size of the local
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ while keeping the other modules
unchanged. To facilitate communication across channels, as the dot product and
convolution operations in $\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ are
channel-wise, we introduce three types of channel transformations: 1) Channel
Mixer is a straightforward one-layer MLP; 2) Recursive Gated Unit (RGU); 3)
Hyper-Channel Interaction. Following these transformations, instance
standardization (Mixer-IS) is applied to ensure the independence of the
channel dimension and the sample dimension. In the subsequent sections, we
provide a detailed introduction to our proposed novel components.

Recursive Gated Unit is an extension of the gated linear unit (GLU) [7] that
incorporates recursion.

$\begin{split}\mathbf{K}&=\mathbf{W}^{K}\mathbf{x},\\\
\mathbf{V}&=\mathbf{W}^{V}\mathbf{x},\\\
\mathbf{Q}&=\sigma\left(\textit{S}\left(\mathbf{K}\odot\mathbf{W}\mathbf{V}\right)\right)+b,\\\
\mathbf{Y}&=\mathbf{W}^{Y}\mathbf{Q},\end{split}$ (10)

where $\sigma$ represents the Gaussian Error Linear Unit (GeLU) nonlinearity,
and S denotes instance standardization. The outputs $\mathbf{K}$,
$\mathbf{V}$, $\mathbf{Q}$, and $\mathbf{Y}$ have the same size as the input
$\mathbf{x}$. We apply the RGU to the input $\mathbf{x}$ and context-dependent
$\hat{\mathbf{K}}_{g}$ and then use their sum as the output of a channel
mixer.

Figure 5: Diagrams illustrate our proposed hyper-channel interaction and hyper interaction mechanisms in our architecture. Block | 0 | 1 | 2 | 3 | 4 | All
---|---|---|---|---|---|---
Slow Network | 73K | 77K | 96K | 153K | 237K | 636K | 8%
1-2[1pt/1pt] Fast Network | Channel mixers | 20K | 27K | 92K | 650K | 2.7M | 3.5M | 45%
2-2[1pt/1pt] | Bottleneck layer | 816 | 19K | 186K | 1.1M | 2.4M | 3.7M | 47%
All | 93K | 123K | 374K | 1.9M | 5.3M | 7.8M |
1.2% | 1.6% | 4.8% | 24.4% | 68.0% |  | 100%
Table 1: Statistics of model parameters in each block, including slow network
and fast network. In order to better know the parameters of each module, we
list channel mixers and bottleneck layer separately from the fast network.

Hyper-Channel Interaction module integrates
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator, which can be viewed as a
more effective version of channel attention mechanisms [17, 46]. As depicted
in Figure 5(a), we begin by aggregating spatial information from a feature map
$\mathbf{x}\in\mathbb{R}^{C\times H\times W}$ using an average-pooling
operation to generate a channel descriptor
$\mathbf{z}_{c}\in\mathbb{R}^{C\times 1\times 1}$. Next, our
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator connects the channel-wise
hyper-weights $\mathbf{w}_{c}\in\mathbb{R}^{C\times 1\times 1}$ and
$\mathbf{z}_{c}$ generated from a slow network through two simple elementwise
multiplications, obtaining gating scores for the channels. Finally, the
channel weights are obtained by applying a Sigmoid function, and these weights
are broadcasted across the spatial dimension for multiplication with
$\mathbf{x}$. Through global cross-channel interaction, our hyper-channel
interaction module achieves channel mixing.

Hyper Interaction combines hyper-channel and hyper-spatial interactions into a
single module, leveraging the gating scores of the high-level output to
automatically extract features from input $\mathbf{x}$. Figure 5(b)
illustrates this process. To extract channel weights, we employ the same
mechanism as described in the hyper-channel interaction. Additionally, for
generating the spatial descriptor $\mathbf{z}_{s}\in\mathbb{R}^{1\times
H\times W}$, we aggregate the channel information of a feature map
$\mathbf{x}\in\mathbb{R}^{C\times H\times W}$ using a channel mean operation.
Our $\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator connects the spatial-
wise hyper-weights $\mathbf{w}_{s}\in\mathbb{R}^{1\times H\times W}$ with
$\mathbf{z}_{s}$ through two simple elementwise multiplications, resulting in
the spatial gating scores. Next, these scores are multiplied with the channel
scores, after broadcasting, to pass through the Sigmoid function for obtaining
the channel-spatial weights. Finally, these weights calculated from the high-
level output are utilized for pixel-level filtering on the input $\mathbf{x}$.
This integration enables the hyper-interaction module to enhance the feature
extraction capability of the SFNE block as a new branch.

Mutual Hyper-Kernel Generation (MuHKGen) module can increases the model’s
complexity and helps prevent over-fitting. For instance, the slow network
integrates $\mathbf{z}_{2}$ to produce a local hyper-kernel
$\mathbf{K}_{l}^{1}$ using Eqn. 8, and vice versa. Subsequently, the fast
networks perform convolution operations on $\mathbf{z}_{1}$ and
$\mathbf{z}_{2}$ using $\mathbf{K}_{l}^{2}$ and $\mathbf{K}_{l}^{1}$,
respectively, to extract multi-scale features.

Channel Bottleneck Layer. The SFNE block concatenate the multi-scale features
along the channel dimension to achieve a comprehensive representation of the
input. The objective of the bottleneck layer is to compress the concatenated
global and local feature maps, effectively eliminating noise present in the
feature maps. To accomplish this, we introduce a bottleneck coefficient
$\lambda$ to control the degree of channel compression, determining the
reduction in the number of concatenated channels. On the other hand, it scales
up the channel dimension of the input by a factor of $\lambda$ ($\lambda\geq
1$). Specifically, when the input is $\mathbf{x}\in\mathbb{R}^{C\times H\times
W}$, passing through the bottleneck layer results in an output
$\mathbf{y}\in\mathbb{R}^{\lambda C\times H\times W}$.

Group-based Standardization. After the channel bottleneck layer, we introduce
a Group-based Instance-Batch Standardization (G-IBS) operation, as illustrated
in Figure 3. Specifically, the output feature maps are divided into non-
overlapping groups based on channels, and Group-based Batch Standardization
(G-BS) and Group-based Instance Standardization (G-IS) are applied alternately
to each corresponding group. Our visualized results shown in Figure 6(b-c)
demonstrate that combining G-IS and G-BS can enhance the model’s expressive
power by increasing diversities of the channel variances.

Method |  #Param. |  sMNIST |  pMNIST |  sCIFAR10
---|---|---|---|---
DilRNN [5] |  44K |  98.0 |  97.2 |  -
LSTM [3] |  70K |  87.2 |  85.7 |  -
GRU [3] |  70K |  96.2 |  87.3 |  -
TCN [3] |  70K |  99.0 |  97.2 |  -
FlexTCN-6 [30] |  375K |  99.6 |  98.6 |  80.8
r-LSTM [42] |  500K |  98.4 |  95.2 |  72.2
Transformer [42] |  500K |  98.9 |  97.9 |  62.2
HiPPO [10] |  500K |  98.9 |  98.3 |  61.1
CKCNN [31] |  1M |  99.32 |  98.54 |  63.74
CCNN [32] |  2M |  99.72 |  98.84 |  93.08
LSSL [12] |  2M |  99.53 |  98.76 |  84.65
S4 [11] |  7.8M |  99.63 |  98.70 |  91.13
Hyena [29] |  7.8M |  - |  - |  91.10
TrellisNet [4] |  8M |  99.20 |  98.13 |  73.42
LRU [28] |  - |  - |  - |  89.00
Liquid-S4 [14] |  - |  - |  - |  92.02
S4D-Inv [13] |  - |  - |  - |  90.69
S5 [38] |  - |  99.65 |  98.67 |  90.10
Terminator |  1.3M |  99.72 |  99.12 |  93.21

Table 2: Pixel-level 1D image classification. The underlined result is the
second best method.

(a) Training convergence

(b) Channel mean

(c) Channel variance

Figure 6: (a) shows fast training convergence of our Terminator. (b) and (c)
visualize the batch-based channel mean and channel variance statistics of the
last hidden layer in ResNet-152 and Terminator. The experiments are
constructed on the STL10 test set.

###  2.5 Slow Neural Loss

The slow neural loss serves as a local feedback error for updating the slow
network (i.e. large hyper-kernel generator). We compute the Mean Squared Error
(MSE) of the $j$-th SFNE block as follows:

$\displaystyle\mathbb{L}_{s}^{j}$
$\displaystyle=\sum_{t=0}^{j-1}\|\mathbf{K}_{g}^{j}-\mathbb{E}(\mathbf{K}_{g}^{t})\|^{2},j=1,...,J$
(11) $\displaystyle\mathbb{L}_{s}$
$\displaystyle=\sum_{j=1}^{J}\mathbb{L}_{s}^{j},$ (12)

where $J$ is the number of SFNE blocks, and $\mathbb{E}$ represents a channel
expansion operation achieved through repeated concatenations for different
blocks ($t=0,...,j-1$). It can exploit the channel-spatial consistency of
$K_{g}$ in deep and shallow blocks to promote context-dependent
$\hat{\mathbf{K}}_{g}$ to obtain more accurate pixel-level scores (see Figure
4). The total slow neural loss $\mathbb{L}_{s}$ is obtained by summing up the
individual block losses, $\mathbb{L}_{s}^{j}$ ($j=1,...,J$). In our
experiments, we combine the $\mathbb{L}_{s}$ loss with the cross-entropy loss
to train the model end-to-end using backpropagation.

Method | #Param. | CIFAR10 | CIFAR100
---|---|---|---
InceptionV2 [40] | 65M | - | 72.49
Xception [6] | 21M | - | 74.93
VGG16 [37] | 20M | 93.91 | 74.08
ViT [8] | 6.3M | 90.92 | 66.54
ConvNeXt [24] | 6.3M | 92.20 | -
WRN-40-2 [49] | 2.2M | 94.67 | -
ResNet-110 [15] | 1.7M | 93.57 | 74.84
Hyena-ISO [29] | 202K | 91.20 | -
CKCNN [31] | 670K | 86.80 | -
FlexNet-16 [30] | 670K | 92.20 | -
CCNN† [32] | 2.0M | 94.56 | 73.58
S4ND-ISO [26] | 5.3M | 94.10 | -
Terminator | 1.3M | 95.22 | 75.38
Table 3: 2D image classification on CIFAR10 and CIFAR100 datasets. † represents the results obtained from the code released by [32]. Method | ResNet-152 [15] w/ BN [19] | ResNet-152 [15] w/ BS | Terminator | w/ Affine | w/ Momentum
---|---|---|---|---|---
Test Accuracy | 85.41 | 77.82 | 86.32 | 85.93 | 85.55
Table 4: Comparison of test accuracy of ResNet-152 and Terminator on STL10 dataset when using normalization and standardization operations. ResNet-152 performs poorly when removing the affine and momentum parameters, while our Terminator does the opposite. Method | Size | #Param. | FLOPs | Acc.
---|---|---|---|---
FlexNet-16 [30] | $96^{2}$ | 670K | - | 68.67
CCNN† [32] | $256^{2}$ | 2M | 115G | 78.01
ResNet-18 [25] | $96^{2}$ | 11.2M | 5G | 78.65
DenseNet-201 [18] | $96^{2}$ | 18.1M | 13G | 80.60
Xception [6] | $96^{2}$ | 20.8M | 10G | 84.15
ResNeXt-152 [48] | $96^{2}$ | 33.3M | 18G | 84.95
ResNet-152 [15] | $96^{2}$ | 58.2M | 34G | 85.41
Terminator | $96^{2}$ | 1.5M | 10G | 80.13
$96^{2}$ | 7.8M | 67G | 86.32
Table 5: Image classification on STL10 dataset. † represents the reproduced
results obtained from the code released by [32].

##  3 Experiments

In this section, we provide quantitative and qualitative results to
demonstrate the superior performance of our Terminator architecture. The
experimental settings, dataset introduction, architectural details, ablation
studies and more discussions are presented in the appendix.

###  3.1 Results on 1D Image Classification

The results shown in Table 2 highlight the superior performance of our
Terminator architecture on the sMNIST, pMNIST [21], and sCIFAR10 [20]
datasets. Our network achieves state-of-the-art results, surpassing temporal
convolutional networks [4], continuous convolutional networks [30, 31, 32] and
state space models [10, 12, 11, 14, 13, 38, 29] in terms of accuracy, despite
having fewer model parameters and not utilizing residual learning. One of the
key factors contributing to our model’s success is the
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator, which effectively models
long-term dependencies through context-dependent fast weights.

###  3.2 Results on 2D Image Classification

Table 3 shows the performance comparison of our Terminator with various models
on the CIFAR10 and CIFAR100 datasets. Remarkably, our Terminator model
surpasses Inception models [40, 6], residual networks [15, 49, 24], and vision
transformers [8] in terms of accuracy while utilizing fewer model parameters.
Moreover, our network outperforms continuous convolutional networks [30, 31,
32] and state space models [26, 29], establishing state-of-the-art results
without relying on residual learning.

Furthermore, the results in Table 5 demonstrate that our Terminator achieves
superior performance compared to ResNet-152 while utilizing only $1/7$ of the
model parameters. Notably, the Terminator model doubles the computation while
efficiently maintaining full context interaction at each layer without
information loss, even on large-resolution images. Consequently, the need for
residual learning is eliminated. Additionally, our
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator utilizes hidden
activations to generate context-dependent fast weights, thereby enhancing the
model’s capacity for in-context learning.

###  3.3 Analysis of Excellent Properties

Training Convergence. The visualization in Figure 6(a) demonstrates the faster
training convergence of our Terminator architecture. It achieves the same test
accuracy as ResNet-152 in approximately $1/6$ of the epochs, requiring only
$1/2$ of the training epochs. Moreover, it is noteworthy that our Terminator
achieves satisfactory results on the sMNIST dataset with just 50 training
epochs, which is only $1/4$ of the epochs required by other networks [31, 30,
32, 12, 11]. The fast convergence of our model is mainly due to the zero-mean
features introduced below, which is a well-known conclusion [22, 19].

Zero-mean Features. The results presented in Figure 6(b) and Table 4
demonstrate that our network achieves stable zero-mean features and improved
performance even without the affine and momentum parameters in standardization
layers. In contrast, training ResNet-152 with batch standardization (BS) leads
to a significant decline in performance, resulting in an accuracy of only
$77.82\%$. Additionally, the channel mean and channel variance experience
notable increases. These elevated batch statistics introduce higher
volatility, which detrimentally affects the model’s convergence and
generalization capabilities.

Model Parameters. The statistical findings in Table 1 reveal that as the model
depth increases, the number of model parameters in each block also increases,
with the last block being particularly significant, accounting for 68% of the
total model parameters. This growth in parameters is primarily attributed to
the escalating number of channels in the feature maps. Notably, the slow
network has a mere 8% of the model parameters. We also would like to highlight
that, as demonstrated in Figure 2, our Terminator network no longer need to be
very deep, because it can achieve effective image denoising by utilizing only
a few SFNE blocks.

##  4 Conclusion

This paper introduces the Terminator architecture, which offers a novel
approach to network design by abandoning residual learning and leveraging
large implicit convolution kernels. The SFNE block provides a new perspective
for building multi-branch networks, while the
$\mathrm{Hyper}\mathcal{Z{\cdot}Z{\cdot}W}$ operator enables dynamic encoding
by providing context-dependent weights for the fast networks through
elementwise multiplication between hyper-kernels and hidden activations. Our
network brings several advantages, including faster training convergence,
stable zero-mean features, and fewer model parameters. The experimental
results demonstrate its superiority in capturing long-range dependencies and
achieving state-of-the-art performance on pixel-level 1D and 2D image
classification benchmarks. Due to limited computing resources, we were unable
to conduct experiments on ImageNet dataset. However, in future research, we
plan to prioritize the exploration of more effective slow neural loss to
improve the accuracy of pixel-level scores.

## References

  * Ba et al. [2016a] Jimmy Ba, Geoffrey E. Hinton, Volodymyr Mnih, Joel Z. Leibo, and Catalin Ionescu.  Using fast weights to attend to the recent past.  In _Neural Information Processing Systems_ , 2016a.
  * Ba et al. [2016b] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton.  Layer normalization.  _arXiv preprint arXiv:1607.06450_ , 2016b.
  * Bai et al. [2018a] Shaojie Bai, J Zico Kolter, and Vladlen Koltun.  An empirical evaluation of generic convolutional and recurrent networks for sequence modeling.  _arXiv preprint arXiv:1803.01271_ , 2018a.
  * Bai et al. [2018b] Shaojie Bai, J Zico Kolter, and Vladlen Koltun.  Trellis networks for sequence modeling.  _arXiv preprint arXiv:1810.06682_ , 2018b.
  * Chang et al. [2017] Shiyu Chang, Yang Zhang, Wei Han, Mo Yu, Xiaoxiao Guo, Wei Tan, Xiaodong Cui, Michael Witbrock, Mark A Hasegawa-Johnson, and Thomas S Huang.  Dilated recurrent neural networks.  _Advances in neural information processing systems_ , 30, 2017.
  * Chollet [2017] François Chollet.  Xception: Deep learning with depthwise separable convolutions.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1251–1258, 2017.
  * Dauphin et al. [2017] Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier.  Language modeling with gated convolutional networks.  In _International conference on machine learning_ , pages 933–941. PMLR, 2017.
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Fathony et al. [2020] Rizal Fathony, Anit Kumar Sahu, Devin Willmott, and J Zico Kolter.  Multiplicative filter networks.  In _International Conference on Learning Representations_ , 2020.
  * Gu et al. [2020] Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré.  Hippo: Recurrent memory with optimal polynomial projections.  _Advances in neural information processing systems_ , 33:1474–1487, 2020.
  * Gu et al. [2021a] Albert Gu, Karan Goel, and Christopher Ré.  Efficiently modeling long sequences with structured state spaces.  _arXiv preprint arXiv:2111.00396_ , 2021a.
  * Gu et al. [2021b] Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré.  Combining recurrent, convolutional, and continuous-time models with linear state space layers.  _Advances in neural information processing systems_ , 34:572–585, 2021b.
  * Gu et al. [2022] Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré.  On the parameterization and initialization of diagonal state space models.  _Advances in Neural Information Processing Systems_ , 35:35971–35983, 2022.
  * Hasani et al. [2022] Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang, Makram Chahine, Alexander Amini, and Daniela Rus.  Liquid structural state-space models.  _arXiv preprint arXiv:2209.12951_ , 2022.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Hinton and Plaut [1987] Geoffrey E Hinton and David C Plaut.  Using fast weights to deblur old memories.  In _Proceedings of the ninth annual conference of the Cognitive Science Society_ , pages 177–186, 1987.
  * Hu et al. [2018] Jie Hu, Li Shen, and Gang Sun.  Squeeze-and-excitation networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 7132–7141, 2018.
  * Huang et al. [2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 4700–4708, 2017.
  * Ioffe and Szegedy [2015] Sergey Ioffe and Christian Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In _International conference on machine learning_ , pages 448–456. pmlr, 2015.
  * Krizhevsky et al. [2009] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * LeCun [1998] Yann LeCun.  The mnist database of handwritten digits.  _http://yann. lecun. com/exdb/mnist/_ , 1998.
  * LeCun et al. [2002] Yann LeCun, Léon Bottou, Genevieve B Orr, and Klaus-Robert Müller.  Efficient backprop.  In _Neural networks: Tricks of the trade_ , pages 9–50. Springer, 2002.
  * Liu et al. [2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 10012–10022, 2021.
  * Liu et al. [2022] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 11976–11986, 2022.
  * Luo et al. [2020] Chunjie Luo, Jianfeng Zhan, Lei Wang, and Wanling Gao.  Extended batch normalization.  _arXiv preprint arXiv:2003.05569_ , 2020.
  * Nguyen et al. [2022] Eric Nguyen, Karan Goel, Albert Gu, Gordon Downs, Preey Shah, Tri Dao, Stephen Baccus, and Christopher Ré.  S4nd: Modeling images and videos as multidimensional signals with state spaces.  _Advances in neural information processing systems_ , 35:2846–2861, 2022.
  * Nøkland and Eidnes [2019] Arild Nøkland and Lars Hiller Eidnes.  Training neural networks with local error signals.  In _International conference on machine learning_ , pages 4839–4850. PMLR, 2019.
  * Orvieto et al. [2023] Antonio Orvieto, Samuel L Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De.  Resurrecting recurrent neural networks for long sequences.  _arXiv preprint arXiv:2303.06349_ , 2023.
  * Poli et al. [2023] Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Stefano Ermon, and Christopher Ré.  Hyena hierarchy: Towards larger convolutional language models.  _arXiv preprint arXiv:2302.10866_ , 2023.
  * Romero et al. [2021a] David W Romero, Robert-Jan Bruintjes, Jakub M Tomczak, Erik J Bekkers, Mark Hoogendoorn, and Jan C van Gemert.  Flexconv: Continuous kernel convolutions with differentiable kernel sizes.  _arXiv preprint arXiv:2110.08059_ , 2021a.
  * Romero et al. [2021b] David W Romero, Anna Kuzina, Erik J Bekkers, Jakub M Tomczak, and Mark Hoogendoorn.  Ckconv: Continuous kernel convolution for sequential data.  _arXiv preprint arXiv:2102.02611_ , 2021b.
  * Romero et al. [2022] David W Romero, David M Knigge, Albert Gu, Erik J Bekkers, Efstratios Gavves, Jakub M Tomczak, and Mark Hoogendoorn.  Towards a general purpose cnn for long range dependencies in $n$ d.  2022\.
  * Rudin et al. [2022] Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong.  Interpretable machine learning: Fundamental principles and 10 grand challenges.  _Statistic Surveys_ , 16:1–85, 2022.
  * Salimans and Kingma [2016] Tim Salimans and Durk P Kingma.  Weight normalization: A simple reparameterization to accelerate training of deep neural networks.  _Advances in neural information processing systems_ , 29, 2016.
  * Schlag et al. [2021] Imanol Schlag, Kazuki Irie, and Jürgen Schmidhuber.  Linear transformers are secretly fast weight programmers.  In _International Conference on Machine Learning_ , pages 9355–9366. PMLR, 2021.
  * Schmidhuber [1992] Jürgen Schmidhuber.  Learning to control fast-weight memories: An alternative to dynamic recurrent networks.  _Neural Computation_ , 4(1):131–139, 1992.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ , 2014.
  * Smith et al. [2022] Jimmy TH Smith, Andrew Warrington, and Scott W Linderman.  Simplified state space layers for sequence modeling.  _arXiv preprint arXiv:2208.04933_ , 2022.
  * Stanley et al. [2009] Kenneth O Stanley, David B D’Ambrosio, and Jason Gauci.  A hypercube-based encoding for evolving large-scale neural networks.  _Artificial life_ , 15(2):185–212, 2009.
  * Szegedy et al. [2016] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2818–2826, 2016.
  * Tan and Le [2019] Mingxing Tan and Quoc Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In _International conference on machine learning_ , pages 6105–6114. PMLR, 2019.
  * Trinh et al. [2018] Trieu Trinh, Andrew Dai, Thang Luong, and Quoc Le.  Learning longer-term dependencies in rnns with auxiliary losses.  In _International Conference on Machine Learning_ , pages 4965–4974. PMLR, 2018.
  * Ulyanov et al. [2016] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky.  Instance normalization: The missing ingredient for fast stylization.  _arXiv preprint arXiv:1607.08022_ , 2016.
  * Vaswani et al. [2017] Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In _Neural Information Processing Systems_ , 2017.
  * Veit et al. [2016] Andreas Veit, Michael J Wilber, and Serge Belongie.  Residual networks behave like ensembles of relatively shallow networks.  _Advances in neural information processing systems_ , 29, 2016.
  * Wang et al. [2020] Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu.  Eca-net: Efficient channel attention for deep convolutional neural networks.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 11534–11542, 2020.
  * Wu and He [2018] Yuxin Wu and Kaiming He.  Group normalization.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 3–19, 2018.
  * Xie et al. [2017] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He.  Aggregated residual transformations for deep neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1492–1500, 2017.
  * Zagoruyko and Komodakis [2016] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks.  _arXiv preprint arXiv:1605.07146_ , 2016.
  * Zagoruyko and Komodakis [2017] Sergey Zagoruyko and Nikos Komodakis.  Diracnets: Training very deep neural networks without skip-connections.  _arXiv preprint arXiv:1706.00388_ , 2017.
  * Zhai et al. [2021] Shuangfei Zhai, Walter Talbott, Nitish Srivastava, Chen Huang, Hanlin Goh, Ruixiang Zhang, and Josh Susskind.  An attention free transformer.  _arXiv preprint arXiv:2105.14103_ , 2021.


# Conductance transition with interacting bosons in an Aharonov-Bohm cage

A. R. Kolovsky  Kirensky Institute of Physics, Federal Research Center KSC SB
RAS, 660036, Krasnoyarsk, Russia  School of Engineering Physics and Radio
Electronics, Siberian Federal University, 660041, Krasnoyarsk, Russia  Center
for Theoretical Physics of Complex Systems, Institute for Basic Science, 34126
Daejeon, Republic of Korea  P. S. Muraev  Kirensky Institute of Physics,
Federal Research Center KSC SB RAS, 660036, Krasnoyarsk, Russia  School of
Engineering Physics and Radio Electronics, Siberian Federal University,
660041, Krasnoyarsk, Russia  IRC SQC, Siberian Federal University, 660041,
Krasnoyarsk, Russia  S. Flach  Center for Theoretical Physics of Complex
Systems, Institute for Basic Science, 34126 Daejeon, Republic of Korea

###### Abstract

We study transport of interacting bosons through an Aharonov-Bohm cage - a
building block of flat band networks - with coherent pump and sink leads. In
the absence of interactions the cage is insulating due to destructive
interference. We find that the cage stays insulating up to a critical value of
the pump strength in the presence of mean field interactions, while the
quantum regime induces particle pair transport and weak conductance below the
critical pump strength. A swift crossover from quantum into the classical
regime upon further pump strength increase is observed. We solve the time
dependent master equations for the density matrix of the many body problem
both in the classical, pure quantum, and pseudoclassical regimes. We start
with an empty cage and switch on driving. We characterize the transient
dynamics, and the complexity of the resulting steady states and attractors.
Our results can be readily realized using experimental platforms involving
interacting ultracold atoms and photons on finetuned optical lattices.

Introduction. Flatbands arise in the band structure of finetuned tigh-binding
networks and are used in various setups in condensed matter physics and
photonics [1, 2, 3, 4]. Particular interest has been paid to the case when all
bands are flat - ABF lattices, as originally observed for particular magnetic
flux values threading the lattice, and therefore coined Aharonov-Bohm cages
(AB) due to the complete destructive interference induced trapping of
noninteracting particles [5, 6]. ABF lattices can be in general diagonalized
with a finite number of local unitary transformations, support compact
localized eigenstates (CLS), and serve as the starting ground for a variety of
single particle and many-body perturbations which lead to different
nonperturbative phases of matter [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
18].

A famous example of such an ABF lattice is the $\pi$-flux rhombic lattice [6]
(also known as the diamond lattice) which is shown in Fig. 1. Here the term
”$\pi$-flux” means that the sum of phases of the nearest-neighbour hopping
matrix elements equals to $\pi$. All eigenstates of the quantum particle in
this lattice are compact localized states which prohibit any transport across
the lattice. This statement, however, is valid only for non-interacting
particles. Finite inter-particle interaction will recover transport in general
[6, 13, 19], though additional single particle and interaction finetuning can
prohibit transport even in the interacting case [12, 13, 11, 20].

In the present work we address the effect of inter-particle interaction on the
transport of Bose particles across the diamond lattice from the viewpoint of
laboratory experiments where one injects bosons into the first site of the
lattice by using an external coherent driving pump and withdraws them from the
last site with a sink. Nowadays such experiments can be performed by using
different physical platforms, for example, super-conducting circuits [21, 22,
23]. In short, one arranges interacting transmons (micro-resonators coupled to
Josephson junctions) in a lattice, drives the first transmon with a microwave
field, and reads the signal off from the last transmon. The crucial impact of
Josephson junctions is that they introduce an effective inter-particle
interaction for photons in the micro-resonators. Hence, that setup can be
modeled by the Bose-Hubbard Hamiltonian. Another promising system are photonic
crystals [24]. In a recent laboratory experiment [25] the authors realized the
$\pi$-flux rhombic lattice and proved the absence of transport for non-
interacting photons.

Figure 1: The flux rhombic lattice (diamond chain). The all bands flat case
corresponds to $\Phi=\pi$.

Model. The above considered laboratory setup is described by the following
master equation for the reduced density matrix $\widehat{{\cal R}}(t)$ of
microwave photons,

$\frac{\partial\widehat{{\cal R}}}{\partial t}=-\frac{i}{\hbar}[\widehat{{\cal
H}},\widehat{{\cal
R}}]-\frac{\gamma}{2}\left(\hat{a}_{L}^{\dagger}\hat{a}_{L}\widehat{\cal
R}-2\hat{a}_{L}\widehat{\cal R}\hat{a}_{L}^{\dagger}+\widehat{\cal
R}\hat{a}_{L}^{\dagger}\hat{a}_{L}\right)\;,$ (1)

where $\gamma$ is the rate of photon absorption by a measurement device and
the Hamiltonian $\widehat{{\cal H}}$ has the form

$\displaystyle\widehat{{\cal
H}}=\hbar\Delta\sum_{\ell=1}^{L}\hat{n}_{\ell}-\sum_{\ell,\ell^{\prime}=1}^{L}\frac{\hbar
J_{\ell,\ell^{\prime}}}{2}\left(\hat{a}_{\ell}^{\dagger}\hat{a}_{\ell^{\prime}}+{\rm
h.c.}\right)$ (2)
$\displaystyle+\frac{\hbar^{2}g}{2}\sum_{\ell=1}^{L}\hat{n}_{\ell}(\hat{n}_{\ell}-1)+\frac{\sqrt{\hbar}\Omega}{2}(\hat{a}_{1}^{\dagger}+\hat{a}_{1})\;.$

In Eqs. (1)-(2) $\hat{a}_{\ell}$ and $\hat{a}_{\ell}^{\dagger}$ are the
standard annihilation and creation operators which commute to unity,
$\hat{n}_{\ell}=\hat{a}_{\ell}^{\dagger}\hat{a}_{\ell}$ is the particle number
operator, $\Omega$ is the Rabi frequency (which is proportional to the
amplitude of the driving field), $\Delta$ the detuning of the driving
frequency from the linear frequency of quantum oscillators, $g$ the
macroscopic interaction constant (which determines the nonlinearity $U=\hbar
g$ of the energy spectrum of quantum oscillators), $J_{\ell,\ell^{\prime}}$
are the hopping matrix elements (i.e., the couplings between oscillators), $L$
is the total number of sites, and $\hbar$ is the dimensionless Planck constant
111Notice, that from the viewpoint of a laboratory experiment the classical
limit $\hbar\rightarrow 0$ corresponds to scaling the nonlinearity of the
oscillator energy spectrum as $U=\hbar g$, and the driving strength as
$\Omega\rightarrow\Omega/\sqrt{\hbar}$.. We label the rhomb sites as shown in
Fig. 1 where $C_{2}\equiv D_{1}$, $C_{3}\equiv D_{2}$, etc., and we shall use
the gauge where $J_{AD}=J_{BD}=J$ and $J_{CA}=-J_{CB}=iJ$. In what follows we
set $J=1$ which implies that all system parameters are measured in units of
the hopping frequency $J$, and time in units of its inverse $1/J$. The case of
non-interacting particles results in destructive interference on the first
$D$-site of the diamond lattice, which blocks all transport. Thus, to
understand the effect of finite inter-particle interactions it suffices to
consider the lattice consisting of a single rhomb.

Mean field approach. First we analyze the classical (mean-field) problem by
using the Gross-Pitaevskii equations on the rhomb lattice, i.e. by replacing
annihilation and creation operators by c-numbers in the presence of damping
and driving[27]. For the lattice consisting of a single rhomb we have

$\displaystyle
i\dot{C}=\left(\Delta+g|C|^{2}\right)C-\frac{i}{2}A+\frac{i}{2}B+\frac{\Omega}{2}$
(3) $\displaystyle
i\dot{A}=\left(\Delta+g|A|^{2}\right)A+\frac{i}{2}C-\frac{1}{2}D$
$\displaystyle
i\dot{B}=\left(\Delta+g|B|^{2}\right)B-\frac{i}{2}C-\frac{1}{2}D$
$\displaystyle
i\dot{D}=\left(\Delta+g|D|^{2}\right)D-\frac{1}{2}A-\frac{1}{2}B-i\frac{\gamma}{2}D\;,$

where $C,A,B,D$ are now time-dependent complex amplitudes of the local
oscillators at the rhomb nodes and $\dot{X}\equiv dX/dt$. We use the Rabi
frequency $\Omega$ as our control parameter and fix $g=0.5$, $\gamma=0.2$ and
$\Delta=-0.5$. The above equations are invariant under a reflection symmetry
$A\rightarrow-B$, $B\rightarrow-A$, with $D=0$. To enforce $D=0$ we need
$A(t)=-B(t)$. Such states are the generalization of a flatband CLS which
persist due to the destructive interference of waves from $A$ and $B$ reaching
site $D$ [20].

Due to the presence of dissipation (more precisely, contraction of the phase
space volume), the long-time dynamics of system (3) is determined by
attractors. Attractors can be stationary (lhs of (3) vanishes) or non-
stationary. Stationary attractors can be obtained from Eq. (3) by setting its
left-hand-side to zero and solving the remaining nonlinear algebraic equations
for the amplitudes $A,B,C,D$. In addition attractors can be symmetric (they
respect the above reflection symmetry) or asymmetric. Asymmetric attractors
have $D(t\rightarrow\infty)\neq 0$ and can well be stationary. The current
$\bar{j}=\gamma|D|^{2}$ across the rhomb is nonzero for asymmetric attractors.
Symmetric attractors correspond to $D(t\rightarrow\infty)=0$ and can not be
stationary. The current $\bar{j}=\gamma|D|^{2}$ across the rhomb vanishes for
symmetric attractors. The time evolution of $C(t)$, $A(t)$, and $B(t)=-A(t)$
amplitudes of symmetric attractors turns periodic or even quasi-periodic
222Attractors of this type were discussed earlier in Refs. [32, 33] with
respect to exciton-polariton dynamics in two coupled micro-cavities. If we
associate the $A$ and $B$ sites of our system with these two cavities then, by
driving the $C$-site, we pump the antibonding mode of the $A-B$ system but it
is the bonding mode which is subject to decay.. For $\Omega<\Omega_{cr}\approx
0.9$ we found two symmetric attractors s1 and s2, with s1 smoothly tuned into
an empty state $A=B=C=0$ for $\Omega\rightarrow 0$ and s2 keeping finite
constant amplitudes $A,B,C$ in that limit. The attractor s1 turns unstable for
$\Omega>\Omega_{cr}$, while attractor s2 stays stable. An asymmetric
stationary conducting attractor a1 exists for $\Omega>\Omega_{cr}$.

Figure 2:  Relaxation of (3) to various attractors with initial condition
$|X|\equiv|A|=|B|=|C|=|D|=0.001$ and random initial phases. The plots show
$|C(t)|^{2}$ (blue, top line) and $|D(t)|^{2}$ (purple, bottom line). (a)
Relaxation to the symmetric nonstationary attractor s1 for $\Omega=0.85$. (b)
Relaxation to the asymmetric stationary attractor a1 for $\Omega=1.5$. (c)
Relaxation to the symmetric nonstationary attractor s2 for $\Omega=1.5$ with a
tiny change in the initial phase values as compared to case (b). For both (b)
and (c) the amplitude $D(t)$ grows exponentially in the interval $0<t<10$, all
amplitudes show chaotic dynamics during $10<t<20$, and the asymptotic regime
is obtained for $t\gg 20$. Parameters are $\Delta=-0.5$, $g=0.5$,
$\gamma=0.2$.

Conductance Transition. Let us analyze the transport across the rhomb as the
function of the Rabi frequency $\Omega$. For small $\Omega$ the steady-state
response of the system to the external driving is the attractor s1 featuring
small quasi-periodic oscillations with $B(t)=-A(t)$ and $D(t)=0$ (see
Fig.2(a)). However, if we increase $\Omega$ above the critical value
$\Omega_{cr}$ this quasi-periodic trajectory becomes unstable and any tiny
perturbation (the numerical round-error suffices) leads to an exponential
growth of the $D$-amplitude. The regime of exponential instability is followed
by some transient regime of chaotic dynamics, where all amplitudes show
irregular oscillations, see Fig. 2(b,c). The transient chaotic regime is then
relaxing into a steady-state regime a1 (Fig.2(b)) or s2 (Fig.2(c)). The exact
attractor choice depends on the fine details of the transient state and is
again affected by tiny details. Using an ensemble of initial conditions with
the absolute values of the complex amplitudes equal to 0.001 and random
phases, we calculate the mean values of the squared amplitudes and plot them
in Fig. 3(a). One clearly identifies the critical driving magnitude
$\Omega_{cr}\approx 0.9$ and this value coincides with the transition from
regular to chaotic dynamics in the Hamiltonian counterpart ($\gamma=0$) of the
considered system. Let us also mention that from the viewpoint of a laboratory
experiment the crossing of the chaos border is similar to some phase
transition where the system is conducting for $\Omega>\Omega_{cr}$ and
insulating for $\Omega<\Omega_{cr}$.

Quantum. We proceed with the quantum dynamics. In the numerical simulations we
evolve the density matrix $\widehat{{\cal R}}(t)$ for initial conditions
corresponding to the empty system and calculate the single particle density
matrix $\hat{\rho}(t)$

$\rho_{\ell,m}(t)={\rm Tr}[\hat{a}^{\dagger}_{\ell}\hat{a}_{m}\widehat{{\cal
R}}(t)]\;.$ (4)

The diagonal elements of this matrix give the population of the rhomb sites
while the off-diagonal elements determine the current between the rhomb sites.
We also mention the density matrix symmetries $\rho_{A,A}(t)=\rho_{B,B}(t)$,
$\rho_{C,B}=-\rho_{C,A}$, and $\rho_{A,B}<0$, which follow from the $\pi$-flux
symmetry of the quantum Hamiltonian.

Figure 3:  Left column: (a)The mean squared amplitudes of the classical
oscillators in the stationary regime versus $\Omega$, averaged over 1008
random initial phase conditions with $|X|=0.001$. (c) The mean populations of
the rhomb sites for $\hbar=2$ in the quantum case versus $\Omega$. Right
column: The mean squared population of the rhomb sites for the pseudoclassical
approach, (b) $\hbar=0.1$ and (d) $\hbar=2$. For all cases $|C|^{2}$ \- blue
curves (top), $|A|^{2}$ and $|B|^{2}$ \- yellow curves (middle), $|D|^{2}$ \-
purple curves (bottom). The other parameters are $\Delta=-0.5$, $g=0.5$, and
$\gamma=0.2$. The number of different realizations of the stochastic force in
the pseudoclassical approach is 1008.

Evolving the system for long enough time, we find the stationary population of
the rhomb sites, see Fig. 3(c). Note the reasonable agreement between the
quantum and classical results for $\Omega>\Omega_{cr}$ and the strong
discrepancy for a small $\Omega$. Unlike the classical system, the quantum
system remains conducting for $\Omega<\Omega_{cr}$.

Figure 4: Transition diagram for two particles $N=2$. The rhombs present the
ten different Fock states which are labeled by the index $j=1,\dots,10$ in the
center of each rhomb. The numbers of particles on the individual sites of a
rhomb in a given Fock state are indicated in the blue and red circles.The
dashed red and black arrows correspond to matrix elements $J=\pm i$ and the
green arrows to $J=1$. If $g=0$ the red-colored Fock states cannot be
populated because of complete destructive interference between different paths
connecting these states with the initial state $j=1$.

To explain the absence of a sharp conductance transition at a critical
$\Omega_{cr}$ in the quantum case we will consider the system dynamics in Fock
space. Let us consider for the moment $\gamma=0$ and $\Omega=0$, and an
initial state of the system with a non-zero population of the $C$-site only.
The Hamiltonian dynamics will propagate this state into other Fock states.
Fig. 4 shows the transition diagram for two particles $N=2$ where different
arrows mean different matrix elements $J$. For the noninteracting case $g=0$
the $\pi$-flux symmetry of the Hamiltonian induces destructive interference
for the Fock states with non-zero occupation of the $D$-site, preventing our
initial Fock state from leaking particles into the $D$-site. This holds true
for larger $N$ as well 333This follows from the fact that for $g=0$ the
single-particle density matrix of $N$ identical particles coincides with the
density matrix of a single particle.. Nonzero interactions $g\neq 0$ will
detune and remove the destructive interference conditions. In the considered
case $N=2$ these interactions change the equation of motion for the occupation
amplitudes of the 5th and the 6th Fock states, which leads to a population of
the symmetry protected Fock states. We mention, in passing, that for $N=2$ the
discussed interaction-induced destruction of Aharonov-Borm caging was
confirmed experimentally in the recent experiment [23] with super-conducting
circuits.

Pseudoclassical. The above presented results prove the quantum system to be
always conducting as soon as $g\neq 0$. Thus, the insulator-to-conductor phase
transition is a particular feature of the classical system. However, since the
transition from the quantum to classical realms is continuous in $\hbar$, one
can find a signature of this classical phase transition in the quantum
dynamics if $\hbar\ll 1$. To quantitatively address this problem we resort to
the pseudo-classical approximation. The pseudo-classical approximation
substitutes the master equation for the reduced density matrix by the Fokker-
Planck equation for the classical distribution function $f=f({\bf a},{\bf
a}^{*},t)$ where $a_{\ell}$ and $a_{\ell}^{*}$ are the pairs of canonically
conjugated variables ($\ell=1,\ldots,L$),

$\frac{\partial f}{\partial
t}=\\{H,f\\}+\frac{\gamma}{2}\left(\frac{\partial(a_{L}f)}{\partial
a_{L}}+{\rm h.c.}\right)+\frac{\gamma\hbar}{2}\frac{\partial^{2}f}{\partial
a_{L}\partial a_{L}^{*}}\;,$ (5)

where $\\{\ldots,\ldots\\}$ denotes the Poisson brackets and $H$ is the
classical counterpart of the Bose-Hubbard Hamiltonian (2). The first term in
the right-hand-side of this equation corresponds to the Hamiltonian evolution
of the system, the next term describes the contraction of the phase space
volume, and the last term is the quantum correction to the classical Fokker-
Planck equation [30]. This diffusion term is of special importance because it
restricts contributions of the other quantum corrections to a value of the
order of $\hbar^{2}$ [31]. Unfolding the Fokker-Planck equation (5) into the
Langevin equation, one arrives to Eq. (3) with an additional stochastic term
$\sqrt{\hbar\gamma/2}\xi(t)$. Here $\xi(t)$ is the $\delta$-correlated complex
white noise. Then the elements of the single-particle density matrix are found
by averaging the solution of the Langevin equation over different realizations
of $\xi(t)$, for example, $\rho_{A,B}(t)=\langle A^{*}(t)B(t)\rangle$. The
results of the pseudoclassical approach are shown in the right column in Fig.
3. It is seen that this approach fairly reproduces the exact quantum result
for $\hbar=2$ and indicates the convergence toward the classical result for
smaller $\hbar$.

Conclusion. To summarize, we analyzed the transport of interacting Bose
particles across the rhombic lattice (diamond chain) by employing the
classical (mean-field), pseudoclassical, and quantum approaches. Within the
classical approach the considered problem reduces to the dynamics of coupled
nonlinear oscillators, where the first oscillator in the lattice is driven by
an external field and the last oscillator is subject to friction. Then the
system is insulating if the stationary amplitude of the last oscillator is
strictly zero and conducting if it is finite. Using the driving strength
$\Omega$ as the control parameter we found the system to be insulating up to a
critical $\Omega_{cr}$. When exceeding this critical value, the system becomes
unstable, and the transient dynamics of the coupled oscillators turns chaotic.
The final state of the system is either conducting or insulating with
probability $P$ and $1-P$, respectively. We found that the discussed
probability crucially depends on the system parameters, in particular, on the
dissipation constant $\gamma$. In brief, for fixed $\Omega>\Omega_{cr}$ this
probability monotonically decreases from unity to zero in the interval
$0<\gamma<0.6$. Thus there is no conducting state for $\gamma>0.6$ (at least,
in the considered interval of $\Omega$). We also studied the many-rhomb system
where, to be closer to laboratory experiments, we included small dissipation
$\tilde{\gamma}=0.01$ to all system sites. Importantly, the classification of
attractors into symmetric (insulating) and asymmetric (conducting) remains
valid. However, for $M>1$ one finds a zoo of possible attractor combinations.
For example, for $M=3$ there are situations where the first and second rhombs
relax to asymmetric attractors but the third rhomb to the symmetric attractor.
Yet, the common feature with the single-rhomb system is that the classical
limit shows a sharp transition from an insulating to a conducting state at
some $\Omega_{cr}$.

Within the quantum approach we numerically solved the master equation for the
density matrix of interacting bosons. We found the stationary populations of
the lattice sites as a function of the Rabi frequency $\Omega$ and compared
them with the classical prediction. While there is qualitative agreement with
the classical results, one striking difference is that the quantum system
conducts even below the threshold $\Omega<\Omega_{cr}$, and the sharp
transition from the classical case turns into a smooth crossover for the
quantum case. The reason for that is that the destructive interference is
destroyed for interacting particles in the quantum case. However, we showed
that the crossover sharpens back to a classical transition by using a
pseudoclassical method upon increasing the number of particles, or decreasing
the effective Planck constant $\hbar$.

Acknowledgements. We thank A. Andreanov for useful discussions. This work was
supported by the Institute for Basic Science, Project Code (Project No.
IBS-R024-D1). PSM acknowledges financial support of the Ministry of High
Education and Science of the Russian Federation through grant FSRZ-2023-0006.

## References

  * Bergholtz and Liu [2013] E. J. Bergholtz and Z. Liu, Topological flat band models and fractional chern insulators, Int. J. Mod. Phys. B 27, 1330017 (2013).
  * Derzhko  _et al._ [2015] O. Derzhko, J. Richter, and M. Maksymenko, Strongly correlated flat-band systems: The route from heisenberg spins to hubbard electrons, Int. J. Mod. Phys. B 29, 1530007 (2015).
  * Leykam  _et al._ [2018] D. Leykam, A. Andreanov, and S. Flach, Artificial flat band systems: from lattice models to experiments, Advances in Physics: X 3, 1473052 (2018).
  * Leykam and Flach [2018] D. Leykam and S. Flach, Perspective: Photonic flatbands, APL Photonics 3, 070901 (2018).
  * Vidal  _et al._ [1998] J. Vidal, R. Mosseri, and B. Douçot, Aharonov-bohm cages in two-dimensional structures, Phys. Rev. Lett. 81, 5888 (1998).
  * Vidal  _et al._ [2000] J. Vidal, B. Douçot, R. Mosseri, and P. Butaud, Interaction induced delocalization for two particles in a periodic potential, Phys. Rev. Lett. 85, 3906 (2000).
  * Tovmasyan  _et al._ [2013] M. Tovmasyan, E. P. van Nieuwenburg, and S. D. Huber, Geometry-induced pair condensation, Phys. Rev. B 88, 220510 (2013).
  * Tovmasyan  _et al._ [2018] M. Tovmasyan, S. Peotta, L. Liang, P. Törmä, and S. D. Huber, Preformed pairs in flat Bloch bands, Phys. Rev. B 98, 134513 (2018).
  * Kuno  _et al._ [2020a] Y. Kuno, T. Orito, and I. Ichinose, Flat-band many-body localization and ergodicity breaking in the Creutz ladder, New J. Phys. 22, 013032 (2020a).
  * Kuno  _et al._ [2020b] Y. Kuno, T. Mizoguchi, and Y. Hatsugai, Interaction-induced doublons and embedded topological subspace in a complete flat-band system, Phys. Rev. A 102, 063325 (2020b).
  * Danieli  _et al._ [2020] C. Danieli, A. Andreanov, and S. Flach, Many-body flatband localization, Phys. Rev. B 102, 041116 (2020).
  * Danieli  _et al._ [2021a] C. Danieli, A. Andreanov, T. Mithun, and S. Flach, Nonlinear caging in all-bands-flat lattices, Phys. Rev. B 104, 085131 (2021a).
  * Danieli  _et al._ [2021b] C. Danieli, A. Andreanov, T. Mithun, and S. Flach, Quantum caging in interacting many-body all-bands-flat lattices, Phys. Rev. B 104, 085132 (2021b).
  * Vakulchyk  _et al._ [2021] I. Vakulchyk, C. Danieli, A. Andreanov, and S. Flach, Heat percolation in many-body flat-band localizing systems, Phys. Rev. B 104, 144207 (2021).
  * Orito  _et al._ [2021] T. Orito, Y. Kuno, and I. Ichinose, Nonthermalized dynamics of flat-band many-body localization, Phys. Rev. B 103, L060301 (2021).
  * Čadež  _et al._ [2021] T. Čadež, Y. Kim, A. Andreanov, and S. Flach, Metal-insulator transition in infinitesimally weakly disordered flat bands, Phys. Rev. B 104, L180201 (2021).
  * Kim  _et al._ [2022] Y. Kim, T. Čadež, A. Andreanov, and S. Flach, Flat band induced metal-insulator transitions for weak magnetic flux and spin-orbit disorder, arXiv preprint arXiv:2211.09410 (2022).
  * Lee  _et al._ [2023] S. Lee, A. Andreanov, and S. Flach, Critical-to-insulator transitions and fractality edges in perturbed flat bands, Physical Review B 107, 014204 (2023).
  * Muraev and Kolovsky [2021] P. S. Muraev and A. R. Kolovsky, Quantum transport in a one-dimensional flux rhombic lattice, Quantum Electronics 51, 502 (2021).
  * Danieli  _et al._ [2018] C. Danieli, A. Maluckov, and S. Flach, Compact discrete breathers on flat-band networks, Low Temperature Physics 44, 678 (2018), https://doi.org/10.1063/1.5041434 .
  * Fitzpatrick  _et al._ [2017] M. Fitzpatrick, N. M. Sundaresan, A. C. Li, J. Koch, and A. A. Houck, Observation of a Dissipative Phase Transition in a One-Dimensional Circuit QED Lattice, Physical Review X 7, 011016 (2017).
  * Fedorov  _et al._ [2021] G. Fedorov, S. Remizov, D. Shapiro, W. Pogosov, E. Egorova, I. Tsitsilin, M. Andronik, A. Dobronosova, I. Rodionov, O. Astafiev, and A. Ustinov, Photon transport in a Bose-Hubbard chain of superconducting artificial atoms, Physical Review Letters 126, 180503 (2021).
  * Martinez  _et al._ [2023] J. G. C. Martinez, C. S. Chiu, B. M. Smitham, and A. A. Houck, Interaction-induced escape from an Aharonov-Bohm cage 10.48550/arXiv.2303.02170 (2023).
  * Szameit  _et al._ [2005] A. Szameit, D. Blömer, J. Burghoff, T. Schreiber, T. Pertsch, S. Nolte, A. Tünnermann, and F. Lederer, Discrete nonlinear localization in femtosecond laser written waveguides in fused silica, Optics Express 13, 10552 (2005).
  * Cáceres-Aravena  _et al._ [2022] G. Cáceres-Aravena, D. Guzmán-Silva, I. Salinas, and R. A. Vicencio, Controlled Transport Based on Multiorbital Aharonov-Bohm Photonic Caging, Physical Review Letters 128, 256602 (2022).
  * Note [1] Notice, that from the viewpoint of a laboratory experiment the classical limit $\hbar\rightarrow 0$ corresponds to scaling the nonlinearity of the oscillator energy spectrum as $U=\hbar g$, and the driving strength as $\Omega\rightarrow\Omega/\sqrt{\hbar}$.
  * Kordas  _et al._ [2015] G. Kordas, D. Witthaut, P. Buonsante, A. Vezzani, R. Burioni, A. Karanikas, and S. Wimberger, The dissipative bose-hubbard model: Methods and examples, The European Physical Journal Special Topics 224, 2127 (2015).
  * Note [2] Attractors of this type were discussed earlier in Refs. [32, 33] with respect to exciton-polariton dynamics in two coupled micro-cavities. If we associate the $A$ and $B$ sites of our system with these two cavities then, by driving the $C$-site, we pump the antibonding mode of the $A-B$ system but it is the bonding mode which is subject to decay.
  * Note [3] This follows from the fact that for $g=0$ the single-particle density matrix of $N$ identical particles coincides with the density matrix of a single particle.
  * Bychek  _et al._ [2020] A. A. Bychek, D. N. Maksimov, and A. R. Kolovsky, Decay of symmetry-protected quantum states, Physical Review A 102, 033324 (2020).
  * Kolovsky [1996] A. R. Kolovsky, Quantum coherence, evolution of the Wigner function, and transition from quantum to classical dynamics for a chaotic system, Chaos: An Interdisciplinary Journal of Nonlinear Science 6, 534 (1996).
  * Rayanov  _et al._ [2015] K. Rayanov, B. Altshuler, Y. Rubo, and S. Flach, Frequency Combs with Weakly Lasing Exciton-Polariton Condensates, Physical Review Letters 114, 193901 (2015).
  * Lledó and Szymańska [2020] C. Lledó and M. H. Szymańska, A dissipative time crystal with or without $\mathrm{Z}_{2}$ symmetry breaking, New Journal of Physics 22, 075002 (2020).


# Categorical characterization of quadrics

Duo Li

###### Abstract.

We give a characterization of smooth quadrics in terms of the existence of
full exceptional collections of certain type, which generalizes a result of
C.Vial for projective spaces.

##  1\. Introduction

Let $X$ be a complex variety and $D^{b}(X)$ be its bounded derived category.
An object $E\in\mathcal{D}^{b}(X)$ is called exceptional if we have

$\text{Hom}(E,E[l])=\left\\{\begin{aligned} &{\mathbb{C}}\quad\text{if}\quad
l=0\\\ &0\quad\text{if}\quad l\neq 0\end{aligned}\right.$

An exceptional collection is a sequence $\\{E_{1}\cdots,E_{n}\\}$ of
exceptional objects which satisfy that $\text{Hom}(E_{j},E_{i}[l])$ vanishes
for all $j>i$ and all $l\in{\mathbb{Z}}.$ An exceptional collection
$\\{E_{1}\cdots,E_{n}\\}$ is full if $\mathcal{D}$ is generated by
$\\{E_{i}\\}.$

In general, full exceptional collections which consist of coherent sheaves are
rare. The existence of such collections would impose strong restrictions on
the geometry of the variety. We now list some varieties with such collections.

  * •

projective spaces: for any $a\in{\mathbb{Z}},$
$\\{\mathcal{O}(a),\mathcal{O}(a+1),\cdots,\mathcal{O}(a+n)\\}$ is a full
exceptional collection of ${\mathbb{P}}^{n}$ (see [1]).

  * •

smooth quadric $Q^{n}\subset{\mathbb{P}}^{n+1}.$ Kapranov [5] shows that

    * –

if $n$ is odd, for any $a\in{\mathbb{Z}},$
$\\{S,\mathcal{O}(a),\mathcal{O}(a+1),\cdots,\mathcal{O}(a+n-1)\\}$ is a full
exceptional collection of $Q^{n},$

    * –

if $n$ is even, for any $a\in{\mathbb{Z}},$
$\\{S^{-},S^{+},\mathcal{O}(a),\mathcal{O}(a+1),\cdots,\mathcal{O}(a+n-1)\\}$
is a full exceptional collection of $Q^{n}$

where $S,$ $S^{-},$ $S^{+}$ are certain spinor bundles.

A _minifold_ is a smooth projective variety $X$ whose $D^{b}(X)$ admits a full
exceptional collection $\mathcal{C}$ of minimal possible length $\dim X+1.$
Galkin, Katzarkov, Mellit and Shinder classify _minifolds_ up to dimension $4$
(see [3]). If $\mathcal{C}$ purely consists of line bundles, C. Vial proves
that $X$ is necessarily a projective space (see [10, Theorem 1.2]). In [7], S.
Kobayashi and T. Ochiai show that if there exists an ample line bundle $H$ on
a smooth projective variety $X$ with $\dim(X)c_{1}(H)\leq c_{1}(X),$ then $X$
is isomorphic to a projective space or a smooth quadric. The purpose of this
article is to generalize C.Vial’s theorem and prove a categorical analog of S.
Kobayashi and T. Ochiai’s classification. In view of Kapranov’s result, it is
reasonable to consider full exceptional collections of length $\dim X+1$ or
$\dim X+2.$

######  Theorem 1.1.

Let $X$ be a smooth projective variety of dimension $n$ $(n\geq 3)$. Suppose
that there is a full exceptional collection $\mathcal{C}$ of $D^{b}(X)$ which
consists of coherent sheaves and is of length $n+1.$ If $\mathcal{C}$ contains
a sub-collection {$\mathcal{L}_{1},$ $\mathcal{L}_{2},\cdots,$
$\mathcal{L}_{n}$} where $\mathcal{L}_{i}$ $(1\leq i\leq n)$ are line bundles,
then $X$ is isomorphic to ${\mathbb{P}}^{n}$ or $Q^{n}.$ Moreover, if $n$ is
an even number, $X$ is isomorphic to ${\mathbb{P}}^{n}.$

######  Theorem 1.2.

Let $X$ be a smooth projective variety of dimension $n$ ($n\geq 3$). Suppose
that there is a full exceptional collection $\mathcal{C}$ of $D^{b}(X)$ which
consists of coherent sheaves and is of length $n+2.$ If $\mathcal{C}$ contains
a sub-collection {$\mathcal{L}_{1},$ $\mathcal{L}_{2},\cdots,$
$\mathcal{L}_{n}$} where $\mathcal{L}_{i}$ $(1\leq i\leq n)$ are line bundles,
then $X$ is an even-dimensional variety and is isomorphic to $Q^{n}.$

To prove Theorem 1.1 and Theorem 1.2, we use the method from C. Vial’s article
[10] and some technical input (see Lemma 2.1).

Convention: In this article, a variety is an integral scheme of finite type
defined over ${\mathbb{C}}$.

##  2\. A technical lemma

Firstly, we note an easy fact about series of numbers.

######  Lemma 2.1.

Let $\\{a_{1},\cdots,a_{n}\\}$ be $n$ distinct integers with $n\geq 5.$ Assume
that the cardinality of the set $A=\\{a_{j}-a_{i}|1\leq i<j\leq n\\}$ is $n.$
Then there exists an integer $d$ such that $(a_{1},\cdots,a_{n})$ is one of
the following series:

  1. (1)

$(a_{1},\cdots,\widehat{a_{1}+kd},\cdots,a_{1}+nd)$ where $1\leq k\leq n-1$ is
an integer.

  2. (2)

$(a_{1},a_{1}+2d,\cdots,a_{1}+(n-1)d,a_{1}+(n+1)d)$

  3. (3)

$\sigma(a_{1},\cdots,a_{1}+(n-1)d)$ where $\sigma$ is a composition of
disjoint permutations $(i_{1},i_{1}+1)\cdots(i_{l},i_{l}+1)$ for some $1\leq
l\leq n.$

###### Proof.

We firstly assume $a_{1}<a_{2}<\cdots<a_{n}$ and claim that
$(a_{1},\cdots,a_{n})$ is of type $(1)$ and $(2).$ It is easy to verify our
claim when $n$ is $5.$ For the case $n>5,$ the cardinality of
$B:=\\{a_{j}-a_{i}|1\leq i<j\leq n-1\\}$ is $n-1$ or $n-2.$ If $\sharp B$ is
$n-2,$ then $(a_{1},\cdots,a_{n-1})$ is an arithmetic progression which can be
denoted by $(a_{1},a_{1}+d,\cdots,a_{1}+(n-2)d)$ for some $d\in{\mathbb{Z}}.$
It is easy to see $a_{n}$ is $a_{1}+nd.$ If $\sharp B$ is $n-1,$ by induction,
we have
$(a_{1},\cdots,a_{n-1})=(a_{1},\cdots,\widehat{a_{1}+kd},\cdots,a_{1}+(n-1)d)$
($1\leq k\leq n-2$) or
$(a_{1},\cdots,a_{n-1})=(a_{1},a_{1}+2d,\cdots,a_{1}+(n-2)d,a_{1}+nd).$ In the
first case, if $k$ is $1,$ then $a_{n}$ equals to $a_{1}+nd$ or
$a_{1}+(n+1)d.$ If $k$ is bigger than $2,$ then $a_{n}$ equals to $a_{1}+nd.$
In the second case, we can assume $a_{n}=a_{1}+md$ for some integer $m>n.$ If
$m$ is $n+1$ or $n+2,$ then $(n-1)d$ belongs to $A.$ If we have $m>n+2,$ then
$(m-2)d$ belongs to $A.$ Both situations lead to the fact $\sharp A\geq n+1,$
so we can exclude the case
$(a_{1},\cdots,a_{n-1})=(a_{1},a_{1}+2d,\cdots,a_{1}+(n-2)d,a_{1}+nd)$ and
prove our claim.

For an arbitrary series $(a_{1},a_{2},\cdots,a_{n}),$ there is a permutation
$\alpha$ satisfying $a_{\alpha(1)}<a_{\alpha(2)}<\cdots<a_{\alpha(n)}.$ Note
that the cardinality of the set $C:=\\{a_{\alpha(j)}-a_{\alpha(i)}|1\leq
i<j\leq n\\}$ is less than $\sharp\\{a_{j}-a_{i}|1\leq i<j\leq n\\}.$ It
follows that $\sharp C$ is $n-1$ or $n.$ If $\sharp C$ is $n-1,$ then there
exists an integer $d$ such that $(a_{\alpha(1)},\cdots,a_{\alpha(n)})$ is
arithmetic and $C$ is $\\{d,2d,\cdots,(n-1)d\\}.$ Note that for any integer
$k\neq 1,$ if $kd$ belongs to $A,$ then $\operatorname{sgn}(k)\cdot d$ belongs
to $A.$ It follows that $\forall k>1,$ $kd\in A$ implies $-kd\notin A.$ Now we
can assume $2d\in B,$ then
$(a_{\alpha(1)},a_{\alpha(1)}+2d,a_{\alpha(1)}+4d,\cdots),$ as well as
($a_{\alpha(1)}+d,a_{\alpha(1)}+3d,a_{\alpha(1)}+5d,\cdots$), is a subseries
of $(a_{1},\cdots,a_{n}).$ If $a_{\alpha(1)}+d$ appears on the right of
$a_{\alpha(1)}$ in $(a_{1},\cdots,a_{n}),$ then we have
$A=\\{-d,d,2d,\cdots,(n-1)d\\}.$ If $a_{\alpha(1)}+d$ appears on the left of
$a_{\alpha(1)}$ in $(a_{1},\cdots,a_{n}),$ then we have $-d\in A$ and
$a_{\alpha(1)}+3d$ appears on the right of $a_{\alpha(1)}.$ It follows that
$A$ is $\\{-d,d,2d,\cdots,(n-1)d\\}.$ So if $\sharp C$ is $n-1,$ the series
$(a_{1},\cdots,a_{n})$ is of type $(3).$ Now suppose that $\sharp C$ is $n,$
then there exists an integer $d$ such that
$(a_{\alpha(1)},\cdots,a_{\alpha(n)})$ is of type $(1)$ or $(2).$ For any
integer $l>0,$ the numbers $ld$ and $-ld$ can not belong to $A$ at the same
time. So we can assume $d\in A$ and $-d\notin A.$ By a similar argument, one
can verify $(a_{1},\cdots,a_{n})=(a_{\alpha(1)},\cdots,a_{\alpha(n)}).$ ∎

######  Remark 2.2.

For the case $n=4,$ $(a_{1},a_{2},a_{3},a_{4})$ is one of the following
series:

  1. (1)

$(a,a+d,a+2d,a+4d)$ or $(a,a+2d,a+3d,a+4d)$ where $d$ is an integer.

  2. (2)

$\sigma(a,a+d,a+2d,a+3d)$ where $d$ is a positive integer and $\sigma\neq id$
is a permutation.

  3. (3)

$\sigma(a,a+d,b,b+d)$ where $d$ is a positive integer with $b>a+d$ and
$\sigma$ is a permutation.

##  3\. Proof of Theorem 1.1 and Theorem 1.2

Let $X$ be a smooth projective variety. Recall that, for any two objects $E$
and $F$ in $D^{b}(X),$ the Euler pairing $\chi$ is the integer
$\chi(E,F):=\sum_{l}(-1)^{l}\dim_{{\mathbb{C}}}\text{Hom}(E,F[l]).$

######  Definition 3.1.

An object $E$ is said to be numerically exceptional if $\chi(E,E)=1.$ A
collection $\\{E_{1},\cdots,E_{r}\\}$ of numerical exceptional objects is a
numerical exceptional collection if $\chi(E_{j},E_{i})=0$ for all $j>i.$

For Fano varieties of Picard number $1,$ we have the following result.

######  Theorem 3.2.

Let Let $X$ be a smooth projective Fano variety of dimension $n$ $(n\geq 3)$
whose $\operatorname{Pic}(X)$ is isomorphic to ${\mathbb{Z}}.$ Assume that
there is a numerical exceptional collection
$\\{\mathcal{L}_{1},\cdots\mathcal{L}_{n}\\}$ where $\mathcal{L}_{i}$ $(1\leq
i\leq n)$ are line bundles. Then $X$ is isomorphic to ${\mathbb{P}}^{n}$ or a
smooth quadric $Q^{n}.$

###### Proof.

We denote $c_{1}(X)$ by $\lambda H$ for some $\lambda\in{\mathbb{Z}}^{+}.$ The
Euler character $\chi(\mathcal{O}_{X}(aH))$ is a polynomial $P$ with rational
coefficients of degree $n$ in the variable $a.$ We write $\mathcal{L}_{i}$ as
$\mathcal{L}_{i}=O_{X}(a_{i}H)$ for some integer $a_{i}$ and we have
$a_{i}\neq a_{j}$ for any $i\neq j,$ as $\chi(\mathcal{O}_{X})$ is $1.$ The
equalities
$\chi(\mathcal{L}_{j},\mathcal{L}_{i})=\chi(\mathcal{L}_{i}\otimes\mathcal{L}_{j}^{-1})=P(a_{i}-a_{j})=0$
$(i<j)$ imply that $a_{i}-a_{j}$ $(i<j)$ are roots of $P(a)=0.$ So the
cardinality $\mu$ of the set {$a_{j}-a_{i}|1\leq i<j\leq n$} is $n-1$ or $n.$

If $\mu$ is $n-1,$ the progression $(a_{1},a_{2},\cdots,a_{n})$ is an
arithmetic progression with $a_{2}-a_{1}=d\in{\mathbb{Z}}.$ The polynomial $P$
vanishes at $-ld$ for $1\leq l\leq n-1.$ Let the remaining root of $P(a)=0$ be
$-N.$ By Riemann-Roch, we have

$P(a)=\chi(\mathcal{O}_{X}(aH))=\frac{\deg(H^{n})}{n!}a^{n}+\frac{\deg(H^{n-1}\cdot
c_{1}(X))}{2(n-1)!}a^{n-1}+\cdots+\chi(\mathcal{O}_{X}).$

Note the following equalities:

$\frac{\deg(H^{n})}{n!}(n-1)!d^{n-1}N=1\qquad(1)$
$\frac{\deg(H^{n})}{n!}(N+\sum_{l=1}^{n-1}ld)=\frac{\deg(H^{n-1}\cdot
c_{1}(X))}{2(n-1)!}\qquad(2).$

One can deduce $d=1,$ $\deg H^{n}=1,$ $\lambda=n+1$ or $d=1,$ $\deg H^{n}=2,$
$\lambda=n.$ Then by [7], the Fano variety $X$ is isomorphic to
${\mathbb{P}}^{n}$ or $Q^{n}.$

If $\mu$ is $n,$ we aim to show that $X$ is isomorphic to ${\mathbb{P}}^{n}.$
We firstly assume $n\geq 5,$ then the progression $(a_{1},a_{2},\cdots,a_{n})$
is classified in Lemma 2.1. If $(a_{1},a_{2},\cdots,a_{n})$ is of type $(1)$
as in Lemma 2.1, then there exists an integer $d$ such that the polynomial
$P(a)$ vanishes exactly at $-ld$ $(1\leq l\leq n).$ By a similar argument, we
have $\deg H^{n}=1,$ $d=1$ and $\lambda=n+1.$ If $(a_{1},a_{2},\cdots,a_{n})$
is of type $(2)$ as in Lemma 2.1, the roots of $P(a)=0$ is
$\\{-d,-2d,\cdots,-(n-1)d,-(n+1)d\\}$ for some integer $d.$ Then we have
$\frac{\deg(H^{n})}{n!}(n-1)!d^{n}(n+1)=1,$ which is absurd as $d$ is an
integer. If $(a_{1},a_{2},\cdots,a_{n})$ is of type $(3)$ as in Lemma 2.1, the
roots of $P(a)=0$ is $\\{\pm d,-2d,\cdots,-(n-1)d\\}$ for some integer $d.$
Then it is easy to deduce $\deg(H^{n})d^{n}=-n$ and
$\lambda=\frac{(n+1)(n-2)}{n}d\in{\mathbb{Z}}^{+},$ which is impossible. Now
suppose $n=3.$ The progression of integers $(a_{1},a_{2},a_{3})$ is not
arithmetic. By a simple calculation, we have
$\lambda=\frac{4}{3}(a_{3}-a_{1})\geq 4\in{\mathbb{Z}}^{+}.$ Now suppose
$n=4.$ If $(a_{1},a_{2},a_{3},a_{4})$ is of type $(1)$ as in Remark 2.2, by a
similar argument, we can deduce that $\lambda$ is $5.$ If
$(a_{1},a_{2},a_{3},a_{4})$ is of type $(2)$ as in Remark 2.2, then there
exists an integer $d$ such that the roots of $P(a)=0$ are
$\\{d,-d,-2d,-3d\\},$ so we have
$\frac{\deg(H^{4})}{4!}\prod_{i<j}(a_{j}-a_{i})<0,$ which is impossible. If
$(a_{1},a_{2},a_{3},a_{4})$ is of type $(3)$ as in Remark 2.2, by the method
of exhaustion, the only possibility satisfying
$\frac{\deg(H^{4})}{4!}\prod_{i<j}(a_{j}-a_{i})=1$ and $\lambda>0$ is
$(a_{1},a_{2},a_{3},a_{4})=(a_{1},a_{1}+1,a_{1}+3,a_{1}+4),$ hence $\lambda$
is $5.$ By all the above argument, if $\mu$ is $n,$ then $\lambda$ is $n+1,$
hence $X$ is isomorphic to ${\mathbb{P}}^{n}.$ ∎

We now prove a lemma about the Picard group of a variety. For similar results,
we refer to [2, Theorem 3.4] and [10, Lemma 2.6].

######  Lemma 3.3.

Let $X$ be a smooth projective variety of dimension $n$. Suppose that there is
a full exceptional collection $\mathcal{C}$ of $D^{b}(X).$ If the length of
$\mathcal{C}$ is $n+1,$ then $\operatorname{Pic}(X)$ is isomorphic to
${\mathbb{Z}}.$ If the length of $\mathcal{C}$ is $n+2$ ($n\geq 3$), then $n$
is an even number and the $\operatorname{Pic}(X)$ is isomorphic to
${\mathbb{Z}}.$

###### Proof.

For any smooth projective variety $X,$ the existence of a full exceptional
collection of $D^{b}(X)$ whose length is $m$ implies that the Grothendieck
group $K_{0}(X)$ of $D^{b}(X)$ is isomorphic to ${\mathbb{Z}}^{m}.$ Moreover,
we have the identity

$m=\operatorname{rk}K_{0}(X)=\sum_{i=0}^{n}\operatorname{rk}\operatorname{CH}^{i}_{{\mathbb{Q}}}(X)$

where every $\operatorname{rk}\operatorname{CH}^{i}_{{\mathbb{Q}}}(X)$ is non-
zero. If $m$ equals to $n+1,$ then $\operatorname{Pic}(X)$ is of rank $1.$ If
$m$ equals to $n+2,$ by [6] or [9, Proposition 3.10], the cycle class maps
$cl_{i}:\operatorname{CH}^{i}_{{\mathbb{Q}}}(X)\rightarrow
H^{2i}(X,{\mathbb{Q}})$ $(0\leq i\leq n)$ are all isomorphisms. So for any
$0\leq i\leq n,$ there exists a perfect pairing
$\operatorname{CH}^{i}_{{\mathbb{Q}}}(X)\times\operatorname{CH}^{n-i}_{{\mathbb{Q}}}(X)\rightarrow{\mathbb{Q}}.$
We have
$\operatorname{rk}\operatorname{CH}^{i}_{{\mathbb{Q}}}(X)=\operatorname{rk}\operatorname{CH}^{n-i}_{{\mathbb{Q}}}(X)=1$
for $i\neq n-i.$ It follows that $n$ is an even number and $X$ is of Picard
number $1.$ By [4, Lemma 2.2] or [10, Lemma 2.6], the Chow group
$\operatorname{CH}^{1}(X)$ is torsion free, so $\operatorname{Pic}(X)$ is
isomorphic to ${\mathbb{Z}}.$ ∎

_The proof of Theorem 1.1._ By [2, Proposition 3.2], the variety $X$ is Fano.
Then Theorem 1.1 is a direct corollary of Theorem 3.2 and Lemma 3.3. ∎

To prove Theorem 1.2, we need the concept of the anticanonical pseudoheight
which is introduced by A. Kuznetsov. For an exceptional collection
$\mathcal{C}=\\{E_{0},\cdots,E_{r}\\},$ the anticanonical pseudoheight is
defined as

$ph_{ac}(\mathcal{C}):=\mathop{\min}\limits_{1\leq a_{o}<\cdots<a_{p}\leq
n}(e(E_{a_{0}},E_{a_{1}})+\cdots+e(E_{a_{p-1}},E_{a_{p}})+e(E_{a_{p}},E_{a_{0}}\otimes
K_{X}^{-1})-p)$

where $\forall F,$ $F^{\prime}\in D^{b}(X),$ the relative height
$e(F,F^{\prime})$ is defined by $\min\\{k|Ext^{k}(F,F^{\prime})\neq 0\\}.$ A.
Kuznetsov shows in [8, Corollary 6.2] that if $ph_{ac}(\mathcal{C})>-\dim X,$
then $\mathcal{C}$ is not full.

_The proof of Theorem 1.2._ By Lemma 3.3, we have
$\operatorname{Pic}(X)\simeq{\mathbb{Z}}H$ for some ample line bundle $H$ and
$\operatorname{rk}K_{0}(X)$ is $n+2,$ which implies that $X$ is even-
dimensional. We keep using the notations in the proof of Theorem 3.2 and aim
to show that $X$ is Fano. If $n>5,$ we can list all the possibilities in the
following diagram:

| $(a_{1},\cdots,a_{n})$ | roots of $P(a)$ | $\lambda$ | $X$
---|---|---|---|---
(1) | $(a_{1},a_{1}+1,\cdots,a_{1}+(n-1))$ | $\\{-1,-2,\cdots,-(n-1)\\}$ | $n$ | $Q^{n}$
(2) | $(a_{1},a_{1}+1,\cdots,a_{1}+(n-1))$ | $\\{-1,-2,\cdots,-n\\}$ | $n+1$ | ${\mathbb{P}}^{n}$
(3) | $(a_{1},a_{1}-1,\cdots,a_{1}-(n-1))$ | $\\{1,2,\cdots,(n-1)\\}$ | $-n$ | general type
(4) | $(a_{1},a_{1}-1,\cdots,a_{1}-(n-1))$ | $\\{1,2,\cdots,n\\}$ | $-n-1$ | general type
(5) | $(a_{1},\cdots,\widehat{a_{1}-k},\cdots,a_{1}-n)$ | $\\{1,2,\cdots,n\\}$ | $-n-1$ | general type

By the Kodaira vanishing theorem, for line bundles $\mathcal{L},$
$\mathcal{L}^{\prime}$ with $\mathcal{L}\otimes\mathcal{L}^{\prime-1}$ ample,
the relative height $e(\mathcal{L},\mathcal{L}^{\prime})$ is bigger than $n.$
Note that for all coherent sheaves $F,$ $F^{\prime}$, $e(F,F^{\prime})$ is
positive. It easily follows that in cases $(3)$ $(4)$ $(5)$ of our diagram,
$ph_{ac}(\mathcal{C})>-n,$ which is a contradiction.

If $n$ is $4,$ for the cases $\mu=3$ or $\mu=4,$ $(a_{1},a_{2},a_{3},a_{4})$
of type $(1)$ and $(2)$ as in Remark 2.2, we can easily deduce that $X$ is
Fano, otherwise there is $ph_{ac}(\mathcal{C})>-4.$ If
$(a_{1},a_{2},a_{3},a_{4})$ is of type $(3).$ Note that if
$\chi(\mathcal{L}_{j},\mathcal{L}_{i})$ vanishes, by the Serre duality,
$\chi(\mathcal{L}_{i},\mathcal{L}_{j}\otimes K_{X})$ also vanishes. It follows
that for any $i<j,$ $a_{j}-a_{i}-\lambda$ is also a root of $P(a)=0.$ Then by
the method of exhaustion, $(a_{1},a_{2},a_{3},a_{4})$ is $(a,a+1,a+3,a+4),$
which also implies that $X$ is also Fano. Note that $K_{0}({\mathbb{P}}^{n})$
is isomorphic to ${\mathbb{Z}}^{n+1},$ by Theorem 3.2, $X$ is isomorphic to
$Q^{n}.$

Acknowledgments. The author is very grateful to Professor Baohua Fu, Yihua
Chen, Zili Zhang for their helpful suggestions and discussions. The author
wishes to thank Professor A. Kuznetsov, C. Vial and Shizhuo Zhang for many
valuable suggestions for the earlier version of this article.

## References

  * [1] Beilinson, A.: Coherent sheaves on ${\mathbb{P}}^{n}$ and problems in linear algebra. Funktsional. Anal. i Prilozhen. 12 (1978), no. 3, 68-69.
  * [2] Bondal, A. I.; Polishchuk, A. E.: Homological properties of associative algebras: the method of helices. Izv. Ross. Akad. Nauk Ser. Mat. $\mathbf{57}$ (1993), no. 2, 3-50
  * [3] Galkin, S.; Katzarkov, L.; Mellit, A.; Shinder, E.: Minifolds and phantoms, preprint available at arXiv:1305.4549v2.
  * [4] Galkin, S.; Shinder, E.: Exceptional collections of line bundles on the Beauville surface. Adv. Math. 244 (2013), 1033-1050.
  * [5] Kapranov, M.: On the derived categories of coherent sheaves on some homogeneous spaces. Invent. Math. 92 (1988), no. 3, 479-508.
  * [6] Kimura, S.: Surjectivity of the cycle map for Chow motives. _Motives and algebraic cycles,_ 157-165, Fields Inst. Commun., 56, Amer. Math. Soc., Providence, 2009.
  * [7] Kobayashi, S.; Ochiai, T.: Characterizaions of complex projective spaces and hyperquadrics. J. Math. Kyoto Univ., 13 (1973), 31-47.
  * [8] Kuznetsov, A.: Height of exceptional collections and Hochschild cohomology of quasiphantom categories. J. reine angew. Math., 708 (2015), 213-243.
  * [9] Vial, C.: Projectors on the intermediate algebraic Jacobians. New York J. Math. 19 (2013), 793-822.
  * [10] Vial, C.: Exceptional collections, and the Néron-Severi lattice for surfaces. Adv. Math. 305 (2017), 895-934.