# Mitigate Domain Shift by Primary-Auxiliary Objectives Association
for Generalizing Person ReID

Qilei Li, Shaogang Gong
Queen Mary University of London
{q.li<EMAIL_ADDRESS>
###### Abstract

While deep learning has significantly improved ReID model accuracy under the
independent and identical distribution (IID) assumption, it has also become
clear that such models degrade notably when applied to an unseen novel domain
due to unpredictable/unknown domain shift. Contemporary domain generalization
(DG) ReID models struggle in learning domain-invariant representation solely
through training on an instance classification objective. We consider that a
deep learning model is heavily influenced and therefore biased towards domain-
specific characteristics, _e.g._ , background clutter, scale and viewpoint
variations, limiting the generalizability of the learned model, and
hypothesize that the pedestrians are domain invariant owning they share the
same structural characteristics. To enable the ReID model to be less domain-
specific from these pure pedestrians, we introduce a method that guides model
learning of the primary ReID instance classification objective by a concurrent
auxiliary learning objective on weakly labeled pedestrian saliency detection.
To solve the problem of conflicting optimization criteria in the model
parameter space between the two learning objectives, we introduce a Primary-
Auxiliary Objectives Association (PAOA) mechanism to calibrate the loss
gradients of the auxiliary task towards the primary learning task gradients.
Benefiting from the harmonious multitask learning design, our model can be
extended with the recent test-time diagram to form the PAOA+, which performs
on-the-fly optimization against the auxiliary objective in order to maximize
the model’s generative capacity in the test target domain. Experiments
demonstrate the superiority of the proposed PAOA model.

##  1 Introduction

Person Re-IDentification (ReID) [21, 45, 40, 18] is a fundamental task which
aims to retrieve the same pedestrian across non-overlapping camera views by
measuring the distances among representations of all the candidates in a pre-
learned discriminative feature space. However, like most deep-learning models,
current ReID techniques are built based on an intrinsic assumption of
independent and identical distribution (IID) between training and test data.
The IID assumption becomes mostly invalid across different domains when
training and test data are not from the same environment. As a result, most
contemporary ReID models suffer from dramatic degradation when applied to a
new domain [25, 4, 34]. Domain Generalization (DG) methods [46, 47, 26], which
aim to learn a generalizable model between a source and a target domain have
been explored by recent studies to address this problem.

Figure 1: Comparing a standard Domain Generalization ReID model and the
proposed Primary-Auxiliary Objectives Association (PAOA) model. A DG model is
typically trained by optimizing an instance classification objective, which
can suffer from overfitting to domain-specific characteristics, _e.g._ ,
luminance, background, scale, and viewpoint. The PAOA model considers learning
jointly a weakly labeled/supervised auxiliary saliency detection task
concurrently with the primary task of the discriminative person ReID. This is
achieved by calibrating the gradient of the auxiliary task against that of the
primary objective as its reference.

A number of DG ReID methods have been developed to mitigate performance
degradation caused by domain shift between training (source) data and test
(target) data. They can be broadly categorized into three main groups: (1)
Learning from diversified training samples [16, 1], (2) Aligning the
distribution of source domains by data statistics [50, 49, 14], (3) Exploiting
meta-learning [4, 47, 5, 42] to mimic source-target distribution
discrepancies. The first category confers advantages to a model through the
utilization of a diversified training dataset by either image sample
augmentation or feature distribution expansion. The second category aims to
learn a source-invariant model by aligning the training data, and expecting it
to be invariant for the target domain. The third category focuses on
simulating the training/testing discrepancy. Despite some performance
improvement from these methods, their overall performances across domains
remain poor, _e.g._ , the latest SOTA models [4, 42] can only achieve below
20% mAP on the MSMT17 benchmark. This highlights the limitation of overfitting
in the current DG ReID models and their inability to learn a more
generalizable cross-domain model representation. We consider this is due to
the not-insignificant interference of domain-specific contextual scene
characteristics such as background, viewpoint, and object distances to a
camera (scale), which are identity-irrelevant but can change significantly
across different domains. Contemporary DG ReID models are mostly trained by an
instance-wise classification objective function, indirectly learning person
foreground attention selection (Figure 1(a)). They are sensitive to such
domain-specific but identity-irrelevant contextual information, resulting in
the misrepresentation of person foreground attention and leading to less
discriminative ReID representation. This likely causes notable ReID
performance degradation on models trained and deployed in different domains.
To mitigate the impact of domain-specific contextual attributes, an intuitive
solution is to isolate the pedestrian object to acquire a domain-invariant
representation. Several endeavors [11, 48, 29] have been made to guide the
person identification network focusing on the pedestrian with the human
saliency prior, which can point out the attentive region relevant to the human
subject. These methods have certain limitations, either relying on exhaustive
manual masking [29] or lacking an appropriate training objective [11, 48] to
ensure the accuracy of the generated segmentation mask. Besides this, it is
crucial to note that these methods fail to consider the potential worst-case
scenario in which the saliency attention prior may be inaccurate, further
leading to negative impacts on identification rather than improvement.

In this work, we address this problem by introducing a novel model learning
regularization method called Primary-Auxiliary Objectives Association (PAOA).
Our aim is to minimize domain-specific contextual interference in model
learning by focusing more on the domain-invariant person’s unique
characteristics. This is achieved by introducing the association of learning
the primary instance classification objective function with an auxiliary
weakly labeled/supervised pedestrian saliency detection objective function,
the idea is illustrated in Figure 1(b). Specifically, PAOA is realized in two
parts: (1) Additionally train a pedestrian saliency detection head with an
auxiliary supervision to assist in focusing the primary ReID discriminative
learning task on more domain-invariant feature characteristics. (2) Eliminate
the interference attributed to inaccurate saliency labels by calibrating the
gradients of the shared feature extractor raised from the weakly-labeled
auxiliary learning task towards that of the primary task as a reference when
they are in conflict [28]. This association mechanism helps ensure the ReID
model learns to attentively focus on generic yet discriminative pedestrian
information whilst both learning tasks are harmoniously trained.

Our contributions are: (1) We introduce the idea of optimizing a more domain-
generic ReID learning task that emphasizes domain-invariant pedestrian
characteristics by associating the ReID instance discriminative learning
objective to an auxiliary pedestrian saliency detection objective in a way
that does not create conflicts or hinder the effectiveness of primary
objective. (2) We formulate a novel regularization called Primary-Auxiliary
Objectives Association (PAOA) to implement the proposed association learning.
It jointly trains the primary and auxiliary tasks with referenced gradient
calibration to solve the conflicting optimization criteria between the two
learning objectives, and promote the learning of a more domain-generic ReID
model. (3) We further explore the target domain test data characteristics by
incorporating the PAOA regularization into a deployment-time model online
optimization process. To that end, we formulate a PAOA+ mechanism for on-the-
fly target-aware model optimization and show its performance benefit.

##  2 Related Work

#### Domain Generalizable ReID

(DG ReID) assuming the absence of target domains during training, aims to
learn a generalizable model which can extract discriminative representations
in any new environment. It’s naturally challenging but practical and has
attracted increasing attention. Contemporary studies typically fall into three
primary classifications: (1) To benefit the model from the diverse training
data achieved by augmentation. (2) To align the target domain with the BN
statistics calculated over the source domain. (3) To mimic the train/test
discrepancy with meta-learning. Despite the improvement obtained by these SOTA
models, significant room for improvement remains, as indicated by the low mAP
scores, _e.g._ , less than 20% on MSMT17 and less than 40% on CUHK03. This is
attributed to the domain-specific interference in the source domain that
limits the learning of a domain-invariant model. In this work, we aim to
tackle this issue by guiding the model to focus on the discriminative
pedestrian area with the tailored auxiliary task, and propose the PAOA
regularization for that end.

Salient Object Detection [3] aims to identify objects or regions that are
visually more attentive than the surrounding areas. It has been significantly
boosted solely by the rapid development of deep learning. Current detection
models are usually trained end-to-end and output a fine-grained saliency map
at the pixel level. In this work, we design the auxiliary task with the
pedestrian saliency detection objective. Instead of exhaustively labeling the
pedestrian area manually as the previous work [29], we propose to use weakly
labeled data generated by a trained salient object detection model, to benefit
from large-scale training. The recent work GASM [11] shares the similar spirit
to ours by employing weakly labelled saliency masks as an additional prior.
However, GASM simply trains the saliency detection layers with the
classification network while omitting the potential worst-case where the weak
label is not accurate and cause potential conflict optimization direction
during model training. In contrast, our method focuses on the association
between instance classification and saliency detection objectives by the
proposed referenced gradient calibration mechanism, which promotes the
learning of the primary objective while mitigating the conflicts between the
primary and auxiliary tasks.

Multitask learning [39] emerges as a solution to learn a single model which is
shared across several tasks, so as to achieve greater efficiency than training
dedicated models individually for each task. Recent work [37] pointed out that
conflicting gradients during multitask learning impede advancement. To break
this condition and achieve positive interactions between tasks, they proposed
to de-conflict such gradients by altering their directions towards a common
orientation. Our model is also constructed in a multitask learning manner, in
which the main and the auxiliary tasks are jointly optimized during training.
However, the auxiliary task is designed to facilitate the main task therefore
it is unsuitable to consider them in the same hierarchy. Instead, we propose
referenced gradient calibration by setting the main task as the reference, and
calibrating the auxiliary gradient towards it, so as to ensure the auxiliary
task can be harmoniously trained alongside the main task, so that it may
provide supervision for the primary model objective.

Test-Time model optimization is an emerging paradigm to tackle distribution
shifts between training and testing environments. The key idea is to perform
post-training model optimization given the test samples during deployment.
Several recent works [33, 32, 13, 7] proposed to optimize the model parameters
by providing proper supervision, such as batch-norm statistics, entropy
minimization, and pseudo-labeling. Another line of work [31, 23] jointly
trains additional self-supervised auxiliary tasks, which are subsequently used
to guide the model optimization during testing. This does not involve any
assumptions about the output and is therefore more generic. It has also been
applied to ReID [9] by considering self-supervised learning tasks for updating
BN statistics. In this work, we formulate PAOA+ by incorporating the proposed
PAOA regularization into the deployment-time optimization framework to seek
further improvement. With the tailored auxiliary objective as the optimization
supervision, PAOA+ effectively exploits the underlying target domain
characteristic and exhibits boosted performance on all the benchmarks.

##  3 Methodology

#### Problem Definition

Given a labeled source domain
$\mathcal{D}_{S}=\left\\{(x_{i},y_{i})\right\\}_{i\in\\{1,\cdots,N\\}}$ for
training, where $N$ is the number of samples, the aim of ReID is to learn a
mapping function parameterized by $\theta$ that projects a person image $x$ to
a high-dimensional feature representation $f_{\theta}$, with the constraint
that features of the same identity have a smaller distance relative to one
another. DG ReID is more practical by assuming the non-availability of the
target domain during training, and expects the model to be able to extract
discriminative feature representations from any target domain. Current models
are designed solely with an instance classification objective, that can be
confused by negative domain-specific information and fall into a local optimum
of the source domain.

Figure 2: Overview of the proposed Primary-Auxiliary Objectives Association
(PAOA) model. The purpose is to derive generic feature representations by
guiding the network to attentively focus on pedestrian information and
mitigate the interference of domain-specific knowledge, which is achieved by
the PAOA regularization of a primary classification objective and an auxiliary
pedestrian saliency detection objective: (a) The auxiliary task is jointly
trained to provide hard-coded spatial attention to the pedestrian region. (b)
The primary task is used as a reference to calibrate the gradients of the
auxiliary objective when they are conflicting.

###  3.1 Overview

In this work, we consider the problem of generalizing a ReID model to any new
deployment target environment subject to unknown domain bias between the
training and the test domains, where there is no labeled training data from
the test domain. To that end, we propose a Primary-Auxiliary Objectives
Association (PAOA) regularization method to enable the model to be more
attentive to learning universal identity generative information that is
applicable in any domain whilst concurrently maximizing ReID discriminative
information from the domain labeled data. Figure 2 shows an overview of PAOA
in model training with two associative steps: (1) Guiding the ReID model to
focus on discriminative pedestrian information with an additional auxiliary
task dedicated to visual saliency detection. (2) Calibrate the gradients of
the auxiliary task when it conflicts with the primary instance classification
objective. To boost the performance, we build PAOA+ to utilize the available
samples in deployment time by minimizing the proposed auxiliary objective, and
demonstrate the plug-and-play merit of our design.

###  3.2 Joint Primary-Auxiliary Objectives Learning

The primary and auxiliary objectives are jointly trained in a multitask
learning architecture, which is composed of a shared feature extractor
$f_{\theta}$, and two dedicated heads $h_{p}$ and $h_{a}$ respectively for the
primary and auxiliary tasks.

#### Primary Objective: Person ReID

Learning a strong instance classification network is fundamentally important
for training a discriminative ReID model. Given a labeled training set
$\mathcal{D}$ = $\\{(x_{i},y^{(p)}_{i})\\}_{i\in\\{1,\cdots,N\\}}$, where
$x_{i}$ is a person image and $y^{(p)}_{i}$ is the corresponding instance
category label, the primary instance classification task is trained with a
softmax cross-entropy (CE) loss $\mathcal{L}_{\text{id}}$ and a triplet loss
$\mathcal{L}_{\text{tri}}$:

$\mathcal{L}_{\text{id}}=-\sum_{i=1}^{N}\sum_{j=1}^{C}p_{i}^{j}\text{log}\hat{p}_{i}^{j},$
(1)

where $p_{i}$ is one-hot vector activated at $y_{i}^{(p)}$, and
$\hat{p}_{i}^{j}$ is the probability for categorized into the $j$th class that
calculated from the classifier. The additional triplet loss constrains the
distance between positive (same identity) and negative (different identities)
sample pairs, which is formulated as

$\mathcal{L}_{\text{tri}}=\sum_{i=1}^{N}[d_{p}-d_{n}+\alpha]_{+},$ (2)

where $d_{p}$ and $d_{n}$ respectively denote the Euclidean distances for the
positive and negative pairs in feature space. $\alpha$ is the margin that
controls the sensitivity and $[s]_{+}$ is $\text{max}(s,0)$. The overall loss
function for the primary task is as follows:

$\mathcal{L}_{\text{prim}}=\mathcal{L}_{\text{id}}+\mathcal{L}_{\text{tri}}.$
(3)

#### Auxiliary Objective: Pedestrian Saliency Detection

As illustrated in [31], an auxiliary task closely aligned with the primary
task can substantially prompt the learning of the primary objective. Inspired
by this, we formulated the auxiliary task as pedestrian saliency detection to
perform pixel-level pedestrian localization within the cropped pedestrian
bounding boxes. Such an auxiliary task is complementary to the primary task by
providing pixel-level hard-coded spatial attention to guide the ReID model to
focus on the pedestrian region. Instead of exhaustively manually annotating
the pedestrian region, we benefit from the large-scale trained model [41] and
perform feed-forward inference to get the weakly labelled samples.
Specifically, given a trained saliency model $\mathcal{G}$, we feed the sample
to obtain the weak label as $y_{i}^{(a)}=\mathcal{G}(x_{i})$, which is a 2D
map to indicate the saliency area. The auxiliary task and it’s essentially a
regression task in the pixel level. To that end, the auxiliary head $h_{a}$ is
designed as a lightweight module composed of cascaded 2D CNN layers to predict
the saliency map. It is optimized by minimizing a conventional $L1$ loss on
the predicted salient label $\hat{y}^{(a)}_{k}$:

$\mathcal{L}_{\text{aux}}=\sum_{k=1}^{N_{k}}|y^{(a)}_{k}-\hat{y}^{(a)}_{k}|.$
(4)

#### Joint Multi-task Learning

To build a joint multitask learning pipeline, we formulate the overall
objective function by combining both $L_{\text{prim}}$ and $L_{\text{aux}}$ as

$\mathcal{L}_{\text{train}}=\frac{1}{N}\sum_{1}^{N}\mathcal{L}_{\text{prim}}(x_{i},y^{(p)}_{i};f_{\theta},h_{p})+\lambda\mathcal{L}_{\text{aux}}(x_{i},y^{(a)}_{i};f_{\theta},h_{a}),$
(5)

where $\lambda$ is the balancing hyperparameter.

#### Limitation:

Despite the auxiliary objective essentially providing hard-coded spatial
attention to guide the network being focused on the salient pedestrian object,
this pipeline is intrinsically limited. This is due to the inherent noise in
the weak label of the auxiliary task that brings a detrimental impact on the
primary task and distracts the shared feature extractor from focusing on the
pedestrian region. This has further resulted in a divergent gradient descent
direction, reflected by the conflicting gradients. We intuitively visualize
the cause of interference in Figure 3. Hence, it becomes necessary to perform
a post-operation that resolves the conflicts between the learning objectives.

###  3.3 Association: Referenced Gradient Calibration

During the model training, the learnable parameter $\theta$ of the shared
feature extractor $f_{\theta}$ is updated based on two loss gradients:
${\boldsymbol{g_{p}}=\frac{\partial{L_{\text{prim}}}}{\partial\theta}}$ from
the primary objective and
$\boldsymbol{g_{a}}=\frac{\partial{L_{\text{aux}}}}{\partial\theta}$ from the
auxiliary objective. However, when $\boldsymbol{g_{p}}$ and
$\boldsymbol{g_{a}}$ are in conflict as reflected by a negative inner product,
_i.e._ , $(\boldsymbol{g_{a}}\cdot\boldsymbol{g_{p}})<0$, their joint effort
cannot provide the network with an informative direction on which to perform
the gradient descent to optimize the parameters. Therefore, collectively they
bring significant difficulty in model convergence and can even lead to
destructive interference [37].

Figure 3:  Illustration of the interference to the ReID objective when the
weak saliency label is inaccurate. Each sample is presented with three
columns: the input pedestrian image on the left, the activation from the
primary ReID head in the middle, and the weak label for the auxiliary saliency
detection head on the right. The gradient descent directions for the two
objectives are contradictory.

To address this fundamental limitation, we propose to break through the
dilemma by calibrating the conflicting gradient yield by the auxiliary
objective with that from the primary objective as a reference. Specifically,
When $\boldsymbol{g_{a}}$ is conflicting with $\boldsymbol{g_{p}}$, we
consider $\boldsymbol{g_{p}}$ as a reference and manually alter the direction
of $\boldsymbol{g_{a}}$ by mapping it to the normal plane of
$\boldsymbol{g_{p}}$ to get the calibrated gradient $\boldsymbol{g_{a}^{c}}$
as

$\boldsymbol{g_{a}^{c}}=\boldsymbol{g_{a}}-\frac{\boldsymbol{g_{a}}\cdot\boldsymbol{g_{p}}}{\|\boldsymbol{g_{p}}\|^{2}}\boldsymbol{g_{a}},\qquad\text{subject
to }(\boldsymbol{g_{a}}\cdot\boldsymbol{g_{p}})<0,$ (6)

Remark: This procedure changes the direction of the conflicting gradient to
ensure it does not conflict with the primary task. With the calibrated
gradient, the model can consider the partial guidance of the auxiliary
objective, ensuring the joint effort is non-conflicting with the primary
objective. It is effective in minimizing the side effects caused by the
inaccurate labeling of the auxiliary task while still performing standard
first-order gradient descent to optimize the model.

###  3.4 Deployment-Time Optimization

We further formulate the PAOA+ to exploit the data characteristic of the
target domain and perform deployment time optimization with the available
samples during testing. Considering that the proposed PAOA is composed of a
shared feature encoder $f_{\theta}$ and two separate task heads $h_{p}$ and
$h_{a}$ that are optimized jointly during model training. When the trained
model is deployed in a new environment, given a batch of identity-unknown
samples $\\{x_{i}^{\prime}\\}_{i\in\\{1,\cdots,B^{\prime}\\}}$, with the
corresponding weakly labels $\\{y_{i}^{\prime(a)}\\}$ generated by the pre-
trained saliency detection model, the shared feature extract $f_{\theta}$ can
be further optimized on the auxiliary task by minimizing the following loss

$\mathcal{L}_{\text{test}}=\frac{1}{B}\sum_{1}^{B}\mathcal{L}_{\text{aux}}(x_{i}^{\prime},{y_{i}^{\prime(a)}};f_{\theta}).$
(7)

So that $f_{\theta}$ can be swiftly adapted by considering the data
distribution of the new environment, further to yield improved performance on
the main task. Note the difference from domain adaptation based methods which
assume the test sample is available during the training phase for explicit
distribution alignment, PAOA+ only requires a batch of samples with arbitrary
numbers for on-the-fly updates, allowing it to seamlessly adapt to new data
distributions.

###  3.5 Model Training and Deployment

#### Training stage:

Given the formulation of the primary and auxiliary tasks, the PAOA model is
designed in multitask learning architecture and can benefit from the
conventional learning supervision by jointly minimizing the primary and
auxiliary losses. The parameters are iteratively optimized with the training
loss (Eq. (5)). As the feature extractor parameterized by $\theta$ is shared
by both the primary and auxiliary tasks, it will be jointly updated with two
gradients: $\boldsymbol{g_{p}}$ for the primary task and $\boldsymbol{g_{a}}$
for the auxiliary task. To seek positive interactions between tasks, the
direction of $\boldsymbol{g_{a}}$ will be calibrated only if it conflicts with
$\boldsymbol{g_{p}}$ by Eq. (6). Note that the cross-entropy loss provides
stronger supervision for person classification, therefore we use its gradients
as the reference to calibrate that of the auxiliary task. This calibrated
gradient ensures the auxiliary task is harmoniously trained with the primary
task by back-propagation and thereby brings benefits to facilitate the
deployment-time optimization. The overall training procedure is depicted in
Algorithm 1.

Algorithm 1 Model Training with PAOA regularization

Input: Labeled dataset $\mathcal{D}=\\{(x_{i},y^{(p)}_{i})\\}$ for primary
task, weak label generator $\mathcal{G}$ for auxiliary task, shared feature
extractor $f_{\theta}$, head modules $h_{p}$/$h_{a}$ for primary/auxiliary
tasks.
Output: Trained $f_{\theta}$, $h_{p}$ and $h_{a}$.
for $i=1$ to $max\\_iter$ do
Randomly sample a mini-batch
$\\{(x_{i},y^{(p)}_{i})\\}_{i\in\\{1,\cdots,N_{\text{B}}\\}}$ from source
dataset $\mathcal{D}$.
Generate the weak label for the auxiliary task by
$\\{y_{i}^{(a)}=\mathcal{G}(x_{i})\\}_{i\in\\{1,\cdots,N_{\text{B}}\\}}$.
Compute the training loss (Eq. (5)) and calculate the gradients.
Calibrate the conflicting gradients (Eq. (6)).
Update the network by gradient descent.
end for

#### Deployment stage:

To make a consistent comparison with DG ReID methods, we can directly apply
the trained PAOA model for identity representation extraction. Additionally,
the improved PAOA+ model further performs deployment time optimization during
the testing stage to mitigate the domain shift between the training and
testing domains. Given the identity representations, subsequent identity
retrieval is performed by a general distance metric.

Figure 4: Example identity samples from different domains and its
corresponding weak labels for the auxiliary task. Significant domain gaps are
caused by the variation on nationality, illumination, viewpoints, resolution,
scenario, etc. As complementary, the pedestrian saliency label can provide a
guide on the most discriminative person area.

##  4 Experiment

###  4.1 Comparison with SOTA methods

###  4.2 Experimental Settings

#### Implementation Details

We used PFAN [41] as the wake label generator for the auxiliary task. The
shared feature extractor is a ResNet50 [10] pre-trained on ImageNet [6] to
bootstrap the feature discrimination. The balancing hyper-parameter in Eq. (5)
was set to 0.1. The batch size was set to 64, including 4 images for 16
randomly sampled identities. All images were resized to $128\times 256$. The
model was trained for 200 epochs with the Adam optimizer [17]. The learning
rate was set to $3.5e-4$. The dimension of the extracted identity
representation was set to 2048. The dimension of the saliency map is $64\times
32$. The learning rate for PAOA+ was set to $1e-6$ and the test batch size was
200. The post-optimization step is set to 1 for balancing performance and
efficiency. All the experiments were implemented on PyTorch [27] on a single
A100 GPU.

#### Datasets and Evaluation Protocol

We conducted multi-source domain generalized ReID on a wide range of
benchmarks. including Market1501 (M) [43], MSMT17 (MS) [34], CUHK03 (C3) [20],
CUHK-SYSU (CS) [35], CUHK02 (C2) [19], VIPeR [8], PRID [12], GRID [24], and
iLIDs [44]. We evaluated the performance of PAOA on the four small-scale
datasets following the traditional setting [30, 15, 2, 38]. We also performed
leave-one-out evaluations by using three datasets for training and the
remaining for the test [42, 4, 22]. Note that the CUHK-SYSU is only for
training given all the images are captured by the same camera. To learn a
discriminative model benefits from diverse identities, all the identities
regardless of the original train/test splits, were used for training. We
adopted Mean average precision (mAP) and Rank-1 of CMC as the evaluation
metrics.

Table 1: Comparison with the SOTA methods on traditional evaluation protocol. The best results are shown in red and the second-best results are shown in blue.  |  | PRID | GRID | VIPeR | iLIDs | Average
---|---|---|---|---|---|---
Source | Method | mAP | R1 | mAP | R1 | mAP | R1 | mAP | R1 | mAP | R1
| DIMN [30] | 52.0 | 39.2 | 41.1 | 29.3 | 60.1 | 51.2 | 78.4 | 70.2 | 57.9 | 47.5
| SNR [16] | 66.5 | 52.1 | 47.7 | 40.2 | 61.3 | 52.9 | 89.9 | 84.1 | 66.3 | 57.3
M+D+C2 +C3+CS | DMG-Net [2] | 68.4 | 60.6 | 56.6 | 51.0 | 60.4 | 53.9 | 83.9 | 79.3 | 67.3 | 61.2
| M3L [42] | 64.3 | 53.1 | 55.0 | 44.4 | 66.2 | 57.5 | 81.5 | 74.0 | 66.8 | 57.2
| MetaBIN [4] | 70.8 | 61.2 | 57.9 | 50.2 | 64.3 | 55.9 | 82.7 | 74.7 | 68.9 | 60.5
| ACL [38] | 73.5 | 63.0 | 65.7 | 55.2 | 75.1 | 66.4 | 86.5 | 81.8 | 75.2 | 66.6
| META [36] | 71.7 | 61.9 | 60.1 | 52.4 | 68.4 | 61.5 | 83.5 | 79.2 | 70.9 | 63.8
| PAOA (Ours) | 74.0 | 65.6 | 67.2 | 56.3 | 76.6 | 66.7 | 87.1 | 83.1 | 76.2 | 67.9
M+C2+ C3+CS | PAOA+ (Ours) | 75.1 | 66.5 | 67.8 | 56.9 | 77.2 | 67.7 | 88.0 | 83.9 | 77.0 | 68.8
Table 2: Comparison with the SOTA methods on large-scale evaluation protocol. The best results are shown in red and the second-best results are shown in blue.  |  | M+MS+CS→C3 | M+CS+C3→MS | MS+CS+C3→M | Average
---|---|---|---|---|---
Method | Reference | mAP | R1 | mAP | R1 | mAP | R1 | mAP | R1
SNR [16] | CVPR2020 | 17.5 | 17.1 | 7.7 | 22.0 | 52.4 | 77.8 | 25.9 | 39.0
QAConv50 [22] | ECCV2020 | 32.9 | 33.3 | 17.6 | 46.6 | 66.5 | 85.0 | 39.0 | 55.0
M3L [42] | CVPR2021 | 35.7 | 36.5 | 17.4 | 38.6 | 62.4 | 82.7 | 38.5 | 52.6
MetaBIN [4] | CVPR2021 | 43.0 | 43.1 | 18.8 | 41.2 | 67.2 | 84.5 | 43.0 | 56.3
ACL [38] | ECCV2022 | 49.4 | 50.1 | 21.7 | 47.3 | 76.8 | 90.6 | 49.3 | 62.7
META [36] | ECCV2022 | 47.1 | 46.2 | 24.4 | 52.1 | 76.5 | 90.5 | 49.3 | 62.9
PAOA | Ours | 49.8 | 50.5 | 25.1 | 51.5 | 77.1 | 90.8 | 50.7 | 64.3
PAOA+ | Ours | 50.3 | 50.9 | 26.0 | 52.8 | 77.9 | 91.4 | 51.4 | 65.0

We compared the proposed PAOA against several recent SOTA methods, and the
comparison results are shown in Table 1 and Table 2. Under a fair comparison
with existing DG ReID methods, the PAOA model outperforms all the competing
methods by a significant margin on both the traditional setting and the large-
scale settings across all the evaluation metrics. It shows a clear advantage
over the recent SOTA methods. Notably, even trained with fewer datasets
compared with [30, 16, 2], the proposed method is still able to extract
discriminative features for identity matching. Besides, we extended our
analysis to include the results from the test-time optimization variant,
PAOA+, which notably improves PAOA consistently across all benchmarks. These
results provide additional evidence on the effectiveness of the associative
learning strategy, where the auxiliary task can promote the primary ReID
objective during test time given the absence of identity labels.

###  4.3 Ablation Studies

Table 3: Effects on mAP (%) value of the proposed modules. Aux: auxiliary
objective. GC: gradient calibration. DTO: deployment-time optimization.

Aux | GC | DTO | C3 | MS | M | Average
---|---|---|---|---|---|---
✗ | ✗ | ✗ | 42.8 | 20.5 | 73.1 | 45.5
✓ | ✗ | ✗ | 44.8 | 20.9 | 73.5 | 46.4
✓ | ✗ | ✓ | 47.0 | 23.1 | 75.2 | 48.4
✓ | ✓ | ✗ | 49.8 | 25.1 | 77.1 | 50.7
✓ | ✓ | ✓ | 50.3 | 26.0 | 77.9 | 51.4

#### Component Analysis

We investigated the effects of different components in PAOA model design to
study their individual contributions. The baseline model is a ResNet50 pre-
trained on ImageNet. The comparison results are shown in Table 3, from which
we can observe that the auxiliary objective and the gradient calibration
strategies can consistently improve performance. With further deployment-time
optimization, our model can be advanced by benefiting from mining the data
characteristics in the target domain. It is notable that the variant without
gradient calibration can always benefit more from that post-optimization
compared with the PAOA+ model, This further illustrates that the referenced
calibration mechanism has already enabled the PAOA model to be more attentive
to the domain-invariant pedestrian region, and therefore it relies less on on-
the-fly optimization.

Table 4: Effects on mAP (%s) of update iterations during deployment
optimization.

Dataset | 0 | 1 | 2 | 3 | 4
---|---|---|---|---|---
C3 | 49.8 | 50.3 | 50.5 | 50.6 | 50.3
MS | 25.1 | 26.0 | 26.5 | 26.0 | 25.0
M | 77.1 | 77.9 | 77.5 | 77.0 | 76.2
Avg. | 50.7 | 51.4 | 51.5 | 51.2 | 50.5

#### Gradient Calibration Designs

We adopted a primary-referenced design for the gradient calibration between
the primary and auxiliary objectives. This was based on the fact that the
primary instance classification objective provides stronger supervision to
identify pedestrians, while the auxiliary objective is to guide the instance
classifier to attentively focus on the pedestrian area and ignore the domain-
specific interference. It’s weakly labeled and therefore is intricately noisy
which can lead to a negative influence on the primary objective, reflected by
the conflicting gradient. We examined the effect of the calibration design by
additionally testing three more formulations as demonstrated in Figure 5.
Table 5 shows the auxiliary-referenced design yielded the worst performance,
given the gradients of the auxiliary objective is noisy and unreliable, using
it as the reference is harmful to the learning of the primary objective. By
contrast, the mutually referenced calibration design includes the primary
gradients as referenced on top of the auxiliary-referenced design, which
alleviates the fallout caused by the gradient destruction, despite it’s still
inferior to the baseline. In comparison, the primary-referenced design
consistently obtained improved performance which supports the design of the
proposed primary referenced gradient calibration.

Figure 5: Illustration of different gradient calibration designs. (a) No
gradient calibration as [29]. (b) Gradients of the primary objective are
calibrated with the auxiliary objective as a reference. (c) Gradients are
calibrated in relation to each other as a reference, as designed in [37]. (d)
Gradients of the auxiliary objective are calibrated with the primary objective
as a reference. Table 5: Comparison of different gradient calibration designs
by mAP (%). Refer to Figure 5 for the corresponding design.

Design | C3 | MS | M | Avg.
---|---|---|---|---
a | 44.8 | 20.9 | 73.5 | 46.4
b | 44.1 | 21.7 | 74.7 | 46.8
c | 47.3 | 23.0 | 75.3 | 48.5
d | 49.8 | 25.1 | 77.1 | 50.7

#### Update iterations for deployment-time optimization

We analyzed the influence of update iterations for optimizing the model with
all test samples in deployment time. Ablating with iterations from 0 to 4
(Table 4), we noted consistent performance improvement by updating the model
at the initial steps. This is attributed to the auxiliary objective guiding
swift adaptation to the test domain. This improvement is attributed to the
auxiliary objective facilitating rapid adaptation to the test domain. However,
excessive updates result in a model forgetting issue by overwhelming the
extractor with the auxiliary. Notably, deployment-time optimization is more
effective for target datasets (_i.e._ , MSMT17) with larger domain shifts,
which further proves that target-aware updates that mitigate domain shifts
more effectively. Balancing efficiency and effectiveness, PAOA+ adopts single-
step updates across all datasets to attain the global optimal solution.

#### Visualization

We visualized the pedestrian images and the model activation maps to
intuitively illustrate the effectiveness of PAOA. We took the feature map of
the final convolutional layer ($4th$ layer) as the activation map, and
compared the baseline model with the proposed PAOA. As can be observed in
Figure 6, the PAOA model can accurately be attentive to the pedestrian area,
while the baseline model is partially focus and some discriminative areas are
missed. This is benefited from the auxiliary objective, as shown in the second
column, which provides assistive supervision on instance classification
learning. Therefore, PAOA model can extract discriminative yet generic
identity representation for ReID. To also visualized the TSNE distribution of
the extracted feature representations in Figure 7. The target domain is
Market1501 and the model was trained with other three source domains. Training
independently with the auxiliary objective can condense the feature space
compared with the baseline, however it’s still prone to domain shift,
especially for CUHK03. As a comparison, the proposed PAOA can significantly
reduce domain shifts with a much more compact feature space.

Figure 6: Visualization of activation maps. For each pedestrian image, the
four columns from left to right are: (1) Person image, (2) Weak label for
auxiliary objective, (3) Activation map from the proposed PAOA model, (4)
Activation map from the baseline. The proposed PAOA helps the model be more
attentive on the pedestrian region to learning domain-invariant
representation.

Figure 7: TSNE visualization on extracted features. 200 samples were randomly
sampled from each domain. Learning with jointly the primary and auxiliary
objectives can condense the feature distribution. The proposed model which
associates the primary and auxiliary objectives can derive a more compact
feature space.

##  5 Conclusions

In this work, we introduced a novel Primary-Auxiliary Objectives Association
(PAOA) regularization to learn a generalizable ReID model for extracting
domain-unbiased representations more generalizable to unseen novel domains for
person ReID. PAOA encourages the model to get rid of the interference of
domain-specific knowledge and to learn from discriminative pedestrian
information by the association of learning an auxiliary pedestrian detection
objective with a primary instance classification objective. To mitigate the
fallout caused by the noisy auxiliary labels, we further derive a referenced-
gradient calibration strategy to alter the gradient of the auxiliary object
when it’s conflicting with the primary object. The PAOA framework is task-
agnostic, making it readily adaptable to other tasks through the incorporation
of a close auxiliary task and a shared learning module.

## References

  * [1] Eugene PW Ang, Lin Shan, and Alex C Kot.  Dex: Domain embedding expansion for generalized person re-identification.  In BMVC, 2021.
  * [2] Yan Bai, Jile Jiao, Wang Ce, Jun Liu, Yihang Lou, Xuetao Feng, and Ling-Yu Duan.  Person30k: A dual-meta generalization network for person re-identification.  In CVPR, 2021.
  * [3] Ali Borji, Ming-Ming Cheng, Qibin Hou, Huaizu Jiang, and Jia Li.  Salient object detection: A survey.  Computational visual media, 5(2), 2019.
  * [4] Seokeon Choi, Taekyung Kim, Minki Jeong, Hyoungseob Park, and Changick Kim.  Meta batch-instance normalization for generalizable person re-identification.  In CVPR, 2021.
  * [5] Yongxing Dai, Xiaotong Li, Jun Liu, Zekun Tong, and Ling-Yu Duan.  Generalizable person re-identification with relevance-aware mixture of experts.  In CVPR, 2021.
  * [6] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In CVPR, 2009.
  * [7] Cian Eastwood, Ian Mason, Christopher KI Williams, and Bernhard Schölkopf.  Source-free adaptation to measurement shift via bottom-up feature restoration.  In ICLR, 2022.
  * [8] Douglas Gray and Hai Tao.  Viewpoint invariant pedestrian recognition with an ensemble of localized features.  In ECCV, 2008.
  * [9] Ke Han, Chenyang Si, Yan Huang, Liang Wang, and Tieniu Tan.  Generalizable person re-identification via self-supervised batch norm test-time adaption.  In AAAI, 2022.
  * [10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, 2016.
  * [11] Lingxiao He and Wu Liu.  Guided saliency feature learning for person re-identification in crowded scenes.  In ECCV, 2020.
  * [12] Martin Hirzer, Csaba Beleznai, Peter M Roth, and Horst Bischof.  Person re-identification by descriptive and discriminative classification.  In Scandinavian conference on Image analysis, 2011.
  * [13] Yusuke Iwasawa and Yutaka Matsuo.  Test-time classifier adjustment module for model-agnostic domain generalization.  NeurIps, 2021.
  * [14] Jieru Jia, Qiuqi Ruan, and Timothy M Hospedales.  Frustratingly easy person re-identification: Generalizing person re-id in practice.  In BMVC, 2019.
  * [15] Xin Jin, Cuiling Lan, Wenjun Zeng, and Zhibo Chen.  Feature alignment and restoration for domain generalization and adaptation.  arXiv, 2020.
  * [16] Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen, and Li Zhang.  Style normalization and restitution for generalizable person re-identification.  In CVPR, 2020.
  * [17] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In ICLR, 2015.
  * [18] Qilei Li, Jiabo Huang, and Shaogang Gong.  Local-global associative frame assemble in video re-id.  BMVC, 2021.
  * [19] Wei Li and Xiaogang Wang.  Locally aligned feature transforms across views.  In CVPR, 2013.
  * [20] Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang.  Deepreid: Deep filter pairing neural network for person re-identification.  In CVPR, 2014.
  * [21] Wei Li, Xiatian Zhu, and Shaogang Gong.  Harmonious attention network for person re-identification.  In CVPR, 2018.
  * [22] Shengcai Liao and Ling Shao.  Interpretable and generalizable person re-identification with query-adaptive convolution and temporal lifting.  In ECCV, 2020.
  * [23] Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi.  Ttt++: When does self-supervised test-time training fail or thrive?  In NeuIPS, 2021.
  * [24] Chen Change Loy, Tao Xiang, and Shaogang Gong.  Time-delayed correlation analysis for multi-camera activity understanding.  IJCV, 2010.
  * [25] Chuanchen Luo, Chunfeng Song, and Zhaoxiang Zhang.  Generalizing person re-identification by camera-aware invariance learning and cross-domain mixup.  In ECCV, 2020.
  * [26] Divyat Mahajan, Shruti Tople, and Amit Sharma.  Domain generalization using causal matching.  In ICML, 2021.
  * [27] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in pytorch.  In NIPS-W, 2017.
  * [28] Ozan Sener and Vladlen Koltun.  Multi-task learning as multi-objective optimization.  Advances in neural information processing systems, 31, 2018.
  * [29] Chunfeng Song, Yan Huang, Wanli Ouyang, and Liang Wang.  Mask-guided contrastive attention model for person re-identification.  In CVPR, 2018.
  * [30] Jifei Song, Yongxin Yang, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales.  Generalizable person re-identification by domain-invariant mapping network.  In CVPR, 2019.
  * [31] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt.  Test-time training with self-supervision for generalization under distribution shifts.  In ICML, 2020.
  * [32] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno A Olshausen, and Trevor Darrell.  Fully test-time adaptation by entropy minimization.  In ICLR, 2021.
  * [33] Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai.  Continual test-time domain adaptation.  In CVPR, 2022.
  * [34] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian.  Person transfer gan to bridge domain gap for person re-identification.  In CVPR, 2018.
  * [35] Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, and Xiaogang Wang.  End-to-end deep learning for person search.  arXiv, 2016.
  * [36] Boqiang Xu, Jian Liang, Lingxiao He, and Zhenan Sun.  Mimic embedding via adaptive aggregation: Learning generalizable person re-identification.  In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIV, pages 372–388. Springer, 2022.
  * [37] Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn.  Gradient surgery for multi-task learning.  NeurIps, 2020.
  * [38] Pengyi Zhang, Huanzhang Dou, Yunlong Yu, and Xi Li.  Adaptive cross-domain learning for generalizable person re-identification.  In ECCV, 2022.
  * [39] Yu Zhang and Qiang Yang.  A survey on multi-task learning.  IEEE Transactions on Knowledge and Data Engineering, 2021.
  * [40] Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Xin Jin, and Zhibo Chen.  Relation-aware global attention for person re-identification.  In CVPR, 2020.
  * [41] Ting Zhao and Xiangqian Wu.  Pyramid feature attention network for saliency detection.  In CVPR, 2019.
  * [42] Yuyang Zhao, Zhun Zhong, Fengxiang Yang, Zhiming Luo, Yaojin Lin, Shaozi Li, and Nicu Sebe.  Learning to generalize unseen domains via memory-based multi-source meta-learning for person re-identification.  In CVPR, 2021.
  * [43] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian.  Scalable person re-identification: A benchmark.  In ICCV, 2015.
  * [44] Wei-Shi Zheng, Shaogang Gong, and Tao Xiang.  Associating groups of people.  In BMVC, 2009.
  * [45] Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, and Jan Kautz.  Joint discriminative and generative learning for person re-identification.  In CVPR, 2019.
  * [46] Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy.  Domain generalization in vision: A survey.  arXiv, 2021.
  * [47] Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang.  Learning to generate novel domains for domain generalization.  In ECCV, 2020.
  * [48] Kuan Zhu, Haiyun Guo, Zhiwei Liu, Ming Tang, and Jinqiao Wang.  Identity-guided human semantic parsing for person re-identification.  In ECCV, 2020.
  * [49] Zijie Zhuang, Longhui Wei, Lingxi Xie, Haizhou Ai, and Qi Tian.  Camera-based batch normalization: an effective distribution alignment method for person re-identification.  IEEE Transactions on Circuits and Systems for Video Technology, 32(1), 2021.
  * [50] Zijie Zhuang, Longhui Wei, Lingxi Xie, Tianyu Zhang, Hengheng Zhang, Haozhe Wu, Haizhou Ai, and Qi Tian.  Rethinking the distribution gap of person re-identification with camera-based batch normalization.  In ECCV, 2020.


# Nonreciprocal optical solitons in a spinning Kerr resonator

Baijun Li  Key Laboratory of Low-Dimensional Quantum Structures and Quantum
Control of Ministry of Education, Department of Physics and Synergetic
Innovation Center for Quantum Effects and Applications, Hunan Normal
University, Changsha 410081, China  Şahin. K. Özdemir  Department of
Engineering Science and Mechanics, and Materials Research Institute,
Pennsylvania State University, University Park, Pennsylvania 16802, USA  Xun-
Wei Xu  Key Laboratory of Low-Dimensional Quantum Structures and Quantum
Control of Ministry of Education, Department of Physics and Synergetic
Innovation Center for Quantum Effects and Applications, Hunan Normal
University, Changsha 410081, China  Lin Zhang  School of Physics and
Information Technology, Shaanxi Normal University, Xi’an 710061, China  Le-Man
Kuang<EMAIL_ADDRESS>Key Laboratory of Low-Dimensional Quantum
Structures and Quantum Control of Ministry of Education, Department of Physics
and Synergetic Innovation Center for Quantum Effects and Applications, Hunan
Normal University, Changsha 410081, China  Hui Jing<EMAIL_ADDRESS>Key
Laboratory of Low-Dimensional Quantum Structures and Quantum Control of
Ministry of Education, Department of Physics and Synergetic Innovation Center
for Quantum Effects and Applications, Hunan Normal University, Changsha
410081, China

###### Abstract

We propose a spinning nonlinear resonator as an experimentally accessible
platform to achieve nonreciprocal control of optical solitons. Nonreciprocity
here results from the relativistic Sagnac-Fizeau optical drag effect, which is
different for pump fields propagating in the spinning direction or in the
direction opposite to it. We show that in a spinning Kerr resonator, different
soliton states appear for the input fields in different directions. These
nonreciprocal solitons are more stable against losses induced by inter-modal
coupling between clockwise and counterclockwise modes of the resonator. Our
work builds a bridge between nonreciprocal physics and soliton science,
providing a promising route towards achieving soliton-wave optical isolators
and one-way soliton communications.

##  I Introduction

Solitons are stable waveforms preserving their shape or energy during
propagation. They widely exist in natural and artificial systems
Hasegawa95Solitons ; Haus96Solitons ; Zabusky65Interaction ;
Mollenauer80Experimental ; Strecker02Formation ; Ackemann09Fundamentals , and
are indispensable in applications such as broadband spectroscopy,
telecommunications, astronomy and low noise microwave generation. In recent
years, rapid advances have been witnessed in creating and utilizing
dissipative Kerr solitons in optical microresonators Herr14Temporal ;
Yi15Soliton ; Brasch16Photonic ; Suh16Microresonator ; Yang17Counter ;
Obrzud17Temporal ; Yi17Single ; Fujii17Effect ; Suh18Gigahertz ; Weng19
Polychromatic ; Karpov19Dynamics ; Shu20A ; Wan20Frequency ; Fan20lSoliton ;
Shen20Integrated ; Runge20The ; Liu20Photonic ; Liu20Monolithic ; Weng20Gain ;
Englebert21Parametrically ; Bao21Quantum . Very recently, such solitons have
been discovered in a high-Q multimode photonic dimer (pair of strongly-
coupled, almost identical nonlinear resonators) Xue19Super ; Tikan20Emergent ;
Xu20Spontaneous ; Komagata21Komagata ; Helgason21Dissipative , revealing a
pleiad of emergent phenomena including gear solitons Tikan20Emergent  and
solitons operated at an exceptional point Komagata21Komagata . These solitons
feature a delicate balance between not only dispersion and nonlinearity but
also loss and gain Kippenberg18Dissipative . Pioneering works in these fields
have provided an attractive platform to build compact and low-power frequency
combs that exhibit femtosecond pulses at tens of gigahertz repetition rates
Kippenberg18Dissipative ; Gaeta19Photonic ; Fortier1920 , which may enable a
wide range of key technologies such as chip-scale clocks, integrated frequency
synthesizer Spencer18An , and ultrafast laser ranging Zhu18Dual ;
Trocha18Ultrafast ; Suh18Soliton ; Riemensberger20Massively ; Wang20Long .

Figure 1: Schematic illustration of nonreciprocal solitons. Different Sagnac
frequency shifts $\Delta_{\mathrm{F}}$ are induced for countercirculating
modes in a spinning Kerr microresonator. (a,b) Solitons appear for the input
laser in one direction but not the other. (c) The Sagnac-Fizeau shift
$\Delta_{\mathrm{F}}$ versus the angular velocity $\Omega$. Increasing the
angular velocity results in a linear opposite frequency shifts for the
counterpropagating modes, leading to the formation of nonreciprocal solitons.

In parallel, nonreciprocal optical devices, allowing one-way flow of
information and thus playing a key role in backaction-immune communications,
have been achieved in nonlinear systems Shen16Experimental ; Shi15Limitations
; Fan11An ; Xia18Cavity ; Scheucher16Quantum ; Malz18Quantum ;
Xu20Nonreciprocity ; Lai20Nonreciprocal ; Sounas17Non-reciprocal , non-
Hermitian systems Peng14Parity ; Chang14Parity ; Arkhipov19Scully , and
spinning resonators Maayani18Flying ; Lu17Optomechanically . Here we study
nonreciprocal control of solitons, which is the first step towards achieving
soliton isolators or routers Yang20Inverse  and chiral soliton combs
Tang20Gain . In contrast to recent works on chiral soliton control using
optical gain or dual pumps Yang17Counter ; Fan20lSoliton ; Tang20Gain , our
scheme utilizes another degree of freedom, i.e., the mechanical spinning of
the resonator Maayani18Flying . We note that very recently, mechanical
rotation has been used in diverse fields such as sound circulation
Fleury14Sound , thermal control Qiang20Tunable , one-way quantum optical
engineering Jiang18Nonreciprocal ; Huang18Nonreciprocal ; Li19Nonreciprocal ;
Jiao20Nonreciprocal  and nanoparticle sensing Jing18Nanoparticle ;
Ahn20Ultrasensitive ; Zhang20Anti . In this work, we find that in a spinning
nonlinear resonator, nonreciprocal solitons can emerge, and exist stably even
in the presence of strong inter-modal coupling between clockwise (CW) and
counter-clockwise (CCW) modes. Our work is also well complementary to other
methods of engineering chiral solitons Tang20Gain ; Fan20lSoliton . In fact,
by combining optical and mechanical techniques, even more flexible one-way
control of solitons can be expected, with practical applications in making and
utilizing nonreciprocal soliton devices Haus96Solitons ; Corcoran20Ultra ;
Parriaux20Electro ; Diddams20Optical .

##  II Solitons in a spinning resonator

The spinning resonator used as the platform in this study can be fabricated by
melting the end of a silica glass cylinder and then mounting it on a turbine,
as shown in a recent experiment Maayani18Flying . A tapered fiber fabricated
from a standard single-mode telecommunications fibre through heat-and-pull
method is positioned near this rotating sphere to couple light evanescently
into or out of the resonator. Because of the Fizeau drag, the optical paths of
counterpropagating lights in the resonator are different, resulting in an
irreversible refractive index for the CW and CCW modes. As a result, a
$99.6\%$ optical isolation can be realized with such a device at the spinning
frequency $\Omega=6.6\mathrm{kHz}$ Maayani18Flying . For such a spinning
device, the rapid rotation drags air into the region between the resonator and
the fiber, forming a thin layer of air, which helps to stabilize the tapered
fiber at a height of several nanometers above the resonator. When there is
external disturbance that causes the taper to rise above the stable
equilibrium height, the taper will float back and self-adjust itself to its
original position Maayani18Flying . The air pressure acting on the fiber is
$\Delta T_{air}=(\rho\Delta\theta)T_{air}/L$ which leads to a tiny
displacement $d$, where $\rho(\theta)$ indicates the radius (angle) of the
winding shape of the deformed region of the fiber, and the whole air pressure
on the fiber is Maayani18Flying

$\displaystyle T_{air}=6.19\mu
R^{5/2}\Omega\int_{0}^{r}\left(h-\sqrt{r^{2}-x^{2}}+r\right)^{-3/2}dx,$ (1)

where $\mu$ is the viscosity of air, $r(R)$ is the radius of the taper
(resonator), and $h=h_{0}+d$ represents the taper-resonator separation, with
the stationary gap between the fiber and the stationary sphere, $h_{0}$. The
tension on this infinitesimal cylinder due to the local deformation of the
fiber is estimated as

$\displaystyle\Delta T_{en}=2F\mathrm{sin}\left(\Delta\theta/2\right)\approx
F\Delta\theta,$ (2)

where $F$ represents the elastic force on the taper and obeys the Hooke law,
$\sigma=\epsilon E$. The $\sigma=F/(\pi r^{2})$ is the uniaxial stress, $E$ is
the Young modulus of silica, and $\epsilon=\delta_{L}/L$ is the strain with
$\delta_{L}=L^{{}^{\prime}}-L$ denotes the length variation between the
original length $L$ and the deformation region $L^{{}^{\prime}}$. In the case
of stable equilibrium, i.e., $\Delta T_{air}=\Delta T_{en}$, we obtain

$\displaystyle T_{air}=2\pi
r^{2}E\left[\mathrm{arcsin}(\phi)-\phi\right]\approx\pi r^{2}\phi^{3}/3,$ (3)

where $\phi=4Ld/(L^{2}+4d^{2})$, and we have used the approximation for
$|\phi|\ll 1$, $\mathrm{arcsin}(\phi)=\phi+\phi^{3}/6+\cdots$. Therefore, the
displacement $d$ can be estimated as

$\displaystyle d=\frac{L}{2}\left(\tau-\sqrt{\tau^{2}-1}\right)/2,$ (4)

where $\tau=[\pi r^{2}E/(3T_{air})]^{1/3}$. Then, the strain of the taper can
be rewritten as $\epsilon=\mathrm{arcsin}(\phi)/\phi-1\approx\phi^{2}/6$. We
find that the strain (i.e., the elastic force) is positively associated with
the taper-resonator separation:

$\displaystyle\frac{\partial F}{\partial h}=\pi
r^{2}E\left(\frac{\partial\epsilon}{\partial d}\right)=\frac{16\pi
r^{2}EL^{2}d(L^{2}-4d^{2})}{3(L^{2}+4d^{2})}>0,$ (5)

i.e., the elastic force becomes stronger when the air gap gets larger than the
stable-equilibrium distance. As a result, the taper is dragged back to its
equilibrium position, enabling separation of the taper from the spinning
resonator and critical coupling of light into the cavity. We remark that even
if the taper is pushed towards the rotating resonator, the taper will not
contact or stick to the resonator, which is opposite to the case observed for
a static resonator (i.e., the taper may stick to the resonator through van der
Waals forces and thus needs to be pulled back to break the connection).

Figure 2: Optical solitons with different numbers in different directions.
(a-g) Three-dimensional representations of optical field energy evolution of
the soliton mode $|\psi_{i}|^{2}$ in the azimuthal direction at $\tau=200$ and
(h) corresponding Kerr combs in a static and a spinning Kerr microresonator
without backscattering. For $\Delta_{\mathrm{F}}=0$, the solitons emerge with
same numbers in different driving directions. For $\Delta_{\mathrm{F}}\neq 0$,
the detuning $\Delta_{\mathrm{p}}$ for the countercirculating modes are
modified by the opposite Sagnac shifts, resulting in solitons with different
numbers in different directions. Here, we have shown three different types of
nonreciprocal solitons in different parameter ranges. The initial conditions
of the soliton optical fields are Gaussian pulses: (a-c,g-h)
$\psi_{0}=0.5+\mathrm{exp}[-(\theta/0.1)^{2}]$, (d-f)
$\psi_{0}=0.5+\mathrm{exp}[-(\theta/0.55)^{2}]$, the rotation velocity (b,c)
$\Omega=2.0~{}\mathrm{kHz}$, (e,f) $\Omega=350~{}\mathrm{Hz}$, (g,h)
$\Omega=2.2~{}\mathrm{kHz}$, the dimensionless detuning (a-f)
$\Delta_{\mathrm{p}}=2$, (g,h) $\Delta_{\mathrm{p}}=2.3$. We used normalized
dispersion $\beta=-0.004$, scaled pump intensity $F_{i}=1.37$ in the
simulations. The figures don’t display the $xyz$ coordinate axis for
convenience since the numbers of solitons can be seen from the waveform.

After confirming the mechanical and optical stabilities of the spinning
device, we consider the driving laser with frequency $\omega_{d}$ from the
left or the right via evanescent field of the optical fiber Spillane03Ideality
(see Fig. 1). The CW and CCW modes of the resonator are coupled to each other
via the backscattering process, that may be induced due to scattering centers
or material inhomogeneity formed during fabrication. In the spinning
resonator, the frequency of incident laser changes slightly, i.e.,
$\omega_{\pm}-\omega=\pm\Omega Rn\omega/c,$ where the subscripts $\pm$ denote
the light propagating against or along the spinning direction, $c$ is the
speed of light in vacuum, and $n$ and $R$ are the refractive index and the
radius of the resonator, respectively. Since $n(\omega_{\pm})\approx
n(\omega)+[n(\omega_{\pm})-\omega]\alpha$ and $\alpha=dn(\omega)/d\omega$, in
view of the Lorentz transformation, the speed of light in the spinning
resonator can be written as $c_{\pm}=({u_{\pm}\pm\Omega R})/({1\pm\Omega
Ru_{\pm}/c^{2}})$, with

$u_{\pm}=\frac{c}{n(\omega_{\pm})}\approx\frac{c}{n}\mp\frac{\omega}{n}\frac{dn}{d\omega}\Omega
R.$ (6)

The optical frequencies of the counter-propagating light fields are then given
as

$\displaystyle\nu_{\pm}=\frac{Nc_{\pm}}{2\pi R}\approx\frac{N}{2\pi
R}\left[\frac{c}{n}\pm\Omega
R\left(1-\frac{1}{n^{2}}-\frac{\omega}{n}\frac{dn}{d\omega}\right)\right],$
(7)

with $N=1,2,3,\cdots,$ leading to the Sagnac shifts [see Fig. 1(c)]
Malykin00The ; Maayani18Flying , i.e.,

$\displaystyle\omega_{0}\rightarrow\omega_{0}\pm\Delta_{\mathrm{F}},$ (8)

with

$\Delta_{\mathrm{F}}=2\pi(v_{\pm}-v_{0})\approx\omega_{0}\frac{n\Omega
R}{c}\left(1-\frac{1}{n^{2}}-\frac{\lambda}{n}\frac{dn}{d\lambda}\right),$ (9)

where $v_{0}=Nc/(2\pi nR)$ and $\omega_{0}=2\pi c/\lambda$ is the pump
frequency for the static resonator. The dispersion term
${\mathrm{d}n}/{\mathrm{d}\lambda}$ that characterizes the relativistic origin
of the Sagnac effect is relatively small in typical materials ($\sim 1\%$)
Malykin00The ; Maayani18Flying . The light drag can be further enhanced by
dispersion, as demonstrated also in an experiment using a moving microcavity
Qin20Fast . Here, for convenience, we consider only the CW rotation of the
microresonator. Also we focus on the fundamental modes denoted by an integer
wave number $l$ around the eigennumber $l_{0}$ of the pump. The
eigenfrequencies of these modes can then be expanded as Herr14Temporal

$\displaystyle\omega_{l}=\omega_{0}+\sum_{n=1}^{N}\frac{\xi_{n}}{n!}(l-l_{0})^{n},$
(10)

where $\xi_{1}=d\omega/dl|_{l=l_{0}}=c/(nR)$ is the free spectral range and
$\xi_{2}=d^{2}\omega/dl^{2}|_{l=l_{0}}$ is the second-order dispersion
coefficient responsible for the asymmetric eigenfrequencies.

Figure 3: Soliton suppression due to random-defect-induced backscattering, and
its revival resulting from the rotation-induced compensation. (a-f) Three-
dimensional representations of the optical field energy evolution of the
soliton mode $|\psi_{i}|^{2}$ in the azimuthal direction at $\tau=200$ in a
static and a spinning microresonator when backscattering ($J=1$) is
considered. (g) The transient dynamics of the soliton mode $|\psi_{i}|^{2}$ at
$\tau=200$ as the function of azimuthal angle $\theta$ and the detuning of CCW
mode $\alpha_{2}$ in the dual pump system as reported in Ref. Yang17Counter
when backscattering ($J=1$) is considered. For $\Delta_{\mathrm{F}}=0$,
solitons are suppressed for increased values of backscattering $J=1$. For
$\Omega=60~{}\mathrm{kHz}$, the soliton can exist stably. The initial
conditions of the soliton optical fields are Gaussian pulses: (a-c,g)
$\psi_{0}=0.5+\mathrm{exp}[-(\theta/0.1)^{2}]$, (d-f)
$\psi_{0}=0.5+\mathrm{exp}[-(\theta/0.55)^{2}]$, the rotation velocity
$\Omega=60~{}\mathrm{kHz}$, and the normalized detuning (a,d)
$\Delta_{\mathrm{p}}=2.0$, (b,e) $\Delta_{\mathrm{p}}=-6.9$, (c,f)
$\Delta_{\mathrm{p}}=11$. Other parameters are the same as Fig. 2 and we have
set the dimensionless driving strength $F_{i}=F_{j}=1.37$ and the scaled
detuning of CW mode $\alpha_{1}=2.2$ in the dual pump system simulations.

Here we consider conventional solitons created in a Kerr resonator, the
dynamics of which can be well described by the spatiotemporal Lugiato-Lefever
equation Lugiato87Spatial ; Chembo13Spatiotemporal ; Kondratiev20Modulational
; Lobanov20Generation ; Skryabin20Hierarchy , with the mode coupling term as
used in the experimental works Yang17Counter ; Yi17Single ; Fujii17Effect ;
Bao21Quantum ,

$\displaystyle\frac{\partial\psi_{i}}{\partial\tau}=$
$\displaystyle-[1+i(\Delta_{\mathrm{p}}-\Delta_{\mathrm{sag}})]\psi_{i}+i(|\psi_{i}|^{2}+2|\psi_{j}|^{2})\psi_{i}$
$\displaystyle-i\frac{\beta}{2}\frac{\partial^{2}\psi_{i}}{\partial\theta^{2}}+iJ\psi_{j}+F_{i},$
$\displaystyle\frac{\partial\psi_{j}}{\partial\tau}=$
$\displaystyle-[1+i(\Delta_{\mathrm{p}}+\Delta_{\mathrm{sag}})]\psi_{j}+i(|\psi_{j}|^{2}+2|\psi_{i}|)\psi_{j}$
$\displaystyle-i\frac{\beta}{2}\frac{\partial^{2}\psi_{j}}{\partial\theta^{2}}+iJ\psi_{i},$
(11)

where $\psi_{i}$ and $\psi_{j}$ denote the normalized field amplitudes of the
pump and the backscattering, respectively. The backscattering process, as
observed in the experiments Weiss95Splitting ; Kippenberg02Modal ;
Yoshiki15Observation , is denoted by the strength $J_{0}$;
$\theta\in[-\pi,\pi]$ is the azimuthal angle along the circumference of the
resonator, and $\tau=\kappa t/2$ is the dimensionless time. Other scaled
parameters are $J=-2J_{0}/\kappa$, $\Delta_{\mathrm{p}}=-2\delta/\kappa$,
$\beta=-2\xi_{2}/\kappa$, $\Delta_{\mathrm{sag}}=2\Delta_{\mathrm{F}}/\kappa$,
where $\kappa=\omega_{0}/Q$ is the optical loss rate, with the quality factor
of the resonator given as $Q$, and $\delta=\omega_{d}-\omega_{0}$ is the
detuning between the frequency of the pump laser and the resonance frequency
of the resonator. Also, the dimensionless external pump field intensity is
found as

$\displaystyle
F_{i}=\sqrt{\frac{4gP}{\hbar\kappa^{2}\omega_{d}}}=\sqrt{\frac{4Pn_{2}c\omega_{d}}{\kappa^{2}n^{2}V_{0}}},$
(12)

where $P$ is the laser power and $g$ is the Kerr coefficient usually given by

$\displaystyle g=\frac{n_{2}c\hbar\omega_{d}^{2}}{n^{2}V_{0}},$ (13)

with $n$ and $n_{2}$ denoting the linear and nonlinear refraction index of the
bulk material, respectively, and $V_{0}$ representing the effective mode
volume of the microresonator. In the following, the cross-phase modulation
between counterpropagating modes is neglected since for short pulses it is
three order lower than the strong linear coupling, as confirmed experimentally
in Ref. Fujii17Effect .

We first consider the case without any backscattering. In our calculations, we
use the experimentally feasible parameter values:
$\lambda=1550~{}\mathrm{nm}$, $Q=10^{9}$ Pavlov17Soliton ,
$R=30~{}\mathrm{um}$, $\beta=-0.004$, $n=1.44$, and
$\Omega=2.2~{}\mathrm{kHz}$. In Fig. 2, we plot the intracavity field energy
of the soliton mode $|\psi_{i}|^{2}$ as a function of the azimuthal angle
$\theta$ at $\tau=200$ and corresponding Kerr combs in a static and a spinning
microresonator. For a static resonator, our results reproduce and confirm the
results of previous studies that have shown the emergence of optical solitons
with different numbers under different initial conditions in a reciprocal way
[see Figs. 2(a,d)] Godey14Stability : For a given pump light, the same soliton
state is formed regardless of the input pump direction. In contrast, for a
spinning device, the system exhibits interesting nonreciprocity. For example,
Figs. 2(b-c,e-f) show that, driving the device from the left or the right
(i.e., in the same or opposite direction of the spinning) can lead to two
different cases: Soliton crystal state emerges for the pump in one direction
whereas a single soliton state emerges for the pump in the other direction
[see Figs. 2(b,c)]. Interestingly, we find that in such a device, it is
possible to realize directional switching of soliton numbers by tuning the
system parameters: As shown in Figs. 2(d-f), in a static resonator, only a
three-soliton state appears for both input directions Godey14Stability ,
whereas in a spinning device (with $\Omega=350~{}\mathrm{Hz}$), a two-soliton
state appears for the input from the left (blue curves) and a five-soliton
state appears for the same input from the right side (red curves).

Clear signature of nonreciprocal solitons is also observed in Figs. 2(g,h). By
tuning the system parameters $\Delta_{\mathrm{p}}$ and $\Omega$ properly, one
can obtain a soliton-diode behavior. For example, for
$\Delta_{\mathrm{p}}=2.3$ and $\Omega=2.2~{}\mathrm{kHz}$, a soliton emerges
only when driving the system from the left, but it is blocked when driving
from the right. This can be explained by different Sagnac drags
$\Delta_{\mathrm{F}}$ for the two counter-circulating modes of the
microresonator which makes it impossible to maintain the double balance
condition (nonlinearity and anomalous dispersion, as well as gain and
dissipation) simultaneously for these two distinct modes.

##  III Backscattering-immune features

Nonreciprocal solitons can also be used to protect optical devices against
losses induced by inter-modal coupling between CW and CCW modes, without using
any specially constructed topological structure or chiral reservoir
Shalaev19Robust ; Yu20Critical . Figs. 3(a,d) show that, solitons disappear in
a static resonator for increased values of backscattering $J=1$. In contrast,
in a spinning device, solitons can exist stably. Figs. 3(b,e) show that by
choosing a spinning velocity, i.e., $\Omega=60\mathrm{kHz}$, the harmful
effect of backscattering can be overcome in a chosen direction. In order to
make this advantage clearer, we show in Fig. 4 the competitive relationship
between the backscattering strength $J$ and the spinning velocity $\Omega$. We
find that as the rotation speed increases, the solitons become more stable
even in the presence of a larger backscattering strength. This suggests that
the performance of soliton devices can be further improved by harnessing the
power of nonreciprocity, which can be useful in nonreciprocal optical
communications or metrology Haus96Solitons ; Corcoran20Ultra ;
Parriaux20Electro ; Diddams20Optical .

Figure 4: The competitive relationship between the backscattering strength and
the spinning velocity in nonreciprocal solitons. The intracavity field energy
of the soliton mode $|\psi_{i}|^{2}$ as a function of the azimuthal angle
$\theta$ at $\tau=200$ and the backscattering strength $J$ with various
spinning velocity $\Omega$. The input signal of the optical soliton field is a
Gaussian pulse: $\psi_{0}=0.5+\mathrm{exp}[-(\theta/0.1)^{2}]$, the
dimensionless detuning is $\Delta_{\mathrm{p}}=2.1+\Delta_{\mathrm{sag}}$, and
the other parameters are the same as Fig. 2.

The underlying physics can be explained as follows. In a static optical
resonator (e.g., microsphere, microring, microtoroid, etc) supporting
frequency-degenerate CW and CCW modes, random-defect-induced backscattering
may lift the degeneracy by inter-modal coupling and hence result in a
symmetric mode splitting Weiss95Splitting ; Kippenberg02Modal ;
Yoshiki15Observation . Various methods have been used to suppress the
backscattering by breaking the time-reversal symmetry of the system and thus
creating different optical densities in the counterpropagating modes
Golubentsev84The ; MacKintosh88Coherent ; Kim19Dynamic ; Svela20Coherent . A
recent experiment has demonstrated that Maayani18Flying , spinning the
resonator provides a new way to break the time-reversal symmetry. Here, we use
spinning-induced nonreciprocity in a nonlinear resonator to achieve one-way
flow of solitons. Such spinning-induced nonreciprocal devices can also be used
to achieve quantum nonreciprocal effects as studied in Refs.
Huang18Nonreciprocal ; Li19Nonreciprocal ; Jiao20Nonreciprocal .

For completeness, in Fig. 3 (g), we also show the transient dynamics in the
dual pump system as reported in Ref. Yang17Counter  when backscattering
($J=1$) is considered. We find that the dual pump scheme can be used to
overcome backscattering by choosing a optical detuning $\alpha_{2}=15$. We
emphasize that our scheme, utilizing the mechanical rotation degree of
freedom, is well compatible with the other established techniques (e.g., using
an optical gain medium or the dual pump technique). In fact, by combining
optical and mechanical techniques, even more flexible one-way control of
solitons can be expected, with practical applications in making and utilizing
backaction-immune soliton devices Haus96Solitons ; Corcoran20Ultra ;
Parriaux20Electro ; Diddams20Optical .

##  IV Conclusions

In summary, we have proposed a system that uses a spinning Kerr resonator to
prepare and engineer nonreciprocal optical solitons. We find that in such a
system, different soliton states appear for the input lasers in the CW and CCW
directions. These nonreciprocal solitons are stable against losses induced by
inter-modal coupling between CW and CCW modes of the resonator, which are
otherwise detrimental for conventional solitons in static devices. We expect
even more flexible control of one-way solitons if mechanical spinning is
combined with other well-established optical techniques. In a broader view,
our results can stimulate more works on preparing nonreciprocal solitons in
systems well beyond spinning photonics, such as in solid devices
Shen16Experimental , atoms Xia18Cavity , or synthetic materials Sounas17Non-
reciprocal ; Peng14Parity ; Chang14Parity  where nonreciprocity has been
demonstrated. In the future, we will also study nonreciprocal engineering of
other kinds of solitons by taking into account the effects of e.g., optical
feedback Shen20Integrated , nonideal geometric shape of the resonator, and
quantum diffusions Bao21Quantum .

Acknowledgements. We thank Zheng-Yu Wang and Fang-Jie Shu for helpful
discussions. H.J. is supported by the National Natural Science Foundation of
China (Grants No. 11935006 and No. 11774086) and the Science and Technology
Innovation Program of Hunan Province (Grant No. 2020RC4047). L.-M.K. is
supported by the NSFC (Grants No. 11935006 and No. 11775075). X.-W.X. is
supported by the NSFC (12064010). L.Z. is supported by the NSFC for emergency
management project (Grant No.11447025). Ş.K.O. acknowledges support from ARO
(Grant No.W911NF-18-1-0043), NSF (Grant No. ECCS 1807485), and AFOSR (Grant
No. FA9550-18-1-0235). B.J.L. is supported also by Hunan Provincial Innovation
Foundation For Postgraduate (No. CX20190339).

## References

  * (1) A. Hasegawa and Y. Kodama, Solitons in Optical Communications (Oxford: Oxford University Press) (1995).
  * (2) H. A. Haus, W. S. Wong, Solitons in optical communications, Rev. Mod. Phys. 68(2), 423–444 (1996).
  * (3) N. J. Zabusky, M. D. Kruskal, Interaction of ”solitons” in a collisionless plasma and the recurrence of initial states, Phys. Rev. Lett. 15, 240–243 (1965).
  * (4) L. F. Mollenauer, R. H. Stolen, J. P. Gordon, Experimental observation of picosecond pulse narrowing and solitons in optical fibers, Phys. Rev. Lett. 45, 1095–1098 (1980).
  * (5) K. E. Strecker, G. B. Partridge, A. G. Truscott, R. G. Hulet, Formation and propagation of matterwave soliton trains, Nature 417, 150–153 (2002).
  * (6) T. Ackemann, W. J. Firth, and G. Oppo, Fundamentals and Applications of Spatial Dissipative Solitons in Photonic Devices, Adv. At. Mol. Opt. Phys. 323–421 (2009).
  * (7) T. Herr, V. Brasch, J. D. Jost, C. Y. Wang, N. M. Kondratiev, M. L. Gorodetsky, and T. J. Kippenberg, Temporal solitons in optical microresonators, Nat. Photon. 8(2), 145–152 (2014).
  * (8) X. Yi, Q.-F. Yang, K. Y. Yang, M.-G. Suh, K. Vahala, Soliton frequency comb at microwave rates in a high-Q silica microresonator, Optica 2, 1078 (2015).
  * (9) V. Brasch, M. Geiselmann, T. Herr, G. Lihachev, M. H. P. Pfeiffer, M. L. Gorodetsky, T. J. Kippenberg, Photonic chip-based optical frequency comb using soliton Cherenkov radiation, Science 351, 357–360 (2016).
  * (10) M.-G. Suh, Q.-F. Yang, K. Y. Yang, X. Yi, K. J. Vahala, Microresonator soliton dual-comb spectroscopy, Science 354, 600 (2016).
  * (11) E. Obrzud, S. Lecomte, T. Herr, Temporal solitons in microresonators driven by optical pulses, Nat. Photon. 11, 600 (2017).
  * (12) Q.-F. Yang, X. Yi, K. Y. Yang and K. Vahala, Counter-propagating solitons in microresonators, Nat. Photon. 11, 560 (2017).
  * (13) X. Yi, Q.-F. Yang, X. Zhang, K. Y. Yang, X. Li, and K. J. Vahala, Single-mode dispersive waves and soliton microcomb dynamics, Nat. Commun. 8, 14869 (2017).
  * (14) S. Fujii, A. Hori, T. Kato, R. Suzuki, Y. Okabe, W. Yoshiki, T. Tanabe, Effect on Kerr comb generation in a clockwise and counter-clockwise mode coupled microcavity, Opt. Express, 25(23), 28969-28982 (2017).
  * (15) M.-G. Suh and K. Vahala, Gigahertz-repetition-rate soliton microcombs, Optica 5(1), 65–66 (2018).
  * (16) W. L. Weng, R. Bouchand, E. Lucas, and T. J. Kippenberg, Polychromatic Cherenkov Radiation Induced Group Velocity Symmetry Breaking in Counterpropagating Dissipative Kerr Solitons, Phys. Rev. Lett. 123, 253902 (2019).
  * (17) M. Karpov, M. H. P. Pfiffer, H. Guo, W. Weng, J. Liu, and T. J. Kippenberg, Dynamics of soliton crystals in optical microresonators, Nat. Phys. 15, 1071–1077 (2019).
  * (18) F. J. Shu, P. J. Zhang, Y. J. Qian, Z. Y. Wang, S. Wan, C. L. Zou, G. C. Guo, and C. H. Dong, A mechanically tuned Kerr comb in a dispersionengineered silica microbubble resonator, Sci. China-Phys. Mech. Astron. 63(5), 254211 (2020).
  * (19) S. Wan, R. Niu, Z.-Y. Wang, J.-L. Peng, M. Li, G.-C. Guo, C.-L. Zou, and C.-H. Dong, Frequency stabilization and tuning of breathing solitons in Si3N4 microresonators, Photon. Res. 8, 1342–1349 (2020).
  * (20) Z. W. Fan and D. V. Skryabin, Soliton blockade in bidirectional microresonators, Opt. Lett. 45, 6446-6449 (2020).
  * (21) B. Q. Shen, L. Chang, J. Q. Liu, H. M. Wang, Q.-F. Yang, C. Xiang, R. N. Wang, J. J. He, T. Y. Liu, W. Q. Xie, J. Guo, D. Kinghorn, L. Wu, Q.-X. Ji, T. J. Kippenberg, K. Vahala, and J. E. Bowers, Integrated turnkey soliton microcombs, Nature 582, 365–369 (2020).
  * (22) A. F. Runge, D. D. Hudson, K. K. Tam, C. M. De Sterke, A. Blanco-Redondo, The pure-quartic soliton laser, Nat. Photon. 14(8), 1–6 (2020).
  * (23) J. Q. Liu, E. Lucas, A. S. Raja, J. J. He, J. Riemensberger, R. N. Wang, M. Karpov, H. Guo, R. Bouchand, and T. J. Kippenberg, Photonic microwave generation in the X- and K-band using integrated soliton microcombs, Nat. Photon. 1 (2020).
  * (24) J. Liu, H. Tian, E. Lucas, A. S. Raja, G. Lihachev, R. N. Wang, J. He, T. Liu, M. H. Anderson, W. Weng, S. A. Bhave, and T. J. Kippenberg, Monolithic piezoelectric control of soliton microcombs, Nature 583, 385 (2020).
  * (25) W. L. Weng, A. Kaszubowska-Anandarajah, J. J. He, P. D. Lakhmijayasimha, E. Lucas, J. Q. Liu, P. M. Anandarajah, and T. J. Kippenberg, Gain-switched semiconductor laser driven soliton microcombs, arXiv: 2006.10662 (2020).
  * (26) N. Englebert, F. De Lucia, P. Parra-Rivas, C. M. Arab, P.-J. Sazio, S.-P. Gorza, and F. Leo, Parametrically driven Kerr cavity solitons, arXiv: 2101.0778 (2021).
  * (27) C. Y. Bao, M.-G. Suh, B. Q. Shen, K. Şafak, A. Dai, H. M. Wang, L. Wu, Z. Q. Yuan, Q.-F. Yang, A. B. Matsko, F. X. Kärtner, and K. J. Vahala, Quantum diffusion of microcavity solitons, Nat. Phys. 17, 462 (2021).
  * (28) X. X. Xue, X. P. Zheng, and B. K. Zhou, Super-efficient temporal solitons in mutually coupled optical cavities, Nat. Photon. 13(9), 616–622 (2019).
  * (29) A. Tikan, J. Riemensberger, K. Komagata, S. Hönl, M. Churaev, C. Skehan, H. Guo, R. N. Wang, J. Liu, P. Seidler, T. J. Kippenberg, Emergent Nonlinear Phenomena in a Driven Dissipative Photonic Dimer, arXiv: 2005.06470 (2020).
  * (30) G. Xu, A. Nielsen, B. Garbin, L. Hill, G.-L. Oppo, J. Fatome, S. G. Murdoch, S. Coen, and M. Erkintalo, Spontaneous symmetry breaking of dissipative optical solitons in a two-component Kerr resonator, arXiv: 2008.13776 (2020).
  * (31) K. Komagata, A. Tikan, A. Tusnin, J. Riemensberger, M. Churaev, H. Guo, and T. J. Kippenberg, Dissipative Kerr solitons in a photonic dimer on both sides of exceptional point, arXiv: 2101.09237 (2021).
  * (32) Ó. B. Helgason, F. R. Arteaga-Sierra, Z. C. Ye, K. Twayana, P. A. Andrekson, M. Karlsson, Jochen SchrÖder, and V. Torres-Company, Dissipative solitons in photonic molecules, Nat. Photon. (2021).
  * (33) T. J. Kippenberg, A. L. Gaeta, M. Lipson, and M. L. Gorodetsky, Dissipative Kerr solitons in optical microresonators, Science 361(6402), 129–162 (2018).
  * (34) A. L. Gaeta, M. Lipson, and T. J. Kippenberg, Photonic-chip-based frequency combs, Nat. Photon. 13, 158–169 (2019).
  * (35) T. Fortier, and E. Baumann, 20 years of developments in optical frequency comb technology and applications, Commun. Phys. 2, 1–16 (2019).
  * (36) D. T. Spencer, T. Drake, T. C. Briles, J. Stone, L. C. Sinclair, C. Fredrick, Q. Li, D. Westly, B. R. Ilic, A. Bluestone, N. Volet, T. Komljenovic, L. Chang, S. H. Lee, D. Yoon Oh, M.-G. Suh, K. Y. Yang, M. H. P. Pfeiffer, T. J. Kippenberg, E. Norberg, L. Theogarajan, K. Vahala, N. R. Newbury, K. Srinivasan, J. E. Bowers, S. A. Diddams, S. B. Papp, An optical-frequency synthesizer using integrated photonics, Nature 557, 81–85 (2018).
  * (37) P. Trocha, M. Karpov, D. Ganin, M. H. Pfeiffer, A. Kordts, S. Wolf, J. Krockenberger, P. Marin-Palomo, C. Weimann, S. Randel, W. Freude, T. J. Kippenberg, C. Kooset, Ultrafast optical ranging using microresonator soliton frequency combs, Science 359(6378), 887–891 (2018).
  * (38) Z. Zhu and G. Wu, Dual-comb ranging, Engineering 4, 772–778 (2018).
  * (39) M.-G. Suh and K. J. Vahala, Soliton microcomb range measurement, Science 359(6378), 884–887 (2018).
  * (40) J. Riemensberger, A. Lukashchuk, M. Karpov, W. L. Weng, E. Lucas, J. Q. Liu, T. J. Kippenberg, Massively parallel coherent laser ranging using a soliton microcomb, Nature 581(7807), 164–170 (2020).
  * (41) J. Wang, Z. Lu, W. Wang, F. Zhang, J. Chen, Y. Wang, X. Zhao, J. Zheng, S. T. Chu, W. Zhao, B. E. Little, X. Qu, and W. Zhang, Long-distance ranging with high precision using a soliton microcomb, Photon. Res. 8, 1964–1972 (2020).
  * (42) Z. Shen, Y.-L. Zhang, Y. Chen, C.-L. Zou, Y.-F. Xiao, X.-B. Zou, F.-W. Sun, G.-C. Guo, and C.-H. Dong, Experimental realization of optomechanically induced non-reciprocity, Nat. Photon. 10, 657 (2016).
  * (43) Y. Shi, Z. Yu, and S. Fan, Limitations of nonlinear optical isolators due to dynamic reciprocity, Nat. Photon. 9, 388 (2015).
  * (44) L. Fan, J. Wang, L. T. Varghese, H. Shen, B. Niu, Y. Xuan, A. M. Weiner, and M. Qi, An All-Silicon Passive Optical Diode, Science 335, 447 (2012).
  * (45) K. Y. Xia, F. Nori, and M. Xiao, Cavity-Free Optical Isolators and Circulators Using a Chiral Cross-Kerr Nonlinearity, Phys. Rev. Lett. 121, 203602 (2018).
  * (46) M. Scheucher, A. Hilico, E. Will, J. Volz, A. Rauschenbeutel, Quantum optical circulator controlled by a single chirally coupled atom, Science 354, 1577 (2016).
  * (47) D. Malz, L. D. Tóth, N. R. Bernier, A. K. Feofanov, T. J. Kippenberg, and A. Nunnenkamp, Quantum-Limited Directional Amplifiers with Optomechanics, Phys. Rev. Lett. 120, 023601 (2018).
  * (48) X.-W. Xu, Y. Li, B. J. Li, H. Jing, and A.-X. Chen, Nonreciprocity via Nonlinearity and Synthetic Magnetism, Phys. Rev. Appl. 13(4), 044070 (2020).
  * (49) D. G. Lai, J. F. Huang, X. L. Yin, B. P. Hou, W. L. Li, D. Vitali, F. Nori, J. Q. Liao, Nonreciprocal ground-state cooling of multiple mechanical resonators, Phys. Rev. A. 102, 011502(R) (2020).
  * (50) D. L. Sounas and A. Alù, Non-reciprocal photonics based on time modulation, Nat. Photon. 11, 774 (2017).
  * (51) B. Peng, S. K. Özdemir, F. Lei, F. Monifi, M. Gianfreda, G. L. Long, S. Fan, F. Nori, C. M. Bender, and L. Yang, Parity-time-symmetric whispering-gallery microcavities, Nat. Phys. 10, 394 (2014).
  * (52) L. Chang, X. Jiang, S. Hua, C. Yang, J. Wen, L. Jiang, G. Li, G. Wang, and M. Xiao, Parity-time symmetry and variable optical isolation in active-passive-coupled microresonators, Nat. Photon. 8, 524 (2014).
  * (53) I. I. Arkhipov, A. Miranowicz, O. Di Stefano, R. Stassi, S. Savasta, F. Nori, S. K. Ozdemir, Scully-Lamb quantum laser model for parity-time-symmetric whispering-gallery microcavities: Gain saturation effects and nonreciprocity, Phys. Rev. A 99, 053806 (2019).
  * (54) S. Maayani, R. Dahan, Y. Kligerman, E. Moses, A. U. Hassan, H. Jing, F. Nori, D. N. Christodoulides, and T. Carmon, Flying couplers above spinning resonators generate irreversible refraction, Nature (London) 558, 569 (2018).
  * (55) H. Lü, Y. Jiang, Y. Z. Wang, and H. Jing, Optomechanically induced transparency in a spinning resonator, Photon. Res. 5, 367–371 (2017).
  * (56) K. Y. Yang, J. Skarda, M. Cotrufo, A. Dutt, G. H. Ahn, M. Sawaby, D. Vercruysse, A. Arbabian, S. H. Fan, A. Alù, and J. Vučković, Inverse-designed nonreciprocal pulse router for chip-based LiDAR, Nat. Photon. 14, 369–374 (2020).
  * (57) T. Tang, H.-J. Chen, Z. Y. Yuan, Y. Yu, Q.-T. Cao, N. An, Q. H. Gong, C. W. Wong, Y. J. Rao, Y.-F. Xiao, B. C. Yao, Gain-assisted chiral soliton microcombs, arXiv: 2008.12510 (2020).
  * (58) R. Fleury, D. L. Sounas, C. F. Sieck, M. R. Haberman, and A. Alù, Sound isolation and giant linear nonreciprocity in a compact acoustic circulator, Science 343, 516 (2014).
  * (59) G. Q. Xu, K. C. Dong, Y. Li, H. G. Li, K. P. Liu, L. Q. Li, J. Q. Wu, and C.-W. Qiu, Tunable analog thermal material, Nat. Commun. 11, 1–9 (2020).
  * (60) Y. Jiang, S. Maayani, T. Carmon, F. Nori, and H. Jing, Nonreciprocal phonon laser, Phys. Rev. Appl. 10, 064037 (2018).
  * (61) R. Huang, A. Miranowicz, J.-Q. Liao, F. Nori, H. Jing, Nonreciprocal Photon Blockade, Phys. Rev. Lett. 121, 153601 (2018).
  * (62) B. J. Li, R. Huang, X.-W. Xu, A. Miranowicz, and H. Jing, Nonreciprocal unconventional photon blockade in a spinning optomechanical system, Photon. Res. 7(6), 630–641 (2019).
  * (63) Y.-F. Jiao, S.-D. Zhang, Y.-L. Zhang, A. Miranowicz, L.-M. Kuang, and H. Jing, Nonreciprocal Optomechanical Entanglement against Backscattering Losses, Phys. Rev. Lett. 125, 143605 (2020).
  * (64) H. Jing, H. Lü, S. K. Özdemir, T. Carmon, and F. Nori, Nanoparticle sensing with a spinning resonator, Optica 5, 1424–1430 (2018).
  * (65) J. Ahn, Z. Xu, J. Bang, P. Ju, X. Gao, and T. Li, Ultrasensitive torque detection with an optically levitated nanorotor, Nat. Nanotechnol. 15, 89 (2020).
  * (66) H. L. Zhang, R. Huang, S.-D. Zhang, Y. Li, C.-W. Qiu, F. Nori, and H. Jing, Breaking Anti-PT Symmetry by Spinning a Resonator, Nano Lett. 20, 7594–7599 (2020).
  * (67) B. Corcoran, M. Tan, X. Xu, A. Boes, J. Y. Wu, T. G. Nguyen, S. T. Chu, B. E. Little, R. Morandotti, A. Mitchell, and D. J. Moss, Ultra-dense optical data transmission over standard fibre with a single chip source, Nat. Commun. 11, 2568 (2020).
  * (68) A. Parriaux, K. Hammani, and G. Millot, Electro-optic frequency combs, Adv. Opt. Photonics 12(1), 223–287 (2020).
  * (69) S. A. Diddams, K. Vahala, T. Udem, Optical frequency combs: Coherently uniting the electromagnetic spectrum, Science 369(6501), 267 (2020).
  * (70) S. M. Spillane, T. J. Kippenberg, O. J. Painter, and K. J. Vahala, Ideality in a Fiber-Taper-Coupled Microresonator System for Application to Cavity Quantum Electrodynamics, Phys. Rev. Lett. 91, 043902 (2003).
  * (71) G. B. Malykin, The Sagnac effect: correct and incorrect explanations, Phys. Usp. 43, 1229 (2000).
  * (72) T. Qin, J. F. Yang, F. X. Zhang , Y. Chen, D. Y. Shen, L. Chen, X. S. Jiang, X. F. Chen, W. J. Wan, Fast-and slow-light-enhanced light drag in a moving microcavity, Commun. Phys. 3, 118 (2020).
  * (73) L. A. Lugiato, R. Lefever, Spatial dissipative structures in passive optical systems, Phys. Rev. Lett. 21, 2209–2211 (1987).
  * (74) Y. K. Chembo, and C. R. Menyuk, Spatiotemporal Lugiato-Lefever formalism for Kerr-comb generation in whispering-gallery-mode resonators, Phys. Rev. Lett. 87, 053852 (2013).
  * (75) N. M. Kondratiev, V. E. Lobanov, Modulational instability and frequency combs in whispering-gallery-mode microresonators with backscattering, Phys. Rev. A 101, 013816 (2020).
  * (76) V. E. Lobanov, A. E. Shitikov, R. R. Galiev, K. N. Min’Kov, and N. M. Kondratiev, Generation and properties of dissipative kerr solitons and platicons in optical microresonators with backscattering, Opt. Express, 28, 36544 (2020).
  * (77) D. V. Skryabin, Hierarchy of coupled mode and envelope models for bi-directional microresonators with Kerr nonlinearity, OSA Continuum 3, 1364–1375 (2020).
  * (78) D. Weiss, V. Sandoghdar, J. Hare, V. Lefevre-Seguin, J.-M. Raimond, and S. Haroche, Splitting of high-Q Mie modes induced by light backscattering in silica microspheres, Opt. Lett. 20, 1835–1837 (1995).
  * (79) T. J. Kippenberg, S. Spillane, and K. Vahala, Modal coupling in traveling-wave resonators, Opt. Lett. 27, 1669–1671 (2002).
  * (80) W. Yoshiki, A. Chen-Jinnai, T. Tetsumoto, and T. Tanabe, Observation of energy oscillation between strongly-coupled counter-propagating ultra-high Q whispering gallery modes, Opt. Express 23, 30851–30860 (2015).
  * (81) N. G. Pavlov, G. Lihachev, S. Koptyaev, E. Lucas, M. Karpov, N. M. Kondratiev, I. A. Bilenko, T. J. Kippenberg, and M. L. Gorodetsky, Soliton dual frequency combs in crystalline microresonators, Opt. Lett. 42, 514 (2017).
  * (82) C. Godey, I. V. Balakireva, A. Coillet, Y. K. Chembo, Stability analysis of the spatiotemporal Lugiato-Lefever model for Kerr optical frequency combs in the anomalous and normal dispersion regimes, Phys. Rev. A 89(6), 063814 (2014).
  * (83) M. I. Shalaev, W Walasik, A. Tsukernik, Y. Xu, and N. M. Litchinitser, Robust topologically protected transport in photonic crystals at telecommunication wavelengths, Nat. Nano. 14, 31–34 (2019).
  * (84) S.-Y. Yu, C. He, X.-C. Sun, H. F. Wang, J.-Q. Wang, Z.-D. Zhang, B.-Y. Xie, Y. Tian, M.-H. Lu, and Y.-F. Chen, Critical couplings in topological-insulator waveguide-resonator systems observed in elastic waves, arXiv: 2008.09547 (2020).
  * (85) A. A. Golubentsev, The suppression of interference effects in multiple scattering of light, Sov. Phys. JETP 59, 26–32 (1984).
  * (86) F. C. MacKintosh and S. John, Coherent backscattering of light in the presence of time-reversal-noninvariant and parity-nonconserving media, Phys. Rev. B 37, 188–1897 (1988).
  * (87) S. Kim, J. M. Taylor, and G. Bahl, Dynamic suppression of Rayleigh backscattering in dielectric resonators, Optica 6, 1016 (2019).
  * (88) A. Ø. Svela, J. M. Silver, L. Del Bino, S. Y. Zhang, M. T. M. Woodley, M. R. Vanner, and P. DeĺHaye, Coherent suppression of backscattering in optical microresonators, Light Sci. Appl. 9, 204 (2020).