# Reciprocal Sequential Recommendation

Bowen Zheng†<EMAIL_ADDRESS>Gaoling School of Artificial Intelligence,
Renmin University of ChinaBeijingChina ,  Yupeng Hou†<EMAIL_ADDRESS>Gaoling School of Artificial Intelligence, Renmin University of
ChinaBeijingChina ,  Wayne Xin Zhao†‡ ✉<EMAIL_ADDRESS>Gaoling School of
Artificial Intelligence, Renmin University of ChinaBeijingChina ,  Yang Song ✉
<EMAIL_ADDRESS>BOSS ZhipinBeijingChina and  Hengshu Zhu
<EMAIL_ADDRESS>BOSS ZhipinBeijingChina

(2023)

###### Abstract.

Reciprocal recommender system (RRS), considering a two-way matching between
two parties, has been widely applied in online platforms like online dating
and recruitment. Existing RRS models mainly capture static user preferences,
which have neglected the evolving user tastes and the dynamic matching
relation between the two parties. Although dynamic user modeling has been
well-studied in sequential recommender systems, existing solutions are
developed in a user-oriented manner. Therefore, it is non-trivial to adapt
sequential recommendation algorithms to reciprocal recommendation. In this
paper, we formulate RRS as a distinctive sequence matching task, and further
propose a new approach _ReSeq_ for RRS, which is short for Reciprocal
Sequential recommendation. To capture dual-perspective matching, we propose to
learn fine-grained sequence similarities by co-attention mechanism across
different time steps. Further, to improve the inference efficiency, we
introduce the self-distillation technique to distill knowledge from the fine-
grained matching module into the more efficient student module. In the
deployment stage, only the efficient student module is used, greatly speeding
up the similarity computation. Extensive experiments on five real-world
datasets from two scenarios demonstrate the effectiveness and efficiency of
the proposed method. Our code is available at
https://github.com/RUCAIBox/ReSeq/.

reciprocal recommendation, self-distillation, sequential recommendation

$\dagger$ Beijing Key Laboratory of Big Data Management and Analysis Methods.

$\ddagger$ Beijing Academy of Artificial Intelligence, Beijing, 100084, China.

✉ Corresponding authors.

††copyright: acmcopyright††journalyear: 2023††doi:
XXXXXXX.XXXXXXX††conference: Proceedings of the 17th ACM Conference on
Recommender Systems; September 18–23, 2023; Singapore††booktitle: Proceedings
of the 17th ACM Conference on Recommender Systems (RecSys ’23), September
18–23, 2023, Singapore††price: 15.00††isbn: 978-1-4503-XXXX-0/22/xx††ccs:
Information systems Recommender systems

##  1\. Introduction

Reciprocal Recommender System (RRS) (Pizzato et al., 2010) has been widely
deployed in online platforms, such as online dating (Pizzato et al., 2010; Xia
et al., 2015; Neve and Palomares, 2019) and recruitment (Le et al., 2019;
Jiang et al., 2020; Yang et al., 2022a). Different from traditional
recommendation algorithms, it emphasizes a two-way (or bilateral) selection
process when establishing the matching between the two parties involved in
recommender systems. Generally, existing RRS works can be roughly divided into
content-based methods and collaborative filtering-based methods. The former
mainly model user interests from their personalized profiles (Pizzato et al.,
2010; Luo et al., 2020b; Yıldırım et al., 2021), while the latter usually
learn static user representations to capture long-term user preference based
on collaborative filtering (Xia et al., 2015; Neve and Palomares, 2019; Luo et
al., 2020a). However, the interests of users tend to dynamically shift along
with alterations in the surrounding environment over time, which is
disregarded by the above two categories of models. For instance, in the
context of online recruitment, as individuals gain more work experience and
qualifications, they are inclined to increase the criteria in job hunting.
Therefore, it is essential to effectively model the dynamic preference changes
in such a two-way preference selection process, thus making more accurate
recommendations that can satisfy the needs of both parties.

To model the dynamic interests of users, sequential recommendation methods
mainly capture user preference by modeling the chronological user behavior
sequences (Kang and McAuley, 2018; Sun et al., 2019) in a unilateral setting.
As a typical approach, they utilize the output in the last time step of the
sequential recommendation model for matching prediction. Different from
conventional recommendation algorithms, which are mainly developed based on
the single-perspective active selection from a user to items, users play a
dual role in reciprocal recommendation, serving not only as the party to
actively select but also as the other party to be passively selected. Thus,
the RRS necessitates the dynamic modeling of both parties, especially in the
context of two-way matching, which is likely to result in a more difficult
learning problem.

As the technical approach, we formulate reciprocal recommendation as a
distinctive sequence matching task, where the recommendation prediction is
performed based on the matching between the behavior sequences of the involved
two parties. Specifically, we establish the behavior sequences in dual
perspectives, and then conduct the two-sided matching by modeling fine-grained
sequential semantic interactions. While, it takes a high time cost to model
fine-grained matching, and we further consider employing self-distillation
techniques (commonly used in model compression (Zhang et al., 2019; Xu et al.,
2020; Liu et al., 2020)) to improve model efficiency.

Inspired by the above motivations, in this paper, we propose _ReSeq_ , a
Reciprocal Sequential recommendation method. There are three major technical
contributions in our approach. Firstly, we model bilateral behavior sequences
by a transformer network(Vaswani et al., 2017) based on user’s dual-
perspective embeddings (_i.e.,_ _active_ and _passive_). Secondly, we design a
multi-scale matching prediction method to capture fine-grained sequential
interactions at both macro and micro levels. Finally, we largely improve the
efficiency of the proposed model via micro-to-macro self-distillation, while
ensuring accuracy and effectiveness.

To the best of our knowledge, this is the first time that dynamic behavior
sequences of two parties are considered in reciprocal recommendation, which
can achieve both accurate and efficient recommendations. Extensive experiments
on five datasets from two reciprocal recommendation scenarios demonstrate the
effectiveness of our proposed method.

##  2\. Related Work

In this section, we review the related work in three aspects, namely
Reciprocal Recommendation, Sequential Recommendation and Self-Distillation.

###  2.1. Reciprocal Recommendation

Reciprocal recommender system (RRS) (Pizzato et al., 2010) is a recommender
system between people and people that significantly deviates from conventional
user-item recommender systems. To emphasize the mutual matching relationship
between users, RRS requires modeling the interests of both parties in the
recommendation, which poses a greater challenge for delivering effective
recommendations. Existing RRS studies can be roughly categorized into two
groups: content-based methods (Pizzato et al., 2010; Alanazi and Bain, 2013;
Tu et al., 2014) and collaborative filtering-based methods (Cai et al., 2012;
Xia et al., 2015). Content-based methods focus on the matching of user
profiles, while collaborative filtering-based methods pay more attention to
modeling historical user interactions. There also exist hybrid models that
integrate user behavior modeling as well as content matching (Akehurst et al.,
2011; Yang et al., 2017; Kleinerman et al., 2018; Hou et al., 2022c). In
recent years, deep learning techniques have been increasingly used to improve
the expressive power of RRS models, including matrix factorization (Qu et al.,
2018), latent factor models (Neve and Palomares, 2019), factorization machines
(Yıldırım et al., 2021), and graph neural networks (GNNs) (Luo et al., 2020a;
Yang et al., 2022a) Furthermore, several studies also make use of multiple
types of user behaviors to learn better bilateral user preferences in RRS
scenarios (Le et al., 2019; Fu et al., 2021). Although effective to some
extent, existing methods mainly encode user behaviors and profiles into static
representations to model long-term user interests, ignoring the dynamic
changes over time. In this work, we formulate reciprocal recommendation as a
distinctive sequence matching task and propose to leverage bilateral behavior
sequences for dynamic interest modeling on both sides.

###  2.2. Sequential Recommendation

In contrast to conventional collaborative filtering algorithms based on all
historical user-item interactions (Koren, 2008; Kabbur et al., 2013; He et
al., 2018; Rendle et al., 2009; He et al., 2017, 2020), sequential
recommendation pays more attention to the temporal aspects of user behaviors
(Hidasi et al., 2015; Sun et al., 2019; Kang and McAuley, 2018). Early
sequential recommendation methods are frequently founded upon Markov Chains
techniques (Rendle et al., 2010; He and McAuley, 2016). With the fast
development of deep neural networks, numerous deep models like Recurrent
Neural Networks (RNNs) are employed to facilitate item sequence modeling
(Hidasi et al., 2015; Tang and Wang, 2018; Yuan et al., 2019). More recently,
GNN-based (Wu et al., 2019; Xu et al., 2019) and Transformer-based methods
(Kang and McAuley, 2018; Sun et al., 2019; Hou et al., 2022a) also show great
effects on sequential recommendation task. Besides direct item sequence
modeling, recent studies also exploited techniques to additionally enhance
model effectiveness and robustness, such as pre-training (Zhou et al., 2020;
Hou et al., 2022b), data augmentation (Wang et al., 2021) and regularized
training (Xie et al., 2022; Zhou et al., 2023).

The above sequential recommendation methods have achieved amazing performance
in conventional user-item recommendation. Nevertheless, in the reciprocal
recommendation scenario, only single-perspective modeling usually fails to
achieve satisfactory results. The other possible reason is that compared with
the static item target in user-item recommendation, the dynamic
characteristics of both users in reciprocal recommendation require more
detailed interactions between both parties. In our approach, we model user
behavior sequences from both active and passive perspectives, and a multi-
scale matching prediction technique is proposed for comprehensive and fine-
grained two-way interactions.

###  2.3. Self-Distillation

Knowledge distillation (Hinton et al., 2015) is a popular technique that can
effectively transfer knowledge from complex teacher networks to more
streamlined student networks. In the realm of recommender systems, it is often
used for domain knowledge transfer (Yang et al., 2022b; Tu et al., 2021) or
model compression (Kang et al., 2021; Tian et al., 2023). However, such
methods mainly necessitate meticulously crafted teacher models, which incur
additional design and experimental costs. In recent years, the self-
distillation technique (Zhang et al., 2019) that distills knowledge from
itself has been proposed, and there are three prevalent self-distillation
methods exist: (a) distillation from deep to shallow layers of the same
network (Zhang et al., 2019; Luan et al., 2019; Liu et al., 2020); (b) using
the self-ensemble model (from the most recent time steps) as a virtual teacher
model (Xu et al., 2020; Shen et al., 2022); (c) data augmentation-based
methods (Huang et al., 2020; Xu and Liu, 2019). In comparison to conventional
knowledge distillation, self-distillation remains an area that is worth
exploring within the realm of recommendation systems (Xia et al., 2022). In
order to ensure low time consumption while engaging in more fine-grained two-
way interactions, we put forth a simple self-distillation framework in this
paper. Our approach is most similar to the first of the above self-
distillation methods, but instead of using the deeper network outputs as
knowledge, we use time-sensitive fine-grained micro-level matching as a
teacher for efficient macro-level matching.

Figure 1. The overall framework of our ReSeq. Note that it only shows the
single perspective of the model from $u_{i}$ to $v_{j}$, the other perspective
from $v_{j}$ to $u_{i}$ is symmetrical.

##  3\. Methodology

In this section, we first formulate our reciprocal sequential recommendation
task, and secondly introduce the proposed ReSeq method in detail. The overall
architecture is shown in Figure 1.

###  3.1. Problem Formulation

In conventional recommender systems, the goal of recommendation algorithms is
typically to predict the user’s interest in items (movies, books, clothes,
etc), which essentially refers to user-oriented item recommendation. Such a
recommendation paradigm is solely based on the user perspective for item
selection. As a comparison, Reciprocal Recommender System (RRS) (Pizzato et
al., 2010) targets a two-sided recommendation setting, in which either side
can select the users or resources on the other side, _e.g.,_ online dating
recommendation between females and males and job recommendation between job
seekers and recruiters. Formally, we denote the two parties in reciprocal
recommendation as $u_{i}\in\mathcal{U}$ (the first side) and
$v_{j}\in\mathcal{V}$ (the second side), respectively. In this setting, we do
not explicitly use the term of _item_ , since each side can play the roles of
_users_ (those who actively select the interactions) and _items_ (those who
are interacted with passively) as in traditional recommender systems. Instead,
we call both sides _users_ and use different notations to discriminate the two
sides. A successful recommendation in RRS indicates that: (a) the user $u_{i}$
demonstrates interest in the user $v_{j}$; (b) the features of $u_{i}$ align
with the preferences and requirements of $v_{j}$. This process can be
characterized as a dual-perspective active selection process (Pizzato et al.,
2010; Xia et al., 2015; Neve and Palomares, 2019; Yang et al., 2022a), the
goal of RRS typically aims at learning a joint function
$\hat{y}_{i,j}=\mathop{g}\limits^{\rightarrow}(u_{i},v_{j})+\mathop{g}\limits^{\leftarrow}(u_{i},v_{j})$
that can model this bilateral preference selection for the recommendation. In
this paper, our approach is established in the aforementioned setting for RRS.
This formulation characterizes a dual-perspective sequence matching task for
the RRS, which is distinctive from previous settings (Pizzato et al., 2010).

To be more specific, at every time step, each user possesses a historical
behavior sequence up to the current time $T$, denoted by
$\mathcal{S}^{T}_{u_{i}}=(v^{t_{1}}_{1},v^{t_{2}}_{2},...,v^{t_{a}}_{a})$ or
$\mathcal{S}^{T}_{v_{j}}=(u^{t_{1}}_{1},u^{t_{2}}_{2},...,u^{t_{b}}_{b})$
where $t_{a}<T$ and $t_{b}<T$ are the timestamps for interactions. In the
sequence $\mathcal{S}^{T}_{u_{i}}$, $v^{t_{j^{\prime}}}_{j^{\prime}}$
represents that $u_{i}$ is matched with $v_{j^{\prime}}$ at time
$t_{j^{\prime}}$. And vice versa, $u^{t_{i^{\prime}}}_{i^{\prime}}$ in
$\mathcal{S}^{T}_{v_{j}}$ represents the historical matching relationship with
$v_{j}$. Our target is to construct a matching function
$\hat{y}_{i,j}=\mathop{g}\limits^{\rightarrow}(u_{i},{S}^{T}_{u_{i}},v_{j},{S}^{T}_{v_{j}})+\mathop{g}\limits^{\leftarrow}(u_{i},{S}^{T}_{u_{i}},v_{j},{S}^{T}_{v_{j}})$
that predicts the matching degree or score $\hat{y}_{i,j}$ between both users
by dual-perspective modeling of their historical behavior sequences.

###  3.2. Dual-Perspective Dynamic Sequential Behavior Modeling

Existing RRS methods primarily utilize user profiles for prediction matching
(Pizzato et al., 2010; Luo et al., 2020b) or learning user static
representations based on historical user interactions (Xia et al., 2015; Neve
and Palomares, 2019). However, such methods tend to mainly focus on capturing
long-term user interests, thereby ignoring the dynamic changes in user
preferences over time. Our approach models reciprocal recommendation as a
distinctive sequence matching task, where the matching of both sides is
conducted based on their dynamic behavior sequences. In this section, we
present our method for modeling user behavior sequences under dual
perspectives. To be more precise, we first illustrate the user dual-
perspective representations in reciprocal recommendation, and then we
elucidate the process of encoding both active and passive aspects of user
behavior sequences.

####  3.2.1. Bilateral User Representation Learning

As mentioned in the previous section, the matching of users on both sides in
reciprocal recommendation is a dual-perspective process. This phenomenon
results in the user playing a dual role in reciprocal recommendation, serving
not only as the active selecting party but also as the passive accepting
party. Hence, methods in reciprocal recommendation frequently use dual-
perspective user representations (Neve and Palomares, 2019; Yang et al.,
2022a).

User representation learning under dual perspectives. For both users in the
reciprocal recommendation scenario, there exist two distinct representations:
one is when they act as active selectors by displaying their preferences over
the other party, and the other is when they act as passive candidates by
exhibiting their own features for being selected. For simplicity, we use “ _p
reference_” as the marker for the user active representations and “ _f
eature_” as the marker for the user passive representations. Particularly, for
users $\mathcal{U}$ on one side, we maintain two separate embedding matrices
$\bm{M}^{U}_{p}\in\mathbb{R}^{|\mathcal{U}|\times d}$ and
$\bm{M}^{U}_{f}\in\mathbb{R}^{|\mathcal{U}|\times d}$, which represent user
active and passive embeddings, respectively. Likewise, there are also two
embedding matrices $\bm{M}^{V}_{p}\in\mathbb{R}^{|\mathcal{V}|\times d}$ and
$\bm{M}^{V}_{f}\in\mathbb{R}^{|\mathcal{V}|\times d}$ for the users
$\mathcal{V}$ on the other side.

Decomposed and shared bilateral user embedding. As discussed previously, the
matching of both parties in reciprocal recommendation is a dual-perspective
alignment of their active interests and passive attractions. This implies that
the representation space of bilateral users should be properly aligned.
Formally, the above embedding matrices are aligned between $\bm{M}^{U}_{p}$
and $\bm{M}^{V}_{f}$, as well as between $\bm{M}^{V}_{p}$ and
$\bm{M}^{U}_{f}$. To reflect this dual-perspective alignment relationship, we
perform a decomposition of the user embedding matrix, and part of the
decomposed matrix is shared between the dual perspectives. Take a perspective
as an example:

(1) $\displaystyle\bm{M}^{U}_{p}=\bm{A}\cdot\bm{C},\ \ \ \ \
\bm{M}^{V}_{f}=\bm{B}\cdot\bm{C},$

where $\bm{A}\in\mathbb{R}^{|\mathcal{U}|\times{d^{\prime}}}$,
$\bm{B}\in\mathbb{R}^{|\mathcal{V}|\times{d^{\prime}}}$, and
$\bm{C}\in\mathbb{R}^{{d^{\prime}}\times d}$ is shared between the user active
representations on one side and the user passive representations on the other
side. Another perspective also applies the identical approach of matrix
decomposition and sharing.

####  3.2.2. Dual-Perspective Behavior Sequence Encoding

After obtaining the decomposed and shared dual-perspective user embedding, we
next encode the user behavior sequence by proposing two different methods. The
purpose is to actively and passively model behavior sequences according to the
different dynamic characteristics of the two roles for the user.

Unidirectional active dynamic encoding. When a user assumes the role of the
active party, each historical matching user in the behavior sequence acts as
the passive party. Intuitively, the features and characteristics of passive
candidates in recommendations are indicative of user interests and
preferences. Therefore, we encode the active dynamic representations of the
user based on the passive representations of historical matching users in the
behavior sequence.

Given a behavior sequence
$\mathcal{S}^{T}_{u_{i}}=(v^{t_{1}}_{1},v^{t_{2}}_{2},...,v^{t_{a}}_{a})$, we
first pad the sequence to a fixed length $n$, and then add a special token
($[CLS]$) at the beginning of the sequence to aggregate the sequence
representations. Next, we apply the lookup operation from $\bm{M}^{V}_{f}$
along with a learnable position embedding $P\in\mathbb{R}^{(n+1)\times d}$ to
form the input sequence passive embeddings
$\bm{E}^{V}_{f}\in\mathbb{R}^{(n+1)\times d}$. Subsequently, we exploit a
widely-used Transformer network (Vaswani et al., 2017) for sequence modeling.
Concretely, a typical Transformer architecture consists of multiple layers of
masked multi-head self-attention (denoted by Masked-MHAttn(·)), and each layer
of self-attention followed by a point-wise feed-forward network (denoted by
FFN(·)). The above sequence embeddings $\bm{E}^{V}_{f}$ combined with position
encoding are used as the initial input of the Transformer. Then, the
calculations can be formally written as:

(2) $\displaystyle\bm{H}^{l}$
$\displaystyle=\operatorname{FFN}\left(\operatorname{Masked-
MHAttn}\left(\bm{H}^{l-1},\bm{W}_{mask}\right)\right),\ l\in\\{1,\ldots,L\\},$

where $\bm{H}^{l}$ is the hidden states in the $l$-th layer, and
$\bm{H}^{0}=\bm{E}^{V}_{f}$. $\bm{W}_{mask}\in\mathbb{R}^{(n+1)\times(n+1)}$
is the attention mask weight matrix used to regulate the attention object for
each part within the sequence.

In our case, to encode the time-sensitive active dynamic behaviors, we use a
unidirectional attention mask weight matrix which allows the model to
concentrate more on recent user behaviors. As shown in Figure 1, compared with
the diagonal matrix unidirectional mask, our mask matrix separately processes
the corresponding row and column of the $[CLS]$ token to ensure both the order
of information transmission and the global aggregation of $[CLS]$
representation. Note that the mask matrix used in our approach is equivalent
to adding the $[CLS]$ token at the end of the sequence. However, compared to
individually inserting $[CLS]$ token to behavior sequences of different
lengths, our approach is more convenient for input construction and model
implementation.

Ultimately, the outputs from the final layer of the $L$-layer Transformer
network serve as the active dynamic representations of the user. Additionally,
we divide $\bm{H}^{L}$ into two scales:

(3) $\displaystyle\bm{p}_{u_{i}}=\bm{H}^{L}_{0},\ \ \ \ \
\widetilde{\bm{p}}_{u_{i}}=\bm{H}^{L}_{1:n+1},$

where $\bm{H}^{L}_{0}$ is the corresponding output of $[CLS]$ token,
$\bm{p}_{u_{i}}\in\mathbb{R}^{d}$ indicates the user’s macro or global active
dynamic representation. And $\bm{H}^{L}_{1:n+1}$ is the other part of the
outputs, $\widetilde{\bm{p}}_{u_{i}}\in\mathbb{R}^{n\times d}$ represents the
user’s micro or fine-grained active dynamic representations.

Bidirectional passive dynamic encoding. Similar to the unidirectional active
dynamic encoding described above, we also employ a Transformer network to
model the passive perspective of user behavior sequences. The difference is
that, in this context, each historical matching user in the behavior sequence
assumes the role of the active party. Empirically, a successful matching means
that the features and characteristics of the current user meet the preferences
and requirements of the active selector. We utilize the lookup operation from
$\bm{M}^{V}_{p}$ instead of $\bm{M}^{V}_{f}$ to form the input sequence active
embeddings $\bm{E}^{V}_{p}\in\mathbb{R}^{(n+1)\times d}$.

Furthermore, in contrast to the time-sensitivity of user active dynamic
behaviors, when a user acts as a passive individual, his features and
characteristics tend to remain relatively stable over time. Thus, as shown on
the right side of Figure 1, we encode the passive dynamic representations by a
bidirectional attention mask, which enables the model to capture essential
user traits across global sequence representations. In the same way, the
outputs of the last layer are also divided into two scales:

(4) $\displaystyle\bm{f}_{u_{i}}=\bm{H}^{L}_{0},\ \ \ \ \
\widetilde{\bm{f}}_{u_{i}}=\bm{H}^{L}_{1:n+1},$

where $\bm{f}_{u_{i}}\in\mathbb{R}^{d}$ is the user’s macro passive dynamic
representation, and $\widetilde{\bm{f}}_{u_{i}}\in\mathbb{R}^{n\times d}$ is
the user’s micro passive dynamic representations.

For user $v_{j}$ on the other side, we also encode his active and passive
dynamic representations based on the symmetrical model structure, which are
denoted by $\bm{p}_{v_{j}}$, $\widetilde{\bm{p}}_{v_{j}}$ and
$\bm{f}_{v_{j}}$, $\widetilde{\bm{f}}_{v_{j}}$.

###  3.3. Multi-Scale Sequence Matching

The above process allows us to obtain multi-scale active and passive dynamic
representations of users by encoding their behavior sequences. In the
following section, we will delve into the dual-perspective multi-scale
matching of both users in reciprocal recommendation.

####  3.3.1. Macro-Level Matching

As mentioned in the previous sections, the matching of both parties in
reciprocal recommendation is a dual-perspective active selection process
(Pizzato et al., 2010; Xia et al., 2015; Neve and Palomares, 2019; Yang et
al., 2022a). Hence, we apply two scores for active-passive matching to model
this dual-perspective propensity. It is worth noting that in Figure 1, only
one perspective matching process is depicted, while the other perspective is
symmetrical. The matching prediction method at the macro level is as follows:

(5) $\displaystyle\begin{split}{\hat{y}}_{i,j}&={\hat{y}}_{i\rightarrow
j}+{\hat{y}}_{j\rightarrow i}\\\
&=(\bm{p}_{u_{i}})^{T}\cdot\bm{f}_{v_{j}}+(\bm{p}_{v_{j}})^{T}\cdot\bm{f}_{u_{i}},\end{split}$

where $\cdot$ is the dot product operation. The first part of the formula
represents the propensity from $u_{i}$ to $v_{j}$, and the second part
represents the propensity from $v_{j}$ to $u_{i}$.

####  3.3.2. Time-Sensitive Micro-Level Matching

Compared with the conventional recommender systems that rely on a fixed number
of static items as candidate-matching targets, our problem formulation
involves a distinctive sequence matching task, which employs dynamic user
behavior sequences as candidate-matching targets. This results in a broader
and more dispersed candidate matching space. Consequently, to alleviate the
sparsity of the candidate matching space, we strive to attain more
discriminative and precise matching predictions by incorporating more detailed
and fine-grained micro-level interactions between users on both sides.
Formally, the overall micro-level matching method is as follows:

(6)
$\displaystyle{\hat{z}}_{i,j}=\operatorname{TiSensiMatch}\left(\widetilde{\bm{p}}_{u_{i}},\
\widetilde{\bm{f}}_{v_{j}}\right)+\operatorname{TiSensiMatch}\left(\widetilde{\bm{p}}_{v_{j}},\
\widetilde{\bm{f}}_{u_{i}}\right).$

Next, as shown in the “Time-Sensitive Micro-Level Matching” section of Figure
1, we take $\operatorname{TiSensiMatch}\left(\widetilde{\bm{p}}_{u_{i}},\
\widetilde{\bm{f}}_{v_{j}}\right)$ as an example to describe the
implementation of time-sensitive matching in detail. Firstly, we compute the
fine-grained matching matrix between the micro active dynamic representations
$\widetilde{\bm{p}}_{u_{i}}$ and the passive dynamic representations
$\widetilde{\bm{f}}_{v_{j}}$:

(7)
$\displaystyle\bm{G}=\widetilde{\bm{p}}_{u_{i}}\cdot{(\widetilde{\bm{f}}_{v_{j}})}^{T},$

where $\bm{G}\in\mathbb{R}^{n\times n}$ stands for the matching degrees
between the active and passive representations at different time steps.
Secondly, we aggregate the matrix $\bm{G}$ through two dimensions of co-
attention. For the passive dimension, we compute attention weights based on
softmax:

(8)
$\displaystyle\bm{\gamma}=\operatorname{softmax}(\widetilde{\bm{f}}_{v_{j}}\cdot\bm{e}^{p}_{u_{i}}),$

where $\bm{e}^{p}_{u_{i}}\in\mathbb{R}^{d}$ is the original active embedding
of the user $u_{i}$ from $\bm{M}^{U}_{p}$. For the active dimension, we
propose a relative time-dependent attention weight to model the time-
sensitivity of user active dynamic behaviors:

(9)
$\displaystyle\bm{\delta}=\operatorname{softmax}(\widetilde{\bm{p}}_{u_{i}}\cdot\bm{e}^{f}_{v_{j}}+\bm{\alpha}^{\tau}),$

where $\bm{e}^{f}_{v_{j}}\in\mathbb{R}^{d}$ is the original passive embedding
of the user $v_{j}$ from $\bm{M}^{V}_{f}$. And $\bm{\alpha}^{\tau}$ is from a
learnable relative time weight $\bm{\alpha}\in\mathbb{R}^{n}$ starting from
the end of the sequence. For a behavior sequence with an actual length of $s$
(no padding), there is $\bm{\alpha}^{\tau}_{s-k}=\bm{\alpha}_{k-1}$, where
$k\in\\{1,2,\ldots,s\\}$. Finally, we aggregate the fine-grained matching
matrix $\bm{G}$ in the following way:

(10) $\displaystyle\begin{split}{\hat{z}}_{i\rightarrow
j}&=\operatorname{TiSensiMatch}\left(\widetilde{\bm{p}}_{u_{i}},\
\widetilde{\bm{f}}_{v_{j}}\right)\\\
&={\bm{\delta}}^{T}\cdot\bm{G}\cdot\bm{\gamma}.\end{split}$

In addition, for the perspective from $v_{j}$ to $u_{i}$, we also get the
micro propensity score ${\hat{z}}_{j\rightarrow i}$ in the same way. The full
micro-level matching score ${\hat{z}}_{i,j}$ is the sum of the dual-
perspective propensities.

###  3.4. Improving Matching Efficiency via Self-Distillation

The rapid development of the internet results in an increased number of users
on online platforms. However, this also leads to a service efficiency problem
that needs to be solved urgently. Specific to the multi-scale sequence
matching method we proposed, macro-level matching is based on two simple and
efficient dot product operations, while micro-level matching provides more
fine-grained bilateral interactions. Nevertheless, the complexity of the
latter also presents efficiency issues that need to be addressed.

Inspired by knowledge distillation techniques (Hinton et al., 2015) that have
been successfully applied in various fields (Zhang et al., 2019; Sanh et al.,
2019; Hofstätter et al., 2020; Tian et al., 2023), we also aspire to improve
the recommendation efficiency in practical applications through self-
distillation from micro-level matching to macro-level matching. More exactly,
as shown in the “Micro-to-Macro Self-Distillation” section of Figure 1, we use
the self-distillation technique to transfer knowledge from finer-grained
micro-level matching to more efficient and simple macro-level matching during
training. Furthermore, only macro-level matching is used in the process of
verification, testing and practical application to achieve efficient and
precise recommendations.

Micro-to-macro self-distillation. As a ranking task, reciprocal recommendation
differs from common classification problems in that it pays more attention to
the relative score gap between positive and negative instances rather than the
absolute category distribution. Therefore, we adopt Margin-MSE (Hofstätter et
al., 2020) as our distillation loss to facilitate the alignment of positive
and negative score margins between micro-level and macro-level matching.
Specifically, the calculation for self-distillation loss is:

(11) $\displaystyle\begin{split}\mathcal{L}_{sd}&=\operatorname{Margin-
MSE}\left({\hat{z}}^{+},\ {\hat{z}}^{-},\ {\hat{y}}^{+},\
{\hat{y}}^{-}\right),\\\
&=\frac{1}{|B|}\sum^{|B|}_{i=1}\left[(\hat{z}^{+}_{i}-\hat{z}^{-}_{i})-(\hat{y}^{+}_{i}-\hat{y}^{-}_{i})\right]^{2},\end{split}$

where $|B|$ represents the batch size of training data. Besides,
${\hat{z}}^{+}$, ${\hat{z}}^{-}$ and ${\hat{y}}^{+}$, ${\hat{y}}^{-}$ are the
positive and negative score sets of micro level and macro level, respectively.

Overall optimization. In our approach, the final matching prediction is
computed by four elements from two users (Eqn. (5) and (6)), which are
indispensable in the actual scenario for successful matching. Thus, we can get
four negative instance scores by replacing the four elements with negative
instances, respectively. Formally, for each positive instance
$(u_{i},{S}^{T}_{u_{i}},v_{j},{S}^{T}_{v_{j}})$, we separately sample
bilateral negative users $(u_{i^{\prime}},{S}^{T}_{u_{i^{\prime}}})$ and
$(v_{j^{\prime}},{S}^{T}_{v_{j^{\prime}}})$. Note that the historical behavior
sequences of negative users are also truncated to the current time $T$. After
inputting negative instances into the model, four sets of negative instances
at the macro level and micro level can be obtained:

(12)
$\displaystyle\bm{D}=\\{(\bm{p}_{u_{i^{\prime}}},\widetilde{\bm{p}}_{u_{i^{\prime}}}),(\bm{f}_{u_{i^{\prime}}},\widetilde{\bm{f}}_{u_{i^{\prime}}}),(\bm{p}_{v_{j^{\prime}}},\widetilde{\bm{p}}_{v_{j^{\prime}}}),(\bm{f}_{v_{j^{\prime}}},\widetilde{\bm{f}}_{v_{j^{\prime}}})\\}.$

By replacing the corresponding elements in Eqn. (5) and (6) with the
aforementioned negative instances, we can compute multiple negative instance
scores for both scales. Without loss of generality, we denote them as two
negative instance score sets, $\bm{Y}^{-}_{i,j}$ and $\bm{Z}^{-}_{i,j}$, each
containing four types of negative instance scores. Then, we use the BPR loss
(Rendle et al., 2009) to optimize the ranking between positive and negative
instances:

(13) $\displaystyle\mathcal{L}_{ma}$
$\displaystyle=-\frac{1}{|B|}\sum_{(u_{i},v_{j})\in B}\
\sum_{\hat{y}^{-}_{i,j}\in\bm{Y}^{-}_{i,j}}\log\left(\sigma(\hat{y}^{+}_{i,j}-\hat{y}^{-}_{i,j})\right),$
(14) $\displaystyle\mathcal{L}_{mi}$
$\displaystyle=-\frac{1}{|B|}\sum_{(u_{i},v_{j})\in B}\
\sum_{\hat{z}^{-}_{i,j}\in\bm{Z}^{-}_{i,j}}\log\left(\sigma(\hat{z}^{+}_{i,j}-\hat{z}^{-}_{i,j})\right),$

where $B$ represents a batch of training data. $\hat{y}^{+}_{i,j}$ refers to
the macro-level matching score of user pairs, and $\hat{y}^{-}_{i,j}$ denotes
its negative instance score. $\hat{z}^{+}_{i,j}$ and $\hat{z}^{-}_{i,j}$ are
corresponding to micro-level matching. In addition, based on multiple negative
instance scores, the self-distillation loss can also be extended to:

(15) $\displaystyle\mathcal{L}_{sd}=\frac{1}{|B|}\sum_{(u_{i},v_{j})\in
B}\sum_{\scriptscriptstyle{\hat{y}^{-}_{i,j}\in\bm{Y}^{-}_{i,j},\hat{z}^{-}_{i,j}\in\bm{Z}^{-}_{i,j}}}\scriptstyle{\left[(\hat{z}^{+}_{i,j}-\hat{z}^{-}_{i,j})-(\hat{y}^{+}_{i,j}-\hat{y}^{-}_{i,j})\right]^{2}}.$

Combining the above three losses, our overall optimization loss function can
be denoted as follows:

(16)
$\displaystyle\mathcal{L}=\mathcal{L}_{ma}+\lambda\mathcal{L}_{mi}+\mu\mathcal{L}_{sd},$

where $\lambda$ and $\mu$ are hyperparameters. $\lambda$ is used to balance
the training of macro and micro, while $\mu$ is for the trade-off between
ranking optimization and self-distillation knowledge transfer.

Note that during validation and testing, we only deploy macro-level matching
module, which makes the model much more efficient in practice.

###  3.5. Discussion

####  3.5.1. Time Complexity Analysis

In a real-world deployment, the model inference associated only with the
entities themselves can be completed in advance. In reference to our approach,
the encoding of a user’s behavior sequence is independent of other
individuals, and the outcomes can be pre-calculated and stored. Therefore, we
mainly discuss the time complexity of the matching prediction part. Matching
at the macro level requires only the time complexity of $\mathcal{O}(d)$,
where $d$ denotes the model output dimension. It is the same as many classical
collaborative filtering-based methods (Rendle et al., 2009; Neve and
Palomares, 2019) and sequential recommendation methods (Kang and McAuley,
2018; Sun et al., 2019). For micro-level matching, it first takes
$\mathcal{O}(n^{2}d)$ time to complete the calculation of the fine-grained
matching matrix, where $n$ denotes the sequence length. Then, the time
complexity for the co-attention of the two dimensions is $\mathcal{O}(nd)$.
The final aggregation operation still takes time of $\mathcal{O}(n^{2})$.
Overall, the time complexity of prediction at the micro level is
$\mathcal{O}(n^{2}d)$. Given that online platforms cater to tens of thousands
of users, such time complexity is unacceptable. As a result, we suggest
utilizing self-distillation from the micro level to the macro level to
optimize the model’s efficiency for practical application.

####  3.5.2. Comparison with Existing Recommendation Methods

Here, we briefly compare with related recommendation methods to highlight the
innovation and novelty of our approach.

Sequential recommendation methods. Existing sequential recommendation
algorithms such as SASRec (Kang and McAuley, 2018) and BERT4Rec (Sun et al.,
2019) only encode user dynamic representations from one perspective, which
lacks the overall modeling of dual perspectives in the reciprocal
recommendation scenario. In contrast, the methodology we proposed involves the
dual-perspective joint modeling of users. And we achieve more precise
recommendations by performing fine-grained matching on more details of both
recommending parties.

Reciprocal recommendation methods. In the current reciprocal recommendation
studies (regardless of content-based or collaborative filtering methods), the
modeling of the user ultimately results in a static representation vector.
Some methods (Pizzato et al., 2010; Yıldırım et al., 2021) model user
preferences based on their fixed profiles, while others (Rendle et al., 2009;
Neve and Palomares, 2019) learn user’s static implicit representations through
matrix factorization and latent factor models. Even methods that are also
based on user historical behaviors (Fu et al., 2021) only aggregate all user
histories in the training set, while ignoring the temporal dynamics of
behavior sequences. As a comparison, our method takes into full consideration
the dynamics on both user active and passive perspectives when constructing
model inputs (Section 3.1), encoding user behaviors (Section 3.2), and
predicting bilateral user matching (Section 3.3).

##  4\. Experiment

###  4.1. Experiment Setup

####  4.1.1. Dataset

We evaluate our model in two reciprocal recommendation scenarios encompassing
five real-world datasets.

  1. (1)

Online recruitment. We conduct the evaluation using three authentic datasets
sourced from a popular Chinese online recruitment platform. These datasets are
created from over 100 days of real online logs, with all user private
information removed. The employer-employee pairs that achieved an interview
are gathered as instances of positive interactions.

  2. (2)

Question answering. We use two public datasets from distinct sections of the
popular online Q&A website “Stack Exchange” (Paranjape et al., 2017), intended
for mutual matching between questioners and answerers. For the answerer side,
we recommend questions to users according to their historical answering
records. For the questioner side, we follow an expert finding (Lin et al.,
2017) paradigm to recommend expert users for questioners.

Regarding data processing, we initially remove inactive users through 5-core
filtering and then create a behavior sequence for each user based on
chronological order. For each interaction record, we only use the sequence
truncated to the current interaction time to prevent data leakage during
training. On the three recruitment datasets, we adopt the last two weeks of
data as the _validation_ and _test_ sets, and the remaining data is used for
_training_. Besides, the datasets for questioner-answerer matching are divided
based on the time span in an 8:1:1 ratio. The overall statistics are shown in
Table 1. Our code is available at https://github.com/RUCAIBox/ReSeq/.

Table 1. Statistics of the experimental datasets.

Recruitment | Datasets | #Candidates | #Recruiters | #Interactions | Sparsity
---|---|---|---|---|---
Design | 12,274 | 9,141 | 139,355 | 99.88%
Sale | 15,831 | 12,757 | 112,340 | 99.94%
Technology | 56,620 | 48,071 | 808,376 | 99.97%
Q&A | Datasets | #Questioners | #Answerers | #Interactions | Sparsity
StackOverflow | 42,381 | 25,004 | 360,063 | 99.97%
AskUbuntu | 6,030 | 3,415 | 69,865 | 99.66%

Table 2. Performance comparison of different methods on the three datasets in
the recruitment scenario. The best-performed and the second-best-performed
methods are indicated in bold and underlined font, respectively. “*” denotes
that the improvements are significant at the level of 0.01 with paired
$t$-test.

Dataset | Perspective | Candidates | Recruiters
---|---|---|---
Metric | HR@5 | MRR@5 | NDCG@5 | HR@5 | MRR@5 | NDCG@5
Design | BPR | 0.1437 | 0.0866 | 0.1008 | 0.1300 | 0.0693 | 0.0842
LFRR | 0.1465 | 0.0855 | 0.1006 | 0.1338 | 0.0694 | 0.0853
NeuMF | 0.1611 | 0.1016 | 0.1163 | 0.1452 | 0.0804 | 0.0964
LightGCN | 0.1496 | 0.0913 | 0.1057 | 0.1349 | 0.0732 | 0.0884
SASRec | 0.2033 | 0.1184 | 0.1394 | 0.2529 | 0.1389 | 0.1670
SSE-PT | 0.1961 | 0.1097 | 0.1311 | 0.2466 | 0.1327 | 0.1608
BERT4Rec | 0.1535 | 0.0874 | 0.1037 | 0.2027 | 0.1044 | 0.1286
FMLP-Rec | 0.1940 | 0.1194 | 0.1379 | 0.2845 | 0.1571 | 0.1886
PJFNN | 0.1056 | 0.0621 | 0.0728 | 0.0604 | 0.0272 | 0.0353
BPJFNN | 0.1755 | 0.1010 | 0.1194 | 0.1589 | 0.0766 | 0.0968
IPJF | 0.1778 | 0.0943 | 0.1149 | 0.1661 | 0.0835 | 0.1038
PJFFF | 0.1839 | 0.1052 | 0.1247 | 0.1235 | 0.0595 | 0.0752
DPGNN | 0.2422 | 0.1396 | 0.1649 | 0.2317 | 0.1273 | 0.1530
| ReSeq | 0.4435* | 0.2554* | 0.3020* | 0.3722* | 0.1994* | 0.2420*
Sale | BPR | 0.1071 | 0.0613 | 0.0726 | 0.0929 | 0.0513 | 0.0615
LFRR | 0.1262 | 0.0730 | 0.0862 | 0.1083 | 0.0602 | 0.0721
NeuMF | 0.1367 | 0.0828 | 0.0962 | 0.1113 | 0.0639 | 0.0756
LightGCN | 0.1447 | 0.0832 | 0.0984 | 0.1228 | 0.0691 | 0.0823
SASRec | 0.1707 | 0.0932 | 0.1123 | 0.1494 | 0.0807 | 0.0976
SSE-PT | 0.1592 | 0.0912 | 0.1080 | 0.1276 | 0.0648 | 0.0802
BERT4Rec | 0.1303 | 0.0685 | 0.0837 | 0.1143 | 0.0579 | 0.0718
FMLP-Rec | 0.1758 | 0.1001 | 0.1188 | 0.1384 | 0.0742 | 0.0901
PJFNN | 0.1097 | 0.0650 | 0.0760 | 0.0568 | 0.0269 | 0.0342
BPJFNN | 0.1355 | 0.0767 | 0.0912 | 0.1130 | 0.0549 | 0.0692
IPJF | 0.1092 | 0.0541 | 0.0677 | 0.1177 | 0.0629 | 0.0763
PJFFF | 0.1564 | 0.0876 | 0.1046 | 0.1052 | 0.0524 | 0.0654
DPGNN | 0.1871 | 0.1026 | 0.1234 | 0.1703 | 0.0930 | 0.1121
| ReSeq | 0.2712* | 0.1499* | 0.1798* | 0.2434* | 0.1335* | 0.1606*
Technology | BPR | 0.2733 | 0.1778 | 0.2015 | 0.2659 | 0.1627 | 0.1883
LFRR | 0.2270 | 0.1420 | 0.1631 | 0.2151 | 0.1232 | 0.1459
NeuMF | 0.2796 | 0.1821 | 0.2063 | 0.2711 | 0.1662 | 0.1922
LightGCN | 0.2775 | 0.1823 | 0.2060 | 0.2726 | 0.1695 | 0.1952
SASRec | 0.4145 | 0.2815 | 0.3148 | 0.4702 | 0.3042 | 0.3456
SSE-PT | 0.3924 | 0.2595 | 0.2926 | 0.4931 | 0.3129 | 0.3578
BERT4Rec | 0.3650 | 0.2408 | 0.2718 | 0.4243 | 0.2718 | 0.3098
FMLP-Rec | 0.4165 | 0.2898 | 0.3215 | 0.5206 | 0.3476 | 0.3908
PJFNN | 0.2707 | 0.1478 | 0.1781 | 0.2636 | 0.1348 | 0.1666
BPJFNN | 0.3791 | 0.2172 | 0.2573 | 0.4261 | 0.2273 | 0.2764
IPJF | 0.3403 | 0.1743 | 0.2151 | 0.3776 | 0.1996 | 0.2435
PJFFF | 0.3425 | 0.2126 | 0.2448 | 0.3715 | 0.1998 | 0.2422
DPGNN | 0.4521 | 0.2696 | 0.3148 | 0.4409 | 0.2599 | 0.3047
| ReSeq | 0.7597* | 0.4945* | 0.5606* | 0.7809* | 0.5099* | 0.5775*

Table 3. Performance comparison of different methods on two datasets in the
questioner-answerer matching scenario. The best-performed and the second-best-
performed methods are indicated in bold and underlined font, respectively. “*”
denotes that the improvements are significant at the level of 0.01 with paired
$t$-test.

Dataset | Perspective | Questioners | Answerers
---|---|---|---
Metric | HR@5 | MRR@5 | NDCG@5 | HR@5 | MRR@5 | NDCG@5
StackOverflow | BPR | 0.3262 | 0.2344 | 0.2572 | 0.2653 | 0.1825 | 0.2029
LFRR | 0.3051 | 0.2113 | 0.2346 | 0.2391 | 0.1522 | 0.1737
NeuMF | 0.3218 | 0.2351 | 0.2567 | 0.2563 | 0.1781 | 0.1975
LightGCN | 0.3352 | 0.2359 | 0.2606 | 0.2780 | 0.1881 | 0.2103
SASRec | 0.4336 | 0.3193 | 0.3478 | 0.2672 | 0.1780 | 0.2001
SSE-PT | 0.4064 | 0.2901 | 0.3190 | 0.2857 | 0.1881 | 0.2123
BERT4Rec | 0.3621 | 0.2501 | 0.2779 | 0.2456 | 0.1548 | 0.1772
FMLP-Rec | 0.4594 | 0.3428 | 0.3718 | 0.2957 | 0.2026 | 0.2257
| ReSeq | 0.4868* | 0.3235 | 0.3640 | 0.4019* | 0.2432* | 0.2825*
AskUbuntu | BPR | 0.1566 | 0.1106 | 0.1220 | 0.1299 | 0.0914 | 0.1009
LFRR | 0.1590 | 0.1003 | 0.1149 | 0.0935 | 0.0622 | 0.0699
NeuMF | 0.2351 | 0.1592 | 0.1780 | 0.1323 | 0.0992 | 0.1074
LightGCN | 0.1606 | 0.1128 | 0.1247 | 0.1245 | 0.0906 | 0.0990
SASRec | 0.2663 | 0.1698 | 0.1937 | 0.1588 | 0.1076 | 0.1202
SSE-PT | 0.2418 | 0.1584 | 0.1791 | 0.1540 | 0.0994 | 0.1129
BERT4Rec | 0.2294 | 0.1404 | 0.1626 | 0.1143 | 0.0669 | 0.0786
FMLP-Rec | 0.2706 | 0.1818 | 0.2039 | 0.1627 | 0.1121 | 0.1246
| ReSeq | 0.5259* | 0.2848* | 0.3447* | 0.2254* | 0.1455* | 0.1653*

####  4.1.2. Baseline Models

We compare ReSeq with the following baseline models (key configurations are in
parentheses):

  * •

BPR (Rendle et al., 2009) is a matrix factorization-based collaborative
filtering method, which is optimized by Bayesian Personalized Ranking (BPR)
loss (embedding size=64).

  * •

LFRR (Neve and Palomares, 2019) applies the latent factor model for reciprocal
recommendation, which utilizes two latent factor models to model the dual
perspectives of both parties (embedding size=64).

  * •

NeuMF (He et al., 2017) combines matrix factorization and neural networks to
improve the effectiveness of personalized recommendation (MLP hidden sizes=
[128,64], dropout ratio=0.1).

  * •

LightGCN (He et al., 2020) is a collaborative filtering method based on
lightweight graph convolutional networks, which simplifies graph convolution
operations and achieves competitive performance on user-item recommendation
(embedding size=64, number of layers=2).

  * •

SASRec (Kang and McAuley, 2018) exploits a unidirectional Transformer-based
neural network to model the item sequences and predict the next item (number
of layers=2, number of attention heads=2, dropout ratio=0.5).

  * •

SSE-PT (Wu et al., 2020) combines user and item embeddings as input of the
Transformer model for personalized sequential recommendation (number of
layers=2, number of attention heads=2, dropout ratio=0.5).

  * •

BERT4Rec (Sun et al., 2019) adopts a bidirectional Transformer model and
combines it with a mask prediction task for the modeling of item sequences
(number of layers=2, number of attention heads=2, fine-tuning ratio=0.5)

  * •

FMLP-Rec (Zhou et al., 2022) proposes an all-MLP model with learnable filters,
which ensures efficiency and reduces noisy signals (number of layers=2,
dropout ratio=0.5).

  * •

PJFNN (Zhu et al., 2018) is a person-job fit model based on CNN to predict the
matching degree between resumes and job descriptions (max sentence number=20,
max sentence length=30).

  * •

BPJFNN (Qin et al., 2018) leverages a bidirectional LSTM network to encode
resumes and job descriptions for person-job fit prediction (max sentence
length=512, number of layers=1, hidden size=32, dropout ratio=0.1)

  * •

IPJF (Le et al., 2019) learns the matching propensities between candidates and
recruiters based on various behavior labels in recruitment scenarios (max
sentence number=20, max sentence length=30, BERT embedding size=768).

  * •

PJFFF (Jiang et al., 2020) fuses the user explicit and implicit
representations. The implicit representations are modeled by LSTM based on
historical user records (hidden size=32, historical item length=100, BERT
embedding size=768, BERT output size=32).

  * •

DPGNN (Yang et al., 2022a) proposes a dual-perspective graph representation
learning method that models recruitment as a two-way selection process (number
of layers=2, BERT embedding size=768, BERT output size=32, contrastive loss
weight=0.05, temperature=0.2).

Our baselines can be categorized into collaborative filtering models (_i.e.,_
BPR, LFRR, NeuMF, LightGCN), sequential recommendation models (_i.e.,_ SASRec,
SSE-PT, BERT4Rec, FMLP-Rec), and person-job fit models (_i.e.,_ PJFNN, BPJFNN,
IPJF, PJFFF, DPGNN). Among these, the person-job fit models are exclusively
applicable in online recruitment scenarios where users possess comprehensive
textual information.

####  4.1.3. Evaluation and Implementation Details

Following (Yang et al., 2022a), we employ three widely used metrics, Hit Ratio
(HR@$k$), Normalized Discounted Cumulative Gain (NDCG@$k$), and Mean
Reciprocal Rank (MRR@$k$) to evaluate ranking tasks from dual perspectives
simultaneously, where $k$ is set to 5. Concretely, for each positive instance
interaction $(u_{i},{S}^{T}_{u_{i}},v_{j},{S}^{T}_{v_{j}})$, we randomly
sample $100$ users from $\mathcal{U}$ and $\mathcal{V}$ as dual-perspective
negative instances, respectively. Then truncate their behavior sequences to
the current interaction time for the evaluation to prevent data leakage (Zhao
et al., 2022b).

For all the compared methods, we use Adam for optimization and grid search for
hyperparameter tuning to get optimal performance. The learning rates of all
models are tuned in {0.005, 0.001, 0.0005, 0.0003, 0.0001, 0.00001}. To ensure
a fair comparison, the dimensions of embeddings are uniformly set to $64$.
Besides, the weight decay coefficient is set to $1e-5$. And we select the best
models by early stopping with the patience of $10$ epochs. All models are
implemented based on a popular open-source recommendation library RecBole
(Zhao et al., 2021, 2022a).

Table 4. Ablation study of our method in two scenarios. The best-performed
methods are indicated in bold.

Dataset | Perspective | Candidates/Questioners | Recruiters/Answerers
---|---|---|---
Metric | HR@5 | MRR@5 | NDCG@5 | HR@5 | MRR@5 | NDCG@5
Design | ReSeq | 0.4435 | 0.2554 | 0.3020 | 0.3722 | 0.1994 | 0.2420
w/o DSE | 0.4224 | 0.2462 | 0.2898 | 0.3409 | 0.1806 | 0.2201
w/o MASK | 0.4305 | 0.2510 | 0.2954 | 0.3406 | 0.1836 | 0.2223
w/o TSA | 0.4139 | 0.2368 | 0.2806 | 0.3315 | 0.1798 | 0.2172
w/o SD | 0.4156 | 0.2372 | 0.2813 | 0.3446 | 0.1889 | 0.2273
StackOverflow | ReSeq | 0.4868 | 0.3235 | 0.3640 | 0.4019 | 0.2432 | 0.2825
w/o DSE | 0.4787 | 0.3102 | 0.3521 | 0.3964 | 0.2391 | 0.2780
w/o MASK | 0.4764 | 0.3114 | 0.3524 | 0.3942 | 0.2387 | 0.2772
w/o TSA | 0.4710 | 0.3128 | 0.3521 | 0.3697 | 0.2239 | 0.2600
w/o SD | 0.4787 | 0.3154 | 0.3560 | 0.3935 | 0.2377 | 0.2763

Figure 2. Performance comparison w.r.t. micro teacher loss coefficient
$\lambda$ and self-distillation loss coefficient $\mu$.

###  4.2. Experiment Results

The results of different methods in the recruitment scenario are shown in
Table 2, including baseline models dedicated to person-job fit (with extra
text data). Table 3 shows the comparison between our model and baseline models
in the questioner-answerer matching scenario. Based on these results, we can
find:

For the four collaborative filtering methods, models based on GCN (_i.e.,_
LightGCN) or MLP (_i.e.,_ NeuMF) for hybrid interaction tend to exhibit
superior performance in comparison to simple matrix factorization method
(_i.e.,_ BPR) and latent factor model (_i.e.,_ LFRR). It indicates that
increased granularity in interactions has a positive impact on the accuracy of
recommendations, even in the case of static representations. As for sequential
recommendation methods, it can be observed that they generally achieve better
performance compared to conventional collaborative filtering methods. We
suppose this can be attributed to their ability to dynamically model user
preferences based on the behavior sequences. Furthermore, FMLP-Rec attains
considerable results compared to transformer-based models (_i.e.,_ SASRec,
SSE-PT, BERT4Rec). One possible reason for this is that the proposed learnable
filtering layer has the ability to efficiently decrease noisy signals.

Additionally, in the recruitment scenario, there are also results of person-
job fit methods. We can observe that models that rely exclusively on user
resumes and job descriptions (_i.e.,_ PJFNN and BPJFNN) are frequently subject
to severe limitations caused by the quality and volume of textual data. These
models perform better on Technology datasets that contain higher quality and
larger volumes of text data compared to that Design and Sale datasets. Among
the hybrid models that combine both text and collaborative behavior signals
(_i.e.,_ IPJF, PJFFF, DPGNN), DPGNN achieves the best performance, which is
inseparable from its modeling capacity of the two-way selection process in
reciprocal recommendation.

Overall, compared with all baseline methods, it can be seen that our approach
ReSeq achieves the best results in almost all the cases across five datasets
from two scenarios. Different from these baselines, we formulate reciprocal
recommendation as a distinctive sequence matching task, where matching
prediction is performed based on the dynamic behavior sequences of both
parties. And for lower time consumption, we introduce the self-distillation
techniques to distill knowledge from the micro to macro level matching, which
greatly speeds up the similarity calculation. As a result, our ReSeq is an
effective and efficient reciprocal recommendation method.

###  4.3. Ablation Study

The major contribution of our method ReSeq lies in four parts: decomposed
embedding, behavior sequence encoding, matching prediction, and efficiency
improvement. In order to verify the effectiveness of specific designs for each
component, we conduct an ablation study on Design and StackOverflow datasets
to verify their contributions. Specifically, we consider the following four
variants of ReSeq:

  * •

ReSeq w/o DSE removes the decomposition and sharing operations of embedding
matrices (Eqn. (1)) and directly learns four embedding matrices for users.

  * •

ReSeq w/o MASK replaces the different mask matrices for encoding user behavior
sequences in Section 3.2.2 with the same bidirectional mask matrix.

  * •

ReSeq w/o TSA exchanges the time-sensitive attention aggregation operation in
micro-level matching (Eqn. (10)) into mean aggregation.

  * •

ReSeq w/o SD removes the self-distillation loss.

From the results in Table 4, it is evident that removing any of the above
components would result in a decline in the overall effect. This indicates
that all of the components in ReSeq play a vital role in delivering precise
recommendations.

###  4.4. Further Analysis

####  4.4.1. Hyperparameter Tuning

In this section, we assess the robustness of our model and conduct a
comprehensive analysis of two crucial loss function coefficients. The
performance of the two datasets is presented in Figure 2. For each loss
coefficient, we show the NDCG@5 of the two perspectives separately and
calculate the average metrics of the two perspectives to reflect the overall
trend.

For the micro teacher loss coefficient $\lambda$, we tune it within the range
from 1 to 10 and additionally designate it as 0.5 to investigate the
ramifications of inadequate teacher preparation. As shown in Figure 2, with
the increasing of coefficient $\lambda$, the model’s performance gradually
improves. At the value of 5, ReSeq achieves the best performance, and further
increment of $\lambda$ does not result in significant performance degradation,
which fully reflects our method’s robustness.

For the self-distillation loss coefficient $\mu$, we tune it in the ranges of
{0.0001, 0.001, 0.003, 0.005, 0.01, 0.05, 0.1}. As shown in Figure 2, our
model achieves the best results on the two datasets when $\mu$ is set to 0.005
and 0.01, respectively. A self-distillation loss coefficient that is too small
cannot effectively transfer knowledge between micro and macro levels, while a
too-large loss coefficient may cause the model to disregard ranking
optimization.

####  4.4.2. Deployment Efficiency Analysis

To investigate the impact of micro-to-macro self-distillation on time latency
in real-world deployment, we compare the efficiency of matching prediction
between SASRec, ReSeq, and ReSeq w/o Self-Distillation on two datasets from
different scenarios, _i.e.,_ Design and StackOverflow. All experiments are
carried out on a Linux system machine with an Intel(R) Xeon(R) CPU E5-2630 v4
@ 2.20GHz CPU and a 12GB NVIDIA TITAN V GPU. For a fair comparison, we use the
PyTorch framework to implement all methods and ensure that the common
parameters such as batch size are uniform. Specifically, we count the average
matching prediction time of each batch of the three methods on the test sets.
Besides, we maintain process synchronization through the synchronize function
in PyTorch to obtain accurate time latency.

As shown in Figure 3, our ReSeq has a latency slightly higher than SASRec. The
additional consumption is an inevitable time increase brought by dual-
perspective matching in reciprocal recommendation to achieve effective
sequence matching. However, without self-distillation, the fine-grained micro-
level matching prediction would result in decades of increase in time
consumption, which is unacceptable in actual deployment. In contrast, the
micro-to-macro self-distillation enables our ReSeq to be an efficient and
effective reciprocal recommendation method.

Figure 3. Per-batch matching prediction latency. Figure 4. Performance after
adding text-based matching.

####  4.4.3. Text Enhancement

Out of all, we conduct a text enhancement experiment on recruitment datasets.
In detail, we directly incorporate a text matching score derived from pre-
trained twin-tower BERT with an additional prediction layer into the macro-
level matching prediction (Eqn. (5)). The results are presented in Figure 4.
Observably, after combining with simple text enhancement, our method basically
obtains performance improvements in both perspectives on the three datasets.
Particularly, ReSeq benefits more on the Technology dataset. The potential
explanation for this phenomenon is that the text quality and quantity vary
prominently across different datasets. Thus, in the Technology dataset, whose
text quality and quantity substantially surpass the other two datasets, the
effect of text enhancement is more significant. The above results illustrate
that our method can be improved by incorporating some supplementary
information. Certainly, this simple text enhancement method may impose some
limitations, and in the future, we will take into account more rational
approaches to merge text or other information.

##  5\. Conclusion

This paper proposes a reciprocal sequential recommendation method, named
_ReSeq_. To model dynamic user interests from both sides, we formulate
reciprocal recommendation as a distinctive sequence matching task.
Specifically, we encode user behavior sequences from both active and passive
perspectives with different module designs. Subsequently, we conduct
comprehensive and fine-grained sequence interactions at both macro and micro
levels through multi-scale sequence matching. In addition, to alleviate the
increased time consumption caused by fine-grained interactions, we improve the
model deployment efficiency via micro-to-macro self-distillation. Extensive
experiments on real-world datasets show that our approach outperforms several
competitive baselines from two perspectives in RRS. In the future, we will
consider incorporating multiple auxiliary sequential features (_e.g.,_ textual
information and categories) into our framework in a universal, effective, and
efficient way. Besides, we would like to further alleviate the sparsity issue
while training reciprocal sequential models, including pre-training, data
augmentation, and self-supervised learning techniques.

## References

  * (1)
  * Akehurst et al. (2011) Joshua Akehurst, Irena Koprinska, Kalina Yacef, Luiz Pizzato, Judy Kay, and Tomasz Rej. 2011\.  CCR—a content-collaborative reciprocal recommender for online dating. In _IJCAI_.
  * Alanazi and Bain (2013) Ammar Alanazi and Michael Bain. 2013.  A people-to-people content-based reciprocal recommender using hidden markov models. In _RecSys_.
  * Cai et al. (2012) Xiongcai Cai, Michael Bain, Alfred Krzywicki, Wayne Wobcke, Yang Sok Kim, Paul Compton, and Ashesh Mahidadia. 2012.  Reciprocal and heterogeneous link prediction in social networks. In _PAKDD_.
  * Fu et al. (2021) Bin Fu, Hongzhi Liu, Yao Zhu, Yang Song, Tao Zhang, and Zhonghai Wu. 2021\.  Beyond matching: Modeling two-sided multi-behavioral sequences for dynamic person-job fit. In _DASFAA_.
  * He and McAuley (2016) Ruining He and Julian McAuley. 2016.  Fusing similarity models with markov chains for sparse sequential recommendation. In _ICDM_.
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020\.  Lightgcn: Simplifying and powering graph convolution network for recommendation. In _SIGIR_.
  * He et al. (2018) Xiangnan He, Zhankui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018\.  Nais: Neural attentive item similarity model for recommendation.  _TKDE_ (2018).
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _WWW_.
  * Hidasi et al. (2015) Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015.  Session-based recommendations with recurrent neural networks.  _arXiv preprint arXiv:1511.06939_ (2015).
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015\.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ (2015).
  * Hofstätter et al. (2020) Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020\.  Improving efficient neural ranking models with cross-architecture knowledge distillation.  _arXiv preprint arXiv:2010.02666_ (2020).
  * Hou et al. (2022a) Yupeng Hou, Binbin Hu, Zhiqiang Zhang, and Wayne Xin Zhao. 2022a.  Core: simple and effective session-based recommendation within consistent representation space. In _SIGIR_.
  * Hou et al. (2022b) Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022b.  Towards Universal Sequence Representation Learning for Recommender Systems. In _SIGKDD_.
  * Hou et al. (2022c) Yupeng Hou, Xingyu Pan, Wayne Xin Zhao, Shuqing Bian, Yang Song, Tao Zhang, and Ji-Rong Wen. 2022c.  Leveraging Search History for Improving Person-Job Fit. In _DASFAA_.
  * Huang et al. (2020) Zeyi Huang, Yang Zou, BVK Kumar, and Dong Huang. 2020\.  Comprehensive attention self-distillation for weakly-supervised object detection.  _NeurIPS_ (2020).
  * Jiang et al. (2020) Junshu Jiang, Songyun Ye, Wei Wang, Jingran Xu, and Xiaosheng Luo. 2020.  Learning effective representations for person-job fit by feature fusion. In _CIKM_.
  * Kabbur et al. (2013) Santosh Kabbur, Xia Ning, and George Karypis. 2013.  Fism: factored item similarity models for top-n recommender systems. In _SIGKDD_.
  * Kang et al. (2021) SeongKu Kang, Junyoung Hwang, Wonbin Kweon, and Hwanjo Yu. 2021\.  Topology distillation for recommender system. In _SIGKDD_.
  * Kang and McAuley (2018) Wang-Cheng Kang and Julian McAuley. 2018.  Self-attentive sequential recommendation. In _ICDM_.
  * Kleinerman et al. (2018) Akiva Kleinerman, Ariel Rosenfeld, Francesco Ricci, and Sarit Kraus. 2018.  Optimally balancing receiver and recommended users’ importance in reciprocal recommender systems. In _RecSys_.
  * Koren (2008) Yehuda Koren. 2008\.  Factorization meets the neighborhood: a multifaceted collaborative filtering model. In _SIGKDD_.
  * Le et al. (2019) Ran Le, Wenpeng Hu, Yang Song, Tao Zhang, Dongyan Zhao, and Rui Yan. 2019\.  Towards effective and interpretable person-job fitting. In _CIKM_.
  * Lin et al. (2017) Shuyi Lin, Wenxing Hong, Dingding Wang, and Tao Li. 2017\.  A survey on expert finding techniques.  _Journal of Intelligent Information Systems_ (2017).
  * Liu et al. (2020) Weijie Liu, Peng Zhou, Zhiruo Wang, Zhe Zhao, Haotang Deng, and Qi Ju. 2020\.  FastBERT: a Self-distilling BERT with Adaptive Inference Time. In _ACL_.
  * Luan et al. (2019) Yunteng Luan, Hanyu Zhao, Zhi Yang, and Yafei Dai. 2019\.  Msd: Multi-self-distillation learning via multi-classifiers within deep neural networks.  _arXiv preprint arXiv:1911.09418_ (2019).
  * Luo et al. (2020a) Linhao Luo, Kai Liu, Dan Peng, Yaolin Ying, and Xiaofeng Zhang. 2020a.  A motif-based graph neural network to reciprocal recommendation for online dating. In _ICONIP_.
  * Luo et al. (2020b) Linhao Luo, Liqi Yang, Ju Xin, Yixiang Fang, Xiaofeng Zhang, Xiaofei Yang, Kai Chen, Zhiyuan Zhang, and Kai Liu. 2020b.  Rrcn: A reinforced random convolutional network based reciprocal recommendation approach for online dating.  _arXiv preprint arXiv:2011.12586_ (2020).
  * Neve and Palomares (2019) James Neve and Ivan Palomares. 2019.  Latent factor models and aggregation operators for collaborative filtering in reciprocal recommender systems. In _RecSys_.
  * Paranjape et al. (2017) Ashwin Paranjape, Austin R Benson, and Jure Leskovec. 2017\.  Motifs in temporal networks. In _WSDM_.
  * Pizzato et al. (2010) Luiz Pizzato, Tomek Rej, Thomas Chung, Irena Koprinska, and Judy Kay. 2010.  RECON: a reciprocal recommender for online dating. In _RecSys_.
  * Qin et al. (2018) Chuan Qin, Hengshu Zhu, Tong Xu, Chen Zhu, Liang Jiang, Enhong Chen, and Hui Xiong. 2018\.  Enhancing person-job fit for talent recruitment: An ability-aware neural network approach. In _SIGIR_.
  * Qu et al. (2018) Yuanhang Qu, Hongzhi Liu, Yingpeng Du, and Zhonghai Wu. 2018\.  Reciprocal ranking: A hybrid ranking algorithm for reciprocal recommendation. In _PRICAI_.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian personalized ranking from implicit feedback. In _UAI_.
  * Rendle et al. (2010) Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010\.  Factorizing personalized markov chains for next-basket recommendation. In _WWW_.
  * Sanh et al. (2019) Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019.  DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.  _arXiv preprint arXiv:1910.01108_ (2019).
  * Shen et al. (2022) Yiqing Shen, Liwu Xu, Yuzhe Yang, Yaqian Li, and Yandong Guo. 2022.  Self-distillation from the last mini-batch for consistency regularization. In _CVPR_.
  * Sun et al. (2019) Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019.  BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In _CIKM_.
  * Tang and Wang (2018) Jiaxi Tang and Ke Wang. 2018.  Personalized top-n sequential recommendation via convolutional sequence embedding. In _WSDM_.
  * Tian et al. (2023) Zhen Tian, Ting Bai, Zibin Zhang, Zhiyuan Xu, Kangyi Lin, Ji-Rong Wen, and Wayne Xin Zhao. 2023.  Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation. In _WSDM_.
  * Tu et al. (2021) Ke Tu, Peng Cui, Daixin Wang, Zhiqiang Zhang, Jun Zhou, Yuan Qi, and Wenwu Zhu. 2021.  Conditional graph attention networks for distilling and refining knowledge graphs in recommendation. In _CIKM_.
  * Tu et al. (2014) Kun Tu, Bruno Ribeiro, David Jensen, Don Towsley, Benyuan Liu, Hua Jiang, and Xiaodong Wang. 2014.  Online dating recommendations: matching markets and learning preferences. In _WWW_. 787–792.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _NIPS_ (2017).
  * Wang et al. (2021) Zhenlei Wang, Jingsen Zhang, Hongteng Xu, Xu Chen, Yongfeng Zhang, Wayne Xin Zhao, and Ji-Rong Wen. 2021.  Counterfactual data-augmented sequential recommendation. In _SIGIR_.
  * Wu et al. (2020) Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. 2020\.  SSE-PT: Sequential recommendation via personalized transformer. In _RecSys_.
  * Wu et al. (2019) Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019\.  Session-based recommendation with graph neural networks. In _AAAI_.
  * Xia et al. (2015) Peng Xia, Benyuan Liu, Yizhou Sun, and Cindy Chen. 2015\.  Reciprocal recommendation system for online dating. In _ASONAM_.
  * Xia et al. (2022) Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Guandong Xu, and Quoc Viet Hung Nguyen. 2022.  On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation. In _SIGIR_.
  * Xie et al. (2022) Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Bolin Ding, and Bin Cui. 2022.  Contrastive learning for sequential recommendation. In _ICDE_.
  * Xu et al. (2019) Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou. 2019\.  Graph Contextualized Self-Attention Network for Session-based Recommendation.. In _IJCAI_.
  * Xu and Liu (2019) Ting-Bing Xu and Cheng-Lin Liu. 2019.  Data-distortion guided self-distillation for deep neural networks. In _AAAI_.
  * Xu et al. (2020) Yige Xu, Xipeng Qiu, Ligao Zhou, and Xuanjing Huang. 2020\.  Improving bert fine-tuning via self-ensemble and self-distillation.  _arXiv preprint arXiv:2002.10345_ (2020).
  * Yang et al. (2022a) Chen Yang, Yupeng Hou, Yang Song, Tao Zhang, Ji-Rong Wen, and Wayne Xin Zhao. 2022a.  Modeling Two-Way Selection Preference for Person-Job Fit. In _RecSys_.
  * Yang et al. (2022b) Chenxiao Yang, Junwei Pan, Xiaofeng Gao, Tingyu Jiang, Dapeng Liu, and Guihai Chen. 2022b.  Cross-task knowledge distillation in multi-task recommendation. In _AAAI_.
  * Yang et al. (2017) Shuo Yang, Mohammed Korayem, Khalifeh AlJadda, Trey Grainger, and Sriraam Natarajan. 2017\.  Combining content-based and collaborative filtering for job recommendation system: A cost-sensitive Statistical Relational Learning approach.  _KBS_ (2017).
  * Yıldırım et al. (2021) Ezgi Yıldırım, Payam Azad, and Şule Gündüz Öğüdücü. 2021\.  biDeepFM: A multi-objective deep factorization machine for reciprocal recommendation.  _Engineering Science and Technology, an International Journal_ (2021).
  * Yuan et al. (2019) Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2019.  A simple convolutional generative network for next item recommendation. In _WSDM_.
  * Zhang et al. (2019) Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. 2019\.  Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In _ICCV_.
  * Zhao et al. (2022a) Wayne Xin Zhao, Yupeng Hou, Xingyu Pan, Chen Yang, Zeyu Zhang, Zihan Lin, Jingsen Zhang, Shuqing Bian, Jiakai Tang, Wenqi Sun, et al. 2022a.  RecBole 2.0: Towards a More Up-to-Date Recommendation Library. In _CIKM_.
  * Zhao et al. (2022b) Wayne Xin Zhao, Zihan Lin, Zhichao Feng, Pengfei Wang, and Ji-Rong Wen. 2022b.  A revisiting study of appropriate offline evaluation for top-N recommendation algorithms.  _TOIS_ (2022).
  * Zhao et al. (2021) Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, et al. 2021\.  Recbole: Towards a unified, comprehensive and efficient framework for recommendation algorithms. In _CIKM_.
  * Zhou et al. (2023) Kun Zhou, Hui Wang, Ji-rong Wen, and Wayne Xin Zhao. 2023\.  Enhancing Multi-View Smoothness for Sequential Recommendation Models.  _ACM Trans. Inf. Syst._ (2023).
  * Zhou et al. (2020) Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020\.  S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In _CIKM_.
  * Zhou et al. (2022) Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. 2022\.  Filter-enhanced MLP is all you need for sequential recommendation. In _WWW_.
  * Zhu et al. (2018) Chen Zhu, Hengshu Zhu, Hui Xiong, Chao Ma, Fang Xie, Pengliang Ding, and Pan Li. 2018\.  Person-job fit: Adapting the right talent for the right job with joint representation learning.  _TMIS_ (2018).


$\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}p_{\mathbf{a}\ast\mathbf{b}}=\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}p_{\mathbf{a}}\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}p_{\mathbf{b}}\leq
1.$

Substituting the above into (4.8) proves that for all $\varepsilon>0$ we have

$|\widehat{\mu}(\xi)|^{2}\lesssim
e^{(2k+1)n\varepsilon}e^{-(2k+1)hn}\cdot\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}\left|\int
e(\xi\varphi_{\mathbf{a}\ast\mathbf{b}}(x))d\mu(x)\right|^{2}+k^{2}e^{-\delta
n}.$

Next, we will reduce the upper bound to exponential sums. We will use the
fundamental theorem of calculus to replace the term
$\varphi_{\mathbf{a}\ast\mathbf{b}}(x)-\varphi_{\mathbf{a}\ast\mathbf{b}}(y)$
with a product of derivatives. We begin by observing that for a fixed
$\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})$ we have

$\displaystyle\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}\left|\int
e(\xi\varphi_{\mathbf{a}\ast\mathbf{b}}(x))d\mu(x)\right|^{2}$
$\displaystyle=\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}\int\int
e(\xi(\varphi_{\mathbf{a}\ast\mathbf{b}}(x)-\varphi_{\mathbf{a}\ast\mathbf{b}}(y)))\,d\mu(x)d\mu(y)$
$\displaystyle\leq\int\int\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\xi(\varphi_{\mathbf{a}\ast\mathbf{b}}(x)-\varphi_{\mathbf{a}\ast\mathbf{b}}(y)))\right|\,d\mu(x)d\mu(y)$

We now focusing on the final term in the above. For any $x,y$ we let

$\eta(x,y):=\xi e^{-2k\lambda
n}(\varphi_{\mathbf{a}_{k}}(x)-\varphi_{\mathbf{a}_{k}}(y)).$

Then by the regularity of $\mathbf{a}_{k}$, we have

$e^{-\varepsilon n}e^{\varepsilon_{0}n}|x-y|\leq|\eta(x,y)|\leq e^{\varepsilon
n}e^{\varepsilon_{0}n}|x-y|.$

Appealing to the fundamental theorem of calculus, and duplicating the
arguments from [55] we have

$\left|\xi(\varphi_{\mathbf{a}\ast\mathbf{b}}(x)-\varphi_{\mathbf{a}\ast\mathbf{b}}(y))-\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|\lesssim
e^{2k}e^{(k+2)\varepsilon n}e^{-\lambda n}e^{\varepsilon_{0}n}.$

This in turn implies that

$\displaystyle\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\xi(\varphi_{\mathbf{a}\ast\mathbf{b}}(x)-\varphi_{\mathbf{a}\ast\mathbf{b}}(y)))\right|$
$\displaystyle\lesssim$
$\displaystyle\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|+\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e^{2k}e^{(k+2)\varepsilon
n}e^{-\lambda n}e^{\varepsilon_{0}n}$ $\displaystyle\lesssim$
$\displaystyle\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|+e^{2k}e^{(2k+2)\varepsilon
n}e^{hkn}e^{-\lambda n}e^{\varepsilon_{0}n}.$

In the final line we have used that
$\sharp\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})\leq e^{\varepsilon
nk}e^{khn}$. Thus

$\displaystyle|\widehat{\mu}(\xi)|^{2}\lesssim$ $\displaystyle
e^{(2k+1)\varepsilon
n}e^{-(2k+1)hn}\cdot\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\int\int\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|d\mu(x)d\mu(y)$
$\displaystyle+\,$ $\displaystyle e^{(2k+1)\varepsilon
n}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}e^{2k}e^{(2k+2)\varepsilon
n}e^{hkn}e^{-\lambda n}e^{\varepsilon_{0}n}$ $\displaystyle+\,$ $\displaystyle
k^{2}e^{-\delta n/2}.$

Now using the inequality
$\sharp\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})\leq
e^{(k+1)\varepsilon n}e^{(k+1)hn},$ we see that the above implies

$\displaystyle|\widehat{\mu}(\xi)|^{2}\lesssim$ $\displaystyle
e^{(2k+1)\varepsilon
n}e^{-(2k+1)hn}\cdot\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\int\int\left|\sum_{b\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|\,d\mu(x)d\mu(y)$
$\displaystyle+$ $\displaystyle e^{2k}e^{(5k+3)\varepsilon n}e^{-\lambda
n}e^{\varepsilon_{0}n}+k^{2}e^{-\delta n/2}.$

As $\varepsilon_{0}\leq\lambda/4$, and $\varepsilon>0$ is chosen small enough
depending on $k$, the latter two terms go to zero exponentially in $n$, which
in turn means polynomially in $|\xi|$. It remains to estimate the term with
the integrals above.

By the self-conformality of $\mu$, and using the assumption that our IFS is
non-trivial, we know that there exists $\kappa>0$ such that

$\mu(B(x,r))\lesssim r^{\kappa}.$ (4.9)

for any $x\in\mathbb{R}$ and $r>0$. The proof of this follows by adapting the
proof of Feng and Lau [25] on overlapping self-similar measures. Using (4.9)
it follows that

$\mu\times\mu(\\{(x,y):|x-y|\leq e^{\varepsilon
n}e^{-\varepsilon_{0}n/2}\\})\lesssim e^{\varepsilon
n\kappa}e^{-\varepsilon_{o}\kappa n/2}.$

Using this bound we have

$\displaystyle e^{\varepsilon
n(2k+1)}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\int\int\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|d\mu(x)d\mu_{(}y)$
$\displaystyle\leq$ $\displaystyle e^{\varepsilon
n(2k+1)}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\int\int\limits_{|x-y|\geq
e^{\varepsilon
n}e^{-\varepsilon_{0}n/2}}\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|d\mu(x)d\mu(y)$
$\displaystyle+e^{\varepsilon
n(2k+1)}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e^{\varepsilon
n\kappa}e^{-\varepsilon_{o}\kappa n/2},$

which is bounded by

$\displaystyle e^{\varepsilon
n(2k+1)}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\int\int\limits_{|x-y|\geq
e^{\varepsilon
n}e^{-\varepsilon_{0}n/2}}\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|d\mu(x)d\mu(y)$
$\displaystyle+e^{\varepsilon n(4k+2)}e^{\varepsilon
n\kappa}\varepsilon^{-\varepsilon_{0}\kappa n/2}.$

Here as $\varepsilon>0$ can be chosen small enough in terms of $k$ and
$\kappa$, the second term here goes to zero exponentially in $n$, so
polynomially in $|\xi|$. Thus we can just focus on bounding the integral term
above.

If a pair of points $x$ and $y$ satisfies $|x-y|\geq e^{\varepsilon
n}e^{-\varepsilon_{0}n/2}$, we have $\eta(x,y)\geq e^{\varepsilon_{0}n/2}$ so
$\eta(x,y)\in J_{n}(\varepsilon,\varepsilon_{0})$. Hence

$\displaystyle e^{\varepsilon
n(2k+1)}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\int\int\limits_{|x-y|\geq
e^{\varepsilon
n}e^{-\varepsilon_{0}n/2}}\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta(x,y)\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|d\mu(x)d\mu(y)$
$\displaystyle\qquad\leq e^{\varepsilon
n(2k+1)}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\sup_{\eta\in
J_{n}(\varepsilon,\varepsilon_{0})}\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|.$

Recall that in Proposition 4.4 we defined $\mathcal{W}$ to be the set of
$(k+1)$-tuples
$\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})$ such that
for all $j=1,\dots,k$, $\eta\in J_{n}(\varepsilon,\varepsilon_{0})$ and
$\sigma\in[R^{-2}|\eta|^{-1},|\eta|^{-\varepsilon_{1}}]$, we have that

$\sharp\\{({\bf
b,c})\in\mathcal{R}_{n}(\varepsilon,\varepsilon_{0})^{2}:|\zeta_{j,{\mathbf{a}}}({\mathbf{b}})-\zeta_{j,{\mathbf{a}}}({\mathbf{c}})|\leq\sigma\\}\leq\sharp\mathcal{R}_{n}(\varepsilon,\varepsilon_{0})^{2}\sigma^{\varepsilon_{0}/4}.$

By Proposition 4.4 there exists $\kappa_{0}>0$ such that

$\frac{\sharp(\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})\setminus\mathcal{W})}{\sharp\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\lesssim
ke^{\varepsilon\kappa_{0}n-\varepsilon_{0}^{2}\varepsilon_{1}n/24}$

Using this bound for $\mathcal{W}$ together with
$\sharp\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})\lesssim
e^{\varepsilon(k+1)n}e^{(k+1)hn}$ and
$\sharp\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})\lesssim e^{\varepsilon
kn}e^{khn},$ we can deduce the following bound

$\displaystyle e^{\varepsilon
n(2k+1)}e^{-(2k+1)hn}\sum_{\mathbf{a}\in\mathcal{R}_{n}^{k+1}(\varepsilon,\varepsilon_{0})}\sup_{\eta\in
J_{n}(\varepsilon,\varepsilon_{0})}\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|$
$\displaystyle\lesssim e^{(3k+2)\varepsilon
n}e^{-khn}\max_{\mathbf{a}\in\mathcal{W}}\sup_{\eta\in
J_{n}(\varepsilon,\varepsilon_{0})}\left|\sum_{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}e(\eta\zeta_{1,\mathbf{a}}(b_{1})\cdots\zeta_{k,\mathbf{a}}(b_{k}))\right|+ke^{(4k+2+\kappa_{0})\varepsilon
n}e^{-\varepsilon_{0}^{2}\varepsilon_{1}n/24}.$

The second term in the above decays to zero exponentially in $n$, and
therefore polynomially in $|\xi|,$ for $\varepsilon$ sufficiently small. It
remains to bound the first term.

Recall now that

$R:=R_{\varepsilon,n}=e^{\varepsilon n}$

and

$\mathcal{Z}_{j}:=\mathcal{R}_{n}(\varepsilon,\varepsilon_{0}),\quad\text{for
all }j=1,\dots,k$

and the maps
$\zeta_{j}:=\zeta_{j,\mathbf{a}}:\mathcal{R}_{n}(\varepsilon,\varepsilon_{0})\to[R^{-1},R]$
will be defined by

$\zeta_{j,\mathbf{a}}(\mathbf{b}):=e^{2\lambda
n}|\varphi_{\mathbf{a}_{j-1}\mathbf{b}}^{\prime}(x_{\mathbf{a}_{j}})|,\quad\mathbf{b}\in\mathcal{R}_{n}(\varepsilon,\varepsilon_{0}),$

If we now fix $\eta\in J_{n}(\varepsilon,\varepsilon_{0})$,
$\mathbf{a}\in\mathcal{W}$ and
$\sigma\in[R^{-2}|\eta|^{-1},|\eta|^{-\varepsilon_{1}}]$, then as

$\sharp\\{(\mathbf{b},\mathbf{c})\in\mathcal{R}_{n}(\varepsilon,\varepsilon_{0})^{2}:|\zeta_{j,\mathbf{a}}(\mathbf{b})-\zeta_{j,\mathbf{a}}(\mathbf{c})|\leq\sigma\\}\leq\sharp\mathcal{R}_{n}(\varepsilon,\varepsilon_{0})^{2}\sigma^{\varepsilon_{0}/4}$

we have by Theorem 4.1 that

$e^{(3k+2)\varepsilon
n}e^{-khn}\max_{\mathbf{a}\in\mathcal{W}}\Big{|}\sum\limits_{{\mathbf{b}\in\mathcal{R}_{n}^{k}(\varepsilon,\varepsilon_{0})}}e(\eta\zeta_{1,{\mathbf{a}}}({\mathbf{b}_{1}})...\zeta_{k,{\mathbf{a}}}({\mathbf{b}}_{k}))\Big{|}\lesssim
e^{(4k+2)\varepsilon n}|\eta|^{-\varepsilon_{2}}\lesssim e^{(4k+2)\varepsilon
n}e^{-\varepsilon_{0}\varepsilon_{2}n/2}$

since $|\eta|\geq e^{\varepsilon_{0}n/2}$ by the definition of
$J_{n}(\varepsilon,\varepsilon_{0})$ as $\eta\in
J_{n}(\varepsilon,\varepsilon_{0})$. By making sure that $\varepsilon>0$ is
chosen small enough, we have proven that $\widehat{\mu}(\xi)$ decays to zero
polynomially in $|\xi|$. ∎

##  5\. Fractal Uncertainty Principles from Fourier decay

Finally, we give the details of the proof of how the Fractal Uncertainty
Principle follows from Fourier decay. The proof method here is adapted from
the proof of the Fractal Uncertainty Principle for the Patterson-Sullivan
measure [9, 22] adapted to the measures we have with weaker regularity. Fix
$\mu_{j}$ as $(C_{j}^{-},\delta_{j}^{-},C_{j}^{+},\delta_{j}^{+},h)$-Frostman
measures, $j=1,2$, and denote their supports by $K_{1}$ and $K_{2}$.

Given $h>0$, define the semiclassical Fourier transform of
$f:\mathbb{R}^{d}\to\mathbb{C}$ by:

$\mathcal{F}_{h}f(\xi):=\frac{1}{(2\pi
h)^{d/2}}\int_{\mathbb{R}^{d}}e^{-ix\cdot\xi/h}f(x)\,dx,\quad\xi\in\mathbb{R}^{d}.$

Then FUP follows for $X=K_{1}+B(0,h)$ and $Y=K_{2}+B(0,h)$ if we can prove:

$\|\mathrm{\mathbf{1}}_{X}\mathcal{F}_{h}^{*}\mathrm{\mathbf{1}}_{Y}\|_{L^{2}(\mathbb{R}^{d})\to
L^{2}(\mathbb{R}^{d})}\lesssim h^{\beta}.$

Define

$\Upsilon_{X}^{\pm}(x)=\frac{1}{4h^{\delta_{1}^{\pm}}}\mu_{1}(B(x,2h))\quad\text{and}\quad\Upsilon_{Y}^{\pm}(y)=\frac{1}{4h^{\delta_{2}^{\pm}}}\mu_{2}(B(y,2h))$

Then for all $x\in X$ we have

$C_{1}^{-}\leq\Upsilon_{X}^{-}(x),\Upsilon_{X}^{+}(x)\leq C_{1}^{+},$

and similarly for all $y\in Y$ we have

$C_{2}^{-}\leq\Upsilon_{Y}^{-}(x),\Upsilon_{Y}^{+}(x)\leq C_{2}^{+}$

Indeed, for example if $x\in X$, then for some $x_{0}\in K_{1}$ we have
$|x-x_{0}|\leq h$. Thus $\mu_{1}(B(x,2h))\geq\mu_{1}(B(x_{0},h))\geq
C_{1}^{-}h^{\delta_{1}^{-}}$.

######  Lemma 5.1.

Suppose for any bounded $u:\mathbb{R}^{d}\to\mathbb{C}$ we have

$\|\sqrt{\Upsilon_{X}^{-}}\mathcal{F}_{h}^{*}\Upsilon_{Y}^{-}u\|_{L^{2}(\mathbb{R}^{d})}\lesssim
h^{\beta}\|\sqrt{\Upsilon_{Y}^{-}}u\|_{L^{2}(\mathbb{R}^{d})}.$

Then

$\|\mathrm{\mathbf{1}}_{X}\mathcal{F}_{h}^{*}\mathrm{\mathbf{1}}_{Y}\|_{L^{2}(\mathbb{R}^{d})\to
L^{2}(\mathbb{R}^{d})}\lesssim h^{\beta}.$

###### Proof.

Let $f\in L^{2}(\mathbb{R}^{d})$ be bounded. Write
$u=\frac{f\mathrm{\mathbf{1}}_{Y}}{\Upsilon_{Y}^{-}}$. Then $u$ is bounded as
$\Upsilon_{Y}^{-}$ is bounded from below. Then by definition

$\|\mathrm{\mathbf{1}}_{X}\mathcal{F}_{h}^{*}\mathrm{\mathbf{1}}_{Y}f\|_{L^{2}(\mathbb{R}^{d})}=\|\mathrm{\mathbf{1}}_{X}\mathcal{F}_{h}^{*}\Upsilon_{Y}^{-}u\|_{L^{2}(\mathbb{R}^{d})}.$

Moreover, as $C_{1}^{-}\lesssim\Upsilon_{X}^{-}$ we obtain

$\|\mathrm{\mathbf{1}}_{X}\mathcal{F}_{h}(\Upsilon_{Y}^{-}u)\|_{L^{2}(\mathbb{R}^{d})}\lesssim\|\Upsilon_{X}^{-}\mathcal{F}_{h}\Upsilon_{Y}^{-}u\|_{L^{2}(\mathbb{R}^{d})}\lesssim
h^{\beta}\|\sqrt{\Upsilon_{X}^{-}}u\|_{L^{2}(\mathbb{R}^{d})}$

by the assumption. Finally,

$\|\sqrt{\Upsilon_{Y}^{-}}u\|_{L^{2}(\mathbb{R}^{d})}=\|f\mathrm{\mathbf{1}}_{Y}\|_{L^{2}(\mathbb{R}^{d})}\leq\|f\|_{L^{2}(\mathbb{R}^{d})}.$

Bounded $L^{2}(\mathbb{R}^{d})$ functions generate $L^{2}(\mathbb{R}^{d})$, so
this implies the claim for all $L^{2}(\mathbb{R}^{d})$ functions. ∎

We will now prove

######  Proposition 5.2.

For any bounded $u:\mathbb{R}^{d}\to\mathbb{C}$ we have

$\|\sqrt{\Upsilon_{X}^{-}}\mathcal{F}_{h}^{*}\Upsilon_{Y}^{-}u\|_{L^{2}(\mathbb{R}^{d})}\lesssim
h^{\beta}\|\sqrt{\Upsilon_{Y}^{-}}u\|_{L^{2}(\mathbb{R}^{d})}.$

###### Proof.

Fix a bounded $u:\mathbb{R}^{d}\to\mathbb{C}$. For $x\in\mathbb{R}^{d}$ and
$t\in\mathbb{R}^{d}$ define the translation of any
$v:\mathbb{R}^{d}\to\mathbb{R}$ by:

$\omega_{t}v(x)=v(x-t).$

Then by Fubini’s theorem

$\|\sqrt{\Upsilon_{X}^{-}}\mathcal{F}_{h}^{*}\Upsilon_{Y}^{-}u\|_{L^{2}(\mathbb{R}^{d})}^{2}=\frac{1}{4h^{\delta_{1}^{-}}}\int_{B(0,2h)}\|w_{t}\mathcal{F}_{h}^{\ast}\Upsilon_{Y}^{-}u\|_{L^{2}(\mu_{1})}^{2}\,dt$

and

$\|\sqrt{\Upsilon_{Y}^{-}}u\|_{L^{2}(\mathbb{R}^{d})}^{2}=\frac{1}{4h^{\delta_{2}^{-}}}\int_{B(0,2h)}\|\omega_{s}v\|_{L^{2}(\mu_{2})}^{2}\,ds.$

Also

$\omega_{t}\mathcal{F}_{h}^{*}\Upsilon_{Y}^{-}u(x)=\frac{1}{4h^{d/2+{\delta_{2}^{-}}}}\int_{B(0,2h)}\int
e^{2\pi i(x-t)\cdot(y-s)/h}\omega_{s}u(y)\,d\mu_{2}(y)\,ds.$

Define an operator:

$B_{t}u(x)=\int e^{2\pi i(x-t)\cdot y/h}u(y)\,d\mu_{2}(y)$

so

$\int e^{2\pi i(x-t)\cdot(y-s)/h}\omega_{s}u(y)\,d\mu_{2}(y)=e^{-2\pi
i(x-t)\cdot s/h}B_{t}(\omega_{s}u)(x).$

Thus

$\displaystyle\|\sqrt{\Upsilon_{X}^{-}}\mathcal{F}_{h}^{*}\Upsilon_{Y}^{-}u\|_{L^{2}(\mathbb{R}^{d})}^{2}$
$\displaystyle=\frac{1}{4h^{\delta_{1}^{-}}}\int_{B(0,2h)}\|w_{t}\mathcal{F}_{h}^{\ast}\Upsilon_{Y}^{-}u\|_{L^{2}(\mu_{1})}^{2}\,dt$
$\displaystyle\lesssim h^{d-{\delta_{1}^{-}}-2{\delta_{2}^{-}}}\sup_{|t|\leq
2h}\int_{B(0,2h)}\|B_{t}(\omega_{s}u)\|_{L^{2}(\mu_{1})}^{2}\,ds.$

Now we are done if we can show

$\|B_{t}v\|_{L^{2}(\mu_{K_{1}})}\lesssim
h^{\alpha/2}\|v\|_{L^{2}(\mu_{K_{1}})}$

for any bounded $v$ and any $|t|\leq 2h$. Indeed, then

$\displaystyle h^{d-{\delta_{1}^{-}}-2{\delta_{2}^{-}}}\sup_{|t|\leq
2h}\int_{B(0,2h)}\|B_{t}(\omega_{s}u)\|_{L^{2}(\mu_{1})}^{2}\,ds$
$\displaystyle\lesssim h^{d-{\delta_{1}^{-}}-2{\delta_{2}^{-}}}\sup_{|t|\leq
2h}\int_{B(0,2h)}\|\omega_{s}u\|_{L^{2}(\mu_{1})}^{2}\,ds$
$\displaystyle\lesssim
h^{d-{\delta_{1}^{-}}-{\delta_{2}^{-}}+\alpha/2}\|\sqrt{\Upsilon_{Y}^{-}}u\|_{L^{2}(\mathbb{R}^{d})}^{2}.$

We can do this step now. By the $TT^{*}$ theorem [28], we have

$\|B_{t}\|_{L^{2}(\mu_{1})\to
L^{2}(\mu_{1})}^{2}=\|B_{t}B_{t}^{*}\|_{L^{2}(\mu_{1})\to L^{2}(\mu_{1})}$

and we can write

$B_{t}B_{t}^{*}v(z)=\int K(z,z^{\prime})v(z^{\prime})\,d\mu_{2}(z^{\prime})$

with kernel

$K(z,z^{\prime}):=\int e^{-2\pi i(z-z^{\prime})y/h}\,d\mu_{2}(y).$

By Schur’s inequality for $K(z,z^{\prime})$, we obtain:

$\|B_{t}\|_{L^{2}(\mu_{1})\to L^{2}(\mu_{1})}^{2}\leq\sup_{z\in
K_{1}}\int|K(z,z^{\prime})|\,d\mu_{2}(z^{\prime}).$

By Fourier decay assumption on $\mu_{2}$, we have:

$|K(z,z^{\prime})|=\Big{|}\widehat{\mu_{2}}\Big{(}\frac{z-z^{\prime}}{h}\Big{)}\Big{|}\lesssim\Big{|}\frac{z-z^{\prime}}{h}\Big{|}^{-\alpha}.$

Splitting

$\int|K(z,z^{\prime})|\,d\mu_{2}(z)=\int_{|z-z^{\prime}|\geq
h^{1/2}}|K(z,z^{\prime})|\,d\mu_{2}(z)+\int_{|z-z^{\prime}|\leq
h^{1/2}}|K(z,z^{\prime})|\,d\mu_{2}(z),$

we get that the first integral is bounded by $\lesssim h^{\alpha/2}$ and the
second integral by $\lesssim h^{\delta_{2}^{+}/2}$ using the Frostman
assumption of $\mu_{2}$. Finally, we know that $2\alpha\leq\delta_{2}^{+}$, so
the claim is done. ∎

## Acknowledgements

We thank Amir Algom, Semyon Dyatlov, Jonathan Fraser, Antti Käenmäki, Connor
Stevens, Sascha Troscheit and Meng Wu for useful discussions during the
preparation of this manuscript, in particular Amir Algom for coordinating the
submission of this and the independent work [4] simultaneously.

## References

  * [1] A. Algom, S. Baker, P. Shmerkin. On normal numbers and self-similar measures, Adv. Math. 399 (2022), Paper No. 108276, 17 pp.
  * [2] A. Algom, F. Hertz, Z. Wang. Pointwise normality and Fourier decay for self-conformal measures Adv. Math., 2021, to appear, arXiv:2012.06529.
  * [3] A. Algom, F. Hertz, Z. Wang. Logarithmic Fourier decay for self-conformal measures Journal of London Math. Soc. (2022)
  * [4] A. Algom, F. Hertz, Z. Wang. Polynomial Fourier decay and a cocycle version of Dolgopyat’s method for self conformal measures, Preprint (2023)
  * [5] A. Avila, S. Gouëzel, J. Yoccoz. Exponential mixing for the Teichmüller flow. Publ. Math. IHES 104, 143–211 (2006).
  * [6] A. Backus, J. Leng, Z. Tao. The fractal uncertainty principle via Dolgopyat’s method in higher dimensions, 2023, arXiv:2302.11708
  * [7] B. Bárány, M. Hochman, A. Rapaport. Hausdorff dimension of planar self-affine sets and measures. Invent. Math. 216, 601–659 (2019)
  * [8] J. Bourgain. The discretized sum-product and projection theorems, J. Anal. Math. 112(2010), 193–236.
  * [9] J. Bourgain and S. Dyatlov.  Fourier dimension and spectral gaps for hyperbolic surfaces.  GAFA 27(2017), 744–771
  * [10] J. Bourgain and S. Dyatlov. Spectral gaps without the pressure condition, Ann. of Math. 187(2018), 825–867
  * [11] J. Brémont. Self-similar measures and the Rajchman property. Annales Henri Lebesgue, Volume 4 (2021), pp. 973-1004.
  * [12] I. Calderón, M. Magee. Explicit spectral gap for Schottky subgroups of $\mathrm{SL}(2,\mathbb{R})$, Preprint, arXiv:2303.17950, 2023.
  * [13] N. Chernov. Markov approximations and decay of correlations for Anosov flows, Ann. of Math. 147, 269-324 (1998)
  * [14] L. Cladek, T. Tao. Additive energy of regular measures in one and higher dimensions, and the fractal uncertainty principle. Analysis and PDE, to appear
  * [15] A. Cohen. Fractal uncertainty in higher dimensions. Preprint (2023), arXiv:2305.05022
  * [16] D. Dolgopyat. On decay of correlations in Anosov flows. Ann. of Math. (2) 147(1998), 357-390.
  * [17] D. Dolgopyat. Prevalence of rapid mixing in hyperbolic flows. ETDS. Volume 18, Issue 5 October 1998, pp. 1097-1114
  * [18] S. Dyatlov, L. Jin.  Semiclassical measures on hyperbolic surfaces have full support.  Acta Math., Volume 220, Number 2 (2018), 297-339.
  * [19] S. Dyatlov, L. Jin. Dolgopyat’s method and the fractal uncertainty principle Anal. PDE 11(6): 1457-1485 (2018).
  * [20] S. Dyatlov, L. Jin, S. Nonnenmacher.  Control of eigenfunctions on surfaces of variable curvature  J. Amer. Math. Soc, to appear
  * [21] S. Dyatlov, J. Zahl. Spectral gaps, additive energy, and a fractal uncertainty principle, GAFA 26(2016), 1011–1094
  * [22] S. Dyatlov. An introduction to fractal uncertainty principle, Journal of Math. Phys. 60(2019), 081505
  * [23] P. Erdös. On the smoothness properties of a family of Bernoulli convolutions. Amer. J. Math., 62:180–186, 1940.
  * [24] K. Falconer, X. Jin. Exact dimensionality and projection properties of Gaussian multiplicative chaos measures. Transactions of the American Mathematical Society, 372, 4, 2019
  * [25] D.-J. Feng, K.-S. Lau. Multifractal formalism for self-similar measures with weak separation condition. J. Math. Pures Appl., 92 (2009) 407–428.
  * [26] J. Fraser, T. Orponen, T. Sahlsten. On Fourier analytic properties of graphs Int. Math. Res. Not., no. 10, 2730-2745, 2014.
  * [27] J. Fraser, T. Sahlsten. On the Fourier analytic structure of the Brownian graph. Analysis & PDE 11, 115-132, 2018.
  * [28] J. Ginibre, G. Velo. Generalized Strichartz Inequalities for the Wave Equation Journal of Functional Analysis Volume 133, Issue 1, October 1995, Pages 50-68
  * [29] M. Hochman. On self-similar sets with overlaps and inverse theorems for entropy. Annals of Mathematics 180 (2014), no. 2, 773-822
  * [30] M. Hochman, A. Rapaport. Hausdorff Dimension of Planar Self-Affine Sets and Measures with Overlaps. J. Eur. Math. Soc. 24, 7, 2361–2441, 2022
  * [31] M. Hochman, B. Solomyak. On the dimension of Furstenberg measure for random matrix products. Invent. Math. 210, 815-875, 2017.
  * [32] W. Hoeffding. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 1963 13–30.
  * [33] J. Li. Discretized Sum-product and Fourier decay in $\mathbb{R}^{n}$, Journal d’Analyse Math. 143, (2021) pp. 763–800.
  * [34] J. Li. Fourier decay, Renewal theorem and Spectral gaps for random walks on split semisimple Lie groups, Annales Scientifiques de l’ÉNS, Tome 55, Fasc.6, pp 1613-1686, 2022
  * [35] J. Li, F. Naud, W. Pan. Kleinian Schottky groups, Patterson-Sullivan measures, and Fourier decay, with an appendix on stationarity of Patterson-Sullivan measures, to appear in Duke Math. J.
  * [36] J. Li, T. Sahlsten. Trigonometric Series and Self-similar Sets, J. Eur. Math. Soc. (2022), to appear
  * [37] J. Li, T. Sahlsten. Fourier transform of self-affine measures, Adv. Math. (2020), Volume 374, 107349
  * [38] J. Li, W. Pan. Exponential mixing of geodesic flows for geometrically finite hyperbolic manifolds with cusps, Inventiones mathematicae (2022), to appear
  * [39] T. Jordan and T. Sahlsten.  Fourier transforms of Gibbs measures for the Gauss map.  Math. Ann. (2016) Vol 364 (3). 983-1023, 2015.
  * [40] R. Kaufman.  Continued fractions and Fourier transforms.  Mathematika, 27(2). 262-267, 1980.
  * [41] O. Khalil. Exponential Mixing Via Additive Combinatorics. Preprint, arXiv:2305.00527, 2023.
  * [42] S. Lalley. Renewal theorems in symbolic dynamics, with applications to geodesic flows, noneuclidean tessellations and their fractal limits. Acta Math. 163: 1-55 (1989)
  * [43] G. Leclerc. Julia sets of hyperbolic rational maps have positive Fourier dimension, Comm. Math. Phys., to appear, 2022
  * [44] G. Leclerc. On oscillatory integrals with Holder phases, 2022, arXiv:2211.08088
  * [45] G. Leclerc. Fourier decay of equilibrium states for bunched attractors, 2023 arXiv:2301.10623
  * [46] R. Lyons. Singularity of Some Random Continued Fractions. Journal of Theoretical Probability volume 13, pages 535–545 (2000)
  * [47] M. Magee, F. Naud. Explicit spectral gaps for random covers of Riemann surfaces. Publ. Math. IHES 132, 137–179 (2020)
  * [48] M. Magee, F. Naud. Extension of Alon’s and Friedman’s conjectures to Schottky surfaces, Preprint, arXiv:2106.02555, 2021.
  * [49] E. Mihailescu. Unstable directions and fractal dimension for skew products with overlaps in fibers. Math. Z. (2011) 269:733–750
  * [50] F. Naud. Expanding maps on Cantor sets and analytic continuation of zeta functions. Ann. ENS, Serie 4, Volume 38 (2005) no. 1, p. 116-153
  * [51] W. Parry, M. Pollicott. Zeta functions and the periodic orbit structure of hyperbolic dynamics. Astérisque, tome 187-188 (1990)
  * [52] S. Pincus. Singular stationary measures are not always fractal. J Theor Probab 7, 199–208 (1994).
  * [53] F. Przytycki, M. Urbański. On the Hausdorff dimension of some fractal sets. Studia Mathematica 93.2 (1989): 155-186.
  * [54] A. Rapaport. On the Rajchman property for self-similar measures on $R^{d}$, Adv. Math. 403, 16, 108375, 2022.
  * [55] T. Sahlsten, C. Stevens. Fourier transform and expanding maps on Cantor sets. Amer. J. Math., to appear, 2022
  * [56] P. Shmerkin. On Furstenberg’s intersection conjecture, self-similar measures, and the $L^{q}$ norms of convolutions. Ann. of Math. (2) 189 (2019), no. 2, 319–391
  * [57] P. Shmerkin and V. Suomala. Spatially independent martingales, intersections, and applications. Mem. Amer. Math. Soc. 251 (2018), no. 1195
  * [58] K Simon, B Solomyak, M Urbański. Invariant measures for parabolic IFS with overlaps and random continued fractions. Transactions of the American Mathematical Society 353 (12), 5145-5164, 2001
  * [59] K Simon, B Solomyak. Hausdorff dimension for horseshoes in $\mathbb{R}^{3}$. Ergodic Theory and Dynamical Systems 19 (5), 1343-1363, 1999.
  * [60] B. Solomyak. Non-Linear Iterated Function Systems with Overlaps. Periodica Mathematica Hungarica 37, 127–141 (1998).
  * [61] B. Solomyak. Fourier decay for self-similar measures. arXiv:1906.12164, Proc. Amer. Math. Soc, to appear.
  * [62] L. Stoyanov. Spectra of Ruelle transfer operators for Axiom A flows. Nonlinearity (2011), Volume 24, Number 4
  * [63] R. Strichartz. Indiana Univ. Math. J.. Vol. 39, No. 3, pp. 797-817, 1990.
  * [64] M. Tsujii. Dyn. Sys. Int. J. 30(4), 2015, ICM Satellite Conference.
  * [65] P. Varjú. Recent progress on Bernoulli convolutions, European Congress of Mathematics, 847–867, Eur. Math. Soc., Zürich, 2018.
  * [66] P. Varjú. Self-similar sets and measures on the line, Proceedings of the ICM 2022.
  * [67] P. Varjú, H. Yu. Fourier decay of self-similar measures and self-similar sets of uniqueness, Analysis & PDE (2020), to appear, arXiv:2004.09358
  * [68] C. Walkden. Transfer operators for contractive Markov systems and stochastic stability of the invariant measure, Dynamical Systems, 28:1, 34-47, 2013.
  * [69] I. Werner. Contractive Markov Systems. J. London Math. Soc. Volume71, Issue1, February 2005 Pages 236-258
  * [70] C. Wormell. Conditional mixing in deterministic chaos 2022, arXiv:2206.09291