# Replace Scoring with Arrangement: A Contextual Set-to-Arrangement Framework
for Learning-to-Rank

Jiarui Jin∗ Shanghai Jiao Tong UniversityShanghai, China
<EMAIL_ADDRESS>,  Xianyu Chen∗ Shanghai Jiao Tong UniversityShanghai,
China<EMAIL_ADDRESS>,  Weinan Zhang  Shanghai Jiao Tong
UniversityShanghai, China<EMAIL_ADDRESS>,  Mengyue Yang  University
College LondonLondon, United Kingdom<EMAIL_ADDRESS>,  Yang Wang†
East China Normal UniversityShanghai, China<EMAIL_ADDRESS>,  Yali Du
King’s College LondonLondon, United Kingdom<EMAIL_ADDRESS>,  Yong Yu
Shanghai Jiao Tong UniversityShanghai, China<EMAIL_ADDRESS>and  Jun Wang
University College LondonLondon, United Kingdom<EMAIL_ADDRESS>
(2023)

###### Abstract.

Learning-to-rank is a core technique in the top-N recommendation task, where
an ideal ranker would be a mapping from an item set to an arrangement (a.k.a.
permutation). Most existing solutions fall in the paradigm of probabilistic
ranking principle (PRP), i.e., first score each item in the candidate set and
then perform a sort operation to generate the top ranking list. However, these
approaches neglect the contextual dependence among candidate items during
individual scoring, and the sort operation is non-differentiable. To bypass
the above issues, we propose Set-To-Arrangement Ranking (STARank), a new
framework directly generates the permutations of the candidate items without
the need for individually scoring and sort operations; and is end-to-end
differentiable. As a result, STARank can operate when only the ground-truth
permutations are accessible without requiring access to the ground-truth
relevance scores for items. For this purpose, STARank first reads the
candidate items in the context of the user browsing history, whose
representations are fed into a Plackett-Luce module to arrange the given items
into a list. To effectively utilize the given ground-truth permutations for
supervising STARank, we leverage the internal consistency property of
Plackett-Luce models to derive a computationally efficient list-wise loss.
Experimental comparisons against 9 the state-of-the-art methods on 2 learning-
to-rank benchmark datasets and 3 top-N real-world recommendation datasets
demonstrate the superiority of STARank in terms of conventional ranking
metrics. Notice that these ranking metrics do not consider the effects of the
contextual dependence among the items in the list, we design a new family of
simulation-based ranking metrics, where existing metrics can be regarded as
special cases. STARank can consistently achieve better performance in terms of
PBM and UBM simulation-based metrics.

Learning-to-Rank, Contextual Set-to-Arrangement, Efficient Supervision
Generation

††journalyear: 2023††copyright: acmlicensed††conference: Proceedings of the
32nd ACM International Conference on Information and Knowledge Management;
October 21–25, 2023; Birmingham, United Kingdom.††booktitle: Proceedings of
the 32nd ACM International Conference on Information and Knowledge Management
(CIKM ’23), October 21–25, 2023, Birmingham, United Kingdom††price:
15.00††isbn: 979-8-4007-0124-5/23/10††doi: 10.1145/3583780.3615031††ccs:
Information systems Learning to rank††ccs: Information systems
Personalization11footnotetext: Equal contributions: Jiarui Jin and Xianyu
Chen.22footnotetext: Correspondence: Yang Wang.

ACM Reference Format:
Jiarui Jin, Xianyu Chen, Weinan Zhang, Mengyue Yang, Yang Wang, Yali Du, Yong
Yu, Jun Wang. 2023. Replace Scoring with Arrangement: A Contextual Set-to-
Arrangement Framework for Learning-to-Rank. In Proceedings of the 32nd ACM
International Conference on Information and Knowledge Management, October
21–25, 2023, Birmingham, United Kingdom. ACM, New York, NY, USA, 10 pages.
https://doi.org/10.1145/3583780.3615031

##  1\. Introduction

Learning-to-rank (LTR) covers a branch of machine learning methods for
optimizing ranking performance and is a core technique for search engine and
top-N recommendation applications (Liu, 2011). Conventional LTR algorithms are
usually designed on the basis of the probability ranking principle (PRP)
(Robertson, 1977), which consists of a scoring function that assigns an
individual score to each item, and subsequently produces the ranking list by
sorting items according to their assigned scores. Note that the crucial
difference between LTR and classical machine learning tasks (e.g., regression,
classification) is that in LTR, the exact score of each item is not important,
but the relative orders of the items matter. It is supported by the behavior
analysis results on search engines (Joachims et al., 2017; Yilmaz et al.,
2014), which manifests that user interactions show strong comparison patterns,
namely users usually compare multiple items before generating a click action.

Being aware of PRP’s limitation of independently assigning the ranking score
for each item and using non-differentiable sort operations (Xia et al., 2008),
there are roughly two lines of previous researches. One direction (Ai et al.,
2018; Pang et al., 2020; Bello et al., 2018) investigates incorporating
context into the ranking process and assigning scores to candidate items while
taking into account their dependencies within the context. The other direction
(Cao et al., 2007; Guiver and Snelson, 2009; Taylor et al., 2008; Xia et al.,
2008) designs a surrogate differentiable list-wise loss based on permutation
probabilities proposed in (Plackett, 1975) to avoid the use of the sort
operation. However, all these methods still fall into a paradigm of first
individually scoring items and then putting them into correct positions in
descending order of their scores.

In this paper, we propose a new Set-To-Arrangement Ranking (STARank)
framework, which can directly generate the permutations of candidate items
without the need for individually scoring and sort operations. Hence, our
proposed framework is end-to-end differentiable, and can be directly
supervised with ground-truth permutations instead of ground-truth scores for
all candidate items. Concretely, as depicted in Figure 1(a), the ultimate goal
is to arrange the candidate items into a permutation close to the ground-truth
permeation based on the historical records from users. For this purpose, we
develop a novel read-arrange-supervise paradigm.

Our reader component, as depicted in Figure 1(b), is required to encode two
types of input items, namely, the user browsed items and the candidate items.
Their major difference lies in that the user browsed items are characterized
by their permutation orders, while the candidate items do not have an inherent
permutation order. Therefore, we first build a permutation-sensitive module to
encode the browsed items, whose outputs are further fed into a permutation-
invariant module to produce the representation of each candidate item. Then,
our arranger, as shown in Figure 1(c), applies a Plackett-Luce (PL) module to
match the candidate items to the remaining positions.

Consider that we only have access to the ground-truth permutations. We
leverage the internal consistency property of the arranger (i.e., PL models)
to derive a list-wise loss function, as shown in Figure 1(d). Compared to
existing loss functions based on the ground-truth relevance scores, our loss
focuses on identifying and correcting the relative permutations of neighboring
pairs that have errors during arranging. In other words, this list-wise loss
is typically designed for our recursive permutation generation. Our
theoretical analysis also reveals that the proposed framework is permutation-
invariant regarding the candidate items (namely any permutation of the input
items would not change the output ranking results).

In addition, we notice that conventional ranking metrics overlook the effects
of contextual dependence among the items (i.e., the probability of a user
favoring an item is affected by other items placed in the same list).
Therefore, we develop a new series of simulation (i.e., click models) based
metrics, where NDCG can be regarded as special cases of using PBM simulations
(Richardson et al., 2007).

We conduct experiments on 2 learning-to-rank benchmark datasets and 3 real-
world recommendation datasets. Empirical results demonstrate that STARank can
consistently outperform 9 the state-of-art baseline methods in terms of both
conventional ranking metrics and new proposed simulation-based metrics.

##  2\. Preliminaries

###  2.1. Problem Formulation

LTR algorithms usually assume that each item $d$ has its utility in the
context of a user (or a query) $q$111In the context of LTR (Liu, 2011), the
terms “query” and “user” normally appear in the search engines and the top-N
recommendation scenarios respectively. For simplicity, we mainly use “user” in
this paper., which is often modelled as the relevance score between $q$ and
$d$. Let $\mathcal{D}$ denote the set of candidate items and $\mathcal{D}_{q}$
denote the set of candidate items associated with $q$. Besides these candidate
items to be arranged (a.k.a., permuted), we would also have access to user
$q$’s historical records, a sequence of the browsed item list, denoted as
$\mathcal{H}_{q}$. Notably, $\mathcal{D}_{q}$ and $\mathcal{H}_{q}$ are
subsets of $\mathcal{D}$ (i.e., $\mathcal{D}_{q}\subseteq\mathcal{D}$,
$\mathcal{H}_{q}\subseteq\mathcal{D}$), and there exists a permutation order
in each $\mathcal{H}_{q}$ but no permutation order in each $\mathcal{D}_{q}$.
We further use $r_{d}$ and $r^{*}_{d}$ to denote the predicted and ground-
truth relevance of $d$ respectively.

Prevailing ranking methods formulate the ranking problem as to predict $r$ for
each item, whose risk function is defined as

(1)
$\mathcal{F}_{\mathtt{point}}=\sum_{q}\sum_{d\in\mathcal{D}_{q}}\ell_{\mathtt{point}}\Big{(}\mathtt{f}_{\mathtt{point}}(d|\mathcal{H}_{q}),r_{d}^{*}\Big{)}$

where $\ell_{\mathtt{point}}(\cdot,\cdot)$ denotes a point-wise loss function,
$\mathtt{f}_{\mathtt{point}}(d|\mathcal{H}_{q})$ is a ranker that makes
prediction over one item $d$ in the context of $\mathcal{H}_{q}$. In practice,
following PRP, the trained ranker is used to assign an individual score for
each item independently and sort the items according to their scores. However,
this pipeline internally suffers from the following limitations: (i) it lacks
the consideration over the contextual dependence among items, as the score of
each item is solely determined by itself; (ii) it requires the sort operation,
which brings the non-differentiable issue.

Table 1. A summary of notations associated with $\pi$.  Notations |  Explanations
---|---
$\pi_{i}\in\mathcal{D}_{q}$ |  The item at the $i$-th position
$\pi_{<i}\subseteq\mathcal{D}_{q}$ |  A set of the items at the higher positions than $i$

Instead, we formulate the ranking problem as to seek for the ground-truth
permutation for the given set of candidate items. Formally, let $\pi(q)$ and
$\pi^{*}(q)$ denote the predicted permutation and ground-truth permutation of
the given item set $\mathcal{D}_{q}$ respectively. In other words, $\pi(q)$ is
the sequence of the arranged items in $\mathcal{D}_{q}$. In the followings, we
use $\pi$ and $\pi^{*}$ instead of $\pi(q)$ and $\pi^{*}(q)$ for convenience.
We list the key notations associated with $\pi$ in Table 1.

Our set-to-arrangement ranking problem can be described as

###### Definition 2.1 (_Set-to-Arrangement Ranking Problem_).

For each user/query $q$, given a tuple
$(\mathcal{D}_{q},\mathcal{H}_{q},\pi^{*})$ where $\mathcal{D}_{q}$ is the set
of candidate items and $\pi^{*}$ is the ground-truth permutation of $q$, and
$\mathcal{H}_{q}$ is the set of browsed items; our goal is to learn a ranker
$\mathtt{f}_{\mathtt{set}}(\mathcal{D}_{q}|\mathcal{H}_{q})$ that generates
the permutation $\pi$ for the given set of items $\mathcal{D}_{q}$ in the
context of $\mathcal{H}_{q}$. The risk function can be written as

(2)
$\mathcal{F}_{\mathtt{set}}=\sum_{q}\ell_{\mathtt{list}}\Big{(}\mathtt{f}_{\mathtt{set}}(\mathcal{D}_{q}|\mathcal{H}_{q}),\pi^{*}\Big{)},$

where $\ell_{\mathtt{list}}(\cdot,\cdot)$ is a list-wise loss function.

We illustrate our task in Figure 1(a). We note that in our task, only the
ground-truth permutation $\pi^{*}$ is accessible, while the ground-truth
relevance score $r^{*}_{d}$ for each item $d$ is not available.

###  2.2. Connections to Related Work

LTR algorithms refer to a group of machine techniques designed to solve
ranking problems that have been successfully applied in multiple areas
including top-N recommendation (Duan et al., 2010), search engine (Joachims,
2002; Liu, 2011). These techniques can be broadly categorized as point-wise,
pair-wise, and list-wise methods regarding the computation of training losses.
The point-wise methods (Friedman, 2001) directly treat the ranking problems as
classifications or regressions by taking one item as the input to predict its
score. The pair-wise methods (Burges et al., 2005; Joachims, 2006) take a pair
of items as the input and optimize their relative positions in the final
ranked list. The list-wise methods (Burges, 2010; Cao et al., 2007; Xia et
al., 2008) further extend the above methods by taking multiple items together
and optimizing the ranking metrics. In addition, there are recently proposed
models to build multi-variant scoring functions for each item (Ai et al.,
2018) or design permutation-invariant encoders (Pang et al., 2020; Lee et al.,
2019) or pointer network encoders (Bello et al., 2018) for the input item set.
However, all the above methods following PBP fall into a paradigm of first
individually scoring items and then putting them into correct positions
through a non-differentiable sort operation (Xia et al., 2008) or a surrogate
loss based on permutation probability (Plackett, 1975). In contrast, STARank
directly learns to generate the permutation of the candidate items does not
involve individually scoring items and the sort operation.

##  3\. The STARank Framework

###  3.1. Overview

In this section, we present the architecture design of
$\mathtt{f}_{\mathtt{set}}(\cdot|\cdot)$ as follows. We first describe our
reader module (as shown in Figure 1(b)) to generate the representations of the
candidate items in the context of the browsed items in Section 3.2. Then, we
introduce our arranger module (as shown in Figure 1(c)) to produce the
permutation of the given candidate items in Section 3.3. We also answer how to
derive a our list-wise loss
$\ell_{\mathtt{list}}(\mathtt{f}_{\mathtt{set}}(\cdot|\cdot),\pi^{*})$ (as
shown in Figure 1(d)) in Section 3.4.

###  3.2. Reader Module

For each user/query $q$ there are two types of inputs: (i) $\mathcal{D}_{q}$
is a set of candidate items, which is expected to be encoded in a permutation-
invariant manner. In other words, any permutation of the input items should
not change the output ranking. (ii) $\mathcal{H}_{q}$ is a sequence of browsed
items that have their natural orders and thus should be encoded in a
permutation-sensitive fashion. Namely, the positions of browsed items should
be considered. In the light of this, we construct
$\mathtt{f}_{\mathtt{set}}(\mathcal{D}_{q}|\mathcal{H}_{q})$ as follows.

Let $\bm{x}_{d}$ denote the feature vector of item $d$. We first introduce a
permutation-sensitive (PS) module, denoted as $\mathtt{PS}(\cdot)$, to
represent the user embedding in a recursive manner as

(3)
$\bm{u}_{q}=\mathtt{PS}\Big{(}\bm{u}_{0},\bm{x}_{1},\ldots,\bm{x}_{|\mathcal{H}_{q}|}\Big{)},$

where $\bm{u}_{q}$ is the embedding vector for user $q$. For simplicity, we
regard the user profiles (e.g., gender, age) as the original user feature,
denoted as $\bm{u}_{0}$ which is fed into our PS module as shown in Figure
1(b). While practical, we adopt a recurrent neural network (RNN), i.e., an
LSTM (Hochreiter and Schmidhuber, 1997) to form the PS module.

Given $\bm{u}_{q}$, we then deploy a permutation-invariant (PI) module,
denoted as $\mathtt{PI}(\cdot)$ to encode the candidate items in
$\mathcal{D}_{q}$, which can be expressed as

(4)
$\\{\bm{h}_{d}|d\in\mathcal{D}_{q}\\}=\mathtt{PI}\Big{(}\\{\bm{x}_{d}|d\in\mathcal{D}_{q}\\},\bm{u}_{q}\Big{)},$

where $\bm{h}_{d}$ is the representation vector of item $d$. In particular,
our PI module is based on an attention mechanism (Vaswani et al., 2017), where
the representation vector of each item $d$ is produced by

(5) $\displaystyle\bm{h}^{\prime}_{d}$
$\displaystyle=\bm{w}_{1}^{\top}\mathtt{tanh}(\bm{W}_{1}\bm{x}_{d}+\bm{b}_{1}),\
d\in\mathcal{D}_{q},$ (6) $\displaystyle\bm{h}_{d}$
$\displaystyle=\beta_{d}\bm{h}^{\prime}_{d},\text{ where
}\beta_{d}=\frac{\bm{h}^{\prime}_{d}\bm{u}_{q}^{\top}}{\sum_{j\in\mathcal{D}_{q}}\bm{h}^{\prime}_{j}\bm{u}_{q}^{\top}},$

where $\bm{w}_{\cdot}$, $\bm{W}_{\cdot}$, $\bm{b}_{\cdot}$ are trainable
matrices and vectors.

Figure 1. An illustrated example of STARank: (a) the task of STARank is to
train a ranker $\mathtt{f}_{\mathtt{set}}(\mathcal{D}_{q}|\mathcal{H}_{q})$
which produces the permutation $\pi$ of the candidate items
$\mathcal{D}_{q}=\\{d_{\text{A}},d_{\text{B}},d_{\text{C}},d_{\text{D}},d_{\text{E}}\\}$
in the context of the user $q$ and the browsed items
$\mathcal{H}_{q}=[d_{\text{X}},d_{\text{Y}}]$. Specifically, (b) STARank reads
a set of candidate items into
$\\{\bm{h}_{\text{A}},\bm{h}_{\text{B}},\bm{h}_{\text{C}},\bm{h}_{\text{D}},\bm{h}_{\text{E}}\\}$,
where the browsed items together with the user profile are first encoded in a
permutation-sensitive fashion, and then are fed into a permutation-invariant
module to embed the candidate items. (c) STARank arranges the candidate items
into their positions (namely generating the predicted permutation $\pi$)
according to the attention scores of placing the item into the position that
are learned via a Plackett-Luce module. (d) STARank is supervised by
optimizing a list-wise loss function $\mathcal{L}$ over $\pi$ and the oracle
permutation $\pi^{*}$ where $\pi^{*}$ is given. Additionally, we introduce a
family of simulation-based ranking metrics.

By incorporating Eqs. (3), (5) and (6) into Eq. (4), we can obtain the
representation vectors of items, being aware of the representations of the
user and her browsed items.

###  3.3. Arranger Module

With the representation vector of each input item, we use a Plackett-Luce (PL)
module to generate their permutation. Formally, the PL module, denoted as
$\mathtt{PL}(\cdot)$, is defined as

(7)
$\pi=\mathtt{PL}\Big{(}\\{\bm{h}_{d}|d\in\mathcal{D}_{q}\\},\bm{u}_{q}\Big{)}.$

Noticing that the original pointer network (Vinyals et al., 2015) using LSTM
as the encoder is permutation-sensitive, we tweak the pointer network by
replacing the LSTM with the aforementioned reader.

More concretely, we construct a batch of one-hot position embedding vectors
$\\{\bm{p}_{i}\\}_{i=1}^{|\mathcal{D}_{q}|}$ to encode the position
information. These embeddings can be obtained by applying a LSTM as the
decoder.

Let $\pi_{i}$ denote the item at the $i$-th position, and $\pi_{<i}$ denote a
set of the items whose positions are smaller than $i$, namely
$\pi_{<i}=\\{\pi_{1},\ldots,\pi_{i-1}\\}$. For convenience, we define
$\pi_{<1}$ as $\emptyset$.

Then, the probability distribution of item $d$ at the $i$-th position can be
produced as

(8) $s^{i}_{d}=\bm{v}^{i}_{d}\bm{u}_{q}^{\top},\text{ where
}\bm{v}^{i}_{d}=\bm{w}_{2}^{\top}\mathtt{tanh}(\bm{W}_{2}\bm{h}_{d}+\bm{W}_{3}\bm{p}_{i}+\bm{b}_{2}),$
(9) $P(\pi_{i}=d|\pi_{<i})=\left\\{\begin{aligned}
&e^{s^{i}_{d}}/\sum_{k\in\mathcal{D}_{q}\backslash\pi_{<i}}e^{s^{i}_{k}},\text{
if }d\not\in\pi_{<i},\\\ &0,\text{ otherwise}.\end{aligned}\right.,$

where $i=1,\ldots,|\mathcal{D}_{q}|,d\in\mathcal{D}_{q}$; $s^{i}_{d}$ is the
trainable attention score associated with placing item $d$ at position $i$ and
$p^{i}_{d}=P(\pi_{i}=d|\pi_{<i})$ is the probability representing the degree
to which the PL module points item $d$ to position $i$. $p^{i}_{d}$ is
computed by a softmax over the remaining items and is set to 0 for items
already been arranged. Once item $d$ is placed at position $i$, its embedding
$\bm{h}_{d}$ is fed as the input at the next step to produce the scores of
position $i+1$.

We demonstrate the arrangement process in Figure 1(c). This way can ensure the
module holds the information on the items already been arranged at each
position to eventually output the predicted permutation $\pi$.

As Eq. (9) shows the step-by-step generation of $\pi$, we follow Eq. (10) in
(Hunter, 2004) to derive the predicted permutation $\pi$ as

(10)
$P(\pi|\mathcal{D}_{q})=\prod^{|\mathcal{D}_{q}|}_{i=1}\frac{e^{s^{i}_{\pi_{i}}}}{\sum^{|\mathcal{D}_{q}|}_{j=i}e^{s^{j}_{\pi_{j}}}},$

where $i=1,\ldots,|\mathcal{D}_{q}|$ and $s^{i}_{\pi_{i}}$ is computed by Eq.
(8). We apply Eq. (10) as $\mathtt{PL}(\cdot)$ in Eq. (7).

###  3.4. List-wise Loss Function

For each user/query $q$, we can compute the predicted permutation $\pi$, then
one direct way to construct $\ell_{\mathtt{list}}(\pi,\pi^{*})$ can be
formulated as
$\ell_{\mathtt{list}}(\pi,\pi^{*})=\sum^{|\mathcal{D}_{q}|}_{i=1}\ell_{\mathtt{point}}(\pi^{*}_{i},\pi_{i})$.
However, this formulation is a simple summation of point-wise loss that fails
to consider the contextual dependence among items.

Notice that the generation of $\pi_{i}$ is conditioned on the arranged items
(i.e., $\pi_{<i}$), as shown in Eq. (9). Therefore, the corresponding
supervision of each $\pi_{i}$ should be $\pi^{*}_{i}|\pi_{<i}$, where
$\pi^{*}|\pi_{<i}$ is the ground-truth permutation given the items in
$\pi_{<i}$ have been arranged, and $\pi^{*}_{i}|\pi_{<i}$ is the item at
position $i$ in $\pi^{*}|\pi_{<i}$. In this regard,
$\ell_{\mathtt{list}}(\pi,\pi^{*})$ can be written as

(11)
$\ell_{\mathtt{list}}(\pi,\pi^{*})=\sum^{|\mathcal{D}_{q}|}_{i=1}\ell_{\mathtt{point}}(\pi^{*}_{i}|\pi_{<i},\pi_{i}|\pi_{<i}).$

However, $\pi^{*}|\pi_{<i}$ is not available, since we only have the ground-
truth permutation $\pi^{*}$. Alternatively, we construct the following
surrogate loss to optimize the arrangement of position $i$ when the previous
items are arranged following the ground-truth. In this regard,
$\ell_{\mathtt{list}}(\pi,\pi^{*})$ can be written as

(12)
$\ell_{\mathtt{list}}(\pi,\pi^{*})=\sum^{|\mathcal{D}_{q}|}_{i=1}\ell_{\mathtt{point}}(\pi^{*}_{i}|\pi^{*}_{<i},\pi_{i}|\pi^{*}_{<i}).$

However, calculating $\pi_{i}$ conditioned on $\pi^{*}_{<i}$ is still
challenging. Therefore, we leverage the internal consistency property of our
arranger (i.e., PL models) to derive a computable version of
$\ell_{\mathtt{list}}(\pi,\pi^{*})$.

We derive the difference between $P(\pi_{i}|\pi^{*}_{<i})$ and
$P(\pi^{*}_{i}|\pi^{*}_{<i})$ as follows.

(13) $\displaystyle P(\pi_{i}|\pi^{*}_{<i})=$
$\displaystyle\frac{P(\\{\pi_{i}\\}\cup\pi^{*}_{<i})}{P(\pi^{*}_{<i})}=\frac{P(\\{\pi_{i}\\}\cup\\{\pi^{*}_{i-1}\\}\cup\pi^{*}_{<i-1})}{P(\pi^{*}_{<i})}$
$\displaystyle=$ $\displaystyle
P(\\{\pi_{i}\\}\cup\\{\pi^{*}_{i-1}\\}|\pi^{*}_{<i-1})\cdot\frac{P(\pi^{*}_{<i-1})}{P(\pi^{*}_{<i})},$

where $P(\pi_{i}|\pi^{*}_{<i})$ denotes the probability of arranging $\pi_{i}$
at position $i$ conditioned on arranged items $\pi^{*}_{<i}$, and
$P(\\{\pi_{i}\\}\cup\pi^{*}_{<i})$ denotes the probability of arranging
$\pi^{*}_{<i}$ at positions from 1 to $i-1$ and arranging $\pi_{i}$ at
position $i$. Eq. (13) tells us the main difference lies in
$P(\\{\pi_{i}\\}\cup\\{\pi^{*}_{i-1}\\}|\pi^{*}_{<i-1})$ and
$P(\\{\pi^{*}_{i}\\}\cup\\{\pi^{*}_{i-1}\\}|\pi^{*}_{<i-1})$.

We then formally describe the internal consistency property of PL models in
the following lemma.

###### Lemma 3.1.

Given a set of items $\mathcal{D}_{q}$ and one subset
$\mathcal{D}^{\prime}_{q}\subseteq\mathcal{D}_{q}$, the internal permutation
of the items in $\mathcal{D}^{\prime}_{q}$, denoted as $\pi^{\prime}$, is
consistent in the context of either $\mathcal{D}_{q}$ or
$\mathcal{D}^{\prime}_{q}$, which can be formulated as

(14)
$P(\pi^{\prime}|\mathcal{D}_{q})=P(\pi^{\prime}|\mathcal{D}^{\prime}_{q}),$

where $P(\pi^{\prime}|\mathcal{D}_{q})$ and
$P(\pi^{\prime}|\mathcal{D}^{\prime}_{q})$ are the probabilities of the items
in $\mathcal{D}^{\prime}_{q}$ satisfying $\pi^{\prime}$ in the context of
using the candidate item sets $\mathcal{D}_{q}$ and $\mathcal{D}^{\prime}_{q}$
as inputs respectively.

This property is originally proposed in (Hunter, 2004), and one can refer to
Eq. (27) in (Hunter, 2004) for the derivation of Lemma 3.1 (i.e., Eq. (14)).

Then, by assigning $\mathcal{D}^{\prime}_{q}=\\{\pi_{i},\pi^{*}_{i-1}\\}$ and
$\mathcal{D}_{q}=\\{\pi_{i}\\}\cup\pi^{*}_{<i}$, we can derive that
$P(\\{\pi_{i}\\}\cup\\{\pi^{*}_{i-1}\\})=P(\\{\pi_{i}\\}\cup\\{\pi^{*}_{i-1}\\}|\pi^{*}_{<i-1})$,
namely the relative permutation of $\pi_{i}$ and $\pi^{*}_{i-1}$ is
independent of how positions from 1 to $i-2$ are arranged. In other words, the
main difference between $P(\pi_{i}|\pi^{*}_{<i})$ and
$P(\pi^{*}_{i}|\pi^{*}_{<i})$, as derived in Eq. (13) exists in
$P(\pi_{i}|\pi^{*}_{i-1})$ and $P(\pi^{*}_{i}|\pi^{*}_{i-1})$.

For convenience, let $\pi^{*}_{0}=\emptyset$. Then, we re-write Eq. (12) as

(15)
$\ell_{\mathtt{list}}(\pi,\pi^{*})=\sum^{|\mathcal{D}_{q}|}_{i=1}\ell_{\mathtt{point}}(\pi_{i}|\pi^{*}_{i-1},\pi^{*}_{i}|\pi^{*}_{i-1}).$

Eq. (15) establishes a list-wise approach to optimize STARank by supervising
the arrangement at each position. In practice, we formulate the overall loss
as

(16) $\mathcal{L}=-\sum_{q}\sum^{|\mathcal{D}_{q}|}_{i=1}\log
P(\pi_{i}=\pi^{*}_{i}|\pi^{*}_{i-1}),$

where $P(\pi_{i}=\pi^{*}_{i}|\pi^{*}_{i-1})$ is the probability of arranging
$\pi^{*}_{i}$ after the arranged item $\pi^{*}_{i-1}$.

Compared to existing loss based on the ground-truth relevance scores,
$\mathcal{L}$ pushes STARank to fit the ground-truth permutation directly in a
position-by-position way, which corresponds to the recursive generation
process in our arranger (as shown in Eq. (9)). In other words, $\mathcal{L}$
focuses on identifying and correcting the relative permutations of neighboring
pairs that have errors during generating the arrangement.

0: item set $\mathcal{D}_{q}$, historical data $\mathcal{H}_{q}$, ground-truth
$\pi^{*}$.

0: ranker $\mathtt{f}_{\mathtt{set}}(\cdot|\cdot)$, predicted permutation
$\pi$.

1: Initialize all parameters.

2: repeat

3: for each user (or query) $q$ do

4: Read $\mathcal{H}_{q}$, $\mathcal{D}_{q}$ into $\bm{u}_{q}$,
$\\{\bm{h}_{d}|d\in\mathcal{D}_{q}\\}$ via Eqs. (3) and (4).

5: Recursively arrange $\mathcal{D}_{q}$ into $\pi$ using Eq. (7).

6: end for

7: Update parameters by minimizing $\mathcal{L}$ using Eq. (16).

8: until convergence

Algorithm 1 STARank

###  3.5. Model Analysis

Complexity. From Algorithm 1, we can see that the main components of STARank
are the reader component and the arranger component.

Let $L$ denote the length of candidate items of user/query $q$, then the time
complexity of computing an attention score of placing an item is
$O(LF_{1}F_{2}+EF_{1})$, where $E$ is the number of input features, and
$F_{1}$ and $F_{2}$ are the number of rows and columns of the attention matrix
respectively. The above process needs to iterate for each item and for each
user/query. Therefore, the overall complexity is
$O(QL^{2}F_{1}F_{2}+QLEF_{1})$ where $Q$ is the number of users/queries.

Property. We theorize that STARank is permutation-invariant regarding the
candidate items in the following lemma:

###### Lemma 3.2.

For the given item space $\mathcal{D}$, assume that its elements are finite,
i.e., $|\mathcal{D}|<\infty$. For each candidate item set
$\mathcal{D}_{q}\subseteq\mathcal{D}$ and its corresponding browsed items
$\mathcal{H}_{q}\subseteq\mathcal{D}$, the ranker
$\mathtt{f}_{\mathtt{set}}(\mathcal{D}_{q}|\mathcal{H}_{q})$ in Eq. (2)
following _STARank_ is permutation-invariant to the elements in
$\mathcal{D}_{q}$.

###### Proof.

We first re-express STARank as a function producing the predicted scores for
all the possible permutations, namely
$\mathtt{f}_{\mathtt{set}}:\mathcal{P}\rightarrow\mathbb{R}$ which can be
specified as

(17)
$P(\pi|\mathcal{D}_{q},\mathcal{H}_{q})=\mathtt{PL}\left(\mathtt{PI}(\mathcal{D}_{q},\mathtt{PS}(\mathcal{H}_{q}))\right).$

Here, as $\mathcal{H}_{q}$ is independent of $\mathcal{D}_{q}$, we can omit
$\mathtt{PS}(\mathcal{H}_{q})$ here. As shown in Eq. (6),
$\mathtt{PI}(\mathcal{D}_{q})$ employing an attention mechanism can be
regarded as the summarization of the items in $\mathcal{D}_{q}$ weighted by
the attention scores. And, the formulation of $\mathtt{PL}(\cdot)$ is provided
in Eq. (10). According to Theorem 2 in (Zaheer et al., 2017), one can easily
conclude that the whole STARank framework is permutation-invariant regarding
the candidate items. ∎

Table 2. Comparison of different rankers on three industrial top-N
recommendation datasets in terms of NDCG and MAP at positions 5, 10. *
indicates $p<0.001$ in significance tests compared to the best baseline.

Ranker | Tmall — | Alipay — | Taobao —
---|---|---|---
N@5 | N@10 | M@5 | M@10 | N@5 | N@10 | M@5 | M@10 | N@5 | N@10 | M@5 | M@10
FM | 0.1233 | 0.1140 | 0.2494 | 0.2572 | 0.1342 | 0.1277 | 0.2634 | 0.2709 | 0.1124 | 0.1106 | 0.2228 | 0.2354
DeepFM | 0.1241 | 0.1181 | 0.2515 | 0.2591 | 0.1276 | 0.1214 | 0.2551 | 0.2629 | 0.1117 | 0.1104 | 0.2209 | 0.2339
PNN | 0.1248 | 0.1185 | 0.2525 | 0.2602 | 0.1296 | 0.1227 | 0.2601 | 0.2666 | 0.1145 | 0.1123 | 0.2266 | 0.2385
LSTM | 0.1341 | 0.1287 | 0.2660 | 0.2716 | 0.1427 | 0.1348 | 0.2783 | 0.2816 | 0.1308 | 0.1284 | 0.2496 | 0.2587
GRU | 0.1311 | 0.1255 | 0.2606 | 0.2668 | 0.1422 | 0.1345 | 0.2773 | 0.2810 | 0.1300 | 0.1275 | 0.2488 | 0.2578
DIN | 0.1345 | 0.1305 | 0.2674 | 0.2725 | 0.1401 | 0.1365 | 0.2705 | 0.2751 | 0.1230 | 0.1227 | 0.2351 | 0.2457
DIEN | 0.1243 | 0.1182 | 0.2518 | 0.2594 | 0.1351 | 0.1293 | 0.2632 | 0.2696 | 0.1132 | 0.1122 | 0.2245 | 0.2366
SetRank | 0.2733 | 0.2640 | 0.4090 | 0.4284 | 0.2969 | 0.2850 | 0.4289 | 0.4458 | 0.2594 | 0.2498 | 0.3876 | 0.4062
Seq2Slate | 0.2385 | 0.2340 | 0.3720 | 0.3948 | 0.2452 | 0.2412 | 0.3771 | 0.3993 | 0.2223 | 0.2180 | 0.3437 | 0.3690
STARank${}^{-}_{\mathtt{PI}}$ | 0.3042 | 0.3251 | 0.4095 | 0.4254 | 0.3214 | 0.3264 | 0.4014 | 0.4446 | 0.2632 | 0.2734 | 0.3974 | 0.4214
STARank${}^{-}_{\mathtt{PS}}$ | 0.2952 | 0.3052 | 0.3974 | 0.4035 | 0.3046 | 0.3035 | 0.4046 | 0.4256 | 0.2678 | 0.2742 | 0.4025 | 0.4264
STARank | 0.3353∗ | 0.3745∗ | 0.4328∗ | 0.4358∗ | 0.4509∗ | 0.4803∗ | 0.5337∗ | 0.5703∗ | 0.3479∗ | 0.4278∗ | 0.4082∗ | 0.4635∗
STARank${}^{+}_{\mathtt{PBM}}$ | 0.3591∗ | 0.4311∗ | 0.4472∗ | 0.4932∗ | 0.4725∗ | 0.5046∗ | 0.5402∗ | 0.5854∗ | 0.4006∗ | 0.4868∗ | 0.4683∗ | 0.5250∗
STARank${}^{+}_{\mathtt{UBM}}$ | 0.3482∗ | 0.4208∗ | 0.4359∗ | 0.4828∗ | 0.4481∗ | 0.4709∗ | 0.5189∗ | 0.5644∗ | 0.3930∗ | 0.4792∗ | 0.4545∗ | 0.5164∗

Table 3. Comparison of different rankers on Yahoo dataset. * indicates
$p<0.001$ in significance tests.

Ranker | Yahoo —
---|---
N@5 | N@10 | M@5 | M@10
FM | 0.2122 | 0.2434 | 0.3574 | 0.3629
DeepFM | 0.2483 | 0.2490 | 0.3587 | 0.3662
PNN | 0.2563 | 0.2693 | 0.3598 | 0.3602
LSTM | 0.2763 | 0.2731 | 0.3602 | 0.3641
GRU | 0.2901 | 0.2876. | 0.3674 | 0.3675
DIN | 0.2932 | 0.2975. | 0.3772 | 0.3706
DIEN | 0.2911 | 0.2983 | 0.3703 | 0.3688
SetRank | 0.3053 | 0.3185 | 0.3932 | 0.3868
Seq2Slate | 0.3121 | 0.3234 | 0.4001 | 0.3987
STARank${}^{-}_{\mathtt{PI}}$ | 0.3188 | 0.3210 | 0.4079 | 0.4154
STARank${}^{-}_{\mathtt{PS}}$ | 0.3110 | 0.3356 | 0.4139 | 0.4201
STARank | 0.3399∗ | 0.3525∗ | 0.4301∗ | 0.4398∗
STARank${}^{+}_{\mathtt{PBM}}$ | 0.3474∗ | 0.3612∗ | 0.4342∗ | 0.4450∗
STARank${}^{+}_{\mathtt{UBM}}$ | 0.3355∗ | 0.3543∗ | 0.4324∗ | 0.4463∗

Table 4. Comparison of different rankers on LETOR dataset. * indicates
$p<0.001$ in significance tests.

Ranker | LETOR —
---|---
N@5 | N@10 | M@5 | M@10
FM | 0.1752 | 0.1721 | 0.3282 | 0.3292
DeepFM | 0.1746 | 0.1689 | 0.3243 | 0.3204
PNN | 0.1775 | 0.1739 | 0.3296 | 0.3301
LSTM | 0.1851 | 0.1795 | 0.3405 | 0.3384
GRU | 0.1739 | 0.1741. | 0.3170 | 0.3205
DIN | 0.1550 | 0.1671. | 0.2815 | 0.2988
DIEN | 0.1818 | 0.1811 | 0.3409 | 0.3402
SetRank | 0.2035 | 0.2246 | 0.4045 | 0.4002
Seq2Slate | 0.2436 | 0.2546 | 0.4122 | 0.4031
STARank${}^{-}_{\mathtt{PI}}$ | 0.2443 | 0.2568 | 0.4172 | 0.4075
STARank${}^{-}_{\mathtt{PS}}$ | 0.2467 | 0.2606 | 0.4231 | 0.4123
STARank | 0.2824∗ | 0.2622∗ | 0.4876∗ | 0.4529∗
STARank${}^{+}_{\mathtt{PBM}}$ | 0.2843∗ | 0.2646∗ | 0.4906∗ | 0.4573∗
STARank${}^{+}_{\mathtt{UBM}}$ | 0.2863∗ | 0.2668∗ | 0.4887∗ | 0.4587∗

Table 5. Comparison of different rankers on three industrial top-N
recommendation datasets in terms of the proposed simulation-based ranking
metrics based on PBM and UBM at positions 5, 10. * indicates $p<0.001$ in
significance tests compared to the best baseline.

Ranker | Tmall — | Alipay — | Taobao —
---|---|---|---
$\mathtt{P}$@5 |  $\mathtt{P}$@10 |  $\mathtt{U}$@5 |  $\mathtt{U}$@10 |  $\mathtt{P}$@5 |  $\mathtt{P}$@10 |  $\mathtt{U}$@5 |  $\mathtt{U}$@10 |  $\mathtt{P}$@5 |  $\mathtt{P}$@10 |  $\mathtt{U}$@5 |  $\mathtt{U}$@10
FM | 0.1026 | 0.1027 | 0.1015 | 0.1016 | 0.1138 | 0.1157 | 0.1013 | 0.1014 | 0.1073 | 0.1067 | 0.1013 | 0.1017
DeepFM | 0.1023 | 0.1023 | 0.1005 | 0.1008 | 0.1037 | 0.1040 | 0.1008 | 0.1009 | 0.1093 | 0.1097 | 0.1008 | 0.1015
PNN | 0.1248 | 0.1185 | 0.1014 | 0.1016 | o.1023 | 0.1021 | 0.1002 | 0.1005 | 0.1142 | 0.1155 | 0.101 | 0.1016
LSTM | 0.2074 | 0.2734 | 0.1301 | 0.1460 | 0.2137 | 0.2068 | 0.1274 | 0.1488 | 0.2498 | 0.2922 | 0.2025 | 0.2842
GRU | 0.2188 | 0.3066 | 0.1299 | 0.1437 | 0.2292 | 0.2208 | 0.1308 | 0.1512 | 0.2541 | 0.3023 | 0.1928 | 0.2747
DIN | 0.2725 | 0.2474 | 0.2493 | 0.1818 | 0.2257 | 0.2333 | 0.1443 | 0.1648 | 0.2748 | 0.2741 | 0.2099 | 0.2846
DIEN | 0.2520 | 0.3693 | 0.1014 | 0.1012 | 0.2366 | 0.2194 | 0.1281 | 0.1456 | 0.2345 | 0.2986 | 0.1891 | 0.2654
SetRank | 0.3778 | 0.3973 | 0.3378 | 0.3650 | 0.4171 | 0.4330 | 0.3346 | 0.3584 | 0.3094 | 0.3191 | 0.3704 | 0.3820
Seq2Slate | 0.3182 | 0.3455 | 0.3210 | 0.3482 | 0.3337 | 0.3600 | 0.3254 | 0.3533 | 0.3285 | 0.3541 | 0.3189 | 0.3470
STARank${}^{-}_{\mathtt{PI}}$ | 0.3962 | 0.3967 | 0.4346 | 0.4426 | 0.5025 | 0.5046 | 0.4325 | 0.4602 | 0.3429 | 0.3478 | 0.4385 | 0.4255
STARank${}^{-}_{\mathtt{PS}}$ | 0.3873 | 0.3863 | 0.4253 | 0.4371 | 0.5010 | 0.5024 | 0.4242 | 0.4523 | 0.3321 | 0.3368 | 0.4257 | 0.4211
STARank | 0.4240∗ | 0.4010∗ | 0.4435∗ | 0.4493∗ | 0.5283∗ | 0.5058∗ | 0.4578∗ | 0.4888∗ | 0.3529∗ | 0.3599∗ | 0.4418∗ | 0.4308∗

##  4\. Experiment

###  4.1. Datasets and Data Pre-processing

Dataset. We introduce 3 industrial recommendation datasets, namely Tmall,
Taobao, and Alipay, and 2 learning-to-rank benchmark datasets, namely Yahoo
and LETOR in our experiment.

  * •

Tmall222https://tianchi.aliyun.com/dataset/dataDetail?dataId=42 is a dataset
consisting of 54,925,331 interactions of 424,170 users and 1,090,390 items.
These sequential histories are collected by the Tmall e-commerce platform from
May 2015 to November 2015 with an average sequence length of 129 and 9 feature
fields.

  * •

Taobao333https://tianchi.aliyun.com/dataset/dataDetail?dataId=649 is a dataset
containing 100,150,807 interactions of 987,994 users and 4,162,024 items.
These user behaviors including several behavior types (e.g., click, purchase,
add to chart, item favoring) are collected from November 2007 to December 2007
with an average sequence length of 101 and 4 feature fields.

  * •

Alipay444https://tianchi.aliyun.com/dataset/dataDetail?dataId=53 is a dataset
collected by Alipay, an online payment application from July 2015 to November
2015. There are 35,179,371 interactions of 498,308 users and 2,200,191 items
with an average sequence length of 70 and 6 feature fields.

  * •

Yahoo555http://webscope.sandbox.yahoo.com is a dataset collected by Yahoo, an
online search engine. It consists of 29,921 queries and 710k items. Each query
document pair is represented by a 700-dimensional feature vector manually
assigned with a label denoting relevance at 5 levels (Chapelle and Chang,
2011).

  * •

LETOR666https://www.microsoft.com/en-us/research/project/letor-learning-rank-
information-retrieval/ is a package of benchmark datasets for research on
LEarning TO Rank, which uses the Gov2 web page collection and two query sets
from the Million Query track of TREC2007 and TREC2008. We conduct experiments
on MQ2007, one of two datsets for supervised learning-to-rank tasks in LETOR.
MQ2007 contains 2,476 queries and 85K documents. Each query-document pair is
represented by a 46-dimensional feature vector with a manually assigned
3-level label (Qin et al., 2010).

Data Pre-processing777We note that our setting on how to divide datasets is
different from previous work such as SetRank (Pang et al., 2020) which would
make the results not consistent with their results reported in original
papers, because our framework requires a sequence of items as the history
(i.e., $\mathcal{H}_{q}$) and another set of items as the candidate items
(i.e., $\mathcal{D}_{q}$).. We split each dataset using the timestep. Let $T$
denote the length of user browsing logs. We first filter out the data
instances whose browsing logs are smaller than 30 (i.e., $T<30$). Next, for
each user $q$, we assign 1-st to ($T$-20)-th items as the training dataset
where 1-st to ($T-30$)-th items are used as the user’s history data (i.e.,
$\mathcal{H}_{q}$) and ($T-29$)-th to ($T-20$)-th items construct the
candidate item set (i.e., $\mathcal{D}_{q}$). In validation, we use 1-st to
($T-20$)-th items as $\mathcal{H}_{q}$ and the ($T-19$)-th to ($T-10$)-th
items as $\mathcal{D}_{q}$; while for evaluation, we use 1-st to ($T-10$)-th
items as $\mathcal{H}_{q}$ and ($T-9$)-th to $T$-th items as
$\mathcal{D}_{q}$. For $\mathcal{H}_{q}$ and $\mathcal{D}_{q}$, we keep their
original browsing orders and do not adopt any initial ranker to generate the
initial ranking list.

###  4.2. Experimental Configurations

Simulation-based Ranking Metrics. Consider that real-world datasets usually
offer ground-truth relevance scores for candidate items, instead of ground-
truth permutations. Therefore, we are required to first generate the ground-
truth permutations. For this purpose, we introduce a ranking metric, denoted
as $\mathtt{R}(\cdot)$. Let $\mathcal{P}$ denote the set of all the possible
permutation of $\mathcal{D}_{q}$. Then, we have
$|\mathcal{P}|=\mathtt{A}^{|\mathcal{D}_{q}|}_{|\mathcal{D}_{q}|}=O(|\mathcal{D}_{q}|!)$
where $\mathtt{A}^{\cdot}_{\cdot}$ represents permutation operator. After we
enumerate all the possible permutations $\pi\in\mathcal{P}$, we select the one
received the highest score in terms of $\mathtt{R}(\cdot)$ as the oracle
permutation $\pi^{*}$. In other words, we have:

(18)
$\pi^{*}=\text{arg}\mathop{\text{max}}_{\pi\in\mathcal{P}}\mathtt{R}(\pi).$

We note that conventional ranking metrics, such as NDCG, only consider the
effects of the positions of the arranged items but overlook the contextual
dependence among the items (i.e., the probability of a user favoring an item
should be dependent on other items posted in the same sequence).

To this end, we propose to construct a virtual user to consider the contextual
dependence and make the evaluation according to the user’s feedbacks. These
virtual user models are known as the click models (Chuklin et al., 2015),
which generate users’ feedback by making some reasonable assumptions based on
some heuristics or eye-tracking experimental results (Joachims et al., 2005).
Formally, we use $\mathtt{CM}$ to denote an arbitrary click model (CM) (e.g.,
position-based model (PBM) (Richardson et al., 2007), user browsing model
(UBM) (Dupret and Piwowarski, 2008)) treated as a black block to generate the
user’s feedback for the input sequence. Then, we introduce a new family of the
ranking metrics based on the aforementioned simulations (i.e., click models),
which is called simulation-based ranking metrics and defined as

(19)
$\displaystyle\mathtt{R}_{\mathtt{CM}}(\pi)=\sum^{|\mathcal{D}_{q}|}_{i=1}P(c_{\pi_{i}}|\pi_{<i};\mathtt{CM})=\sum^{|\mathcal{D}_{q}|}_{i=1}P(o_{\pi_{i}}|\pi_{<i};\mathtt{CM})\cdot
P(r_{\pi_{i}}|\pi_{<i};\mathtt{CM}),$

which reflects a simple fact that a user clicks an item $d$ (i.e.,
$P(c_{d}=1)$) only when it is both observed (i.e., $P(o_{d}=1)$) and perceived
as relevant (i.e., $P(r_{d}=1)$), and the observation probability and the
relevance probability are determinated by the click model $\mathtt{CM}$. We
use PBM and UBM as examples for $\mathtt{CM}$ and introduce their detailed
configurations as follows. PBM (Richardson et al., 2007) simulates user
browsing behavior based on the assumption that the bias of a document only
depends on its position, which can be formulated as
$P(o_{i})=\rho_{i}^{\tau}$, where $\rho_{i}$ represents position bias at
position $i$ and $\tau\in[0,+\infty]$ is a parameter controlling the degree of
position bias. The position bias $\rho_{i}$ is obtained from an eye-tracking
experiment in (Joachims et al., 2005) and the parameter $\tau$ is set as 1 by
default. It also assumes that a user decides to click a document $d_{i}$
according to the probability $P(c_{i})=P(o_{i})\cdot P(r_{i})$. UBM (Dupret
and Piwowarski, 2008) is an extension of the PBM model that has some elements
of the cascade model. The examination probability depends not only on the rank
of an item $d_{i}$ but also on the rank of the previously clicked document
$d_{i^{\prime}}$ as
$P(o_{i}=1|c_{i^{\prime}}=1,c_{i^{\prime}+1}=0,\ldots,c_{i-1}=0)=\upgamma_{0}$.
Similarly, we get $\upgamma_{0}$ from the eye-tracking experiments in
(Joachims et al., 2005; Dupret and Piwowarski, 2008). The click probability is
determined by $P(c_{i})=P(o_{i})\cdot P(r_{i})$.

In this regard, NDCG metric is a special case of the proposed ranking metrics
using PBM as $\mathtt{CM}$. Concretely, the formulation of NDCG can be written
as

(20)
$\mathtt{R}_{\mathtt{NDCG}}(\pi)=\frac{1}{N}\sum^{|\mathcal{D}_{q}|}_{i=1}(\frac{1}{\log_{2}{(i+1)}})(2^{r_{\pi_{i}}}-1),$

where $N$ normalizes for the number of relevant items, and $r_{\pi_{i}}$ is
the relevance of item $\pi_{i}$. By comparing Eq. (20) to Eq. (19), we reveal
that $\mathtt{R}_{\mathtt{NDCG}}(\cdot)$ is a special case of
$\mathtt{R}_{\mathtt{CM}}(\cdot)$ by assigning
$P(o_{\pi_{i}}|\pi_{<i};\mathtt{CM})=\frac{1}{N_{o}}(\frac{1}{\log_{2}{(i+1)}}),P(r_{\pi_{i}}|\pi_{<i};\mathtt{CM})=\frac{1}{N_{r}}(2^{r_{\pi_{i}}}-1),$
where $N=N_{o}\cdot N_{r}$. We can see that the observation probability of an
arbitrary item is solely determined by its position, showing that CM here is a
special case of PBM.

Noting that STARank metrics can be seamlessly adopted in the supervision
generation, we can re-formulate Eq. (18) as

(21)
$\pi^{*}=\text{arg}\mathop{\text{max}}_{\pi\in\mathcal{P}}\mathtt{R}_{\mathtt{CM}}(\pi).$

Note that the supervisions of STARank are determined by the ranking metrics.
For those datasets described in with binary scores in the browsing logs, there
might exist multiple oracle permutations when applying either Eq. (18) or
(21). In these cases, we randomly choose one oracle permutation as the
supervision. Also, these cases would occur during the evaluation phase, where
we act similarly. We notice that there is a related paper (Sun et al., 2019)
investigating the impacts of different evaluation scheme regarding the same
scores, which is out of the scope of this paper and we leave it for future
work.

Hyperparameter Setting. The learning rate is decreased from the initial value
$1\times 10^{-2}$ to $1\times 10^{-6}$ during the training process. The batch
size is set as 100. The weight for the L2 regularization term is $4\times
10^{-5}$. The dimension of embedding vectors is set as 64. The dropout rate is
0.5. All experimental comparisons are conducted with 10 different random
seeds. All the models are trained under the same hardware settings with 6-Core
AMD Ryzen 9 5950X (2.194GHZ), 62.78GB RAM, NVIDIA GeForce RTX 3080 cards.

###  4.3. Baseline Methods and Ranking Metrics

Baseline Descriptions. We compare STARank against 9 strong baselines,
introduced as follows. FM (Rendle, 2010) is the factorization machine using
the linear projection and inner product of features to measure the user-item
relevance. DeepFM (Guo et al., 2017) is a generalized model consisting of a FM
as a wide component and a MLP as a deep component. PNN (Qu et al., 2016) is
product-based neural networks including an embedding layer and a product layer
to model user-item interactions. DIN (Zhou et al., 2018) designs a local
activation unit to capture user interests from historical records. LSTM
(Hochreiter and Schmidhuber, 1997) is a standard long short memory approach
widely used for modeling user’s sequential pattern. GRU (Hidasi et al., 2015)
uses the gate recurrent units to model sequential user behaviors. DIEN (Zhou
et al., 2019) is an extension of DIN that builds an interest extractor layer
to model user’s temporal interests. SetRank (Pang et al., 2020) uses a stack
of self-attention blocks based on set transformer (Lee et al., 2019) to model
the cross-item interactions. Seq2Slate (Bello et al., 2018) employs the
pointer network (Qi et al., 2017) to sequentially encode previously selected
items and predict the next one. All the methods are trained based on click
feedback only.

Evaluation Metrics. We evaluate the performance of these methods according to
two series of ranking metrics, namely conventional ranking metrics and
simulation-based ranking metrics built upon the specific click models. To be
more specific, for the former ones, we choose MAP (Mean Average Precision) and
NDCG (Normalized Distributed Cumulative Gain) at position 5, 10, denoted as
M@K and N@K where K is 5, 10. For the later ones, we use PBM and UBM as the
click models and compute the cumulative click probability at positions 5, 10,
which are denoted as $\mathtt{P}$@K and $\mathtt{U}$@K and K is 5, 10.

Considering that the implementation of STARank is conditioned on which click
model we choose. Here, besides the standard implementation of STARank using
NDCG as the metric, we further introduce two variants using PBM and UBM as the
click models:

  * •

STARank is the proposed framework, which uses NDCG over the whole ranking list
to compute $\pi^{*}$ following Eq. (18).

  * •

STARank${}^{+}_{\mathtt{PBM}}$ is a variant of STARank using
$\mathtt{R}_{\mathtt{PBM}}(\cdot)$ in Eq. (19) to produce $\pi^{*}$ (i.e., PBM
(Richardson et al., 2007) as the click model).

  * •

STARank${}^{+}_{\mathtt{UBM}}$ is another variant of STARank using
$\mathtt{R}_{\mathtt{UBM}}(\cdot)$ in Eq. (19) to generate $\pi^{*}$ (i.e.,
UBM (Dupret and Piwowarski, 2008) as the click model).

We also develop the following variants to investigate the impact of each
component of the proposed framework:

  * •

STARank${}^{-}_{\mathtt{PI}}$ is a variant of STARank using multi-layer
perceptron (MLP) layer as PI module to encode the input items instead of the
attention network, where the user representation is concatenated with item
embeddings to be fed into the MLP layer.

  * •

STARank${}^{-}_{\mathtt{PS}}$ is a variant of STARank using the MLP layer as
PS module to encode the sequential records instead of the LSTM.

###  4.4. Performance Comparison

Tables 2, 3, 4 and 5 summarize the comparison results on the top-N
recommendation datasets and learning-to-rank datasets, in terms of
conventional ranking metrics and simulation-based ranking metrics. Our major
findings are listed as follows.

STARank consistently outperforms all the baselines scoring and sorting. From
these tables, we can clearly see that STARank outperforms all these baselines
including classical tower-based methods (e.g., DeepFM), sequential methods
(e.g., DIEN), and recently proposed set-to-sequence methods (e.g., SetRank) in
terms of all the conventional and new proposed simulation-based ranking
metrics. Since all these baselines are supervised by the relevance score of
each item instead of directly optimizing the model by the permutation of the
items in an end-to-end fashion, these results would indicate the superiority
of the overall design of the proposed framework.

STARank works well on the proposed simulation-based metrics. From comparisons
between Tables 2 and 5, we can see that there are larger performance gains
from STARank in terms of the simulation-based metrics than the conventional
configurations. A possible explanation is that compared to the conventional
ranking metrics, the new proposed simulation-based configurations consider the
effects from the contextual dependence; and thus supervising the permutations
of the ranking list would perform better in this case.

SetRank and Seq2Slate consistently outperforms the other baselines. From these
results, we also can observe that among 9 baseline methods, SetRank and
Seq2Slate can consistently achieve the best performance. One possible reason
is that due to our experimental configuration typically assigns a set of items
served as the historical records, then these methods can effectively leverage
the contexts to better capture the user needs.

Figure 2.  Visualization of attention probabilities of placing item $d_{i}$ at
position $p_{j}$ on Alipay dataset.

###  4.5. Ablation Studies

Impact of Permutation-Invariant Module. To investigate the performance gain
from our permutation-invariant module, we evaluate the performance of
STARank${}^{-}_{\mathtt{PI}}$ and report the corresponding results in Tables
2, 3, 4, and 5. From comparisons between STARank and
STARank${}^{-}_{\mathtt{PI}}$, the results demonstrate the superiority of our
proposed attention networks introduced in Section 3.2 which incorporates user
representation to produce the representation vector for each item.

Impact of Permutation-Sensitive Module. We use STARank${}^{-}_{\mathtt{PS}}$
to show the performance changes by replacing our RNN module with the MLP layer
which is a permutation-invariant module and is not suitable to encode the
permutation-sensitive user historical data. Results summarized in Tables 2, 3,
4, and 5 verify the performance gain by using a PS module to model the user’s
history.

Figure 3.  Performance comparisons of STARank against baselines in terms of
accuracy at top 5 positions on Alipay dataset.  Figure 4. Comparisons of
training and inference time of STARank and its variants against baselines on
Alipay dataset.

Impact of Supervision Generation. As described in the experimental
configuration, the proposed simulation-based ranking metrics can not only be
used for evaluation but also can be adopted in the ground-truth permutation
generation (i.e., Eq. (21)). As Table 5 shows the experimental results of each
model under these simulation-based ranking metrics, we further form the
variants STARank${}^{+}_{\mathtt{PBM}}$ and STARank${}^{+}_{\mathtt{UBM}}$
whose supervision generations are simulation-based but evaluations are under
conventional ranking metrics. From the corresponding results shown in Table 2,
we can see that PBM-based supervisions consistently improve the ranking
performance, while UBM-based supervisions succeed in Tmall and Taobao datasets
and fail in Alipay dataset. One possible explanation is that these supervision
signals encourage the model to encode the sequence dependence based on the
assumptions of certain click models. Therefore, if these assumptions hold in
the dataset, the ranking performance would be improved; otherwise, the ranking
performance would be hurt.

###  4.6. In-Depth Analysis

Visualization Analysis. To better understand the behavior of the model, we
visualize the probabilities of the attention scores from Eq. (9), which shows
the average probability of placing item $d_{i}$ (corresponding to $d$ in Eq.
(9)) at position $p_{j}$ (corresponding to $\pi_{i}$ in Eq. (9)) at each step.
As Figure 2 depicts, the model produces the permutations that are close to the
input order, but with the some items are placed to the neighboring positions.
A possible reason is that the input orders provided by Taobao platform are
expected to achieve a good performance; and our method in this case can be
regarded as a re-ranking model to refine the input orders.

Also, these attention values provide another perspective to explain why our
framework performs better than existing ranking methods following PRP. As
revealed in (Dai et al., 2020), considering there are multiple bias in
ranking, the maximization of the utility can be seen as solving the maximum-
weight matching on the item-position bipartite graph, where the edge weight
between an item and a position denotes the utility of placing the item at this
position. In this regard, our attention scores could be the estimation of the
utility. In contrast, PRP learning to assign individual score to each item,
does not learn to estimate these the edge weights.

Accuracy at Each Position. In order to precisely investigate the model
performance at each position, we compute the accuracy (ACC) of the ranking
list provided by PNN, LSTM, DIN, Seq2Slate, and STARank, and report the
corresponding results on Alipay dataset in Figure 3. Given the predicted
permutation $\pi$ and the oracle permutation $\pi^{*}$, for each data
instance, ACC value equals 1 at position $i$ when $\pi_{i}=\pi^{*}_{i}$ holds;
0 otherwise. From Figure 3, we can see that our method can consistently
outperform all these baseline methods at all the positions. We also note that
ACC values at the highest and lowest positions are higher than other
positions. One reasonable explanation is that the items that a user extremely
favors or dislikes are much easier to distinguish than other items.

###  4.7. Robustness Analysis

We evaluate the robustness of STARank under different amounts of training data
in Alipay dataset. For comparison, we also employ LSTM as a baseline method.
Results are reported in Figure LABEL:fig:data, which demonstrate that after
reaching a threshold (60% in this case), STARank is almost as robust as LSTM
regarding the amounts of training data.

##  5\. Conclusion and Future Work

In this paper, we present a novel set-to-arrangement framework named STARank
to directly generate the permutations for input sets of items without the
needs of scoring and sorting items. Our architecture and supervision designs
together enable STARank to be fully differentiable and also allow STARank to
operate when only ground-truth permutations instead of ground-truth relevance
scores for items are accessible. For future work, it would be interesting to
investigate possible applications of the proposed framework in real-world
e-commerce platforms.

Acknowledgments. The work is partially supported by National Natural Science
Foundation of China (No. 62177033) and Shanghai Artificial Intelligence
Innovation and Development Fund (No. 2020-RGZN-02026). Jiarui Jin would like
to thank Wu Wen Jun Honorary Doctoral Scholarship from AI Institute, Shanghai
Jiao Tong University.

## References

  * (1)
  * Ai et al. (2018) Qingyao Ai, Keping Bi, Jiafeng Guo, and W Bruce Croft. 2018\.  Learning a deep listwise context model for ranking refinement. In _SIGIR_.
  * Bello et al. (2018) Irwan Bello, Sayali Kulkarni, Sagar Jain, Craig Boutilier, Ed Chi, Elad Eban, Xiyang Luo, Alan Mackey, and Ofer Meshi. 2018.  Seq2slate: Re-ranking and slate optimization with rnns.  _arXiv preprint arXiv:1810.02019_ (2018).
  * Burges et al. (2005) Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005.  Learning to rank using gradient descent. In _ICML_.
  * Burges (2010) Christopher JC Burges. 2010\.  From ranknet to lambdarank to lambdamart: An overview.  _Learning_ (2010).
  * Cao et al. (2007) Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007.  Learning to rank: from pairwise approach to listwise approach. In _ICML_.
  * Chapelle and Chang (2011) Olivier Chapelle and Yi Chang. 2011.  Yahoo! learning to rank challenge overview. In _Proceedings of the learning to rank challenge_. 1–24.
  * Chuklin et al. (2015) Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. 2015\.  Click models for web search.  _Synthesis lectures on information concepts, retrieval, and services_ 7, 3 (2015), 1–115.
  * Dai et al. (2020) Xinyi Dai, Jiawei Hou, Qing Liu, Yunjia Xi, Ruiming Tang, Weinan Zhang, Xiuqiang He, Jun Wang, and Yong Yu. 2020.  U-rank: Utility-oriented learning to rank with implicit feedback. In _CIKM_.
  * Duan et al. (2010) Yajuan Duan, Long Jiang, Tao Qin, Ming Zhou, and Heung Yeung Shum. 2010.  An empirical study on learning to rank of tweets. In _Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)_.
  * Dupret and Piwowarski (2008) Georges E Dupret and Benjamin Piwowarski. 2008.  A user browsing model to predict search engine click data from past observations.. In _SIGIR_.
  * Friedman (2001) Jerome H Friedman. 2001\.  Greedy function approximation: a gradient boosting machine.  _Annals of statistics_ (2001), 1189–1232.
  * Guiver and Snelson (2009) John Guiver and Edward Snelson. 2009.  Bayesian inference for Plackett-Luce ranking models. In _proceedings of the 26th annual international conference on machine learning_. 377–384.
  * Guo et al. (2017) Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017.  DeepFM: a factorization-machine based neural network for CTR prediction. In _IJCAI_.
  * Hidasi et al. (2015) Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015.  Session-based recommendations with recurrent neural networks.  _ICLR_ (2015).
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ 9, 8 (1997), 1735–1780.
  * Hunter (2004) David R Hunter. 2004\.  MM algorithms for generalized Bradley-Terry models.  _The annals of statistics_ (2004).
  * Joachims (2002) Thorsten Joachims. 2002\.  Optimizing search engines using clickthrough data. In _KDD_.
  * Joachims (2006) Thorsten Joachims. 2006\.  Training linear SVMs in linear time. In _KDD_.
  * Joachims et al. (2017) Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2017.  Accurately interpreting clickthrough data as implicit feedback. In _SIGIR_.
  * Joachims et al. (2005) Thorsten Joachims, Laura A Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005.  Accurately interpreting clickthrough data as implicit feedback. In _SIGIR_.
  * Lee et al. (2019) Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. 2019\.  Set transformer: A framework for attention-based permutation-invariant neural networks. In _ICML_.
  * Liu (2011) Tie-Yan Liu. 2011\.  Learning to rank for information retrieval.  (2011).
  * Pang et al. (2020) Liang Pang, Jun Xu, Qingyao Ai, Yanyan Lan, Xueqi Cheng, and Jirong Wen. 2020\.  Setrank: Learning a permutation-invariant ranking model for information retrieval. In _SIGIR_.
  * Plackett (1975) Robin L Plackett. 1975\.  The analysis of permutations.  _Journal of the Royal Statistical Society: Series C (Applied Statistics)_ (1975).
  * Qi et al. (2017) Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017\.  Pointnet: Deep learning on point sets for 3d classification and segmentation. In _CVPR_.
  * Qin et al. (2010) Tao Qin, Tie-Yan Liu, Jun Xu, and Hang Li. 2010\.  LETOR: A benchmark collection for research on learning to rank for information retrieval.  _Information Retrieval_ (2010).
  * Qu et al. (2016) Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016.  Product-based neural networks for user response prediction. In _ICDM_.
  * Rendle (2010) Steffen Rendle. 2010\.  Factorization machines. In _ICDM_.
  * Richardson et al. (2007) Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007\.  Predicting clicks: estimating the click-through rate for new ads. In _WWW_.
  * Robertson (1977) Stephen E Robertson. 1977\.  The probability ranking principle in IR.  _Journal of documentation_ (1977).
  * Sun et al. (2019) Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, and Yiming Yang. 2019\.  A re-evaluation of knowledge graph completion methods.  _ACL_ (2019).
  * Taylor et al. (2008) Michael Taylor, John Guiver, Stephen Robertson, and Tom Minka. 2008.  Softrank: optimizing non-smooth rank metrics. In _WSDM_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In _NIPS_.
  * Vinyals et al. (2015) Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015\.  Pointer networks.  _arXiv preprint arXiv:1506.03134_ (2015).
  * Xia et al. (2008) Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008.  Listwise approach to learning to rank: theory and algorithm. In _ICML_.
  * Yilmaz et al. (2014) Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, and Peter Bailey. 2014.  Relevance and effort: An analysis of document utility. In _CIKM_.
  * Zaheer et al. (2017) Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. 2017.  Deep sets.  _arXiv preprint arXiv:1703.06114_ (2017).
  * Zhou et al. (2019) Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019.  Deep interest evolution network for click-through rate prediction. In _AAAI_.
  * Zhou et al. (2018) Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018\.  Deep interest network for click-through rate prediction. In _KDD_.


# GarNet: A Continuous Robot Vision Approach for Predicting Shapes and
Visually Perceived Weights of Garments

Li Duan1 and Gerardo Aragon-Camarasa1 1 Li Duan and Gerardo Aragon-Camarasa
are with the School of Computing Science, University of Glasgow, G12 8QQ,
Scotland, United Kingdom<EMAIL_ADDRESS>and
<EMAIL_ADDRESS>
###### Abstract

We present a Garment Similarity Network (GarNet) that learns geometric and
physical similarities between known garments by continuously observing a
garment while a robot picks it up from a table. The aim is to capture and
encode geometric and physical characteristics of a garment into a manifold
where a decision can be carried out, such as predicting the garment’s shape
class and its visually perceived weight. Our approach features an early stop
strategy, which means that GarNet does not need to observe a garment being
picked up from a crumpled to a hanging state to make a prediction. In our
experiments, we find that GarNet achieves prediction accuracies of $92\%$ for
shape classification and $95.5\%$ for predicting weights and advances state-
of-art approaches by $~{}21\%$ for shape classification.

##  I Introduction

Deformable objects remains an open problem for robotic perception and
manipulation. This is because garments have an infinite number of possible
configurations, such as crumpled and irregular shapes, that cannot be modelled
easily via simulations [1, 2]. Specifically, the main challenges in deformable
objects perception and manipulation are twofold. First, they usually have a
complex initial configuration, which means they are wrinkled, crumpled or
folded, and not in a known configuration state that can be used for
manipulation tasks. Second, garments usually deform in unpredictable ways,
making predictions of their deformations difficult during dexterous robotic
manipulations.

Robots manipulating deformable objects without prior knowledge about their
geometric and physical properties (e.g. shapes, weights or stiffness
parameters) can result in robots requiring to plan actions using a complex and
high-dimensional space. This, therefore, causes failures in motion planning
since robots are prone to fail due to minor variations in the deformable
object’s configurations. We, therefore, propose in this paper the first step
towards an online continuous perception approach that equips a robot with the
ability to predict garment shapes and, consequently, allows a robot to stop a
grasping or manipulation task if the prediction belief is above a threshold.

Figure 1: An example of our experimental setup for capturing a database of
garment’s deformations. That is, garments are grasped and dropped by a dual-
armed robot. An Xtion camera is placed in the front of the robot to record a
video sequence, consisting of RGB images and depth images.

Compared to traditional research focusing on wrinkles [3] and other local
features [4, 5], we propose to learn the dynamic properties of garments from
video sequences and allow a robotic system to recognise the shape and weight
of a garment continuously. For this, we devise a Garment similarity Network
(GarNet) that learns the physical similarity between garments to predict
shapes (geometric) and visually perceived weights (physical) of unseen
garments. We define a visually perceived weight in three discretised levels
using an electric scale to physically weigh garments; namely, light, medium
and heavy weights. We hypothesise that GarNet can predict shapes and
discretised weights in approximately 0.1 seconds per image frame by learning
geometric and physical properties and predicting the garment’s shape and
weights continuously during a robotic garment pick and place task.

To test the above hypothesis, we have built a database that consists of RGBD
video sequences of a robot grasping and dropping garments on a table111This
database can be downloaded from https://liduanatglasgow.github.io/GarNet/, see
Fig. 1. This database simulates a sorting scenario (e.g. [4, 3]) where a robot
can sort based on shape or weights. We then train GarNet to learn garments’
geometric and physical similarities based on their shape and discretised
weight labels. GarNet’s objective is thus to cluster garments of the same
categories (shapes or discretised weights) together and pull garments of
different categories apart using a triplet loss function, and these clusters
are mapped into a Garment Similarity Map (GSM). To predict unseen garment
shapes and weights, we introduce the concept of decision points which depend
on previously mapped points in the similarity map. We use those decision
points to devise an early-stop strategy by fitting confidence intervals for
each cluster and allow us to determine whether decision points are within a
statistical significant interval around a given cluster. An overview of our
approach is shown in Figure 2.

The contributions of this paper are threefold;

  1. 1.

we have advanced the state-of-art by introducing GarNet that improves the
prediction accuracy from 70.8% to 92% for shape classification;

  2. 2.

GarNet can visually estimate weights of unseen garments with a 95.5%
prediction accuracy;

  3. 3.

we propose an early stop strategy so that GarNet is faster during inference
compared to the state-of-art, where a frame can be processed in 0.1 seconds.

Figure 2: Top: Training GarNet. A positive, negative and anchor image samples
from a video sequence of the training dataset are input into GarNet. Training,
therefore, consists of identifying whether any two of the input triplet come
from the same shape or discretised weight categories. GarNet maps input image
frames into a Garment Similarity Map (GSM) in which input frames are mapped
into clusters if they are similar; otherwise, new points are pulled apart from
the cluster. Confidence intervals are computed for each cluster in the GSM as
described in Section III-A. Bottom: Continuous Perception (Testing GarNet). An
image from a video sequence of the testing dataset is input into a trained
GarNet to get the mapping onto the garment similarity map. A video sequence of
a garment in our database contains 60 frames. The plots show that GarNet gains
confidence in predicting that the tested garment is a shirt. That is, as
knowledge is being accumulated into the GSM, most of the decision points
belong to the shirt category. In the example shown, the prediction is stopped
at frame 30 because 80% of all decision points belong to the shirt category.

##  II Related Work

Previous approaches have proposed learning geometric and physical properties
of deformable objects before manipulation. Geometric characteristics of
garments include shapes[4][5][3], their accessories (e.g. zips, buttons),
geometric properties (lengths and heights), and physical characteristics such
as weight [6], stiffness (e.g. bending, stretching and shearing) [1], damping
factors and elasticity [7]. For instance, Mariolis et al. [8] proposed to use
a CNN network to classify garment shapes, which are rotated by a dual-arm
robot. The CNN network learns garment dynamics via depth images and achieves
an accuracy rate of 89% after training. However, they train and test the
network on synthetic datasets of simulated garments and their validation image
set contains the garments that are already in the training dataset, failing to
generalise for unseen garments of similar classes.

Sun et al.[4] presented an approach where local (local B-spline path and
locality-constrained linear coding) and global (shape index, local binary
patterns, and topology spatial distances) features are extracted from single-
shot images and are used to predict unseen garment shapes. This approach makes
use of local and global visual characteristics of garments, such as wrinkle
features. Compared to [8], their approach does not require interactions with
garments, so it is faster to predict shapes and is robust while being
presented with unseen samples. However, prediction accuracies are constrained
by the inability of the robot to interact with the garments, and no new
knowledge can be captured. For this, Sun et al. [3] proposed a Gaussian
process regression classifier to predict unseen garment shapes. That is, the
robot in their experiment shakes or flips and then drops garments on a table
to obtain a new state to improve the classification score. If the
classification score of a garment is above a threshold, the garment is sorted
based on their shape. This approach, therefore, demonstrated that interacting
with garments enables an autonomous system to improve its prediction
confidence over interactions and leads to higher classification accuracies.

However, [3] captured the garment’s state while being static and on a table
which results in making the system slow at predicting shapes as it requires
multiple interactions. Martinez et al.[5] removed this limitation by
introducing the concept of continuous perception to enable a robot to predict
shapes by continuously observing video frames from an Xtion depth-sensing
camera rather than single image frames. They showed higher accuracy in
predicting unseen garment shapes compared to [3] and [4]. However, the
limitation in [5] is that they let the robot to observe the entire video
sequence before a decision can be made, which means that the robot takes a
significant amount of time to predict a garment shape category, and this is
given by the length of the video. In their work, they sample a garment for
approximately 6 seconds which consists of sampling the garment from a crumpled
to a hanging states. In this paper, we, therefore, explore the possibility of
adopting the continuous perception paradigm to allow a robot to change its
manipulation strategy on the fly, i.e as soon as the class of the garment is
predicted even if the garment is still being manipulated from a crumpled to a
hanging state.

Simulated environments [6][2] that model deformable objects, have been used in
the literature to learn the physical properties of these objects and
extrapolate the learned knowledge into the real world. For example, Ruina et
al.[6] devised an approach where they predicted the area weights of fabrics by
learning the physical similarities between simulated fabrics and real fabrics
using a spectral decomposition network (SDN). Closing the gap between a
simulation and the real world is effective because the physical property
parameters of simulated objects are easily accessible compared to those of
real objects. However, their approach is not applicable for online evaluation
because it requires aligning the simulation with reality, creating an extra
overhead before a prediction can be carried out. Hoque et al. [2] propose
learning dynamic physical properties of towels via a VisionSpatial Foresight
network (VSF), which is trained on simulated towels but tested on real towels.
The VSF predicts a sequence of towel deformations and corresponding robot
actions based on the towels’ initial and desired configurations. They used RGB
plus depth images instead of RGB images in their experiment, and they obtained
a success rate of $90\%$ on manipulating (flattening) the towels. Even though
the robot used to deploy VSF takes unnecessary actions while folding a towel,
their proposed approach demonstrates that prior knowledge on understanding
geometric and physical properties of deformable objects enables an effective
manipulation of those objects. Therefore, This paper investigates whether a
continuous perception approach coupled with an early-stop strategy can extend
beyond simple fabrics and one single shape class. That is, we propose learning
the similarity between garments to predict unseen garment shapes and
discretised weights based on a ‘garment similarity map’. Compared to previous
works (e.g. [5], and [3]), our work features an early-stop strategy, where a
prediction can be halted earlier without observing the entire interaction that
can enable an online manipulation strategy.

##  III GarNet: Garment Similarity Network

Our proposed Garment similarity Network (GarNet) consists of a Siamese network
[9] which clusters garments into groups according to their shape and
discretised weight categories. The objective of clustering garments is to
learn common geometric and physical features of garments of the same
categories. Our GarNet network comprises a residual convolutional block that
extracts features from input data and a fully connected layer that maps
features onto a 2D Garment Similarity Map (GSM). A garment similarity map is a
2D manifold that encodes a garment’s geometric and physical characteristics
according to its shape and discretised weight categories. Garments of the same
categories are clustered together, while garments of different categories are
pulled apart. Figure 4 shows the garment similarity map in our experiments
where each cluster in the map is called a Garment Cluster ($\mathcal{GC}$).
Our GarNet training process is expressed mathematically as:

$\mathcal{P}=\mathcal{F}[\mathcal{C}(I)]$ (1)

where $\mathcal{C}$ denotes residual convolutional layers, $\mathcal{F}$,
fully connected layers, and $I$, input video frames. We define $\mathcal{P}$
as a garment similarity point ($\mathcal{GSP}$). Each frame in the input video
sequence of the garments is converted into one garment similarity point
($\mathcal{GSP}$). We also define a Garment Similarity Distance
($\mathcal{GSD}$) as:

$\mathcal{GSD}(x,y)=\mathcal{P}_{i}-\mathcal{P}_{j}$ (2)

where $i$ and $j$ are the $ith$ and $jth$ garment similarity point.
$\mathcal{GSD}$ increases between garments with different labels and decreases
between garments with the same labels. Therefore, to train GarNet, we use a
triplet loss [10]:

$\displaystyle{PP}$
$\displaystyle=\mathinner{\\!\left\lvert\mathcal{P}_{positive}-\mathcal{P}_{anchor}\right\rvert}$
(3) $\displaystyle{NP}$
$\displaystyle=\mathinner{\\!\left\lvert\mathcal{P}_{negative}-\mathcal{P}_{anchor}\right\rvert}$
$\displaystyle{TripletLoss}$ $\displaystyle=max(0,PP-NP+margin)$

where $PP$ is a positive pair between positive and anchor samples and $NP$ is
a negative pair between negative and anchor samples. A positive sample is an
image of a garment of the same category with respect to the anchor. A negative
sample is an image of a garment with different category with respect to the
anchor. The $margin$ is a value that promotes the network to learn to map
positive and negative samples further away from each other. In this
experiment, we set this $margin$ to 1 as suggested in [6] experiments.

Two GarNets are trained separately and independently predict the shapes and
discretised weights independently. That is, the GarNet for shape predictions
is trained on the shape categories, while GarNet for discretised weight
predictions is trained on discretised weight categories, e.g. light, medium
and heavy weights.

###  III-A Garment Cluster Confidence Intervals

To decide which category (either shapes or discretised weights) a mapped
garment similarity point in the similarity map belongs to, we propose to fit
statistical confidence intervals to each garment cluster in this map. That is,
we define a confidence interval using a non-parametric probability density
function for each garment cluster, $\mathcal{GC}$ via a kernel density
estimator [11] that is defined as:

$\displaystyle\hat{f_{h}}(\mathcal{GC})$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}K_{h}(\mathcal{P}-\mathcal{P}_{i})$
(4)
$\displaystyle=\frac{1}{nk}\sum_{i=1}^{n}K(\frac{\mathcal{P}-\mathcal{P}_{i}}{h})$

where $\mathcal{GC}$ is the garment cluster, $K$ is a Gaussian kernel, $h>0$
is a smoothing parameter called bandwidth which regulates the amplitude of
confidence intervals, and $\hat{f_{h}}$ is an estimated probability density
function for a garment cluster. We have conducted an ablation study on the
confidence interval’s bandwidths ($h$) and results are presented in section
V-C. After training a GarNet, the centroid of each garment cluster is defined
as:

$\mathcal{GC}_{mean}=(\frac{1}{m}\sum_{i=1}^{m}x_{\mathcal{P}_{i}},\frac{1}{m}\sum_{i=1}^{m}y_{\mathcal{P}_{i}})$
(5)

where $\mathcal{GC}_{mean}$ is the mean value of garment similarity points
mapped from one garment cluster (in Figure 4) and $m$ is the number of garment
similarity points in the cluster.

In our experiments, we directly input unseen garments image frames acquired by
the robot to GarNet to map them into the garment similarity map. To decide the
shapes and discretised weights, we define a Decision Point ($\mathcal{DP}$)
that is the mean value of garment similarity points ($\mathcal{GSP}$s):

$\mathcal{DP}=(\frac{1}{n}\sum_{i=1}^{n}x_{\mathcal{P}_{i}},\frac{1}{n}\sum_{i=1}^{n}y_{\mathcal{P}_{i}})$
(6)

where $n$ is the total number of frames observed. To predict the shapes and
discretised weights, we find whether a $\mathcal{DP}$ is within any confidence
interval and has the minimum distance to the confidence interval’s
$\mathcal{GC}_{mean}$. For this paper, we use the Euclidean distance to
evaluate how close a $\mathcal{DP}$ is with respect to $\mathcal{GC}_{mean}$.

Each video sequence has 60 frames; therefore, we will have 60 decision points.
To predict the shape and discretised weight, we establish that a predicted
category should have at least $80\%$ of decision points belonging to a garment
cluster. If none of the categories fulfils this requirement, we denote that
the observed garment does not have a known class. That is, if a decision point
is outside any confidence interval, the network is not confident about which
category the input garment belongs to. By clustering garments and defining
confidence intervals, it is possible to define an early-stop strategy to allow
a robotic system to stop its execution if it is confident about the garment
shape or weight. After observing a number of image frames of a garment, if any
of the trained categories takes $80\%$ of the decision points, the process is
terminated, and the category is chosen as the predicted category.

##  IV Experiments

###  IV-A GarNet Architecture

We implement our code in Pytorch [12]. Our GarNet comprises a ResNet18 [13] as
a feature extraction and fully connected networks (FC). The FC networks
comprise three linear layers, where a PReLU activation layer is placed between
adjacent linear layers. The source code for GarNet and experimental scripts
are available at https://liduanatglasgow.github.io/GarNet/.

We use an Intel-i7 equipped computer with an Nvidia GTX 1080 Ti to train the
network. We use the Adam optimizer with an initial learning rate of
${10}^{-3}$, controlled by a learning scheduler with a decay rate of
${10}^{-1}$ and a step size of 8. The network is trained for 270K iterations,
taking approximately 30 minutes.

###  IV-B Data Collection and Experiments

The video database in this experiment consists of 20 garments of five
different shapes, namely, pants, shirts, sweaters, towels and t-shirts. Figure
3 shows garment samples for each garment instance in our database, i.e. five
categories and four garments for each category. For each shape, there are four
garments of different colours and materials. We used an electric scale to
weigh every garment and divide their weights into three discretised levels,
namely, light, medium and heavy weights. Therefore, in these experiments, we
do not predict the real weight values of tested garments but predict the
discretised weights levels in order to enable a robot sort garments as we do
before putting garments into a washing machine.

To validate our network and test our hypothesis (Section I), we propose to
carry out a leave-one-out cross-validation methodology. That is, we group all
garments into four groups and each garment category as shown in Fig. 3, has
four different garment instances. Hence, four experiments are conducted, where
three groups served as training groups and one group served as a testing
group. We averaged accuracies for each category output from the four
experiments and used testing group to validate the classification performance
of our approach. For each of the four experiments, the training group
represent 80% of our video sequence database, while the testing group
represent 20% of our video sequence database.

Figure 3: The garment database used in our experiments: Five categories (pants
(jeans), shirts, sweaters, towels and t-shirts), and each category has four
garment instances. For each garment instance, we show an RGB image frame and
its corresponding segmented depth image

We have used a Baxter robot to manipulate garments. Garments are initially
placed on a table, and, next, they are lifted to $1m$ above the table and
dropped from this height. An Xtion depth-sensing camera is used to capture
garment video sequences. Each garment is captured ten times, which means that
the grasp-and-drop operation is conducted ten times. There are 200 videos in
total, and each video contains 60 frames (sampling frequency is 10Hz; video
sequence length is 6 seconds). Therefore there are 12,000 image frames in
total. Figure 1 shows the experimental setup of the robot grasping and
dropping garments.

Our experiments include 50 unseen garment videos containing ten videos for
each of the five categories for each of the four leave-one-out cross-
validation experiments. For each video sequence, we predict the shape and
discretised weight of the garment in the video. Therefore, we have ten
predictions for each category (one prediction for each video) and 50
predictions in total. The prediction accuracy for each category is defined as
the percentage of correctly predicted videos of each category.

##  V Results

Figure 4: The Garment Similarity Map (GSM) after training GarNet.

###  V-A GarNet Training

While training GarNet, our approach achieves a validation classification
performance of $93.9\%$ for shapes and $94.9\%$ for discretised weights.
Figure 4 shows the mappings of testing garments onto the similarity map, where
it is possible to observe that garments of different categories are pulled
apart; while garments of the same categories are clustered together. These
results confirm that GarNet coupled with a triplet loss function (Eq. 3) is
able to extract physical dissimilarities between categories while maintaining
inter-class physical properties within well-defined clusters.

###  V-B Continuous Perception Experiments

Figure 5: Examples of the early-stop strategy proposed in this paper. The
early-stop strategy allows the robotic system to make a prediction without
observing the full motion. As observed in both plots, GarNet becomes confident
over time, and the early-stop strategy activates if 80% of decision points in
the garment similarity map are within a correct category. Plots for all unseen
predictions can be accessed at https://liduanatglasgow.github.io/GarNet/

Examples of our experimental results are shown in Figure 5. We can see that
although GarNet does not recognise garments correctly from the beginning
(because most of the decision points belong to an incorrect category), GarNet
gradually gains confidence in predicting a correct category for each garment
because more decision points are within the correct category and eventually
percentages of decision points are over 80%. The classification task is
consequently stopped early, and the system does not need to observe the full
robot motion (and video sequence) to make a correct prediction of the garment
class.

We conduct two ablation studies for the continuous perception experiment. The
first study is about comparing predictions only on local garment similarity
points ($\mathcal{GSP}$s) rather than on decision points ($\mathcal{DP}$s).
The second ablation study compares the performance of GarNet trained on RGB
and depth images.

Tables I and II show the results of the leave-one-out cross-validation
experiments, where the network achieved $92\%$ for shape classification and
$96\%$ for discretised weight classification. The results show that our
network has an expected ability to classify shapes and discretised weights of
unseen garments.

We use ‘decision points’, $\mathcal{DP}$s, (Eq. 6) to make predictions on
unseen garment shapes and discretised weights. That is, the position of a
decision point on the garment similarity map (in Figure 4) depends on all
previously observed image frames rather than on currently observed image
frames. From Tables I and II, we can observe that using decision points has
better performance than using garment similarity points ($92\%$ vs $78\%$ for
shapes and $95.5\%$ vs. $80\%$ for discretised weights, respectively). This
shows that GarNet benefits from using accumulated knowledge via decision
points rather than local and episodic knowledge as in [8, 4].

TABLE I: Table: Prediction Results (shapes) Category | depth, $\mathcal{DP}$ | depth, $\mathcal{GSP}$ | RGB, $\mathcal{DP}$ | RGB, $\mathcal{GSP}$
---|---|---|---|---
pants | 97.5% | 82.5% | 97.5% | 87.5%
shirts | 72.5% | 75% | 87.5% | 62.5%
sweaters | 97.5% | 85% | 25% | 20%
towels | 92.5% | 65% | 50% | 17.5%
t-shirts | 100% | 82.5% | 32.5% | 22.5%
Average | 92% | 78% | 53.5% | 42%
TABLE II: Table: Prediction Results (discretised weights) Category | depth, $\mathcal{DP}$ | depth, $\mathcal{GSP}$ | RGB, $\mathcal{DP}$ | RGB, $\mathcal{GSP}$
---|---|---|---|---
lights | 97.5% | 50% | 37.5% | 5%
mediums | 93.75% | 87.5% | 72.5% | 58.75%
heavies | 96.25% | 87.5% | 71.25% | 71.25%
Average | 95.5% | 80% | 65% | 53%

To investigate whether the type of image affects the overall prediction of a
garment class, we trained GarNet using RGB and depth images. Tables I and II
show that a GarNet trained on depth images outperforms a GarNet trained on RGB
images (92% vs 53.5% for shapes and 95.5% vs 65% for discretised weights,
respectively). The increase in performance is because depth images capture
structural and dynamic information of the garment being manipulated and are
better suited to capture the physical properties of garments as opposed to RGB
images as proposed by [6].

Note that the inter-class variability for the pants category is consistent
(i.e. we use jeans for this category, see Fig. 3, top row). Therefore,
classification scores in Table I for pants are high across the ablation
studies with respect to other shape categories of which they have high inter-
class variability. This result shows that in order to generalise to unseen
garments, depth images and decision points offer the best combination for the
continuous perception task.

###  V-C Ablation study on the confidence intervals bandwidths

A bandwidth, as defined in section III-A, determines the size of a confidence
interval. In this study, we, therefore, evaluate the effect of the bandwidth
selection with respect to the performance of GarNet. A confidence interval of
a garment cluster is a region in the garment similarity map of which a certain
percentage of $\mathcal{GSP}$s are grouped together.

A decrease in the bandwidth value denotes a decrease in the percentage of
$\mathcal{GSP}$s included within the garment cluster. An increase in the
bandwidth means that almost all $\mathcal{GSP}$s should be included, while a
small portion of points are relatively far away from a garment cluster. This
means that a confidence interval may overlap with other confidence intervals,
or even multiple confidence intervals will be generated for one garment
cluster. In this case, the performance of GarNet will have an impact in the
final classification prediction.

Figure 6: Bandwidth Ablation Study. Left: Bandwidth values versus accuracy for
shape classification. Right: Bandwidth values versus Accuracy for discretised
weights classification.

In Figure 6, we find that a bandwidth of $95\%$ has the best performance, and
we use $95\%$ as the bandwidth for the rest of the experiments. However, at a
$99\%$ bandwidth, we can observe that the prediction accuracy drops while
classifying shapes; this is because $\mathcal{GSP}$s are grouped into
incorrect categories.

###  V-D Comparison with the state-of-art

We have compared our results with the results from [4][5][3][14]. As observed
in Table III, our GarNet outperforms previous work on predicting unseen
garment shapes. There are several reasons why our network has the best
performance and we discuss these below.

####  V-D1 The use of a garment similarity map to encode knowledge of garment
shapes and weights

Inspired by [6], where the authors proposed learning the physical similarity
between simulated fabrics and predicting physical properties of real fabrics,
we find that the similarity network effectively predicts unseen deformable
objects, such as garments and fabrics. In our previous work [14], where we
focused on utilising solely classification approach rather than the clustering
approach in our network, we found that our clustering approach presented in
this paper improves over the classification approach. Compared with a
traditional classifier that regresses embeddings of data into labels (which is
equivalent to asking which shape/discretised weight classes the data belongs
to), our GarNet network learns geometric and physical characteristics that
makes them the same or different (which is equivalent to asking why the data
presents the same or different shapes/discretised weights). Therefore, for
unseen garments, the network only needs to decide similarities of the garments
for each garment cluster rather than classify them into certain classes,
reducing the prediction difficulty.

####  V-D2 Continuous Perception.

Traditional methods such as [4, 8] that predict shapes and weights of garments
are based on static garment features such as wrinkles, outlines, creases, to
name a few. Instead, we propose to carry out predictions on encoded knowledge
in the GSM while learning the dynamic properties of garments.

####  V-D3 Early-Stop strategy.

Compared with [3][5], where the proposed approaches consist of having a robot
observing the entire interaction with a garment, our approach only requires a
robot to observe interactions partly if termination requirements are
satisfied. Therefore, our approach has a mechanism to stop a manipulation on
the fly as GarNet can process images every 0.1 seconds, taking an average of 4
seconds to generate a prediction.

TABLE III: Comparisons with the state-of-art. Method | Accuracy (%)
---|---
CNN-LSTM (classification) [14] | $48\%$
Interactive Perception[3] | $64.2\%$
Single-shot category recognition[4] | $67.0\%$
Continuous Perception in [5] | $70.8\%$
GarNet (Continuous Perception, Ours) | $92.0\%$

##  VI Conclusion

We have presented a garment similarity network (GarNet) that learns the
similarity of the garments and predicts continuously their shapes and their
visually perceived weights. We have also introduced a Garment Similarity Map
($\mathcal{GSM}$) that encodes garment shapes and weights knowledge into
clusters. These clusters were then used to decide on which cluster unseen
garment samples belongs to heuristically. Our experimental validation shows
that, GarNet obtains high prediction accuracies while classifying shapes (92%)
and discretised weights (95.5%), Fig. 5. Similarly, we have compared GarNet’s
performance with the state of the art, and GarNet showed an increase of 21.2%
of classification accuracy performance (Table III).

Compared with previous work on continuous perception [5], GarNet has the
advantage of an ‘early stop’ strategy. That is, a robot does not need to
observe the full motion (video sequences) to make predictions which could
enable robots to be more responsive and effective while manipulating garments
and deformable objects in a laundry pipeline. However, GarNet, in this paper,
does not support online learning of unknown garment shapes. For instance, we
train GarNet on five shape categories, and it can predict shapes of unseen
garments from those categories.

In future work, we plan to devise an online-learning approach for GarNet to
investigate complex manipulations interactions (enabled by a behaviour-based
reinforcement learning agent [15]), such as twisting garments, shaking
garments or rotating garments. From those interactions, differences in
stretching and bending characteristics of garments can be exploited to
evaluate garments’ stiffness parameters, which can potentially help to develop
a robot dexterous garment manipulation approach for folding [16] [17],
flattening [18], to name a few, that requires fewer iterations. Furthermore,
knowledge of the garments weights, i.e. whether it is light, medium or heavy,
can enable a robot to plan for these complex manipulation interactions since
it will be possible to reduce the search space while estimating the dynamic
physical properties of garments.

##  VII Acknowledgement

We would like to thank George Killick and Nikos Pitsillos for their valuable
comments while reviewing this paper.

## References

  * [1] H. Wang, J. F. O’Brien, and R. Ramamoorthi, “Data-driven elastic models for cloth: modeling and measurement,” _ACM transactions on graphics (TOG)_ , vol. 30, no. 4, pp. 1–12, 2011.
  * [2] R. Hoque, D. Seita, A. Balakrishna, A. Ganapathi, A. K. Tanwani, N. Jamali, K. Yamane, S. Iba, and K. Goldberg, “Visuospatial foresight for multi-step, multi-task fabric manipulation,” 2021.
  * [3] L. Sun, S. Rogers, G. Aragon-Camarasa, and J. P. Siebert, “Recognising the clothing categories from free-configuration using gaussian-process-based interactive perception,” in _2016 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2016, pp. 2464–2470.
  * [4] L. Sun, G. Aragon-Camarasa, S. Rogers, R. Stolkin, and J. P. Siebert, “Single-shot clothing category recognition in free-configurations with application to autonomous clothes sorting,” 2017.
  * [5] L. Martínez, J. R. del Solar, L. Sun, J. P. Siebert, and G. Aragon-Camarasa, “Continuous perception for deformable objects understanding,” _Robotics and Autonomous Systems_ , vol. 118, pp. 220 – 230, 2019.
  * [6] T. F. Runia, K. Gavrilyuk, C. G. Snoek, and A. W. Smeulders, “Cloth in the wind: A case study of physical measurement through simulation,” _arXiv preprint arXiv:2003.05065_ , 2020.
  * [7] B. Tawbe and A. Crétu, “Acquisition and neural network prediction of 3d deformable object shape using a kinect and a force-torque sensor †,” _Sensors (Basel, Switzerland)_ , vol. 17, 2017.
  * [8] I. Mariolis, G. Peleka, A. Kargakos, and S. Malassiotis, “Pose and category recognition of highly deformable objects using deep learning,” in _2015 International Conference on Advanced Robotics (ICAR)_. IEEE, 2015, pp. 655–662.
  * [9] G. Koch, R. Zemel, and R. Salakhutdinov, “Siamese neural networks for one-shot image recognition,” in _ICML deep learning workshop_ , vol. 2. Lille, 2015.
  * [10] E. Hoffer and N. Ailon, “Deep metric learning using triplet network,” in _International workshop on similarity-based pattern recognition_. Springer, 2015, pp. 84–92.
  * [11] M. Rosenblatt, “Remarks on Some Nonparametric Estimates of a Density Function,” _The Annals of Mathematical Statistics_ , vol. 27, no. 3, pp. 832 – 837, 1956.
  * [12] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems 32_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds. Curran Associates, Inc., 2019, pp. 8024–8035.
  * [13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015.
  * [14] L. Duan. and G. Aragon-Camarasa., “Continuous perception for classifying shapes and weights of garments for robotic vision applications,” in _Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP,_ , 2022, pp. 348–355.
  * [15] A. Pore and G. Aragon-Camarasa, “On simple reactive neural networks for behaviour-based reinforcement learning,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 7477–7483.
  * [16] A. Doumanoglou, J. Stria, G. Peleka, I. Mariolis, V. Petrik, A. Kargakos, L. Wagner, V. Hlaváč, T.-K. Kim, and S. Malassiotis, “Folding clothes autonomously: A complete pipeline,” _IEEE Transactions on Robotics_ , vol. 32, no. 6, pp. 1461–1478, 2016.
  * [17] J. Stria, D. Průša, V. Hlaváč, L. Wagner, V. Petrík, P. Krsek, and V. Smutný, “Garment perception and its folding using a dual-arm robot,” in _2014 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2014, pp. 61–67.
  * [18] L. Sun, G. Aragon-Camarasa, S. Rogers, and J. P. Siebert, “Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening,” in _2015 IEEE International Conference on Robotics and Automation (ICRA)_ , 2015, pp. 185–192.