# Hierarchical Relationship Alignment Metric Learning

Lifeng Gu

###### Abstract

Most existing metric learning methods focus on learning a similarity or
distance measure relying on similar and dissimilar relations between sample
pairs. However, pairs of samples cannot be simply identified as similar or
dissimilar in many real-world applications, e.g., multi-label learning, label
distribution learning. To this end, relation alignment metric learning (RAML)
framework is proposed to handle the metric learning problem in those
scenarios. But RAML learn a linear metric, which can’t model complex datasets.
Combining with deep learning and RAML framework, we propose a hierarchical
relationship alignment metric leaning model HRAML, which uses the concept of
relationship alignment to model metric learning problems under multiple
learning tasks, and makes full use of the consistency between the sample pair
relationship in the feature space and the sample pair relationship in the
label space. Further we organize several experiment divided by learning tasks,
and verified the better performance of HRAML against many popular methods and
RAML framework.

Machine Learning, ICML

##  1 Introduction

In many computer vision and pattern recognition tasks, e.g., face recognition
(Guillaumin et al., 2009), image classification (Mensink et al., 2012), and
person re-identification (Liao et al., 2015), it is crucial to learn a
discriminative distance metric to measure the similarity between pairs of
samples. However, for some learning tasks, e.g., multi-label learning (Zhang &
Wu, 2015) and label distribution learning (Geng, 2016), relations between
sample pairs cannot be simply identified as similar or dissimilar. Thus, the
existing metric learning methods cannot work on the above tasks. The problem
arises that it is difficult to classify two images into similar or dissimilar
sample pair. Above discussions encourage us to propose a generalized metric
learning method, which can be flexibly adopted to various kinds of tasks. RAML
(Zhu & Yang, 2018) framework was proposed to handle the problem. Combing with
deep neural networks and RAML, we propose a hierarchical relationship
alignment metric leaning model HRAML, which uses the concept of relationship
alignment to model metric learning problems under multiple learning tasks, and
makes full use of the consistency between the sample pair relationship in the
feature space and the sample pair relationship in the label space.
00footnotetext: Affiliations 1Tju 00footnotetext: Correspondence Lifeng Gu -
<EMAIL_ADDRESS>
##  2 Related works

deep metric learning  deep metric learning want to learn a good metric to
measure the similarity of samples. Contrative loss (Hadsell et al., 2006) was
used to combine with Siamese network and achieved good results. The loss of
two-tuples can be used to model several problems. It is also a simple and
feasible method. The sampling complexity is $o(n^{2})$, easier than many
subsequent variants. Wen et al. (2016) combines softmax and center loss for
face recognition. Center loss is used to minimize the distance from the sample
representation to the center of the category. Ge (2018) combined the triple
loss and hierarchical tree to propose a new loss function. He used the
hierarchical tree to encode the context of the sample, and then used the
hierarchical tree to calculate the intra-class distance and the inter-class
distance. Sample the more suitable triples, and finally use the dynamic
interval mechanism to form the loss function. Wang et al. (2017) proposed a
novel angle loss to improve the discriminative ability of sample
representation. Different from the two-tuple and triple-tuple loss, the angle
loss uses the angle of the sample to the center of the category to impose
constraints. The angle constraint has rotation and scaling invariance. It
naturally encodes the geometric relationship between the three samples. You
can set the angle The size to change the constraint strength has the advantage
of being very robust. The slow convergence speed of binary or ternary loss is
mainly due to the fact that only one negative sample is compared in each
update and other negative samples are ignored. Npair loss (Sohn, 2016)
improves this point, and compares batches in each update All other negative
samples in each update. Circle loss (Sun et al., 2020) was proposed, Circle
loss re-weights the similarity between samples to highlight the incomplete
sample similarity. Circle loss unifies two basic learning methods in the field
of feature learning: learning based on category labels and learning based on
pair label. Cakir et al. (2019) proposed FastAP, which learns features by
optimizing the average accuracy of the ranked list. FastAP connects feature
learning and ranking problems together, opening up new ideas.

##  3 Background

Zhu & Yang (2018) proposed the concept of relational alignment in the field of
metric learning. We briefly describes it as follows: For metric learning,
pairwise constraints are often used to describe the sample pair relationship
in the decision space. Here, we introduce relation alignment learning to
metric learning. $f({\bf{x}}_{i},{\bf{x}}_{j},{\bf{M}},b)$ is used to measure
the sample relationship in the feature space, $g(\bf{y_{i}},\bf{y_{j}})$ are
used to measure the sample relationship in the decision space.
$g(\bf{y_{i}},\bf{y_{j}})$ is specifically designed to handle different tasks.
In addition, $\bf{A}\in{\mathbb{R}}^{n\times n}$ and
$\bf{E}\in{\mathbb{R}}^{n\times n}$ are used as The sample relationship matrix
between the feature space and the decision space, the relationship alignment
means The sample relationship of the feature space should be consistent with
the sample relationship of the decision space. This is a wide-ranging idea,

$\displaystyle\small\left[{\begin{array}[]{*{20}{c}}{{a_{11}}}&{...}&{{a_{i1}}}&{...}&{{a_{n1}}}\\\
{...}&{...}&{...}&{...}&{...}\\\
{{a_{1i}}}&{...}&{{a_{ii}}}&{...}&{{a_{ni}}}\\\
{...}&{...}&{...}&{...}&{...}\\\
{{a_{1n}}}&{...}&{{a_{in}}}&{...}&{{a_{nn}}}\end{array}}\right]=\left[{\begin{array}[]{*{20}{c}}{{e_{11}}}&{...}&{{e_{i1}}}&{...}&{{e_{n1}}}\\\
{...}&{...}&{...}&{...}&{...}\\\
{{e_{1i}}}&{...}&{{e_{ii}}}&{...}&{{e_{ni}}}\\\
{...}&{...}&{...}&{...}&{...}\\\
{{e_{1n}}}&{...}&{{e_{in}}}&{...}&{{e_{nn}}}\end{array}}\right]$ (11)

$a_{ij}$ and $e_{ij}$ represent the relationship between the sample
${\bf{x}}_{i}$ and ${\bf{x}}_{j}$ in the feature space and the decision space.
In order to maintain consistency, here is a requirement

$f({\bf{x}}_{i},{\bf{x}}_{j},{\bf{M}},b)=g({\bf{y}}_{i},{\bf{y}}_{j}).$ (12)

$g(\bf{y_{i}},\bf{y_{j}})$ represents the different degrees of the two samples
in the decision space. $g({\bf{y}}_{i},{\bf{y}}_{j})$ reflects the
relationship between sample pairs in the decision space and is used to guide
the learning in the feature space $({\bf{M}},b)$ .

$\begin{array}[]{l}f({{\bf{x}}_{i}},{{\bf{x}}_{j}},{\bf{M}},b)={\left({{{\bf{x}}_{i}}-{{\bf{x}}_{j}}}\right)^{T}}{\bf{M}}\left({{{\bf{x}}_{i}}-{{\bf{x}}_{j}}}\right)+b\\\
{\rm{}}\qquad\qquad\qquad\;=\left\langle{{\bf{M}},{{\bf{T}}_{ij}}}\right\rangle+b\end{array}$
(13)

$\left\langle{\cdot,\cdot}\right\rangle$ is defined as the inner product of
two Frobenius matrices, $b$ is the offset, and
${{\bf{T}}_{ij}}=\left({{{\bf{x}}_{i}}-{{\bf{x}}_{j}}}\right){\left({{{\bf{x}}_{i}}-{{\bf{x}}_{j}}}\right)^{T}}$.
(12) can be rewritten as

$g({{\bf{y}}_{i}},{{\bf{y}}_{j}})=\left\langle{{\bf{M}},{{\bf{T}}_{ij}}}\right\rangle+b$
(14)

Once the relational function $g(\bf{y_{i}},\bf{y_{j}})$ is selected, the
formula (14) can be regarded as a linear regression problem. Here, the metric
learning problem is transformed into solving a sample pair regression problem,
and the input is a sample pair $({\bf{x}}_{i},{\bf{x}}_{j})$.

##  4 HRAML

Given sample pairs ${\bf{x}}_{i}$ and ${\bf{x}}_{j}$, after being modeled by a
neural network of $M+1$ layer, they can finally be formalized as
$f(\bf{x_{i}})={\bf{h}}_{i}^{(M)}=s({\bf{W}}^{{(m)}^{T}}{\bf{x}}_{i}^{(m-1)}+{\bf{b}}^{(m)})$
and $f({\bf
x_{j}})={\bf{h}}_{j}^{(M)}=s({\bf{W}}^{{(m)}^{T}}{\bf{x}}_{j}^{(m-1)}+{\bf{b}}^{(m)})$,
where the mapping $s:{\mathbb{R}}\to{\mathbb{R}}$ is a nonlinear activation
function, and $m$ is the number of layers.
${\bf{x_{i}}}^{(m)}={\bf{W}}^{{(m)}^{T}}{\bf{x}}_{i}^{(m-1)}+{\bf{b}}^{(m)}$.

Given a sample pair $x_{i}$ and $x_{j}$, using the idea of relationship
alignment, the alignment relationship between the sample pair in the feature
space and the decision space can be formalized as:

$R(f(x_{i}),f(x_{j}))=g(y_{i},y_{j})$ (15)

Where $R$ represents the sample pair relation function in the feature space,
and $g$ represents the relation function in the feature space. By specifying
the relationship functions $R$ and $g$, different relationships specifically
used for alignment in the idea of relationship alignment can be expressed. We
defines the sample pair relationship of the feature space as the distance
between them: $R(f(x_{i}),f(x_{j}))=D_{ij}=\left\|f({\bf x_{i}})-f({\bf
x_{j}})\right\|_{2}$, and then use the mean square error loss optimize formula
(15) to derive the objective function of HRAML:

$\mathop{\min}J=\frac{1}{4}\sum\nolimits_{(i,j)}^{n}(D_{ij}^{2}-g(y_{i},y_{j}))^{2}+r(\theta)$
(16)

$r(\theta)=\lambda\sum_{m=1}^{M}\left(\left\|{\bf{W}}^{(m)}\right\|_{F}^{2}+\left\|{\bf{b}}^{(m)}\right\|_{2}^{2}\right)$
is a regularization item, $g(y_{i},y_{j})$ is relation function. In order to
optimize (16), we use stochastical gradient descent and we have flowing
equations:
for layer $m=M$ in neural network:

$\begin{split}\frac{\partial
J(i,j)}{\partial{\bf{W}}^{(m)}}&=\left(D^{2}_{ij}-g(y_{i},y_{j})\right)\Big{(}\frac{\partial
s({\bf{x}}_{i}^{(m)})}{\partial{\bf{x}}_{i}^{(m)}}\left({f({\bf{x}}_{i})}-{f({\bf{x}}_{j})}\right)\\\
&{\bf{h}}_{i}^{{(m-1)}^{T}}-\frac{\partial
s({\bf{x}}_{j}^{(m)})}{\partial{\bf{x}}_{j}^{(m)}}\left({f({\bf{x}}_{i})}-{f({\bf{x}}_{j}})\right){\bf{h}}_{j}^{{(m-1)}^{T}}\Big{)}\\\
&={\bf{\delta}}_{i}^{(m)}{\bf{h}}_{i}^{{(m-1)}^{T}}-{\bf{\delta}}_{j}^{(m)}{\bf{h}}_{j}^{{(m-1)}^{T}}\end{split}$
(17) $\begin{split}\frac{\partial
J(i,j)}{\partial{\bf{b}}^{(m)}}&=\left(D^{2}_{ij}-g(y_{i},y_{j})\right)\Big{(}\frac{\partial{s({\bf{x}}_{i}^{(m)})}}{\partial{\bf{x}}_{i}^{(m)}}-\frac{\partial{s({\bf{x}}_{j}^{(m)}})}{\partial{\bf{x}}_{j}^{(m)}}\Big{)}\\\
&\left(f({\bf{x}}_{i})-{f({\bf{x}}_{j})}\right)={\bf{\delta}}_{i}^{(m)}-{\bf{\delta}}_{j}^{(m)}\end{split}$
(18)

for layer $m=1,2\dots M-1$ in neural network:

$\begin{split}\frac{\partial
J(i,j)}{\partial{\bf{W}}^{(m)}}&=\frac{\partial{s({\bf{x}}_{i}^{(m)}})}{\partial{\bf{x}}_{i}^{(m)}}{{\bf{W}}^{(m+1)}}^{T}\frac{\partial
J(i,j)}{\partial{{\bf{{x}}_{i}}^{(m+1)}}}{{\bf{h}}_{i}^{(m-1)}}^{T}\\\
&-\frac{\partial{s({\bf{x}}_{j}^{(m)}})}{\partial{\bf{x}}_{j}^{(m)}}{{\bf{W}}^{(m+1)}}^{T}\frac{\partial
J(i,j)}{\partial{{\bf{x}}_{j}}^{(m+1)}}{{\bf{h}}_{j}^{(m-1)}}^{T}\\\
&={\bf{\delta}}_{i}^{(m)}{{\bf{{h}}}_{i}^{(m-1)}}^{T}-{\bf{\delta}}_{j}^{(m)}{{{\bf{h}}_{j}^{(m-1)}}}^{T}\end{split}$
(19) $\begin{split}\frac{\partial
J(i,j)}{\partial{\bf{b}}^{(m)}}&=\frac{\partial{s({\bf{x}}_{i}^{(m)}})}{\partial{\bf{x}}_{i}^{(m)}}{\bf{W}}^{{(m+1)}^{T}}\frac{\partial
J(i,j)}{{\partial{\bf{x}}_{i}}^{(m+1)}}-\frac{\partial{s({\bf{x}}_{j}^{(m)}})}{\partial{\bf{x}}_{j}^{(m)}}\\\
&{{\bf{W}}^{{{(m+1)}}^{T}}}\frac{\partial
J(i,j)}{\partial{{({{\bf{x}}_{j}}^{(m+1)})}}}={\bf{\delta}}_{i}^{(m)}-{\bf{\delta}}_{j}^{(m)}\end{split}$
(20)

The corresponding metric learning HRAML algorithm is summarized in Algorithm
1.

Algorithm 1 The algorithms of our proposed HRAML

0: Training data $\bf{X}\in{\mathbb{R}}^{d\times m}$, where $d$ and $m$ are
the numbers of feature dimension and samples, respectively. number of network
layers $M$, learning rate $\mu$, iterative number $T$

1: Generate sample pairs $({\bf{x}}_{i1},{\bf{x}}_{i2})$, $i=1,2,...,n$.

2: Compute sample relation $g({\bf{x}}_{i1},{\bf{x}}_{i2})$, $i=1,2,...,n$.

3: repeat

4: Randomly select a pair$(\bf{x}_{i},\bf{x}_{j})$ from sample pairs.

5: Set ${\bf{h}}_{i}^{0}$= ${\bf{x}}_{i}$, ${\bf{h}}_{j}^{0}$ =
${\bf{x}}_{j}$.

6: for m = 1,2,…,M do

7: Do forward propagation to get ${\bf{h}}^{m}_{i}$ and ${\bf{h}}^{m}_{j}$.

8: end for

9: for m = M,M-1,…,1 do

10: Do back propagation to get $\frac{\partial
J(i,j)}{\partial{\bf{W}}^{(m)}}$ and $\frac{\partial
J(i,j)}{\partial{\bf{b}}^{(m)}}$ by (17),(18),(19),(20)

11: end for

12: update ${\\{{\bf{W}}^{m},{\bf{b}}^{m}\\}},m=1,2,\dots,M$ by gradient
descent.

13: until converge

13: Weights and biases: {Wm, bm}, m=1,2,…,M

##  5 Implementation Details

###  5.1 Training Details

Due to the diversification of the training data that needs to be processed,we
uses general mlp as the encoder and tanh as the activation function. In
addition, we normalizes the output of the network so that a better performance
can be obtained (Wu et al., 2017), the normalized form is

$f(x)=\frac{f(x)}{\|f(x)\|}$ (21)

f(x) is the output of the network.
We initializes ${\bf{b}}^{m},m=1,2,\dots,M$ is 0, and ${\bf{W}}^{m},m=1,2,\
dots,M$ is initialized to the distribution $U[-0.2,0.2]$.

###  5.2 Relation function

In the feature space, we need to use differentiable functions as the relation
function to ensure the stability and ease of learning. We uses ordinary
Euclidean distance as the relation function. In the decision space, we can
imitate the feature space and use the function with parameters, which will
make HRAML method has a broader form and can be linked to more methods, but
for the convenience of the sample selection stage, we are consistent with (Zhu
& Yang, 2018), using a fixed function without parameters as the decision space
Function, we consider the $l_{1}$ norm: Let ${\bf{y}}_{i}$ and ${\bf{y}}_{j}$
represent label of ${\bf{x}}_{i}$ and ${\bf{x}}_{j}$. Relation function will
be :

$g({{\bf{y}}_{i}},{{\bf{y}}_{j}}){\rm{=}}{\left\|{{{\bf{y}}_{i}}-{{\bf{y}}_{j}}}\right\|_{1}}$
(22)

where ${\left\|{\bf{a}}\right\|_{1}}{\rm{}}$ is $l_{1}$-norm of item $\bf{a}$.

###  5.3 Sample selection problem

In the process of neural network learning, sample selection is very important
and has a great impact on the results. We can uses difficult sample mining
(Shrivastava et al., 2016).

##  6 Experiment

In this section, we conduct experiments to validate the performance of the
proposed HRAML. We consider three applications, including single-label
classification, multi-label classification, label distribution learning . The
following part will be organized as the corresponding parts.

###  6.1 Classification

Experiment setup. Following (Zhu & Yang, 2018), we use "S/F/C" represents the
number of samples, features and classes. We compare our method with popular
methods, e.g., ITML (Davis et al., 2007), LMNN (Weinberger & Saul, 2009), DML
(Ying & Li, 2012), DoubletSVM (DSVM) (Wang et al., 2015),GMML (Zadeh et al.,
2016) and RAML(Zhu & Yang, 2018). For fair comparison, the parameters of all
compared methods are set as the default setting of the original references.
Experimental analysis. Table 1 list the classification accuracy of different
metric learning methods on many datasets, respectively, where the best results
are marked in bold face. HRAML get the best performance, it beat others
methods, not only linear methods but also kernel methods based nolinear
methods. HRAML has a clear and sample objective function Under classification
task, neural network can optimise easily and learn discriminate metric.

Data | S/F/C | ITML | LDML | LMNN | DSVM | GMML | DML | RAML-SVR | RAML-KRR | HRAML
---|---|---|---|---|---|---|---|---|---|---
binalpha | 1404/320/36 | 0.6303$\pm$0.0501 | 0.6542$\pm$0.0317 | 0.6112$\pm$0.0358 | 0.5625$\pm$0.0322 | 0.5338$\pm$0.1986 | 0.5063$\pm$0.0251 | 0.7250$\pm$0.0348 | 0.6850$\pm$0.0351 | 0.7243$\pm$0.0262
caltech101 | 8641/256/101 | 0.5803$\pm$0.0162 | 0.5528$\pm$0.0157 | 0.5795$\pm$0.0126 | 0.5584$\pm$0.0159 | 0.5500$\pm$0.0117 | 0.3936$\pm$0.0123 | 0.5855$\pm$0.0095 | 0.5803$\pm$0.0147 | 0.6015$\pm$0.0161
MnistDat | 3495/784/10 | 0.8695$\pm$0.0142 | 0.8858$\pm$0.0124 | 0.8721$\pm$0.0255 | 0.8848$\pm$0.0194 | 0.8589$\pm$0.0171 | 0.8323$\pm$0.0239 | 0.9019$\pm$0.0175 | 0.9087$\pm$0.0137 | 0.8983$\pm$0.0168
Mpeg7 | 1400/6000/70 | 0.8214$\pm$0.0333 | 0.7971$\pm$0.0365 | 0.8253$\pm$0.0232 | 0.8271$\pm$0.0353 | 0.8429$\pm$0.0228 | 0.7071$\pm$0.0267 | 0.8450$\pm$0.0305 | 0.7936$\pm$0.0341 | 0.8043$\pm$0.0248
news20 | 3970/8014/4 | 0.8678$\pm$0.0200 | 0.8816$\pm$0.0145 | 0.8734$\pm$0.0290 | 0.8594$\pm$0.0159 | 0.8647$\pm$0.0143 | 0.8166$\pm$0.0222 | 0.9025$\pm$0.0132 | 0.9217$\pm$0.0145 | 0.9436$\pm$0.0133
TDT2_20 | 1938/3677/20 | 0.9587$\pm$0.0358 | 0.9531$\pm$0.0306 | 0.9352$\pm$0.0197 | 0.9499$\pm$0.0175 | 0.9437$\pm$0.0275 | 0.6333$\pm$0.0176 | 0.9679$\pm$0.0244 | 0.9762$\pm$0.0164 | 0.9860$\pm$0.0154
uspst | 2007/256/10 | 0.8979$\pm$0.0261 | 0.9084$\pm$0.0243 | 0.9096$\pm$0.0217 | 0.9125$\pm$0.0172 | 0.8858$\pm$0.0168 | 0.8030$\pm$0.0330 | 0.9525$\pm$0.0147 | 0.9447$\pm$0.0157 | 0.9537$\pm$0.0202

Table 1: classification accuracy on many datasets | Data | emotion | flags | corel800
---|---|---|---|---
MLKNN |  Hamming Loss$\downarrow$ | 0.2137 | 0.3099 | 0.0137
Ranking Loss$\downarrow$ | 0.1729 | 0.2228 | 0.1888
One Error$\downarrow$ | 0.3317 | 0.2154 | 0.6825
Coverage$\downarrow$ | 1.9158 | 3.8154 | 88.5100
Average Precision$\uparrow$ | 0.7808 | 0.8084 | 0.3276
RAML-SVR |  Hamming Loss$\downarrow$ | 0.2054 | 0.2967 | 0.0135
Ranking Loss$\downarrow$ | 0.1577 | 0.2179 | 0.1882
One Error$\downarrow$ | 0.2376 | 0.2000 | 0.6425
Coverage$\downarrow$ | 1.8960 | 3.8115 | 88.2350
Average Precision$\uparrow$ | 0.8101 | 0.8128 | 0.3386
RAML-KRR |  Hamming Loss$\downarrow$ | 0.2046 | 0.2967 | 0.0134
Ranking Loss$\downarrow$ | 0.1382 | 0.2113 | 0.1888
One Error$\downarrow$ | 0.2574 | 0.2000 | 0.6550
Coverage$\downarrow$ | 1.7327 | 3.7692 | 88.5100
Average Precision$\uparrow$ | 0.8225 | 0.8112 | 0.3388
HRAML |  Hamming Loss$\downarrow$ | 0.2060 | 0.2791 | 0.0135
Ranking Loss$\downarrow$ | 0.1690 | 0.2021 | 0.1861
One Error$\downarrow$ | 0.2871 | 0.2154 | 0.6075
Coverage$\downarrow$ | 1.8861 | 3.6769 | 87.6025
Average Precision$\uparrow$ | 0.8052 | 0.8244 | 0.3582
Table 2:  results on muti-label datasets

.

Criterion |  Chebyshev$\downarrow$ |  Clark$\downarrow$ |  Canberra$\downarrow$ |  Cosine$\uparrow$ |  Intersection$\uparrow$
---|---|---|---|---|---
AAKNN | 0.3261 | 1.8448 | 4.3412 | 0.6905 | 0.5506
RAML-SVR | 0.3102 | 1.6986 | 3.8576 | 0.7051 | 0.5739
RAML-KRR | 0.3139 | 1.6865 | 3.8419 | 0.7057 | 0.5743
HRAML | 0.2907 | 1.7664 | 4.0922 | 0.7321 | 0.5929
Table 3:  results on Nature Scene dataset.

###  6.2 Multi-label Learning

Dataset. Like (Zhu & Yang, 2018), we use three multi-label datasets to
evaluate performance111http://mulan.sourceforge.net/datasets-mlc.html, emotion
(Trohidis et al., 2008), flags, and corel800 dataset (Hoi et al., 2006).

Evaluation Method. We use the performance of the MLKNN algorithm to evaluate
the discriminative ability of the learned metric, We use multiple popular
multi-label learning indicators used in (Zhu & Yang, 2018) to evaluate the
permormance of MLKNN. The up arrow represents the higher the indicator, the
better, and the down arrow represents the lower the indicator, the better.

Experimental Analysis. Table 2 shows the results on multi-label datasets,
Compared with the original performance of MLkNN, RAML and HRAML learn more
discriminative metric in multi-learning task. Compared with RAML algorithm,
HRAML has better experimental results. Thanks to the improvement of the
encoder, it has better performance than RAML. Beyond linear transform or
kernel methods, HRAML has more power to extract features. And it can
distinguish samples under different labels well through its optimization
goals, .

###  6.3 Label Distribution Learning

dataset. We use the Nature Scene dataset used in (Zhu & Yang, 2018).
Evaluation Method. We use the performance of AAKNN to evaluate the learned
metric, and use AAKNN and RAML as comparison algorithms. We use multiple
evaluation indicators used in (Zhu & Yang, 2018) as the evaluation indicators.
In order to maintain consistency with other experimental parts, We use (22) as
the relationship function of the decision space. Of course, some better
distribution functions such as kl divergence can be used as the relationship
function, which should achieve better effect. Experimental analysis. Table 3
shows the comparison of AAKNN, RAML and HRAML. Thanks to the HRAML’s encoder
and optimization goal, it’s performance is better than RAML.

##  7 Conclution

This paper proposes a hierarchical relationship alignment model HRAML, which
uses the concept of relationship alignment to model metric learning problems
under multiple learning tasks, and makes full use of the consistency between
the sample pair relationship in the feature space and the sample pair
relationship in the label space. Finally, the performance of HRAML is verified
under a variety of tasks.

## References

  * Cakir et al. (2019) Cakir, F., He, K., Xia, X., Kulis, B., and Sclaroff, S.  Deep metric learning to rank.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 1861–1870, 2019.
  * Davis et al. (2007) Davis, J. V., Kulis, B., Jain, P., Sra, S., and Dhillon, I. S.  Information-theoretic metric learning.  In _ICML_ , pp. 209–216, 2007.
  * Ge (2018) Ge, W.  Deep metric learning with hierarchical triplet loss.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 269–285, 2018.
  * Geng (2016) Geng, X.  Label distribution learning.  _IEEE TKDE_ , 28(7):1734–1748, 2016.
  * Guillaumin et al. (2009) Guillaumin, M., Verbeek, J., and Schmid, C.  Is that you? metric learning approaches for face identification.  In _ICCV_ , pp. 498–505. IEEE, 2009.
  * Hadsell et al. (2006) Hadsell, R., Chopra, S., and LeCun, Y.  Dimensionality reduction by learning an invariant mapping.  In _2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06)_ , volume 2, pp. 1735–1742. IEEE, 2006.
  * Hoi et al. (2006) Hoi, S. C., Liu, W., Lyu, M. R., and Ma, W.-Y.  Learning distance metrics with contextual constraints for image retrieval.  In _CVPR_ , volume 2, pp. 2072–2078, 2006.
  * Liao et al. (2015) Liao, S., Hu, Y., Zhu, X., and Li, S. Z.  Person re-identification by local maximal occurrence representation and metric learning.  In _CVPR_ , pp. 2197–2206, 2015.
  * Mensink et al. (2012) Mensink, T., Verbeek, J., Perronnin, F., and Csurka, G.  Metric learning for large scale image classification: Generalizing to new classes at near-zero cost.  _ECCV_ , pp. 488–501, 2012.
  * Shrivastava et al. (2016) Shrivastava, A., Gupta, A., and Girshick, R.  Training region-based object detectors with online hard example mining.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 761–769, 2016.
  * Sohn (2016) Sohn, K.  Improved deep metric learning with multi-class n-pair loss objective.  _Advances in neural information processing systems_ , 29:1857–1865, 2016.
  * Sun et al. (2020) Sun, Y., Cheng, C., Zhang, Y., Zhang, C., Zheng, L., Wang, Z., and Wei, Y.  Circle loss: A unified perspective of pair similarity optimization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 6398–6407, 2020.
  * Trohidis et al. (2008) Trohidis, K., Tsoumakas, G., Kalliris, G., and Vlahavas, I. P.  Multi-label classification of music into emotions.  In _ISMIR_ , volume 8, pp. 325–330, 2008.
  * Wang et al. (2015) Wang, F., Zuo, W., Zhang, L., Meng, D., and Zhang, D.  A kernel classification framework for metric learning.  _IEEE TNNLS_ , 26(9):1950–1962, 2015.
  * Wang et al. (2017) Wang, J., Zhou, F., Wen, S., Liu, X., and Lin, Y.  Deep metric learning with angular loss.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 2593–2601, 2017.
  * Weinberger & Saul (2009) Weinberger, K. Q. and Saul, L. K.  Distance metric learning for large margin nearest neighbor classification.  _Journal of Machine Learning Research_ , 10(Feb):207–244, 2009.
  * Wen et al. (2016) Wen, Y., Zhang, K., Li, Z., and Qiao, Y.  A discriminative feature learning approach for deep face recognition.  In _European conference on computer vision_ , pp. 499–515. Springer, 2016.
  * Wu et al. (2017) Wu, C.-Y., Manmatha, R., Smola, A. J., and Krahenbuhl, P.  Sampling matters in deep embedding learning.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 2840–2848, 2017.
  * Ying & Li (2012) Ying, Y. and Li, P.  Distance metric learning with eigenvalue optimization.  _Journal of Machine Learning Research_ , 13(Jan):1–26, 2012.
  * Zadeh et al. (2016) Zadeh, P., Hosseini, R., and Sra, S.  Geometric mean metric learning.  In _ICML_ , pp. 2464–2471, 2016.
  * Zhang & Wu (2015) Zhang, M.-L. and Wu, L.  Lift: Multi-label learning with label-specific features.  _IEEE TPAMI_ , 37(1):107–120, 2015.
  * Zhu & Yang (2018) Zhu, Pengfei, R. Q. Q. H. Q. W. C. Z. and Yang, L.  Beyond similar and dissimilar relations.: A kernel regression formulation for metric learning.  In _IJCAI_ , pp. 3242–3248, 2018.