# From Visual Prompt Learning to Zero-Shot Transfer:
Mapping Is All You Need

Ziqing Yang 111The first two authors made equal contributions. Zeyang Sha
111The first two authors made equal contributions. Michael Backes Yang Zhang

CISPA Helmholtz Center for Information Security

###### Abstract

Visual prompt learning, as a newly emerged technique, leverages the knowledge
learned by a large-scale pre-trained model and adapts it to downstream tasks
through the usage of prompts. While previous research has focused on designing
effective prompts, in this work, we argue that compared to prompt design, a
good mapping strategy matters more. In this sense, we propose SeMap, a more
effective mapping using the semantic alignment between the pre-trained model’s
knowledge and the downstream task. Our experimental results show that SeMap
can largely boost the performance of visual prompt learning. Moreover, our
experiments show that SeMap is capable of achieving competitive zero-shot
transfer, indicating that it can perform the downstream task without any fine-
tuning on the corresponding dataset. This demonstrates the potential of our
proposed method to be used in a broader range of applications where the zero-
shot transfer is desired. Results suggest that our proposed SeMap could lead
to significant advancements in both visual prompt learning and zero-shot
transfer. We hope with SeMap, we can help the community move forward to more
efficient and lightweight utilization of large vision models.

##  Introduction

Figure 1:  Overview of methods adapting the pre-trained model to downstream
tasks. (a) illustrates fine-tuning, which optimizes the whole model based on
the downstream dataset. (b) demonstrates linear probing, which replaces the
final layer of the pre-trained model with a new linear layer, then trains only
the new layer on the downstream dataset. (c) is visual prompt learning, which
includes a prompt and a mapping strategy. The prompt is optimized based on the
downstream dataset while the backbone model is kept frozen.

Recently, large-scale pre-trained models such as ViT [11] and CLIP [30] have
shown their superiority in many applications [36, 18, 21, 4, 29]. As such
large-scale models are typically trained on massive datasets, they capture
profound knowledge which can be adapted to many downstream tasks. As shown in
Figure 1, fine-tuning [10, 24] and linear probing [15, 30] are two commonly
used methods for this adaptation. Fine-tuning updates the parameters of the
pre-trained model by training on the downstream datasets. However, optimizing
a large pre-trained model is computationally intensive. Additionally, the
fine-tuned model usually becomes too specific to the training data and does
not generalize well to new data. Linear probing involves replacing the final
layer of the pre-trained model with a new, untrained linear layer, and then
training only the parameters of the new layer on the downstream dataset. As
only the new linear layer is optimized, linear probing adapts the pre-trained
model to new tasks more efficiently. However, both methods require some level
of modifications to the pre-trained model, e.g., modifying some parameters or
layers.

To overcome such limitations, visual prompt learning [12, 32, 20, 3] has
emerged as an efficient and effective alternative to fine-tuning for pre-
trained vision models. As shown in Figure 1 (d), visual prompt learning keeps
the pre-trained model frozen, then learns a perturbation (i.e., a _prompt_)
added to the input image and maps the output of the pre-trained model to fit
the downstream task. Such a paradigm enables visual prompt learning to achieve
efficient and lightweight knowledge transfer from pre-trained models. However,
existing methods [3, 20] only focus on how to design and optimize the prompt,
neglecting the importance of the mapping strategy, i.e., mapping the output of
the pre-trained model to the downstream task. More importantly, in practice,
the performance of visual prompt learning varies with different downstream
tasks and is relatively low on many datasets.

In this paper, instead of focusing on designing proper prompts, we find that
the mapping strategies matter more regarding boosting the performance of
visual prompt learning. More specifically, we propose SeMap (Semantics-based
Mapping), which explores the semantic alignment between the pre-trained task
and the downstream tasks. We further design SeMap-1 and SeMap-a, respectively,
to enhance the capability of visual prompt learning concretely. In SeMap-1, we
follow the intuition that there exist implicit connections between the output
of the pre-trained model and the downstream task. We try to map the
semantically closest categories from the pre-trained task to downstream
classes. To achieve this, we leverage the text encoder of CLIP [30] to
generate text embeddings of both pre-trained class labels and downstream class
labels. Then, we measure the semantic similarity by calculating the cosine
similarity between two embeddings. Thus the downstream class would be mapped
to its most similar class in the pre-trained task.

Unlike 1-on-1 mapping in SeMap-1, SeMap-a tries to map multiple indices to one
downstream class to leverage the pre-trained model’s information better, i.e.,
an $k$-on-1 mapping. To find the best $k$ for different downstream classes, we
propose an adaptive method SeMap-a. Our adaptive method can find the most
representative $k$ based on the ideology of clustering. Given a downstream
class, our algorithm will first sort the similarities of all pre-trained
classes. We further find a gap between different similarities and then divide
the pre-trained classes into top-$k$ classes and other classes. Indices
correspond to top-$k$ classes will be mapped to the given downstream class.

To evaluate the effectiveness and efficiency of our proposed SeMap, we first
compare it with existing visual prompt learning [7, 3]. Our evaluation results
demonstrate that with SeMap, visual prompt learning can have much better
performance than previous work. For instance, methods proposed by Bahng et al.
[3] can only achieve 0.412 accuracy on CIFAR10 [1] ResNet18 [16], while SeMap
(including SeMap-1 and SeMap-a) can achieve 0.570/0.642, which is much higher.
We also find that even the downstream datasets are totally different from pre-
trained datasets, e.g., Fashion-MNIST [33], SeMap based on semantic meaning
mapping can still have better performance than just random map in previous
works. Apart from visual prompt learning, we further point out that with the
mapping strategies in SeMap, we can also achieve great zero-shot transfer
performance without training the prompts. For instance, with SeMap-a, the
zero-shot transfer can have 0.658 accuracy on STL10 [2] ResNet50 [16], which
is even higher than the visual prompt learning methods proposed in Bahng et al
[3].

Our contributions can be summarized as follows.

  * •

We are the first to systematically summarize the existing visual prompt
learning on pre-trained image classifiers.

  * •

We are the first to demonstrate that in visual prompt learning, mapping
strategies matter more than the design of prompts. We further proposed SeMap,
a mapping method based on semantics.

  * •

Extensive experiments show that SeMap can largely boost the performance of
visual prompt learning.

  * •

SeMap itself can achieve great zero-shot transfer performance, and its
effectiveness and efficiency are shown in many downstream tasks.

##  Related Work

###  Prompt Learning

Prompt Learning in NLP. Prompt learning [26] has been extensively studied and
applied in the field of natural language processing (NLP). Take a pre-trained
language model as the backbone, a prompt is a task-specific template used to
reformulate the downstream task’s input. Consider the task of predicting the
emotion (positive/negative) of a given text. A suitable prompt for this task
might be “[INPUT] All in all, I felt [MASK].” Given the input “The sunshine is
warm and beautiful,” the backbone model might fill the mask as “wonderful”,
which conveys a positive emotion, enabling the output to be classified
accordingly. Hand-crafting an effective textual prompt for a downstream task
can improve prompt learning. However, this process often requires significant
domain expertise and a great deal of trial-and-error experimentation.
AutoPrompt [31] is the first to study how to automatically get the appropriate
prompt from the vocabulary. However, determining proper words is discrete and
hard to optimize. Recent approaches [13, 25, 23, 27, 14, 5] aim to address
these issues by focusing on learning a "soft prompt," which is a continuous
vector that can be optimized via backpropagation while keeping the backbone
model frozen. Notable examples of this approach include prompt tuning [25] and
hybrid tuning [27].

Prompt Learning in CV. Building on the success of prompt learning in NLP,
researchers have explored the possibility of adapting this approach to the
domain of computer vision (CV), resulting in the emergence of visual prompt
learning. Before prompt learning, researchers identified an attack known as
adversarial reprogramming in computer vision. Adversarial reprogramming [12]
is an adversarial attack that repurposes pre-trained models without modifying
any of their parameters by adding a perturbation to the input image. Despite
having different motivations, the input perturbation acts as a prompt.
Enlightened by this, works on visual prompt learning [7, 20, 3] are proposed,
aiming at transferring the knowledge of the pre-trained model to downstream
tasks. For example, Chen et al. [7] further enhance adversarial reprogramming
as a transfer learning strategy. And they propose a frequency-based mapping
strategy. Jia et al. [20] proposes VPT that is specific to Vision Transformers
[11]. Bahng et al. [3], instead, focus on pre-trained vision models including
image classification models and CLIP [30] models, and propose a prompt
learning framework. The prompt is an image perturbation learned via
backpropagation while keeping the backbone model frozen. However, they only
use the random mapping strategy, which maps the top indices of the output to
the downstream task’s labels. In this paper, we will follow this framework and
mainly investigate visual prompt learning on image classification models.

###  Zero-Shot Transfer

The concept of zero-shot transfer is first proposed by Radford et al. [30].
Zero-shot transfer is a method that aims to perform unseen tasks by utilizing
the knowledge learned from related tasks, _without_ additional training or
fine-tuning. It focuses on measuring the task learning capabilities of machine
learning models. Different from zero-shot learning, zero-shot transfer
emphasizes without training or fine-tuning on “unseen” tasks. Recently, there
are also some extensions of zero-shot transfer [19, 35, 8].

##  Preliminaries

Figure 2:  Overview of SeMap, which is based on semantic similarity. For each
label $l^{i}\in\mathcal{L}$, we compute its semantic similarity with all
labels $\mathcal{L}_{p}$ in the pre-trained task. We then sort the indices by
their similarities from high to low. Specifically, (a) SeMap-1 maps the index
with the highest similarity to the $i$-th class. (b) SeMap-a is an adaptive
$k$-on-1 mapping method that defines $k$ for each label automatically by
introducing a hyperparameter $\epsilon$.

An image classification model is a parameterized function
$f_{\Theta}:\mathbb{R}^{d\times d}\to\mathbb{R}^{n}$ that maps a $d\times d$
image $x$ to an output $y\in\mathbb{R}^{n}$. Here, $\Theta$ represents the
parameters of the function $f$, $d$ is the size of the image, and $n$
corresponds to the number of classes to be predicted. More specifically, the
output $y$ is a probability distribution over $n$ possible classes. The label
of the image is predicted as the class with the highest probability, i.e.,
$\arg\max_{i\in[1,n]}{y^{i}}$.

##  Mapping Is All You Need

In this section, we first systematically summarize the framework of visual
prompt learning. Then we propose our method SeMap as the mapping strategy to
enhance visual prompt learning. We also show that our proposed SeMap can adapt
large pre-trained visual models to new downstream tasks without _any_
optimization, i.e., can achieve zero-shot transfer in downstream tasks.

###  Visual Prompt Learning

We first define how visual prompt learning works. The framework consists of a
backbone model $f_{\Theta}$, a visual prompt $\Delta$, and a mapping strategy
$h$.

Backbone Model. The backbone model is an image classification model
$f_{\Theta}$ pre-trained on a pre-trained dataset
$(\mathcal{X}_{p},\mathcal{Y}_{p})$, where
$\mathcal{X}_{p}\subset\mathbb{R}^{d\times d}$ contains images, and
$\mathcal{Y}_{p}\subset\mathbb{R}^{n}$ corresponds to the classes.

Visual Prompt. The visual prompt is normally a trainable perturbation $\Delta$
added to an image $x$ so that images with that perturbation in the downstream
dataset $(\mathcal{X},\mathcal{Y})$ can be adapted to the input space of the
backbone model, where $\mathcal{X}\subset\mathbb{R}^{d\times d}$ and
$\mathcal{Y}\subset\mathbb{R}^{m}$. Specifically, we can present the prompted
image as $\oplus(x,\Delta)\in\mathbb{R}^{d\times d}$, where
$\oplus(\cdot,\cdot)$ represents adding the perturbation to the image. The
prompted image has the capability of spurring the backbone model to perform a
certain downstream task. As discussed in Bahng et al. [3], $\Delta$ can be a
pixel patch at a random/fixed location or padding around the image. Chen et
al. [7] perturb the whole image so that $\Delta$ can also have the same size
as the input image. In this work, we focus on the prompt that is padding
around the image as padding achieves the best performance over other design
choices [3].

Mapping Strategy. The mapping strategy is an output transformation that maps
the output of the backbone model to the labels of the downstream task, i.e.,
$h(y_{p})=y,y_{p}\in\mathbb{R}^{n},y\in\mathbb{R}^{m}$. There are several
existing mapping strategies.

  * •

Random Mapping (RM). We start from this simplest mapping strategy that is used
in many existing works [12, 7, 3]. RM randomly selects $m$ indices from the
output $y_{p}\in\mathcal{Y}_{y}$ and maps them to the output $y\in\mathcal{Y}$
of the downstream task One simplest way is to map the top $m$ indices of
$y_{p}$ to $y$. It can be written as:

$y^{i}=y_{p}^{i},i\in[1,m].$ (1)

The remaining $n-m$ indices of $y_{p}$ are abandoned.

  * •

Frequency-Based Mapping (FM). Although RM is simple and easy to conduct, such
a strategy ignores the connection between the pre-trained task and the
downstream task. Chen et al. [7] propose FM, which makes use of the
information contained in the backbone model. Suppose $\mathcal{X}^{i}$ denotes
the images in class $i$ in $\mathcal{X}$. For $x\in\mathcal{X}^{i}$, we can
get $y_{p}=f_{\Theta}(x)$, which is the original output without prompt. And
the class index of the pre-trained task it corresponds to is
$\arg\max_{j\in[1,n]}{y_{p}^{j}}$. Based on the outputs of all
$x\in\mathcal{X}^{i}$, we can assign the most frequent class index $j$ to the
corresponding class $i$ in the downstream task. For each $y^{i}\in
y,i\in[1,m]$, it is mapped to $y_{p}^{j}\in y_{p},j\in[1,n]$ based on
frequency. And $j$ is defined as the most frequent class index for images in
class $i$ in the downstream dataset.

$j=\arg\max_{j\in[1,n]}{\mathbb{E}(f_{\Theta}{(x)}\text{ is
}j|x\in\mathcal{X}^{i})}.$ (2)

Overall. As demonstrated in Figure 1 (c), visual prompt learning first adds a
prompt $\Delta$ on the input image and then puts it into the backbone model
$f_{\Phi}$, finally, the output is mapped by the mapping strategy $h$. Based
on the training dataset $(\mathcal{X},\mathcal{Y})$, visual prompt learning
optimizes the prompt $\Delta$ via backpropagation. It aims to optimize:

$\arg\min_{\Delta}\mathbb{E}_{(x,y)\in(\mathcal{X},\mathcal{Y})}{L(h\circ
f\circ\oplus(x,\Delta),y)}.$ (3)

Here, $L(\cdot,\cdot)$ denotes the cross-entropy loss, which is widely used in
supervised learning.

We observe that current mapping strategies have some limitations. As shown
above, RM randomly assigns the indices, ignoring any connection between the
pre-trained and downstream tasks. As for FM, it maps the most frequent index
to the corresponding class. However, it requires obtaining a set of examples
first. For each class, FM first requires some images in this class, then
obtains their outputs from the model and computes the frequency. Therefore, FM
is time-consuming and requires access to the downstream dataset before
conducting visual prompt learning.

###  SeMap

In this paper, we propose SeMap, which only requires the label information of
the pre-trained and downstream tasks instead of access to the downstream
dataset. We now discuss how our proposed SeMap works. We divide our method
into two parts, i.e., SeMap-1 and SeMap-a. SeMap-1 adopts the intuitive way to
conduct the map, e.g., finding the class of the pre-train dataset with the
closest semantics of the downstream class. SeMap-a is an adaptive method,
which adopts $k$-on-1, instead of 1-on-1, to map multiple classes to one
downstream class. SeMap-a aims to find the best $k$ for each downstream class
automatically. We will introduce these two methods below.

SeMap-1. We start with a simple setting that we map one index of the backbone
model’s output to each downstream class. Recall that the pre-trained dataset
and the downstream dataset are labeled. We denote the labels of the pre-
trained dataset as $\mathcal{L}_{p}$ and that of the downstream dataset as
$\mathcal{L}$. Each label is a natural language description that contains the
semantic information of the corresponding class. To gain a better mapping
strategy, we make use of the semantic alignment between $\mathcal{L}_{p}$ and
$\mathcal{L}$. For each label $l^{i}\in\mathcal{L}$, we measure its semantic
similarity with all labels in $\mathcal{L}_{p}$. Then we sort the indices by
the similarity from high to low as
$\mathcal{S}=\\{s_{1},s_{2},\cdots,s_{n}\\}$. For example, for a label
$l^{i}\in\mathcal{L}$, the most similar label would be $l_{p}^{s_{1}}$, as
shown in Figure 2 (a). In other words,
$s_{1}=\arg\max_{j\in[1,n]}{\text{Sim}(l_{p}^{j},l^{i})}$, where
$\text{Sim}(\cdot,\cdot)$ represents the semantic similarity. Naturally, based
on the semantic similarity, SeMap maps the index with the most similar label
to the target downstream class:

$y^{i}=y_{p}^{s_{1}},i\in[1,m].$ (4)

We first put the label descriptions into the text encoder of the CLIP model
[30], then the semantic similarity is given by computing the cosine similarity
between two text embeddings. The reason we choose CLIP is that CLIP learns the
alignment between texts and images so that its text and image embeddings are
in the same space. We notice that there could be one label that is the most
similar one with several downstream labels. For example, “bird” and “airplane”
in STL110 are both quite similar to “fly” in ImageNet-1k. In this case,
SeMap-1 would find the next similar label and assign its index to the
downstream label.

SeMap-a. Although SeMap-1 successfully aligns the semantic meaning, it only
takes advantage of limited information from the output of the backbone model.
Therefore, we develop SeMap-a to overcome such drawbacks. SeMap-a generalizes
SeMap-1 to a multi-label mapping setting, i.e., we map multiple indices of the
model’s output to one class in $\mathcal{Y}$. We define $k$ as the number of
multiple indices mapped to one class.

(a) ResNet18

(b) ResNet50

(c) BiT-M ResNet50

(d) Instagram ResNeXt

Figure 3: The performance of RM-VP, FM-VP, SeMap-1, SeMap-a, and FT on
ResNet18, ResNet50, Instagram ResNeXt ,and BiT-M ResNet50. The x-axis
represents different downstream datasets (CIFAR10, CIFAR100, STL10, and
F-MNIST). The y-axis represents the test accuracy.

(a) ResNet18

(b) ResNet50

(c) BiT-M ResNet50

(d) Instagram ResNeXt

Figure 4: The performance of zero-shot transfer of Random Guess, SeMap-1,
SeMap-a, and RM-VP on ResNet18, ResNet50, Instagram ResNeXt ,and BiT-M
ResNet50. The x-axis represents different downstream datasets (CIFAR10,
CIFAR100, STL10, and F-MNIST). The y-axis represents the test accuracy.

Different from defining a unified $k$ for all classes, SeMap-a aims at
automatically assigning a $k$ for each label $l^{i}\in\mathcal{L}$. We first
introduce two concepts named _reachability_ and _connectivity_. Recall that
for a label $l^{i}\in\mathcal{L}$, the corresponding indices sorted by the
similarity from high to low are $\mathcal{S}=\\{s_{1},s_{2},\cdots,s_{n}\\}$.
As shown in Figure 2, $\mathcal{S}$ can be considered as a set of points lying
on a line, the value of each point is the corresponding semantic similarity.
For any two points in $\mathcal{S}$, they are _reachable_ if the distance
between them, i.e., their semantic similarity difference, is smaller than a
threshold. Based on transitivity, two points are _connected_ if there exists a
path that any two points next to each other are reachable. For example, for
$s_{s},s_{t}\in\mathcal{S}$, if there exists a path/sequence
$s_{s}\leftrightarrow s_{a}\leftrightarrow s_{b}\leftrightarrow s_{t}$ where
$\leftrightarrow$ denotes reachable, then $s_{s}$ and $s_{t}$ are connected.

SeMap-a aims to find all points that are connected with $s_{1}$. Here we
introduce two hyperparameters, a distance threshold $\epsilon$ and a decay
rate $\gamma$. The distance $\epsilon$ denotes the initial distance threshold
in SeMap-a, i.e., two points whose distance is shorter than $\epsilon$ will be
considered reachable. After we go to the next point, the distance threshold
will decrease by the decay rate $\gamma$. For example, when we centered on
$s_{3}$, the distance threshold would be ${\gamma}^{2}\epsilon$. In this way,
the mapping function can be written as:

$y^{i}=\sum_{j\in[1,k_{i}]}^{k}{y_{p}^{s_{j}}}.$ (5)

Here, $k_{i}$ is assigned for each label adaptively, which denotes the number
of points that are connected with $s_{1}$. Note that SeMap-1 is the special
case of SeMap-a when $k=1$.

Computational Complexity. SeMap is efficient from the perspective of
computational complexity. For a label $l^{i}\in\mathcal{L}$, the complexity
for getting the index with the highest similarity is $O(\log n)$ using max
heap, where $n$ corresponds to the number of pre-trained labels. There are $m$
labels in $\mathcal{L}$, thus the time complexity of SeMap-1 is $m\log n$,
respectively. As for SeMap-a, for $l^{i}\in\mathcal{L}$, the worst case is to
go through all points, i.e., $O(n\log n)$. But we can bound the complexity by
setting a maximum number $C$, i.e., after we get $C$ points, we will stop
searching. In this sense, the complexity for SeMap-a is $O(m*C\log n)=O(m\log
n)$.

###  From Visual Prompt Learning To Zero-Shot Transfer

We further find that SeMap can also be adapted to zero-shot transfer settings.
As recently in-context learning in NLP [6, 28] has been quite popular, the
mapping process can be considered as providing the context of a given image
for the pre-trained model. Therefore, users can only use the mapping strategy
we have proposed to adapt the pre-trained model to their own downstream tasks
without any optimization. The whole process can be described as

$y_{\text{predict}}=\arg\max_{j\in[1,m]}{(h\circ f_{\Theta}(x))^{j}}$ (6)

Here $h(\cdot)$ represents SeMap-a and is defined by Equation 5, and
$f_{\Theta}(\cdot)$ denotes the pre-trained model, respectively. We further
demonstrate that with SeMap, zero-shot transfer can even have stronger
performance than existing visual prompt learning works.

##  Experiments

(a) Zero-Shot: CIFAR10

(b) VP: CIFAR10

(c) Zero-Shot: STL10

(d) VP: STL10

Figure 5: The performance gap between different $k$ on different datasets. The
x-axis represents different models. The y-axis represents the accuracy gap.

(a) RM-VP: CIFAR10

(b) SeMap: CIFAR10

(c) RM-VP: CIFAR100

(d) SeMap: CIFAR100

Figure 6: The performance gap between different prompts on different datasets.
The x-axis represents different models The y-axis represents the test
accuracy.

###  Experimental Settings

Models and Datasets. Our target model is pre-trained on ImageNet-1k[9]. We use
four different kinds of model structures: ResNet18 (RN18) [16], ResNet50
(RN50) [16], Instagram ResNeXt (Ins) [34], and BiT-M ResNet50 (BiT) [22]. Note
that all of our models are pre-trained on ImageNet-1k. Specifically, Instagram
ResNeXt is first pre-trained on Instagram images with different tags and then
fine-tuned on ImageNet-1k. BiT-M ResNet50 also pre-trained on ImageNet-21k [9]
and then fine-tuned on ImageNet-1k. For downstream tasks, we evaluate SeMap on
four datasets: CIFAR10 [1], CIFAR100 [1], F-MNIST [33], and STL10 [2]. Our
implementation is based on a visual prompt learning framework [3].

Baseline Methods. We compare our methods with the most recent visual prompt
learning methods [7, 3], where random mapping and frequency-based mapping are
adopted. Therefore, we name the methods as RM-VP and FM-VP. Further, we
compare our methods with traditional fine-tuning, which is denoted as FT. Note
that visual prompt learning and fine-tuning are two different ways to adapt
pre-trained models to downstream tasks.

###  Performance of SeMap

(a) Padding

(b) Fixed Patch

(c) Random Patch

Figure 7: Different prompts for visual prompt learning.

(a) ResNet18

(b) ResNet50

(c) BiT-M-ResNet50

(d) Instagram ResNeXt

Figure 8: The relationship between FID scores and the gap of performance. The
x-axis represents different FID scores between the pre-trained dataset and
different downstream datasets (sorted as STL10, CIFAR10, CIFAR100, and
F-MNIST). The y-axis represents the accuracy gap between SeMap and previous
works.

In this section, we show that by taking SeMap as the mapping strategy in
visual prompt learning, we can achieve much better performance than RM-VP and
FM-VP. Further, we demonstrate that SeMap can even achieve good performance in
zero-shot transfer without the existence of prompts.

Visual Prompt Learning. We first show the effectiveness of our proposed SeMap
in visual prompt learning. The results are summarized in Figure 3. It is shown
that SeMap consistently outperforms existing methods by a large margin. For
instance, when RM-VP/FM-VP can only achieve 0.412/0.174 accuracy on CIFAR10
ResNet18, SeMap (including SeMap-1 and SeMap-a) can achieve the accuracy of
0.570/0.643, which is much higher. We also find that compared to the intuitive
1-on-1 mapping in SeMap-1, taking advantage of mapping multiple indices to one
class can have better performance, which preserves more knowledge from the
pre-trained model. For instance, on CIFAR100, SeMap-1 can achieve 0.236
accuracy on ResNet50 while the accuracy of SeMap-a is 0.256. Also, even if the
distribution of downstream datasets is far away from pre-trained datasets, for
example, on Fashion-MNIST Instagram-ResNeXt, the mapping strategy can still
have better performance (0.880 accuracy) than previous works (0.836 accuracy).
This result demonstrates that our SeMap does not just simply benefit from the
clustering results. Compared to traditional fine-tuning, SeMap can have better
performance on STL10, but is not so good on CIFAR10 and CIFAR100.

Visual prompt learning was proposed to achieve efficient and lightweight
knowledge transfer. It is reasonable to have a worse performance than
traditional fine-tuning shown in previous works [12, 7, 3], as fine-tuning
takes more time and computational resources for the transfer. In the next
section, we will introduce zero-shot transfer results, which demonstrate that
even without optimization, SeMap can still better leverage the pre-trained
model.

Zero-Shot Transfer. The above section has shown the distinct performance of
SeMap on visual prompt learning. Now we demonstrate that SeMap can also be
adapted to zero-shot transfer without optimization. The results are shown in
Figure 4. We observe that in most datasets, such as CIFAR10 and STL10, even
without optimization, SeMap can outperform RM-VP a lot. This demonstrates that
the mapping process is much more important than the prompt optimization
process. We will show more analysis on this conclusion in Section 5.3. Also,
different from SeMap of visual prompt learning, the performance of zero-shot
transfer depends more on the downstream tasks. For instance, since STL10 is
close to ImageNet-1k, it achieves 0.742 accuracy with SeMap on Instagram-
ResNeXt. Fashion-MNIST is a totally different dataset from ImageNet-1k, it
achieves 0.250 accuracy, which is relatively low but still much higher than
random guessing.

###  Ablation Study

Here, we further conduct ablation studies to explore the effectiveness of our
proposed SeMap.

Influence of $k$ in SeMap-a. We first evaluate the influence of mapping
different numbers of classes of the pre-trained model to one downstream class.
As we have demonstrated, SeMap-a can have better performance than SeMap-1.
This makes us wonder how the different numbers of target classes, i.e., $k$,
impact the overall performance. Figure 5 shows the performance of different
$k$ on both visual prompt learning and zero-shot transfer. With different $k$,
the performance varies a lot. For instance, on CIFAR10 ResNet50, if $k$ is
100, the accuracy reaches 0.516, while it can raise to 0.580 if $k$ is 10. We
also find that our proposed adaptive method SeMap-a can always find a good $k$
for different classes in different datasets. For instance, on CIFAR10
ResNet18, different $k$ can have an accuracy ranging from 0.496 to 0.651,
while SeMap-a achieves 0.642 accuracy. Figure 5 also indicates that $k$ has a
larger impact on zero-shot transfer than visual prompt learning. For instance,
on STL10 ResNet50, the performance on zero-shot transfer can range from 0.423
to 0.576, while that of visual prompt learning ranges from 0.577 to 0.712,
where the gap is smaller.

Mapping VS. Prompt Optimization. As we have stated, in visual prompt learning,
mapping matters more than prompt design. To better illustrate our conclusion,
we conduct experiments to explore the performance of different prompts on both
the RM-VP and SeMap, where different prompts are depicted in Figure 7. The
results are shown in Figure 6. Compared to the impact of $k$ shown in Figure
5, the performance gap between good prompts (with a higher performance) and
bad prompts (with a lower performance) is larger. For instance, for CIFAR10 on
BiT-M-ResNet50, padding (shown in 7(a)) achieves 0.692 accuracy, while the
accuracy of fixed patch (shown in 7(b)) is only 0.453. Also, we observe that
even the prompt with the worst performance in SeMap can have a better
performance than the best prompt in RM-VP. For instance, bad prompt of SeMap
on STL10 on ResNet50 achieves 0.778 accuracy, which is still higher than the
0.473 accuracy on good prompts of RM-VP. This observation supports our claim
that in visual prompt learning, mapping is much more important than the design
of prompts.

Relationship Between Downstream and Pre-trained Datasets. Since our methods
rely on the semantic relationship between the downstream and pre-trained
datasets, it is worthwhile for us to explore the impacts of the relationship
between these two datasets. Therefore, we introduce FID score [17], which is a
metric that measures the similarity between two datasets. We compute the FID
score between ImageNet-1k (as the pre-trained dataset) and the four downstream
datasets we have tried. For visual prompt learning, we calculate the accuracy
gap of SeMap and RM-VP. For zero-shot transfer, we calculate the accuracy gap
of SeMap and random guess for a fair comparison. Figure 8 shows the results.
We observe that, if the downstream dataset is closer to the pre-trained
dataset, e.g., STL10, both zero-shot transfer and visual prompt learning can
have better performance. Also, we find that if the downstream datasets are
similar to pre-trained datasets, prompt optimization can largely boost the
visual prompt learning performance better. From Figure 8 we also can argue
that even though downstream datasets are totally different from the pre-
trained dataset, our proposed SeMap can still have better performance than
previous works as the gap is always greater than 0 in all cases.

##  Conclusion

In this paper, we focus on exploring the mapping strategies in visual prompt
learning. We first systematically summarize the various paradigms of visual
prompt learning. We demonstrate that mapping is much more important than the
design of prompts in visual prompt learning. We further propose the SeMap
based on semantic closeness. More specifically, we propose SeMap-1 and SeMap-a
with 1-on-1 mapping and $k$-on-1 mapping. To find the best $k$ for SeMap-a, we
propose adaptive methods to detect the natural gap among similarities.
Extensive experiments show that our SeMap can have much better performance
than previous works. Moreover, we find that with the mapping strategies in
SeMap, we can take advantage of large visual models for zero-shot transfer
tasks without optimizing prompts. Further experiments show that with SeMap,
the zero-shot transfer can even have better performance than previous
training-based methods in visual prompt learning. It better demonstrates the
advantages of our proposed SeMap. We also conduct ablation studies to show
that mapping matters more than the optimization of prompts by computing the
gap of the accuracy of different prompts and different $k$ in $k$-on-1. We
hope that our work will help the community better understand visual prompt
learning. As our proposed SeMap can achieve that good performance, we hope to
provide new insights for the efficient usage of large models with limited
computational cost.

## References

  * [1] https://www.cs.toronto.edu/~kriz/cifar.html.
  * [2] https://cs.stanford.edu/%7Eacoates/stl10/.
  * [3] Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola.  Exploring Visual Prompts for Adapting Large-scale Models.  CoRR abs/2203.17274, 2022.
  * [4] Hangbo Bao, Li Dong, and Furu Wei.  BEiT: BERT Pre-Training of Image Transformers.  CoRR abs/2106.08254, 2021.
  * [5] Eyal Ben-David, Nadav Oved, and Roi Reichart.  PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains.  Transactions of the Association for Computational Linguistics, 2022.
  * [6] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.  Language Models are Few-Shot Learners.  In Annual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS, 2020.
  * [7] Lingwei Chen, Yujie Fan, and Yanfang Ye.  Adversarial Reprogramming of Pretrained Neural Networks for Fraud Detection.  In ACM International Conference on Information and Knowledge Management (CIKM), pages 2935–2939. ACM, 2021.
  * [8] Xi Chen, Xiao Wang, Soravit Changpinyo, A. J. Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Burcu Karagol Ayan, Carlos Riquelme, Andreas Steiner, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut.  PaLI: A Jointly-Scaled Multilingual Language-Image Model.  CoRR abs/2209.06794, 2022.
  * [9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  ImageNet: A large-scale hierarchical image database.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. IEEE, 2009.
  * [10] Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah A. Smith.  Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping.  CoRR abs/2002.06305, 2020.
  * [11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.  In International Conference on Learning Representations (ICLR), 2021.
  * [12] Gamaleldin F. Elsayed, Ian J. Goodfellow, and Jascha Sohl-Dickstein.  Adversarial Reprogramming of Neural Networks.  In International Conference on Learning Representations (ICLR), 2019.
  * [13] Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May.  WARP: Word-level Adversarial ReProgramming.  In Annual Meeting of the Association for Computational Linguistics (ACL), pages 4921–4933. ACL/IJCNLP, 2021.
  * [14] Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun.  PTR: Prompt Tuning with Rules for Text Classification.  AI Open, 2022.
  * [15] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick.  Masked Autoencoders Are Scalable Vision Learners.  CoRR abs/2111.06377, 2021.
  * [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. IEEE, 2016.
  * [17] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium.  In Annual Conference on Neural Information Processing Systems (NIPS), pages 6626–6637. NIPS, 2017.
  * [18] Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, and Jianlong Fu.  Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers.  CoRR abs/2004.00849, 2020.
  * [19] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig.  Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision.  In International Conference on Machine Learning (ICML), pages 4904–4916. PMLR, 2021.
  * [20] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge J. Belongie, Bharath Hariharan, and Ser-Nam Lim.  Visual Prompt Tuning.  In European Conference on Computer Vision (ECCV), pages 709–727. Springer, 2022.
  * [21] Virapat Kieuvongngam, Bowen Tan, and Yiming Niu.  Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2.  CoRR abs/2006.01997, 2020.
  * [22] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby.  Big Transfer (BiT): General Visual Representation Learning.  In European Conference on Computer Vision (ECCV), pages 491–507. Springer, 2020.
  * [23] Brian Lester, Rami Al-Rfou, and Noah Constant.  The Power of Scale for Parameter-Efficient Prompt Tuning.  In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3045–3059. ACL, 2021.
  * [24] Hao Li, Pratik Chaudhari, Hao Yang, Michael Lam, Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto.  Rethinking the Hyperparameters for Fine-tuning.  In International Conference on Learning Representations (ICLR), 2020.
  * [25] Xiang Lisa Li and Percy Liang.  Prefix-Tuning: Optimizing Continuous Prompts for Generation.  In Annual Meeting of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL/IJCNLP), pages 4582–4597. ACL, 2021.
  * [26] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.  Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing.  CoRR abs/2107.13586, 2021.
  * [27] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang.  GPT Understands, Too.  CoRR abs/2103.10385, 2021.
  * [28] Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer.  Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?  In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 11048–11064. ACL, 2022.
  * [29] Ron Mokady, Amir Hertz, and Amit H. Bermano.  ClipCap: CLIP Prefix for Image Captioning.  CoRR abs/2111.09734, 2021.
  * [30] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning Transferable Visual Models From Natural Language Supervision.  In International Conference on Machine Learning (ICML), pages 8748–8763. PMLR, 2021.
  * [31] Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh.  AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts.  In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235. ACL, 2020.
  * [32] Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. M. Ali Eslami, Oriol Vinyals, and Felix Hill.  Multimodal Few-Shot Learning with Frozen Language Models.  In Annual Conference on Neural Information Processing Systems (NeurIPS), pages 200–212. NeurIPS, 2021.
  * [33] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms.  CoRR abs/1708.07747, 2017.
  * [34] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He.  ResNeXt: Aggregated Residual Transformations for Deep Neural Networks.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017.
  * [35] Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer.  LiT: Zero-Shot Transfer with Locked-image text Tuning.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.
  * [36] Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan.  DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation.  CoRR abs/1911.00536, 2019.


# The Social Triad model of Human-Robot Interaction*

David Cameron1 and Emily Collins2 and Stevienna de Saille3 and James Law4
*This work was funded by the UKRI projects EP/V00784X/1 Trustworthy Autonomous
Systems Hub1David Cameron is with the Information School, University of
Sheffield, UK<EMAIL_ADDRESS>Collins is with the Institute
for Experiential Robotics, Northeastern University, Mass, USA
<EMAIL_ADDRESS>de Saille is with the Department for
Sociologial Studies, University of Sheffield, UK
<EMAIL_ADDRESS>Law is with the Department of Computer
Science, University of Sheffield, UK<EMAIL_ADDRESS>
###### Abstract

Despite the increasing interest in trust in human-robot interaction (HRI),
there is still relatively little exploration of trust as a social construct in
HRI. We propose that integration of useful models of human-human trust from
psychology, highlight a potentially overlooked aspect of trust in HRI: a
robot’s apparent trustworthiness may indirectly relate to the user’s
relationship with, and opinion of, the individual or organisation deploying
the robot. Our Social Triad for HRI model (User, Robot, Deployer), identifies
areas for consideration in co-creating trustworthy robotics.

##  I INTRODUCTION

Trust is increasingly recognised as being an important aspect of successful
Human-Robot Interaction (HRI). Substantial strategic funding has been invested
(e.g., Trustworthy Autonomous Systems, www.tas.ac.uk) to address the topic,
from direct study of HRI through to understanding the wider socio-technical
contexts for creating trustworthy robotics. This short paper proposes a means
by which the wider contexts for trustworthy robotics may shape trust during
HRI and potentially the outcomes of HRI studies; specifically, we consider how
wider relationships in an HRI study could affect trust - as a social construct
- towards robots.

Despite being relatively well-trodden ground in psychological literature that
evaluations of others (and trust towards others) can be characterised as a
social construct having both cognitive and affective components [1, 2, 3],
exploration of trust as a social construct is still a relatively new area of
study for HRI. While established models of trust such as the Human, Robot, and
Environment model [4, 5, 6] cover a wide range of factors, people’s
experiences of trust towards robots - and even social robots at that - has
been largely explored solely in terms of cognitive factors such as beliefs of
capability or reliability [7, 8, 9, 10, 11] rather than affective factors.

###  I-A Social Robotics

At its heart, the field of social robotics takes two key directions:
simulating social processes in robotic agents and studying people’s social
experiences and interaction with robotics [12]. In terms of trust, this could
include presenting various social-like behaviours in robots to shape
interaction experience [13], and exploring the potential for social models
(e.g., [1, 2, 3]) to further understand and explain trust in HRI.

Ahead of the development of any structured model of trust as a social
construct for HRI, a range of studies have nonetheless demonstrated user trust
can be influenced through robots’ simulated social or affective interactions.
Examples include using rhetorical persuasion [14], using gesture and
expression [15], taking blame [16] or offering apologies for errors [18, 19],
and making promises to change behaviours [20]. Emerging social models, drawing
from such examples, argue that trust as a social construct, as seen in human-
human interaction [1, 3], has relevance in HRI [18, 21]. People’s trust
towards a social robot may not be just based on factors such as reliability or
capability (the ‘competence’ dimension [3]), but also along the ‘warmth’
dimension based on factors such as apparent integrity or benevolence.

A substantial challenge in integrating social models of trust into HRI
research is clearly understanding people’s views of a robot as being a social
entity and its capacities for such social aspects. Evaluations of early
measures of trust as a social construct in HRI highlight a sizable percentage
of individuals declaring that concepts such as ‘intention’ and ‘benevolence’
to not apply to robots [17]. Yet, the social strategies identified above
appear to affect people’s trust towards robots; indeed, one study finds
apologies promote intentions to use while adversely affecting perceptions of a
robot’s capability [18].

This apparent contradiction still needs resolving; it is unclear how people
who do not believe a robot has capacity for (e.g.) ‘warmth’ based social
interaction and experiences may still be affected in their trust judgements
towards a robot exhibiting simulations of such interactions. In brief, are
users in some way being deceived or is there an alternative possibility?

###  I-B Can a social robot be trusted?

The issue of deception in social robotics is still hotly contested [22],
though one perspective [23] offers some points towards clarity in
understanding trust as a social construct in HRI.

As a starting point, the paper argues robots have no intention; despite their
apparent independence and agency, their behaviours are reflections of the
limitations set out by programmers [23]. That said, the paper recognises that
it is possible to deceive without intention, but again argues the deception
comes not from the robot per se; instead the deception arises from the design
and programming of the robot, the circumstances to create the deception, and
the users participation in the deception [23]. Similar arguments liken social
robots to that of puppets, in which people interact with the object as if they
are social agents [24], actively participating in the deception.

This stance, while ultimately arguing against social robots as being seen as
social agents, does nonetheless inform how trust as a social construct has
relevance to HRI. Interacting with robots as if they are social agents may
enable the effects from social strategies to shape trust (outlined earlier) to
be seen. Concurrently, knowledge or beliefs that the robot is a depiction of a
social agent rather than having the genuine capacity of a social agent, permit
honest responses that attributes or behaviours such as integrity or
benevolence are not applicable to robots. We pose that trust as a social
construct (e.g., understanding its benevolence) is not directed towards the
robot itself but towards it as an extension of the person behind the robot.
The classic model of a dyad in social robotics HRI is perhaps better examined
as a triad.

##  II The Social Triad of HRI

The robot as puppet analogy [24] highlights the role of the authority behind
the character in controlling its interaction with the participating audience.
The character serves as both as a participant in the interaction and as a
mediator for the interaction between audience and authority. We draw from this
to develop the Social Triad of HRI, specifically modifying the nature of the
authority to better accommodate experienced HRI. In the analogy the authority
directly controls the passive puppet; in HRI the authority (we have termed
elsewhere the ‘Deployer’ [25]) may have no direct control over the robot but
rather has assumed responsibility for the deployment of the robot and creation
of the HRI scenario.

The Deployer may have programmed the robot’s interactions and behaviour,
although this is not a necessary requirement. The Deployer may be present
during the HRI scenario and active or passive in the interaction between the
remaining agents: User and Robot, although again this is not entirely
necessary. The Deployer is the individual(s) the User believes has
responsibility for the HRI scenario. Concrete examples of a Deployer would
include the researcher conducting an HRI experiment or the manager bringing a
robot into the workplace. Interactions between the User and the Robot take
place within the human-human interaction social context (between User and
Deployer), seemingly peripheral to the HRI scenario underway.

Within the Social Triad of HRI, trust as a social construct relates to the the
User’s opinions of the Deployer and potentially of the Robot as an extension,
or realisation, of the Deployer’s intentions. A Robot may behave with apparent
integrity or benevolence, without itself being capable of either. Users may be
wholly deceived, actively participate in the deception by responding as if it
is a social agent, and/or evaluate the Deployer based on the Robot’s
behaviour. Trust as a social construct as experienced by the User towards the
Robot is bound by limits of the User’s trust towards the Deployer.

Figure 1: Interaction pathways recognising the role of the Deployer as both
external to, and influential on, the HRI scenario. While a robot is contained,
the User may enter and exit, the interaction scenario.

###  II-A Interactions in the Triad

The inclusion of a third agent (the Deployer) in HRI adds interaction pathways
and relationships to consider. We briefly outline each of the six interaction
pathways.

1\. HRI is traditionally recognised as just that - the, often reciprocal,
human-robot interaction. A User may influence a Robot’s behaviour through
their own behaviour and communication, either directly controlling or
otherwise affecting the robot.

2\. In return, a Robot may shape User trust through its own behaviour,
including social-like behaviours. Specifically, User trust may be affected by
the robot’s appearance and actions [4] and use of simulated social strategies
to regulate trust [18, 26].

3\. The interaction from Deployer to Robot includes the specification of robot
behaviours (e.g., through Wizard of Oz control; specifying goals, or use of
architectures for generating behaviour) and specifying the contexts for which
the robot may be used, ultimately setting the bounds for a robot’s interaction
with the User [27].

4\. In return, the Robot provides the Deployer information on the interaction,
passively through the behaviour or actively through recorded metrics of the
interaction, potentially of both Robot and User.

5\. Just as the Deployer determines the bounds of a Robot’s behaviour
(interaction 3), their formation of the HRI scenario directs the bounds of
User behaviour. Existing or emerging relationships between Deployer and User
may shape a User’s approach to HRI, as would any communication from the
Deployer (either directly or via robot behaviour).

6\. In return, the User either passively or actively provides feedback to the
Deployer on the scenario (in research, this may be from observation,
questionnaires etc.; in industry, performance appraisals etc.). Where a User
may ostensibly make evaluations of the Robot and HRI scenario, these may
reflect their views towards the Deployer’s behaviours and intentions; i.e.
mistrust towards a Deployer expressed as mistrust towards the Robot.

##  III Co-Creation for Trust

In sum, where Users place their trust in Robots, they indirectly place their
trust in Deployers (cf [24]). Simulated social communication from a Robot to
engender trust indirectly asks the User to trust the Deployer (i.e. a Robot
suggesting its benevolence or it having integrity or showing contrition is a
product of the Deployer intending for the User to think of them both in these
terms). As with deceit not coming from the robot, but as a collaborative
effort between the user and those behind the robot and interaction
circumstances[23], user trust as a social construct is not necessarily towards
the robot but around the robot.

In controlled, laboratory (often university) research, trust towards the
Deployer may almost be taken for granted through solemn assurances that the
User will come to no harm (physical or psychological); the Robot is programmed
to operate appropriately; the specified HRI scenario complies with necessary
safety/ethical regulations; and information gathered from HRI will not be used
carelessly or maliciously. As participants in such research, Users also have
control over engagement with the scenario and power to disengage with the HRI
at will to no detriment. Collectively these factors regarding trust towards
the Deployer could present an artificially high cap on the User’s potential
trust towards the Deployer’s agent: the Robot. Users may be willing to
actively engage with the - as such - deception of social strategies from a
robot affecting trust, even while recognising these are not genuinely the
robot’s.

The above circumstances in laboratory research may offer little ecological
validity, especially in attempts to import to circumstances where there is
little trust towards the Deployer and/or little opportunity for them to
demonstrate trustworthiness (e.g., introducing a social robot to the
workforce). Without a user willing to engage in the deception, a robot’s
apologies or expressions of benevolence could ring hollow and offer no
meaningful indication of the Deployer’s intentions. It becomes incumbent on
the Deployer to authentically obtain trust before a robot can simulate
obtaining trust.

Co-creation as dialogue with the User may be a mechanism by which a Deployer
can themselves earn trust, in turn scaffolding trust as a social construct for
HRI. Where co-creation of specific robotic systems can require extensive
resources and investment, co-creation of the interaction scenarios may
meaningfully allow potential Users to receive assurances and control over
interaction (akin to the above laboratory research) at comparatively low
investment. User control over the types of simulated social behaviours and
communications they would be willing to accept - as such, the deceptions they
are happy to engage in - may raise their footing from individuals with HRI put
upon them to that equal of the Deployer through shared responsibility for the
creation of the scenario.

##  IV Summary

Trust research in HRI typically considers just the interactions between user
and robot. We propose that while this offers insight into the users’
experiences, it omits the important context of relevant human-human
interactions that enable or create the HRI scenario. Examining user trust
towards the individual(s) that a user believes responsible for the HRI
scenario (we have termed Deployer) may further understanding of trust in HRI
and inform effective and appropriate means to build trust. Co-creation as a
method could be a useful means to explore how changes in relationships between
the User and Deployer (namely empowering Users and building trust between User
and Deployer) may shape trust towards robotics in HRI.

## References

  * [1] Weiss, A., Michels, C., Burgmer, P., Mussweiler, T., Ockenfels, A., Hofmann, W., Trust in everyday life. Journal of Personality and Social Psychology 121(1), 95, 2021.
  * [2] McAllister, D.J., Affect-and cognition-based trust as foundations for interpersonal cooperation in organizations. Academy of management journal 38(1), 24–59, 1995.
  * [3] Fiske, S.T., Cuddy, A.J., Glick, P., Universal dimensions of social cognition: Warmth and competence. Trends in cognitive sciences 11(2), 77–83, 2007.
  * [4] Hancock, P.A., Billings, D.R., Schaefer, K.E., Chen, J.Y., De Visser, E.J., Parasuraman, R., A meta-analysis of factors affecting trust in human-robot interaction. Human factors 53(5), 517–527, 2011.
  * [5] Schaefer, K.E., Chen, J.Y., Szalma, J.L., Hancock, P.A., A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human factors 58(3), 377–400, 2016.
  * [6] Hancock, P., Kessler, T.T., Kaplan, A.D., Brill, J.C., Szalma, J.L.: Evolving trust in robots, specification through sequential and comparative meta-analyses. Human factors 63(7), 1196–1229, 2021.
  * [7] Ruff, H.A., Narayanan, S., Draper, M.H., Human interaction with levels of automation and decision-aid fidelity in the supervisory control of multiple simulated unmanned air vehicles. Presence: Teleoperators & Virtual Environments 11(4), 335–351, 2002.
  * [8] de Visser, E., Parasuraman, R., Adaptive aiding of human-robot teaming, Effects of imperfect automation on performance, trust, and workload. Journal of Cognitive Engineering and Decision Making 5(2), 209–231, 2011.
  * [9] Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A., Yanco, H., Impact of robot failures and feedback on real-time trust. In: 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 251–258, 2013.
  * [10] Salem, M., Lakatos, G., Amirabdollahian, F., Dautenhahn, K., Would you trust a (faulty) robot? effects of error, task type and personality on human-robot cooperation and trust. In: 2015 10th ACM/IEEE International Con- ference on Human-Robot Interaction (HRI), pp. 1–8, 2015.
  * [11] Hancock, P.A., Billings, D.R., Schaefer, K.E., Can you trust your robot? Ergonomics in Design 19(3), 24–29, 2011.
  * [12] Sheridan, T.B., A review of recent research in social robotics. Current opinion in psychology 36, 7–12, 2020.
  * [13] Law, T., Scheutz, M., Trust: Recent concepts and evaluations in human-robot interaction. In: Nam, C.S., Lyons, J.B. (eds.) Trust in Human-Robot Interaction, pp. 27–57. Academic Press, Massachusetts, 2021.
  * [14] Lee, S.A., Liang, Y.J., Robotic foot-in-the-door: Using sequential-request persuasive strategies in human-robot interaction. Computers in Human Behavior 90, 351-356, 2019.
  * [15] Salem, M., Eyssel, F., Rohlfing, K., Kopp, S., Joublin, F., To err is human (-like):, Effects of robot gesture on perceived anthropomorphism and likability. International Journal of Social Robotics 5(3), 313-323, 2013.
  * [16] Kaniarasu, P., Steinfeld, A.M., Effects of blame on trust in human robot interaction. In The 23rd IEEE international symposium on robot and human interactive communication, pp. 850-855, 2014.
  * [17] Chita-Tegmark, M., Law, T., Rabb, N., Scheutz, M., Can you trust your trust measure? In: Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, pp. 92-100, 2021.
  * [18] Cameron, D., de Saille, S., Collins, E.C., Aitken, J.M., Cheung, H., Chua, A., Loh, E.J., Law, J.; The effect of social-cognitive recovery strategies on likability, capability and trust in social robots. Computers in Human Behavior 114, 106561, 2021.
  * [19] Kox, E.S., Siegling, L.B., Kerstholt, J.H., Trust development in military and civilian human-agent teams: The effect of social-cognitive recovery strategies. International Journal of Social Robotics, 2022.
  * [20] Robinette, P., Howard, A.M., Wagner, A.R., Timing is key for robot trust repair. In: International Conference on Social Robotics, pp. 574-583, 2015.
  * [21] Malle, B.F., Ullman, D., A multidimensional conception and measure of human-robot trust. In: Trust in Human-Robot Interaction, pp. 3-25. Academic Press, Massachusetts, 2021.
  * [22] Collins, E.C., Vulnerable users: deceptive robotics. Connection Science, 29, 223-229, 2017.
  * [23] Sharkey, A., Sharkey, N. We need to talk about deception in social robotics!. Ethics and Information Technology, 23, 309-316, 2021.
  * [24] Clark, H.H., Fisher, K., Social robots as depictions of social agents. Behavioral and Brain Sciences, 33, 2022.
  * [25] Cameron, D., Collins, E.C., User, robot, deployer: A new model for measuring trust in HRI, International Conference on Robot-Human Interactive Communication (SCRITA Workshop), 2021.
  * [26] Geiskkovitch D.Y and Young, J.E., Children’s overtrust: Intentional use of robot errors to decrease trust, in 29th International Conference on Robot-Human Interactive Communication (SCRITA Workshop), 2020.
  * [27] Cameron, D., Aitken, J., Collins, E., Boorman, L., Chua, A., Fernando, S., McAree, O., Martinez Hernandez, U., Law, J., Framing factors: The importance of context and the individual in understanding trust in human-robot interaction,” in IROS 2015: Workshop on designing and evaluating social robots for public settings, 2015.


[a]Sergio Chaves García-Mascaraque

# Meson thermal masses at different temperatures

Gert Aarts  Chris Allton  Timothy Burns  Simon Hands  Benjamin Jäger

###### Abstract

We determine the ground state meson masses at low temperature using
simulations with $N_{f}=2+1$ flavours of improved Wilson-clover fermions.
Subsequently we study the effect of increasing the temperature of the hadron
gas, including the transition to the quark-gluon plasma, as well as the
restoration of SU(2)A chiral symmetry. We use the FASTSUM anisotropic, fixed-
scale Generation 2L ensembles and consider mesons with light, strange and
charm content.

##  1 Introduction

The fate of hadrons as the temperature of the hadron gas is increased and the
transition to the quark-gluon plasma is made, has been a topic of longstanding
interest. In principle, nonperturbative simulations of lattice QCD should
provide theoretical insight, but the analysis is complicated due to the
difficulty in defining the notions of ground state and mass at nonzero
temperature. This can be resolved by considering hadronic spectral functions,
which include thermal mass shifts and widths, but their construction is
hindered by the finite number of points in the Euclidean time direction
available for the required analytic continuation. Nevertheless, interesting
results for light and strange baryons have been obtained [1, 2], shedding
light on parity doubling and chiral symmetry restoration.

In this contribution, we present the temperature dependence of masses of
mesons. We take the following conservative approach: we first determine the
masses of the ground states in various channels at low temperature, using a
regression analysis based on the one presented in Ref. [3]. We aim to minimise
possible bias by performing variational fits and avoiding cherry-picking.
Subsequently we extend this analysis to nonzero temperature and attempt to
systematically generate comparable estimates of ground state masses for all
temperatures. While we expect this approach to break down at higher
temperatures and definitely in the quark-gluon plasma phase, where light
mesons no longer exist, we demonstrate that it nevertheless provides insight
into thermal effects. In particular, we study the restoration of the SU(2)A
symmetry as the temperature increases, by comparing the masses of the
$\rho(770)$ and $a_{1}(1260)$. Finally, we identify limits to this approach by
contrasting results obtained with local and smeared sources at high
temperature.

##  2 Lattice setup and mesonic correlation functions

We use the anisotropic FASTSUM ensembles described in detail in Ref. [5], with
$N_{f}=2+1$ flavours of clover-improved Wilson fermions. The strange quark
mass is at its physical value [4], but the light quarks are heavier than in
nature. Relevant parameters are given in Table 1. In the fixed-scale approach,
the temperature is varied by changing $N_{\tau}$, using the relation
$T=1/(N_{\tau}a_{\tau})$.The lowest (highest) temperature we study here is
$T=47$ ($300$) MeV, obtained using $N_{\tau}=128$ ($20$).

$1/a_{\tau}$ [GeV] |  $a_{s}$ [fm] | $\xi=a_{s}/a_{\tau}$ | $N_{s}$ |  $M_{\pi}$ [MeV] | $M_{\pi}L$ |  $T_{c}$ [MeV]
---|---|---|---|---|---|---
$5.997(34)$ | $0.01136(6)$ | $3.453(6)$ | $32$ | $236(2)$ | $4.36$ | $164(2)$
Table 1:  Parameters relevant for the ensembles: $a_{\tau}$ ($a_{s}$) is the
temporal (spatial) lattice spacing; $\xi$ is the renormalised anisotropy;
$N_{s}$ is the number of points in the spatial direction; $M_{\pi}$ is the
mass of the pion; $T_{c}$ is the critical temperature, computed using the
inflection point of the renormalised chiral condensate [5].

Our analysis focuses on mesonic correlation functions. The mesonic operators
are $O(a)$ improved through the Symanzik improvement scheme. In the
correlation functions the source and sink operators are identical. As we are
interested in thermal observables, correlation functions are expressed in a
time-momentum representation, with the external momentum set to zero.
Consequently, our correlators only depend on Euclidean lattice time $\tau$,
$C(\tau)$ with $\tau/a_{\tau}\in[0,N_{\tau})$. In the simulations, three quark
fields are present: $u$, $d$ and $s$, with $u$ and $d$ being degenerate. We
also have access to observables containing charm ($c$), which is however not
included in the fermion determinant. As a result, we can simulate six flavour
combinations: $uu$, $us$, $uc$, $ss$, $sc$ and $cc$. We do not calculate
disconnected contributions, therefore, we only have access to non-singlet
flavour observables.

Channel | pseudoscalar | vector | axial-vector | scalar
---|---|---|---|---
Operator | $\gamma_{5}$ | $\gamma_{\mu}$ | $\gamma_{\mu}\gamma_{5}$ | $1\\!\\!1$
$J^{PC}$ | $0^{-+}$ | $1^{--}$ | $1^{++}$ | $0^{++}$
Table 2:  Channels available in our simulations, indicated by the Dirac matrix
and the $J^{PC}$ quantum number.

We have analysed a number of channels: pseudoscalar, vector, axial vector and
scalar, see Table 2. Here we report results for the first three channels only.
Besides the specification of the flavour combination and operator used, we can
choose the type of source used in the inversion of the Dirac operator. We have
used local and smeared (using Gaussian smearing with parameters
$N=100,\kappa=5.5$) sources. The latter are designed to have better overlap
with the ground state at zero temperature, but in principle both types should
lead to the same ground state mass. We construct correlation functions in
which either local sources were used for both quark propagators, and denote
these with $l$-$l$, or smeared sources, denoted with $s$-$s$. As a result, we
have two different estimates for the same mesonic correlator, for each channel
and flavour combination.

##  3 Regression analysis

###  3.1 Spectral decomposition

Every parametric regression analysis needs a model. Here we start from the
simplest Ansatz: a sum of isolated states, characterised by a mass $M_{s}$,
amplitude $A_{s}$ and vanishing width (i.e. simple poles in the mesonic
correlator, or delta-functions in the corresponding spectral function). The
correlator then takes the form

$C(\tau)=\sum_{s=0}^{\infty}A_{s}\cosh\left[M_{s}(\tau-1/(2T)\right].$ (1)

In principle, we can fit our estimates of the correlation function to the
equation above to extract the masses $M_{s}$ and amplitudes $A_{s}$ of the
different states. We refer to the state with the lowest mass as the ground
state; extracting its mass is our main objective. Provided $M_{0}\ll M_{s\neq
0}$, the ground state dominates at large times $\tau>0$, which at nonzero
temperature is $\tau/a_{\tau}\simeq N_{\tau}/2$. It is noted that Eq. (1) is
only expected to be valid in the low-temperature limit, $T\to 0$. Once the
temperature is increased, the validity of the model starts being questionable,
due to in-medium effects.

To perform a regression analysis, we assume that the correlation functions
are, at all temperatures, described by the model

$C(\tau)=f(\tau;\theta,N_{\rm tr})+u,$ (2)

where the Ansatz, $f$, is a truncation of Eq. (1) at the order $N_{\rm tr}$.
It depends on the parameters $\theta=\\{A_{s},M_{s}|s=0,\ldots,N_{\rm tr}\\}$,
which need to be found; the parameters $A_{s}$ and $M_{s}$ correspond to the
ones in Eq. (1). The error term, $u$, is assumed normally distributed and
conditionally independent of $\tau$.

To set up notation, from now on $\hat{C}(\tau)$ is our estimate of the
correlation function, which is estimated by averaging over all configurations
at a fixed temperature. Note that the regression analysis presented in this
section is based on the procedure presented in Ref. [3].

###  3.2 Sources of problems in a regression analysis

Performing a regression analysis is a nontrivial task. Here we briefly review
some of the problems present in the analysis and discuss solutions to mitigate
them.

Correlation function data is heavily correlated – The correlation function
estimate at different times $\tau$ is heavily correlated, hence the data
includes less information than expected. The source of this autocorrelation is
the computation of the propagators; the entire ensemble is used to produce an
estimate of the correlation function at all $\tau$. Consequently, the samples
are correlated at different $\tau$, which violates the usual assumption of
independence in the errors in Eq. (2).

Not taking into account the correlation in the regression analysis tends to
underestimate the uncertainty in the parameter estimates and even produce
wrong results. One way to take correlation into account is by using the so-
called correlated maximum likelihood estimate. This estimate can be derived by
maximising the likelihood function of the intersection of $N$ correlated
normally distributed random variables. The problem is equivalent to minimising
the following objective function:

$\chi^{2}=\left[\hat{C}-f(\tau;\theta)\right]K^{-1}\left[\hat{C}-f(\tau;\theta)\right]^{T},$
(3)

$K$ being the covariance matrix of the error term, which can be estimated
using the correlation function estimate $\hat{C}(\tau)$. Minimising Eq. (3)
tends to be unstable; starting at different initial parameters leads to
significantly dissimilar results.

One can always try to break down the correlation by producing a bootstrap
estimate of the data. This can be done by selecting different random samples
from all configurations available for each time $\tau$. This procedure can
produce biased estimates of $\hat{C}(\tau)$ [Efron:1986hys].

Multistate fits are a must – As explained before, the lowest state tends to
dominate the correlation function at large $\tau$. However, without having
information about the values of the masses of the excited states, we do not
know when those states start becoming relevant as $\tau\searrow 0$. As a
consequence, if only one-state fits are produced, then bias is included in the
result. Hence one should always try including as many states as possible in
the model.

Varying the fit windows – The fit window used in the regression should be
varied to produce as many estimates as possible of the ground state mass.
Varying the fit window reduces the bias included by manually selecting a
region of fit. For the remainder of the contribution, we define a fit window
$FW(\tau_{0},\tau_{f})$ as all times $\tau$ included in the interval
$[\tau_{0},\tau_{f}]$. We fix $\tau_{f}/a_{\tau}=N_{\tau}/2$.

Parameter initialisation – Minimising Eq. (3) using initial parameters that
are close to the “true” values is crucial to avoid instabilities. We produce
our initial parameters using a combination of effective mass computations and
fits to a different number of states. The mass of the $n^{\rm th}$ state can
be estimated using the effective mass if knowledge about the parameters of the
lighter $n-1$ states is available. To do so, we subtract the model composed of
the $n-1$ states from the correlation function estimate, eliminating those
contributions from the correlation function; see Eq. (1). We can use this
procedure iteratively to estimate the masses of the wanted excited states.
Afterwards, one should always fit a one-state model to the subtracted
correlation function to produce a more robust estimate of the initial
parameter. As errors propagate, this technique is less reliable for higher
order excited states.

Figure 1:  Ground state mass of the local-local pion correlation function
($\gamma_{5},uu$) as a function of the starting time $\tau_{0}$ defining the
fit window used in the regression; the fit windows used contain all points in
the region $[\tau_{0},N_{\tau}/2]$. The error bars correspond to the estimate
of the ground state mass at a given time window. In some cases the error bars
are too small to be visible. Error bars change rapidly in regions where
additional excited states become relevant. The line corresponds to the median
mass using all fit windows; this result is independent of the fit window used.
The temperature of the system is $T=46$ MeV ($N_{\tau}=128$).

###  3.3 Regression at fixed fit window and extraction of the final ground
state mass

We will now explain how regression is carried out using a fixed fit window. As
we do not know when excited states become relevant, we should include models
with a different number of states. For each of the models, we can produce an
estimate of the ground state mass. At the end, we need to select the best
estimate of the ground state mass at the current fit window. To obtain this
value, we use the so-called corrected Akaike Information Criterion (AICc) [7,
8]. The $AICc$ can be used to measure the relative likelihood of the data
description between two models; the model with the lowest $AICc$ is the most
likely to describe the data. It can be defined using

$AICc=N_{\theta}-\log(\hat{L})+\frac{N_{\theta}^{2}+N_{\theta}}{N_{\rm
fw}-N_{\theta}-1},$ (4)

where $N_{\theta}$ is the number of parameters in the model, $\hat{L}$ is the
likelihood function evaluated at the estimated parameters and $N_{\rm fw}$ is
the number of points included in the fit.

Using the $AICc$, we can compute the relative likelihood between two models
$m_{1}$ and $m_{2}$,

$l(m_{1},m_{2})=\exp\left(-\frac{1}{2}[AICc(m_{1})-AICc(m_{2})]\right).$ (5)

This quantity measures how likely $m_{2}$ is to describe the data compared to
$m_{1}$. Provided we set $m_{1}$ to the model with the lowest $AICc$ among all
models available, then we can use the values of $l$ to measure the relative
model quality. We can utilise this information to compute the best estimate of
the mass at the given time window by calculating the weighted average of all
masses; we use $l$ as weights. This technique allows us to promote the
influence of high-quality models in our result while avoiding manually
discarding any models. Finally, the error in the weighted mass can be
extracted using a bootstrap analysis.

For each fit window used in the fit, we collect an estimate of the ground mass
$M_{0}(FW)$. Our final estimate of the mass, independent of the fit window, is
extracted using the median of all $M_{0}(FW)$. We use the median as it is a
robust statistic; outliers tend to be present due to the unstable nature of
correlated fits. Confidence intervals on the median estimation can be
constructed using bootstrap; confidence intervals are not guaranteed to be
symmetric.

##  4 Results

Pseudoscalar channel

Vector channel

Figure 2:  Temperature dependence of the ground state masses in the
pseudoscalar ($\gamma_{5}$) and the vector ($\gamma_{\mu}$) channels. The
vertical gray line corresponds to the pseudocritical temperature $T_{c}$.

We will now present some results for the ground state masses for different
states. Fig. 2 contains the temperature dependence of the different flavour
combinations in the pseudoscalar and vector channels. Two different trends can
be seen. In the low-temperature regime, where the system is in the hadronic
phase, the masses show minimal temperature dependence. For the light quarks,
some dependence can be seen in the vector channel. On the other hand, above
the pseudocritical temperature ($T_{c}\sim 164$ MeV), which is estimated using
the inflection point of the chiral condensate, where the system is a quark-
gluon plasma, the masses vary with the temperature, when the results are taken
at face value. This effect is stronger in the light sector ($uu$, $us$ and
$ss$). For the heavier sector, involving charm, the effect is milder and the
charmonium $cc$ states are almost unaffected by thermal effects. In the high-
temperature regime, the masses of the light mesons increase until becoming
degenerate. The mass increase and the degeneration of the light flavour
combinations can be explained by thermal effects: the inherent light-quark-
energy scales are smaller than the temperature scale and the system is
dominated by thermal excitations. Note that the uncertainties in the estimates
grow with temperature due to the combined fact that the model used in the fit
is not expected to be accurate as the temperature increases (to be discussed
next) and the fact that the number of points available in the fit decreases
with $T$.

The apparent presence of thermal effects for light mesons in the high-
temperature regime leads to a natural question: is the spectral decomposition
underpinning Eq. (1) valid at high $T$? The answer is clearly no: at high
temperature, thermal effects, collective excitations and screening are
expected to impact mesonic correlators. Light meson states will disappear
(deconfinement) and although states involving charm may survive in the quark-
gluon plasma, they are expected to be no longer describable by delta
functions/simple poles. The inability of Eq. (1) to represent the high-
temperature regime generates a problem: the quantities extracted in this
region cannot be interpreted as masses. This problem is more severe in the
light sector, where thermal effects are more dominant. Nevertheless, the
results contain meaningful information about the status of the system in the
sense that degeneracies and symmetry restoration can be studied from the
outcome of the regression analysis, as it gives information on the underlying
correlators, irrespective of the interpretation as ground state mass.

Figure 3:  Temperature dependence of two $SU(2)_{A}$ related channels: vector
– $\rho(770)$ – and axial-vector – $a_{1}(1260)$. The estimate of the
correlation function in the axial-vector channel at the lowest temperature is
remarkably noisy, affecting the confidence intervals.

We demonstrate this in more detail in the context of SU(2)A chiral symmetry
restoration and the temperature dependence of states related by this symmetry.
Here we discuss the vector and axial-vector channels in the lightest flavour
combination, connected to physical states $\rho(770)$ and $a_{1}(1260)$
respectively, see Fig. 3. At low temperature, the states are non-degenerate.
However, at higher temperature, they are expected to become degenerate as the
SU(2)A symmetry is restored. The results of our simulations in Fig. 3 appear
to confirm this expectation, with the degeneracy emerging at or near the
transition temperature.

Finally, in Fig. 4 we present some results comparing the masses extracted
using different types of sources: local and smeared ones. Smearing is designed
to decouple the excited states in a correlation function. As a consequence,
the ground state signal is enhanced, thus leading to a better estimate of the
ground state mass. Where the ground state is well-defined, local and smeared
ground state mass estimates should be equal. In the hadronic phase, our
results indeed are comparable. However, in the deconfined phase the estimates
are no longer equal. The implication is that the single-pole Ansatz given in
Eq. (1) is not valid and the ground state is no longer present. The larger
errors at high temperature are another indication the fits are no longer
working.

Pseudoscalar channel

Vector channel

Figure 4:  Temperature dependence of the difference between the ground state
masses extracted using local and smeared sources, $M_{0}^{l-l}-M_{0}^{s-s}$,
in the pseudoscalar ($\gamma_{5}$) and vector ($\gamma_{\mu}$) channels for
all flavour combinations.

##  5 Conclusions

The regression analysis presented allows the estimation of ground state masses
in a systematic manner. However, as stated before, these masses are not
reliable at all temperatures. In the hadronic phase, the Ansatz consisting of
a sum of simple poles, see Eq. (1), is sufficient to obtain reliable
estimates. A minimal temperature dependence of the masses in this regime is
observed. In contrast, as the temperature increases, thermal effects dominate
and Eq. (1) is no longer valid. More complex models and algorithms would be
needed to fully capture the information present in the correlation functions
at high temperature, reconstructing e.g. the spectral functions directly.
Although the approach used here is not completely accurate, pushing the
boundaries of this established method is not useless. Information about
degeneracies and symmetry restoration at the level of the correlation
functions can be extracted using simple regression, and those results can be
used as a firm ground on which more complex analysis tools are built.

A remarkable finding of our analysis is the degeneracy of SU(2)A related
states. Our results confirm the non-degeneracy of the $\rho(770)$ and the
$a_{1}(1260)$ states in the hadronic phase. As the temperature increases, the
states get closer and become degenerate at or around the transition
temperature to the quark-gluon plasma and remain degenerate at higher
temperatures.

As a final important exercise, we compared correlators obtained using local
and smeared sources. An agreement between the extracted ground state masses
provides an important boost for the validity of the approach. This is observed
in the hadronic phase. On the other hand, likely reasons for a discrepancy at
high temperature are that the ground state is no longer given by a simple pole
or no longer clearly discernible, due to thermal effects and/or deconfinement.
This was seen in the quark-gluon plasma, in all flavour combinations and
channels studied. This leaves the question of the impact of smearing at high
temperature; further research on the effects of smearing on high-temperature
observables is therefore needed.

Acknowledgements – This work is supported by the UKRI Science and Technology
Facilities Council (STFC) Consolidated Grant No. ST/P00055X/1 and
ST/T000813/1. We are grateful to DiRAC, PRACE and Supercomputing Wales for the
use of their computing resources and to the Swansea Academy for Advanced
Computing for support. This work was performed using the PRACE Marconi-KNL
resources hosted by CINECA, Italy and the DiRAC Extreme Scaling service and
Blue Gene Q Shared Petaflop system at the University of Edinburgh operated by
the Edinburgh Parallel Computing Centre. The DiRAC equipment is part of the
UK’s National e-Infrastructure and was funded by UK’s BIS National
e-infrastructure capital grant ST/K000411/1, STFC capital grants ST/H008845/1
and ST/R00238X/1, and STFC DiRAC Operations grants ST/K005804/1, ST/K005790/1
and ST/R001006/1.

## References

  * [1] G. Aarts, C. Allton, D. De Boni and B. Jäger, Phys. Rev. D 99 (2019) no.7, 074503 [arXiv:1812.07393 [hep-lat]].
  * [2] G. Aarts, C. Allton, D. De Boni, S. Hands, B. Jäger, C. Praki and J. I. Skullerud, JHEP 06 (2017), 034 [arXiv:1703.09246 [hep-lat]].
  * [3] A. Bazavov, S. Dentinger, H. T. Ding, P. Hegde, O. Kaczmarek, F. Karsch, E. Laermann, A. Lahiri, S. Mukherjee and H. Ohno, et al. Phys. Rev. D 100 (2019) no.9, 094510 [arXiv:1908.09552 [hep-lat]].
  * [4] R. G. Edwards, B. Joo and H. W. Lin, Phys. Rev. D 78 (2008), 054501 [arXiv:0803.3960 [hep-lat]].
  * [5] G. Aarts, C. Allton, J. Glesaaen, S. Hands, B. Jäger, S. Kim, M. P. Lombardo, A. A. Nikolaev, S. M. Ryan and J. I. Skullerud, et al. [arXiv:2007.04188 [hep-lat]].
  * [6] Efron, B. & Tibshirani, R. J. (1994). An introduction to the bootstrap. CRC press.
  * [7] Akaike, H. (1974). A new look at the statistical model identification. IEEE transactions on automatic control, 19(6), 716-723.
  * [8] Akaike, H. (1998). Information theory and an extension of the maximum likelihood principle. In Selected papers of Hirotugu Akaike (pp. 199-213). Springer, New York, NY.