# An In-depth Investigation of User Response Simulation for Conversational
Search

Zhenduo Wang<EMAIL_ADDRESS>1234-5678-9012 University of Utah ,
Zhichao Xu<EMAIL_ADDRESS>University of Utah ,  Qingyao Ai
<EMAIL_ADDRESS>Tsinghua University and  Vivek Srikumar
<EMAIL_ADDRESS>University of Utah

###### Abstract.

Conversational search has seen increased recent attention in both the IR and
NLP communities. It seeks to clarify and solve a user’s search need through
multi-turn natural language interactions. However, most existing systems are
trained and demonstrated with recorded or artificial conversation logs.
Eventually, conversational search systems should be trained, evaluated, and
deployed in an open-ended setting with unseen conversation trajectories. A key
challenge is that training and evaluating such systems both require a human-
in-the-loop, which is expensive and does not scale. One strategy for this is
to simulate users, thereby reducing the scaling costs. However, current user
simulators are either limited to only respond to yes-no questions from the
conversational search system, or unable to produce high quality responses in
general.

In this paper, we show that current state-of-the-art user simulation system
could be significantly improved by replacing it with a smaller but advanced
natural language generation model. But rather than merely reporting this new
state-of-the-art, we present an in-depth investigation of the task of
simulating user response for conversational search. Our goal is to supplement
existing works with an insightful hand-analysis of what challenges are still
unsolved by the advanced model, as well as to propose our solutions for them.
The challenges we identified include (1) dataset noise, (2) a blind spot that
is difficult for existing models to learn, and (3) a specific type of
misevaluation in the standard empirical setup. Except for the dataset noise
issue, we propose solutions to cover the training blind spot and to avoid the
misevaluation. Our proposed solutions lead to further improvements. Our best
system improves the previous state-of-the-art by 10.53 BLEU-4, 13.4 ROUGE, and
14.59 METEOR on Qulac dataset, and 10.4 BLEU-4, 13.29 ROUGE, 14.49 METEOR on
ClariQ dataset.

conversational search, user response simulation, natural language generation

††ccs: Information systems Users and interactive retrieval

##  1\. Introduction

A study by Spink et al. (Spink et al., 2001) suggested that almost 60% of web
search queries have fewer than three words. Conventional search systems
usually perform a single-turn result retrieval based on a potentially
ambiguous user query, implicitly placing the burden on users to go through the
entire search result page to hopefully get the information they need. User
behavior analysis and advanced natural language processing models have made it
easier to clarify and iterate over complex user needs through conversations.
This has led to the development of interactive information-seeking systems ,
usually referred to as _Conversational Search systems_ (Zamani et al., 2022):
an increasingly popular research topic and an essential frontier of IR (Anand
et al., 2021; Culpepper et al., 2018).

Most research about conversational search (Aliannejadi et al., 2019, 2020;
Wang and Ai, 2022; Sekulić et al., 2022; Bi et al., 2021; Sekulić et al.,
2021; Zamani et al., 2020a) is limited to training their system on datasets
with observed or artificial conversation logs. Such a dataset would lack
training signal and evaluation references when a conversation veers away from
the dataset; especially, when the system generates a question not listed in
the dataset and steers the conversation in an unseen direction. We refer to
this as the open-ended nature of a multi-turn conversational system, as
opposed to training and evaluating the system with the recorded conversation
trajectories from a dataset. Eventually, conversational search systems should
be trained, evaluated, and deployed in an open-ended setting.

However, training and evaluating them in such a setting is challenging; it
requires humans to generate responses to open clarifying questions, which is
expensive and does not scale.

Past work (e.g., Salle et al., 2021; Sekulić et al., 2022) has demonstrated
that a user response simulator that automatically generates human responses
can help evaluate conversational search systems. The goal of a such a system
is to generate user-like answers to system-generated clarifying questions
based on a query and the user’s search intent. A user response simulation
system can also enable studies such as training a multi-turn conversational
search system, by generating synthetic conversations and rewards, and perhaps
using Reinforcement Learning from Human Feedback (RLHF) (Ouyang et al., 2022).

The primary goal of this paper is to analyze and provide insights into the
task of user response simulation for conversational search, focusing on the
design and challenges with existing models and how the challenges may be
addressed, and identifying what is left to be solved. As far as we know, we
are the first to suggest that the task of user response simulation for
conversational search generally resembles the task of question answering,
while also having distinguishing characteristics. Our experiments show that
question answering datasets can provide transferrable knowledge for training a
user response simulation system.

We also conduct a manual analysis on a subset of a widely-used conversational
search dataset Qulac (Aliannejadi et al., 2019). We study all the low-scoring
cases where a baseline model struggles. From the analysis, we conclude a
categorization of low-scoring cases of its failings, which is shown in detail
and with examples in Section 4. Our analysis suggests that, among all the low-
scoring cases for our baseline model: (1) 10% of them contain out-of-context
information either in the question or the reference response and therefore are
extremely hard to be simulated by any system. (2) 38% generated responses are
bad, mainly due to the baseline sequence-to-sequence model’s inability to
understand the task, which is shared by the existing state-of-the-art system.
(3) at least 45% of the generations are reasonable but are mis-evaluated by
the imperfect evaluation metrics.

Further, we demonstrate a simple two-step generation system that aims to
address a subset of the bad responses to improve the generations in the
abovementioned problem (2). In addition, we notice that a significant portion
of the mis-evaluations shares the same reason that is unique to the user
response simulation task: the current standard for evaluation does not account
for user cooperativeness. We propose a data partition and new evaluation
heuristic to account for user cooperativeness as solution to the
abovementioned problem (3).

Finally, we conduct experiments comparing our proposed systems and heuristic
with existing approaches. Evaluating generations from all the systems are done
from three perspectives. The first is automatic evaluation metrics, such as
BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), and METEOR (Banerjee and
Lavie, 2005), as well as a classification f1-score. Next, we employ crowd
workers to score the generated responses in terms of generation relevance and
naturalness. The last is from the document retrieval performance perspective,
following (Aliannejadi et al., 2019; Salle et al., 2021; Sekulić et al.,
2022). We compare the retrieval performances of adding the clarifying question
with generated response as additional input with a retrieval baseline using
only the initial query.

Our experiment results show that our best-proposed system improves previously
published state-of-the-art by 10.53 BLEU-4, 13.4 ROUGE, 14.59 METEOR on the
Qulac dataset, and 10.4 BLEU-4, 13.29 ROUGE, 14.49 METEOR on the ClariQ
dataset. Evaluations from human judgement confirm that our proposed system
significantly improves over a strong T5 baseline. Further, for document
retrieval, all our proposed systems significantly improve previous state-of-
the-art and are on par with oracle generation utility.

##  2\. Related Work

This section gives a high-level overview of previous work from two related
fields: conversational search and user response simulation. Other related work
is introduced in their appropriate context.

#### Conversational Search

Conversational search is a novel search paradigm that aims to clarify and
solve users’ complex search needs through natural language conversations
(Zamani et al., 2022). Recent work (e.g., Anand et al., 2021; Culpepper et
al., 2018) has identified it as one of the research frontiers of IR, and it
has been the focus of a large volume of seminars and surveys (e.g., Anand et
al., 2021; Hauff et al., 2021; Fu et al., 2020; Gao et al., 2018; Zamani et
al., 2022; Gao et al., 2022). The most desired feature of conversational
search is that both the user and the system can take the conversational
initiative as suggested in the theoretical framework by Radlinski and Craswell
(Radlinski and Craswell, 2017). Zamani and Craswell (Zamani and Craswell,
2020) later proposed a conceptual pipeline of such a mixed-initiative
conversational search system. In their and most existing conversational search
systems (Rao and Daumé III, 2019; Zamani et al., 2020b; Aliannejadi et al.,
2019), system-initiative is implemented as proactively ask clarifying
questions about the search query. Evaluating these systems and scaling their
functions to multi-turn systems require an actual human to provide feedback
for their clarifying questions. Because human-in-the-loop is extremely
expensive and does not scale, evaluation and scaling remain challenging for
conversational search systems.

#### User Response Simulation for conversational systems

User Simulation for conversational systems has been broadly studied by NLP and
IR communities in the past (Eckert et al., 1997; Komatani et al., 2005;
Schatzmann et al., 2006; Scheffler and Young, 2000). One of the earlier user
simulation methods is agenda-based simulation (Schatzmann et al., 2007), where
users are assumed to generate responses around a specific dialogue objective.
It has been shown to be effective for various close-domain tasks such as task-
oriented dialogue systems and conversational recommendation. In these tasks, a
simple set of rules can usually lead to highly realistic systems (Hou et al.,
2019; Li et al., 2016; Zhang and Balog, 2020). Some recent works (Chandramohan
et al., 2011; Chen et al., 2019; Zhao et al., 2019) use deep learning and
reinforcement learning to learn user simulation from data.

User simulation for open-domain conversational information retrieval is
relatively under-explored (Salle et al., 2021; Sekulić et al., 2022). Salle et
al. (Salle et al., 2021) demonstrated a multi-turn search intent clarification
process of conversational search with a clarifying question selection model
and a user response generation model named CoSearcher. In their system, the
user simulator is initialized with one search intent. It needs to respond to
clarifying questions in the form of whether the search intent is a guessed
intent from the clarifying question selection model. Their model also
considers various user parameters such as the user’s patience for engaging in
the conversation and cooperativeness for supplying the user response with more
information about the search intent. However, their system is limited to only
responding to ‘yes-no’ questions and selects a response from the dataset
instead of generating a response by itself. In this work, we propose to answer
all types of clarifying questions; Section 3.2 presents our categorization of
clarifying question types.

Sekulić et al. (Sekulić et al., 2022) proposed a generative user simulation
model named USi, which can generate responses to any clarifying question for
open-domain conversational search. Their model—based on the GPT-2 (Radford et
al., 2019) model—is shown to have human-like performance. They also explore
user simulation in a multi-turn setting with conversational history as part of
their input but fail to evaluate their multi-turn variant quantitatively due
to data limitations.

In this paper, we show that GPT-2 is a structurally suboptimal choice for
modeling user responses. We also show that the similarity of user response
simulation to question answering (QA) can improve the former with knowledge
learned from QA. Further, we conduct an in-depth investigation on the low-
scoring cases for an existing baseline, propose solutions, and provide
insights into the challenges of simulating user responses for conversational
search.

##  3\. User Response Simulation Task

This section defines the user response simulation task for conversational
search and briefly introduces the datasets we use.

###  3.1. Task Definition

A conversational search session starts with the user issuing a potentially
ambiguous search query to the search system. For example, a user may be
looking for an anti-spyware program called Defender and may type “Tell me
about defender” in the search system. The word ‘defender’ is ambiguous: it can
also refer to other concepts, such as a TV series, a vehicle model named, or a
video game. The conversational search system may want to clarify whether the
user is looking for the TV series by “Are you interested in a television
series?” A good human response could be “No.”, or “No, I am looking for a
software.”

Following USi (Sekulić et al., 2022), our user response simulation task aims
to train a system to generate responses as above. Formally, given the user
search query $q$, search intent $i$, and clarifying question $cq$, a user
response simulation system should generate an answer $a$ in natural language
that is consistent with the user’s search intent $i$. Specifically, in the
above example,

$i=$ “I am looking for an anti-spyware program called Defender.” $q=$ “Tell me
about defender” $cq=$ “Are you interested in a television series?” $a=$ “No, I
am looking for a software.”

Some previous work approached the task of user response simulation for
conversational search differently. Recall from Section 2 that CoSearcher
(Salle et al., 2021) builds a system to respond to yes-no questions, while we
propose to answer all types of clarifying questions.

###  3.2. Datasets

We use two publicly available datasets in this work: Qulac (Aliannejadi et
al., 2019) and ClariQ (Aliannejadi et al., 2020). Qulac is built based on
faceted queries from TREC Web Track (Clueweb) 09-12 (Clarke et al., 2009). It
contains rows of query, facet, clarifying question, and answer representing
one turn of conversational search, where the clarifying questions and answers
are crowd-sourced. The dataset can be seen as tree-structured, where each
faceted query can have multiple facets and multiple reasonable clarifying
questions. We use the Qulac training set for finetuning all our systems, the
development set for studying the problem, and the test set for evaluation.
ClariQ extends Qulac with additional queries and facets and creates a new test
set with topics not included in Clueweb09-12. However, because the indexed
documents for retrieval experiments are not publicly available along with the
dataset, we choose to use the development set of ClariQ for evaluation. ClariQ
has a small multi-turn set with five hundred conversations. Previous user
simulation works, such as CoSearcher (Salle et al., 2021) and USi (Sekulić et
al., 2022), have used both datasets.

#### Clarifying Types in Datasets

As we previously mentioned, the CoSearcher (Salle et al., 2021) system aims to
simulate a specific type of clarifying questions that can be answered by ‘yes’
or ‘no’ (also referred to as check (Krasakis et al., 2020) or verification
(Keyvan and Huang, 2022) questions). To do this, they filter the dataset with
heuristically by whether the word ‘yes’ or ‘no’ is within the first three
words of the answers. This suggests that the clarifying questions in the
datasets can have other types not or cannot be answered by ‘yes’ or ‘no’. We
extend their heuristic rules and study the distribution of the type of
clarifying questions in these two datasets. We find that questions not
answerable by ‘yes’ or ‘no’ are mostly open questions or Wh-questions. In
addition, we notice a significant number of questions with answers such as “I
don’t know.” or “This is not related to my search.”, etc. This category is
necessary to respond to questions to indicate that it is irrelevant, and does
not actually answer the question. As an extension to CoSearcher, we categorize
all the questions in our datasets by their answer types into four classes:
$\\{\text{yes, no, open, irrelevant}\\}$, as shown in Table 1. We note that
our heuristic categorization (which builds on the one used for CoSearcher) is
not perfect. From random sampling, we estimate that about 10% of the answers
that are labeled as ‘open’ (i.e., does not contain ‘yes’ or ‘no’) can have
similar meanings as ‘yes’ or ‘no’ answers. For example, consider:

$i=$ “what are typical and good ranges for sat scores, what is the average
score?” $q=$ “sat” $cq=$ “are you searching for your sat scores?” $a=$ “i am
looking for the average score.”

The answer above does not contain a ‘no’ explicitly, but essentially says
‘(no,) i am looking for the average score.’

Table 1. Clarifying question distributions by answer type. Answer Types | Qulac/train | ClariQ/train
---|---|---
yes | 18.3% | 19.6%
no | 57.9% | 54.6%
open | 20.3% | 22.3%
irrelevant | 3.6% | 3.5%
Dataset Size # | 5273 | 8566

##  4\. T5 Baseline: An Unpublished State-of-the-Art

We now introduce the T5 baseline and its failings in user simulation.

###  4.1. T5 Model

Our baseline for this task is a pretrained T5 (Raffel et al., 2020) finetuned
on our training set. T5 is a general-purpose text generation model pretrained
on a large collection of text-to-text tasks, such as summarization, machine
translation, and question answering. It has the same structure as a
conventional encoder-decoder transformer (Vaswani et al., 2017), with
simplified layer norm and relative positional embeddings. T5 first encodes the
input text sequence and then generates an output text sequence with step-by-
step decoding. The input and output sequence we present to T5 are formatted
as:

$\begin{array}[]{l}\textit{input\\_seq}=i\;.\;q\;.\;cq\\\
\textit{output\\_seq}=a\end{array}$

Here, $i$ is the user search intent, $q$ is the user query, and $cq$ is the
system-generated clarifying question, and $a$ is the answer to the clarifying
question. The input sequence is formatted similarly to USi (Sekulić et al.,
2022), which uses GPT-2 as its generation model. The only difference is that
there are no counterparts of [SEP], [BOS], or [EOS] tokens in T5. Therefore,
to separate $i$, $q$, and $cq$, we use the period (‘.’). During finetuning, we
pass the input and output examples to pretrained T5 and tune it until
convergence with decoder cross-entropy loss.

Although the input and output of T5 and GPT-2 are almost identical, they are
structurally different. With a complete encoder-decoder structure, T5 (Raffel
et al., 2020) outperforms GPT-2, which only uses the decoder. In our task, T5
also performs significantly better than GPT-2. Because the T5 model has been
publicly available since 2019, we consider it an unpublished state-of-the-art.
The comparison of T5 and GPT-2, i.e., the existing state-of-the-art (Sekulić
et al., 2022), will be shown in Section 7, where we will see that simply
replacing GPT-2 with T5 brings significant improvements while halving its
parameters. However, does this mean that the future of user simulation
research is merely fine-tuning more advanced sequence-to-sequence natural
language generation (NLG) models?

###  4.2. A Deep Dive into T5 Generations

Instead of upgrading with the next NLG powerhouse, we take a different
approach in this work with an in-depth examination of the task. In particular,
we study the cases of T5 with significantly low ROUGE (Lin, 2004) scores.

Our analysis is conducted on the Qulac development set with the output from
Qulac-train-finetuned-T5. We first evaluate the generations from the T5 with
several automatic evaluation metrics, including BLEU (Papineni et al., 2002),
ROUGE (Lin, 2004), and METEOR (Banerjee and Lavie, 2005), as well as an
auxiliary classification f1-score of the answer type defined in Section 3.2.
We then keep all the examples with ROUGE lower than 0.2. This results in a
subset of the Qulac development set with 360 generation examples, which is
27.9% of the development set. From our result tables 3 and 4, the automatic
scores usually align reasonably with each other. In other words, these kept
generations will likely have low scores on all automatic metrics. Thus, from
the perspective of the automatic metrics, if taken seriously, these examples
will be considered as hard examples meriting closer study.

We investigate these examples by playing the role of users and evaluating the
human-generated responses and the T5 generations to find out why the scores
are so low. In doing so, we identify a few common types of low scoring
examples, described below.

#### Type 1: Answering clarifying question requires extra information.

There are two subtypes for why T5 fails in this class. The first subtype is
that the clarifying question asks for specific information about the user.
Usually, the specific information requires personal information like address,
age, etc., or personal preference. For example, (In examples for this section,
$g$ is the T5-generated responses, and $r$ is the human-generated response):

$i=$ “where can i find cheat codes for playstation 2 games?” $q=$ “ps 2 games”
$cq=$ “what types of ps 2 games do you like to play?” $g=$ “i want to find
cheat codes for playstation 2 games.” $r=$ “role playing.”

In this example, there is no explicit information indicating what game genre
the user likes to play. Thus the system does its best - to answer with the
user’s true intent.

The following example demonstrates the second subtype of this class, where the
clarifying question does not ask for extra personal information. However, the
human-generated response contains or implies information that the system by no
means would know.

$i=$ “find a schedule of events taking place at the orange county convention
center” $q=$ “orange county convention center” $cq=$ “do you need a hotel?”
$g=$ “no i need a schedule of events.” $r=$ “yes at the orange county
convention center.”

In this example, the system (and us) would not know whether the user will need
a hotel, given that they are finding a schedule of events at the location.
Therefore the system rejects the search intent in the question that seems
semantically unfamiliar from the actual intent. One may argue that it would be
reasonable for the user to need a hotel when organizing events. Such cases
could be improved by integrating a common-sense reasoning model. Note that our
goal is not to claim that these examples are impossible to simulate, because a
model can still randomly guess it correctly. However, these examples are
extremely hard to be simulated faithfully without additional user profiles,
world knowledge, etc. We find 37 examples in this class, about 10.3% of the
studied set.

#### Type 2: Both generations are valid.

The examples in this class have equally valid T5-generated responses as the
human-generated responses. However, they are not correctly evaluated by the
current automatic evaluation metrics. It is sometimes because the word-
overlap-based metrics cannot effectively evaluate paraphrases, and sometimes
it is not just a matter of paraphrasing. For example:

$i=$ “find the homepage of the president of the united states.” $q=$
“president of the united states” $cq=$ “are you looking for a list of all us
presidents?” $g=$ “no i want the homepage of the president of the united
states.” $r=$ “i need to go to his web site.”

In this example, both the system and human-generated responses are valid.
However, because they almost do not share words in common, the ROUGE score for
the system generation is low. Among all the cases, 50 examples fall into this
class, which is about 13.9%.

#### Type 3: Cooperativeness mismatch.

The idea of cooperativeness is mentioned by Salle et al. in (Salle et al.,
2021), where they propose to model it as a variable of their system.
Cooperativeness captures the following phenomenon: For yes-no questions, users
have multiple ways to answer the question, with the difference in how much
information they would like to express or simply how many words they want to
say. For example:

$i=$ “how do i register to take the sat exam?” $q=$ “sat” $cq=$ “do you need
information about the san antonio international airport?” $g=$ “no i need to
register to take the sat exam.” $r=$ “no.”

In the above example, the user is looking for instructions on the SAT exam
registration while being asked whether they need information about the airport
coded as SAT. In this case, both the human-generated and T5-generated answers
are valid answers to the question, but the T5 system adds more details. The
generation from T5 represents a cooperative user willing to supply the search
system with more information. In contrast, the generation from the human
represents an uncooperative user who tends to answer with minimal effort. We
want to highlight that, across the entire dataset, the human’s behavior seems
random. There is no indicator of whether the human-generated response would be
cooperative or not, given only the information in $(i,q,cq)$.

All the automatic metrics are sensitive to generation lengths. Therefore the
direct result of cooperative mismatch is the low scores of the automatic
metrics, whether the system generation is cooperative and human reference is
uncooperative or vice versa. This reason applies to 112 examples among all the
low-scoring cases, which is about 31.1%. We argue that the dataset needs a
column named user cooperativeness. Only with that can any system generate
responses of the same cooperativeness as human references and be appropriately
evaluated.

In fact, this class is a subtype of the “Both generations are valid” class.
However, since the number of examples in this subtype is so large and its
reason is so particular and straightforward, we decide to list this as a
separate class. Another reason for doing this is that we will later suggest an
evaluation strategy to avoid this specific type of misevaluation in the
results.

#### Type 4: Generating wrong answer type.

As mentioned in Section 3.2, we introduced a 4-way categorization of answer
types. This class contains examples where the T5 generation has the wrong
answer type, e.g., when it needs to say ‘yes’, instead it says ‘no’ or vice
versa. When this type of mistake happens in answering clarification questions,
the response mostly has wrong meaning. For example:

$i=$ “i’m looking for web sites that do antique appraisals” $q=$ “appraisals”
$cq=$ “do you need an antique appraised?” $g=$ “no i want to know about
antique appraisals.” $r=$ “yes.”

In this above example, the user is looking for websites that do antique
appraisals, which implies that they are very likely to need their antique
appraised according to simple reasoning. The T5-generated response is a
‘no’-type answer, which determines that the meaning of the answer is entirely
contradictory to the human-generated answer. Note that just the difference of
one word between ‘yes’ and ‘no’ may not be the main reason why the generation
receives a low ROUGE score, but this class represents a type of mistake that
T5 makes, which is getting the answer type wrong. We find 122 examples of such
mistakes, which is 33.9% of all the cases with low ROUGE.

#### Type 5: Noise in data.

It may not be surprising that the clarifying questions or the human-generated
responses can be of poor quality sometimes because they are crowd-sourced. A
bad clarifying question does not clarify the search intent, while a bad human-
generated response can be incoherent or inconsistent with the search intent.
If the clarifying question is bad, it can be challenging to generate a valid
response. Moreover, a good generation can have low evaluation scores if the
human reference is bad. For example:

$i=$ “find information on various types of computer memory, and how they are
different.” $q=$ “memory” $cq=$ “who was the first to study the brain and
memory?” $g=$ “i want to know how different they are.” $r=$ “herman
ebbinguaus.”

This is an example of bad clarifying questions in the dataset. To understand
this example, we searched the internet and verified that Herman Ebbinguaus was
considered the pioneer of brain and memory studies. The problem in this
example is that the clarifying question ‘who was the first to study the brain
and memory?’ does not serve the purpose of clarifying search intent. Perhaps,
the system actually wants to ask ‘do you want to know who was the first to
study the brain and memory’. However, the human-generated response is still
unnatural, considering the user is the information seeker not provider. The T5
generated response $g$ is not perfect, but had it generated a reasonable
response such as ‘no, i want to find information about different types of
computer memory.’, it will still be mis-evaluated due to the noisy data. There
are 21 examples of such noisy data among all the examples, which is 5.8%.

#### Type 6: Miscellaneous.

The rest of the examples are all in this class, where we find the T5
generations are wrong for various reasons, but different from any of the
abovementioned classes. Most of them are isolated, wrong generations for a
plethora of reasons. There appear to be 15 examples in this class, about 4.2%
of the studied set.

###  4.3. A Summary of T5’s Failings

In Table 2, we summarize the types and the distribution of low-scoring cases
of T5 on the Qulac development set. From the table, we see that the main
reasons for T5 generations getting low ROUGE is generating answers with the
wrong answer type or user type with the human-generated answer. A few other
observations are worth noting. At least 45% of the low-scoring generations are
good generations. This number is obtained by adding the ‘Cooperativeness
mismatch’ and ‘Both valid’ cases. This suggests that the system-generated
responses could be better than the numbers seem. Another observation is that
only 38% of the low-scoring generations should be solely attributed to the T5
model, corresponding to the ‘Wrong answer type’ and ‘Miscellaneous’ types.
What this suggests is that the room for improvement is quite limited. However,
this analysis does show us that, among all these classes, the most realistic
way to improve system performance is to address the answer type errors in the
system. On the other hand, the most realistic way to improve evaluation is to
address the misevaluations caused by cooperativeness mismatch. Our proposals
to address these two problems will be described in Section 5.1, and 5.2, and
Section 5.3 respectively.

Table 2. Categorization of reasons for low ROUGE Reasons | T5
---|---
Wrong answer type | 33.9%
Cooperativeness mismatch | 31.1%
Both valid | 13.9%
Extra information | 10.3%
Noisy reference | 5.8%
Miscellaneous | 4.2%
Total # ROUGE¡0.2 | 360 (27.9%)

##  5\. Remedies for the Failings

This section describes our proposed models, which aim to address failings of
T5 and the misevaluation.

###  5.1. Learning from Question Answering

From the analysis in Section 4, we see that most low-scoring generations of T5
are caused by answer type error. We note that simulating user responses to
clarifying questions is similar to the natural language processing task of
question answering (QA) in that both require a response to a question. Can we
improve user simulators from knowledge in these tasks, and QA tasks in
general? We instantiate our experiments using one of the current state-of-the-
art models for QA—UnifiedQA (Khashabi et al., 2020), which extends T5 by
pretraining on twenty QA datasets across four formats. One of the twenty
datasets is BoolQ (Clark et al., 2019), which consists of yes-no questions
that need to be answered with a short paragraph provided as context. The
verification clarifying questions in user simulation such as “Are you looking
for Y?”, given the search intent “I am looking for X.”, can be considered a
particular case akin to the examples in BoolQ. As another QA dataset included
in UnifiedQA’s training set, SQuAD (Rajpurkar et al., 2016) contains reading
comprehension questions with answers as spans from the context. The wh-
questions in SQuAD are also similar to the open clarifying questions in user
response simulation. Further, even questions that do not directly map to user
simulation can potentially increase the general reasoning ability of the
simulation system.

In this work, we fine-tune the pretrained UnifiedQA on our data. To do so, we
follow the format described in UnifiedQA. We treat the intent $i$, query $q$
as the context, and $cq$ as the question. During training, we found that the
query $q$ does not provide performance gain and can be dropped from the
context. Therefore, our final input and output format for finetuning UnifiedQA
is as follows:

$\begin{array}[]{l}\textit{input\\_seq}=cq\text{ ? }\textbackslash n\text{ i
am looking for }i\\\ \textit{output\\_seq}=a\end{array}$

where ‘\n’ is not the newline character, but a special backslash-n character,
as explained in UnifiedQA (Khashabi et al., 2020). The reason for adding ‘i am
looking for’ before the search intent is that most of the intents from the
dataset (the facet column) are imperative sentences such as “Find information
about human memory”. Therefore, we add the prefix to mimic questions and
contexts from UnifiedQA pretraining tasks. It can also be seen as a way of
prompting (Radford et al., 2019; Petroni et al., 2019; Liu et al., 2021).

###  5.2. Conceptualization-Instantiation Generation

Not every word in the user response generation is equally important.
Naturally, the most important word in the answer to a yes-no question will be
the ‘yes’ or ‘no’; it almost solely determines the semantics and sentiment of
the rest of the answer. Therefore, there should be a higher priority to at
least generate the ‘yes’ or ‘no’ correctly over other words in the response.
We hypothesize that this also imitates the response generation process of a
human. A human will likely first conceptualize their answer as a confirmation,
negation, or irrelevancy assertion to the clarifying question before
instantiating the entire answer.

However, the training loss of the text sequence-to-sequence models (of which
T5 and UnifiedQA are instances) averages the cross-entropy loss across all the
decoding steps. In other words, the ‘yes’s and ‘no’s are weighted the same as
other words in the loss. While this loss is widely used for general text
generation tasks, it may not be the best strategy for our goal of emphasizing
the correctness of answer types. To demonstrate that this loss is indeed
ineffective, we inspect the top 10 generations from UnifiedQA for a set of
randomly chosen yes-no questions. While we expect the top 10 generations to be
united in being a ‘yes’ or ‘no’ answer, we find that the top 10 generations
contain both yes-answers and no-answers.

Also, we find two incongruous situations of the top 10 answers during the
examination. When the search intent is “I am looking for X.”, suppose a
clarifying question asks, “are you looking for Y”. In one situation, UnifiedQA
generates multiple potential answers in its top 10 generations. For example,
the top 10 answers contain “yes, I am looking for Y” and “no, I am looking for
X.” simultaneously. This case potentially shows that the sequence-to-sequence
learned model is good at learning distributions, i.e., answering with ‘yes’ or
‘no’ for a yes-no question, while being weak in deciding between ‘yes’ and
‘no’. The second situation is that, for the same question, UnifiedQA generates
contradictory answers such as both “yes, I am looking for X” and “no, I am
looking for X” in its top 10 generations. We conjecture that this is because
the model picks up the pattern of repeating/rephrasing words from the search
intent but cannot efficiently learn to decide whether the question accords
with the search intent, i.e., to answer the question.

The above observation shows that UnifiedQA is good at generating various
possible answers but may not be good at predicting which of them has the
correct answer type. Therefore, we can potentially leave the latter to a
specialized model, such as a classification model. To this end, we propose to
train a classifier that predicts answer types to guide the UnifiedQA through
constrained generation. We use RoBERTa (Liu et al., 2019) to instantiate our
classifier, which improves upon BERT (Devlin et al., 2018) by optimizing
hyperparameter choices. We fine-tune the pre-trained RoBERTa-base on the
answer-type classification task, as defined in Table 1. During inference, we
convert its prediction to generator decoding constraints as follows: If the
predicted answer type is ‘yes’ or ‘no’, then the generation starts with ‘yes’
or ‘no’ correspondingly and generates the rest. If the predicted answer type
is ‘irrelevant’, the generation starts with ‘I don’t know’. Otherwise, we will
not place any constraints on the generator, in which case it falls back to the
standard UnifiedQA. Since this the-step generation procedure attempts to our
hypothesized human generating process of conceptualizing the answer type first
before instantiating each word, we name this model the Conceptualizaton-
Instantiation (CI) generation model.

###  5.3. Cooperativeness-Aware Evaluation

The analysis in Section 4 shows that a large proportion of mis-evaluation
comes from the cooperativeness mismatch between the T5 generations and the
human-generated responses. We also propose that the dataset, if it is to be
used for user response simulation, is missing a necessary column indicating
the cooperativeness of the human-generated response. As we have seen, user
cooperativeness can shape the final response generation. Moreover, it is is
independent and cannot be predicted from the other inputs. However, this issue
is rarely mentioned in previous work. Only CoSearcher (Salle et al., 2021)
addresses this by modeling cooperativeness as a controllable variable in their
model, but their model randomly decides between cooperative or uncooperative
based on random seeds during inference. This fails to fundamentally solve the
problem, since randomly choosing cooperativeness cannot guarantee to match the
actual user.

Therefore, in this paper, we propose a cooperativeness-aware data partitioning
heuristic to train and evaluate user simulation systems. The idea is simple:
if cooperativeness mismatch is a major cause of responses being judged to have
poor quality, then we should evaluate with only matched cooperativeness. To
implement this, we partition the dataset into two subsets: one with only
generations fewer than three words as the uncooperative group and the rest as
cooperative group. Next, we train two simulation systems on each partition,
and evaluate them separately on each partition.

##  6\. Experimental Setup

In our experiments, we aim to test whether our proposed models can effectively
improve upon existing models, and how well the challenges we identify in our
analyses can be resolved. All the experiments use the two datasets we
introduced in Section 3.2: Qulac and ClariQ. On Qulac, we finetune all of the
models on the Qulac train set and test them on the Qulac test set. The Qulac
dev set is mainly used for our analyses. On ClariQ, we use the ClariQ train
set for finetuning and the ClariQ dev set for testing. This is because ClariQ
test sets are created using search topics and documents outside TREC
Clueweb09-12. Therefore the corresponding document and .qrel files necessary
for document retrieval performance evaluations are not publicly available.

We first conduct two experiments: one to compare the previous state-of-the-art
GPT-2 model and the baseline T5 model, and the other to evaluate our proposed
Conceptualization-Instantiation (CI) generation model. The goal of the first
experiment is to verify our claim that the previous state-of-the-art can be
easily improved by replacing with a smaller but advanced T5 model. The T5
model we use in our experiments is T5-small, which has only 60M parameters,
while GPT-2 has 124M. The goal of our second experiment is to verify whether
the RoBERTa in the proposed CI model can predict more precise answer types
than T5 and whether UnifiedQA can generate better responses with knowledge
learned from question-answering tasks. Similar to the analyses in Section 4,
we conduct another analysis on our proposed CI model on the Qulac dev set to
study whether the categorization of CI model is different from T5.

Next, we conduct a cooperativeness-aware evaluation experiment as described in
Section 5.3. In this experiment, we apply the cooperativeness-aware evaluation
on both T5 and the proposed CI model. Then, we compare the cooperativeness-
aware evaluation results with current cooperativeness-unaware evaluation
results. The goal is to show that cooperativeness-aware evaluation reduces
mis-evaluations and gives better insights into generation quality.

We include three complementary evaluation paradigms to comprehensively compare
all the results in the experiments.

#### Generation Metrics

The generation metrics measure the similarity between generated response and
human reference by word overlap, such as BLEU (Papineni et al., 2002),
ROUGE(Lin, 2004), and METEOR (Banerjee and Lavie, 2005). An issue with these
metrics is that they assume all the words are of equal importance. However,
there are highly important words in the task of user response simulation, such
as ‘yes’ and ‘no’. Thus, these metrics can cause responses with the exact
opposite meaning to get high scores and good generations to get low scores. As
described earlier in Section 3.2, we extend the method in CoSearcher (Salle et
al., 2021) to classify responses into four types, which are inherently
different ways of answering a question. The idea is that if the generated
response disagrees with the answer type of human reference, it is highly
improbable that they are similar in meaning. Therefore, we propose to use the
answer type f1 score as another metric to measure the quality of generated
responses.

#### Human Judgement

In addition to automatic evaluation, we crowd workers from Amazon MTurk to
evaluate our generated responses. The workers are required to have 500
lifetime HITs approvals and 95+% approval rate. We provide workers 200
randomly sampled tuples of Qulac query, search intent, clarifying question,
and generated response. They receive shuffled list of generations from T5, CI,
and the copy-intent oracle without knowing source of each. We ask the workers
to score the generated responses according to two criteria, relevance and
naturalness, from 1 to 5. The workers get paid at a rate of twelve dollars per
hour work.

We define relevance as whether the response is consistent with the search
intent and whether it helps the search system better understand the user’s
unspecified intention. Relevant answers perfectly align with the intent, while
irrelevant responses contradict the search intent or can be randomly off-
topic. Similar existing metrics are adequacy (Stent et al., 2005),
informativeness (Chuklin et al., 2019), and usefulness (Sekulić et al., 2022).

Naturalness measures whether the generated response is fluent, grammatical,
and human-like. Natural answers are fluent and like our daily language, while
unnatural answers can be hard to understand. Moreover, natural answers should
not provide information beyond what the question is asking for. As an example
of unnatural generations, we include a _copy-intent oracle_ in our evaluation,
which always copies the search intent for response. From the conversational
search system perspective, such an answer will provide the perfect
information. However, it is not a reasonable model because a real user will
not answer every question by repeating their search intent. Also, imagine
training a conversational search system to ask clarifying questions with the
copy-intent baseline as the user simulator. The system will never learn to
differentiate good clarifying questions from bad ones since both will receive
true search intent leaked by the user simulator. Therefore, we add naturalness
as another essential criterion for user response simulation.

#### Retrieval Metrics

Evaluating the document retrieval performance is a standard paradigm used by
existing work (Aliannejadi et al., 2019; Sekulić et al., 2022; Salle et al.,
2021). This evaluation is based on the motivation of asking clarifying
questions in conversational search: the additional information from the
clarifying question and its answer should help retrieve better results. Hence,
these works assume that better responses will generally have higher document
retrieval performances. We follow these works and use a query-likelihood model
(Ponte and Croft, 2017) as our document retrieval model to provide comparable
results with these works.

Table 3. Model performances on Qulac dataset. Our proposed CI model outperforms the unpublished state-of-the-art T5 model. Refer to Section 7 for detailed explanations. ${\dagger}$ and ${\ddagger}$ indicate $p<0.05$ and $p<0.01$ statistical significance of improvements over the T5 baseline using permutation test. Model | Generation Metrics |  | Retrieval Metrics
---|---|---|---
BLEU-3 | BLEU-4 | ROUGE-L | METEOR | F1 |  | nDCG1 | nDCG5 | nDCG20 | P@1 | MRR
USi (GPT-2) | 12.56 | 9.08 | 28.24 | 28.90 | 24.43 |  | 0.1845 | 0.1864 | 0.1732 | 0.2440 | 0.3517
T5 | 23.68 | 19.00 | 40.80 | 43.15 | 34.07 |  | 0.2170 | 0.2099 | 0.1879 | 0.2809 | 0.3896
UnifiedQA | 23.33 | 18.66 | 40.90 | 43.43 | 41.65 |  | 0.2148 | 0.2101 | 0.1882 | 0.2788 | 0.3904
CI | 24.44‡ | 19.61† | 41.64‡ | 43.49 | 43.28‡ |  | 0.2125 | 0.2083 | 0.1869 | 0.2751 | 0.3877
T5+Coop | 28.62 | 23.02 | 48.14 | 46.01 | 40.69 |  | 0.2061 | 0.2029 | 0.1816 | 0.268 | 0.3773
CI+Coop | 30.06 | 24.29 | 50.54 | 46.70 | 50.11 |  | 0.2032 | 0.2024 | 0.1816 | 0.2648 | 0.3759
Query only | - | - | - | - | - |  | 0.1329 | 0.1461 | 0.1525 | 0.1900 | 0.2938
Human | 100.00 | 100.00 | 100.00 | 93.65 | 100.00 |  | 0.1982 | 0.1946 | 0.1776 | 0.2593 | 0.3674
Copy-intent | 17.30 | 13.77 | 31.42 | 29.40 | 8.28 |  | 0.2161 | 0.2123 | 0.1931 | 0.2831 | 0.3913
Table 4. Model performances on ClariQ dataset. Our proposed CI model outperforms the unpublished state-of-the-art T5 model. ${\dagger}$ and ${\ddagger}$ indicate $p<0.05$ and $p<0.01$ statistical significance of improvements over the T5 baseline using permutation test. Model | Generation Metrics |  | Retrieval Metrics
---|---|---|---
BLEU-3 | BLEU-4 | ROUGE-L | METEOR | F1 |  | nDCG1 | nDCG5 | nDCG20 | P@1 | MRR
USi (GPT-2) | 13.51 | 9.79 | 28.82 | 28.56 | 22.76 |  | 0.1346 | 0.1222 | 0.1061 | 0.1603 | 0.2331
T5 | 24.30 | 19.51 | 41.00 | 43.32 | 36.64 |  | 0.1497 | 0.1335 | 0.1182 | 0.1756 | 0.2485
UnifiedQA | 24.32 | 19.41 | 41.63 | 43.58 | 45.90 |  | 0.1476 | 0.1349 | 0.1192 | 0.1704 | 0.2471
CI | 25.24† | 20.19† | 42.11‡ | 43.05 | 46.27‡ |  | 0.1488 | 0.1364 | 0.1194 | 0.1722 | 0.2484
T5+Coop | 29.11 | 23.43 | 48.10 | 45.44 | 41.76 |  | 0.1472 | 0.1328 | 0.1165 | 0.1725 | 0.2463
CI+COOP | 30.39 | 24.31 | 49.95 | 45.60 | 53.41 |  | 0.1409 | 0.1330 | 0.1164 | 0.1625 | 0.2422
Query only | - | - | - | - | - |  | 0.1184 | 0.1087 | 0.0894 | 0.1316 | 0.2021
Human | 100.00 | 100.00 | 100.00 | 93.43 | 100.00 |  | 0.1387 | 0.1272 | 0.1121 | 0.1629 | 0.2377
Copy-intent | 18.11 | 14.38 | 31.71 | 29.86 | 8.71 |  | 0.1494 | 0.1369 | 0.1212 | 0.1749 | 0.2497
Table 5. Average relevance and naturalness from crowd workers. Our proposed CI mode outperforms T5. $\star$ indicates $p<0.01$ statistical significance of improvements over copy-intent baseline. ${\dagger}$ indicates $p<0.05$ statistical significance of improvements over T5. Both are done with paired t-test. model | Relevance | Naturalness
---|---|---
T5 | 4.21 | 4.16⋆
CI | 4.35 † | 4.30⋆†
Copy-intent | 4.64 | 3.57
Table 6. Categorization of reasons for low ROUGE. Our proposed CI and Cooperativeness-aware evaluation effectively address T5’s failings. ${\dagger}$ and ${\ddagger}$ indicate statistical significance of improvement over T5 with $p<0.05$ and $p<0.01$ using fisher’s exact test.  Reasons | T5+Coop | T5 | CI
---|---|---|---
Wrong answer type | 46.4% |  33.9%$\rightarrow$ | 11.8%‡
Cooperativeness miss | 2.4%‡ |  $\leftarrow$ 31.1% | 44.6%
Both valid | 16.4% | 13.9% | 15.2%
Extra information | 15.6% | 10.3% | 13.0%
Noisy reference | 11.6% | 5.8% | 7.4%
Miscellaneous | 7.6% | 4.2% | 7.1%
Total #ROUGE¡0.2 | 250 (19.4%‡) | 360 (27.9%) | 323 (25.0%†)

##  7\. Results and Analyses

We present the automatic evaluations for comparing GPT-2, T5, UnifiedQA, and
our proposed CI model in Table 3 and Table 4, human evaluations in Table 5 and
Figure 1, and additional results for data partition and evaluation in Table 6.

#### Oracle Models.

Besides the model we plan to compare in the experiments section, we also
include a few oracle models. The ‘Query only’ model shows the document
retrieval performances of searching only with initial query. From the table,
all the other rows have significantly higher scores than it. This result is
consistent with previous studies about the benefit of asking clarifying
questions in conversational search. The ‘Human’ row is the human-generated
response. Thus it has the perfect score for generation metrics against itself.
The ‘Copy intent’ row is an oracle model that always directly copies the
search intent to simulate user response. Its generation scores are noticeably
low, showing that human-generated responses tend to differ from the original
search intent. Also, notice that the GPT-2 based generator has even lower
scores than this copy-intent model, which implies that it has limited
generation capacity. More interestingly, the copy-intent model usually tops
retrieval metrics, even higher than human ones. This result is reasonable
because, for the search system, the best possible response is the exact search
intent, regardless of clarifying question it asks, while the human generated
response may not contain the search intent.

#### GPT-2 versus T5.

From the first two rows from both tables, we can see that the T5 baseline
significantly improves GPT-2 performances measured by both generation metrics
and retrieval metrics on both the Qulac and the ClariQ datasets. This result
confirms our previous claim that T5 is an unpublished state-of-the-art.

Figure 1. Kernel Density Estimate plot for crowd worker scores of T5 and CI
model

Kernel Density Estimate plot for crowd worker scores of T5 and CI model

#### T5 versus CI.

Next, we compare the performances of T5, UnifiedQA, and the CI model. We bold
the numbers for best performing model among the three and see that CI is the
best in most columns. Being the best in generation metrics suggests it
produces the most human-like generations. For retrieval metrics, all three
models outperform the existing state-of-the-art, and all three models are on
par with the copy-intent oracle. None from the three consistently outperforms
the others, and none of the differences is significant enough. This suggests
the three models have indistinguishable utilities for document retrieval.
However, useful answers for retrieval are not necessarily good answers, as
discussed in the copy-intent oracle. Therefore, to gain better insights into
the CI model, we need to combine the above with the human evaluation results
in Table 5.

#### Crowd-sourced evaluation.

Human evaluation shows that copy-intent oracle generates the most relevant
responses, while being poor in naturalness. T5 improves the naturalness over
copy-intent, but drops in relevance. Our proposed CI model improves both over
T5. Paired-t test verifies that these improvements are significant. We use a
kernel distribution estimation (KDE) plot to visualize how our model improves
T5 in Fig 1. The 2-D KDE plot in the middle also shows two 1-D KDEs in the top
and right. Each dot represents an averaged pair of $(relevance,naturalness)$.
From the 2-D KDE, we see that the blue T5 distribution has more low-scoring
region compared to CI. Also, from the peaks of 1-D KDEs, we see that the
orange CI model has more 5-scoring examples.

#### Manual analysis.

Next, we conduct a manual analysis for CI model generations, shown in Table 6.
We find 323 low-scoring examples from the CI model’s generation, which is
significantly fewer than T5. The table shows that CI model has significantly
fewer wrong answer types, reducing it from 33.9% to 11.8%. The percentage of
low-scoring generation caused by model mistakes, by summing up wrong answer
type and miscellaneous, also drops from 38.1% to 18.9%, meaning the CI model
only generates about half as many bad generations as T5. Combining the
automatic, human evaluations and our manual analysis, we can conclude that our
proposed CI model significantly improves the unpublished state-of-the-art T5
baseline by addressing the wrong answer types in generations.

#### Evaluating with cooperativeness.

The T5+Coop and CI+Coop rows in Table 3 and Table 4 are results of our data
partitioning evaluation. These numbers in the table are aggregates that
combine different cooperativeness. We see that both T5+Coop and CI+Coop see
significant improvements with cooperativeness-aware evaluation in Table 3 and
Table 4, compared with T5 and CI. However, we still need to show that these
automatic score improvements is a result of reduction in cooperativeness
mismatch to sufficiently argue our new evaluation heuristic is effective. For
this purpose, we do a manual analysis for T5+Coop, shown in Table 6. In the
table, we see the cooperativeness mismatches drop significantly from 31.1% to
only 2.4%, and the total of low-scoring examples subsequently declines. This
result, plus the automatic metrics from Table 3 and Table 4, sufficiently
shows that the inaccessibility of cooperativeness information is an
outstanding reason for mis-evaluating these systems, and our proposed
cooperativeness-aware evaluation can effectively avoid this mis-evaluation.

#### Final words.

The above observations and analyses show that our proposed solutions for
correcting wrong answer types in the generation model and reducing mis-
evaluations are both effective. Interestingly, we also observe a repeated
pattern that the generation and retrieval metrics exhibits negative
correlations when they are above certain thresholds, i.e., a generation that
is more human-like tends to get lower retrieval performances. However, the
difference are not statistical significant. We conjecture that this is for the
same reason as the human oracle underperforming the copy-intent oracle in
retrieval metrics. Because the best answer for document retrieval is always
repeating the search intent, but human will not always do that. However, the
verification is left as a future work.

##  8\. Conclusion

In this paper, we study the task of simulating user responses for clarifying
questions in conversational search. We find that replacing the existing state-
of-the-art GPT-2-based model with T5 can significantly improve generation
performance. Rather than reporting it as the new state-of-the-art, we cast the
question of what can be learned from the task instead of waiting for the next
powerful generation model. As part of the answering process, we conduct an in-
depth manual analysis of the low-performance generations of T5. It shows that
aside from data noise, 38% of the generations are bad because T5 cannot
effectively learn to generate responses of the correct answer type, and at
least 45% of the generations have low performances due to mis-evaluations. We
then propose a simple two-step model named the Conceptualization-Instantiation
model to correct the wrong answer types in generations, which is shown to
improve the T5 model significantly in our experiments evaluated by three
complementary evaluations. Further, we propose a data partition heuristic to
account for an essential variable for user simulation, the cooperativeness,
which substantially improves upon the existing evaluation standard.

## References

  * (1)
  * Aliannejadi et al. (2020) Mohammad Aliannejadi, Julia Kiseleva, Aleksandr Chuklin, Jeff Dalton, and Mikhail Burtsev. 2020\.  ConvAI3: Generating clarifying questions for open-domain dialogue systems (ClariQ).  _arXiv preprint arXiv:2009.11352_ (2020).
  * Aliannejadi et al. (2019) Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W Bruce Croft. 2019.  Asking clarifying questions in open-domain information-seeking conversations. In _Proceedings of the 42nd international acm sigir conference on research and development in information retrieval_. 475–484.
  * Anand et al. (2021) Avishek Anand, Lawrence Cavedon, Matthias Hagen, Hideo Joho, Mark Sanderson, and Benno Stein. 2021\.  Dagstuhl Seminar 19461 on Conversational Search: Seminar Goals and Working Group Outcomes.  _SIGIR Forum_ 54, 1, Article 3 (feb 2021), 11 pages.  https://doi.org/10.1145/3451964.3451967
  * Banerjee and Lavie (2005) Satanjeev Banerjee and Alon Lavie. 2005.  METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In _Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization_. Association for Computational Linguistics, Ann Arbor, Michigan, 65–72.  https://aclanthology.org/W05-0909
  * Bi et al. (2021) Keping Bi, Qingyao Ai, and W Bruce Croft. 2021.  Asking Clarifying Questions Based on Negative Feedback in Conversational Search. In _Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval_. 157–166.
  * Chandramohan et al. (2011) Senthilkumar Chandramohan, Matthieu Geist, Fabrice Lefevre, and Olivier Pietquin. 2011.  User simulation in dialogue systems using inverse reinforcement learning. In _Twelfth annual conference of the international speech communication association_.
  * Chen et al. (2019) Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, and Le Song. 2019\.  Generative adversarial user model for reinforcement learning based recommendation system. In _International Conference on Machine Learning_. PMLR, 1052–1061.
  * Chuklin et al. (2019) Aleksandr Chuklin, Aliaksei Severyn, Johanne R Trippas, Enrique Alfonseca, Hanna Silen, and Damiano Spina. 2019.  Using audio transformations to improve comprehension in voice question answering. In _Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10_. Springer, 164–170.
  * Clark et al. (2019) Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019.  BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Association for Computational Linguistics, Minneapolis, Minnesota, 2924–2936.  https://doi.org/10.18653/v1/N19-1300
  * Clarke et al. (2009) Charles L. A. Clarke, Nick Craswell, and Ian Soboroff. 2009\.  Overview of the TREC 2009 Web Track. In _TREC_.
  * Culpepper et al. (2018) J Shane Culpepper, Fernando Diaz, and Mark D Smucker. 2018\.  Research frontiers in information retrieval: Report from the third strategic workshop on information retrieval in lorne (swirl 2018). In _ACM SIGIR Forum_ , Vol. 52. ACM New York, NY, USA, 34–90.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Eckert et al. (1997) Wieland Eckert, Esther Levin, and Roberto Pieraccini. 1997\.  User modeling for spoken dialogue system evaluation. In _1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings_. IEEE, 80–87.
  * Fu et al. (2020) Zuohui Fu, Yikun Xian, Yongfeng Zhang, and Yi Zhang. 2020\.  Tutorial on Conversational Recommendation Systems. In _Fourteenth ACM Conference on Recommender Systems_. 751–753.
  * Gao et al. (2018) Jianfeng Gao, Michel Galley, and Lihong Li. 2018.  Neural approaches to conversational ai. In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_. 1371–1374.
  * Gao et al. (2022) Jianfeng Gao, Chenyan Xiong, Paul Bennett, and Nick Craswell. 2022.  Neural approaches to conversational information retrieval.  _arXiv preprint arXiv:2201.05176_ (2022).
  * Hauff et al. (2021) Claudia Hauff, Julia Kiseleva, Mark Sanderson, Hamed Zamani, and Yongfeng Zhang. 2021.  Conversational Search and Recommendation: Introduction to the Special Issue.
  * Hou et al. (2019) Yutai Hou, Meng Fang, Wanxiang Che, and Ting Liu. 2019\.  A corpus-free state2seq user simulator for task-oriented dialogue. In _Chinese Computational Linguistics: 18th China National Conference, CCL 2019, Kunming, China, October 18–20, 2019, Proceedings_. Springer, 689–702.
  * Keyvan and Huang (2022) Kimiya Keyvan and Jimmy Xiangji Huang. 2022.  How to Approach Ambiguous Queries in Conversational Search? A Survey of Techniques, Approaches, Tools and Challenges.  _ACM Computing Surveys (CSUR)_ (2022).
  * Khashabi et al. (2020) D. Khashabi, S. Min, T. Khot, A. Sabhwaral, O. Tafjord, P. Clark, and H. Hajishirzi. 2020.  UnifiedQA: Crossing Format Boundaries With a Single QA System.  _EMNLP - findings_.
  * Komatani et al. (2005) Kazunori Komatani, Shinichi Ueno, Tatsuya Kawahara, and Hiroshi G Okuno. 2005.  User modeling in spoken dialogue systems to generate flexible guidance.  _User Modeling and User-Adapted Interaction_ 15 (2005), 169–183.
  * Krasakis et al. (2020) Antonios Minas Krasakis, Mohammad Aliannejadi, Nikos Voskarides, and Evangelos Kanoulas. 2020.  Analysing the effect of clarifying questions on document ranking in conversational search. In _Proceedings of the 2020 acm sigir on international conference on theory of information retrieval_. 129–132.
  * Li et al. (2016) Xiujun Li, Zachary C Lipton, Bhuwan Dhingra, Lihong Li, Jianfeng Gao, and Yun-Nung Chen. 2016\.  A user simulator for task-completion dialogues.  _arXiv preprint arXiv:1612.05688_ (2016).
  * Lin (2004) Chin-Yew Lin. 2004\.  ROUGE: A Package for Automatic Evaluation of Summaries. In _Text Summarization Branches Out_. Association for Computational Linguistics, Barcelona, Spain, 74–81.  https://aclanthology.org/W04-1013
  * Liu et al. (2021) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021\.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _arXiv preprint arXiv:2107.13586_ (2021).
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _arXiv preprint arXiv:2203.02155_ (2022).
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002\.  Bleu: a Method for Automatic Evaluation of Machine Translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 311–318.  https://doi.org/10.3115/1073083.1073135
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. 2019.  Language models as knowledge bases?  _arXiv preprint arXiv:1909.01066_ (2019).
  * Ponte and Croft (2017) Jay M Ponte and W Bruce Croft. 2017.  A language modeling approach to information retrieval. In _ACM SIGIR Forum_ , Vol. 51. ACM New York, NY, USA, 202–208.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Radlinski and Craswell (2017) Filip Radlinski and Nick Craswell. 2017.  A theoretical framework for conversational search. In _Proceedings of the 2017 conference on conference human information interaction and retrieval_. 117–126.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.  _Journal of Machine Learning Research_ 21, 140 (2020), 1–67.  http://jmlr.org/papers/v21/20-074.html
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016.  Squad: 100,000+ questions for machine comprehension of text.  _arXiv preprint arXiv:1606.05250_ (2016).
  * Rao and Daumé III (2019) Sudha Rao and Hal Daumé III. 2019.  Answer-based adversarial training for generating clarification questions.  _arXiv preprint arXiv:1904.02281_ (2019).
  * Salle et al. (2021) Alexandre Salle, Shervin Malmasi, Oleg Rokhlenko, and Eugene Agichtein. 2021.  Studying the effectiveness of conversational search refinement through user simulation. In _European Conference on Information Retrieval_. Springer, 587–602.
  * Schatzmann et al. (2007) Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, and Steve Young. 2007.  Agenda-based user simulation for bootstrapping a POMDP dialogue system. In _Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers_. 149–152.
  * Schatzmann et al. (2006) Jost Schatzmann, Karl Weilhammer, Matt Stuttle, and Steve Young. 2006.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies.  _The knowledge engineering review_ 21, 2 (2006), 97–126.
  * Scheffler and Young (2000) Konrad Scheffler and Steve Young. 2000.  Probabilistic simulation of human-machine dialogues. In _2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No. 00CH37100)_ , Vol. 2. IEEE, II1217–II1220.
  * Sekulić et al. (2021) Ivan Sekulić, Mohammad Aliannejadi, and Fabio Crestani. 2021\.  Towards Facet-Driven Generation of Clarifying Questions for Conversational Search. In _Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval_. 167–175.
  * Sekulić et al. (2022) Ivan Sekulić, Mohammad Aliannejadi, and Fabio Crestani. 2022\.  Evaluating Mixed-Initiative Conversational Search Systems via User Simulation. In _Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining_ (Virtual Event, AZ, USA) _(WSDM ’22)_. Association for Computing Machinery, New York, NY, USA, 888–896.  https://doi.org/10.1145/3488560.3498440
  * Spink et al. (2001) Amanda Spink, Dietmar Wolfram, Major BJ Jansen, and Tefko Saracevic. 2001.  Searching the web: The public and their queries.  _Journal of the American society for information science and technology_ 52, 3 (2001), 226–234.
  * Stent et al. (2005) Amanda Stent, Matthew Marge, and Mohit Singhai. 2005\.  Evaluating Evaluation Methods for Generation in the Presence of Variation.. In _CICLing_ , Vol. 2005. Springer, 341–351.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wang and Ai (2022) Zhenduo Wang and Qingyao Ai. 2022.  Simulating and Modeling the Risk of Conversational Search.  _ACM Trans. Inf. Syst._ 40, 4, Article 85 (mar 2022), 33 pages.  https://doi.org/10.1145/3507357
  * Zamani and Craswell (2020) Hamed Zamani and Nick Craswell. 2020.  Macaw: An extensible conversational information seeking platform. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 2193–2196.
  * Zamani et al. (2020a) Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, and Gord Lueck. 2020a.  Generating clarifying questions for information retrieval. In _Proceedings of The Web Conference 2020_. 418–428.
  * Zamani et al. (2020b) Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul N Bennett, Nick Craswell, and Susan T Dumais. 2020b.  Analyzing and learning from user interactions for search clarification. In _Proceedings of the 43rd international acm sigir conference on research and development in information retrieval_. 1181–1190.
  * Zamani et al. (2022) Hamed Zamani, Johanne R Trippas, Jeff Dalton, and Filip Radlinski. 2022.  Conversational information seeking.  _arXiv preprint arXiv:2201.08808_ (2022).
  * Zhang and Balog (2020) Shuo Zhang and Krisztian Balog. 2020.  Evaluating conversational recommender systems via user simulation. In _Proceedings of the 26th acm sigkdd international conference on knowledge discovery & data mining_. 1512–1520.
  * Zhao et al. (2019) X Zhao, L Xia, L Zou, D Yin, and J Tang. 2019.  _Simulating User Feedback for Reinforcement Learning Based Recommendations_.  Technical Report. Technical report.


# A Globally Stable Practically Implementable PI Passivity-based Controller
for Switched Power Converters

Alexey Bobtsov1, Romeo Ortega1,2, Nikolay Nikolaev1, Wei He*3

###### Abstract

In this paper we propose a PI passivity-based controller, applicable to a
large class of switched power converters, that ensures global state regulation
to a desired equilibrium point. A solution to this problem was reported in
[12] but it requires full state-feedback, which makes it practically
unfeasible. To overcome this limitation we construct a state observer that is
implementable with measurements that are available in practical applications.
The observer reconstructs the state in finite-time, ensuring global
convergence of the PI. The excitation requirement for the observer is very
weak and is satisfied in normal operation of the converters. Simulation
results illustrate the excellent performance of the proposed PI.

1Department of Control Systems and Robotics, ITMO University, Kronverkskiy av.
49, Saint Petersburg, 197101, Russia, (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
2Université Paris-Saclay, CNRS, CentraleSupélec, Laboratoire des signaux et
systèmes, 91190, Gif-sur-Yvette, France, (e-mail<EMAIL_ADDRESS>
3School of Automation, Nanjing University of Information Science & Technology,
Nanjing, China, (e-mail<EMAIL_ADDRESS>
##  1 Introduction

Switching power converters are, nowadays, an essential component of most
electrical engineering applications. The ever increasing performance
requirements on these applications translates into more stringent
specifications on the quality of the converters control. The dynamics of power
converters is highly nonlinear, even with fast switching when their averaged
model is valid, and the validity of their linear approximation is restricted
to a small neighborhood of the corresponding operating point [16]. The vast
majority of power converters are controlled with classical PI loops and there
is a widely accepted belief that, if they are properly tuned, their behavior
is acceptable [17, 30, 34]. The qualifier “if” in the previous sentence is
very important because, if the range of operation of the system to be
controlled is “wide”—as it is required in modern applications—the task of
tuning the gains of a PI (or, for that matter, of any other controller for
nonlinear systems) is far from obvious. Unfortunately, to date, the only
systematic procedure to carry-out this task, is invoking standard linear
systems theory arguments, e.g., pole location, stability margins, and applying
them to the linearized model of the converter. Various procedures to re-tune
the PI gains, including gain-scheduling [32], relay auto-tuning [5] and
adaptation [20], have been proposed but they all suffer from, well-documented,
serious limitations and drawbacks [6, 27].

The main objective of this paper is to propose a practically implementable PI
controller, applicable to a large class of power converters described by
average nonlinear models, which has the following properties.

  1. F1

Global stability of the closed-loop is guaranteed for all positive values of
the PI tuning gains.

  2. F2

By “practically implementable” we mean that it does not assume the
availability of the full state of the system, and it only requires the
measurements used in standard PIs.

  3. F3

The proposed PI has a very close connection with the widely popular PQ
instantaneous power controllers of [1].

Providing a theoretical framework for controller design is a topic of
paramount importance. On one hand, it allows power engineer practitioners to
apply the control law with confidence and, on the other hand, considerably
simplifies the commissioning stage, which now that stability is guaranteed for
all positive tuning gains, can concentrate on the transient performance
specifications.

The approach adopted in this paper is in the line of [22] which relies on the
use of energy balance concepts to control a system. Unlike most classical
nonlinear control techniques found in the literature, which try to impose some
predetermined dynamic behavior—usually through nonlinearity cancellation,
domination of nonlinearites and high gain—energy-based methods exploit and
respect the physical structure of the system. Passivity-based control (PBC) is
the generic name of this controller design methodology, which achieves
stabilization by exploiting the passivity properties of the system. Due to the
physical appealing that this methodology has, a vast literature has been
advocated to its application to mechanical, electrical and electromechanical
systems, see [9, 21].

Passivity is the key property of power converters that is exploited in this
paper. The first time that passivity principles were applied for power
converter control is in the foundational paper [28]. It is well-known [21, 31]
that wrapping a PI (or a PID) around a passive output ensures input-output
stability of the system and convergence to zero of the passive output. In many
applications, including power converters, the objective is to drive this
output to a value different from zero, which is associated with the steady-
state behavior of the system. In this case, we are interested in proving that
the incremental model is passive. In [25] it was shown that, for a general
class of models of power converters, it is possible to define an output signal
such that the incremental model is passive—this result was later generalized
in several directions in [12, 13]. This fundamental property, called
“passivity of the nonlinear incremental model” in [14], is now referred as
shifted passivity [31] and has played a central role in many recent
developments of the control community. In the present context the main
interest of this property is that, driving the shifted passive output to zero
with a PI control ensures, under a precise condition on the systems
dissipation, that the full state of the system converges to a desired
equilibrium.

The PI controller of [12] has enjoyed a very wide popularity—with more than
120 cites in Google Scholar. However, its main drawback is that it requires
the measurement of the state that, for technological and economic reasons, is
unattainable in most practical applications of power converters. The main
contribution of this note is to propose an observer that, using standard
measurements, reconstructs the system state in finite-time giving rise to a,
practically reasonable, globally stable PI design. Two additional features of
the observer are, on one hand, that convergence of the state estimates is
achieved under very weak excitation assumptions usually met in normal power
converter operation and, on the other hand, that it does not rely on noise-
sensitive high-gain injection or approximate differentiation.

The design of the observer relies on the use of the following two theoretical
developments recently reported by the authors.

  1. T1

Parameter Estimation-based Observer (PEBO) design technique first reported in
[23], whose main novelty is that it translates the task of state observation
into an on-line _parameter estimation_ problem. In this work we use a
generalization of this design, called GPEBO, recently used in [24] for the
state observation of chemical-biological reactors.

  2. T2

To estimate the parameters required by GPEBO, we use the Dynamic Regressor
Extension and Mixing (DREM) procedure first proposed in [3]. The central
property of DREM is that it generates, out of a $q$-dimensional Linear
Regression Equation (LRE), $q$ scalar LREs—one for each of the unknown
parameters—considerably simplifying the estimation task and ensuring
convergence under very weak excitation assumptions.

The remainder of the paper is organized as follows. In Section 2 we present
the mathematical model of the power converters we consider in the paper,
recall the state-feedback PI-PBC of [12] and formulate the observer problem,
for which we give a solution in Section 3. Simulation results, which
illustrate the performance of the proposed PI-PBC, are presented in Section
LABEL:sec4. The paper is wrapped-up with concluding remarks and future
research in Section LABEL:sec5.

Notation. $I_{n}$ is the $n\times n$ identity matrix. For
$x\in\mathbb{R}^{n}$, we denote the Euclidean norm $|x|^{2}:=x^{\top}x$. All
mappings are assumed smooth. Given a function $f:\mathbb{R}^{n}\to\mathbb{R}$
we define the differential operator $\nabla
f:=\left(\frac{\displaystyle\partial f}{\displaystyle\partial
x}\right)^{\top}$.

##  2 The PI-PBC of [12] and the Observer Problem Formulation

We consider in this paper switched power converters, with switched external
sources, described in port-Hamiltonian (pH) form [11, 31]

$\displaystyle\dot{x}=\left(J_{0}+\sum^{m}_{i=1}J_{i}u_{i}-R\right)\nabla
H(x)+\left(G_{0}+\sum^{m}_{i=1}G_{i}u_{i}\right)E,$ (1)

where $x(t)\in\mathbb{R}^{n}$, consisting of inductor fluxes and capacitor
charges, is the converter state, which is assumed unknown,
$u(t)\in\mathbb{R}^{m}$ is the duty ratio, $E\in\mathbb{R}^{n}$ is the vector
of constant external sources, $J_{i}=-J^{\top}_{i}\in\mathbb{R}^{n\times
n},\;i\in\bar{m}:=\\{0,\cdots,m\\}$, are the interconnection matrices,
$R=R^{\top}>0$ denotes the dissipation matrix,
$H:\mathbb{R}^{n}\to\mathbb{R}_{+}$ is the total stored energy, and
$G_{i}\in\mathbb{R}^{n\times n},\;i\in\bar{m}$, are the input matrices.
Assuming, linear capacitors and inductors, we have that

$H(x)=\frac{1}{2}x^{\top}Qx,$

with $Q\in\mathbb{R}^{n\times n},\;Q>0$, a diagonal matrix. We assume that all
the parameters of the converter are known. The model (1) describes the
behavior of most power converters used in practice [16, 17, 30, 34].

Control Objective Fix an admissible equilibrium point
${x^{\star}}\in\mathbb{R}^{n}$, that is, ${x^{\star}}$ satisfies

$\displaystyle 0$ $\displaystyle=$
$\displaystyle\left(J_{0}+\sum_{i=1}^{m}J_{i}u_{i}^{*}-R\right)Q{x^{\star}}+\left(G_{0}+\sum_{i=1}^{m}G_{i}u_{i}^{\star}\right)E,$
(2)

for some $u^{\star}\in\mathbb{R}^{m}$. Design a controller such that global
state regulation is achieved, namely, that for all system and controller
initial conditions,

$\lim_{t\to\infty}x(t)={x^{\star}},$ (3)

with all signals bounded.

###  2.1 The full-state feedback PI-PBC of [12]

The following full-state feedback PI solution to the aforementioned problem
was proposed in [12].

######  Proposition 1.

Consider the switched power converter described by (1) in closed loop with the
PI-PBC

$\displaystyle\dot{x}_{c}$ $\displaystyle=$ $\displaystyle\tilde{y}$
$\displaystyle u$ $\displaystyle=$ $\displaystyle-K_{P}\tilde{y}-K_{I}x_{c},$
(4)

with $K_{P}=K_{P}^{\top}\geq 0,\;,K_{I}=K_{I}^{\top}\geq 0$ and

$\tilde{y}={\cal C}(x-{x^{\star}}),$ (5)

with ${\cal C}\in\mathbb{R}^{m\times n}$ given by

${\cal
C}=\left[\begin{array}[]{c}E^{\top}G_{1}^{\top}-({x^{\star}})^{\top}QJ_{1}\\\
\vdots\\\
E^{\top}G_{m}^{\top}-({x^{\star}})^{\top}QJ_{m}\end{array}\right]Q=:G_{N}^{\top}(x^{\star})Q,$
(6)

For all initial conditions
$(x(0),x_{c}(0))\in\mathbb{R}^{n}\times\mathbb{R}^{m}$ the trajectories of the
closed-loop system are bounded and satisfy (3). $\Box\Box\Box$

The proof of Proposition 1 is established in [12] showing that the system (1)
in closed-loop with the PI-PBC (4) is described by

$\dot{\tilde{x}}=\left(J_{0}+\sum^{m}_{i=1}J_{i}u^{\star}_{i}-R\right)Q\tilde{x}+G_{N}(x)\tilde{u},$
(7)

where the matrix $G_{N}(\cdot)$ is defined in (6) and we (generically) use the
notation $\tilde{(\cdot)}:=(\cdot)-{(\cdot)^{\star}}$. Therefore, the Lyapunov
function candidate

$W(\tilde{x},\tilde{x}_{c})={1\over 2}\tilde{x}^{\top}Q\tilde{x}+{1\over
2}\tilde{x}_{c}^{\top}K_{I}\tilde{x}_{c}$

where $x_{c}^{\star}:=K_{I}^{-1}u^{\star}$, verifies

$\dot{W}=-\tilde{x}^{\top}QRQ\tilde{x}-\tilde{y}^{\top}K_{P}\tilde{y}.$

In [12] it is assumed that the dissipation matrix $R$ is only positive
semidefinite. Consequently, to ensure asymptotic stability, it is necessary to
impose a detectability condition that, as shown in [9, 12, 13, 19], is
satisfied in practical converter topologies. For ease of presentation, we
strengthen here the assumption on $R$, which is tantamount to making the
physically reasonable assumption that all energy storage elements—that is,
inductors and capacitors—are leaky.

######  Remark 1.

In [25] it is shown that. for a classical rectifier, one of the components of
the passive output $\tilde{y}$, defined in (5), is related with the difference
between supplied and extracted instantaneous active powers (of a suitably
scaled representation) of the rectifier, the second output being the component
of the input current. In this way, we make a nice connection with the widely
popular PQ instantaneous power controllers of [1], where an outer PI loop
around the output voltage is used to generate a reference for an inner PI loop
acting on the aforementioned power difference. It is also possible to show
that driving the passive output to zero can be reinterpreted as a power
equalization objective identical to the one used in Akagi’s PQ method.

###  2.2 State observation problem formulation

The main objective of this paper is to design a globally convergent state
observer that, combined with the PI-PBC of Proposition 1, will give rise to a
practically implementable version of it. To pose the observer design problem
we assume that there exists a full-rank matrix $C\in\mathbb{R}^{p\times
n},\;p<n$, such that the vector

$y_{m}=Cx,$ (8)

is available for measurement. Typically, this signal will be the voltage fed
to the converter, the input current flowing into it and/or the voltage
provided to the load [16]. It will be shown below that—as expected—the level
of excitation required to ensure convergence of the observer will reduce if
the number of measurements increases—see Remark LABEL:rem3.

Finite-convergence Time (FCT) Observer Problem Formulation Consider the power
converter (1) with known parameters and measurable output signals (8). Design
a dynamical system

with $\chi\in\mathbb{R}^{n_{\chi}}$, such that, for all initial conditions
$(x(0),\chi(0))\in\mathbb{R}^{n+n_{\chi}}$, all signals remain bounded and
there exists $t_{c}\in[0,\infty)$ such that

$\hat{x}(t)=x(t),\;\forall t\geq t_{c}.$

######  Remark 2.

Following standard practice in observer theory [7] we assume that the the
control $u$ is such that the state trajectories of (1) are bounded.

##  3 Observer Design

In this section we present the main result of the paper, namely an FCT
GPEBO+DREM to reconstruct the state of the system (1) with the measurable
outputs (8). The excitation assumption for global asymptotic convergence is
very week and, as illustrated in the simulations in Section LABEL:sec5, is
satisfied in normal operating conditions of the converters. We also present
other asymptotically convergent observers, which have a below par performance
with respect to the GPEBO.

###  3.1 GPEBO+DREM design with FCT

To streamline the presentation of the result we define the matrix

$\Lambda(u):=\left(J_{0}+\sum^{m}_{i=1}J_{i}u_{i}-R\right)Q,$ (9)

and make the following sufficient excitation condition [18].

######  Assumption 1.

Fix a small constant $\mu\in(0,1)$. There is a time $t_{c}>0$ such that

$\int^{t_{c}}_{0}\Delta^{2}(\tau)d\tau\geq-\frac{1}{\gamma}\ln(1-\mu).$ (10)

######  Proposition 2.

Consider the system (1) with the measurable outputs (8). Define the GPEBO via

Figure 2: Transients of $i_{1}-\hat{i}_{1}$ of the FCT-GPEBO for different
adaptation gains Figure 3: Transients of $\upsilon_{2}-\hat{\upsilon}_{2}$ of
the FCT-GPEBO for different adaptation gains Figure 4: Transients of
$i_{3}-\hat{i}_{3}$ of the FCT-GPEBO for different adaptation gains Figure 5:
Transients of $u_{4}$ of the FCT-GPEBO for step change in reference signal and
load

#### Comparison of the performance of FCT-GPEBO with other observers

Second, to compare the performance of the FCT-GPEBO of Proposition 2 with
other observers, we present now simulations of the output-feedback PI-PBC
where we replaced this observer by one of the observers discussed in
Subsection LABEL:subsec32 and the asymptotic GPEBO of Remark LABEL:rem5. For
all observers, the initial conditions of the converter were taken as
$x(0)=\mbox{col}(0.75L_{1},15C_{1},-1.5L_{2},-18C_{2})$. For the PEBOs we used
$\lambda=5$ and $\gamma=10^{12}$. For the KBF we used $S=I_{4\times 4}$ and
$H(0)=I_{4\times 4}$. For the classical gradient estimator we set to zero
$\hat{\theta}(0)$ and used $\gamma=10^{8}$. For the asymptotic GPEBO we used
$\gamma=10^{17}$.

Figs. 6-8 show the transients of the state estimation errors for the following
estimation methods: FCT-GPEBO, asymptotic GPEBO, open-loop emulator, KBF and
GPEBO with classical gradient estimator. The following observations are in
order.

  1. O1

As predicted by the theory all state estimation errors of FCT-GPEBO converge
in finite time with $t_{c}\approx 0.03$s, and the performance of this observer
is superior to all the other ones.

  2. O2

The emulator and the KBF have almost identical behavior. Actually, in the
plots the difference is indistinguishable. The performance of the KBF didn’t
change with other choices of $S$.

  3. O3

The asymptotic GPEBO outperforms all other observers, except the FCT-GPEBO.

  4. O4

The GPEBO plus gradient also converges, but has an erratic behavior, whose
origin we fail to understand.

Figure 6: Transients of $i_{1}$ estimation errors for diffrerent algorithms
Figure 7: Transients of $\upsilon_{2}$ estimation errors for diffrerent
algorithms Figure 8: Transients of $i_{3}$ estimation errors for diffrerent
algorithms

#### Performance of the PI-PBC with full-state measurement and FCT-GPEBO

Second, we compare the performance of the full-state feedback PI-PBC with the
one using the observed state. We look in this case at the effect of the
initial conditions. For simulations we considered the initial conditions:
Figure 11: The response curves of $x_{4}$ of the closed-loop system under
traditional PI with $K_{P}=0.008,K_{I}=8$ and the output-feedback PI-PBC, with
$x_{4}^{\star}=-15$

It may be argued that the response of the classical PI can be improved re-
tuning the gains. The usual procedure to improve the transient performance of
a PI is to increase the integral gain $K_{I}$, see [34]. This is shown in Fig.
3.1 where the transient performance is, indeed, improved but is still inferior
to the proposed PI-PBC.

Figure 12: The response curves of $x_{4}$ of the closed-loop system under
traditional PI with $K_{P}=0.008$ and different values of $K_{I}$, with
$x_{4}^{\star}=-15$

##  6 Conclusions an Future Work

We have provided a solution to the main drawback of the popular PI-PBC of
[12], namely the need to measure the full state, by proposing a globally
convergent state observer. The convergence of this observer occurs in finite
time and requires only a very weak excitation assumption.

In view of the high uncertainty on the converter parameters—in particular, the
variations of the load—our current research focuses on the development of
adaptive versions of the observer and the PI-PBC. Some results for particular
converter topologies are already available and will be reported elsewhere.

We are also working on the experimental implementation of the proposed PI-PBC
for a Ćuk and a quadratic converter, that we expect to have ready for the
final version of this paper.

## References

  * [1] H. Akagi Instantaneous Power Theory and Applications to Power Conditioning, Wiley, Newark, 2007.
  * [2] B. Anderson, Stability properties of Kalman-Bucy filters, Journal of the Franklin Institute, vol. 291, no. 2, pp. 137-144, 1971.
  * [3] S. Aranovskiy, A. Bobtsov, R. Ortega, A. Pyrkin, Performance enhancement of parameter estimators via dynamic regressor extension and mixing, IEEE Trans. Automatic Control, vol. 62, pp. 3546-3550, 2017.
  * [4] A. Astolfi, D. Karagiannis and R. Ortega, Nonlinear and Adaptive Control with Applications, Springer-Verlag, Berlin, Communications and Control Engineering, 2008.
  * [5] K. J. Astrom and T. Hagglund, Automatic Tuning of PID Controllers, Instrument Society of America, NC, USA, 1988.
  * [6] K. J. Astrom and R. M. Murray, Feedback Systems An Introduction for Scientists and Engineers, 2nd edition, Princeton University Press, 2019.
  * [7] P. Bernard, Observer Design for Nonlinear Systems, Springer, Vol. 479, 2019.
  * [8] G. Chowdhary, T. Yucelen, M. Muhlegg and E. Johnson, Concurrent learning adaptive control of linear systems with exponentially convergent bounds, Int. J. on Adaptive Control and Signal Processing, vol. 27, no. 4, pp. 280-301, 2013.
  * [9] R. Cisneros-Montoya, PI Passivity-Based Control: Application to Physical Systems, Ph D. University, Paris-Saclay, HAL Id. https://tel.archives-ouvertes.fr/tel-01368308, 2016.
  * [10] D. Dochain, State and parameter estimation in chemical and biochemical processes: a tutorial, Journal of Process Control, vol. 13, no. 8, pp. 801-818, 2003.
  * [11] G. Escobar, A. van der Schaft and R. Ortega, A Hamiltonian viewpoint in the modeling of switching power converters, Automatica, vol. 35, pp. 445-452, 1999.
  * [12] M. Hernandez-Gomez, R. Ortega, F. Lamnabhi-Lagarrigue and G. Escobar: Adaptive PI stabilization of switched power converters, IEEE Trans. Control Syst. Technology, vol. 18, no. 3, pp. 688-698, 2010.
  * [13] M. Hernandez-Gomez, R. Ortega, F. Lamnabhi-Lagarrigue and G. Escobar, Adaptive PI stabilization of switched power converters described by port-Hamiltonian models, Dynamics and Control of Switched Electronic Systems, eds. F. Vasca and L. Iannelli, Advances in Industrial Control, Springer, Heidelberg, pp. 355-390, 2012.
  * [14] B. Jayawardhana, R. Ortega, E. Garcia-Canseco and F. Castaños, Passivity of nonlinear incremental systems: Application to PI stabilization of nonlinear RLC circuits, Systems and Control Letters, vol. 56, pp.618-622, 2007.
  * [15] R. Kamalapurkar, B. Reish, G. Chowdhary and W. Dixon, Concurrent learning for parameter estimation using dynamic state-derivative estimators, IEEE Trans. Automatic Control, vol. 62, no. 7, pp. 3594-3601, 2017.
  * [16] J. Kassakian, M. Schlecht and G. Verghese, _Principles of Power Electronics_ , Addison-Wesley, Reading, 1991.
  * [17] M.P. Kazmierkowski, R. Krishnan, F. Blaabjerg and J.D. Irwin, Control in Power Electronics: Selected Problems, Academic Press Series in Engineering. Elsevier Science, 2002.
  * [18] G. Kreisselmeier and G. Rietze-Augst, Richness and excitation on an interval—with application to continuous-time adaptive control, IEEE Trans. Automatic Control, vol. 35, no. 2, pp. 165-171, 1990.
  * [19] C. Meza, D. Biel, D. Jeltsema and J. M. A. Scherpen, Lyapunov-based control scheme for single-phase grid-connected PV central inverters, IEEE Trans. Control Systems Technology, vol. 20, no. 2, pp. 520-529, 2011.
  * [20] R. Ortega and R. Kelly, PID self-tuners: some theoretical and practical aspects, IEEE Trans. on Ind. Electr., vol. 31, no.4, 1984.
  * [21] R. Ortega, A. Loria, P. J. Nicklasson and H. Sira–Ramirez, Passivity–based control of Euler-Lagrange systems, Springer-Verlag, Berlin, Communications and Control Engineering, 2014.
  * [22] R. Ortega, A. J. van der Schaft, I. Mareels, and B. Maschke, Putting energy back in control, IEEE Control Systems Magazine, vol. 21, no. 2, pp. 18-33, 2001.
  * [23] R. Ortega, A. Bobtsov, A. Pyrkin and A. Aranovskyi, A parameter estimation approach to state observation of nonlinear systems, Systems and Control Letters, Vol. 85, pp 84-94, 2015.
  * [24] R. Ortega, A. Bobtsov, D. Dochain and N. Nikolayev, State observers for reaction systems with improved convergence rate, Journal of Process Control, Vol. 83, pp. 53-62, 2019.
  * [25] M. Perez, R. Ortega and J. Espinoza, Passivity-based PI control of switched power converters, IEEE Transactions on Control Systems Technology, vol. 12, pp. 881-890, 2004.
  * [26] W.J. Rugh, Linear Systems Theory, 2nd Edition, Prentice hall, NJ, 1996.
  * [27] W.J. Rugh and J. Shamma, Research on gain scheduling, Automatica, vol. 36, no. 10, pp. 1401-1425, 2000.
  * [28] S. Sanders and G. Verghese, Lyapunov-based control for switched power converters, IEEE Trans. Power Electronics, vol. 7, No. 1, pp. 17-24, 1992.
  * [29] S. Sastry and M. Bodson, Adaptive Control: Stability, Convergence and Robustness, Prentice Hall, Englewood Cliffs, NJ, 1989.
  * [30] H. Sira-Ramirez amd R. Silva-Ortigoza, Control Design Techniques in Power Electronics Devices, Springer-Verlag, Berlin, Communications and Control Engineering, 2006.
  * [31] A. van der Schaft, $L_{2}$–Gain and Passivity Techniques in Nonlinear Control, Springer, Berlin, 3rd Edition, 2016.
  * [32] V. Vesely and A. Ilka, Gain-scheduled PID controller design. Journal of Process Control, vol. 23, pp.1141-1148, 2013.
  * [33] M. Vidyasagar, Decomposition techniques for large-scale systems with non-additive interactions: stability and stabilizability, IEEE Trans. Automatic Control, vol. 25, no. 4, pp. 773-779, 1980.
  * [34] L. Wang, S. Chai, D. Yoo, L. Gan and K. Ng, PID and Predictive Control of Electrical Drives and Power Converters using MATLAB/Simulink, First Edition, John Wiley & Sons Singapore, 2015.