# EnDSUM: Entropy and Diversity based Disaster Tweet Summarization

Piyush Kumar Garg  IIT PatnaIndia<EMAIL_ADDRESS>,  Roshni
Chakraborty  Aalborg UniversityDenmark<EMAIL_ADDRESS>and  Sourav Kumar
Dandapat  IIT PatnaIndia<EMAIL_ADDRESS>
###### Abstract.

The huge amount of information shared in Twitter during disaster events are
utilized by government agencies and humanitarian organizations to ensure quick
crisis response and provide situational updates. However, the huge number of
tweets posted makes manual identification of the relevant tweets impossible.
To address the information overload, there is a need to automatically generate
summary of all the tweets which can highlight the important aspects of the
disaster. In this paper, we propose an entropy and diversity based summarizer,
termed as EnDSUM, specifically for disaster tweet summarization. Our
comprehensive analysis on $6$ datasets indicates the effectiveness of EnDSUM
and additionally, highlights the scope of improvement of EnDSUM.

Entropy, Disaster tweets, Social media, Summarization

##  1\. Introduction

Social media platforms, like Twitter, are highly important mediums of
information during disasters. For example, humanitarian organizations and
government agencies rely on Twitter to identify relevant information on
different categories, such as affected population, urgent need of resources,
infrastructure damage, etc (Imran and Castillo, 2015). However, the huge
number of tweets posted and the high vocabulary diversity (Castillo, 2016;
Chakraborty et al., 2018) make it a challenging to manually find the relevant
information (Vieweg et al., 2014; Imran et al., 2015). In order to address
this issue, several research works (Rudra et al., 2015; Dutta et al., 2018)
have proposed specific tweet summarization approaches for disaster events.

Existing disaster tweet summarization approaches could be segregated into
content based (Rudra et al., 2015), graph based (Dutta et al., 2018), deep
learning based (Dusart et al., 2021), and ontology based (Garg et al., 2022)
approaches on the basis of the mechanism they follow. While content based
approaches (Rudra et al., 2015, 2018a) rely on only the importance of the
words present in a tweet to determine its selection to the summary, deep
learning based approaches (Dusart et al., 2021) consider both content and
contextual importance of the tweet. However, none of these approaches consider
the vocabulary diversity and therefore, fails to always ensure diversity in
summary and coverage of all the important categories present in the tweets. In
order to address these, graph based approaches (Dutta et al., 2018, 2015)
initially group similar tweets together such that each group represents a
category by community detection algorithms, thereby handling the vocabulary
diversity followed by selecting representative tweets from each group to
create the summary to ensure coverage. However, automatic community detection
algorithms fails to automatically segregate the tweets into different
categories due to the vocabulary overlap among tweets of different categories.
Therefore, Garg et al. (Garg et al., 2022) initially identify the category of
each tweet by an ontology based approach and then, select tweets from each
category to generate the summary. However, none of these approaches try to
handle the vocabulary diversity simultaneously while selecting the tweets into
the summary. For example, these existing approaches are dependent on
identifying the categories initially which lead to bad summaries, such as
reduced diversity in summary, if the categories are not identified correctly.

In order to resolve this, we propose EnDSUM, an entropy and diversity based
disaster summarizer where we automatically select that tweet into summary
which provides the best information coverage of all the tweets, i.e., entropy
and most novel information, i.e., diversity. Therefore, EnDSUM can generate
the summary automatically without explicitly identifying the category of a
tweet. Although there are few single and multiple document summarization
approaches (Khurana and Bhatnagar, 2022; Feigenblat et al., 2017; Aji and
Kaimal, 2012; Luo et al., 2010; Ravindra et al., 2004) that have highlighted
the relevance of entropy based selection of sentences into summary, those
approaches are not directly applicable to disaster tweets. The reason being
the informal structure of tweets, absence of storyline in tweets and the high
vocabulary diversity in user generated tweets. Our evaluation of EnDSUM with
existing state-of-the-art disaster tweet summarization approaches on $6$
different disasters shows its high effectiveness on $5$ datasets. However, we
observe that the performance of EnDSUM degrades when there is considerable
vocabulary overlap among the tweets which belong to different categories of
the same disaster event. The reason being we consider only content based
information for calculation of entropy and diversity. The organization of the
paper is as follows. We discuss problem definition and proposed approach in
Section 2 followed by the experiment details in Section 3 and conclusions in
Section 4.

Table 1. F1-score of ROUGE-1, ROUGE-2 and ROUGE-L score of EnDSUM and
baselines on $6$ datasets is shown.

Dataset | Approaches | ROUGE-1 | ROUGE-2 | ROUGE-L | Dataset | Approaches | ROUGE-1 | ROUGE-2 | ROUGE-L
---|---|---|---|---|---|---|---|---|---
|  | F1-score | F1-score | F1-score |  |  | F1-score | F1-score | F1-score
| $EnDSUM$ | 0.55 | 0.21 | 0.27 |  | $EnDSUM$ | 0.51 | 0.16 | 0.24
| $B_{1}$ | 0.49 | 0.22 | 0.29 |  | $B_{1}$ | 0.20 | 0.04 | 0.20
${D_{1}}$ | $B_{2}$ | 0.48 | 0.18 | 0.25 | ${D_{4}}$ | $B_{2}$ | 0.47 | 0.14 | 0.21
| $B_{3}$ | 0.52 | 0.21 | 0.23 |  | $B_{3}$ | 0.45 | 0.11 | 0.21
| $B_{4}$ | 0.56 | 0.23 | 0.29 |  | $B_{4}$ | 0.50 | 0.15 | 0.23
| $EnDSUM$ | 0.52 | 0.17 | 0.24 |  | $EnDSUM$ | 0.52 | 0.13 | 0.24
| $B_{1}$ | 0.48 | 0.13 | 0.22 |  | $B_{1}$ | 0.19 | 0.04 | 0.18
${D_{2}}$ | $B_{2}$ | 0.47 | 0.14 | 0.22 | ${D_{5}}$ | $B_{2}$ | 0.48 | 0.10 | 0.20
| $B_{3}$ | 0.44 | 0.12 | 0.22 |  | $B_{3}$ | 0.50 | 0.12 | 0.22
| $B_{4}$ | 0.49 | 0.15 | 0.23 |  | $B_{4}$ | 0.51 | 0.13 | 0.22
| $EnDSUM$ | 0.52 | 0.14 | 0.26 |  | $EnDSUM$ | 0.55 | 0.27 | 0.44
| $B_{1}$ | 0.45 | 0.13 | 0.23 |  | $B_{1}$ | 0.53 | 0.26 | 0.33
${D_{3}}$ | $B_{2}$ | 0.46 | 0.14 | 0.24 | ${D_{6}}$ | $B_{2}$ | 0.52 | 0.22 | 0.29
| $B_{3}$ | 0.44 | 0.14 | 0.23 |  | $B_{3}$ | 0.48 | 0.20 | 0.27
| $B_{4}$ | 0.48 | 0.16 | 0.25 |  | $B_{4}$ | 0.51 | 0.20 | 0.29

##  2\. Proposed Approach

Given a disaster event, $E$, that consists of $m$ tweets, $T$ =
{$T_{1},T_{2},...,T_{m}\\}$, we aim to prepare a summary, $S$, by selecting
$L$ tweets from $T$ such that it provides the maximum information coverage
from $T$ with minimum redundant information in the final summary. Therefore,
we propose EnDSUM where we iteratively selecting the tweet that can ensure the
maximum entropy of all the tweets and maximum diversity in summary. While
selection of the tweet with maximum entropy ensures information coverage of a
category, selection of the tweet with the maximum diversity ensures not
multiple tweets from the same category are selected (Chakraborty et al., 2019,
2017). Therefore, at every iteration, we select the tweet ($T^{*}$), which has
the maximum score by Equation 1.

(1) $\displaystyle T^{*}=\operatorname{\operatorname*{arg\,max}}\sum_{T_{i}\in
T}\alpha*E(T_{i})+\beta*D(T_{i},S^{{}^{\prime}})$

where, $E(T_{i})$ represents the entropy of tweet, $T_{i}$, and
$D(T_{i},S^{{}^{\prime}})$ represents the information diversity provided by
$T_{i}$ with respect to the already selected tweets in summary,
$S^{{}^{\prime}}$. $\alpha$ and $\beta$ are the tunable parameters which
represent the importance of $E(T_{i})$ and $D(T_{i},S^{{}^{\prime}})$
respectively. We consider $\alpha$ and $\beta$ as $0.5$ to provide equal
importance to both entropy and diversity. Although there are several available
mechanisms to calculate $E(T_{i})$, we rely on Karci Entropy (Hark and Karcı,
2020) for EnDSUM. Karci Entropy can resolve the inherent vocabulary diversity
in disaster tweets as it calculates the entropy of a tweet, $E(T_{i})$, by
considering the similarity of $T_{i}$ with the other tweets as shown in
Equation 2.

(2) $\displaystyle E(T_{i},K)=\sum_{j=1}^{\lvert K\rvert}|-P_{ij}^{\gamma}\log
P_{ij}|,\ 0<\gamma$

where, $\gamma$ represents the importance of similarity. We consider $\gamma$
as $0.5$ as highlighted by Hark et al. (Hark and Karcı, 2020). $K$ is the list
of similar tweets of $T_{i}$, where a tweet is said to be similar to $T_{i}$
if the content based cosine similarity, i.e. $P_{ij}$ between them is higher
than $0$ (as shown in (Hark and Karcı, 2020)) and $P_{ij}$ is the normalized
number of overlapping between $T_{i}$ and $T_{j}$ normalized by the total
number of overlapping keywords of $T_{i}$ with any tweet. We calculate
$D(T_{i},S^{{}^{\prime}})$ as (1-$Sim(T_{i},S^{{}^{\prime}})$) where
$Sim(T_{i},S^{{}^{\prime}})$ represents the overlap in keywords between
$T_{i}$ and $S^{{}^{\prime}}$ by

(3) $\displaystyle Sim(T_{i},S^{{}^{\prime}})=\sum_{k\in
S^{{}^{\prime}}}\frac{Overlap(T_{i},T_{k})}{Length(T_{i})}$

where, $Length(T_{i})$ is the number of keywords of $T_{i}$. We follow Khan et
al. (Khan et al., 2013) to identify the keywords of $T_{i}$ as the nouns,
verbs, adjectives present in $T_{i}$ and similarly, for $S^{{}^{\prime}}$, we
consider the distinct set of nouns, verbs, adjectives present in all the
tweets of $S^{{}^{\prime}}$. Therefore, a lower $Sim(T_{i},S^{{}^{\prime}})$
ensures $T_{i}$ has minimum redundant content information with respect to
already generated summary, $S^{{}^{\prime}}$, and a higher $E(T_{i})$ ensures
$T_{i}$ has higher information coverage of the category.

##  3\. Experiments and Results

In this Section, we provide details of the experiment and results. For the
datasets, we consider Los Angeles International Airport Shooting
111https://en.wikipedia.org/wiki/2013_Los_Angeles_International_Airport_shooting
($D_{1}$) provided by Olteanu et al. (Olteanu et al., 2015), Hurricane Matthew
222https://en.wikipedia.org/wiki/Hurricane_Matthew ($D_{2}$), Puebla Mexico
Earthquake 333https://en.wikipedia.org/wiki/2017_Puebla_earthquake ($D_{3}$),
Pakistan Earthquake 444https://en.wikipedia.org/wiki/2019_Kashmir_earthquake
($D_{4}$) and Midwestern U.S. Floods
555https://en.wikipedia.org/wiki/2019_Midwestern_U.S._floods ($D_{5}$)
provided by Alam et al. (Alam et al., 2021) and Sandy Hook Elementary School
Shooting
666https://en.wikipedia.org/wiki/Sandy_Hook_Elementary_School_shooting
($D_{6}$) provided by Dutta et al. (Dutta et al., 2018). We perform
lemmatization, convert to lower case and remove of Twitter specific keywords
(Arachie et al., 2020) and retweets as pre-processing. We consider the ground
truth summary provided by Garg et al. (Garg et al., 2022) for $D_{1}$-$D_{5}$
and by Dutta et al. (Dutta et al., 2018) for $D_{6}$. We compare EnDSUM with
content based (Rudra et al., 2019) ($B_{1}$), graph based (Dutta et al., 2018)
($B_{2}$), sub-event based (Rudra et al., 2018b) ($B_{3}$) and ontology based
(Garg et al., 2022) ($B_{4}$) disaster summarization approaches.

Results and Discussion : We evaluate the performance of EnDSUM and the
existing research with the ground truth summary using ROUGE-N (Lin, 2004)
F1-score score when N=$1$, $2$, and L. Our observations from Table 1 indicate
that EnDSUM ensures better ROUGE-N F1-score over all baselines for
$D_{2}$-$D_{6}$. The improvement is highest over $B_{1}$ baseline and lowest
over $B_{4}$ baseline. EnDSUM performs worse than $B_{4}$ for Rouge-N scores
and worse than $B_{1}$ for Rouge-2 and Rouge-L scores on $D_{1}$. Therefore,
although EnDSUM has highly effective performance in most scenarios, it
sometimes fails to resolve the vocabulary overlap across different categories
in a disaster, as seen for $D_{1}$. Therefore, to resolve this, we are working
towards making EnDSUM resilient irrespective of the vocabulary diversity by
considering semantic and contextual similarity along with the already
considered content similarity for entropy and diversity calculation.

##  4\. Conclusions and Future Works

In this paper, we propose a novel entropy and diversity based tweet
summarizer, EnDSUM for disaster events. Our experimental analysis on $6$
disaster datasets indicates both the effectiveness of EnDSUM and its scope of
improvement. For example, to handle the the high vocabulary overlap among
categories, we are working to both include semantic and contextual similarity
while calculating entropy and diversity in EnDSUM. Furthermore, while most
summarization algorithms generate a predefined length summary, we intend to
extend EnDSUM such that it provides complete information coverage of the
disaster event. For example, intuitively, the summary length varies on the
basis of the information diversity in a disaster event, therefore a summary of
length less than the required length leads to less information coverage
whereas a summary with more number of tweets than the required will reduce the
information diversity. We believe by incorporating these changes in EnDSUM, it
would provide an effective performance irrespective of the disaster.

## References

  * (1)
  * Aji and Kaimal (2012) Subhanpurno Aji and Ramachandra Kaimal. 2012.  Document summarization using positive pointwise mutual information.  _AIRCC’s International Journal of Computer Science and Information Technology_ 4, 2 (2012), 47–55.
  * Alam et al. (2021) Firoj Alam, Umair Qazi, Muhammad Imran, and Ferda Ofli. 2021\.  HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks.  _arXiv preprint arXiv:2104.03090_ (2021).
  * Arachie et al. (2020) Chidubem Arachie, Manas Gaur, Sam Anzaroot, William Groves, Ke Zhang, and Alejandro Jaimes. 2020\.  Unsupervised detection of sub-events in large scale disasters. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 354–361.
  * Castillo (2016) Carlos Castillo. 2016\.  _Big crisis data: social media in disasters and time-critical situations_.  Cambridge University Press.
  * Chakraborty et al. (2017) Roshni Chakraborty, Maitry Bhavsar, Sourav Dandapat, and Joydeep Chandra. 2017.  A network based stratification approach for summarizing relevant comment tweets of news articles. In _International Conference on Web Information Systems Engineering_. Springer, 33–48.
  * Chakraborty et al. (2019) Roshni Chakraborty, Maitry Bhavsar, Sourav Kumar Dandapat, and Joydeep Chandra. 2019.  Tweet summarization of news articles: An objective ordering-based perspective.  _IEEE Transactions on Computational Social Systems_ 6, 4 (2019), 761–777.
  * Chakraborty et al. (2018) Roshni Chakraborty, Abhijeet Kharat, Apalak Khatua, Sourav Kumar Dandapat, and Joydeep Chandra. 2018.  Predicting Tomorrow’s Headline using Twitter Deliberations.. In _CIKM Workshops_.
  * Dusart et al. (2021) Alexis Dusart, Karen Pinel-Sauvagnat, and Gilles Hubert. 2021\.  TSSuBERT: Tweet Stream Summarization Using BERT.  _arXiv preprint arXiv:2106.08770_ (2021).
  * Dutta et al. (2018) Soumi Dutta, Vibhash Chandra, Kanav Mehra, Asit Kumar Das, Tanmoy Chakraborty, and Saptarshi Ghosh. 2018.  Ensemble algorithms for microblog summarization.  _IEEE Intelligent Systems_ 33, 3 (2018), 4–14.
  * Dutta et al. (2015) Soumi Dutta, Sujata Ghatak, Moumita Roy, Saptarshi Ghosh, and Asit Kumar Das. 2015.  A graph based clustering technique for tweet summarization. In _2015 4th international conference on reliability, infocom technologies and optimization (ICRITO)(trends and future directions)_. IEEE, 1–6.
  * Feigenblat et al. (2017) Guy Feigenblat, Haggai Roitman, Odellia Boni, and David Konopnicki. 2017.  Unsupervised query-focused multi-document summarization using the cross entropy method. In _Proceedings of the 40th International ACM SIGIR Conference on research and development in information retrieval_. 961–964.
  * Garg et al. (2022) Piyush Kumar Garg, Roshni Chakraborty, and Sourav Kumar Dandapat. 2022\.  OntoRealSumm: Ontology based Real-Time Tweet Summarization.  _arXiv preprint arXiv:2201.06545_ (2022).
  * Hark and Karcı (2020) Cengiz Hark and Ali Karcı. 2020.  Karcı summarization: A simple and effective approach for automatic text summarization using Karcı entropy.  _Information Processing & Management_ 57, 3 (2020), 102187\.
  * Imran and Castillo (2015) Muhammad Imran and Carlos Castillo. 2015.  Towards a data-driven approach to identify crisis-related topics in social media streams. In _Proceedings of the 24th International Conference on World Wide Web_. 1205–1210.
  * Imran et al. (2015) Muhammad Imran, Carlos Castillo, Fernando Diaz, and Sarah Vieweg. 2015.  Processing social media messages in mass emergency: A survey.  _ACM Computing Surveys (CSUR)_ 47, 4 (2015), 1–38.
  * Khan et al. (2013) Muhammad Asif Hossain Khan, Danushka Bollegala, Guangwen Liu, and Kaoru Sezaki. 2013.  Multi-tweet summarization of real-time events. In _2013 International Conference on Social Computing_. IEEE, 128–133.
  * Khurana and Bhatnagar (2022) Alka Khurana and Vasudha Bhatnagar. 2022.  Investigating Entropy for Extractive Document Summarization.  _Expert Systems with Applications_ 187 (2022), 115820.
  * Lin (2004) Chin-Yew Lin. 2004\.  Rouge: A package for automatic evaluation of summaries. In _Text summarization branches out_. 74–81.
  * Luo et al. (2010) Wenjuan Luo, Fuzhen Zhuang, Qing He, and Zhongzhi Shi. 2010\.  Effectively leveraging entropy and relevance for summarization. In _Asia Information Retrieval Symposium_. Springer, 241–250.
  * Olteanu et al. (2015) Alexandra Olteanu, Sarah Vieweg, and Carlos Castillo. 2015\.  What to expect when the unexpected happens: Social media communications across crises. In _Proceedings of the 18th ACM conference on computer supported cooperative work & social computing_. 994–1009.
  * Ravindra et al. (2004) G Ravindra, N Balakrishnan, and KR Ramakrishnan. 2004\.  Multi-document automatic text summarization using entropy estimates. In _International Conference on Current Trends in Theory and Practice of Computer Science_. Springer, 289–300.
  * Rudra et al. (2018a) Koustav Rudra, Niloy Ganguly, Pawan Goyal, and Saptarshi Ghosh. 2018a.  Extracting and summarizing situational information from the twitter social media during disasters.  _ACM Transactions on the Web (TWEB)_ 12, 3 (2018), 1–35.
  * Rudra et al. (2015) Koustav Rudra, Subham Ghosh, Niloy Ganguly, Pawan Goyal, and Saptarshi Ghosh. 2015.  Extracting situational information from microblogs during disaster events: a classification-summarization approach. In _Proceedings of the 24th ACM International on Conference on Information and Knowledge Management_. 583–592.
  * Rudra et al. (2019) Koustav Rudra, Pawan Goyal, Niloy Ganguly, Muhammad Imran, and Prasenjit Mitra. 2019.  Summarizing situational tweets in crisis scenarios: An extractive-abstractive approach.  _IEEE Transactions on Computational Social Systems_ 6, 5 (2019), 981–993.
  * Rudra et al. (2018b) Koustav Rudra, Pawan Goyal, Niloy Ganguly, Prasenjit Mitra, and Muhammad Imran. 2018b.  Identifying sub-events and summarizing disaster-related information from microblogs. In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_. 265–274.
  * Vieweg et al. (2014) Sarah Vieweg, Carlos Castillo, and Muhammad Imran. 2014\.  Integrating social media communications into the rapid assessment of sudden onset disasters. In _International Conference on Social Informatics_. Springer, 444–461.


$\displaystyle\leq\left(2p-6c-2\right)-M\cdot\left(p+1\right).$

We rearrange this as

$M\cdot\left(p+1\right)\leq 2p-6c-2<p+6c-6c-2=p-2,$

and notice that the only nonnegative integer $M$ satisfying this is $M=0$.

  2. (2)

Using part (1), we calculate the dimension of $Q^{2p-1}$ as

$\displaystyle\dim{Q}^{2p-1}$
$\displaystyle=\dim{M_{1,c}^{2p-1}(\mathrm{Stand})/\left<v_{p-3c},v_{+},v_{-},v_{p+3c}\right>}-$
$\displaystyle\quad-K\cdot\dim{L_{1,c}^{p-3c-1}(\mathrm{Sign})}-N\cdot\dim{L_{1,c}^{3c-1}(\mathrm{Triv})}$
$\displaystyle=\left(2\cdot
2p-(2p-1-p+3c+1)-2(2p-1-p+1)-(2p-1-p-3c+1)\right)-$
$\displaystyle\quad-K\cdot(p-3c-1+1)-N\cdot(3c-1+1)$
$\displaystyle=-K\cdot(p-3c)-N\cdot 3c.$

Using that $\dim{Q}^{2p-1}\geq 0$, $p-3c>0$ and $3c>0$, the only nonnegative
$K,N$ satisfying this are $K=N=0.$

  3. (3)

This follows immediately from (1) and (2).

∎

###  7.4. Irreducible representation $L_{1,c}(\mathrm{Stand})$, part 2

######  Conjecture 7.25.

The irreducible representation $L_{1,c}(\mathrm{Stand})$ of the rational
Cherednik algebra $H_{1,c}(S_{3},\h)$ over an algebraically closed field of
characteristic $p>3$ for $c\in\mathbb{F}_{p}$, $p/3<c<p/2$, is the quotient of
the Verma module $M_{1,c}(\mathrm{Stand})$ by the submodule generated by
vectors in the following degrees:

  * •

$-p+3c$ \- one dimensional space, of type $\mathrm{Sign}$

  * •

$3p-3c$ \- one dimensional space, of type $\mathrm{Triv}$

  * •

$p+3c$ \- one dimensional space, of type $\mathrm{Sign}$

  * •

$5p-3c$ \- one dimensional space, of type $\mathrm{Triv}$

Its character and Hilbert polynomial are

$\displaystyle\chi_{L_{1,c}(\mathrm{Stand})}(z)$
$\displaystyle=\chi_{S\h^{*}}(z)\left([\mathrm{Stand}]-z^{-p+3c}[\mathrm{Sign}]-z^{3p-3c}[\mathrm{Triv}]-\right.$
$\displaystyle\quad\quad\left.-z^{p+3c}[\mathrm{Sign}]-z^{5p-3c}[\mathrm{Triv}]+z^{4p}[\mathrm{Stand}]\right)$
$\displaystyle h_{L_{1,c}(\mathrm{Stand})}(z)$
$\displaystyle=\frac{2-z^{-p+3c}-z^{3p-3c}-z^{p+3c}-z^{5p-3c}+2z^{4p}}{(1-z)^{2}}.$

Compare this with Lemma 3.19, where an analogous result is proved for $p=2$.
Notice also that the vectors $v_{+},v_{-}$ from Lemma 6.7, which are singular
in the generic case, are in this case contained in the submodule generated in
degrees $-p+3c$ and $3p-3c$ so do not have to be included in the list of
generators of the maximal proper submodule here.

We have checked this conjecture using Magma [BCP] for all $p\leq 13$ and all
$c$, and a small number of larger $p$ and specific $c$. For example, when
$p=13$ and $c=6$, $L_{1,c}(\mathrm{Stand})$ has $50$ nonzero graded pieces and
dimension $886$.

## References

  * [BCa] Martina Balagović and Harrison Chen.  Category $\mathcal{O}$ for rational Cherednik algebras $H_{t,c}(GL_{2}(\mathbb{F}_{p}),\h{})$ in characteristic $p$.  Journal of Pure and Applied Algebra, 217(9):1683–1699 (2013).
  * [BCb] Martina Balagović and Harrison Chen.  Representations of Rational Cherednik Algebras in Positive Characteristic.  Journal of Pure and Applied Algebra, 217(4):716–740 (2013).
  * [BCP] Wieb Bosma, John Cannon, and Catherine Playoust.  The Magma algebra system. I. The user language.  Journal of Symbolic Computation, 24(3-4):235–265 (1997).
  * [BEGa] Yuri Berest, Pavel Etingof, and Victor Ginzburg.  Cherednik Algebras and Differential Operators on Quasi-invariants.  Duke Mathematical Journal, 118(2):279–337 (2003).
  * [BEGb] Yuri Berest, Pavel Etingof, and Victor Ginzburg.  Finite Dimensional Representations of Rational Cherednik algebras.  International Mathematics Research Notices, 2003(19):1053–1088 (2003).
  * [BFG] Roman Bezrukavnikov, Michael Finkelberg, and Victor Ginzburg.  Cherednik algebras and Hilbert Schemes in characteristic $p$.  Represention Theory, 10:254-298 (2006).
  * [BM13] Gwyn Bellamy and Maurizio Martino.  On the Smoothness of Centres of Rational Cherednik Algebras in Positive Characteristic.  Glasgow Mathematical Journal, 55(A):27–54 (2013), (2013).
  * [CE] Tatyana Chmutova and Pavel Etingof.  On Some Representations of the Rational Cherednik Algebra.  Representation Theory, 7:641–650 (2003).
  * [Chea] Ivan Cherednik.  Double Affine Hecke Algebras and Macdonald‘s Conjectures.  Annals of Mathematics, 141(1):191–216 (1995).
  * [Cheb] Ivan Cherednik.  Double Affine Hecke algebras, Knizhnik-Zamolodchikov Equations, and Macdonald’s Operators.  International Mathematics Research Notices, 9:171–180 (1992).
  * [CK] Merrick Cai and Daniil Kalinov.  The Hilbert Series of the Irreducible Quotient of the Polynomial Representation of the Rational Cherednik Algebra of Type $A_{n-1}$ in Characteristic $p$ for $p\mid n-1$.  Journal of Algebra and Its Applications, 21:6 (2021).
  * [DJO] Charles F. Dunkl, Marcel F. E. De Jeu, and Eric M. Opdam.  Singular Polynomials for Finite Reflection Groups.  Transactions of the American Mathematical Society, 346(1):237–256 (1994).
  * [DSa] Sheela Devadas and Steven V. Sam.  Representations of rational Cherednik algebras of $G(m,r,n)$ in positive characteristic.  Journal of Commutative Algebra 6(4):525–559 (2014).
  * [DSb] Sheela Devadas and Yi Sun.  The polynomial representation of the type $A_{n-1}$ rational Cherednik algebra in characteristic $p\mid n$.  Communications in Algebra 45(5):1926–1934 (2016).
  * [Dun] Charles F. Dunkl.  Singular Polynomials for the Symmetric Groups.  International Mathematics Research Notices, 2004(67):3607-–3635 (2004).
  * [EG] Pavel Etingof and Victor Ginzburg.  Symplectic Reflection Algebras, Calogero-Moser Space, and Deformed Harish-Chandra Homomorphism.  Inventiones mathematicae, 147:243-–348 (2002).
  * [EM11] Pavel Etingof and Xiaoguang Ma.  Lecture notes on Cherednik algebras.  (2011).
  * [GGOR] Victor Ginzburg, Nicolas Guay, Eric Opdam, and Raphael Rouquier.  On the category $\mathcal{O}$ for rational cherednik algebras.  Inventiones mathematicae, 154:617–651 (2003).
  * [Gor] Iain G. Gordon.  Baby Verma Modules for Rational Cherednik Algebras.  Bulletin of the London Mathematical Society, 35(3):321–336 (2003).
  * [Gri08] Stephen Griffeth.  Jack polynomials and the coinvariant ring of $g(r,p,n)$.  (2008).
  * [Kem] Gregor Kemper.  Calculating Invariant Rings of Finite Groups over Arbitrary Fields.  Journal of Symbolic Computation, 21:351–366 (1996).
  * [Lat] Frédéric Latour.  Representations of rational Cherednik Algebras of Rank 1 in Positive Characteristic.  Journal of Pure and Applied Algebra, 195(1):97–112 (2005).
  * [Lia12] Carl Lian.  Representations of Cherednik Algebras Associated to Symmetric and Dihedral Groups in Positive Characteristic.  (2012).
  * [Nak] Haruhisa Nakajima.  Invariants of Finite Groups Generated by Pseudo-Reflections in Positive Characteristic.  Tsukuba Journal of Mathematics, 3(1):109–122 (1979).


# Development of a Trust-Aware User Simulator for Statistical Proactive Dialog
Modeling in Human-AI Teams (Preprint)

Matthias Kraus  University of AugsburgAugsburgGermany<EMAIL_ADDRESS>,  Ron Riekenbrauck  Ulm UniversityUlmGermany<EMAIL_ADDRESS>and
Wolfgang Minker  Ulm UniversityUlmGermany<EMAIL_ADDRESS>
(2023)

###### Abstract.

The concept of a Human-AI team has gained increasing attention in recent
years. For effective collaboration between humans and AI teammates,
proactivity is crucial for close coordination and effective communication.
However, the design of adequate proactivity for AI-based systems to support
humans is still an open question and a challenging topic. In this paper, we
present the development of a corpus-based user simulator for training and
testing proactive dialog policies. The simulator incorporates informed
knowledge about proactive dialog and its effect on user trust and simulates
user behavior and personal information, including socio-demographic features
and personality traits. Two different simulation approaches were compared, and
a task-step-based approach yielded better overall results due to enhanced
modeling of sequential dependencies. This research presents a promising avenue
for exploring and evaluating appropriate proactive strategies in a dialog game
setting for improving Human-AI teams.

user simulation, proactive dialog, corpus-based methods, human-AI team, human-
AI trust

††journalyear: 2023††copyright: rightsretained††conference: Adjunct
Proceedings of the 31st ACM Conference on User Modeling, Adaptation and
Personalization; June 26–29, 2023; Limassol, Cyprus††booktitle: Adjunct
Proceedings of the 31st ACM Conference on User Modeling, Adaptation and
Personalization (UMAP ’23 Adjunct), June 26–29, 2023, Limassol, Cyprus††doi:
10.1145/3563359.3597403††isbn: 978-1-4503-9891-6/23/06

##  1\. Introduction

The concept of a Human-AI team (HAIT) is intriguing, but despite the
availability of sophisticated conversational assistants such as Alexa, Siri,
and ChatGPT, we are still far from achieving an AI that is not only able to
solve specific tasks but could also socialize and form a personal bond with
its user to build an effective team. HAIT requires close coordination between
humans and AI teammates to work together towards a common goal (Zhang et al.,
2021). Effective communication, prediction of teammates’ actions, and high-
level coordination are essential components of this collaborative effort. In
this regard, the proactive behavior of AI-based systems and the communication
thereof during collaboration is an important research topic concerning HAITs,
e.g., see Horvitz et al. (Horvitz, 1999). Proactivity can be defined as an
AI’s self-initiating, anticipatory behavior for contributing to effective and
efficient task completion. It has been shown to be essential for human
teamwork as it leads to higher job and team performance and is associated with
leadership and innovation (Crant, 2000). However, the design of adequate
proactivity for AI-based systems to support humans is still an open question
and a challenging topic. It is essential to study the impact of proactive
system actions on the human-agent trust relationship and how to use
information about an AI agent’s perceived trustworthiness to model appropriate
proactive dialog strategies for forming effective HAITs. There are different
experimental approaches to achieving this goal, such as developing strategies
and testing them with real users or training statistical-based strategies and
testing them using simulated experimental approaches (McTear, 2020).
Experiments with real users can either take place under laboratory conditions
or ”in the wild,” i.e., in real-life scenarios. However, there are
disadvantages to both approaches, including the lack of real-life usage in
laboratory conditions and the difficulty of obtaining a full-functioning
system during the early stages of development for in-the-wild studies. In
addition, recruiting a sufficient number of users to allow a valid
interpretation of data is a challenge. To overcome this challenge, user
simulation techniques have been developed (Eckert et al., 1997; Levin et al.,
2000; Schatzmann et al., 2006; Kreyssig et al., 2018; Jain et al., 2018).
These techniques allow the testing of dialog agent prototypes with a large
number of simulated ”subjects” and facilitate the exploration of dialog
strategies that may enhance HAIT.

In this paper, we present the development of a corpus-based user simulator for
training and testing proactive dialog policies. To create the simulator, we
collected a proactive dialog corpus with user trust annotations (Kraus et al.,
2022a), utilizing informed knowledge about proactive dialog and its effect on
user trust from previous studies (Kraus et al., 2020b, 2021b, 2022b, a) to
ensure high-quality data collection. Our main goal was to replicate realistic
user characteristics, tasks, and trusting behaviors for exploring and
evaluating appropriate strategies in a dialog game setting that we designed as
a sequential decision-making task in a company management environment (Kraus
et al., 2022a). We simulated user behavior and personal information, including
socio-demographic features and personality traits, using relevant data from
the corpus collection. This enabled us to estimate the current trustworthiness
of system behavior (Kraus et al., 2021a) and integrate trust in the dialog
state and reward function for creating trust-adaptive proactive dialog
strategies (Kraus et al., 2022c). To simulate the user’s task behavior, we
developed and compared two different simulation approaches: A complexity-based
method and a task-step-based method. Both methods were found to be applicable
for training statistical proactive dialog strategies, but the task-step-based
approach yielded better overall results due to better modeling of sequential
dependencies.

##  2\. Related work

Over the past 25 years, user simulation for training statistical and primarily
RL-based dialog systems has been extensively studied (Singh et al., 2002). The
primary focus has been on task-oriented dialog systems that are designed to
assist users in achieving a specific task or goal. Examples of such systems
include conversational agents used for hotel room bookings or ordering food
from a restaurant (Young et al., 2013). These systems conduct so-called slot-
filling dialogs, where the system retrieves specific values for pre-defined
entities, or slots, of a particular domain, such as food type, price range,
and location, via dialog with the user for providing the desired information.
In task-oriented dialog systems, user utterances are typically encoded in
semantic representations by a natural language understanding module. The
dialog management module then takes these representations as input for filling
the respective semantic slots and selecting an appropriate system response
according to a specific dialog policy. Therefore, user simulation for training
and evaluating task-oriented systems usually produces output in the form of
semantic representations of user actions (Eckert et al., 1997; Levin et al.,
2000; Lee and Eskenazi, 2012; Lin et al., 2021). However, there are also
alternative approaches that can generate natural language utterances (Kreyssig
et al., 2018; Shi et al., 2019). (López-Cózar et al., 2003) even presented a
user simulator that generates spoken utterances based on pre-recorded speech
files. User simulators can be classified into two types: rule-based and
corpora-based (Schatzmann et al., 2006). Rule-based methods rely on hand-
crafted rules and a range of user profiles (Pietquin, 2005), while corpora-
based approaches use probabilistic data-driven methods to model natural user
behavior (Schatzmann et al., 2006; Lee and Eskenazi, 2012). More recent
approaches use sequence-to-sequence or transformer models to produce output on
a semantic or natural language level (Kreyssig et al., 2018; Shi et al., 2019;
Lin et al., 2021). Social behavior modeling is also of interest when it comes
to user simulation. Egges et al. (2004) simulated personality, mood, and
emotion, and Ferreira and Lefevre (2015) incorporated social signals in the
reward function of the RL-based system for training socially-effective dialog
behavior. Several social signals, such as user satisfaction, rapport-building,
small talk, and self-disclosure, have been used in different user simulation
approaches (Ultes et al., 2015; Sun et al., 2021; Jain et al., 2018). We also
aim to build a user simulator that can express task-related behavior and model
specific user characteristics. However, as our interaction with the agent only
involves a limited set of user actions, we opted for a simplistic combination
of rule-based and data-driven mechanisms for simulating adequate user
behavior. We compared two simulation approaches: using the complexity-based
method, the user’s task behavior was simulated depending on the agent’s action
and the complexity of a task step, while the task-step-based method
incorporated information from a particular task step and the agent’s action.
For deciding, which approach to use for training our RL-based proactive dialog
agent, we evaluated the approaches according to common metrics for estimating
the quality of user simulators (Pietquin and Hastie, 2013). Here, we utilized
the mean square error (MSE) and Kullback-Leibler (KL) Divergence for measuring
the error and difference between the behavior of real users and simulated
users.

##  3\. Simulated Proactive Dialog Environment

As we chose to utilize a corpus-based method to appropriately model a user’s
task and trusting behavior, it was necessary to have access to a proactive
dialog corpus that includes trust annotations. Although there are various data
corpora that exist for conventional dialog modeling (Williams et al., 2014;
Budzianowski et al., 2018), none of them are sufficient for modeling proactive
dialog. This is due to the fact that proactive behavior is either absent or
underrepresented in these corpora (Balaraman and Magnini, 2020), and trust-
related features are not adequately annotated. To fill this gap, a new data
corpus was created in previous work (Kraus et al., 2022a), which involved
developing an AI agent prototype for personal advising with a proactive dialog
model. The agent collected personal and dialog data in a serious gaming
scenario, resulting in a trust-annotated data corpus containing interactions
with the proactive assistance system. To accurately model and predict trust
during mixed-initiative dialog, a trust prediction model was required to
simulate a user’s trusting behavior. Previous research on trust has identified
various user-, system-, and context-related factors that influence the trust
relationship (Parasuraman and Riley, 1997; Muir, 1994; Lee and See, 2004; Hoff
and Bashir, 2015). For including this in the development of proactive dialog
policies, we collected the necessary features during data collection and build
a trust estimation model in previous work (Kraus et al., 2021a). The next
sections provide further details regarding the data collection method, the
corpus, and the techniques employed to predict the HCT relationship.

###  3.1. Data Collection

We acquired a dataset of 308 dialogs, which included a total of 3696 exchanges
between users and a proactive dialog agent (Kraus et al., 2022a). The data was
gathered using an online game that utilized the clickworker framework
111www.clickworker.de, where users had to make strategic decisions to manage a
company with the agent’s assistance. Each exchange was marked with the user’s
self-reported measures of trust in the system, including competence,
predictability, and reliability. The data also included objective features
such as task complexity, exchange duration, user actions, and static user
information such as age, gender, personality, and domain expertise. The game
was designed as a turn-based planning task where the user had to make
decisions based on the agent’s provided options. The agent selected various
proactive dialog action types and employed natural language to offer
suggestions and information to help the user make the best choice. We made
sure that the agent’s suggestions generated the most points based on the task
step to avoid any unintended negative impacts on the system’s trustworthiness.

###  3.2. Trust Modelling and Estimation

Based on the collected data set, we created a new user model to predict online
trust when interacting with proactive AI agents (Kraus et al., 2021a). For
this, we selected corpus parameters as features for prediction, including both
numerical values (such as age and trust propensity) and categorical values
(such as proactivity type). The resulting feature vector contained 57
features, which were categorized as personal user parameters, interaction
parameters, and temporal interaction parameters. To predict trust, the
variables for trust, competence, reliability, and predictability were combined
to create a target variable with a range of 1 to 5 on a Likert-scale. The
prediction problem was treated as a multi-class classification task with
distinct trust values as target classes. We applied a Support Vector Machine
(SVM) as a trust classifier, which provided the best results compared to Gated
Recurrent Unit (GRU) Networks and Extreme Gradient Boosting. The trained SVM
was used to predict the user’s trust state at each task step and to evaluate
the simulated user’s trust.

In summary, the created simulated environment consisted of a corpus-based user
simulator, a supervised learning-based trust state model which allows for
estimating the simulated user’s current trust in the agent’s action and to be
able to adapt the dialog to the estimated user trust in the agent, and an RL-
based proactive dialog agent, which is extensively described in our previous
work (Kraus et al., 2022c). In Fig 1, the schematic description of the
simulated environment is depicted. In the following, we present the
architecture of the user simulation model.

##  4\. User Simulator Architecture

Figure 1. The proposed RL framework for implementing trust-adaptive proactive
dialog agents. We formulate the collaboration process as an MDP and train an
RL-based proactive dialog agent on interactions with a simulated user based on
our data collection. For integrating user trust estimations in the state and
the reward function of the agent, we utilize our trust estimation module for
predicting user trust in the agent’s actions in real time.

User simulation was based on two components similar to the work by Jain et al.
(2018): a user model and a user dialog manager. The user model contained all
the necessary information for modeling distinct user types whose specific task
and trust behaviors were imitated. The user dialog manager was designed as a
rule-based agent that triggered various behaviors dependent on the proactive
system’s actions and the current task context. See Table 1 for an overview of
all parameters used for user simulation. First, the user model is described in
detail.

###  4.1. User Model

Parameter |  Description |  Type
---|---|---
Age |  numerical value |  user trait
Gender |  categorical value: male, female, other |  user trait
Technical Affinity |  avg. numerical value based on 5-point Likert scale by Karrer et al. (2009) |  user trait
Trust Propensity |  avg. numerical value based on 5-point Likert scale by Merritt et al. (2013) |  user trait
Domain Expertise |  avg. numerical value based on self-developed 5-point Likert scale |  user trait
Big 5 personality traits |  avg. numerical value for each trait based on 5-point Likert scale by (Rammstedt et al., 2013) |  user trait
Proactive DialAct |  categorical value: None, Notification, Suggestion, Intervention |  system action
Task Difficulty |  numerical value based on 5-point Likert scale |  task property
Task Complexity |  numerical value: 3,4,5 |  task property
Task Duration |  numerical value |  task property
User Selection |  numerical value according to game score of the respective option |  user action
Suggestion Request |  categorical value: True, False |  user action
Help Request |  categorical value: True, False |  user action
Table 1. Overview and description of the user and interaction parameters from
the ProDial-corpus.

To create distinct user types, we utilized user-dependent information from the
corpus, including age, gender, technical proficiency, the propensity to trust,
domain expertise, and the Big 5 personality traits. We generated random
distributions for these variables, with the exception of gender, which was
based on its likelihood of occurrence in the corpus. Truncated Gaussian
distributions were used for all other variables, as they were rated on 5-point
Likert scales or were bounded due to study restrictions (user age was limited
from 18 to 60). Our definition of a user’s task behavior comprised the
selection of options, which was represented as the game score, help requests
about the game and suggestion requests towards the system. While all user-type
features were used for trust estimation, only three variables (domain
expertise, propensity to trust, and technical proficiency) were deemed
relevant to the specific user’s task behavior. Domain expertise influences
decision-making and novices may ask for more recommendations. Propensity to
trust affects a user’s reactions to proactive behavior, with low trust leading
to rejections of offers or not asking for assistance. Technical proficiency
affects decision-making when collaborating with an autonomous technical
system. To simplify the selection of specific task behavior, these three user
traits were transformed into binary values based on a threshold of 3 on the
5-point Likert scale. This was done to reduce the rule space for simulating
user behavior. User-specific task behavior and the system’s actions affect
task duration and perceived difficulty. Both variables were randomized also
using truncated Gaussian distributions as task duration was always greater
than 20 seconds and had timing limitation, and perceived difficulty was
measured on a 5-point Likert scale. The user’s task behavior was based on a
pre-defined rule set generated by a user dialog manager, which is described in
the next section.

###  4.2. User Dialog Manager

Two distinct approaches, namely complexity-based and task-step-based, were
used to generate task behavior in the serious dialog game. The game had 12
task steps, each with a varying level of complexity, which referred to the
number of options a user had to select from for decision-making. The number of
options per task varied sequentially in the order of 3, 4, 5, 3, 4, and 5. The
complexity-based method simulated user behavior based on the system’s action
and the complexity level of the current task step. For instance, if the
current proactive dialog act type was Notification and the current task step
had a complexity level of 3, the simulator would use the corpus data
distributions for these specific cases for generating task behavior. On the
other hand, the task-step-based method incorporated information from a
specific task step and the system’s action to simulate user behavior. For
instance, if the current proactive dialog act type was Notification and the
user was working on the seventh task step, the simulator would use the corpus
data distributions for these specific cases for generating task behavior. The
advantage of incorporating task complexity in dialog management was that user
behavior could be generated in a more generalized way, not dependent on the
specific task steps. However, the task-step-based method modeled sequential
dependencies between task steps better. Hence, there existed a certain trade-
off between both variants.

The simulation of user behavior in both approaches involved generating values
for the game score, whether a user-initiated a suggestion or help request, the
corresponding duration of the task step, and perceived task difficulty. The
probabilities for each specific user behavior were based on structured
datasets that depended on the user model and the current dialog situation. The
overall dataset was sorted based on the occurrences of user behavior,
dependent on the relevant user traits, such as domain experience, propensity
to trust, and technical affinity. These user traits were represented as tuples
of three binary values, i.e., ”000” to ”111”. For instance, ”000” represented
low domain experience, low propensity to trust, and low technical affinity,
while ”111” represented high domain experience, high propensity to trust, and
high technical affinity. In the complexity-based method, the user-dependent
data was first summarized based on tasks of the same complexity, and then it
was summarized according to the types of assistant proactivity, i.e., None,
Notification, Suggestion, and Intervention. Lastly, the resulting data was
structured according to the occurrences of help and suggestion requests,
represented as binary values. In contrast, the task-step-based approach used
the same method but summarized the user-dependent data based on the respective
task step number. In case the occurrences of specific parameters did not
exceed a specific threshold, both approaches used fallback datasets. If there
was not enough data for a particular user trait, i.e., occurrences for a trait
were below 10, then this parameter was omitted, and the means and standard
deviations or counts of all user traits were used for calculating the
probabilities for user behavior generation.

The following outlines the process for simulating using the task-step-based
approach 222The code for the user simulator is available online at
https://github.com/MattKraus90/ProactiveTraining for reproducibility..
Similarly, the complexity-based approach algorithm was structured, but with
complexity-based data distributions instead of task-step-based distributions.
The algorithmic process begins with generating a user type with specific
traits and loading approach-specific structured data sets. Then, the dialog
game is initialized, and the task steps (1-12) with respective complexities
(3, 4, 5) are iterated. For each task step, values for help and suggestion
requests, duration, perceived difficulty, and the achieved game score are
calculated based on the user type and the system’s action. Relevant trait
categories are queried, and context is determined, such as proactive system
action and complexity or task step number. The fallback threshold is checked,
and if not exceeded, personality traits are neglected, and overall means are
used for probability calculation. A simulation is conducted to determine if a
help and/or suggestion request would be set, with a fallback check for
respective request types. Depending on the specific case, the perceived
difficulty, task step duration, and the achieved game score are simulated.

##  5\. Experiments and Results

To determine the most suitable approach for training and testing proactive
dialog strategies that adapt to trust, we conducted an evaluation that focused
on assessing the realism of each user simulation method. Our goal was to
identify the approach that best approximated the actual behavior of users, as
observed in the data. To achieve this, we employed both methods to simulate
user behavior, based on the user types and system actions recorded during data
collection with real users. We then compared the simulated behavior with the
real behavior, using the KL distance to measure the difference between the two
distributions. This approach has been suggested by previous researchers such
as Pietquin and Hastie (2013) and was used in a study by Jain et al. (2018).
KL distance is a measure of how one probability distribution $Q$ is different
from a second, reference probability distribution $P$:

(1) $D_{KL}(P\parallel
Q)=\sum_{x\in\mathcal{X}}P(x)\log\left(\frac{P(x)}{Q(x)}\right)$

where the distances range from 0, i.e. distributions are equal, to 1, i.e
distributions are completely different. The lower the distance between the
distributions, the more realistic is the respective user simulator. For
evaluation, we calculated the distances of distributions between the
complexity-, respectively task-step-based approach and actual behavior for
each task step. In Table 2, the overall mean distances for each task step as
well as the individual distances for the game score, duration, help and
suggestion request, and perceived difficulty are listed.

|  Complexity-based M (SD) |  Task-step-based M (SD)
---|---|---
|  KL |  MSE |  KL |  MSE
Game Score |  0.369 (.185) |  73.19 (64.6) |  0.354 (.166) |  70.94 (64.7)
Duration |  0.261 (.064) |  1722 (844) |  0.244 (.079) |  1530 (104)
Difficulty |  0.145 (.011) |  1.909 (.155 ) |  0.149 (.008) |  1.887 (.217)
Help Request |  0.029 (.009) |  0.088 (.028) |  0.031 (.011) |  0.097 (.035)
Suggestion Request |  0.084 (.006) |  0.352 (.025) |  0.082 (.010) |  0.337 (.034)
Overall |  0.178 (.151) |  359.5 (780) |  0.172 (.142) |  320.6 (765)
Table 2. Descriptive statistics of the KL distances and MSEs for each user
simulator type with regard to the measures of game score, duration, help and
suggestions request, and perceived difficulty.

##  6\. Discussion and Conclusion

The study findings indicated that both types of user simulators had similar
performance as there were no notable differences in all measured features
(with all p-values being greater than 0.05). Table 2 reveals that the task-
step-based approach produced behavior that was slightly more realistic than
the complexity-based method. This was mainly attributed to the fact that the
former approach simulated more authentic game scores and durations of specific
task steps. One possible explanation for this was that the task-step approach
used averages of individual task steps to generate distributions, while the
complexity-based method used average values across four task steps of the same
complexity. Help and suggestion requests were simulated nearly the same way as
observed during data collection. The result of the simulators was in the same
region as the performance of the user simulator by Jain et al. (2018) by
achieving a KL-score of 0.109 which they then used to build an RL-based dialog
manager (Pecune and Marsella, 2020). Thus, we deem both applicable for
training and testing statistical proactive dialog strategies. Given the
slightly more realistic outcomes, the task-step-based approach was chosen for
constructing the train- and test environment in developing statistical
proactive dialog. We trained and tested an RL-based proactive dialog agent
using the described user simulation approach and presented the results in
Kraus et al. (2022c). The evaluation showed the utility of our approach, as
the user simulator provided similar on task effectiveness and user trust as
observed in studies with real users (Rau et al., 2013; Kraus et al., 2020b).

As usual, our approach has limitations. Firstly, although we found that 308
participants provided meaningful results for creating user types, increasing
the number of participants for data collection could be beneficial. This is
especially important for very specific user types where there is limited
information, necessitating the need for fallback strategies. Gathering data
from more users could alleviate this issue. Secondly, we only focused on a
restricted decision-making interaction where users used template natural
language utterances to interact with the agent. Therefore, we used a
simplistic approach which was deemed reasonable for the task at hand. However,
for future work, using more sophisticated approaches such as Hidden Markov
Models or RL-based approaches could be beneficial, especially when extending
our approach to utilize a greater number of dialog acts. In addition, we aim
to integrate natural language interaction, for which transformer-based user
simulation approaches could be helpful in creating sophisticated models for
more complex use cases. Nonetheless, our approach is among the first to
incorporate user, task, and system information for modeling a user’s task and
trusting behavior during mixed-initiative interaction with a proactive dialog
agent. Thus, this work is an important step towards enabling socially
responsible and task-effective HAITs.

## References

  * (1)
  * Balaraman and Magnini (2020) Vevake Balaraman and Bernardo Magnini. 2020.  Proactive Systems and Influenceable Users: Simulating Proactivity in Task-oriented Dialogues. In _Proceedings of the 24th Workshop on the Semantics and Pragmatics of Dialogue-Full Papers, Virually at Brandeis, Waltham, New Jersey, July. SEMDIAL_.
  * Budzianowski et al. (2018) Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018.  MultiWOZ–A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling.  _arXiv preprint arXiv:1810.00278_ (2018).
  * Crant (2000) J Michael Crant. 2000\.  Proactive behavior in organizations.  _Journal of management_ 26, 3 (2000), 435–462.
  * Eckert et al. (1997) Wieland Eckert, Esther Levin, and Roberto Pieraccini. 1997\.  User modeling for spoken dialogue system evaluation. In _1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings_. IEEE, 80–87.
  * Egges et al. (2004) Arjan Egges, Sumedha Kshirsagar, and Nadia Magnenat-Thalmann. 2004\.  Generic personality and emotion simulation for conversational agents.  _Computer animation and virtual worlds_ 15, 1 (2004), 1–13.
  * Ferreira and Lefevre (2015) Emmanuel Ferreira and Fabrice Lefevre. 2015.  Reinforcement-learning based dialogue system for human–robot interactions with socially-inspired rewards.  _Computer Speech & Language_ 34, 1 (2015), 256–274.
  * Hoff and Bashir (2015) Kevin Anthony Hoff and Masooda Bashir. 2015.  Trust in automation: Integrating empirical evidence on factors that influence trust.  _Human factors_ 57, 3 (2015), 407–434.
  * Horvitz (1999) Eric Horvitz. 1999\.  Principles of mixed-initiative user interfaces. In _Proceedings of the SIGCHI conference on Human Factors in Computing Systems_. ACM, 159–166.
  * Jain et al. (2018) Alankar Jain, Florian Pecune, Yoichi Matsuyama, and Justine Cassell. 2018.  A user simulator architecture for socially-aware conversational agents. In _Proceedings of the 18th International Conference on Intelligent Virtual Agents_. 133–140.
  * Karrer et al. (2009) Katja Karrer, Charlotte Glaser, Caroline Clemens, and Carmen Bruder. 2009.  Technikaffinität erfassen–der Fragebogen TA-EG.  _Der Mensch im Mittelpunkt technischer Systeme_ 8 (2009), 196–201.
  * Kraus et al. (2020a) Matthias Kraus, Marvin Schiller, Gregor Behnke, Pascal Bercher, Michael Dorna, Michael Dambier, Birte Glimm, Susanne Biundo, and Wolfgang Minker. 2020a.  ”Was That Successful?” On Integrating Proactive Meta-Dialogue in a DIY-Assistant Using Multimodal Cues. In _Proceedings of the 2020 International Conference on Multimodal Interaction_ (Virtual Event, Netherlands) _(ICMI ’20)_. Association for Computing Machinery, New York, NY, USA, 585–594.  https://doi.org/10.1145/3382507.3418818
  * Kraus et al. (2021b) Matthias Kraus, Nicolas Wagner, Zoraida Callejas, and Wolfgang Minker. 2021b.  The Role of Trust in Proactive Conversational Assistants.  _IEEE Access_ 9 (2021), 112821–112836.
  * Kraus et al. (2020b) Matthias Kraus, Nicolas Wagner, and Wolfgang Minker. 2020b.  Effects of Proactive Dialogue Strategies on Human-Computer Trust. In _Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization_ (Genoa, Italy) _(UMAP ’20)_. Association for Computing Machinery, New York, NY, USA, 107–116.  https://doi.org/10.1145/3340631.3394840
  * Kraus et al. (2021a) Matthias Kraus, Nicolas Wagner, and Wolfgang Minker. 2021a.  Modelling and Predicting Trust for Developing Proactive Dialogue Strategies in Mixed-Initiative Interaction. In _Proceedings of the 2021 International Conference on Multimodal Interaction_. 131–140.
  * Kraus et al. (2022a) Matthias Kraus, Nicolas Wagner, and Wolfgang Minker. 2022a.  ProDial – An Annotated Proactive Dialogue Act Corpus for Conversational Assistants using Crowdsourcing. In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_. European Language Resources Association, Marseille, France, 3164–3173.  https://aclanthology.org/2022.lrec-1.339
  * Kraus et al. (2022b) Matthias Kraus, Nicolas Wagner, Wolfgang Minker, Ankita Agrawal, Artur Schmidt, Pranav Krishna Prasad, and Wolfgang Ertel. 2022b.  KURT: A Household Assistance Robot Capable of Proactive Dialogue. In _Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction_. 855–859.
  * Kraus et al. (2022c) Matthias Kraus, Nicolas Wagner, Ron Riekenbrauck, and Wolfgang Minker. 2022c.  Towards Improving Proactive Dialog Agents Using Socially-Aware Reinforcement Learning. In _arXiv_. arXiv.  https://doi.org/10.48550/ARXIV.2211.15359
  * Kreyssig et al. (2018) Florian Kreyssig, Iñigo Casanueva, Paweł Budzianowski, and Milica Gasic. 2018\.  Neural User Simulation for Corpus-based Policy Optimisation of Spoken Dialogue Systems. In _Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue_. 60–69.
  * Lee and See (2004) John D Lee and Katrina A See. 2004.  Trust in automation: Designing for appropriate reliance.  _Human factors_ 46, 1 (2004), 50–80.
  * Lee and Eskenazi (2012) Sungjin Lee and Maxine Eskenazi. 2012.  An unsupervised approach to user simulation: toward self-improving dialog systems. In _Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. 50–59.
  * Levin et al. (2000) Esther Levin, Roberto Pieraccini, and Wieland Eckert. 2000\.  A stochastic model of human-machine interaction for learning dialog strategies.  _IEEE Transactions on speech and audio processing_ 8, 1 (2000), 11–23.
  * Lin et al. (2021) Hsien-chin Lin, Nurul Lubis, Songbo Hu, Carel van Niekerk, Christian Geishauser, Michael Heck, Shutong Feng, and Milica Gašić. 2021.  Domain-independent user simulation with transformers for task-oriented dialogue systems.  _arXiv preprint arXiv:2106.08838_ (2021).
  * López-Cózar et al. (2003) Ramón López-Cózar, A De la Torre, José C Segura, and Antonio J Rubio. 2003.  Assessment of dialogue systems by means of a new simulation technique.  _Speech Communication_ 40, 3 (2003), 387–407.
  * McTear (2020) Michael F McTear. 2020\.  Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots.  _Synthesis Lectures on Human Language Technologies_ 13, 3 (2020), 1–251.
  * Merritt et al. (2013) Stephanie M Merritt, Heather Heimbaugh, Jennifer LaChapell, and Deborah Lee. 2013.  I trust it, but I don’t know why: Effects of implicit attitudes toward automation on trust in an automated system.  _Human factors_ 55, 3 (2013), 520–534.
  * Muir (1994) Bonnie M Muir. 1994\.  Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems.  _Ergonomics_ 37, 11 (1994), 1905–1922.
  * Parasuraman and Riley (1997) Raja Parasuraman and Victor Riley. 1997.  Humans and automation: Use, misuse, disuse, abuse.  _Human factors_ 39, 2 (1997), 230–253.
  * Pecune and Marsella (2020) Florian Pecune and Stacy Marsella. 2020.  A framework to co-optimize task and social dialogue policies using Reinforcement Learning. In _Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents_. 1–8.
  * Pietquin (2005) Olivier Pietquin. 2005\.  _A framework for unsupervised learning of dialogue strategies_.  Presses univ. de Louvain.
  * Pietquin and Hastie (2013) Olivier Pietquin and Helen Hastie. 2013.  A survey on metrics for the evaluation of user simulations.  _The knowledge engineering review_ 28, 1 (2013), 59–73.
  * Rammstedt et al. (2013) Beatrice Rammstedt, Christoph Kemper, Mira Céline Klein, Constanze Beierlein, and Anastassiya Kovaleva. 2013.  Eine kurze Skala zur Messung der fünf Dimensionen der Persönlichkeit: Big-Five-Inventory-10 (BFI-10).  _Methoden, Daten, Analysen (mda)_ 7, 2 (2013), 233–249.
  * Rau et al. (2013) Pei-Luen Patrick Rau, Ye Li, and Jun Liu. 2013.  Effects of a social robot’s autonomy and group orientation on human decision-making.  _Advances in Human-Computer Interaction_ 2013 (2013), 11.
  * Schatzmann et al. (2006) Jost Schatzmann, Karl Weilhammer, Matt Stuttle, and Steve Young. 2006.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies.  _The knowledge engineering review_ 21, 2 (2006), 97–126.
  * Shi et al. (2019) Weiyan Shi, Kun Qian, Xuewei Wang, and Zhou Yu. 2019\.  How to build user simulators to train rl-based dialog systems.  _arXiv preprint arXiv:1909.01388_ (2019).
  * Singh et al. (2002) Satinder Singh, Diane Litman, Michael Kearns, and Marilyn Walker. 2002.  Optimizing dialogue management with reinforcement learning: Experiments with the NJFun system.  _Journal of Artificial Intelligence Research_ 16 (2002), 105–133.
  * Sun et al. (2021) Weiwei Sun, Shuo Zhang, Krisztian Balog, Zhaochun Ren, Pengjie Ren, Zhumin Chen, and Maarten de Rijke. 2021.  Simulating user satisfaction for the evaluation of task-oriented dialogue systems. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 2499–2506.
  * Ultes et al. (2015) Stefan Ultes, Matthias Kraus, Alexander Schmitt, and Wolfgang Minker. 2015.  Quality-adaptive spoken dialogue initiative selection and implications on reward modelling. In _Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. 374–383.
  * Williams et al. (2014) Jason D Williams, Matthew Henderson, Antoine Raux, Blaise Thomson, Alan Black, and Deepak Ramachandran. 2014\.  The dialog state tracking challenge series.  _AI Magazine_ 35, 4 (2014), 121–124.
  * Young et al. (2013) Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams. 2013.  Pomdp-based statistical spoken dialog systems: A review.  _Proc. IEEE_ 101, 5 (2013), 1160–1179.
  * Zhang et al. (2021) Rui Zhang, Nathan J McNeese, Guo Freeman, and Geoff Musick. 2021.  ” An ideal human” expectations of AI teammates in human-AI teaming.  _Proceedings of the ACM on Human-Computer Interaction_ 4, CSCW3 (2021), 1–25.