# COFFEE: Counterfactual Fairness for Personalized Text Generation in
Explainable Recommendation

Nan Wang1, Shaoliang Nie2, Qifan Wang2, Yi-Chia Wang2, Maziar Sanjabi2,
Jingzhou Liu2, Hamed Firooz2, Hongning Wang1<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>1University of Virginia, Charlottesville, VA, USA2Meta, Menlo Park, CA, USA

###### Abstract.

Personalized text generation has broad industrial applications, such as
explanation generation for recommendations, conversational systems, etc.
Personalized text generators are usually trained on user written text, e.g.,
reviews collected on e-commerce platforms. However, due to historical, social,
or behavioral reasons, there may exist bias that associates certain
_linguistic quality_ of user written text with the users’ protected attributes
such as gender, race, etc. The generators can identify and inherit these
correlations and generate texts discriminately w.r.t. the users’ protected
attributes. Without proper intervention, such bias can adversarially influence
the users’ trust and reliance on the system. From a broader perspective, bias
in auto-generated contents can reinforce the social stereotypes about how
online users write through interactions with the users.

In this work, we investigate the fairness of personalized text generation in
the setting of explainable recommendation. We develop a general framework for
achieving measure-specific counterfactual fairness on the linguistic quality
of personalized explanations. We propose learning disentangled representations
for counterfactual inference and develop a novel policy learning algorithm
with carefully designed rewards for fairness optimization. The framework can
be applied for achieving fairness on any given specifications of linguistic
quality measures, and can be adapted to most of existing models and real-world
settings. Extensive experiments demonstrate the superior ability of our method
in achieving fairness while maintaining high generation performance.

counterfactual fairness, personalized text generation

##  1\. Introduction

Figure 1. Left: average length of ground-truth explanations written by male
and female users in the train set of Amazon Games reviews. Right: average
length of explanations generated for male and female users on the test set by
PETER.

Personalized text generation (PTG) has broad applications in e-commerce
systems, such as explainable recommendation (Zhang and Chen, 2020; Chen et
al., 2021; Yang et al., 2022), conversational systems (Zhang et al., 2018a,
2019; Lee et al., 2021), review summarization (Dong et al., 2017; Li and
Tuzhilin, 2019; Zhang et al., 2019), etc. The auto-generated text, serving at
the frontier of human-machine interaction, influences users’ decisions and
transforms their way of thinking and behaving. Due to its powerful
capabilities and wide reach, PTG can imperceptibly incur fairness issues that
lead to counter-effects on the society at large (Alim et al., 2016; Bordia and
Bowman, 2019; Blodgett et al., 2020). In this paper, we investigate the
fairness problem in one of the mostly studied setting: generating natural
language explanations for recommendations (Wang et al., 2018; Chen et al.,
2021; Yang et al., 2021; Li et al., 2021b), and propose a general framework
for achieving fairness on any given quality measures of generated text.

Personalized explanation generation in recommendation aims to provide the user
with a paragraph of text to describe an item according to the user’s
interests, so as to help users make more informative and accurate decisions.
The generators are language models usually trained on user written reviews in
online platforms (Zhang et al., 2018a; Ni et al., 2019; Yang et al., 2021),
where sentences related to item descriptions are retained to construct the
training data. However, due to historical, social, or behavioral reasons,
there may exist bias that associates certain linguistic characteristics of the
review text with the users’ protected attributes, e.g., gender, race, income,
education, etc. (Newman et al., 2008; Alim et al., 2016; Volz et al., 2020).
Some of the characteristics that feature the diversity of language use (Newman
et al., 2008; Groenwold et al., 2020) are proper to be captured by
personalization together with the users’ preferences on items. While others
that are related to the _linguistic quality_ of the reviews (Louis, 2013),
such as detailedness, structure, fluency, informativeness, etc., may not be
kept in explanation generation, which otherwise will lead to unfair treatments
when serving users’ with different protected attribute values. As an example,
we study the Amazon Games reviews111The details about data preparation and
experiment setup is discussed in Section 6., and use the length of reviews as
a case study to evaluate the text quality, since length is shown to strongly
correlate and indicate text quality in writing assessment and easy to measure
(Powers, 2005; Guo et al., 2013; Louis, 2013; Fleckenstein et al., 2020). As
shown in Figure 1 (left), we found that male users generally write longer and
more detailed reviews about games than female users. If directly trained on
such data, a language model can inherit the bias and generate explanations
discriminately towards users’ gender—when a new game is recommended, the model
may generate short and generic explanations for female users (e.g., “Good
game.”), but long and detailed explanations for male users. This is shown in
Figure 1 (right), where we train a personalized transformer model (PETER) (Li
et al., 2021b) on the training set, and generate explanations on the test set.

In particular, the bias in observed user behavioral data originates from the
subjectivity of users, over which we have no control. But the system should
not keep the bias and act discriminately in providing explanations of
different quality for different users. The essence of the problem lies in the
discrepancy between the purposes of the data collected from users (for
training models) and the explanations generated (for serving users)—the lack
of detailed reviews from users of a given gender does not mean they do not
want to or deserve to receive detailed explanations. Without breaking the
spurious correlations, the model reproduces the bias, which distorts the
users’ perception and may run counter to the original purposes of the
explanations. When the users realize such bias, it can also adversarially
affect the users’ reliance and trust in the platform (Tintarev and Masthoff,
2015). From a broader perspective, the bias presented can reinforce the
stereotype by influencing how users write online in turn, and jeopardizes
fairness in the long run (Schwartz et al., 2016; Alim et al., 2016; Bordia and
Bowman, 2019).

In this work, we take a causal perspective in analyzing the biased
correlations and enforce counterfactual fairness (CF) (Kusner et al., 2017) in
explanation generation: the quality of the generated explanations should not
change when a user’s protected attribute is changed from one value to another
(e.g., male $\rightarrow$ female). This problem is essential and unique
compared to fairness problems studied in the literature (see Section 2), and
also imposes unique technical challenges. Although CF has been studied in
other NLP tasks (Huang et al., 2020; Garg et al., 2019), they mainly study CF
in terms of how the model’s outputs depend on the attribute-specific words
directly encapsulated in the input text (Blodgett et al., 2020; Liang et al.,
2021; Sheng et al., 2021). In this case, counterfactual inference can be
easily performed on the text itself, e.g., changing male pronouns to female
pronouns in the prompt for sentence completion (Huang et al., 2020), or in the
input text for text classification (Garg et al., 2019). However, how the
quality of generated text can be unfair towards the user being served has
never been considered. In personalized explanation generation, user and item
IDs are input tokens to the model, which are represented by embedding vectors
learned by the model (Li et al., 2017; Wang et al., 2018; Li et al., 2021b).
The user’s preferences and the protected attribute are entangled by an
implicit mixing mechanism in the user’s history data, so as in the learned
representations (Locatello et al., 2019a). The attribute implicitly encoded
cannot be directly manipulated for counterfactual inference. Moreover, the
review history of a counterfactual user does not exist, and thus there is no
supervision for learning counterfactual representations. One way explored for
CF in personalization is to simply remove all information about the protected
attribute in the users’ representations (Zemel et al., 2013; Locatello et al.,
2019a; Li et al., 2021a; Wang et al., 2022), e.g., by adding an adversarial
discriminator, such that the dependence between the model’s output and the
protected attribute is totally broken. This helps improve fairness but greatly
harm the personalization or any desired properties from the protected
attribute. In contrast, our goal for fairness is to enforce the independence
between the protected attribute and only a given _specific measure_ on the
linguistic quality of generated explanations, while keeping the dependence
from content of explanations for maintaining high personalization performance.

To achieve the goal, we develop a general framework COFFEE for achieving
COunterFactual FairnEss in Explanation generation. In particular, we treat
each user’s protected attribute value as a separate token input to the model,
together with the user and item IDs, and disentangle the attribute’s
representation from the user’s representation (Ma et al., 2019; Locatello et
al., 2019b; Zheng et al., 2021). By switching between real and counterfactual
input attribute values for counterfactual inference, we impose a measure-
specific CF constraint (Russell et al., 2017) on the quality of generated
explanations for training the generator. To tackle the constrained
optimization that generalizes to any given quality measures, we develop a
novel policy learning scheme to optimize CF, for which a reward allocation
mechanism is designed to quantify each sampled explanation’s contribution to
the fairness constraint.

Last but not least, recommendation happens in two-sided markets (Wang and
Joachims, 2021), and we should also consider the fairness for items in
explanation generation to protect the business owners (or people to be
recommended, e.g., in a job applicant recommendation system). We use user-side
fairness by default in the discussions, but COFFEE is general to ensure
fairness on either user or item side protected attributes, handle any given
specification of linguistic quality measures, and adapted to different models,
as will be shown in Sections 5 and 6. Extensive experiments on three datasets
and two popular models show COFFEE’s superiority in achieving fairness
compared to baselines, without hurting the generation performance.

##  2\. Background

Uniqueness and Importance: Fairness in machine learning (ML) is originally
studied in automatic decision-making systems which directly impose a
”significant” or ”legal” effect on individuals (Voigt and Bussche, 2017). The
fairness considerations are mostly on _resource allocation_ by ML models, such
as loan assessment (Lee and Floridi, 2021), job application (Singh and
Joachims, 2018), where the model predictions can unfavorably affect people in
a protected group w.r.t. their protected attributes (Du et al., 2021; Mehrabi
et al., 2021). For example, Bartlett et al. (2022) show that mortgage
algorithms have been discriminatory towards Latino and African Americans,
causing them to pay significantly higher interest rates than other risk-
equivalent borrowers. There have been extensive efforts on developing
algorithms for making fair predictions in resource allocation (Zafar et al.,
2017; Kleinberg et al., 2017; Zhang et al., 2018b; Hu and Chen, 2020; Wang et
al., 2020, 2021; Wang and Joachims, 2021).

In NLP, instead of studying resource allocation fairness, researchers focus on
the _representational fairness_ (Blodgett et al., 2020; Liang et al., 2021;
Sheng et al., 2021) of how language models shape social biases and stereotypes
through natural language understanding (NLU) or generation (NLG). The
representational biases contain stereotyping that propagate negative
generalizations about particular social groups, as well as differences in
system performance for different social groups, or text that misrepresents the
distribution of different social groups (Blodgett et al., 2020). In NLU, as an
example, Kiritchenko and Mohammad (2018) find that sentiment analysis systems
yield different sentiment scores for sentences containing names associated
with African Americans vs. European Americans. The fairness in NLG is even
more emphasized (Sheng et al., 2021) since NLG applications more directly
interact with different users to generate novel content in various domains.
Sheng et al. (2019) enumerate how the generated text can propagate varying
social perceptions (aka regard) of different demographics indicated in the
prompts. Huang et al. (2020) analyze the sentence continuations generated by a
GPT-2 model, and find different sentiment distributions of the completed
sentences when the occupation word is changed in the prompts. Bordia and
Bowman (2019) show that doctor co-occurs more frequently with male pronouns
while nurse co-occurs more often with female pronouns in generated text.
However, these biases are easier to analyze, since both the demographic and
the textual signals of bias are encapsulated within the text. None of them
considers the essential case that the protected attribute of a user being
served is the subject to bias in PTG.

To the best of our knowledge, we are the first to study how personalization
links the bias in NLG to individual user’s protected attributes, leading to
unfair treatment. We propose a unique and important perspective in PTG
fairness: the linguistic quality of generated text should not differentiate
w.r.t. user’s protected attribute.

Fairness Notions: There exist different fairness notions in the literature,
among which group-wise fairness notions are the firstly studied ones (Zafar et
al., 2015; Hardt et al., 2016; Zafar et al., 2017). However, group-wise
fairness has different quantitative definitions and different criteria such as
demographic parity (DP) or equal opportunity (EO) are incompatible in general
(Kleinberg et al., 2017; Berk et al., 2021). Depending on the relationship
between the protected attribute and the data, some definitions can even
increase discrimination (Kusner et al., 2017): e.g., if the protected
attribute do not actually influence the ratio of positive labels in different
groups, requiring DP will make the classifier more discriminatory. Individual
fairness (Zemel et al., 2013; Joseph et al., 2016) requires similar users to
receive similar predictions. But it depends on the specific distance metric
for similarity, which must be carefully chosen, and requires an understanding
of the domain at hand (Dwork et al., 2012). In contrast, counterfactual
fairness (CF) (Kusner et al., 2017) considers fairness from a causal
perspective and requires fairness on an individual in the real-world vs. the
counterfactual world where _only_ the protected attribute of the user is
changed. CF has been widely adopted recently as a more robust fairness notion
(Russell et al., 2017; Wu et al., 2019; Makhlouf et al., 2020), which can also
improve group-wise fairness in certain scenarios (Zhang and Bareinboim, 2018;
Khademi et al., 2019).

Explainable Recommendation: Providing explanations for recommendations have
been shown to help users make better decisions and increase users’ reliance
and trust in the system (Wang et al., 2018; Chen et al., 2021; Zhang and Chen,
2020). Natural language explanations (Chen et al., 2021; Li et al., 2021b;
Yang et al., 2021), which are expressive and flexible, have been extensively
studied recently owing to the advancement of NLG technology. Since natural
language explanations can be applied to different types of recommendations, we
focus on the fairness only in explanation generation, and refer interested
readers to other works (Li et al., 2021a) for fairness in recommendation.

##  3\. Problem Formulation

In the following discussions, we consider a single protected attribute on the
user side. But the proposed framework can be easily adapted to multiple
attributes on either the user or the item side. The value of a user’s
protected attribute is treated as a variable $A\in\mathcal{A}$, where
$\mathcal{A}$ is the set of possible attribute values, e.g.,
$\mathcal{A}=\\{male,\,female\\}$ when gender is the protected attribute. In
the data set for training the explanation generator, each entry is a tuple of
$(u,i,a,s,e)$, corresponding to user ID, item ID, observed user’s protected
attribute value, user’s rating on the item, and the ground-truth explanation.
The explanation generator $G_{\theta}$ is a language model parameterized by
$\theta$. Given a user $u$, an item $i$, and the observed attribute value $a$,
an explanation $y\sim G_{\theta}(u,i|A=a)$ can be sampled from the generator.
The linguistic quality of any generated explanation $y$ is measured by a
function $Q(y)$. In particular, we assume $Q$ is a _black box oracle_ and can
only be queried, which is specified according to the fairness needs. This
design is essential for practical applications, since the linguistic quality
can have different definitions in different scenarios and be evaluated in
different ways such as via a explicit function, human-evaluation, or a
government provided tool. Without loss of generality, we assume higher $Q$
means higher quality. Counterfactual fairness (CF) on the linguistic quality
of generated explanations requires that, given user $u$ and item $i$,

(1) $P(Q(y_{A\leftarrow a})|u,i,A=a)=P(Q(y_{A\leftarrow
a^{\prime}})|u,i,A=a),$

where $y_{A\leftarrow a^{\prime}}\sim G_{\theta}(u,i|A=a^{\prime})$ is the
explanation generated when we counterfactually assign the value of the user’s
protected attribute by $A\leftarrow a^{\prime},a^{\prime}\neq a$. The right
side of Eq. 1 evaluates the quality distribution of explanations generated
_had the user’s protected attribute value been $a^{\prime}$_, given that the
_observed attribute value is $a$_. We refer interested readers to (Kusner et
al., 2017; Li et al., 2021a) for deeper understandings on the rationale and
notation for the counterfactual inference.

Denote the loss of a generated explanation for a given user $u$ and item $i$
by $\mathcal{L}_{gen}(G_{\theta}(u,i|A=a),e)$, which is typically the negative
log-likelihood (NLL) loss or a combination of several losses (Li et al., 2017;
Yang et al., 2021; Li et al., 2021b). We consider training the generator for
fair explanation generation as a constrained optimization problem:

(2) $\displaystyle\min\mathcal{L}_{gen}(G_{\theta}(u,i|A=a),e)$ $\displaystyle
s.t.\;\mathbb{E}_{y_{A\leftarrow a}}[Q(y_{A\leftarrow
a})|u,i,A=a]=\mathbb{E}_{y_{A\leftarrow a^{\prime}}}[Q(y_{A\leftarrow
a^{\prime}})|u,i,A=a].$

For ease of presentation, we consider a single user-item pair, and the total
loss on a data set is simply summed over all user-item pairs with the
constraint applied to each pair. In this work, we apply the first-order moment
(expectation) of the quality of generated explanations to construct the
constraint, and leave the extension to other moment-matching constraints for
future work. We further simplify the expression of the constraint as
$\mathbb{E}[Q(y_{A\leftarrow a})]=\mathbb{E}[Q(y_{A\leftarrow a^{\prime}})]$.

##  4\. Proposed Framework

We propose COFFEE for achieving CF on the quality of personalized explanation
generation. For counterfactual inference on the user’s protected attribute, we
disentangle the attribute representation from the user representation. We then
develop a policy learning method with carefully designed reward for optimizing
CF.

###  4.1. Disentangled Attribute Representation

As discussed in the introduction, existing models for explanation generation
usually take the user and item IDs as input, which map to their embedding
vectors learned by the model to represent the user and the item (Li et al.,
2017, 2021b). The protected attribute and the user’s personal preference are
entangled in the user’s history, and bias that associates the explanation
quality to the protected attribute is implicitly encoded in the embeddings,
making it impossible to perform counterfactual inference. To overcome this, we
consider the protected attribute’s value as a special token input to the model
along with the associated user and item IDs, and propose to learn disentangled
user representations for counterfactual inference (Locatello et al., 2019a, b;
Ma et al., 2019; Zheng et al., 2021). Concretely, we separate the
representations for the protected attribute and the user’s preferences that
are _independent_ from the protected attribute. This is similar to works in
controllable text generation (Hu et al., 2017; Oraby et al., 2018; Shu et al.,
2020; Dathathri et al., 2020), where disentangled attribute tokens are used as
input to control the topic, sentiment, or style of the generated text. Here we
use the attribute token to capture the protected attribute of the target user
in explanation generation.

For a given tuple of $(u,i,a)$, we denote the representation for the protected
attribute of value $a$ as $\mathbf{r}_{a}$, the user’s preference (non-
protected) representation as $\mathbf{r}_{u}$ and item representation as
$\mathbf{r}_{i}$. Therefore, the complete user representation is
$\mathbf{r}_{u}^{a}=[\mathbf{r}_{a},\mathbf{r}_{u}]$. Correspondingly, when
performing counterfactual inference $A\leftarrow a^{\prime}$ on user $u$, we
change the input attribute token from $a$ to $a^{\prime}$, and the new user
representation becomes $\mathbf{r}_{u}^{A\leftarrow
a^{\prime}}=[\mathbf{r}_{a}^{\prime},\mathbf{r}_{u}]$. Each attribute value
has its own representation, learned separately from the user’s preference
representation. The representation of a specific attribute value is shared
across all users having that value, i.e., all male users’ attribute
representation is the same vector $\mathbf{r}_{male}$. We can do the same for
item-side attributes as $\mathbf{r}_{i}^{a}=[\mathbf{r}_{a},\mathbf{r}_{i}]$.
To generalize to $K$ protected attributes, we can simply add $K$ attribute
tokens in the input, each of which is mapped to its separately learned
representations.

Note that disentangling the protected attribute from the user’s preference
representation can not guarantee that $\mathbf{r}_{u}$ will not contain any
information about the protected attribute, potentially inhibiting the accuracy
of counterfactual inference. To further enforce the disentanglement, we
introduce a discriminator $D(r_{u})$, which attempts to predict the protected
attribute value from $\mathbf{r}_{u}$. We then add an adversarial loss on
$\mathbf{r}_{u}$ in Eq. 2 as

(3) $\displaystyle\min\ \
\mathcal{L}_{gen}(G_{\theta}(u,i|A=a),e)+\lambda_{D}\log(D(r_{u},a)),$
$\displaystyle s.t.\;\mathbb{E}[Q(y_{A\leftarrow
a})]=\mathbb{E}[Q(y_{A\leftarrow a^{\prime}})]$

where $D(r_{u},a)$ is the probability of predicting the correct attribute
value $a$ by the discriminator. In this way, we adversarially remove the
protected attribute information from $\mathbf{r}_{u}$, and enforce
$\mathbf{r}_{a}$ to capture all the attribute information. During optimization
with mini-batches, we alternate between the parameter updates of the model and
the discriminator as follows: (1) $X$ batches of updates minimizing the loss
Eq. 3 with $D$ fixed, and (2) $Z$ batches of updates maximizing the the loss
Eq. 3 with the generator $G_{\theta}$ fixed.

###  4.2. Policy Learning for Fairness

Once the user, item, and attribute representations are _learned and fixed_ ,
we can perform counterfactual inference and optimize the generator w.r.t. the
CF constraint. Due to the non-convexity of the constrained fairness
optimization problem, closed-form solutions are impossible. Thus we add the
fairness constraint with Lagrangian relaxation as a regularization in the loss
function (Russell et al., 2017)

(4) $\min\mathcal{L}_{all}+\lambda\big{|}\mathbb{E}[Q(y_{A\leftarrow
a})]-\mathbb{E}[Q(y_{A\leftarrow a^{\prime}})]\big{|},$

where
$\mathcal{L}_{all}=\mathcal{L}_{gen}(G_{\theta}(u,i|A=a),e)+\lambda_{D}\log(D(r_{u},a))$
in Eq. 3, denote all other losses except for the CF constraint, and $\lambda$
is a hyper-parameter for fairness-performance trade-off. However, standard
gradient methods can not be directly derived and applied to optimize this
constraint—the explanations are discretely sampled from the generator, the
quality function $Q$ is an oracle function. Instead, we consider policy
learning for optimization, where the explanation distribution imposed by the
generator is considered as the policy for explanation generation, and the
sampled explanations are actions. We then derive the policy gradients on the
generator’s parameters based on carefully designed rewards of sampled
explanations for optimizing CF. Concretely, for estimating the expectations in
the regularization, we need to sample $N$ explanations and calculate the
average quality for explanations sampled in both the real-world and the
counterfactual world. Denote the regularization term as $\mathcal{L}_{fair}$,
the expectations in the regularization can be decomposed as

(5) $\displaystyle\mathcal{L}_{fair}=$
$\displaystyle\big{|}\mathbb{E}[Q(y_{A\leftarrow
a})]-\mathbb{E}[Q(y_{A\leftarrow a^{\prime}})]\big{|}$ $\displaystyle=$
$\displaystyle\Bigg{|}\frac{1}{N}\sum_{k=1}^{N}Q\big{(}y^{k}_{A\leftarrow
a}\big{)}-\frac{1}{N}\sum_{k=1}^{N}Q\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}\Bigg{|}$ $\displaystyle=$
$\displaystyle\text{sgn}(\Delta)\Bigg{(}\frac{1}{N}\sum_{k=1}^{N}Q\big{(}y^{k}_{A\leftarrow
a}\big{)}-\frac{1}{N}\sum_{k=1}^{N}Q\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}\Bigg{)},$

where $\Delta=\mathbb{E}[Q(y)|x_{A\leftarrow
a},a]-E[Q(y^{\prime})|x_{A\leftarrow a^{\prime}},a^{\prime}]$ and sgn is the
sign function. For an explanation $y^{k}_{A\leftarrow a}$ sampled from the
real-world, its contribution to the regularization term is
$\frac{1}{N}\text{sgn}(\Delta)Q\big{(}y^{k}_{A\leftarrow a}\big{)}$. Since we
are minimizing the regularization to improve fairness, the reward for
explanation $y^{k}_{A\leftarrow a}$ is thus

(6) $r\big{(}y^{k}_{A\leftarrow
a}\big{)}=-\frac{1}{N}\text{sgn}(\Delta)Q\big{(}y^{k}_{A\leftarrow a}\big{)}.$

Similarly, the reward for the a sampled explanation $y^{k}_{A\leftarrow
a^{\prime}}$ in the counterfactual world can be calculated as

(7) $r\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}=\frac{1}{N}\text{sgn}(\Delta)Q\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}.$

In this way, we convert the problem of optimizing CF into maximizing the
rewards of sampled explanations. To maximize the rewards, the policy gradient
(Sutton et al., 1999; Yang et al., 2021) on the generator’s parameters can be
estimated as

(8) $\displaystyle\triangledown_{\theta}\mathcal{L}_{fair}=$
$\displaystyle\sum_{k=1}^{N}\triangledown_{\theta}\log
G_{\theta}\big{(}y^{k}_{A\leftarrow a}\big{)}r\big{(}y^{k}_{A\leftarrow
a}\big{)}$ $\displaystyle+$
$\displaystyle\sum_{k=1}^{N}\triangledown_{\theta}\log
G_{\theta}\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}r\big{(}y^{k}_{A\leftarrow a^{\prime}}\big{)}$

Intuitively, during optimization, the probability of sampled explanations that
contribute more to the unfairness will be demoted to a larger extent, while
the probability of those that contribute to fairness will be promoted
relatively.

Although the rewards are designed to minimize the difference in the qualities
of explanations sampled in the real vs. counterfactual world, we have no
control over how the difference should be minimized during optimization. The
model may arbitrarily improve or decrease the quality of explanations to
achieve fairness, e.g., always generating low quality explanations, which
greatly hurt the generation performance and harm its usefulness in practice.
Therefore, we further design a weighting mechanism to calibrate the rewards
such that the optimization can focus more on improving (or decreasing) the
quality of explanations for achieving fairness, which will empower the
designer to better control the fairness optimization and the performance-
fairness trade-off. Specifically, we introduce a quality promotion weight
$\eta\in[0,1]$ to re-weigh the rewards of explanations in the world with lower
expected quality, and use $1-\eta$ to reweigh the rewards of explanations in
the other world. The weighted rewards are

(9) $\displaystyle r_{w}\big{(}y^{k}_{A\leftarrow
a}\big{)}=-\frac{1}{N}\text{sgn}(\Delta)Q\big{(}y^{k}_{A\leftarrow
a}\big{)}\cdot w(\Delta),$ (10) $\displaystyle r_{w}\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}=\frac{1}{N}\text{sgn}(\Delta)Q\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}\cdot(1-w(\Delta)),$

where
$w=\frac{\text{sgn}(\Delta)+1}{2}(1-\eta)+\Big{(}1-\frac{\text{sgn}(\Delta)+1}{2}\Big{)}\eta$.
We can leverage this weight to direct the optimization—if $\eta>0.5$, the
algorithm will focus more on improving the quality of low-quality explanations
for fairness. Finally, for stability of training, we apply the advantage trick
(Mnih et al., 2016) to regularize the rewards, and each reward becomes its
difference from the average reward in its corresponding world:

(11) $\displaystyle r_{adv}\big{(}y^{k}_{A\leftarrow
a}\big{)}=r_{w}\big{(}y^{k}_{A\leftarrow
a}\big{)}-\bar{r}_{w}\big{(}y^{k}_{A\leftarrow a}\big{)},$ (12) $\displaystyle
r_{adv}\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}=r_{w}\big{(}y^{k}_{A\leftarrow
a^{\prime}}\big{)}-\bar{r}_{w}\big{(}y^{k}_{A\leftarrow a^{\prime}}\big{)}.$

We use $r_{adv}$ for the final policy gradient estimates in Eq. 8.

To train a COFFEE model, we first pre-train the model without the fairness
constraint. Then we fix the latent representations, and add the fairness
constraint for fine-tuning the model.

##  5\. Applying COFFEE to different models

There exist various models for personalized explanation generation (Li et al.,
2017; Chen et al., 2021; Yang et al., 2021; Li et al., 2021b). Almost all of
them use either Transformer (Li et al., 2021b) or RNN (Li et al., 2017; Chen
et al., 2021; Yang et al., 2021) as the generator, which takes the user and
item IDs as input and maps them to their learned embeddings. In this section,
we demonstrate the application of COFFEE to two models: (1) PETER (Li et al.,
2021b) based on Transformer, which is the state-of-the-art explanation
generation model; and (2) NRT (Li et al., 2017) based on RNN, which is one of
the most popularly adopted model.

###  5.1. PETER as Base Model

In PETER (Li et al., 2021b), the user and item IDs are treated as two special
tokens appended at the beginning of each word sequence of the explanation
before inputting to transformer layers. With the PETER masking (Section 4.2 in
(Li et al., 2021b)), the user and the item tokens can attend to each other,
while the word tokens can only attend to past tokens. Either the user or the
item embedding can capture the information of the user-item pair. For better
personalization, a context prediction loss is introduced in addition to the
NLL loss, which aims to predict the words in the explanation without caring
about their order. Besides, rating prediction is also made possible by feeding
the output user or item embedding to an MLP. By default, the user token is
used for rating prediction and the item token is used for context prediction.
We denote the overall loss function of PETER as $\mathcal{L}_{peter}$, which
is the summation of the NLL loss, context prediction loss, and MSE for rating
prediction (Section 4 in (Li et al., 2021b)).

Figure 2. Modified PETER with disentangled representations, where user, item,
and attribute IDs are treated as special tokens appended at the beginning of
each sequence.

To apply COFFEE to PETER, we first disentangle the attribute representation by
adding an attribute token at the beginning of each sequence, besides the user
and item tokens, as illustrated in Figure 2. The user, item and attribute
tokens can attend to each other, while explanation words can only attend to
past tokens. Note that the discriminator for disentangling representations is
applied to the input embedding layer, rather than the output embeddings. The
loss function of applying COFFEE is constructed by replacing the
$\mathcal{L}_{gen}$ in Eq. 3 by $\mathcal{L}_{peter}$.

###  5.2. NRT as Base Model

There are mainly two modules in NRT (Li et al., 2017)—a MLP for rating
prediction, and an RNN for explanation generation. Both modules share the same
user and item embeddings mapped from input user and item IDs for
personalization. In particular, the user and item embeddings are used in the
initial hidden state of the RNN for explanation generation (see Figure 2 in
(Li et al., 2017)). To apply COFFEE to NRT, we simply need to use the
concatenation of disentangled user preference embedding and the attribute
embedding instead of the holistic user embedding, exactly as introduced in
Section 4.1. Similar to PETER, there are also three loss terms in the original
NRT, corresponding to the explanation, context, and rating prediction (see
Section 3 in (Li et al., 2017)). We denote the total loss of NRT by
$\mathcal{L}_{nrt}$, which is used to replace $\mathcal{L}_{gen}$ in Eq. 3 for
applying COFFEE.

##  6\. Experiments

We conduct comprehensive experiments with both PETER and NRT on three popular
datasets: Amazon Video Games, Amazon Movies and TV, and Yelp Restaurant. We
compare COFFEE with several baselines and will release all the codes and
datasets used for the experiments for reproducibility upon publication of the
paper. Due to space limit, we show some qualitative examples of generation
explanations and put the results based on NRT in Appendix B.

###  6.1. Experimental Setup

Table 1. Statistics of the datasets and the protected attributes used in
experiments. The candidate values of the attributes are indicated in
parentheses.

#Users #Items #Reviews Attribute Video Games 9,511 5,173 59,374 User’s gender
(male, female) Movies & TV 28,001 12,888 265,646 User’s gender (male, female)
Yelp 35,714 24,900 1,499,291 Restaurant’s price ($, $$, $$$)

####  6.1.1. Datasets

The first two datasets are the _Video Games_ and the _Movies & TV_ categories
from Amazon reviews222https://nijianmo.github.io/amazon/index.html. These two
datasets only provide the names of users. We use a gender classification
tool333https://pypi.org/project/gender-guesser/ to predict the gender of users
from their names. To guarantee the accuracy of the attribute values, we only
reserve the users whose names can be confidently classified as _male_ or
_female_ and remove those classified as _unknown_. We also study the influence
of the item’s protected attribute on the Yelp
dataset444https://www.yelp.com/dataset, where we use the restaurants price
range as the protected attribute, since the platform should not discriminate a
restaurant simply because the restaurant owner sets lower prices for equally
good food to its counterparts. There are originally four price ranges ($, $$,
$$$, $$$$) provided. However, we found that there are much less four-dollar
restaurants than in other price ranges, and thus we merge the four-dollar
restaurants into the category of three-dollar ones for experiments. The
statistics of the datasets are summarized in Table 1. To construct ground-
truth explanations from reviews, we follow the standard natural language
explanation processing steps (Chen et al., 2021; Yang et al., 2021; Li et al.,
2021b) by first extracting item feature words from the all reviews. Then we
extract from each review sentences that contain at least one of the feature
words as the explanation about a user’s preference on an item. We randomly
split each dataset into training, validation, and testing sets by 80%, 10%,
10%, and ensure that there is at least one record in each subset for every
user and item.

####  6.1.2. Baselines

Since we are the first to study the fairness in terms of the quality of
personalized text generation, there is no existing work that directly
addresses this problem. Therefore, we adapt popular methods in the literature
for the purpose, including pre-processing, in-processing, and post-processing
methods. None of these methods is able to achieve measure-specific fairness
optimization from counterfactual perspectives.

$\bullet$ RAW: Original PETER (Li et al., 2021b) and NRT (Li et al., 2017)
without modifications.
$\bullet$ ADV (in-processing): Adversarial training for fairness by adding a
discriminator to adversarially remove the sensitive information in the learned
representations Li et al. (2021a).
$\bullet$ NORM (pre-processing): We normalize the training data to remove the
bias on group-level. With two demographic groups, we remove the explanations
with the higher (lower) quality in the group with more reviews, until the
difference of average quality between the two groups is below $10\%$ of the
original difference. With more than two groups, we recursively apply the
procedure to the two groups with the maximum difference until the maximum
difference is below the threshold. NORM removes less than $2\%$ of training
data on Amazon datasets, and less than $18\%$ on Yelp.
$\bullet$ BT (pre-processing): Back-translation has been shown in NLP to help
normalize text and reduce bias (Rabinovich et al., 2017; Christiansen et al.,
2021). We pre-process the training data by translating the explanations to
Chinese and then back to English. We use the multilingual model mBART (Tang et
al., 2020) from EasyNMT555https://github.com/UKPLab/EasyNMT to perform the
translation.
$\bullet$ ATTR: In order to evaluate the effectiveness of optimizing the
fairness constraint in COFFEE, we use the model trained without the constraint
($\lambda=0$ in Eq. 4) as a baseline. We call the model ATTR, which means the
attribute is disentangled from the user’s or item’s representations as
introduced in Section 5 but without adding the fairness constraint.
$\bullet$ NATTR (post-processing): We disable the protected attribute token in
ATTR for generating explanations during inference. Specifically, for NATTR-
NRT, we replace the learned attribute embeddings with random embeddings with
values uniformly sampled in $[-1,1]$ for explanation generation. For NATTR-
PETER, we disable the other tokens’ attention on the attribute token during
inference.

For all methods, we train the model on the training set, tune the hyper-
parameters on the validation set, and report the results on the testing set.
Each result point is averaged over 3 runs. We put the detailed model
specifications and parameter tuning in Appendix A.

####  6.1.3. Evaluation Metrics

Our evaluation consists of two categories—fairness evaluation and performance
evaluation.
$\bullet$ Fairness metrics. To specify the quality function $Q$, we first use
length of the text $Q_{len}$ as the measure, which has been shown to strongly
correlate with different linguistic quality traits (e.g., detailedness,
structure, fluency, etc.) as well as human-judged text quality (Louis, 2013;
Fleckenstein et al., 2020). Besides, one of the main purposes of explanation
is to provide informative descriptions on item features for the user to make
more informed decisions. We also use the number of features mentioned in the
explanations $Q_{feat}$ as a quality measure. Different quality measures may
correlate to each other, and we are interested in seeing how optimizing
fairness on one will affect fairness on the other, as well as the
compositional effect of optimizing fairness on both together (Bose and
Hamilton, 2019). The methods applied to optimize fairness on $Q_{len}$,
$Q_{feat}$, and $Q_{len}+Q_{feat}$ are denoted as [model]-L/F/LF,
respectively. One can simply specify $Q$ to any other quality measures under
concern in practice.

We evaluate both individual-level and group-level fairness, with and without
counterfactual perspectives. The first metric denoted as Ind-CF evaluates the
originally defined CF on individuals (Kusner et al., 2017), which is the same
as the regularization term in Eq. 4 averaged over all user-item pairs on the
test set $\mathcal{D}$:

(13) $\text{Ind-
CF}=\frac{1}{|\mathcal{D}|}\sum_{u,i\in\mathcal{D}}\big{|}\mathbb{E}[Q(y_{A\leftarrow
a})|u,i]-\mathbb{E}[Q(y_{A\leftarrow a^{\prime}})|u,i]\big{|}.$

We also evaluate the counterfactual effect on group-level (Coston et al.,
2020). For each attribute value $a$ and the corresponding demographic group,
we counterfactually change the attribute value of all users in this group by
$a^{\prime}\neq a$, and calculate the change on the average quality of this
group. We call this metric Group-CF:

(14) $\displaystyle\text{Group-CF}=$
$\displaystyle\frac{1}{|\mathcal{A}|(|\mathcal{A}|-1)}\sum_{a\in\mathcal{A}}\sum_{a^{\prime}\neq
a}$
$\displaystyle\frac{1}{|\mathcal{D}_{a}|}\Bigg{|}\sum_{u,i\in\mathcal{D}_{a}}\mathbb{E}[Q(y_{A\leftarrow
a})|u,i]-\sum_{u,i\in\mathcal{D}_{a}}\mathbb{E}[Q(y_{A\leftarrow
a^{\prime}})|u,i]\Bigg{|},$

where $\mathcal{D}_{a}$ denotes the demographic group with attribute value
$a$.

Finally, besides counterfactual metrics, we also evaluate COFFEE’s
generalization to improve group-wise fairness by the popular notion of
demographic disparity (DDP) (Zafar et al., 2015):

(15) $\displaystyle\text{DDP}=$
$\displaystyle\frac{2}{|\mathcal{A}|(|\mathcal{A}|-1)}\sum_{a,a^{\prime}\in\mathcal{A}}$
$\displaystyle\Bigg{|}\frac{1}{|\mathcal{D}_{a}|}\sum_{u,i\in\mathcal{D}_{a}}\mathbb{E}[Q(y)|u,i]-\frac{1}{|\mathcal{D}_{a^{\prime}}|}\sum_{u,i\in\mathcal{D}_{a^{\prime}}}\mathbb{E}[Q(y)|u,i]\Bigg{|}.$

All fairness metrics are _lower the better_ , and the quality expectation on
each user-item pair is calculated over $N=3$ explanation samples.

Figure 3. The average length (left) and average number of features (right) in
generated explanations by PETER on the three datasets. Amazon Games and Movies
use users’ gender, and Yelp uses restaurants’ price as protected attributes.

$\bullet$ Performance metrics. We adopt several metrics to comprehensively
evaluate the performance of explanation generation. In addition to the
popularly used BLEU-{1,4} (Papineni et al., 2002) and ROUGE-L (Lin, 2004)
scores, we also include the recently prevalent BERTScore (Zhang* et al.,
2020), which has been shown to better capture the semantic meaning between the
target and generated text and better correlate to human judgements. We
calculate the BERTScore using RoBERTa-large with rescaled
scores666https://github.com/Tiiiger/bert_score. We use F-1 score for ROUGE and
BERTScore by default. Although we focus on explanation generation, we also
evaluate the rating prediction performance by RMSE to examine if different
methods will influence the recommendation performance. The ground-truth
ratings range from $1-5$ for all three datasets. RMSE is lower the better and
the other metrics are higher the better.

###  6.2. Fairness in Explanation Generation

Figure 4. Length distributions of explanations generated by ATTR-PETER. The
left figure plots the distributions for real male users vs. their counterpart
female users in the counterfactual world. The right figure is similarly
plotted for female users vs. their counterfactual male users. The smoothed
distribution curves are produced by kernel density estimation.

We first analyze, without any fairness consideration, how length and number of
mentioned features in model generated explanations can be unfair towards
certain protected attributes on the three datasets, which provides insights
into the need of fairness improvement. The statistics of explanations
generated on the test sets by PETER is shown in Figure 3. On both Amazon Games
and Movies & TVs, the model tends to generate longer explanations with more
features for male users than female users, similarly as in the observed
ground-truth training data. On Yelp, the model generates longer and more
detailed feature descriptions for more expensive restaurants than cheaper
restaurants, similar to the observations in the training set.

We also conduct a case study to understand the fairness issue from a
counterfactual perspective. In particular, We study how changing the user’s or
item’s protected attribute counterfactually will affect the length of
generated explanations, which advocates the needs for CF. We analyze the
generated explanations on Amazon Games by ATTR-PETER and the observations are
similar on the other two datasets. The analysis is displayed in Figure 4,
where we plot the length distributions of generated explanations before and
after counterfactually changing users’ gender. The left figure corresponds to
ground-truth male users and their counterfactual female users. As is shown,
when we change male user’s gender to female, the length of generated
explanations are becoming shorter, with more short explanations and less long
explanations. Similarly, in the right figure, the length will become longer if
we change ground-truth female users gender to male. These results show that
the model generates explanations of different lengths depending on the users’
gender, leading to unfair treatment when serving users. Furthermore, this
study also proves the effectiveness of disentangling the protected attribute’s
representations. The separate attribute value input can successfully correlate
to the quality of generated explanations, laying the foundation for subsequent
counterfactual inference for fairness optimization.

###  6.3. Comparison to Baselines

Table 2. Comparison between COFFEE and baselines based on the PETER model. BL
stand for BLEU and RG denotes ROUGE. BLEU, ROUGE and BERTScore are in
percentage values and others are in absolute values. The best results are
boldfaced, and the second best are underlined. * indicates $p<0.05$ for
significance test over the second best baseline.

PETER Fairness on Length Fairness on #Feature Performance Ind-CF$\downarrow$
Grp-CF$\downarrow$ DDP$\downarrow$ Ind-CF$\downarrow$ Grp-CF$\downarrow$
DDP$\downarrow$ BL1$\uparrow$ BL4$\uparrow$ RG1$\uparrow$ RG2$\uparrow$
RGL$\uparrow$ BERT$\uparrow$ RMSE$\downarrow$ Amazon Games (User’s Gender as
Protected Attribute) RAW - - 5.84 - - 1.18 8.82 1.49 19.95* 5.37* 15.54* 14.14
1.21 ADV - - 3.04 - - 0.62 8.78 1.50 16.13 3.96 13.06 12.84 1.21 NORM-L - -
5.37 - - 1.02 8.79 1.54 18.28 4.80 14.60 14.08 1.21 NORM-F - - 5.67 - - 1.13
8.83 1.51 18.20 4.66 14.42 13.91 1.21 BT - - 4.00 - - 0.65 10.43* 1.55 13.42
2.08 10.44 9.55 1.26 NATTR - - 3.24 - - 0.63 8.76 1.52 16.55 3.86 13.02 12.72
1.21 ATTR 12.42 3.63 6.81 2.68 0.75 1.40 8.91 1.57 17.73 4.45 14.13 14.08 1.19
COFFEE-L 3.87 0.47 1.09 1.18 0.03 0.11 10.14 1.24 18.48 4.75 14.69 14.09 1.19
COFFEE-F 6.98 2.11 3.99 1.56 0.32 0.68 9.77 1.58 16.73 3.98 13.43 14.12 1.20
COFFEE-LF 3.72* 0.37* 0.86* 1.15 0.01 0.08* 10.23 1.24 17.38 3.70 13.62 14.02
1.19 Amazon Movies & TV (User’s Gender as Protected Attribute) RAW - - 3.02 -
- 0.57 11.25 2.20 18.04 4.53 15.48 19.93 1.11 ADV - - 1.43 - - 0.26 11.08 2.16
17.20 4.32 14.37 18.94 1.14 NORM-L - - 1.48 - - 0.28 10.98 2.11 17.11 4.28
14.78 19.53 1.10 NORM-F - - 1.51 - - 0.27 11.23 2.17 17.46 4.42 15.00 19.67
1.14 BT - - 0.94 - - 0.16 11.25 2.18 12.91 1.66 11.02 15.01 1.13 NATTR - -
1.56 - - 0.27 11.24 2.19 17.35 4.40 14.92 19.34 1.17 ATTR 6.46 0.31 1.87 1.47
0.10 0.37 11.19 2.16 17.43 4.63 15.98 19.72 1.13 COFFEE-L 3.47 0.07 0.19 1.16
0.01 0.02 10.37 1.52 19.16 4.46 15.88 19.77 1.08 COFFEE-F 4.91 0.06 0.84 1.13
0.01 0.07 11.28 2.12 17.77 4.60 15.35 20.13* 1.08 COFFEE-LF 3.31* 0.16 0.10*
1.16 0.01 0.00* 10.05 1.39 19.34* 4.56 15.86 19.52 1.08 Yelp (Restaurant’s
Price as Protected Attribute) RAW - - 3.34 - - 0.51 10.24 1.57 18.71 3.11
14.64 19.82 1.12 ADV - - 1.89 - - 0.28 9.87 1.60 17.25 2.64 13.99 18.34 1.18
NORM-L - - 2.09 - - 0.21 8.57 1.00 19.38* 3.27* 14.76 19.14 1.18 NORM-F - -
0.53 - - 0.09 9.96 1.54 17.56 2.81 14.10 19.67 1.16 BT - - 2.02 - - 0.27 10.69
1.61 16.42 2.36 13.29 18.89 1.20 NATTR - - 1.97 - - 0.32 10.08 1.55 18.53 3.06
14.57 19.75 1.17 ATTR 9.64 2.65 3.92 1.50 0.44 0.64 10.00 1.53 18.83 3.16
14.75 19.81 1.12 COFFEE-L 2.42 0.22* 0.17* 0.61 0.03 0.04 11.19 1.64 17.73
2.96 14.49 20.75* 1.11 COFFEE-F 5.78 0.58 0.71 0.84 0.08 0.13 11.17 1.77 17.97
3.00 14.39 20.58 1.10 COFFEE-LF 2.27* 0.31 0.26 0.54* 0.03 0.03* 11.61* 1.72
17.41 2.93 14.31 20.21 1.11

We compare COFFEE to the baselines on both fairness and performance of
personalized explanation generation. In particular, an ideal method should
improve fairness without hurting the performance. We show the results based on
PETER in Table 2 and the results on NRT is in Appendix B. As is shown in the
table, COFFEE always achieves superior fairness on DDP compared to baselines
with a large margin, which demonstrates its strong ability in achieving
fairness via the novel fair policy learning scheme. Besides, the comparison
between COFFEE and ATTR on Ind-CF and Grp-CF proves the efficacy of optimizing
the CF constraint. Moreover, optimizing fairness on $Q_{len}$ helps the
fairness on $Q_{feat}$, while the reverse is not as significant. This verifies
that length is a general and strong indicator for text quality (Guo et al.,
2013; Louis, 2013; Fleckenstein et al., 2020). When optimizing on
$Q_{len}+Q_{feat}$ together, COFFEE can further improve the fairness on either
$Q_{len}$ and $Q_{feat}$, showing that the correlations between $Q_{len}$ and
$Q_{feat}$ assist the fairness optimization.

Importantly, according to the performance metrics, COFFEE achieves strong
fairness results while maintaining high performance on explanation generation.
Although COFFEE may slightly drop the performance compared to the baselines,
e.g., on Amazon Games, it can sometimes even outperform the baselines, as
shown in the results on Amazon Movies & TV and Yelp datasets. This is because
the disentanglement mechanism enables better representation learning (Ma et
al., 2019; Zheng et al., 2021) and increases the flexibility and accuracy of
interactions between the user and item for better explanation generation. It
is worth noting that ADV and NORM can also improve fairness at a decent level,
especially on Amazon Movies and Yelp. However, the improvement on fairness is
less significant than COFFEE, with a similar level of decrease in performance.
Finally, BT can also improve fairness by simply back-translating the training
text to and from another language, which shows the interesting normalization
effect of back-translation. But BT improves fairness with a dramatic sacrifice
on performance, as it may eliminate lots of information about personal
preference during translation.

All the observations and conclusions above can be similarly reached on the NRT
model, as is shown in Appendix B. These promising results demonstrates
COFFEE’s ability to generalize the fairness improvement to different types of
models, and also reflects its flexibility and practicality in real-world
scenarios.

Figure 5. Relative Ind-CF and BERTScore ratio w.r.t. to ATTR-PETER (equal to
$\lambda=0$ in COFFEE). The Ind-CF and BERTScore of ATTR-PETER are $12.42$ and
$14.08$, respectively.

###  6.4. Fairness and Performance Trade-off

We study the trade-off between fairness and performance by tuning the weight
$\lambda$ in Eq. 4 for optimizing the fairness constraint. We use Ind-CF on
length for fairness evaluation since it is the original CF definition (Kusner
et al., 2017) COFFEE directly optimizes, and use BERTScore for performance
evaluation. We plot the results on Amazon Games in Fig. 5, where the y-axis is
the metric values ratio to the results of ATTR with $\lambda=0$. As is shown
in the figure, COFFEE can significantly improve the fairness with a drop of
about 77% on Ind-CF while barely hurt the performance with a drop on BERTScore
only about 2% (when $\lambda=5$). This study shows COFFEE’s outstanding
efficiency and effectiveness in fairness optimization.

##  7\. Conclusion

In this work, we investigate the fairness in personalized text generation,
which has significant social implications in real-world applications. We
propose a general framework called COFFEE for achieving counterfactual
fairness in explanation generation, ensuring that the quality of generated
explanations will not be influenced by the user’s or item’s protected
attributes. Comprehensive results on three datasets shows the efficacy of
COFFEE in achieving fairness without hurting the explanation generation
performance.

This work opens the new direction of fairness in personalized text generation.
We expect this work to encourage researchers to investigate novel fairness
notions in this problem based on different quality measures, and conduct in-
depth analysis on their social impacts. We also plan to generalize the COFFEE
to other personalized text generation settings, such as conversational
systems.

## References

  * (1)
  * Alim et al. (2016) H. Samy Alim, John R. Rickford, and Arnetha F. Ball. 2016\.  _Raciolinguistics: How Language Shapes Our Ideas About Race_.  Oxford University Press.
  * Bartlett et al. (2022) Robert Bartlett, Adair Morse, Richard Stanton, and Nancy Wallace. 2022.  Consumer-lending discrimination in the FinTech Era.  _Journal of Financial Economics_ (2022).
  * Berk et al. (2021) Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. 2021.  Fairness in Criminal Justice Risk Assessments: The State of the Art.  _Sociological Methods & Research_ (2021).
  * Blodgett et al. (2020) Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020.  Language (Technology) is Power: A Critical Survey of “Bias” in NLP. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.
  * Bordia and Bowman (2019) Shikha Bordia and Samuel R. Bowman. 2019.  Identifying and Reducing Gender Bias in Word-Level Language Models. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop_. Association for Computational Linguistics, Minneapolis, Minnesota.
  * Bose and Hamilton (2019) Avishek Joey Bose and William Hamilton. 2019.  Compositional Fairness Constraints for Graph Embeddings.  (2019).
  * Chen et al. (2021) Hanxiong Chen, Xu Chen, Shaoyun Shi, and Yongfeng Zhang. 2021\.  Generate Natural Language Explanations for Recommendation.  https://doi.org/10.48550/ARXIV.2101.03392
  * Christiansen et al. (2021) Jonathan Gabel Christiansen, Mathias Gammelgaard, and Anders Søgaard. 2021.  The Effect of Round-Trip Translation on Fairness in Sentiment Analysis. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.
  * Coston et al. (2020) Amanda Coston, Alan Mishler, Edward H. Kennedy, and Alexandra Chouldechova. 2020.  Counterfactual Risk Assessments, Evaluation, and Fairness. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ _(FAT* ’20)_. Association for Computing Machinery.
  * Dathathri et al. (2020) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020\.  Plug and Play Language Models: A Simple Approach to Controlled Text Generation. In _International Conference on Learning Representations_.
  * Dong et al. (2017) Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, and Ke Xu. 2017\.  Learning to Generate Product Reviews from Attributes. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_. Association for Computational Linguistics.
  * Du et al. (2021) Mengnan Du, Fan Yang, Na Zou, and Xia Hu. 2021\.  Fairness in Deep Learning: A Computational Perspective.  _IEEE Intelligent Systems_ (2021).
  * Dwork et al. (2012) Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012.  Fairness through Awareness. In _Proceedings of the 3rd Innovations in Theoretical Computer Science Conference_ _(ITCS ’12)_. Association for Computing Machinery.
  * Fan et al. (2018) Angela Fan, Mike Lewis, and Yann Dauphin. 2018.  Hierarchical Neural Story Generation. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics.
  * Fleckenstein et al. (2020) Johanna Fleckenstein, Jennifer Meyer, Thorben Jansen, Stefan Keller, and Olaf Köller. 2020\.  Is a Long Essay Always a Good Essay? The Effect of Text Length on Writing Assessment.  _Front Psychol._ (2020).
  * Garg et al. (2019) Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. 2019\.  Counterfactual Fairness in Text Classification through Robustness. In _Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’19)_. Association for Computing Machinery, New York, NY, USA.
  * Groenwold et al. (2020) Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, and William Yang Wang. 2020.  Investigating African-American Vernacular English in Transformer-Based Text Generation. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics.
  * Guo et al. (2013) Liang Guo, Scott A. Crossley, and Danielle S. McNamara. 2013\.  Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study.  _Assessing Writing_ (2013).
  * Hardt et al. (2016) Moritz Hardt, Eric Price, Eric Price, and Nati Srebro. 2016\.  Equality of Opportunity in Supervised Learning. In _Advances in Neural Information Processing Systems_. Curran Associates, Inc.
  * Hu and Chen (2020) Lily Hu and Yiling Chen. 2020.  Fair Classification and Social Welfare. In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ _(FAT* ’20)_. Association for Computing Machinery.
  * Hu et al. (2017) Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017.  Toward Controlled Generation of Text. In _Proceedings of the 34th International Conference on Machine Learning - Volume 70_ _(ICML’17)_. JMLR.org.
  * Huang et al. (2020) Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2020.  Reducing Sentiment Bias in Language Models via Counterfactual Evaluation. In _Findings of the Association for Computational Linguistics: EMNLP 2020_. Association for Computational Linguistics.
  * Joseph et al. (2016) Matthew Joseph, Michael Kearns, Jamie H. Morgenstern, Seth Neel, and Aaron Roth. 2016.  Rawlsian Fairness for Machine Learning.  _ArXiv_ abs/1610.09559 (2016).
  * Khademi et al. (2019) Aria Khademi, Sanghack Lee, David Foley, and Vasant Honavar. 2019\.  Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality. In _The World Wide Web Conference_ _(WWW ’19)_. Association for Computing Machinery.
  * Kiritchenko and Mohammad (2018) Svetlana Kiritchenko and Saif Mohammad. 2018.  Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In _Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics_. Association for Computational Linguistics.
  * Kleinberg et al. (2017) Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017\.  Inherent Trade-Offs in the Fair Determination of Risk Scores. In _8th Innovations in Theoretical Computer Science Conference (ITCS 2017)_ _(Leibniz International Proceedings in Informatics)_. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik.
  * Kusner et al. (2017) Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017.  Counterfactual Fairness. In _Advances in Neural Information Processing Systems_. Curran Associates, Inc.
  * Lee et al. (2021) Jing Yang Lee, Kong Aik Lee, and Woon Seng Gan. 2021\.  DLVGen: A Dual Latent Variable Approach to Personalized Dialogue Generation.
  * Lee and Floridi (2021) Michelle Seng Ah Lee and Luciano Floridi. 2021.  Algorithmic Fairness in Mortgage Lending: From Absolute Conditions to Relational Trade-Offs.  _Minds Mach._ (mar 2021).
  * Li et al. (2021b) Lei Li, Yongfeng Zhang, and Li Chen. 2021b.  Personalized Transformer for Explainable Recommendation. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. Association for Computational Linguistics.
  * Li and Tuzhilin (2019) Pan Li and Alexander Tuzhilin. 2019.  Towards Controllable and Personalized Review Generation.
  * Li et al. (2017) Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. 2017.  Neural Rating Regression with Abstractive Tips Generation for Recommendation. In _Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval_ _(SIGIR ’17)_. Association for Computing Machinery.
  * Li et al. (2021a) Yunqi Li, Hanxiong Chen, Shuyuan Xu, Yingqiang Ge, and Yongfeng Zhang. 2021a.  Towards Personalized Fairness Based on Causal Notion. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ _(SIGIR ’21)_. Association for Computing Machinery, New York, NY, USA.
  * Liang et al. (2021) Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021.  Towards Understanding and Mitigating Social Biases in Language Models. In _ICML_.
  * Lin (2004) Chin-Yew Lin. 2004\.  ROUGE: A Package for Automatic Evaluation of Summaries. In _Text Summarization Branches Out_. Association for Computational Linguistics.
  * Locatello et al. (2019a) Francesco Locatello, Gabriele Abbati, Thomas Rainforth, Stefan Bauer, Bernhard Schölkopf, and Olivier Bachem. 2019a.  On the Fairness of Disentangled Representations. In _Advances in Neural Information Processing Systems_. Curran Associates, Inc.
  * Locatello et al. (2019b) Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. 2019b.  Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations. In _ICML_.
  * Louis (2013) Annie Louis. 2013\.  Predicting Text Quality: Metrics for Content, Organization and Reader Interest.  _Dissertations available from ProQuest_ (2013).
  * Ma et al. (2019) Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019.  _Learning Disentangled Representations for Recommendation_.  Curran Associates Inc., Red Hook, NY, USA.
  * Makhlouf et al. (2020) Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. 2020\.  Survey on Causal-based Machine Learning Fairness Notions.
  * Mehrabi et al. (2021) Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021\.  A Survey on Bias and Fairness in Machine Learning.  _ACM Comput. Surv._ (jul 2021).
  * Mnih et al. (2016) Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016.  Asynchronous Methods for Deep Reinforcement Learning.  (2016).
  * Newman et al. (2008) Matthew Newman, Carla Groom, Lori Handelman, and James Pennebaker. 2008.  Gender Differences in Language Use: An Analysis of 14,000 Text Samples.  _Discourse Processes - DISCOURSE PROCESS_ (05 2008).
  * Ni et al. (2019) Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019.  Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China.
  * Oraby et al. (2018) Shereen Oraby, Lena Reed, Shubhangi Tandon, Sharath T.S., Stephanie Lukin, and Marilyn Walker. 2018\.  Controlling Personality-Based Stylistic Variation with Neural Natural Language Generators. In _Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue_. Association for Computational Linguistics.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002\.  BLEU: A Method for Automatic Evaluation of Machine Translation. In _Proceedings of the 40th Annual Meeting on Association for Computational Linguistics_ _(ACL ’02)_. Association for Computational Linguistics, USA.
  * Powers (2005) Donald E. Powers. 2005\.  Wordiness: A Selective Review of Its Influence, and Suggestions for Investigating Its Relevance in Tests Requiring Extended Written Responses.  _Research Memorandum, Educational Testing Service_ (2005).
  * Rabinovich et al. (2017) Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lucia Specia, and Shuly Wintner. 2017.  Personalized Machine Translation: Preserving Original Author Traits. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_. Association for Computational Linguistics.
  * Russell et al. (2017) Chris Russell, Matt J. Kusner, Joshua R. Loftus, and Ricardo Silva. 2017.  When Worlds Collide: Integrating Different Counterfactual Assumptions in Fairness. In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ _(NIPS’17)_. Curran Associates Inc.
  * Schwartz et al. (2016) H. Schwartz, Maarten Sap, Margaret Kern, Johannes Eichstaedt, Adam Kapelner, Megha Agrawal, Eduardo Blanco, Lukasz Dziurzynski, Gregory Park, David Stillwell, Michal Kosinski, Martin Seligman, and Lyle Ungar. 2016.  Predicting Individual Well-being Through the Language of Social Media.
  * Sheng et al. (2019) Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019.  The Woman Worked as a Babysitter: On Biases in Language Generation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics.
  * Sheng et al. (2021) Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2021.  Societal Biases in Language Generation: Progress and Challenges. In _Proceedings of the Conference of the 59th Annual Meeting of the Association for Computational Linguistics (ACL)_.
  * Shu et al. (2020) Lei Shu, Alexandros Papangelis, Yi-Chia Wang, Gokhan Tur, Hu Xu, Zhaleh Feizollahi, Bing Liu, and Piero Molino. 2020\.  Controllable Text Generation with Focused Variation. In _Findings of the Association for Computational Linguistics: EMNLP 2020_. Association for Computational Linguistics.
  * Singh and Joachims (2018) Ashudeep Singh and Thorsten Joachims. 2018.  Fairness of Exposure in Rankings. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ _(KDD ’18)_. Association for Computing Machinery.
  * Sutton et al. (1999) Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999.  Policy Gradient Methods for Reinforcement Learning with Function Approximation. In _Advances in Neural Information Processing Systems_. MIT Press.
  * Tang et al. (2020) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020\.  Multilingual Translation with Extensible Multilingual Pretraining and Finetuning.  (2020).
  * Tintarev and Masthoff (2015) Nava Tintarev and Judith Masthoff. 2015.  _Explaining Recommendations: Design and Evaluation_.  Springer US.
  * Voigt and Bussche (2017) Paul Voigt and Axel von dem Bussche. 2017.  _The EU General Data Protection Regulation (GDPR): A Practical Guide_.  Springer Publishing Company, Incorporated.
  * Volz et al. (2020) Sarah Volz, Marc-André Reinhard, and Patrick Müller. 2020\.  Why don’t you believe me? Detecting deception in messages written by nonnative and native speakers.  _Applied Cognitive Psychology_ (2020).
  * Wang and Joachims (2021) Lequn Wang and Thorsten Joachims. 2021.  User Fairness, Item Fairness, and Diversity for Rankings in Two-Sided Markets. In _Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval_ _(ICTIR ’21)_. Association for Computing Machinery.
  * Wang et al. (2022) Nan Wang, Lu Lin, Jundong Li, and Hongning Wang. 2022\.  Unbiased Graph Embedding with Biased Graph Observations. In _Proceedings of the ACM Web Conference 2022_ _(WWW ’22)_. Association for Computing Machinery, New York, NY, USA.
  * Wang et al. (2021) Nan Wang, Zhen Qin, Xuanhui Wang, and Hongning Wang. 2021\.  Non-Clicks Mean Irrelevant? Propensity Ratio Scoring As a Correction. In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_ (Virtual Event, Israel) _(WSDM ’21)_. Association for Computing Machinery.
  * Wang et al. (2018) Nan Wang, Hongning Wang, Yiling Jia, and Yue Yin. 2018\.  Explainable Recommendation via Multi-Task Learning in Opinionated Text Data. In _The 41st International ACM SIGIR Conference on Research and Development in Information Retrieval_ _(SIGIR ’18)_. Association for Computing Machinery.
  * Wang et al. (2020) Serena Wang, Wenshuo Guo, Harikrishna Narasimhan, Andrew Cotter, Maya Gupta, and Michael Jordan. 2020\.  Robust Optimization for Fairness with Noisy Protected Groups. In _Advances in Neural Information Processing Systems_. Curran Associates, Inc.
  * Wu et al. (2019) Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. 2019\.  _PC-Fairness: A Unified Framework for Measuring Causality-Based Fairness_.  Curran Associates Inc.
  * Yang et al. (2022) Aobo Yang, Nan Wang, Renqin Cai, Hongbo Deng, and Hongning Wang. 2022.  Comparative Explanations of Recommendations. In _Proceedings of the ACM Web Conference 2022_ _(WWW ’22)_. Association for Computing Machinery.
  * Yang et al. (2021) Aobo Yang, Nan Wang, Hongbo Deng, and Hongning Wang. 2021\.  Explanation as a Defense of Recommendation. In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_ _(WSDM ’21)_. Association for Computing Machinery, New York, NY, USA, 9 pages.
  * Zafar et al. (2017) Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2017.  Fairness Beyond Disparate Treatment and Disparate Impact: Learning Classification without Disparate Mistreatment. In _Proceedings of the 26th International Conference on World Wide Web_ _(WWW ’17)_. International World Wide Web Conferences Steering Committee.
  * Zafar et al. (2015) Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2015.  Learning Fair Classifiers.  _arXiv: Machine Learning_ (2015).
  * Zemel et al. (2013) Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013.  Learning Fair Representations. In _Proceedings of the 30th International Conference on Machine Learning_ _(Proceedings of Machine Learning Research)_. PMLR.
  * Zhang et al. (2018b) Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018b.  Mitigating Unwanted Biases with Adversarial Learning. In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_ _(AIES ’18)_. Association for Computing Machinery.
  * Zhang and Bareinboim (2018) Junzhe Zhang and Elias Bareinboim. 2018.  Equality of Opportunity in Classification: A Causal Approach. In _Advances in Neural Information Processing Systems_. Curran Associates, Inc.
  * Zhang* et al. (2020) Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020\.  BERTScore: Evaluating Text Generation with BERT. In _International Conference on Learning Representations_.
  * Zhang et al. (2019) Wei-Nan Zhang, Qingfu Zhu, Yifa Wang, Yanyan Zhao, and Ting Liu. 2019.  Neural Personalized Response Generation as Domain Adaptation.  _World Wide Web_ 22, 4 (2019).
  * Zhang and Chen (2020) Yongfeng Zhang and Xu Chen. 2020.  Explainable Recommendation: A Survey and New Perspectives.  _Found. Trends Inf. Retr._ 14 (2020), 1–101.
  * Zhang et al. (2018a) Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018a.  Towards Conversational Search and Recommendation: System Ask, User Respond. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_ (Torino, Italy) _(CIKM ’18)_. Association for Computing Machinery, New York, NY, USA.
  * Zheng et al. (2021) Yu Zheng, Chen Gao, Xiang Li, Xiangnan He, Yong Li, and Depeng Jin. 2021\.  Disentangling User Interest and Conformity for Recommendation with Causal Embedding. In _Proceedings of the Web Conference 2021_ _(WWW ’21)_. Association for Computing Machinery.

##  Appendix A Experiment Protocols

Here we elaborate the experiemntal settings, model specifications, and hyper-
parameters. For all models, we set the size of vocabulary to 20,000 by keeping
the most frequent words. By default, we use top-5 sampling (Fan et al., 2018)
as the decoding strategy, and the maximum decoding sequence length is $128$.

For the raw PETER model (Li et al., 2021b), we mostly follow the original
paper’s hyper-parameters. The token embedding dimension is $512$ and the
dimension of feed-forward network is $2,048$. The number of transformer layers
and attention heads are both two. The dropout rate during training is $0.2$.
For NRT (Li et al., 2017), we follow the original paper and set the embedding
dimension to $300$ for all users, items and words. The dimension of hidden
layers is $400$. The dropout rate during training is $0.1$.

COFFEE | PETER | NRT
---|---|---
Games | Movies | Yelp | Games | Movies | Yelp
$\lambda$ | 0.2 | 0.2 | 0.2 | 0.3 | 0.1 | 0.1
$\eta$ | 0.6 | 0.6 | 0.5 | 0.5 | 0.5 | 0.5
Table 3. Weight $\lambda$ for the fairness constraint and the promotion weight
$\eta$ in reward calibration of COFFEE when applied to PETER and NRT on the
three datasets.

COFFEE consists of pre-training the ATTR models with disentangled attribute
representations and then fine-tune them with the counterfactual fairness
constraints. For ATTR-PETER, we just add an additional attribute token to
PETER, with the same embedding dimension for the attribute. For ATTR-NRT, we
disentangle the attribute representation by concatenating to the user or item
embedding an attribute embedding of dimension $100$. The other model
specifications for ATTR-PETER and ATTR-NRT are the same as raw PETER and NRT.
When applying the discriminator for removing the information about the
protected attribute from user or item embeddings, we use a $2$-layer MLP with
hidden size of $512$ as the attribute discriminator. For both ATTR-PETER and
ATTR-NRT, the weight $\lambda_{D}$ of the adversarial loss is set to $0.5$ on
Amazon Games and Yelp datasets, and $0$ on Amazon Movies. We iterate between
one epoch of model training and one epoch of discriminator training until
convergence. After the pre-training of ATTR models, we fix the user, item and
attribute embeddings, remove the loss on rating prediction but keep the loss
on explanation generation, and tune the model with the fairness constraint.
The weight $\lambda$ for the fairness constraint and the promotion weight
$\eta$ in reward calibration are tuned for different models and datasets, and
are listed in Table 3.

For model training, we use Adam as optimizer and the initial learning rate for
training raw models and ATTR models is 1e-4. The initial learning rate is 1e-5
for COFFEE during fine tuning by the fairness constraint. By default, the
batch size is 16. But we use batch size of $8$ during the tuning phase of
COFFEE when applying to PETER models, mainly due to memory issues. For
training raw models and ATTR models, we evaluate the total loss on the
validation set after each epoch, and use the epoch checkpoint when the total
loss is higher in the next 5 consecutive epochs. When COFFEE fine-tunes the
pre-trained ATTR models, we always use the checkpoint after a single epoch,
which already yields promising results from COFFEE.

Table 4. Generated explanations by different models on Amazon Games. We select one male user _Chadwick_ and one female user _Noemi_ , and present the explanations for them on the same item ” _Wii Nunchuk Controller_ ”. PETER | user: _Chadwick_ , item: _Wii Nunchuk Controlle_ | user: _Noemi_ , item: _Wii Nunchuk Controller_
---|---|---
Ground-truth |  | this review is for the white wii nunchuk controller. it is a
---
necessary component in most wii games, and attaches to
the bottom port of every wii remote. it is well-designed,
sturdy, and comfortable to hold. the price is relatively low
for such a controller.
arrived as expected.
RAW |  | i bought it to play wii u and this is a great addition to the wii
---
remote. the gamepad is a nice addition to the wii u gamepad.
the wii u is a must have for any wii u console owner.
worked very well.
COFFEE-L |  | i bought this controller to use the nunchuck. this is a great
---
controller for wii - u owners especially the wii u. the
controller is very responsive and the triggers are awesome.
| i bought this controller for my wii u. it was exactly
---
what it would be expected. i love the feel of the box
and it works great.

##  Appendix B Additional Results

We list some examples of PETER generated explanations on Amazon Games in Table
4. In particular, we select a male user _Chadwick_ who wrote long reviews and
a female user _Noemi_ who wrote short reviews for the same item ” _Wii Nunchuk
Controller_ ”. The raw PETER model follows the ground-truth reviews and
generate detailed long explanation for _Chadwick_ but short explanation for
_Noemi_. In contrast, after applying COFFEE to improve fairness on length of
explanations, the model tend to generate longer and more descriptive
explanation for _Noemi_. These examples demonstrate the effect of COFFEE in
generating fair and high-quality explanations for users with different
protected attribute values.

We present the results on NRT models in Table 5. Similar to the results on
PETER, COFFEE when applied to NRT can significantly outperform the baselines
in terms of fairness improvement. Again, when optimizing both $Q_{len}$ and
$Q_{feat}$ together, COFFEE achieves the best fairness performance, verifying
the effect of correlations between different quality measures. In general,
PETER can generate better explanations than NRT, e.g., on BERTScore, with or
without fairness optimizations, showing the capability of transformers over
RNNs. However, COFFEE still maintains high generation performance based on NRT
when compared to baselines, showing its advantage in generalizing the results
to different models.

Table 5. Comparison between COFFEE and baselines based on the NRT model. BL stand for BLEU and RG denotes ROUGE. BLEU, ROUGE and BERTScore are in percentage values and others are in absolute values. The best results are boldfaced, and the second best are underlined. * indicates $p<0.05$ for significance test over the second best baseline. NRT | Fairness on Length | Fairness on #Feature | Performance
---|---|---|---
Ind-CF$\downarrow$ | Grp-CF$\downarrow$ | DDP$\downarrow$ | Ind-CF$\downarrow$ | Grp-CF$\downarrow$ | DDP$\downarrow$ | BL1$\uparrow$ | BL4$\uparrow$ | RG1$\uparrow$ | RG2$\uparrow$ | RGL$\uparrow$ | BERT$\uparrow$ | RMSE$\downarrow$
Amazon Games (User’s Gender as Protected Attribute)
RAW | - | - | 7.00 | - | - | 1.42 | 8.33 | 1.30 |  19.21 | 3.97 | 14.35* | 13.79 | 1.31
ADV | - | - | 5.16 | - | - | 1.25 | 8.13 | 1.29 | 17.67 | 3.37 | 13.27 | 12.85 | 1.28
NORM-L | - | - | 7.50 | - | - | 1.47 | 8.37 |  1.32 |  19.21 |  3.92 | 14.27 | 13.66 | 1.30
NORM-F | - | - | 8.83 | - | - | 1.79 | 8.17 | 1.26 | 19.89* | 4.27 | 13.65 | 13.49 | 1.30
BT | - | - | 8.46 | - | - | 1.47 | 8.34 | 1.17 | 16.44 | 2.71 | 12.23 | 10.73 |  1.29
NATTR | - | - | 0.78 | - | - | 0.12 | 8.39 | 1.39 | 13.07 | 2.71 | 10.83 | 11.11 | 1.30
ATTR | 15.54 | 7.81 | 8.93 | 6.70 | 1.46 | 1.65 |  8.98 | 1.29 | 18.94 | 3.89 |  14.29 | 13.84 | 1.30
COFFEE-L |  3.74 |  0.72 |  0.54 |  0.90 |  0.03 | 0.03 | 8.99 | 1.25 | 17.65 | 3.07 | 12.77 | 13.82 | 1.30
COFFEE-F | 8.97 | 3.37 | 3.47 | 1.81 | 0.52 | 0.51 | 8.91 | 1.30 | 17.94 | 3.50 | 13.63 |  13.83 | 1.30
COFFEE-LF | 3.23* | 0.56* | 0.49* | 0.77* | 0.02 |  0.04 | 8.67 | 1.26 | 17.11 | 3.11 | 12.46 | 13.69 | 1.30
Amazon Movies & TV (User’s Gender as Protected Attribute)
RAW | - | - | 2.74 | - | - | 0.46 | 10.55 | 1.86 | 18.37 | 3.98 |  15.16 | 19.20 | 1.06
ADV | - | - | 1.56 | - | - | 0.24 | 10.48 | 1.83 |  18.44 | 3.95 | 15.10 | 19.09 | 1.06
NORM-L | - | - | 2.74 | - | - | 0.47 | 10.27 | 1.95 | 17.99 | 3.81 | 14.91 | 19.06 | 1.06
NORM-F | - | - | 2.56 | - | - | 0.46 | 10.53 | 1.91 | 18.09 | 3.89 | 14.96 | 19.03 | 1.06
BT | - | - | 2.69 | - | - | 0.43 | 10.63 | 1.62 | 14.84 | 1.87 | 11.95 | 14.36 | 1.06
NATTR | - | - | 1.87 | - | - | 0.33 | 9.84 |  2.07 | 15.90 | 3.37 | 13.77 | 16.57 | 1.07
ATTR | 7.75 | 0.66 | 2.88 | 1.74 | 0.14 | 0.48 | 10.53 | 1.84 | 18.68* | 4.10 | 15.34* | 19.21 | 1.07
COFFEE-L |  4.66 |  0.15 |  1.38 |  1.22 | 0.04 |  0.23 | 11.06 |  2.07 | 17.42 | 3.91 | 14.90 |  19.29 | 1.07
COFFEE-F | 5.75 | 0.12 | 1.67 | 1.23 | 0.04 | 0.24 |  11.05 | 2.08 | 17.90 |  4.02 | 15.11 | 19.27 | 1.07
COFFEE-LF | 4.30* | 0.12 | 1.19* | 1.10* | 0.04 | 0.19* | 10.52 | 2.02 | 17.04 | 3.92 | 14.77 | 19.35* | 1.07
Yelp (Restaurant’s Price as Protected Attribute)
RAW | - | - | 3.76 | - | - | 0.56 | 10.09 | 1.49 |  19.14 | 3.17 |  14.83 | 19.50 | 1.01
ADV | - | - | 0.93 | - | - | 0.21 | 10.35 | 1.97 | 16.78 | 2.49 | 13.14 | 17.88 | 1.24
NORM-L | - | - | 3.21 | - | - | 0.40 | 8.76 | 1.06 | 18.73 | 2.87 | 14.64 | 18.58 | 1.07
NORM-F | - | - | 1.25 | - | - | 0.15 | 9.88 | 1.42 | 19.20 |  3.14 | 14.85 | 19.48 |  1.06
BT | - | - | 2.06 | - | - | 0.30 | 10.56* | 1.52 | 16.44 | 2.31 | 13.24 | 18.95 | 1.01
NATTR | - | - | 0.54 | - | - | 0.19 | 10.38 | 2.23* | 15.89 | 2.26 | 12.58 | 17.26 | 1.44
ATTR | 10.64 | 3.19 | 3.41 | 1.58 | 0.42 | 0.50 | 10.08 | 1.85 | 18.92 | 3.09 | 14.72 | 19.74 | 1.10
COFFEE-L |  4.17 |  0.29 | 0.19 |  0.75 |  0.12 | 0.00 | 10.29 | 2.02 | 17.97 | 2.87 | 14.12 | 19.53 | 1.10
COFFEE-F | 6.88 | 0.68 | 0.43 | 0.93 | 0.17 | 0.10 | 10.25 | 1.75 | 18.34 | 2.92 | 14.33 | 19.49 | 1.10
COFFEE-LF | 3.97* | 0.16* |  0.21 | 0.69* | 0.09* |  0.02 |  10.45 |  2.07 | 18.65 | 2.86 | 14.25 |  19.67 | 1.10