# Choosing the Right Weights:
Balancing Value, Strategy, and Noise in Recommender Systems

Smitha Milli  Cornell Tech
<EMAIL_ADDRESS>Emma Pierson  Cornell Tech
<EMAIL_ADDRESS>Nikhil Garg  Cornell Tech
<EMAIL_ADDRESS>
###### Abstract

Many recommender systems are based on optimizing a linear weighting of
different user behaviors, such as clicks, likes, shares, etc. Though the
choice of weights can have a significant impact, there is little formal study
or guidance on how to choose them. We analyze the optimal choice of weights
from the perspectives of both users and content producers who strategically
respond to the weights. We consider three aspects of user behavior: value-
faithfulness (how well a behavior indicates whether the user values the
content), strategy-robustness (how hard it is for producers to manipulate the
behavior), and noisiness (how much estimation error there is in predicting the
behavior). Our theoretical results show that for users, upweighting more
value-faithful and less noisy behaviors leads to higher utility, while for
producers, upweighting more value-faithful and strategy-robust behaviors leads
to higher welfare (and the impact of noise is non-monotonic). Finally, we
discuss how our results can help system designers select weights in practice.

##  1 Introduction

Most widely-used recommender systems are based on prediction and optimization
of multiple behavioral signals. For example, a video platform may predict
whether a user will click on a video, how long they will watch it, and whether
or not they will give it a thumbs-up. These predictions need to then be
aggregated into a final score that items for a user will be ranked by.
Typically, the aggregation is done through a linear combination of the
different signals. For example, leaked documents from TikTok (Smith, 2021)
described the objective for ranking as $\mathbb{P}(\text{like)}\cdot
w_{\text{like}}+\mathbb{P}(\text{comment})\cdot
w_{\text{comment}}+\mathbb{E}[\text{playtime}]\cdot
w_{\text{playtime}}+\mathbb{P}(\text{play})\cdot w_{\text{play}}$. Twitter
also recently open-sourced the exact weights on the ten behaviors they use for
ranking (Twitter, 2023).

Unfortunately, the chosen weights can often lead to unintended consequences.
For example, when Facebook introduced emoji reactions, they gave all emoji
reactions a weight five times that of the standard thumbs-up. However, after
evidence that the high weight on the angry reaction led to more
misinformation, toxicity, and low-quality content, its weight was eventually
reduced from five to four, to one and a half, and then finally, to zero times
that of a thumbs up (Merrill and Oremus, 2021). Furthermore, the weights can
have a significant impact on content producers who strategically adapt in
response to them. For example, leaked documents from Facebook stated that,
_“Research conducted in the EU reveals that political parties feel strongly
that the change to the algorithm has forced them to skew negative in their
communications on Facebook, with the downstream effect of leading them into
more extreme policy positions,”_ (Hagey and Horwitz, 2021; Morris, 2021).

Even though they can have a major effect on the emergent dynamics of the
platform, these weights are rarely the topic of formal research and there
exist few guidelines for system designers on how to choose them. In this
paper, we study how to optimally choose weights (for users and producers) when
behaviors can vary along three dimensions that designers consider in practice:
_value-faithfulness_ , _strategy-robustness_ , and _noisiness_. Firstly,
value-faithfulness is how indicative a behavior is of whether the user values
the content or not. This concept is referenced, for example, by TikTok, which
has stated that behaviors are “weighted based on their value to a user”
(TikTok, 2020). Secondly, strategy-robustness refers to how hard it is for
producers to manipulate the behavior. A prime example of this is YouTube’s
shift from focusing less on views and more on explicit user behaviors such as
likes and dislikes, in an effort to curb the rise of clickbait video titles
(Youtube, 2019). Lastly, noisiness, refers to the variance in machine learning
predictions of the behavior. Variance is a common consideration in machine
learning and depends, among other factors, on training set size. Netflix
increasingly relied on implicit behaviors (e.g. views) over explicit behaviors
(e.g. ratings) due to their greater prevalence (Gomez-Uribe and Hunt, 2015).

To study the optimal weight design problem, we posit a model in which two
producers compete for the attention of one user. The recommender system ranks
producers based on a linear combination of predictions of $k$ behaviors.
However, producers can strategically adapt their items to increase the
probability of different user behaviors. User utility depends on being shown a
high value producer, and a producer’s utility is the probability they are
ranked highly minus their costs of strategic manipulation. We find that, for
the user, upweighting behaviors that are more value-faithful and less noisy
leads to higher utility (and strategy-robustness has no impact), while for
producers, upweighting behaviors that are more value-faithful and strategy-
robust leads to higher welfare, i.e., higher average utility (and the impact
of noise is non-monotonic).

In practice, the weight vector that platforms use is chosen by employees,
typically product and engineering teams, based on both performance in A/B
tests and qualitative human judgment (Twitter, 2023; TikTok, 2020; Merrill and
Oremus, 2021; Hagey and Horwitz, 2021). Understanding how different behaviors
compare on the three aspects studied—value-faithfulness, noisiness, and
strategy-robustness—can help system designers narrow down the most relevant
weight vectors to test. This is particularly important as a full grid search
is often too expensive to run as the number of behaviors grows, and some
aspects (such as long-run strategic behavior) are difficult to measure. We
provide examples in three settings—e-commerce, TikTok, and Twitter—to
illustrate how this analysis may be applied. Considering these aspects can
help anticipate potential negative side effects, and in certain cases, our
theoretical results imply constraints on the user-optimal weight vector that
can be used to narrow the search space of weights.

##  2 Related Work

##### Designing weights in recommender systems.

Many recommender systems use a weighted combination of various behavior
predictions to rank items (Smith, 2021; Twitter, 2023). The choice of weights
is typically not automated and is chosen by employees based on performance in
A/B tests, insights from user surveys, and qualtitative judgement (TikTok,
2020; Twitter, 2023; Cameron et al., 2022). The weights can have a large
impact on the emergent dynamics of the platform. For instance, when Facebook
modified their weights in 2018 as part of their transition to the “Meaningful
Social Interactions Metric,” Buzzfeed CEO Jonah Peretti warned that the
changes were promoting the virality of divisive content, thereby incentivizing
its production (Hagey and Horwitz, 2021).

To bypass the manual weight creation process, some prior work tries to rank
content by directly optimizing for users’ latent _value_ for content (Milli et
al., 2021; Maghakian et al., 2022). Latent value is unobserved and must be
inferred from observed behaviors, but importantly, the relationship between
value and the behaviors is empirically learned rather than manually specified
through weights. While more elaborate, automated approaches are intriguing,
the use of simple linear weights is widespread as it provides system designers
with a more interpretable design lever that can be used to shape the platform.
Moreover, the models used in these automated approaches are typically not
capable of accounting for complex effects like strategic behavior, which
humans may be more adept at factoring into their selection.

Despite the importance of the weights, there exists little formal study of
them or guidelines on how to select them. In this paper, we study how to
choose weights when behaviors can vary along three dimensions: _value-
faithfulness_ , _strategy-robustness_ , and _noisiness_. Value-faithfulness
indicates the degree to which a behavior reveals a user’s genuine preference
for an item. Although defining true value can be challenging (Lyngs et al.,
2018), we focus on users’ reflective preferences. Recent research has shown
that an overreliance on learning from implicit, less value-faithful signals
can cause recommender systems to be misaligned with users’ stated, reflective
preferences (Lu et al., 2018; Kleinberg et al., 2022; Agan et al., 2023). Our
work aims to offer guidelines for weight selection when behaviors exhibit
variability in not just value-faithfulness, but also noisiness and strategy-
robustness. For instance, in Section 4, we demonstrate that, for users, there
is a trade-off between choosing value-faithful behaviors and behaviors with
lower estimation noise.

##### Strategic classification, ranking, and recommendation.

Our model considers how strategic producer behavior (alongside estimation
noise and behavior value-faithfulness) should affect the design of recommender
weights. Strategic behavior by content producers, particularly motivated
organizations like news outlets and political parties, has been well-
documented (Morris, 2021; Christin, 2020; Hagey and Horwitz, 2021; Smith,
2023; Meyerson, 2012). In our theoretical model, as in practice, producers
compete against each other (and so, for example, effort by multiple producers
may cancel each other out in equilibria).

In machine learning, strategic adaptation has been primarily been studied in
the field of strategic _classification_ (Brückner et al., 2012; Hardt et al.,
2016). Kleinberg and Raghavan (2020) study how to set linear weights on
observed features that can be strategically changed; Braverman and Garg (2020)
show that noisier signals can lead to better equilibrium outcomes when there
is heterogeneity in producer’s cost functions. Relatively less work in machine
learning has focused on the problem of strategic _contests_ in which
participants must compete to receive desired outcomes, with Liu et al. (2022)
being a notable exception – they consider a rank competition setting in a
single dimension. There exists a rich theory of contests in economics (Hillman
and Riley, 1989; Baye et al., 1996; Lazear and Rosen, 1981; Che and Gale,
2000; Tullock, 1980), and our model is most similar to the classic model of
rank-order tournaments by Lazear and Rosen (1981). To this literature, our
work contributes an analysis of how strategic behavior interacts with value-
faithfulness and noisiness in order to influence the recommender system weight
design problem.

Prior work has also specifically analyzed strategy in recommender systems
(Ben-Porat and Tennenholtz, 2018; Ben-Porat et al., 2020; Jagadeesan et al.,
2022; Hron et al., 2022), studying properties such as genre formation and
producer profit at equilibria (Jagadeesan et al., 2022) or the algorithmic
factors that lead to existence of equilibria (Hron et al., 2022). However, the
prior work does not model the fact that the ranking objective on a recommender
system is typically a linear weighting of _multiple_ behaviors with some
behaviors being easier to game than others. In our work, we focus on the
design of these weights.

##  3 Model

We model a system with one user and two producers. Each producer
$i\in\\{-1,+1\\}$ creates an item whose true value111Here, we consider an
item’s true value to the user to be how the user would value the item upon
reflection (as opposed to their immediate, automatic preference for the item).
to the user is $v(i)$, where $v(1)>v(-1)$. Users can interact with a
producer’s item through $k$ different behaviors. For example, a user may
_click_ , _like_ , and/or _watch_ a video. To rank the producers, the
recommender system creates predictions $\mathbf{y}\in\mathbb{R}^{k}$ of
whether the user will engage with the item using each of these $k$ behaviors
(using historical data from the user and the producers). It then combines the
predictions into a final score $\mathbf{w}^{T}\mathbf{y}$, using a weight
vector $\mathbf{w}\in\mathbb{R}^{k}_{\geq 0}$. User utility depends on being
shown a high value producer, and producer utility depends on being ranked
first.

The platform’s design challenge is to choose weights
$\mathbf{w}\in\mathbb{R}_{\geq 0}^{k}$, to maximize user utility or producer
welfare, i.e., the average utility of both producers. We assume
$\|\mathbf{w}\|_{p}=1$ for some $p$-norm $\|\cdot\|_{p}$, examples of which
include the $\ell^{1}$ or $\ell^{2}$ norm.

We postulate that the predictions $\mathbf{y}(i)\in\mathbb{R}^{k}$
corresponding to each producer $i\in\\{-1,+1\\}$ are:

$\displaystyle\mathbf{y}(i)=v(i)+\mathbf{b}(i)+\bm{\xi}(i)+\mathbf{e}(i),$ (1)

where $v$ reflects the item’s true value; $\mathbf{b}_{j}(i)$ corresponds, for
each behavior $j$ and item $i$, to the user’s bias for engaging with that
item; $\bm{\xi}(i)$ is a noise vector, reflecting variance in the predictions
due to finite sample sizes (Domingos, 2000); $\mathbf{e}_{j}(i)$ corresponds
to producer $i$’s effort in strategically manipulating users to engage in
behavior $j$ on their item.

#### Behavior characteristics

We now detail each component of the prediction $\mathbf{y}(i)$ introduced
above, which represent the three primary behavior characteristics we study:
value-faithfulness, noisiness, and strategy-robustness.

Value-faithfulness. Some behaviors are more indicative of whether the user
values the item than others. For example, explicitly liking an item is more
indicative of value than simply clicking on the item. To model this, each item
$i$ has a behavior-specific bias: $\mathbf{b}(i)\in\mathbb{R}^{k}$. The sum
$v(i)+\mathbf{b}(i)\in\mathbb{R}^{k}$ captures how likely a user is to engage
in a behavior on producer $i$’s item (in the absence of any strategic effort
from the producer). Then, the _value-faithfulness_ of a behavior $j\in[k]$ is

$\displaystyle\mathbf{VF}_{j}=\mathbb{E}[\mathbf{y}_{j}(1)-\mathbf{y}_{j}(-1)]=(v(1)+\mathbf{b}_{j}(1))-(v(-1)+\mathbf{b}_{j}(-1))\,.$
(2)

The higher a behavior’s value-faithfulness, the more likely the user is to
engage in that behavior on the higher-valued item compared to the lower-valued
item. For example, a like has higher value-faithfulness than a click because a
user is more likely to only like a high-valued item while they may click on
both high and low-valued items. Without loss of generality, we assume that
$\mathbf{VF}>\mathbf{0}$ (if a behavior does not have positive value-
faithfulness, we can always consider the opposite of the behavior instead,
e.g. not clicking instead of clicking).

Noisiness. The predictions for the behaviors are made by a machine learning
model trained on a finite dataset. Some behavior predictions may be noisier
than others, for example, especially if that behavior is much sparser (e.g.
likes) and the system has less data on it than other behaviors (e.g. clicks)
(Chen et al., 2018). We model this estimation error as random behavior-
specific noise $\bm{\xi}(i),\bm{\xi}(-i)\sim\mathcal{N}(0,\Sigma)$ where
$\Sigma\in\mathbb{R}^{k\times k}$ is a diagonal matrix. We say that a behavior
$j$ is _noisier_ than a behavior $k$ if its variance $\Sigma_{jj}$ is higher
than $\Sigma_{kk}$.

Strategy-robustness. Given a weight vector $\mathbf{w}$, producers will
strategically adapt their items to get a higher score under $\mathbf{w}$.
Though we consider an item’s true value to be fixed, a producer can put effort
$\mathbf{e}\in\mathbb{R}^{k}_{\geq 0}$ into increasing the probability that
the user interacts with their item with each of the $k$ behaviors. For
example, without increasing the quality of their content, the producer may
craft a clickbait title to entice the user into clicking on it. The producer
incurs a cost $c(\mathbf{e})$ for their effort. The cost is quadratic with
some behaviors being higher cost to manipulate than others:
$c(\mathbf{e})=\frac{1}{2}\mathbf{e}^{\intercal}A\mathbf{e}$ where $A$ is a
diagonal matrix and all entries on the diagonal are unique and positive. We
say that a behavior $i$ is more _strategy-robust_ than behavior $j$ if
$A_{ii}>A_{jj}$.

#### Ranking, utility, and equilibria

Producer one is ranked first if
$\mathbf{w}^{\intercal}\mathbf{y}(1)-\mathbf{w}^{\intercal}\mathbf{y}(-1)>0$.
Or equivalently, producer one is ranked first if
$\epsilon(\mathbf{w})<\mathbb{E}[\mathbf{w}^{\intercal}\mathbf{y}(1)-\mathbf{w}^{\intercal}\mathbf{y}(-1)]$,
where
$\epsilon(\mathbf{w})=\mathbf{w}^{\intercal}\bm{\xi}(-1)-\mathbf{w}^{\intercal}\bm{\xi}(1)$
is the difference in noise terms. Letting $F_{\epsilon}$ be the distribution
of $\epsilon(\mathbf{w})$, the probability that producer $i$ is ranked first
is

$\displaystyle\mathbb{P}_{\mathbf{w}}(R(i)=1\mid\mathbf{e})=\begin{cases}F_{\epsilon}(\mathbb{E}[\mathbf{w}^{\intercal}\mathbf{y}(1)-\mathbf{w}^{\intercal}\mathbf{y}(-1)])&i=1\\\
1-F_{\epsilon}(\mathbb{E}[\mathbf{w}^{\intercal}\mathbf{y}(1)-\mathbf{w}^{\intercal}\mathbf{y}(-1)])&i=-1\end{cases}$
(3)

where $R(i)$ is a random variable indicating producer $i$’s rank.

An individual producer’s expected utility is the probability they are ranked
first minus the incurred cost of manipulation:

$\displaystyle\mathcal{U}_{\text{prod}}^{i}(\mathbf{e}(i),\mathbf{e}(-i);\mathbf{w})=\mathbb{P}_{\mathbf{w}}(R(i)=1\mid\mathbf{e})-c(\mathbf{e}(i))\,.$
(4)

The producer welfare is defined as the average utility of the producers,

$\displaystyle\mathcal{W}_{\text{prod}}(\mathbf{e}(1),\mathbf{e}(-1);\mathbf{w})$
$\displaystyle=\frac{1}{2}\sum_{i}\mathcal{U}_{\text{prod}}^{i}(\mathbf{e}(i),\mathbf{e}(-i);\mathbf{w})=\frac{1}{2}-\frac{\sum_{i}c(\mathbf{e}(i))}{2}\,.$
(5)

The user’s utility is the probability the higher-valued producer is ranked
first:

$\displaystyle\mathcal{U}_{\text{user}}(\mathbf{e}(1),\mathbf{e}(-1);\mathbf{w})=\mathbb{P}_{\mathbf{w}}(R(1)=1\mid\mathbf{e})\,.$
(6)

Producer $i$’s best response given fixed features for the other producer is

$\displaystyle\mathbf{BR}^{i}(\mathbf{e}(-i);\mathbf{w})=\operatorname*{arg\,max}_{\mathbf{q}\in\mathbb{R}^{k}_{\geq
0}}\,\mathcal{U}_{\text{prod}}^{i}(\mathbf{q},\mathbf{e}(-i);\mathbf{w})\,.$
(7)

At a (pure Nash) equilibrium, the best responses of both producers are at a
fixed point, defined formally below.

######  Definition 3.1 (Equilibrium).

Given a fixed weight vector $\mathbf{w}$, an equilibrium consists of a pair of
efforts $(\mathbf{e}^{*}(1),\mathbf{e}^{*}(-1))$ that satisfy
$\mathbf{BR}^{i}(\mathbf{e}^{*}(-i);\mathbf{w})=\mathbf{e}^{*}(i)$ for
$i\in\\{-1,+1\\}$.

##  4 User utility without strategic adaptation

First, we analyze user utility in the absence of strategic adaptation from
producers, i.e., when $\mathbf{e}\triangleq\mathbf{0}$. Even without any
strategic adaptation from producers, it is not obvious what the weights a
practitioner should choose are. (We do not analyze producer welfare in the
non-strategic setting because producer welfare is constant when producers do
not manipulate, i.e.,
$\mathcal{W}_{\text{prod}}(0,0;\mathbf{w})=\frac{1}{2}$.)

One might assume that for the user it would be best to give the most weight to
the most value-faithful behaviors. However, the value-faithful behaviors may
not be the easiest to predict, and thus, may introduce more noise into the
rankings. Indeed, our analysis suggests that the optimal weight vector depends
on a trade-off between choosing behaviors that are more value-faithful and
choosing behaviors with lower estimation noise. This conclusion is formalized
in Theorem 4.1 and illustrated in Figure 1. Omitted proofs for this section
can be found in Appendix A.

Figure 1: This example demonstrates how the user’s optimal weight vector
$\mathbf{w}$ is affected by two factors: value-faithfulness and noise. There
are two behaviors being considered, and the second one is both more value-
faithful and noisier than the first. Here, the weight vector is normalized so
that $||\mathbf{w}||_{1}=1$, and thus, a weight of $0.5$ corresponds to equal
weight on both behaviors. The optimal weight vector assigns more weight
$\mathbf{w}_{2}$ to the second behavior as its relative value-faithfulness
increases and less weight as its relative noisiness increases. The exact
parameters used for the figure can be found in Appendix C.

######  Theorem 4.1.

Without any strategic adaptation, the weight vector that maximizes user
utility is

$\displaystyle\mathbf{w}^{*}=(\Sigma^{-1}\mathbf{VF})/\|\Sigma^{-1}\mathbf{VF}\|_{p}\,.$
(8)

Consequently, the optimal weight on a behavior increases as its value-
faithfulness increases or its variance decreases, stated formally below.

######  Corollary 4.1.

For any behavior, $j\in[k]$, the user-optimal weight on the behavior
$\mathbf{w}^{*}_{j}$ monotonically increases in the behavior’s value-
faithfulness $\mathbf{VF}_{j}$ and monotonically decreases in the behavior’s
variance $\Sigma_{jj}$.

User utility under the optimal weight vector also increases in value-
faithfulness and decreases in variance. Under any weight vector, increasing
the value-faithfulness or decreasing the variance of a behavior will increase
the probability that the higher-valued producer is ranked first, i.e., user
utility. Since this is true for _any_ weight vector, it is also true under the
optimal weight vector.

######  Theorem 4.2.

Let
$\mathcal{U}_{\text{user}}(\mathbf{0},\mathbf{0};\mathbf{w}^{*}(\mathbf{VF},\Sigma))$
be user utility under the user-optimal weight vector $\mathbf{w}^{*}$. For any
behavior, $j\in[k]$, optimal user utility
$\mathcal{U}_{\text{user}}(\mathbf{0},\mathbf{0};\mathbf{w}^{*}(\mathbf{VF},\Sigma))$
monotonically increases in the behavior’s value-faithfulness $\mathbf{VF}_{j}$
and monotonically decreases in the behavior’s variance $\Sigma_{jj}$.

###### Proof.

We can write user utility $\mathcal{U}_{\text{user}}$ as

$\displaystyle\mathcal{U}_{\text{user}}(\mathbf{0},\mathbf{0};\mathbf{w})=\mathbb{P}_{\mathbf{w}}(R(1)=1)=F_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})=\frac{1}{2}\left[1+\mathrm{erf}\left(\frac{\mathbf{w}^{\intercal}\mathbf{VF}}{2\sqrt{\mathbf{w}^{\intercal}\Sigma\mathbf{w}}}\right)\right]\,.$
(9)

For _any_ weight vector $\mathbf{w}\in\mathbb{R}^{k}_{\geq 0}$, user utility
$\mathcal{U}_{\text{user}}(\mathbf{0},\mathbf{0};\mathbf{w})$ is monotonically
increasing in $\mathbf{VF}_{j}$ and monotonically decreasing in $\Sigma_{jj}$
for any behavior $j\in[k]$. Thus, user utility under the _optimal_ weight
vector must also be monotonically increasing in $\mathbf{VF}_{j}$ and
monotonically decreasing in $\Sigma_{jj}$. ∎

Aspect of behavior $j$ | Optimal user utility $\mathcal{U}_{\text{user}}^{*}$ | Optimal producer welfare $\mathcal{W}_{\text{prod}}^{*}$
---|---|---
(5.1) | (Theorem 5.1)
Value-faithfulness $\mathbf{VF}_{j}$ | Increases | Increases
Noise $\Sigma_{jj}$ | Decreases | Non-monotonic
Strategy-robustness $A_{jj}$ | Constant | Increases
Table 1: A summary of our results on the impact of value-faithfulness, noise,
and strategy-robustness on user utility under the user-optimal weight vector
and producer welfare under the producer-optimal weight vector. Figure 2: This
example demonstrates how the user-optimal and producer-optimal weight vector
is affected by three aspects of behavior: value-faithfulness, variance, and
strategy-robustness. There are two behaviors being considered, and the second
one scores higher on all three aspects. The exact parameters used for the
figure can be found in Appendix C.

##  5 User and producer welfare under strategic adaptation

In this section, we analyze user utility and producer welfare under the full
model described in Section 3. Omitted proofs can be found in Appendix B. Given
the weights $\mathbf{w}$ chosen by the designer, producers strategically exert
effort $\mathbf{e}\in\mathbb{R}^{k}_{\geq 0}$ into increasing the probability
that the user engages with their item through each behavior. The choice of
weights must balance between three aspects of behavior at once: value-
faithfulness, strategy-robustness, and noisiness.

In 5.1, we derive the unique equilibrium strategy for producers and find that
the strategy is symmetric, i.e., both producers exert the same effort at
equilibrium.

######  Proposition 5.1 (Equilibrium).

The unique equilibrium strategy for both producers is
$\mathbf{e}^{*}(1)=\mathbf{e}^{*}(-1)=f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})A^{-1}\mathbf{w}$
where $f_{\epsilon}$ is the density of the difference in noise terms
$\epsilon(\mathbf{w})\sim\mathcal{N}(0,2\mathbf{w}^{\intercal}\Sigma\mathbf{w})$.

Symmetric equilibria are commonly seen in the literature on _contests_ in
which agents exert effort towards attaining outcomes that are allocated based
on relative rank (Olszewski and Siegel, 2016; Bodoh-Creed and Hickman, 2018;
Liu et al., 2022). The intuition for the symmetry in our setting is that
producer utility is linear in the probability of being ranked first (which is
zero-sum between producers) and the cost of manipulation (where the cost
function is the same between producers). Thus, if one producer had found it
valuable to expend effort to improve their ranking probability (at the expense
of the other), then the other producer would equally have found it valuable to
do the same. Hence, they have to have the same effort at equilibrium.

Since the equilibrium is symmetric, the probability that producer one is
ranked first is the same as in the non-strategic setting, as producer effort
cancels out. Consequently, for users, both the optimal weight vector and user
utility under the optimal weight vector remain the same in the strategic
setting. Thus, for users, it is better to up-weigh value-faithful behaviors
and down-weigh noisy behaviors (and strategy-robustness has no impact on the
optimal weight vector). (This result would not hold in a model in which effort
is productive, improving producer quality for the user.)

######  Corollary 5.1.

Even with strategic adaptation, the user-optimal weight vector is the same as
in Theorem 4.1. As in the non-strategic setting (Theorem 4.2), for any
behavior $j$, user utility under the user-optimal weight vector is
monotonically increasing in value-faithfulness $\mathbf{VF}_{j}$,
monotonically decreasing in noisiness $\Sigma_{jj}$, and is constant in
strategy-robustness $A_{jj}$.

###### Proof.

From 5.1, the equilibrium strategy for producers is symmetric:
$\mathbf{e}^{*}(1)=\mathbf{e}^{*}(-1)$. When the strategies are symmetric,
then user utility
$\mathcal{U}_{\text{user}}(\mathbf{e}^{*}(1),\mathbf{e}^{*}(-1);\mathbf{w})$
is equal to user utility without strategic adaptation
$\mathcal{U}_{\text{user}}(\mathbf{0},\mathbf{0};\mathbf{w})$. Thus, for
users, the results from the non-strategic setting still hold in the strategic
setting. ∎

At equilibrium, the probability that each producer is ranked first is the same
as it would be if neither producer exerted any effort. Thus, a producer’s
effort is essentially wasted effort but is required in order to “keep up” with
their competitor. Hence, the weight vector that maximizes producer welfare is
the one that most disincentives manipulation among producers. Unlike the user-
optimal weight vector, it is difficult to find a closed-form solution for the
producer-optimal weight vector. The following proposition shows that the
optimal weight vector for producers is the solution to a non-convex
optimization problem. Solving for the weight vector requires minimizing the
product of a convex quadratic form ($\mathbf{w}^{\intercal}A^{-1}\mathbf{w}$)
and a non-convex term (the Gaussian density
$f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})$).

######  Proposition 5.2.

The weight vector that maximizes producer welfare at equilibrium is

$\displaystyle\mathbf{w}^{*}\in\operatorname*{arg\,min}_{\mathbf{w}:||\mathbf{w}||=1}f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})^{2}\mathbf{w}^{\intercal}A^{-1}\mathbf{w}$
(10)

where $f_{\epsilon}$ is the density of the difference in noise terms
$\epsilon(\mathbf{w})\sim\mathcal{N}(0,2\mathbf{w}^{\intercal}\Sigma\mathbf{w})$.

###### Proof.

By 5.1, the unique and symmetric equilibrium strategy for producers is
$\mathbf{e}^{*}(1)=\mathbf{e}^{*}(-1)=f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})A^{-1}\mathbf{w}$.
Thus, producer welfare at equilibrium is equal to

$\displaystyle\mathcal{W}_{\text{prod}}(\mathbf{e}^{*}(1),\mathbf{e}^{*}(-1);\mathbf{w})=\frac{1}{2}-c(\mathbf{e}^{*}(1))=\frac{1}{2}-\frac{f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})^{2}}{2}\mathbf{w}^{\intercal}A^{-1}\mathbf{w}\,,$
(11)

and the optimal weight vector minimizes
$f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})^{2}\mathbf{w}^{\intercal}A^{-1}\mathbf{w}$.
∎

We can, however, characterize how producer welfare at equilibrium under the
optimal weight vector changes as the three aspects of behavior—value-
faithfulness, strategy-robustness, and noisiness—change (Theorem 5.1) We find
that producer welfare under the optimal weight vector increases as strategy-
robustness and value-faithfulness increase: both strategy-robustness and
value-faithfulness disincentivize producer manipulation; strategy-robustness
does so directly, and value-faithfulness does so by making the gap between the
producers’ pre-manipulation scores larger. On the other hand, the relationship
between noisiness and producer welfare is non-monotonic. Intuitively, when
noise is very high, producer welfare is high because manipulation is
disincentivized since the ranking outcome is primarily determined by
randomness rather than their scores. Conversely, when noise is very low,
producer welfare is also high because producers gain little from any
incremental increase in their score, so manipulation is also disincentivized.

######  Theorem 5.1.

Let $\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)$ be the optimal
producer welfare given the exogenous parameters for value-faithfulness
$\mathbf{VF}$, noise $\Sigma$, and strategy-robustness $A$, i.e.,

$\displaystyle\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)=\max_{\mathbf{w}}\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})$
(12)

where $\mathbf{e}_{\mathbf{w}}^{*}(1)$ and $\mathbf{e}_{\mathbf{w}}^{*}(-1)$
are the unique equilibrium strategies in response to $\mathbf{w}$.

For any behavior, $j\in[k]$, the optimal producer welfare
$\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)$ is monotonically
increasing in the behavior’s strategy robustness $A_{jj}$, monotonically
increasing in its value-faithfulness $\mathbf{VF}_{j}$, and is not necessarily
monotonic in its variance $\Sigma_{jj}$.

Furthermore, at the limit, as any of strategy robustness $A_{jj}$, value-
faithfulness $\mathbf{VF}_{j}$, or variance $\Sigma_{jj}$ go to infinity,
producer welfare under the optimal weight vector
$\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)$ reaches the maximum
possible value, i.e.,

$\displaystyle\lim_{z\rightarrow\infty}\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)=1/2~{}~{}~{}\text{for
}~{}z\in\\{A_{jj},\mathbf{VF}_{j},\Sigma_{jj}\\}\,.$ (13)

Figure 2 shows an example of how the user-optimal and producer-optimal weight
on a behavior changes as its value-faithfulness, variance, and strategy-
robustness increase. For producers, the optimal weight is calculated through
numerical approximation. The producer-optimal weight changes in the same way,
as described in Theorem 5.1, that the optimal producer welfare changes as a
function of these three aspects. Though the user and producer-optimal weight
vector remain similar as value-faithfulness changes, they diverge drastically
as noisiness or strategic-robustness change.

##  6 Discussion

We analyzed how three aspects of behavior — value-faithfulness, noisiness, and
strategy-robustness — affect the optimal weight vector and welfare for users
and producers. In practice, the weight vector is chosen based on both
performance in A/B tests and qualitative human judgment (Twitter, 2023;
Merrill and Oremus, 2021). Understanding how these three aspects of behavior
affect user and producer welfare could help platform designers (typically the
product and engineering teams) narrow the search space of weights that are
most relevant to test, since a full grid search may become prohibitively
expensive as the number of behaviors grows.

###  6.1 Example: the three aspects in e-commerce, TikTok, and Twitter

Through an example, we illustrate how system designers could think through
these three aspects to help choose the weights for their platform. First, they
could start by listing out the behaviors and theorizing rankings for each of
the three aspects. Table 2 shows one hypothesized ranking of three behaviors
for three different settings: an e-commerce setting, TikTok, and Twitter. Note
that for certain platforms, such as TikTok or Twitter, users are typically
both consumers and producers. Our hypothesized rankings for value-faithfulness
and strategy-robustness coincide: value-faithful behaviors tend to be more
explicit, reflective behaviors that are harder to game. For the noisiness
rankings, we assume that less frequent behaviors (i.e., smaller training sets)
result in greater variance in estimation, as often seen in theory and practice
(Chen et al., 2018).

For example, for Twitter, we consider three behaviors: _like_ , _retweet_ ,
and _reply_. From the recent open-sourcing of the Twitter algorithm, we know
that these behaviors are used in practice by Twitter. Since Twitter does not
use a separate behavior specifically for quote retweets222On Twitter, users
can “retweet” tweets posted by others to share them with their followers. A
regular retweet simply shares the original tweet. In a quote tweet, the user
shares the original tweet with their own message added to it., we assume that
_retweet_ includes both regular retweets and quote retweets (Twitter, 2023).
We hypothesize that the value-faithfulness and strategy-robustness ordering is
$\mathtt{like}\succ\mathtt{retweet}\succ\mathtt{reply}$ (meaning that likes
are more value-faithful and strategy-robust than the others). Retweets are
ranked lower than likes because quote retweets often express disagreement with
the original poster (Minot et al., 2021). Based on the prevalence of the
behaviors (McClain et al., 2021), we hypothesize the reverse ranking for
noisiness: $\mathtt{reply}\succ\mathtt{retweet}\succ\mathtt{like}$.

Ultimately, the rankings provided here are just for illustration. Platforms
can use a combination of domain expertise and measurement studies to determine
how behaviors rank on these three aspects. Indeed, Facebook conducted surveys
to measure how much users value different kinds of interactions (i.e.,
measuring value-faithfulness) and used them in choosing the weights: _“the
base weight of all the interactions are derived based on producer-side
experiments which measure value to the originator/producer (of the content)”_
(Cameron et al., 2022).333The quote is from a leaked document from the
Facebook Files titled “The Meaningful Social Interactions Metric Revisited:
Part Two”.

The exercise of thinking through these aspects may help foresee certain
negative side effects, especially when it comes to strategic adaptation, which
often takes longer to measure than the typical duration of an A/B test. For
instance, on Twitter, producers sometimes post offensive or sensationalized
messages that receive few likes but many replies and quote tweets expressing
disagreement (a phenomenon called “ratioing” (Troy, 2022; Minot et al.,
2021)). By theorizing why some behaviors may be less strategy-robust,
designers may recognize that overweighing replies or retweets could
inadvertently incentivize producers to game the rankings by intentionally
posting offensive messages that are widely disliked but still gain engagement
in the form of replies or retweets.

Hypothesized behavior rankings
---
| e-commerce | TikTok | Twitter
Value-faithfulness | $\mathtt{order}\succ\mathtt{cart}\succ\mathtt{click}$ | $\mathtt{like}\succ\mathtt{comment}\succ\mathtt{play}$ | $\mathtt{like}\succ\mathtt{RT}\succ\mathtt{reply}$
Strategy-robustness | $\mathtt{order}\succ\mathtt{cart}\succ\mathtt{click}$ | $\mathtt{like}\succ\mathtt{comment}\succ\mathtt{play}$ | $\mathtt{like}\succ\mathtt{RT}\succ\mathtt{reply}$
Noisiness | $\mathtt{order}\succ\mathtt{cart}\succ\mathtt{click}$ | $\mathtt{comment}\succ\mathtt{like}\succ\mathtt{play}$ | $\mathtt{reply}\succ\mathtt{RT}\succ\mathtt{like}$
Theory-implied constraints on user-optimal $\mathbf{w}^{*}$
| e-commerce | TikTok | Twitter
Implied constraints | None | $\mathbf{w}^{*}_{\mathtt{like}}>\mathbf{w}^{*}_{\mathtt{comment}}$ | $\mathbf{w}^{*}_{\mathtt{like}}>\mathbf{w}^{*}_{\mathtt{RT}}>\mathbf{w}^{*}_{\mathtt{reply}}$
Table 2: Three types of platforms and hypothesized rankings of their behaviors
according to the three aspects: value-faithfulness, strategy-robustness, and
noisiness.

Further, our results suggest certain constraints on the ordering of the user-
optimal weights based on the rankings of value-faithfulness and noisiness.
Such constraints can help reduce the range of weights requiring testing, which
is particularly beneficial as the total number of behaviors increases (e.g.,
Twitter uses 10 different behaviors for ranking (Twitter, 2023)). In our
hypothesized ranking for TikTok, likes are both more value-faithful and less
noisy than comments, and thus, in the user-optimal weight vector, we must have
$\mathbf{w}^{*}_{\mathtt{like}}>\mathbf{w}^{*}_{\mathtt{comment}}$. Similarly,
for Twitter, the dominance relations would imply that
$\mathbf{w}^{*}_{\mathtt{like}}>\mathbf{w}^{*}_{\mathtt{retweet}}>\mathbf{w}^{*}_{\mathtt{reply}}$.

###  6.2 Limitations and open questions

Our insights into the three aspects of behavior were derived from a simple
theoretical model that examined one user and two items or producers. In
reality, however, there are numerous users and items—an important direction
for extension of our model. Furthermore, the three aspects of each behavior
might differ depending on the item and user in question. For instance, some
users may be more likely to click on items that they don’t value. In the
simulations provided in Appendix C, we demonstrate how heterogeneity may
affect the optimal weight vector.

Moreover, addressing the issue of producers’ strategic adaptation remains a
challenge in practical applications. Theoretical models of strategic response
are often simplistic, and further research is needed to connect theory and
practice more effectively. One prevalent method for accounting for strategic
effects involves periodically retraining (in this case, periodically changing
the weights), but this may not always be optimal (Perdomo et al., 2020). In
the context of recommender systems, conducting producer-side A/B tests to
gauge strategic effects can be complex due to the need to avoid interference
with ongoing user-side A/B tests and to minimize violations of the Stable Unit
Treatment Value Assumption (SUTVA) (Nandy et al., 2021).

Broader impacts. The weights chosen can have a large effect on the emergent
dynamics of the platform, in some cases even increasing toxicity and
misinformation (Merrill and Oremus, 2021). Our paper describes guidelines for
choosing weights, but it is essential to always be evaluating properties of
the system, e.g., tracking measures of toxicity or misinformation, to mitigate
any possible unintended side effects. Ultimately, we hope that our framework
and its three aspects—value-faithfulness, noisiness, and strategy-
robustness—can be a helpful one for recommender system designers to use when
grappling with the difficult question of what weights to choose.

## Acknowledgements

We thank Gabriel Agostini, Sidhika Balachandar, Ben Laufer, Raj Movva, Kenny
Peng, and Luke Thorburn for feedback on a draft version of the paper.

## References

  * Smith [2021] Ben Smith.  How tiktok reads your mind, Dec 2021.  URL https://www.nytimes.com/2021/12/05/business/media/tiktok-algorithm.html.
  * Twitter [2023] Twitter.  Twitter’s recommendation algorithm - heavy ranker and twhin embeddings, Mar 2023.  URL https://github.com/twitter/the-algorithm-ml/tree/main/projects/home/recap.
  * Merrill and Oremus [2021] Jeremy B Merrill and Will Oremus.  Five points for anger, one for a ‘like’: How facebook’s formula fostered rage and misinformation.  _The Washington Post_ , 26, 2021.
  * Hagey and Horwitz [2021] Keach Hagey and Jeff Horwitz.  Facebook tried to make its platform a healthier place. it got angrier instead.  _The Wall Street Journal_ , 16, 2021.
  * Morris [2021] Loveday Morris.  In poland’s politics, a “social civil war” brewed as facebook rewarded online anger, Oct 2021.  URL https://www.washingtonpost.com/world/2021/10/27/poland-facebook-algorithm/.
  * TikTok [2020] TikTok.  How tiktok recommends videos #foryou, Jun 2020.  URL https://newsroom.tiktok.com/en-us/how-tiktok-recommends-videos-for-you.
  * Youtube [2019] Youtube.  Continuing our work to improve recommendations on youtube, Jan 2019.  URL https://blog.youtube/news-and-events/continuing-our-work-to-improve/.
  * Gomez-Uribe and Hunt [2015] Carlos A Gomez-Uribe and Neil Hunt.  The netflix recommender system: Algorithms, business value, and innovation.  _ACM Transactions on Management Information Systems (TMIS)_ , 6(4):1–19, 2015.
  * Cameron et al. [2022] Dell Cameron, Shoshana Wodinsky, Mack DeGeurin, and Thomas Germain.  Read the facebook papers for yourself, Apr 2022.  URL https://gizmodo.com/facebook-papers-how-to-read-1848702919.
  * Milli et al. [2021] Smitha Milli, Luca Belli, and Moritz Hardt.  From optimizing engagement to measuring value.  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , pages 714–722, 2021.
  * Maghakian et al. [2022] Jessica Maghakian, Paul Mineiro, Kishan Panaganti, Mark Rucker, Akanksha Saran, and Cheng Tan.  Personalized reward learning with interaction-grounded learning (igl).  _arXiv preprint arXiv:2211.15823_ , 2022.
  * Lyngs et al. [2018] Ulrik Lyngs, Reuben Binns, Max Van Kleek, and Nigel Shadbolt.  So, tell me what users want, what they really, really want!  In _Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems_ , pages 1–10, 2018.
  * Lu et al. [2018] Hongyu Lu, Min Zhang, and Shaoping Ma.  Between clicks and satisfaction: Study on multi-phase user preferences and satisfaction for online news reading.  In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_, pages 435–444, 2018.
  * Kleinberg et al. [2022] Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan.  The challenge of understanding what users want: Inconsistent preferences and engagement optimization.  In _Proceedings of the 23rd ACM Conference on Economics and Computation_ , pages 29–29, 2022.
  * Agan et al. [2023] Amanda Y Agan, Diag Davenport, Jens Ludwig, and Sendhil Mullainathan.  Automating automaticity: How the context of human choice affects the extent of algorithmic bias.  Technical report, National Bureau of Economic Research, 2023.
  * Christin [2020] Angèle Christin.  _Metrics at work: journalism and the contested meaning of algorithms_.  Princeton University Press, 2020.
  * Smith [2023] Ben Smith.  _Traffic: Genius, rivalry, and delusion in the billion-dollar race to go viral_.  Penguin Press, 2023.
  * Meyerson [2012] Eric Meyerson.  Youtube now: Why we focus on watch time.  _YouTube Creator Blog. August_ , 10:2012, 2012.
  * Brückner et al. [2012] Michael Brückner, Christian Kanzow, and Tobias Scheffer.  Static prediction games for adversarial learning problems.  _The Journal of Machine Learning Research_ , 13(1):2617–2654, 2012.
  * Hardt et al. [2016] Moritz Hardt, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters.  Strategic classification.  In _Proceedings of the 2016 ACM conference on innovations in theoretical computer science_ , pages 111–122, 2016.
  * Kleinberg and Raghavan [2020] Jon Kleinberg and Manish Raghavan.  How do classifiers induce agents to invest effort strategically?  _ACM Transactions on Economics and Computation (TEAC)_ , 8(4):1–23, 2020.
  * Braverman and Garg [2020] Mark Braverman and Sumegha Garg.  The Role of Randomness and Noise in Strategic Classification.  In Aaron Roth, editor, _1st Symposium on Foundations of Responsible Computing (FORC 2020)_ , volume 156 of _Leibniz International Proceedings in Informatics (LIPIcs)_ , pages 9:1–9:20, Dagstuhl, Germany, 2020\. Schloss Dagstuhl–Leibniz-Zentrum für Informatik.  ISBN 978-3-95977-142-9.  doi: 10.4230/LIPIcs.FORC.2020.9.  URL https://drops.dagstuhl.de/opus/volltexte/2020/12025.
  * Liu et al. [2022] Lydia T Liu, Nikhil Garg, and Christian Borgs.  Strategic ranking.  In _International Conference on Artificial Intelligence and Statistics_ , pages 2489–2518. PMLR, 2022.
  * Hillman and Riley [1989] Arye L Hillman and John G Riley.  Politically contestable rents and transfers.  _Economics & Politics_, 1(1):17–39, 1989.
  * Baye et al. [1996] Michael R Baye, Dan Kovenock, and Casper G De Vries.  The all-pay auction with complete information.  _Economic Theory_ , 8:291–305, 1996.
  * Lazear and Rosen [1981] Edward P Lazear and Sherwin Rosen.  Rank-order tournaments as optimum labor contracts.  _Journal of political Economy_ , 89(5):841–864, 1981.
  * Che and Gale [2000] Yeon-Koo Che and Ian Gale.  Difference-form contests and the robustness of all-pay auctions.  _Games and Economic Behavior_ , 30(1):22–43, 2000\.
  * Tullock [1980] Gordon Tullock.  Efficient rent seeking.  _Toward a Theory of the Rent-Seeking Society_ , 1980.
  * Ben-Porat and Tennenholtz [2018] Omer Ben-Porat and Moshe Tennenholtz.  A game-theoretic approach to recommendation systems with strategic content providers.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Ben-Porat et al. [2020] Omer Ben-Porat, Itay Rosenberg, and Moshe Tennenholtz.  Content provider dynamics and coordination in recommendation ecosystems.  _Advances in Neural Information Processing Systems_ , 33:18931–18941, 2020.
  * Jagadeesan et al. [2022] Meena Jagadeesan, Nikhil Garg, and Jacob Steinhardt.  Supply-side equilibria in recommender systems.  _arXiv preprint arXiv:2206.13489_ , 2022.
  * Hron et al. [2022] Jiri Hron, Karl Krauth, Michael I Jordan, Niki Kilbertus, and Sarah Dean.  Modeling content creator incentives on algorithm-curated platforms.  _arXiv preprint arXiv:2206.13102_ , 2022.
  * Domingos [2000] Pedro Domingos.  A unified bias-variance decomposition.  In _Proceedings of 17th international conference on machine learning_ , pages 231–238. Morgan Kaufmann Stanford, 2000.
  * Chen et al. [2018] Irene Chen, Fredrik D Johansson, and David Sontag.  Why is my classifier discriminatory?  _Advances in neural information processing systems_ , 31, 2018.
  * Olszewski and Siegel [2016] Wojciech Olszewski and Ron Siegel.  Large contests.  _Econometrica_ , 84(2):835–854, 2016.
  * Bodoh-Creed and Hickman [2018] Aaron L Bodoh-Creed and Brent R Hickman.  College assignment as a large contest.  _Journal of Economic Theory_ , 175:88–126, 2018.
  * Minot et al. [2021] Joshua R Minot, Michael V Arnold, Thayer Alshaabi, Christopher M Danforth, and Peter Sheridan Dodds.  Ratioing the president: An exploration of public engagement with obama and trump on twitter.  _PloS one_ , 16(4):e0248880, 2021.
  * McClain et al. [2021] Colleen McClain, Regina Widjaya, Gonzalo Rivero, and Aaron Smith.  The behaviors and attitudes of us adults on twitter.  2021\.
  * Troy [2022] Cassandra Lynn-Collins Troy.  Get ratioed: Questioning the fossil fuel industry’s social license to operate on twitter.  _The Journal of Public Interest Communications_ , 6(1):4–4, 2022.
  * Perdomo et al. [2020] Juan Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt.  Performative prediction.  In _International Conference on Machine Learning_ , pages 7599–7609. PMLR, 2020.
  * Nandy et al. [2021] Preetam Nandy, Divya Venugopalan, Chun Lo, and Shaunak Chatterjee.  A/b testing for recommender systems in a two-sided marketplace.  _Advances in Neural Information Processing Systems_ , 34:6466–6477, 2021.

##  Appendix A Proofs for Section 4

######  Lemma A.1.

Let $\|\cdot\|_{p}$ be a $p$-norm. The partial derivative of
$\|\mathbf{w}\|_{p}$ is

$\displaystyle\frac{\partial}{\partial\mathbf{w}_{i}}\|\mathbf{w}\|_{p}=\left(\frac{|\mathbf{w}_{i}|}{\|\mathbf{w}\|_{p}}\right)^{p-1}\operatorname{sgn}(\mathbf{w}_{i})\,.$
(14)

###### Proof.

The $p$-norm of a vector $\mathbf{w}$ is equal to
$\|\mathbf{w}\|_{p}=\left(\sum_{j}|\mathbf{w}_{j}|^{p}\right)^{(1/p)}$ for
some $p\geq 1$. Taking the derivative with respect to a component
$\mathbf{w}_{i}$ yields,

$\displaystyle\frac{\partial}{\partial\mathbf{w}_{i}}\|\mathbf{w}\|_{p}=\frac{1}{p}\left(\sum_{j}|\mathbf{w}_{j}|^{p}\right)^{\frac{1}{p}-1}\cdot
p|\mathbf{w}_{i}|^{p-1}\operatorname{sgn}(\mathbf{w}_{i})=\left(\frac{|\mathbf{w}_{i}|}{\|\mathbf{w}\|_{p}}\right)^{p-1}\operatorname{sgn}(\mathbf{w}_{i})\,.$
(15)

∎

### Proof of Theorem 4.1

###### Proof.

User utility in the non-strategic setting is equal to

$\displaystyle\mathcal{U}_{\text{user}}(\mathbf{0},\mathbf{0};\mathbf{w})=\mathbb{P}_{\mathbf{w}}(R(1)=1)=F_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})=\frac{1}{2}\left[1+\mathrm{erf}\left(\frac{\mathbf{w}^{\intercal}\mathbf{VF}}{2\sqrt{\mathbf{w}^{\intercal}\Sigma\mathbf{w}}}\right)\right]\,.$
(16)

Since the error function $\mathrm{erf}$ is monotonically increasing on
$[0,\infty)$, the optimal weight vector is simply one which solves the
following optimization problem (for now, let us ignore the constraint that the
weight vector satisfy $\|\mathbf{w}\|_{p}=1$):

$\displaystyle\max_{\mathbf{w}\geq 0}g(\mathbf{w})\text{ where
}g(\mathbf{w})=(\mathbf{w}^{\intercal}\mathbf{VF})/\sqrt{\mathbf{w}^{\intercal}\Sigma\mathbf{w}}\,.$
(17)

Since the objective function is scale-invariant, i.e.,
$g(\mathbf{w})=g(a\mathbf{w})$ for any scalar $a\geq 0$, one can rewrite the
problem as

$\displaystyle\max_{\mathbf{w}\geq 0}\mathbf{w}^{\intercal}\mathbf{VF}\text{
such that }\mathbf{w}^{\intercal}\Sigma\mathbf{w}=1$ (18)

because one can always scale any optimal weight vector for the original
problem in Equation 17 so that $\mathbf{w}^{\intercal}\Sigma\mathbf{w}=1$ is
satisfied. Let $\tilde{\mathbf{w}}=\Sigma^{1/2}\mathbf{w}$ and
$\mathbf{z}=\Sigma^{-1/2}\mathbf{VF}$. Then, the constrained optimization
problem in Equation 18 can be rewritten as

$\displaystyle\max_{\tilde{\mathbf{w}}\geq
0}\tilde{\mathbf{w}}^{\intercal}\mathbf{z}\text{ such that
}\tilde{\mathbf{w}}^{\intercal}\tilde{\mathbf{w}}=1\,.$ (19)

The unique optimal solution to Equation 19 is
$\tilde{\mathbf{w}}=\mathbf{z}/\|\mathbf{z}\|_{2}$. Thus, the unique optimal
weight vector that solves Equation 18 is
$\mathbf{w}=(\Sigma^{-1}\mathbf{VF})/\|\Sigma^{-1}\mathbf{VF}\|_{2}$.
Therefore, the solution set to the original problem in Equation 17 consists of
all vectors $\\{\alpha\Sigma^{-1}\mathbf{VF}\mid\alpha>0\\}$. Thus, the
optimal weight vector that is unit-norm with respect to the $p$-norm
$\|\cdot\|_{p}$ is
$\mathbf{w}=(\Sigma^{-1}\mathbf{VF})/\|\Sigma^{-1}\mathbf{VF}\|_{p}$. ∎

### Proof of 5.1

###### Proof.

Define the vector
$\mathbf{z}=\Sigma^{-1}\mathbf{VF}=(\Sigma_{11}^{-1}\mathbf{VF}_{1},\dots,\Sigma_{kk}^{-1}\mathbf{VF}_{k})$.
By Theorem 4.1, the user-optimal weight vector is
$\mathbf{w}^{*}=\mathbf{z}/\|\mathbf{z}\|_{p}$. To prove that the optimal
weight vector $\mathbf{w}^{*}$ is increasing in the value-faithfulness
$\mathbf{VF}_{i}$ and decreasing in the variance $\Sigma_{i}$ of a behavior
$j\in[k]$, it suffices to prove that for $\mathbf{z}\geq 0$, the optimal
weight vector $\mathbf{w}^{*}$ is increasing in $\mathbf{z}_{j}$. To do so, we
can show that the partial derivative is non-negative:

$\displaystyle\frac{\partial}{\partial\mathbf{z}_{j}}\frac{\mathbf{z}_{j}}{\|\mathbf{z}\|_{p}}$
$\displaystyle=\frac{1}{\|\mathbf{z}\|_{p}^{2}}\left(\|\mathbf{z}\|_{p}-\mathbf{z}_{j}\frac{\partial}{\partial\mathbf{z}_{j}}\|\mathbf{z}\|_{p}\right)$
(20)
$\displaystyle=\frac{1}{\|\mathbf{z}\|_{p}^{2}}\left(\|\mathbf{z}\|_{p}-\mathbf{z}_{j}\left(\frac{\mathbf{z}_{j}}{\|\mathbf{z}\|_{p}}\right)^{p-1}\right)$
(21)
$\displaystyle=\frac{\|\mathbf{z}\|_{p}^{p}-\mathbf{z}_{j}^{p}}{\|\mathbf{z}\|_{p}^{p+1}}\geq
0\,,$ (22)

∎

where Equation 21 uses Lemma A.1, the partial derivative of the $p$-norm, and
the fact that $\mathbf{z}\geq 0$.

##  Appendix B Proofs for Section 5

### Proof of 5.1

###### Proof.

The utility for producer $i$ is equal to

$\displaystyle\mathcal{U}_{\text{prod}}^{i}(\mathbf{e}(i),\mathbf{e}(-i);\mathbf{w})=\begin{cases}F_{\epsilon}(\mathbb{E}[\mathbf{w}^{\intercal}\mathbf{y}(1)-\mathbf{w}^{\intercal}\mathbf{y}(-1)])-c(\mathbf{e}(1))&i=1\\\
1-F_{\epsilon}(\mathbb{E}[\mathbf{w}^{\intercal}\mathbf{y}(1)-\mathbf{w}^{\intercal}\mathbf{y}(-1)])-c(\mathbf{e}(-1))&i=-1\end{cases}\,,$
(23)

where $F_{\epsilon}$ is the CDF of the difference in noise terms
$\epsilon(\mathbf{w})=\mathbf{w}^{\intercal}\bm{\xi}(-1)-\mathbf{w}^{\intercal}\bm{\xi}(1)\sim\mathcal{N}(0,2\mathbf{w}^{\intercal}\Sigma\mathbf{w})$
and the mean difference in producer scores is equal to
$\mathbb{E}[\mathbf{w}^{\intercal}\mathbf{y}(1)-\mathbf{w}^{\intercal}\mathbf{y}(-1)]=\mathbf{w}^{\intercal}\mathbf{VF}+\mathbf{w}^{\intercal}\mathbf{e}(1)-\mathbf{w}^{\intercal}\mathbf{e}(-1)$.

At the equilibrium, the first-order conditions for both producers must be
satisfied:

$\displaystyle\nabla_{\mathbf{e}(1)}\mathcal{U}_{\text{prod}}^{1}(\mathbf{e}(1),\mathbf{e}(-1);\mathbf{w})=f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF}+\mathbf{w}^{\intercal}\mathbf{e}(1)-\mathbf{w}^{\intercal}\mathbf{e}(-1))\mathbf{w}-A\mathbf{e}(1)=0\,,$
(24)
$\displaystyle\nabla_{\mathbf{e}(-1)}\mathcal{U}_{\text{prod}}^{-1}(\mathbf{e}(-1),\mathbf{e}(1);\mathbf{w})=f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF}+\mathbf{w}^{\intercal}\mathbf{e}(1)-\mathbf{w}^{\intercal}\mathbf{e}(-1))\mathbf{w}-A\mathbf{e}(-1)=0\,,$
(25)

where $f_{\epsilon}$ is the density of $\epsilon(\mathbf{w})$. Subtracting
Equations 24 and 25 shows that the equilibrium strategy is symmetric, i.e,
$\mathbf{e}(1)=\mathbf{e}(-1)$ at equilibrium. Substituting
$\mathbf{e}(1)=\mathbf{e}(-1)$ into either equation yields
$\mathbf{e}(i)=f_{\mathbf{w}}(\mathbf{w}^{\intercal}\mathbf{VF})A^{-1}\mathbf{w}$
for $i\in\\{-1,+1\\}$.

To prove sufficiency, we need to consider the second-order conditions and show
that the Hessian of each producer’s utility is negative-definite at the
equilibrium efforts. The Hessian is given by

$\displaystyle\nabla^{2}_{\mathbf{e}(i)}\mathcal{U}_{\text{prod}}^{i}(\mathbf{e}(i),\mathbf{e}(-i);\mathbf{w})$
$\displaystyle=\nabla_{\mathbf{e}(i)}f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF}+\mathbf{w}^{\intercal}\mathbf{e}(i)-\mathbf{w}^{\intercal}\mathbf{e}(-i))\mathbf{w}-A\mathbf{e}(i)$
(26)
$\displaystyle=f^{\prime}_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF}+\mathbf{w}^{\intercal}\mathbf{e}(i)-\mathbf{w}^{\intercal}\mathbf{e}(-i))\mathbf{w}\mathbf{w}^{\intercal}-A\,.$
(27)

When both producers exert equal effort, the Hessian simplifies to
$f^{\prime}_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})\mathbf{w}\mathbf{w}^{\intercal}-A$.
By assumption $\mathbf{VF}>0$, $\mathbf{w}\geq 0$, and $||\mathbf{w}||=1$,
which ensures that the dot product $\mathbf{w}^{\intercal}\mathbf{VF}$ is
positive. When $\mathbf{w}^{\intercal}\mathbf{VF}>0$, the derivative of the
zero-mean Gaussian density
$f^{\prime}_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})$ is negative, making
the Hessian negative-definite. Consequently,
$\mathbf{e}(1)=\mathbf{e}(-1)=f_{\mathbf{w}}(\mathbf{w}^{\intercal}\mathbf{VF})A^{-1}\mathbf{w}$
represents the unique equilibrium. ∎

### Proof of Theorem 5.1

###### Proof.

By 5.1, the unique and symmetric equilibrium strategy for producers is
$\mathbf{e}_{\mathbf{w}}^{*}(1)=\mathbf{e}_{\mathbf{w}}^{*}(-1)=f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})A^{-1}\mathbf{w}$
where $f_{\epsilon}$ is the density of the difference in noise terms
$\epsilon(\mathbf{w})=\mathbf{w}^{\intercal}\bm{\xi}(-1)-\mathbf{w}^{\intercal}\bm{\xi}(1)\sim\mathcal{N}(0,2\mathbf{w}^{\intercal}\mathbf{w})$.
Thus, producer welfare at equilibrium is equal to

$\displaystyle\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})$
$\displaystyle=\frac{1}{2}-\frac{1}{2}c(\mathbf{e}^{*}(1))-\frac{1}{2}c(\mathbf{e}^{*}(-1))$
(28)
$\displaystyle=\frac{1}{2}-\frac{f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})^{2}}{2}\mathbf{w}^{\intercal}A^{-1}\mathbf{w}$
(29)
$\displaystyle=\frac{1}{2}-\frac{1}{4\sqrt{\pi}\mathbf{w}^{\intercal}\Sigma\mathbf{w}}\exp\left(-\frac{(\mathbf{w}^{\intercal}\mathbf{VF})^{2}}{\mathbf{w}^{\intercal}\Sigma\mathbf{w}}\right)\mathbf{w}^{\intercal}A^{-1}\mathbf{w}\,.$
(30)

From the above expression, it is clear that for any fixed weight vector
$\mathbf{w}$, producer welfare
$\mathcal{W}_{\text{prod}}(\mathbf{e}^{*}(1),\mathbf{e}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})$
is monotonically increasing as the strategy-robustness $A_{jj}$ or
value-faithfulness $\mathbf{VF}_{j}$ of any behavior $j\in[k]$ increases.
Thus, producer welfare under the optimal weight vector
$\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)$ must also be
monotonically increasing in strategy-robustness and value-faithfulness.

However, the optimal producer welfare
$\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)$ is not necessarily
monotonic in the behavior’s variance $\Sigma_{jj}$. The variance only affects
producer welfare by changing
$f_{\epsilon}(\mathbf{w}^{\intercal}\mathbf{VF})$, the density of the
difference in noise terms
$\epsilon(\mathbf{w})\sim\mathcal{N}(0,2\mathbf{w}^{\intercal}\mathbf{w})$.
Let $\sigma^{2}=2\mathbf{w}^{\intercal}\Sigma\mathbf{w}$ be the variance of
$\epsilon(\mathbf{w})$. For any fixed weight vector
$\mathbf{w}\in\mathbb{R}_{\geq 0}^{k}$, as $\sigma^{2}$ approaches $0^{+}$ or
$+\infty$, producer welfare approaches its maximum possible value:

$\displaystyle\lim_{\sigma^{2}\rightarrow
0^{+}}\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})=1/2\,,$
(31)
$\displaystyle\lim_{\sigma^{2}\rightarrow\infty}\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})=1/2\,.$
(32)

Thus, the optimal producer welfare also approaches the maximum possible value
as $\sigma^{2}$ approaches either $0^{+}$ or $+\infty$:

$\displaystyle 1/2\geq\lim_{\sigma^{2}\rightarrow
0^{+}}\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)\geq\lim_{\sigma^{2}\rightarrow
0^{+}}\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})=1/2\,,$
(33) $\displaystyle
1/2\geq\lim_{\sigma^{2}\rightarrow\infty}\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)\geq\lim_{\sigma^{2}\rightarrow\infty}\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})=1/2\,,$
(34)

Therefore, the only way for optimal producer welfare to be monotonic in a
behavior’s variance is if it is constant over $\sigma^{2}\in(0,\infty)$, i.e.,
is always equal to $1/2$, for any given value-faithfulness vector
$\mathbf{VF}$ or cost matrix $A$. This is clearly untrue in general, and thus,
optimal producer welfare is not necessarily monotonic in a behavior’s variance
$\Sigma_{jj}$.

Finally, for any fixed weight vector $\mathbf{w}$, we have that producer
welfare approaches the maximal possible value as any of the three aspects of
behavior go to $+\infty$:

$\displaystyle\lim_{z\rightarrow\infty}\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})=1/2$
(35)

for any $z\in\\{A_{jj},\mathbf{VF}_{j},\Sigma_{jj}\mid j\in[k]\\}$.
Furthermore,

$\displaystyle
1/2\geq\lim_{z\rightarrow\infty}\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)\geq\lim_{z\rightarrow\infty}\mathcal{W}_{\text{prod}}(\mathbf{e}_{\mathbf{w}}^{*}(1),\mathbf{e}_{\mathbf{w}}^{*}(-1);\mathbf{VF},\Sigma,A,\mathbf{w})=1/2\,,$
(36)

and thus, the optimal producer welfare also approaches the maximum possible
value:
$\lim_{z\rightarrow\infty}\mathcal{W}_{\text{prod}}^{*}(\mathbf{VF},\Sigma,A)=1/2$.
∎

##  Appendix C Simulations

### Parameters for Figure 1

The parameters used for the simulation are

$\displaystyle\text{value: }v(1)=1,v(-1)=0\,,$ (37)
$\displaystyle\text{behavioral biases:
}\mathbf{b}(-1)=\mathbf{0},\mathbf{b}(1)_{1}=0.75\,,$ (38)
$\displaystyle\text{variance: }\Sigma_{11}=1\,,$ (39) $\displaystyle\text{cost
of manipulation: }A=I\,.$ (40)

The bias $\mathbf{b}(1)_{2}$ and variance $\Sigma_{22}$ of the second behavior
is adjusted so that the second behavior has the relative value-faithfulness
and variance given by the $x$ and $y$-axes.

### Parameters for Figure 2

The default parameters used for each of the subplots are

$\displaystyle\text{value: }v(1)=1,v(-1)=0\,,$ (41)
$\displaystyle\text{behavioral biases:
}\mathbf{b}(-1)=\mathbf{0},\mathbf{b}(1)_{1}=\mathbf{b}(1)_{2}=0.75\,,$ (42)
$\displaystyle\text{variance: }\Sigma_{11}=1,\Sigma_{22}=3\,,$ (43)
$\displaystyle\text{cost of manipulation: }A=I\,.$ (44)

The three subplots are generated by adjusting the bias $\mathbf{b}(1)_{2}$,
variance $\Sigma_{22}$, or cost $A_{22}$ of the second behavior so that it has
the relative value-faithfulness, variance, or strategy-robustness given by the
$x$-axis.

Figure 3: How the optimal weight vector changes as value-faithfulness and
variance change in the homogeneous setting. The default parameters for the
simulations are $\mu_{\texttt{click}}=\mu_{\texttt{rec}}=1$ and
$\Sigma_{11}=\Sigma_{22}=2$. The left figure is generated by plotting the
optimal weight vector as $\mu_{\texttt{rec}}$ increases (and consequently,
when value-faithfulness increases), and the right figure is generated by
increasing $\Sigma_{22}$.

Figure 4: A comparison of the empirical and theoretical optimal weight vector
in the homogeneous and heterogeneous setting. In both settings, the variance
on click is $\Sigma_{11}=0.1$ and the variance on recommend is
$\Sigma_{22}=2$. In the homogeneous setting, $\mu_{\texttt{click}}=1$ and
$\mu_{\texttt{rec}}=3$. In the heterogeneous setting,
$\alpha_{\texttt{click}}=0$, $\beta_{\texttt{click}}=2$,
$\alpha_{\texttt{rec}}=2$, and $\beta_{\texttt{click}}=4$. Thus, the mean
predictions across items are the same in both the homogeneous and
heterogeneous setting: $\mu_{\texttt{click}}=(\mu_{\texttt{i, click}})/n_{+}$
and $\mu_{\texttt{rec}}=(\mu_{\texttt{i, rec}})/n_{+}$. However, while the
theory optimal weight vector and empirical optimal weight vector closely match
in the homogeneous setting, they have a distinct gap in the heterogeneous
setting.

###  C.1 Additional simulations with $n>2$ items

In simulation, we consider the non-strategic setting, i.e., when
$\mathbf{e}\triangleq\mathbf{0}$, and extend it to a setting with $n$
producers or items. Here, the user values $n_{+}$ items and doesn’t value
$n_{-}$ items. All valued items are assumed to have the same positive value
$v_{+}$ while unvalued items have a value of zero. In the two-item setting, we
defined a user’s utility as the probability that the higher-valued item is
ranked first (Equation 6). Then, a natural metric to optimize for in the $n$
item setting is the probability that a randomly-picked valued item is ranked
above a randomly-picked unvalued item, i.e., the AUC.

The user can interact with the items using two different behaviors: (1)
_click_ and (2) _recommend_. In our simulations, we assume that recommend is
more value-faithful than click but also higher variance. To extend value-
faithfulness to the setting with $n$ items, we define the mean value-
faithfulness $\overline{\mathbf{VF}}$ as the mean difference in behavior
scores between valued items and unvalued items:

$\displaystyle\overline{\mathbf{VF}}=\frac{1}{n_{+}}\sum_{i:v(i)>0}\mathbf{y}(i)-\frac{1}{n_{-}}\sum_{i:v(i)=0}\mathbf{y}(i)\,.$
(45)

We investigate two settings: one in which, given their value, each item has
the same mean behavior predictions, and the other, in which items are
heterogeneous. In both, we compare (a) the weight vector that maximizes the
empirical AUC and (b) the user-optimal weight vector given by Theorem 4.1 (in
which we substitute $\overline{\mathbf{VF}}$ for $\mathbf{VF}$).

In the homogeneous setting, the predictions are simulated as

$\displaystyle\mathbf{y}(i)\sim\begin{cases}\mathcal{N}(\mathbf{0},\Sigma)&v(i)=0\\\
\mathcal{N}\left(\begin{bmatrix}\mu_{\texttt{click}}\\\
\mu_{\texttt{rec}}\end{bmatrix},\Sigma\right)&v(i)=v_{+}\end{cases}\,,$ (46)

i.e., all unvalued items have the same mean prediction of $0$ for both clicks
and recommend while all valued items have the same mean prediction
$\mu_{\texttt{click}}>0$ and $\mu_{\texttt{rec}}>0$ for clicks and recommend.

In the heterogeneous setting, the behavior predictions for unvalued items are
simulated the same way, i.e. $\mathbf{y}(i)\sim(\mathbf{0},\Sigma)$ for $i$
such that $v(i)=0$. But for valued items, the mean of the behavior predictions
is heterogeneous, i.e.,

$\displaystyle\mu_{i,\texttt{click}}$
$\displaystyle\sim\mathrm{Unif}[\alpha_{\texttt{click}},\beta_{\texttt{click}}]\,,$
(47) $\displaystyle\mu_{i,\texttt{rec}}$
$\displaystyle\sim\mathrm{Unif}[\alpha_{\texttt{rec}},\beta_{\texttt{rec}}]\,,$
(48) $\displaystyle\mathbf{y}(i)$
$\displaystyle\sim\mathcal{N}\left(\begin{bmatrix}\mu_{i,\texttt{click}}\\\
\mu_{i,\texttt{rec}}\end{bmatrix},\Sigma\right)\,.$ (49)

Figure 3 shows that in the homogeneous setting, the theory-optimal weight
vector and the empirical-optimal weight vector match closely even as value-
faithfulness and variance change. However, Figure 4 demonstrates that in the
heterogeneous setting, the theory-optimal weight vector and the empirical-
optimal weight vector may not necessarily coincide.