PET short=PET, long=Positron Emission Tomography SE short=SE, long=squeeze and
excitation

11institutetext: Biomedical Engineering, Yale University, New Haven, CT, USA
22institutetext: Radiology and Biomedical Imaging, Yale University, New Haven,
CT, USA

# Anatomy-Constrained Contrastive Learning for Synthetic Segmentation without
Ground-truth

Bo Zhou 11 Chi Liu 1122 James S. Duncan 1122

###### Abstract

A large amount of manual segmentation is typically required to train a robust
segmentation network so that it can segment objects of interest in a new
imaging modality. The manual efforts can be alleviated if the manual
segmentation in one imaging modality (e.g., CT) can be utilized to train a
segmentation network in another imaging modality (e.g., CBCT/MRI/PET). In this
work, we developed an anatomy-constrained contrastive synthetic segmentation
network (AccSeg-Net) to train a segmentation network for a target imaging
modality without using its ground-truth. Specifically, we proposed to use
anatomy-constraint and patch contrastive learning to ensure the anatomy
fidelity during the unsupervised adaptation, such that the segmentation
network can be trained on the adapted image with correct anatomical
structure/content. The training data for our AccSeg-Net consists of 1) imaging
data paired with segmentation ground-truth in source modality, and 2) unpaired
source and target modality imaging data. We demonstrated successful
applications on CBCT, MRI, and PET imaging data, and showed superior
segmentation performances as compared to previous methods. Our code is
available at https://github.com/bbbbbbzhou/AccSeg-Net

###### Keywords:

Contrastive Learning, Anatomy-Constraint, Synthetic Segmentation, Unsupervised
Learning

##  1 Introduction

Deep learning based image segmentation has wide applications in various
medical imaging modalities. Over the recent years, numerous segmentation
networks have been proposed to continuously improve the segmentation
performance [1, 2, 3, 4, 5, 6]. These segmentation networks require training
from large amounts of segmentation ground-truth on their target domain to
achieve robust performance. However, a large amount of ground-truth data is
not always available to several imaging modalities, such as intra-procedural
CBCT, gadolinium-enhanced T1 MRI, and PET with different tracers, thus it is
infeasible to directly obtain robust segmentation networks for them. In this
work, we aim to obtain a robust segmenter on target modality without using
target modality’s ground-truth by leveraging the large amounts of source
domain data (e.g., CT) with segmentation ground-truth.

Previous works on synthetic segmentation can be classified into two
categories, including two-stage method [7] and end-to-end methods [8, 9].
Zhang et al. [7] developed a two-step strategy, called TD-GAN, for chest x-ray
segmentation, where they first use a CycleGAN [10] to adapt the target domain
image to the domain with a well-trained segmenter, and then predict the
segmentation on the adapted image. However, the segmentation performance
relies on the image adaptation performance, thus the two-step process may
prone to error aggregation. On the other hand, Kamnitsas et al. [8] developed
an end-to-end unsupervised domain adaptation for MRI brain lesion
segmentation. However, they only used overlapping MRI modalities (e.g., FLAIR,
T2, PD, MPRAGE) in both source and target imaging modalities to ensure
performance. Later, Huo et al. [9] proposed to directly concatenate CycleGAN
and segmenter as an end-to-end network, called SynSeg-Net, and performed
studies on two independent imaging modalities (e.g., MRI and CT). While
SynSeg-Net achieved reasonable performance, there are still several issues.
First, the training image of the segmentation network relies on high-quality
adapted images from the CycleGAN part of SynSeg-Net. Without preserving
anatomy structures and contents during the adaptation, the image could be
adapted to a target domain image with incorrect structure/content, and
negatively impact the subsequent segmentation network’s training. Second,
SynSeg-Net is a heavy design that relies on training 5 different networks
simultaneously which requires careful hyper-parameter tuning, long training
time, and high GPU memory consumption.

To tackle these issues, we developed an anatomoy-constrained contrastive
learning for synthetic segmentation network (AccSeg-Net), where we proposed to
use anatomy-constraint and patch contrastive learning to preserve the
structure and content while only using only 3 sub-networks. Given large
amounts of CT data with segmentation ground-truth available from public
dataset, we used CT as our source domain and validated our method’s
segmentation performance on target domains of CBCT, MRI, and PET. Our
experimental results demonstrated that our AccSeg-Net achieved superior
performance over previous methods. We also found our AccSeg-Net achieved
better performance as compared to fully supervised methods trained with
relatively limited amounts of ground-truth on the target domain.

Figure 1: The architecture of AccSeg-Net. Our AccSeg-Net consisting of a
generator, a discriminator, and a segmented, is trained in an end-to-end
fashion by 5 loss functions.

##  2 Methods

Our AccSeg-Net includes two parts: 1) an anatomy-constraint contrastive
translation network (Acc-Net), and 2) a segmentation network for the target
domain segmentation. The architecture and training/test stages are shown in
Figure 1. Acc-Net aims to adapt images from domain $\mathbb{A}$ to domain
$\mathbb{B}$. The anatomy-constraint loss and contrastive loss ensure
structural information are not lost during the unpaired domain adaptation
process, thus critical for training a robust segmenter in domain $\mathbb{B}$.
On the other hand, the segmentation loss from the segmentation network is also
back-propagated into the Acc-Net, providing addition supervision information
for Acc-Net to synthesize image with correct organ delineation. More
specifically, our AccSeg-Net contains a generator, a discriminator, and a
segmenter. The generator $G$ adapts images from domain $\mathbb{A}$ to domain
$\mathbb{B}$, the discriminator $D$ identifies real image from domain
$\mathbb{B}$ or the adapted ones from $G$, and the segmenter $M_{B}$ predicts
the segmentation $\hat{S_{B}}$ on adapted image from generator $G$. Training
supervision comes from five sources:
(a) adversarial loss $\mathcal{L}_{adv}$ uses discriminator to minimize the
perpetual difference between the generative image and the ground truth image
in domain $\mathbb{B}$ by:

$\displaystyle\mathcal{L}_{adv}(G,D,I_{B},I_{A})=\operatorname{\mathbb{E}}_{I_{B}\sim\mathbb{B}}\left[\log{D(I_{B})}\right]+\operatorname{\mathbb{E}}_{I_{A}\sim\mathbb{A}}\left[\log{(1-D(G(I_{A})))}\right]$
(1)

(b) identity loss $\mathcal{L}_{idt}$ ensures the generator does not change
target domain appearance when real sample in domain $\mathbb{B}$ is fed:

$\displaystyle\mathcal{L}_{idt}=\operatorname{\mathbb{E}}_{I_{B}\sim\mathbb{B}}\left[\|G(I_{B})-I_{B}\|_{2}^{2}\right]$
(2)

(c) segmentation loss $\mathcal{L}_{seg}$ is computed from the segmentation
prediction from the adapted image $\hat{I_{B}}$ and the ground truth
segmentation label from domain $\mathbb{A}$:

$\displaystyle\mathcal{L}_{seg}=\operatorname{\mathbb{E}}_{I_{A}\sim\mathbb{A}}\left[1-\frac{2|M_{B}(G(I_{A}))\cap
S_{A}|}{|M_{B}(G(I_{A}))|+|S_{A}|}\right]$ (3)

Figure 2: Details of our patch contrastive loss ($\mathcal{L}_{PCT}$). The
encoder (E) of the generator (G) followed by a fully-connected layer encodes
the image patches into patch feature vectors. The $\mathcal{L}_{PCT}$ is
computed using the patch feature vectors.

(d) patch contrastive loss $\mathcal{L}_{PCT}$ aims to associate the shared
parts between the input image and the output image, while disassociating the
non-shared parts. As illustrated in Figure 2, the kidney in the synthesized
image $\hat{I}_{B}$ should have stronger association with the kidney in the
input image $I_{A}$ than the other parts of the input image. We use the
already trained encoder $E$ of generator $G$ (Figure 1) to generate a feature
stack of the image by choosing L layers of interest in E. Each spatial
location within the individual level of the stack represents a patch of the
input image [11]. In our implementation, each feature level goes through an
additional fully-connected layer $H_{l}$. Thus, the feature stack can be
formulated as $\\{f_{l}\\}_{L}=\\{H_{l}(E_{l}(I_{A}))\\}_{L}$, where $E_{l}$
is the l-th layer in E and $H_{l}$ is the full-connected layer. Denoting the
number of spatial location in each layer as $s\in\\{1,...,S_{l}\\}$ and the
number of feature channel in each layer as $C_{l}$, we can write the input
associated feature as $f_{l}^{s}\in\mathbb{R}^{1\times{C_{l}}}$ and the input
disassociated feature as $f_{l}^{S\backslash
s}\in\mathbb{R}^{{(S_{l}-1)}\times{C_{l}}}$. Similarly, the generator output’s
associated feature stack can be written as
$\\{\hat{f}_{l}\\}_{L}=\\{H_{l}(E_{l}(G(I_{A})))\\}_{L}$. Then, we can compute
our patch contrastive loss [12] by:

$\displaystyle\mathcal{L}_{PCT}=\operatorname{\mathbb{E}}_{I_{A}\sim\mathbb{A}}\left[\sum_{l=1}^{L}\sum_{s=1}^{S_{l}}\mathcal{L}_{CE}(\hat{f}_{l}^{s},f_{l}^{s},f_{l}^{S\backslash
s})\right]$ (4)
$\displaystyle\mathcal{L}_{CE}(f,f^{+},f^{-})=-\log\left[\frac{\exp{(f\cdot
f^{+}/\alpha)}}{\exp{(f\cdot f^{+}/\alpha)}+\sum_{n=1}^{N}\exp{(f\cdot
f^{-}/\alpha)}}\right]$ (5)

where $\alpha$ is a temperature hyper-parameter for scaling the feature vector
distance [13], and is empirically set to $\alpha=0.07$ here. The patches are
randomly cropped during the training.

(e) anatomy-constraint loss $\mathcal{L}_{AC}$ ensures the adaptation only
alter the appearance of image while maintaining the anatomical structure, such
that segmentation network $M_{B}$ can be trained correctly to recognize the
anatomical content in the adapted image. We use both MIND loss [14] and
correlation coefficient (CC) loss to preserve the anatomy in our unpaired
adaptation process:

$\mathcal{L}_{AC}=\lambda_{cc}\operatorname{\mathbb{E}}_{I_{A}\sim\mathbb{A}}\left[\frac{Cov(G(I_{A}),I_{A})}{\sigma_{G(I_{A})}\;\sigma_{I_{A}}}\right]+\lambda_{mind}\operatorname{\mathbb{E}}_{I_{A}\sim\mathbb{A}}\left[\|F(G(I_{A}))-F(I_{A})\|_{1}\right]$
(6)

where the first term is the correlation coefficient loss. $Cov$ is the
variance operator and $\sigma$ is the standard deviation operator. The second
term is the MIND loss, and $F$ is a modal-independent feature extractor
defined as $F_{x}(I)=\frac{1}{Z}exp\left(-\frac{K_{x}(I)}{V_{x}(I)}\right)$.
Specifically, $K_{x}(I)$ is a distance vector of image patches around voxel
$x$ with all the neighborhood patches within a non-local region in image $I$.
$V_{x}(I)$ is the local variance at voxel x in image $I$. Here, dividing
$K_{x}(I)$ with $V_{x}(I)$ reduce the influence of image modality and
intensity range, and $Z$ is a normalization constant to ensure that the
maximum element of $F_{x}$ equals to $1$. We set
$\lambda_{cc}=\lambda_{mind}=1$ here to achieve a balanced training.

Finally, the overall objective is a weighted combination of all loss listed
above:

$\displaystyle\mathcal{L}_{all}=\lambda_{1}\mathcal{L}_{adv}+\lambda_{2}\mathcal{L}_{idt}+\lambda_{3}\mathcal{L}_{seg}+\lambda_{4}\mathcal{L}_{PCT}+\lambda_{5}\mathcal{L}_{AC}$
(7)

where weighting parameters are set to
$\lambda_{1}=\lambda_{2}=\lambda_{3}=\lambda_{4}=\lambda_{5}=1$ to achieve a
balanced training. We use a decoder-encoder network with 9 residual bottleneck
for our generators, and a 3-layer CNN for our discriminators. For segmenter,
we use a default setting of a 5-level UNet with concurrent SE module [3]
concatenated to each level’s output, named DuSEUNet. The segmenter in AccSeg-
Net is interchangeable with other segmentation networks, such as R2UNet[4],
Attention-UNet[2], and UNet[1].

##  3 Experimental Results

Dataset Preparation As liver segmentation is commonly available in CT domain,
we chose CT as our source domain (domain $\mathbb{A}$), and aims to obtain
CBCT/MRI/PET liver segmenters without their segmentation ground truth. For CT,
we collected 131 and 20 CT volumes with liver segmentation from LiTS [15] and
CHAOS [16], respectively. In the CBCT/MR domain, we collected 16 TACE patients
with both intraprocedural CBCT and pre-operative MRI for our segmentation
evaluations. In the PET domain, we collected 100 18F-FDG patients with
abdominal scan. All the CBCT were acquired using a Philips C-arm system with a
reconstructed image size of $384\times 384\times 297$ and voxel size of
$0.65\times 0.65\times 0.65mm^{3}$. The MRI and PET were acquired using
different scanners with different spatial resolutions. Thus, we re-sampled all
the CBCT, MR, PET and CT to an isotropic spatial resolution of $1\times
1\times 1mm^{3}$. As a result, we obtained $13,241$ 2D CT images with liver
segmentation, $3,792$ 2D CBCT images, $1,128$ 2D MR images, and $6,150$ 2D PET
images. All the 2D images were resized to $256\times 256$. With 16 CBCT and
MRI patients in our dataset, we performed four-fold cross-validation with 12
patients used as training and 4 patients used as testing in each validation.
For PET, we used 60 patients as training and 40 patients as testing, and
qualitatively evaluated the results. Implementation details are summarized in
our supplemental materials.

Figure 3: Comparison of CBCT and MRI segmentation using different methods. Red mask: liver segmentation prediction. Green contour: liver segmentation ground truth. The coronal and saggital view are shown on left and bottom, respectively.  Table 1: Quantitative comparison of CBCT and MRI segmentation results using DSC and ASD(mm). * means supervised training with ground truth segmentation on the target domain (CBCT/MRI domain). -PCT means without patch contrastive learning. Best results are marked in blue, and $\dagger$ means the difference between ours and SynSeg-Net are significant at $p<0.05$. CBCT | TD-GAN | SynSeg-Net |  SegCBCT* | Ours-PCT | Ours
---|---|---|---|---|---
Median DSC | $0.685$ | $0.870$ | $0.882$ | $0.827$ | $0.885$
Mean(Std) DSC | $0.695(0.092)$ | $0.862(0.051)$ | $0.874(0.035)$ | $0.831(0.079)$ | $0.893(0.029)^{\dagger}$
Median ASD | $10.144$ | $7.289$ | $9.190$ | $9.824$ | $5.539$
Mean(Std) ASD | $10.697(2.079)$ | $7.459(2.769)$ | $10.742(4.998)$ | $10.381(3.739)$ | $5.620(1.352)^{\dagger}$
MRI | TD-GAN | SynSeg-Net |  SegMRI* | Ours-PCT | Ours
Median DSC | $0.907$ | $0.915$ | $0.907$ | $0.893$ | $0.920$
Mean(Std) DSC | $0.900(0.044)$ | $0.912(0.029)$ | $0.859(0.102)$ | $0.898(0.037)$ | $0.921(0.018)$
Median ASD | $1.632$ | $1.681$ | $2.838$ | $1.743$ | $1.468$
Mean(Std) ASD | $2.328(2.070)$ | $1.916(1.142)$ | $3.660(2.522)$ | $2.478(2.014)$ | $1.769(1.002)$

Segmentation Results After AccSeg-Nets are trained, we can extract the
segmenters for liver segmentation on CBCT, MRI, and PET. We used Dice
Similarity Coefficient (DSC) and Average Symmetric Surface Distance (ASD) to
evaluate the quantitative segmentation performance. First, we compared our
segmentation performance with TD-GAN[7], SynSeg-Net[9], and segmenter directly
supervised trained on target domain images with limited liver annotations
(SegCBCT / SegMRI). For a fair comparison, we used DuSEUNet as segmentation
network in all the compared methods. The visual comparison is shown in Figure
3, and the quantitative comparison is summarized in Table 1. As we can
observe, the image quality of CBCT is degraded by metal artifacts and low CNR.
TD-GAN’s results are non-ideal as it requires adapting the input CBCT to CT
first, and the segmentation relies on the translated image quality. The
unpaired and unconstrained adaption from CBCT to CT is challenging as it
consists of metal artifact removal and liver boundary enhancement. The multi-
stage inference in TD-GAN thus prone to aggregate prediction errors into the
final segmentation. SynSeg-Net with single-stage segmentation help mitigate
the prediction error aggregation, but the segmentation is still non-ideal due
to lack of structure and content constraint in the unpaired adaptation. On the
other hand, our AccSeg-Net with patch contrastive learning and anatomy
constraint achieved the best segmentation results. Compared to the segmenters
trained on target domains using relatively limited annotation data (12
CBCT/MRI patients), our AccSeg-Net trained from large-scale conventional CT
data (151 patients) can also provide slightly better segmenters.

Figure 4: Comparison of CBCT and MRI segmentation using different anatomy-constraint settings. Red mask: liver segmentation prediction. Green contour: liver segmentation ground truth. Table 2: Quantitative comparison of CBCT and MRI segmentation results using different anatomy-constraint settings in AccSeg-Net. -MIND and -CC means without MIND loss and without CC loss, respectively. Best results are marked in blue.  CBCT | -MIND-CC | +MIND-CC | -MIND+CC | +MIND+CC
---|---|---|---|---
Mean(Std) DSC | $0.869(0.059)$ | $0.883(0.041)$ | $0.885(0.039)$ | $0.893(0.029)$
Mean(Std) ASD | $7.397(2.268)$ | $6.648(2.147)$ | $6.424(2.110)$ | $5.620(1.352)$
MRI | -MIND-CC | +MIND-CC | -MIND+CC | +MIND+CC
Mean(Std) DSC | $0.913(0.030)$ | $0.916(0.024)$ | $0.918(0.020)$ | $0.921(0.018)$
Mean(Std) ASD | $1.923(1.153)$ | $1.886(1.138)$ | $1.803(1.109)$ | $1.769(1.002)$

Then, we analyzed the effect of using different anatomy-constraint settings in
our AccSeg-Net. The results are visualized in Figure 4, and summarized in
Table 2. As we can see, adding either MIND loss or CC loss help improve our
segmentation performance, while combining both anatomy constraint losses
yields the best segmentation performance of our AccSeg-Net. We also performed
ablation studies on using different segmentation networks in our AccSeg-Net,
including R2UNet[4], Attention-UNet[2], and UNet[1]. The visualization and
quantitative evaluation are summaried in our supplemental materials. We
observed that AccSeg-Net can be adapted to different segmentation networks,
and yields reasonable segmentation results. Additional PET liver segmentation
results from our AccSeg-Net are shown in Figure 5 for visual evaluation. Our
AccSeg-Net can also provide reasonable segmentation on PET data without using
any ground-truth annotation in PET domain.

Figure 5: PET liver segmentation prediction on 5 different patients at three
different latitudes.

##  4 Discussion and Conclusion

In this work, we proposed a novel framework, called AccSeg-Net, for synthetic
segmentation without the target domain’s ground truth. Specifically, we
proposed to use anatomy-constraint and patch contrastive learning in our
AccSeg-Net to ensure the anatomy fidelity during the unsupervised adaptation,
such that the segmenter can be trained on the adapted image with correct
anatomical contents. We demonstrated successful applications on CBCT, MRI, and
PET imaging data, and showed superior segmentation performances as compared to
previous methods. The presented work also has potential limitations. First,
our segmentation performance is far from perfect, and extending our framework
to 3D and with enhanced segmentation loss may further improve the performance.
Moreover, our method is an open framework, and substituting the segmentation
network with a more advanced network may also help improve the performance. In
our future work, we will also explore diseases and multi-organ segmentation on
other imaging modalities, such as ultrasound and radiography.

## References

  * [1] Ronneberger, O., Fischer, P., Brox, T.:  U-net: Convolutional networks for biomedical image segmentation.  In: International Conference on Medical image computing and computer-assisted intervention, Springer (2015) 234–241
  * [2] Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., et al.:  Attention u-net: Learning where to look for the pancreas.  arXiv preprint arXiv:1804.03999 (2018)
  * [3] Roy, A.G., Navab, N., Wachinger, C.:  Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks.  In: International conference on medical image computing and computer-assisted intervention, Springer (2018) 421–429
  * [4] Alom, M.Z., Yakopcic, C., Hasan, M., Taha, T.M., Asari, V.K.:  Recurrent residual u-net for medical image segmentation.  Journal of Medical Imaging 6(1) (2019) 014006
  * [5] Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.:  nnu-net: a self-configuring method for deep learning-based biomedical image segmentation.  Nature Methods (2020) 1–9
  * [6] Yu, Q., Yang, D., Roth, H., Bai, Y., Zhang, Y., Yuille, A.L., Xu, D.:  C2fnas: Coarse-to-fine neural architecture search for 3d medical image segmentation.  In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. (2020) 4126–4135
  * [7] Zhang, Y., Miao, S., Mansi, T., Liao, R.:  Task driven generative modeling for unsupervised domain adaptation: Application to x-ray image segmentation.  In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer (2018) 599–607
  * [8] Kamnitsas, K., Baumgartner, C., Ledig, C., Newcombe, V., Simpson, J., Kane, A., Menon, D., Nori, A., Criminisi, A., Rueckert, D., et al.:  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks.  In: International conference on information processing in medical imaging, Springer (2017) 597–609
  * [9] Huo, Y., Xu, Z., Moon, H., Bao, S., Assad, A., Moyo, T.K., Savona, M.R., Abramson, R.G., Landman, B.A.:  Synseg-net: Synthetic segmentation without target modality ground truth.  IEEE transactions on medical imaging 38(4) (2018) 1016–1025
  * [10] Zhu, J.Y., Park, T., Isola, P., Efros, A.A.:  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In: Proceedings of the IEEE international conference on computer vision. (2017) 2223–2232
  * [11] Chen, T., Kornblith, S., Norouzi, M., Hinton, G.:  A simple framework for contrastive learning of visual representations.  In: International conference on machine learning, PMLR (2020) 1597–1607
  * [12] Park, T., Efros, A.A., Zhang, R., Zhu, J.Y.:  Contrastive learning for unpaired image-to-image translation.  In: European Conference on Computer Vision, Springer (2020) 319–345
  * [13] Wu, Z., Xiong, Y., Yu, S.X., Lin, D.:  Unsupervised feature learning via non-parametric instance discrimination.  In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (2018) 3733–3742
  * [14] Yang, H., Sun, J., Carass, A., Zhao, C., Lee, J., Prince, J.L., Xu, Z.:  Unsupervised mr-to-ct synthesis using structure-constrained cyclegan.  IEEE Transactions on Medical Imaging (2020)
  * [15] Bilic, P., Christ, P.F., Vorontsov, E., Chlebus, G., Chen, H., Dou, Q., Fu, C.W., Han, X., Heng, P.A., Hesser, J., et al.:  The liver tumor segmentation benchmark (lits).  arXiv preprint arXiv:1901.04056 (2019)
  * [16] Kavur, A.E., Gezer, N.S., Barış, M., Conze, P.H., Groza, V., Pham, D.D., Chatterjee, S., Ernst, P., Özkan, S., Baydar, B., et al.:  Chaos challenge–combined (ct-mr) healthy abdominal organ segmentation.  arXiv preprint arXiv:2001.06535 (2020)


# How A/B testing changes the dynamics of information spreading on a social
network

Matteo Ottaviani111Corresponding author, email<EMAIL_ADDRESS>Scuola
Normale Superiore, Piazza dei Cavalieri 7, 56126 Pisa - Italy Max Planck
Institute for Human Development, Lentzeallee 94, 14195 Berlin - Germany
Pietro Leonardo Nickl  Max Planck Institute for Human Development, Lentzeallee
94, 14195 Berlin - Germany  Stefan M. Herzog  Max Planck Institute for Human
Development, Lentzeallee 94, 14195 Berlin - Germany  Philipp Lorenz-Spreen
Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin -
Germany

(Mar 12, 2021 )

###### Abstract

A/B testing methodology is generally performed by private companies to
increase user engagement and satisfaction about online features. Their usage
is far from being transparent and may undermine user autonomy (e.g. polarizing
individual opinions, mis- and dis- information spreading). For our analysis we
leverage a crucial case study dataset (i.e. Upworthy) where news headlines
were allocated to users and reshuffled for optimizing clicks. Our center of
focus is to determine how and under which conditions A/B testing affects the
distribution of content on the collective level, specifically on different
social network structures. In order to achieve that, we set up an agent-based
model reproducing social interaction and an individual decision-making model.
Our preliminary results indicate that A/B testing has a substantial influence
on the qualitative dynamics of information dissemination on a social network.
Moreover, our modeling framework promisingly embeds conjecturing policy (e.g.
nudging, boosting) interventions.

##  1 Introduction

The online ecosystem has been an exceptional ground for private companies to
build choice environments that draw people’s attention in order to make and
increase profits. Although the their goal may vary from influencing political
opinions to selling a pair of shoes, the methods which are involved often
share the same algorithmic principles. One example in that respect is A/B
testing, a method that optimizes pieces of the online environment to some goal
function via a randomized experiment with two variants, the currently used
version A (control) and its modification in some respect B (treatment). A/B
tests are generally performed by private companies to increase user engagement
and satisfaction about online features. Large platforms like Facebook,
Instagram, Google, Groupon, LinkedIn, Microsoft, Netflix, Yahoo and Amazon use
A/B testing to make user experiences more profitable and as a way to
streamline interface of their services (Kohavi et al., 2014; Xu et al., 2015).
Surfing the web, users undergo A/B tests without awareness; for example
different users may see different versions of the same web page at the same
moment. Thereby, the A/B testing machinery is far from being transparent;
private companies don’t share their data, their methods, their direct purposes
and, particularly, the different versions they are testing.

We may discern two different levels of impact: the way A/B testing influence
individual decision making and the way how it scales up to collective
behavior. The first research question that we will address here aims to find
out which are the most relevant features that A/B testing is typically
amplifying. In order to do that, we will leverage real world data from a
particular field of application of A/B testing. In the second research
question we will ask how and under which conditions A/B testing affects the
distribution of content on the collective level, specifically on different
social network structures. Understanding the impact A/B testing has on the
online ecosystem and the mechanisms at work may help to develop intervention
and tools that dampen potentially unwanted outcomes of A/B testing and help to
promote “user autonomy” (Lorenz-Spreen et al., 2020; Kozyreva et al., 2020).

We will leverage a crucial case study regarding the effectiveness of A/B
testing: the Upworthy archive Matias and Munger (2019). In November 2019 Good
& Upworthy and a team of researchers announced a dataset of 32,487 A/B tests
carried out by Upworthy from January 2013 to April 2015 (Matias and Munger,
2019). The company conducted its A/B tests on its own website, randomly
varying different presentations for a single story. Presentations were
_packages_ of news/story headlines that were randomly assigned to people on
the website as part of a test. A/B test machinery compared package “success”
to pick winners employing the click-through rate (CTR; i.e. the ratio of
clicks to impressions). The present data set is by definition a clickbait
dataset: The different headlines where designed and tested in an effort to
maximise clicks (Blom and Hansen, 2015; Chen et al., 2015). This data is
particularly suitable for systematically analyzing which linguistic features
successfully attract clicks and are then amplified through A/B testing.

Linguistic cues occur on different levels. Particular attention in each
package should be paid on the lexical or semantic level, the structural or
grammatical one and the formal or formatting one. For example, different words
may recall different emotional reactions, some topics could have been more
successful than others, a specific text and word lengths may impact in a
certain way.

We will combine rule-based and machine learning methods to extract linguistic
features. For the semantic analysis we will use a topic model and a text
analysis tool that incorporates a number of freely available sentiment
dictionaries. We will use topic modeling to detect the presence of semantic
topics (e.g. feminism, racial equality, LGBT+ issues, etc). For the topic
modelling we will use Latent Dirichlet Allocation (LDA). LDA is an
unsupervised learning algorithm that detects the co-occurence of words across
documents (e.g. a headline) in a corpus (the collection of all headlines).
Words that cluster together form a “topic”, which may or may not be an
interpretable semantic topic. In order to detect the presence of emotion,
arousal etc. we will use Sentiment Analysis and Cognition Engine (SEANCE;
Crossley et al., 2017), a freely available text analysis tool that embeds a
number of sentiment dictionaries. Mainly we are using SEANCE to analyse
headlines in terms of valence, arousal and specific emotions. Once we obtained
linguistic features characterizing packages, we associate to each package an
input vector made of its extracted features. The decision-making model $DM$ we
assume is a simple linear relationship that binds the linguistic features
present in a generic package with its probability to be clicked, i.e., the
click-through rate of that package. We assume that the decision-making model
holds globally for each package of the Upworthy data set. The continuous
outcome for each package (i.e. the click-through rate), together with the
latter assumption, allows for a linear regression method. Therefore, we
perform a LASSO regression (Least Absolute Shrinkage and Selection Operator).
The regression coefficients coming from its training quantify the overall
impact of each linguistic feature on click-through rate; they serve the
decision-making function for simulating an individual interacting in front of
an hypothetical (e.g. clickbait) headline. Therefore, we put forward a data-
driven human decision-making model for clicking headlines online, based on
linguistic/semantic features as cues.

To asses the global impact of A/B testing on the distribution of content, we
compare two situations: (1) clicks on headline refer to a social sharing
process, thereby no global actor has an agency and just the attractiveness of
linguistic features plays a role in the distribution and (2) a global A/B
tester is introduced who is aiming to additionally maximize the clicks by
systematically varying the headlines following an A/B testing scheme. The two
main assumptions of our agent-based model lay on:

  * •

all agents are equal: each of them reacts to a given package following the
same decision-making model (mentioned above);

  * •

the click for an agent means ‘sharing’ a package (partially of entirely) to
the nearest neighbors (with a friction parameter: the infection rate).

In order to mimic an online discourse we couple agents via a stylized social
network structure, along which they can share information (headlines in this
case). We employ several network topologies (e.g. Albert-Barabasi, Erdős-
Rényi, Stochastic Block Model) and tune their densities ( i.e. sparsity of the
network).

We consider synthetic randomly generated headline, which consists of a random
combination of a fixed number of linguistic features.

In a pilot studies we have tested the procedure mentioned above. It has been
achievable thanks to the availability of an exploratory data set from the
Upworthy Research Archive Matias and Munger (2019). The preliminary results
coming up from the exploration of the exploratory data set allowed us to
generate research questions that we wish to address with the full data. In
particular, the final distributions of feature distribution from our agent-
based model, either in a pure social spreading scenario or in the A/B testing
condition. We could observe that the A/B testing mechanism increases the
homogeneity of information that is spread. In other words, we observe that A/B
testing performed on a social network structure reduces the exploration and
amplifies exploitation of successful features of early pieces of information,
ignoring others. With the full data from the Upworthy archive, we will be able
to answer what types of information this typically is and is not.

##  2 A/B testing: a brief overview

Private companies which sell their products online, including Amazon, eBay,
Etsy, Facebook, Google, Groupon, LinkedIn, Microsoft, Netflix or Yahoo (Kohavi
et al., 2014; Xu et al., 2015), have been using online controlled experiments
(i.e. A/B testing) for at least two decades, in order to shape their
platforms. However, in their words, the aim that drives them is the continuous
improvement of users’ online experiences. Companies run and analyse thousands
concurrent experiments per day in order to validate new ideas, which
materially translate in changes in the online environment shown to customers.
The latter range from entire redesigns and infrastructure changes (e.g. a post
on a social network, mobile app interface, etc) to bug fixes. Finally, the
whole collection of successes and failures is then employed in learnings on
customer behaviour (Kaufman et al., 2017; Xu et al., 2015). Private companies
are used to build in-house experiment infrastructures to maximize their
products’ success, strongly relying on A/B testing (Kaufman et al., 2017; Xu
et al., 2015).

As briefly mentioned before, A/B testing is a randomised controlled trial for
assessing the causal effect of introducing a new idea (i.e. treatment) on some
wished output. Traditionally, the effect of the treatment is estimated on the
whole intended population, since the outcome for a given individual cannot be
computed for both versions A and B. It is referred to as the Average Treatment
Effect (ATE; Rubin, 1974): it measures the difference in the randomized
treatment and control groups as following: $ATE=E[Y=1]-E[Y=0]$. The ATE is
then the difference in the expected values of the treatment and control
group’s output. For a given causal effect of interest, the ATE value would
suggest the effectiveness of some treatment applied. Statistical inference is
usually employed to determine whether an ATE estimate is statistically
consistent (either positively or negatively; Kaufman et al., 2017). However,
since ATE is a measure of the average effect, a positive or negative ATE does
not assess whether and how a particular individual would react to a given
treatment. In the last years, much more attention has been focused on the
latter requirement. In particular, the rising of huge data sets containing
personalized treatments, along with their individual profiles, has allowed to
explore how treatment effects vary across individuals. Accordingly, a new
measure has been put forward: Conditional Average Treatment Effect (CATE,
$\tau(x)=E[Y(1)-Y(0)|X=x]$) which is the treatment effect conditional on
observed covariates. Meta-algorithms are employed in order to compute CATE.
The most common of them takes two steps: it employs the so called “base
learners” to measure the conditional expectations of the outcomes both for
control and treatment groups, separately; afterwards, it takes the difference
between these estimates. To date, most used meta-algorithms include:T-learner,
S-learner and X-learner (Künzel et al., 2019).

##  3 A case of study: Upworthy.com

We took into account a crucial case study regarding the effectiveness of A/B
testing: Upworthy Archive. Upworthy.com is a company founded in 2012 by the
author of The Filter Bubble (Pariser, 2011) and cofounder Peter Koechley;
their aim was to reach large audiences with pieces of news (in their words
“stuff that matters”) which would have drawn most of people’s attention on the
web. A key component of their strategy was to setting up attractive
“packages,” each of them consisting of a headline, a subheading and an image,
similar to the one individuals were shown when an article was posted on a
social media. In order to establish which package would influence the most,
Upworthy adopted A/B testing practices. They were already popular among
technology industry and political campaigns. The company optimized content to
deliver packages, measure responses, and compute probability of a viewer
clicking on different versions of the same story. In the history of American
media, Upworthy was a leading actor from 2013–2015; people associated its
success to the idea of “clickbait.” Editors and website engineers reported
that they set only one experiment per page in order to reduce correlations
and/or dependencies among experiments. The system recorded the number of
individuals that were shown a given package (i.e. impressions) and the number
that clicked on the package (i.e. clicks). After a while, editors decided
either to choose the best performing package for finalizing or to keep
modifying packages for increasing performance (Matias and Munger, 2019). In
November 2019 Good & Upworthy (Upworthy merged with Good Worldwide in 2017)
and a team of researchers composed by J. Nathan Matias (Cornell University),
Kevin Munger (Penn State University), and Marianne Aubin Le Quere (Cornell
University) announced The Upworthy Research Archive, a dataset of 32,487 A/B
tests carried out by Upworthy from January 2013 to April 2015. In their words,
the release of the dataset aimed to enhance knowledge in many fields,
including: communication, political science, psychology, statistics, and
computer science (Matias and Munger, 2019).

###  3.1 The Upworthy Research Archive

The Upworthy Research Archive dataset consists of 32,488 tests carried out
from January 24, 2013 through April 14, 2015, just after an editorial shifts
was announced by the company. For each test, the dataset includes viewer
reactions to each package in an experiment; there is a median of 4 packages
per test. The dataset contains over 150 thousand packages. These packages
altogether received over 538 million impressions and over 8 million clicks.
Each test includes a median of 14,342 impressions and a median of 201 clicks
per test. Each package consists of: the experiment ID; the headline; the
subhead, the social media summary (where Upworthy used them); the preview
image; the number of impressions and clicks received by the package during the
test (see Fig. 1; Matias and Munger, 2019).

Figure 1: An example of package features from the Upworthy Research Archive,
as described in Matias (2020).

###  3.2 Clickbait: definition and characteristics, linguistic features

Potthast et al. (2018) distinguish between four definitions of clickbait by:
emergence, intention, effect and perception. We may define clickbait by its
effect (as whatever attracts the most clicks) or as a certain style (e.g.
anything being perceived as sensationalist). A New York Times headline might
be very effective without employing clickbait style. It is worth
distinguishing between clickbait in the effective definition and “clickbait
style.” It could be that clickbait style actually is not the most effective
way of grabbing a reader’s attention, or that it even annoys some people—for
example the makers and users of downworthy (ZOT), a browser extension that
changes clickbait style elements, replacing words like “Awesome” and “WHOA.”
While we expect our present dataset to fulfil all four definitions of
clickbait, it is important to keep in mind the distinction between clickbait
as a certain style and clickbait as a click-optimised piece of text.

###  3.3 Clickbait style

The present data set is by definition a clickbait dataset: It is the result of
an effort to optimize for number of clicks. Though the effectiveness of any
headline may not be due to clickbait style, it is worth examining this style
in the context of the Upworthy dataset, as it actually coined much of what we
think of as “clickbait style.” In fact, studies on clickbait actually take
material from upworthy (Blom and Hansen, 2015; Chen et al., 2015). Clickbait
style includes formal features (the two-sentence headline, the listicle,
unusual punctuation) as well as semantic features (colloquial, overly
emotional language). We cast a wide net: We try to include as many clickbait
characteristics from the literature as possible.

##  4 Methods

In Fig. 2 we show a broad outline of the main steps of the procedure we
follow. After employing linguistic methods on the data set in order to infer
linguistic features, we convert our data set in input-output relationships for
each package, where the features presented in each bundle are linked to its
click-though rate. By performing LASSO regression over the latter, we infer
feature coefficients which serve us their overall impacts (negatively or
positively) on the click-though rate. Our first research question about what
are the linguistic features most mattering in an A/B testing can be addressed
with this procedure. For the second research question we will assume a simple
linear model for decision making that binds the linguistic features present in
a generic package with its probability to be clicked, directly linking to the
click-through rate of that package. LASSO regression coefficients are part of
this individual decision-making model to quantify the feature impacts. In
order to pursue our second research question, i.e. under which conditions a
message enhanced by AB testing spread in a social network structure, we need
to set up a general framework in which we can ideally ’switch on’ the A/B
testing machinery and make a comparison afterwards. Therefore, we design a
simplistic agent-based model, where we can distinguish between two scenarios.
In the first one, messages shared by individuals undergo a pure social
spreading; in the second one, instead, A/B tests influences the message
selection dynamics. In the following we go through a full description of all
methods that we will employ for achieving this.

Figure 2: Broad round-up of the temporal order of our procedure.

###  4.1 Linguistic features extraction

We combine rule-based and machine learning methods to extract formal and
semantic linguistic features. Most linguistic features are extracted by the
application of an explicit rule, e.g. the presence of a certain character or
word. For the topic modelling, we employ a data-driven approach, by capturing
the underlying semantic structure of the corpus. All code is implemented in
python. For preprocessing we employ a separate pipelines based on the python
libraries nltk, and spaCy separately, as they come with different functions
for tokenization, lemmatization and different word lists e.g. for stop word
recognition. The output of the feature extraction process is a pandas
dataframe. It is a headline-feature matrix with the headlines as rows and the
extracted linguistic features, as well as the click-rate as columns. We will
use this matrix both to describe the upworthy corpus linguistically as well as
to see how features are associated with the click rate.

####  4.1.1 Formal Features

Clickbait style is characterised by formal features, such as a preference for
fully formed sentences as titles. This entails overall more words, a higher
stop-word to content-word ratio and longer syntactic dependencies in
comparison to the succinct headlines of established newspapers, while average
word length might be low. Such features are easy to operationalise. For our
feature extraction, we take inspiration from Kuiken et al. (2017) and Zheng et
al. (2017) who list these and more features (see table 1 for their respective
lists). As formal features Safran (2013) lists five headlines types: normal,
Question (headline forms a question), How to (headline starts with “how to”),
Number (headline introduces a listicle), and reader-addressing (contains a
form of “you”). We would like to include the feature of forward referencing
(Blom and Hansen, 2015), another characteristic of clickbait style: In
headlines like “She Did Not Expect THIS,” or “What Happened Next Will Blow
Your Mind.” Here, one has to click the article to resolve what these headlines
are referring to. We are still working on an adequate operationalisation of
this feature, perhaps simplifying it to the presence of a demonstrative and/or
pronoun (like this, she).

Zheng et al. (2017) |  Kuiken et al. (2017)
---|---
Presence/number of exclamation mark |  Number of characters
Presence/number of question mark |  Number of words
Presence of pronoun |  Average word length
Presence of interrogative |  Number of sentences
Number of words |  Number of sentimental words
Number of dots |  Readability score
Stop words ratio |  Containing question
tf-idf weight of words |  Containing quote
tf-idf weight of bigrams |  Containing signal words
|  Containing pronouns
|  Containing number
|  First word type
Table 1: Clickbait feature lists to compute a clickbait score.

####  4.1.2 Semantic features

##### Sentimental Analysis

Emotional valence (positive/negative) and arousal (strong/weak) are important
factors for reading behaviour (Chen et al., 2015). Clickbait is further
associated with overly positive sentiment. For sentiment analysis we will use
the Sentiment Analysis and Cognition Engine (SEANCE, Crossley et al. 2017).
SEANCE is a large knowledge base including many word lists for nuanced
sentiment analysis (251 core indeces). Since SEANCE includes word lists that
are interpretable as a topic (e.g. “politics, economics, and religion,”
“Dominance, respect, money, and power,” “social relations,” “Arts and
Academics”), it may also serve as a knowledge base to detect these topics.

##### Topic model

We will employ topic modeling to detect the presence of semantic topics (e.g.
feminism, racial equality, LGBT+ issues). For the topic modelling we will use
Latent Dirichlet Allocation (LDA). LDA is an unsupervised learning algorithm
that detects the co-occurence of words across documents (e.g. a headline) in a
corpus (the collection of all headlines). Words that cluster together form a
“topic,” which may or may not be an interpretable semantic topic. For the LDA,
we will define a document as each unique story plus all its different
headlines, i.e. we combine the unique “lede” with all its unique headlines.
This yields larger, more semantically coherent documents, which give LDA a
better chance of picking up semantically related words. This means that we
will perform LDA on the story-level, not the headline level. Since we aim to
detect semantic topics, preprocessing for the LDA will include lowercasing,
lemmatization, stopword removal, punctuation removal. While we may lose
potentially informative formal features by removing formal features as
“noise,” we will amplify the semantic signal. We will retrieve information
about stopwords, uppercasing and punctuation in other feature extraction
steps.

###  4.2 Assessing linguistic features importance in the data set

Through the linguistic methods mentioned above, we will obtain in total $M$
linguistic features extracted along with their relative weights. In other
words, for each package $k$ and for each feature $i$ included in it, a
relative weight $F_{i}^{k}$ is measured. Each package $k$ identifies an online
environment and it can be formally described as follows:
$\\{F_{1}^{k},...,F_{M}^{k}\\}$, where $F_{i}^{k}$ is the weight assigned by
linguistic methods to the linguistic feature $i$ for the package $k$;
$F_{i}^{k}=0$ if the linguistic feature $i$ was not found in the package $k$.

####  4.2.1 Individual decision-making model

Our main purpose is to build up a decision-making model able to surrogate the
interaction of an individual with a news headline presented online. The latter
may be thought of consisting of a set of linguistic features:
$OE_{k}=\\{j,n,...,s\\}$, where $j,n,...,s$ are the indices belonging to the
linguistic features present in the headline $k$; $k=1,...,K$, $K$ is the total
number of packages available. Let us call $Y_{k}$ the click-through rate value
for package $k$.

The decision-making model $DM$ we assume is a simple linear relationship that
binds the linguistic features present in a generic package $k$ with the click-
through rate of that package. Formally:

$DM_{k}=\sum_{i\in{OE_{k}}}w_{i}=Y_{k}\hskip 28.45274pt\propto p_{k}$ (1)

where $w_{i}$ are the coefficients quantifying the overall impact of feature
$i$ and $p_{k}\in[0,1]$.

In order to assess features importance $w_{i}$’s, let us assume that our
decision-making model (eq.1) holds for each package of the Upworthy data set,
that is a simple linear relationship which binds the linguistic features
present in the package with its click-through rate. Since we are provided of a
continuous outcome for each package (i.e. the click-through rate), together
with the latter assumption, then a linear regression method fits these
requirements.

####  4.2.2 LASSO

In order to measure parameters for all features, we will perform a Lasso
regression (Least Absolute Shrinkage and Selection Operator). It is a
modification of linear regression. In Lasso, the loss function is modified to
minimize the complexity of the model by limiting the sum of the absolute
values of the model coefficients (also called the $l1-$norm).

Formally, the estimates of LASSO coefficient $(w_{1},...,w_{M})$ are the
quantities that minimize:

$\sum_{k=1}^{K}Y_{k}-\left(w_{0}+\sum_{i=1}^{M}w_{i}F_{i}^{k}\right)^{2}+\lambda\sum_{i=1}^{M}|w_{i}|$
(2)

where $w_{0}$ is the regression intercept and $\lambda$ is a tuning parameter
which to be determined apart. The LASSO regression searches for the best
coefficient estimates that fit the data set.

LASSO differs form other regulation methods in this context (as Ridge, OLS,
etc) thanks to its second term, which is called “shrinkage penalty;” it has
the effect of shrinking the coefficient estimates towards zero. While several
regression methods tend to generate a model involving all features, the LASSO
will not only shrink parameter estimates but it will also push some of them to
zero. We could say that LASSO performs variable selection, in a certain sense
(Georges and Pereira, 2019).

####  4.2.3 LASSO tuning parameter selection

LASSO implementation needs a way to selecting the tuning parameter $\lambda$.
Cross-validation (CV) provides a simple method to achieve this task and it is
consistent with the goal of minimizing overfitting. In particular, we will
employ $k$-fold CV that consists in randomly splitting the set of observed
data into $k$ folds of equal size. The first fold works as a validation set,
and the LASSO regression is performed on the remaining $k-1$ folds. The fold
kept out (i.e. the validation set) is used to compute the mean squared error,
$MSE_{1}$. By iterating this procedure $k$ times with a different fold of data
set used as the validation set, the process generates $k$ estimates of the
test error, $MSE_{1},MSE_{2},...,MSE_{k}$. The average of the latter yields
the k-fold CV error:

$CV_{k}=\frac{1}{k}\sum_{j=1}^{k}MSE_{j}\,\,\text{.}$ (3)

The choices of $\lambda$ and $k$ are affected by a bias-variance trade-off. It
has been shown empirically that $k=5$ and $k=10$ values produce test error
rate estimates with relatively small bias and variance and it is then common
practise to compute $k$-fold CV employing them (Georges and Pereira, 2019). In
light of this, we will perform LASSO with respect to a grid of $\lambda$
tuning parameters, we will compute the cross-validation error for each of
them, and then we will select the one for which the k-fold CV error is the
lowest value.

Once the tuning parameter is chosen, we will finally obtain the best set of
LASSO regression coefficients; they represent the overall impact of features
on the click-through rate.

###  4.3 Agent based model of messages social spreading

Once obtained from LASSO performance the impact of each feature on the click-
though rate, the decision-making function shown in Eq. 1 is fully working for
simulating an individual interaction in front of an hypothetical (e.g.
clickbait) online environment. Let us figure individuals in a social network
which are given of different stimuli (in the spirit of Upworthy); their social
interactions may promote sharing of popular messages (i.e. linguistic
features). A synthetic reproduction of that dynamics may be thought as
following. In our simulation frameworks, $N$ agents are given. Our first
assumption lies on the fact that they are all equal: they behave through the
same decision-making function (Eq. 1) in front of a package, provided of the
same feature coefficients. The agents are linked one another with respect of a
network structure which reproduce social interconnections among them. In order
to mimic a social network structure, the best modeling tool which fits our
requirements is borrowed from graph theory; we are endowed with a stylized
complex network whose nodes are agents presented in the environment and whose
edges are the possible interactions among different individuals. We employ
several network topologies (e.g. Albert-Barabasi, Erdős-Rényi, Stochastic
Block Model) and tune their densities (i.e. sparsity of the network). Let us
consider synthetic packages generated by randomly picking $n$ linguistic
features. When an agent is given of a package, its individual decision-making
generates an output with respect to the features presented in the package.
This response is then converted in a probability to click and a random
threshold determines if the agent clicks or not on the content. Once an agent
clicks, it shares the successful package to its directly connected nearest
neighbors, according to an infection rate $\eta in(0,1]$. The latter is our
second main assumption: the click for an agent means “sharing” a package to
the nearest neighbors (with a friction parameter $\eta$). In order to pursue
our second research question, i.e. under which conditions a piece of news
enhanced by A/B testing spread in a social network structure, we need to set
up a general framework in which we can ideally ’switch on’ the A/B testing
machinery and make a comparison afterwards. Therefore, we design the agent-
based model and we can distinguish between two scenarios. In the first
scenario, messages shared by individuals undergo a pure social spreading; in
the second one, instead, A/B tests lead the message selection dynamics.

####  4.3.1 Pure social spreading scenario

The case of a pure social spreading is meant to be our benchmark scenario. The
simulation starts by drawing two randomly generated packages (i.e. packages
are done of $n$ linguistic features randomly picked) and are randomly
allocated to agents all over the network, package $A$ to half population and
package $B$ to the other half. Agents click or not according to our decision-
making model and the ones who clicked may spread (according to the infection
rate $\eta$) to those nearest neighbors of theirs who were shown the other
package. This spreading dynamics stops in a few steps, once the opportunities
of sharing a new content to neighbors saturate (i.e. each agent may at most
see both packages and share them around if it clicked). At this point, a new
two packages drawing takes place: one between the old $A$ and $B$ is kept and
the other one is partially reassembled according to a mutation rate $\mu$;
i.e. $\mu$ is the percentage of package features which is randomly varied.
Another test administration round starts again by randomly splitting the
population in two and randomly assigning the two packages to agents and so on.

####  4.3.2 A/B testing led social spreading scenario

The case of social spreading in which A/B testing is performed by a third
party (e.g. a private company) is a slight but pretty crucial modification of
the benchmark framework mentioned above. The A/B testing machinery takes place
from the second round package selection on. By comparing click results between
$A$ and $B$ of the previous round, it chooses for the new test administration
the $B$ package if its performance overcomes A click-though rate and builds a
new package by varying $B$ of a mutation rate. In the opposite case, $A$ is
kept for the new round and a new package $B$ is built by varying $A$ package.
Exploring A/B testing literature, several statistical methods have been
employed over time for assessing if $B$ click-through rate greater than $A$
one is statistically consistent in an A/B test. Nevertheless, the main
difference is between frequentist and Bayesian approaches.

##### A/B testing methodologies

In general, one performing an A/B test is basically collecting click-through
rates from the control $A$ and the variation (B) and then uses a statistical
method to determine which of the two performed better. In a frequentist world,
one would use p-values and make a choice between the null hypothesis (i.e.
there is no relevant difference between $A$ and $B$ , then we keep $A$) and
the alternative hypothesis that the variation $B$ is better than the control
$A$. As soon as the p-value reaches statistical significance or large amount
of data, the experiment is considered done. The frequentist approach requires
a wider difference in performances in order to prefer $B$.

On the contrary, following the Bayesian methodology, the click-though rate of
each variant is modeled as a random variable with a probability distribution.
Even in case the improvement of a new variant is small, Bayesian statistics is
more willing to choose the variation $B$ against the control $A$. On one hand,
this means that this methodology results more prone to false positive rate; on
the other hand, it lets either to control the magnitude of wrong decisions or
to perform a quick series of experiments so that one may accumulate small
gains from incremental improvements once making a lot of variations is
effortless and free of charge (Frasco, 2018).

Since our aim is to simulate how a private company would perform A/B testing,
our choice of which methodology to reproduce solely depends on which choice
they would make. They actually depend on their purposes and their contexts.
For example, one may prefer the latter in situations in which the treatment
requires relevant expenses (e.g. engineering maintenance, disruption to the
user experience, the costs of implementing it, etc) which would only be offset
by large benefits (i.e. wide difference between $A$ and $B$ click-through
rates). On the contrary, for situations similar to ours a Bayesian approach is
rather more preferred. Each small improvement is very welcome since no
additional expenses one incurs in choosing the variation $B$ against $A$, even
if it is a false positive. It doesn’t worth putting effort against the false
positive rate; it is much more relevant administrate more tests (i.e. explore
wider all the possible combinations) and quickly.

####  4.3.3 Bayesian A/B testing

The Bayesian methodology for performing A/B testing lies in a nicety of the
Bayes theorem in combination with the nature of processes underlying A/B
tests. It is usually very hard or impossible in most of cases to obtain closed
form solutions to the Bayes theorem; for this reason approximation methods
(e.g. Markov Chain Monte Carlo) have been developed. An A/B test can be
described by Bernoulli trial since it is a random experiment with only two
possible outcomes: “click” and “not to click.” The Beta distribution is the
conjugate prior of such process; that means the posterior function of the
Bayes Theorem lies in the same family of the prior one and then a final
function can be built with an iterative process. Therefore, an exact solution
of the Bayes formula (i.e. closed-form solution) exists. As just mentioned, a
convenient prior distribution form modeling a binomial parameter $q$ is the
beta distribution. Starting from a flat, uninformative prior, defined by
$Beta(1,1)$, the distribution of $p$ after $C$ clicks and $F$ failures (i.e.
impressions minus clicks) is given by $Beta(C+1,F+1)$. $C+1$ and $F+1$ are the
two parameters of the beta distribution for the belief. Therefore, in our case
we have two beta functions and accordingly their two Bayesian beliefs; they
are one for the experimental branch $A$ and the other one for $B$:$p_{A}\sim
Beta(C_{A}+1,F_{A}+1)$ and $p_{B}\sim Beta(C_{B}+1,F_{A}+1)$.

In line with Miller (2015) and without explicitly report the calculation, the
probability that $B$ will perform better than $A$ in the long run is given by:

$Pr(p_{B}>p_{A})=\sum_{i=0}^{C_{B}-1}\frac{Beta(C_{A}+i,F_{B}+F_{A})}{(F_{B}+i)Beta(1+i,F_{B})Beta(C_{A},F_{A})}\,\,\text{.}$
(4)

Let us define the uplift as how much $B$ variation increases the click-through
rate with respect to the control $A$:

$uplift=\frac{c_{B}-c_{A}}{c_{A}}\,\,\text{,}$ (5)

where $c_{A}$ and $c_{B}$ are click-through rates. Let us suppose the latter
is positive. In order to assess how trustful is the result, Eq. 4 estimates
the probability that $B$ option to perform better than $A$. Note that in the
frequentist approach there is no way to calculate this probability. In that
case, one computes a p-value and in case it falls under an arbitrary
threshold, one would state that “within a certain percentage of confidence
level (i.e. the $\alpha$ value, usually $95\%$), the null hypothesis can be
rejected.” This is a way different from the Bayesian case here where one would
state “this hypothesis is better then the other with a certain probability
percentage.”

Figure 3: Comparison of the fist-ranked linguistic feature dynamics in both
explored scenarios, a pure social spreading setting and an A/B testing led
one. The network structure employed in the simulation is the Albert-Barabasi
one.

##  5 Example of pilot data exploration

All the procedure explained in details above has been tested exploratory data
set from the Upworthy Research Archive (Matias and Munger, 2019) and we are
happy to share the code either to replicate results or to perform new
experiments. All the machinery described above may be considered pretty
general and, in principle, it could employ successfully other similar data
sets.

In the following we present some results obtained from our procedure’s
performance. In particular, we set up a social spreading framework according
to the settings previously described. We simulate $N=500$ agents allocated
according to the Albert-Barabasi network structure for total $T=500$ time
steps. Each $5$ time steps a new $A$-$B$ couple drawing takes place. We
perform this simulation either the pure social spreading framework or in the
A/B led one. The first A-B couple drawing is common to both of the settings:
$n_{F}$ features (in the shown example $n_{F}=7$) are randomly picked for
generating the control package and the variation $B$ is assembled varying the
control. From the second drawing on, the two setting differ in the A-B couples
way of selection. In the pure social spreading case, one of the two previous
packages is chosen for the new round (new control $A$) and challenged against
its varied version (new variation $B$). In the A/B led social spreading
setting, the Bayesian A/B testing methodology is employed on the gathered
click-through rates at the end of every round; in case the variation $B$
overperforms the control $A$ with a probability greater than a fixed threshold
(in the shown simulation the threshold is set at $95\%$), then the successful
$B$ becomes the new control $A$ for the next round and a new variation $B$ is
generated varying the control. Al the contrary, if successful conditions for
the A/B are not satisfied, the control $A$ remains the same and a new
variation is generated from it. In every case, generating the variation
depends on a mutation rate $\mu$(in the example shown, $\mu=3/7$). The
simulation of every scenario was performed for $R=100$ Monte Carlo replicas.
Every replica was built according to a unique set of random seeds for pseudo-
random generators; this guarantees either that replicas differ one another in
random drawings employed or simulation reproducibility.

A crucial measure has been assessing the dynamics of the most successful
linguistic features over time. We show a comparison of the two scenarios in
Fig.3. In a pure social spreading, the ’winning’ linguistic feature behavior
over time (averages over $100$ replicas) shows an evolution roughly fitted by
a third degree polynomial function. At the contrary, in an A/B testing led
scenario, the successful linguistic feature behavior follows a trend which can
be fitted by a linear function.

Figure 4: Comparison of the final ranking distributions of messages in both
explored scenarios, a pure social spreading setting and an A/B testing led
one. The network structure employed in the simulation is the Albert-Barabasi
one.

##  6 Conclusions

This preliminary results coming up from this exploration are indicating that
A/B testing has a substantial influence on the qualitative dynamics of
information dissemination on a social network. Through measuring homogeneity
and heterogeneity of final distribution of successful messages in a social
network framework, either in a pure social spreading or in a AB testing-leaded
one, we clearly observe (Fig.4 ) that the A/B testing-led framework changes
the dynamics of the pure social spreading qualitatively, by drawing crowd
attention on leading linguistic features. In other words, we observe that A/B
testing performed on synthetic social networks structure kills message
heterogeneity by promoting the most successful linguistic features identified
during the dynamics. In the subsequent data analysis, leveraging the full,
confirmatory upworthy data set, we will be able to quantify specifically which
of the linguistic features are the ones particularly favoured by A/B testing
strategies and allow inferences about its potential impact on the online
discourse. Due to the dramatic shifts of the public discourse to
algorithmically driven platforms, this assessment is of interest for the
future design of platforms and regulation that preserve an exchange of
arguments while content delivery stays relevant.

## References

  * Blom and Hansen (2015) Blom, J. N. and K. R. Hansen (2015).  Click bait: Forward-reference as lure in online news headlines.  Journal of Pragmatics 76, 87 – 100.
  * Chen et al. (2015) Chen, Y., N. J. Conroy, and V. L. Rubin (2015).  Misleading online content: Recognizing clickbait as ”false news”.  In Proceedings of the 2015 ACM on Workshop on Multimodal Deception Detection, WMDD ’15, New York, NY, USA, pp. 15–19. Association for Computing Machinery.
  * Crossley et al. (2017) Crossley, S. A., K. Kyle, and D. S. McNamara (2017).  Sentiment analysis and social cognition engine (seance): An automatic tool for sentiment, social cognition, and social-order analysis.  Behavior research methods 49(3), 803–821.
  * Frasco (2018) Frasco, M. (2018).  The power of bayesian a/b testing.
  * Georges and Pereira (2019) Georges, C. and J. Pereira (2019).  Market stability with machine learning agents.  Available at SSRN 3374666.
  * Kaufman et al. (2017) Kaufman, R. L., J. Pitchforth, and L. Vermeer (2017).  Democratizing online controlled experiments at booking. com.  arXiv preprint arXiv:1710.08217.
  * Kohavi et al. (2014) Kohavi, R., A. Deng, R. Longbotham, and Y. Xu (2014).  Seven rules of thumb for web site experimenters.  In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1857–1866.
  * Kozyreva et al. (2020) Kozyreva, A., S. Lewandowsky, and R. Hertwig (2020).  Citizens versus the internet: Confronting digital challenges with cognitive tools.  Psychological Science in the Public Interest 21(3), 103–156.
  * Kuiken et al. (2017) Kuiken, J., A. Schuth, M. Spitters, and M. Marx (2017).  Effective headlines of newspaper articles in a digital environment.  Digital Journalism 5(10), 1300–1314.
  * Künzel et al. (2019) Künzel, S. R., J. S. Sekhon, P. J. Bickel, and B. Yu (2019).  Metalearners for estimating heterogeneous treatment effects using machine learning.  Proceedings of the national academy of sciences 116(10), 4156–4165.
  * Lorenz-Spreen et al. (2020) Lorenz-Spreen, P., S. Lewandowsky, C. R. Sunstein, and R. Hertwig (2020).  How behavioural sciences can promote truth, autonomy and democratic discourse online.  Nature Human Behaviour, 1–8.
  * Matias and Munger (2019) Matias, J. N. and K. Munger (2019).  The upworthy research archive: A time series of 32,488 experiments in us advocacy.
  * Matias (2020) Matias, J.N. Aubin Le Quere, M. (2020).  Asking questions of the upworthy research archive.
  * Miller (2015) Miller, E. (2015).  Formulas for bayesian a/b testing.
  * Pariser (2011) Pariser, E. (2011).  The filter bubble: How the new personalized web is changing what we read and how we think.  Penguin.
  * Potthast et al. (2018) Potthast, M., T. Gollub, M. Hagen, and B. Stein (2018).  The clickbait challenge 2017: Towards a regression model for clickbait strength.  arXiv preprint arXiv:1812.10847.
  * Rubin (1974) Rubin, D. B. (1974).  Estimating causal effects of treatments in randomized and nonrandomized studies.  Journal of educational Psychology 66(5), 688.
  * Safran (2013) Safran, N. (2013).  Data insights into the headlines readers click.  Moz, July 13, 5.
  * Xu et al. (2015) Xu, Y., N. Chen, A. Fernandez, O. Sinno, and A. Bhasin (2015).  From infrastructure to culture: A/b testing challenges in large scale social networks.  In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2227–2236.
  * Zheng et al. (2017) Zheng, H.-T., X. Yao, Y. Jiang, S.-T. Xia, and X. Xiao (2017).  Boost clickbait detection based on user behavior analysis.  In L. Chen, C. S. Jensen, C. Shahabi, X. Yang, and X. Lian (Eds.), Web and Big Data, Cham, pp. 73–80. Springer International Publishing.