# GAMIVAL: Video Quality Prediction on Mobile Cloud Gaming Content

Yu-Chih Chen, Avinab Saha, Chase Davis, Bo Qiu, Xiaoming Wang, Rahul Gowda,
Ioannis Katsavounidis, and Alan C. Bovik The work of Alan C. Bovik was
supported by the National Science Foundation AI Institute for Foundations of
Machine Learning (IFML) under Grant 2019844. This work was supported by Meta
Platfroms, Inc.Yu-Chih Chen, Avinab Saha and Alan C. Bovik are with the
Laboratory for Image and Video Engineering (LIVE), Department of Electrical
and Computer Engineering, The University of Texas at Austin, Austin, TX, 78712
USA (email<EMAIL_ADDRESS><EMAIL_ADDRESS>bovik@ece.utexas.edu).Chase Davis, Bo Qiu, Xiaoming Wang, Rahul Gowda and
Ioannis Katsavounidis are with Meta Platforms Inc., One Hacker Way Menlo Park,
CA 94025 USA (email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>ikatsavounidis@fb.com).

###### Abstract

The mobile cloud gaming industry has been rapidly growing over the last
decade. When streaming gaming videos are transmitted to customers’ client
devices from cloud servers, algorithms that can monitor distorted video
quality without having any reference video available are desirable tools.
However, creating No-Reference Video Quality Assessment (NR VQA) models that
can accurately predict the quality of streaming gaming videos rendered by
computer graphics engines is a challenging problem, since gaming content
generally differs statistically from naturalistic videos, often lacks detail,
and contains many smooth regions. Until recently, the problem has been further
complicated by the lack of adequate subjective quality databases of mobile
gaming content. We have created a new gaming-specific NR VQA model called the
Gaming Video Quality Evaluator (GAMIVAL), which combines and leverages the
advantages of spatial and temporal gaming distorted scene statistics models, a
neural noise model, and deep semantic features. Using a support vector
regression (SVR) as a regressor, GAMIVAL achieves superior performance on the
new LIVE-Meta Mobile Cloud Gaming (LIVE-Meta MCG) video quality database.

###### Index Terms:

image/video quality assessment, mobile cloud gaming, natural scene statistics,
no-reference, perceptual quality, temporal statistics

©Copyright © 2023 IEEE. Personal use of this material is permitted. However,
permission to use this material for any other purposes must be obtained from
the IEEE by sending a request to<EMAIL_ADDRESS>DOI:
10.1109/LSP.2023.3255011

##  I Introduction

The development of broadband wireless Internet technology and mobile devices
has significantly boosted the popularity of mobile games. Cloud gaming
provides users a way to access many genres of games by remotely rendering
streaming games in the cloud as videos. Client devices, such as smartphones
and tablets capture users’ interactions, and transmit them to cloud servers.
This approach makes it possible to deliver high-computation video games to any
modern mobile device. However, along with the massive computations required to
render real-time 3D video games, other significant challenges arises, such as
avoiding response delays, and dealing with high bandwidth transmission while
ensuring users’ Quality of Experience (QoE). For example, “first person
shooter” games have very short delay tolerances (about 100ms [1]). If the
latency increases, players may leave a game because of worsened interactive
experiences. When there are large numbers of simultaneous users, bandwidth
requirements may explode. When transmitting over traffic-stressed networks,
unstabilities and errors can greatly degrade users’ QoE.

Being able to monitor distorted video quality on the client side, sending
feedback to cloud servers to adjust their encoding recipes when streaming
video can decrease computation and bandwidth requirements while enabling high
QoE at the users’ side. The basic tools are objective video quality assessment
(VQA) algorithms, of which several are currently utilized in streaming and
social video applications over very large scales, to produce perceptually
accurate predictions with low computational expense.

In the cloud gaming space, reference videos are not accessible on the client
side; hence NR VQA models provide reasonable solutions. Many general-purpose
NR VQA models have been designed to predict the perceived qualities of real-
world video types. Early models, operated by extracting features defined under
spatial natural video statistics models, include NIQE [2], BRISQUE [3],
HIGRADE [4], GM-LOG [5], and FRIQUEE [6], among others. Since analyzing
temporal distortions is essential to the prediction of dynamic video quality,
space-time VQA models have been devised, including V-BLIINDS [7], ChipQA [8,
9] and RAPIQUE [10], which computes spatial and temporal bandpass statistical
features, along with semantic features computed by a Convolutional Neural
Network (CNN). However, gaming videos rendered by computer graphics engines
differ statistically from naturalistic videos, which effects the relevance and
performances of existing NR VQA models. Indeed, existing methods struggle on
recent gaming video databases [11, 12, 13].

Figure 1: Schematic flow diagram of the GAMIVAL model. The top portion depicts
the spatial and temporal NSS feature computations. The lower portion shows the
CNN feature extraction process following NDNetGaming [14]. All of the features
are concatenated and utilized to train an SVR model.

In recent years, the success of CNNs on many image analysis problems [15, 16,
17], has motivated their application to NR VQA [18, 19]. CNN based VQA models
have been shown to perform well on existing UGC datasets, but training them
requires adequately large numbers of labeled image samples. Since the advent
of widespread cloud gaming technologies and live streaming gaming services
over the past decade, only six gaming VQA databases [20, 21, 13, 22, 23, 24]
have been proposed. However, unlike large-scale UGC VQA databases containing
thousands of videos and highly diverse content, existing gaming video
databases contain only a few labeled videos and fewer unique contents. Toward
addressing these data limitations, several previous NR gaming VQA models [21,
25, 26] have been trained and evaluated using labels generated by the full-
reference quality metric, Video Multi-Method Assessment Fusion (VMAF) [27],
which has been shown to deliver superior performance on gaming content
datasets [11]. Using VMAF to generate quality labels makes it possible to
train deep networks from scratch, although VMAF is not a perfect predictor of
perceptual quality. Another approach that is often effective is transfer
learning. The aim is to transfer knowledge learned from one or more source
tasks, even if the training and test datasets have somewhat different data
distributions [28], which can be rectified by fine-tuning on the target data.
For example, a CNN-based NR VQA model NDNetGaming [14] was first trained on
243,000 images with associated VMAF scores serving as proxy perceptual quality
labels, and then fine-tuned on a smaller dataset. This approach was able to
achieve high correlations against the subjective scores for gaming content
[20, 21].

Here we design an NR VQA model called GAMIVAL, which combines the merits of
conventional feature-based VQA algorithms with deep CNN-based VQA models. It
achieves superior performance with low computational complexity on LIVE-Meta
MCG dataset [24], as compared to state-of-the-art NR VQA algorithms.

##  II Gaming Video Quality Evaluator (GAMIVAL)

Fig. 1 shows the processing flow of the GAMIVAL VQA model. It employs the
spatial and temporal components of the RAPIQUE model [10], and the CNN-based
features from NDNetGaming [14]. It also employs a simple “neural noise” model.
These features are concatenated and further used to train an SVR model.

###  II-A Spatial Domain Features + Neural Noise

Previous spatial-temporal bandpass statistics-based video quality models, like
RAPIQUE [10], have been shown to efficiently capture the perceptual impacts of
complex real-world distortions. In these models, bandpass and divisive
normalization processes yield mean-subtracted, contrast-normalized (MSCN)
coefficients applied on the input image (or on a previously computed feature
map) $I(i,j)$:

$\hat{I}=\frac{I(i,j)-\mu(i,j)}{\sigma(i,j)+C},$ (1)

where $(i,j)$ are spatial indices, $C$ = 1 is a saturation constant that
prevents instabilities, and $\mu$ and $\sigma$ are weighted local means and
standard deviations [2, 3] within a gaussian-weighted spatial window centered
at location $(i,j)$.

|  |
---|---|---
(a) building2 | (b) Plants vs. Zombies | (c) Sonic
Figure 2: Exemplar test images and frames: (a) natural image from the LIVE IQA
database [29], and (b)-(c) two gaming video frames from the LIVE-Meta MCG
database [24].

It has been widely observed that the bandpass MSCN coefficients of natural
image or video frames reveal an underlying statistical regularity. However,
visual contents rendered by computer graphics, like gaming videos, typically
contain fewer details, and are generally smoother, hence their bandpass
statistics differ from those of naturalistic videos or images. Feature
computations on these regions can be less stable. Following [30], we introduce
a neural noise model:

$\widetilde{I}(i,j)=I(i,j)+W_{s}$ (2)

by which we add white Gaussian noise $W_{s}\sim N(0,\sigma^{2}_{W_{s}})$ to
the image before computing the MSCN coefficients.

As in [30], we have observed that the distributions of the MSCN coefficients
of high quality gaming content video frames tends towards Gaussianity. To
visualize this, we selected one natural image from the LIVE IQA database [29]:
building2, and two pristine video frames (720p) from two games in the LIVE-
Meta MCG database [24]: Plants vs. Zombies and Sonic, as shown in Fig. 2. Fig.
3 top portion shows the histograms of the MSCN coefficients of each of these
images before and after adding the simulated neural noise
($\sigma_{W_{s}}$=1.5). It may be observed that histograms of the gaming video
frames contain singular spikes, but after the noise was added to the gaming
video frames, their MSCN histograms became very similar to those of the
natural image, with a Gaussian appearance. Another gaming frame (State of
Survival) was chosen to further visualize how these regularities are affected
by distortion. Fig. 3 lower portion shows the MSCN coefficients of this frames
after resampling to different resolutions and bitrates. Quantifying these
deviations to predict perceived quality is effectively accomplished using the
basic statistical feature extraction module NSS-34 in RAPIQUE [10]. We apply
this module to the luma and chromatic feature maps at two scales (the original
scale resized to 540p and 270p) as in RAPIQUE [10], by uniformly sampling 2
frames per second, thus producing 680 spatial features.

---
Figure 3: Histograms of the MSCN coefficients of the one natural image and two
gaming video frames shown in Fig. 2 before adding noise (upper left) and after
adding noise (upper right), and the MSCN coefficients of the gaming video
“State of Survival” over varying resolutions (lower left) and bitrates (at
720p) (lower right).

###  II-B Temporal Domain Features + Neural Noise

Since temporal bandpass statistics have been shown to be predictive of frame
rate-dependent video quality [31], the authors of RAPIQUE [10] proposed a
temporal model utilizing temporal bandpass coefficients and subsequentially
applying spatial MSCN transforms, as in Eq. (1). The MSCN coefficients of
temporal bandpass coefficients of gaming videos also exhibit discontinuous
histograms, as shown in Fig. 4 top portion. Hence again apply additive neural
noise before computing the spatial MSCN coefficients

$\widetilde{Y_{k}}(\textbf{x},t)={Y_{k}}(\textbf{x},t)+W_{t}$ (3)

as before, where ${Y_{k}}(x,t)$ are the temporal bandpass coefficients,
$k=1,...,7$ denotes subband indices, $\textbf{x}=(x,y)$ and _t_ are spatial
and temporal coordinates, and $W_{t}\sim N(0,\sigma^{2}_{W_{t}})$ is the noise
added to the temporal model.

As shown in Fig. 4 lower portion, after adding the noise
($\sigma_{W_{t}}$=1.5), the histograms of the MSCN coefficients of temporal
bandpass coefficients of gaming videos also present Gaussian appearances.
These regularities are modified by the presence of distortions, which provides
a way of quantifying deviations, and therefore, to predict video quality
scores. Towards this end, we deploy the NSS-34 operator set in RAPIQUE [10] on
the temporal bandpass coefficients of each analyzed gaming video at two scales
(the original scale resized to 540p and 270p), sampling at 8 frames per second
and then applying a temporal Haar filter as in [10] to extract 7 bandpass
responses, yielding 34$\times$7$\times$2 = 476 temporal features at each time
sample. Section V-A provides additional information on the selection of the
parameters $\sigma_{W_{s}}$ and $\sigma_{W_{t}}$ of the additive noise
elements ${W_{s}}$ and ${W_{t}}$ in equations (2) and (3), respectively. It is
also shown that model performance significantly improves by the addition of
noise, and remains robust over even orders of magnitude of the parameters.

Figure 4: Histograms of raw temporal subband coefficients before adding noise
(upper left) and after adding noise (lower left), and the corresponding
spatial MSCN coefficients of the gaming video “PGA Golf Tour” before adding
noise (upper right) and after adding noise (lower right).

###  II-C CNN-based Features

Since existing gaming video content datasets contain too few videos to allow
training of CNN feature extractors from scratch, we instead applied
NDNetGaming model [14]. Unlike the CNN branch in RAPIQUE [10], (a pre-trained
ResNet-50 model [32]), the authors of [14] instead fine-tuned a DenseNet-121
model [33]) in two steps: first, they retrained the last 57 convolutional
layers of a DenseNet-121 by replacing the fully connected (FC) layer with a
dense layer, using VMAF values as proxy subjective quality labels. Second,
they fine-tuned the resulting pre-trained model by retraining the last 36
layers based on human subjective image quality ratings. Finally, they assigned
a different weight to each frame, based on its temporal complexity. Without
being retrained on different cloud game video databases, this model might not
deliver robust and generalized performance. The authors of [34] showed that,
without fine-tuning, the simple feature vector of an FC layer could be a
useful quality indicator if a shallow regressor is trained on top. Therefore,
we discarded the pooling layer of the original NDNetGaming model, thereby
yielding 1024 activation features. By also taking into account the time
complexity, we also devised a temporal sampling strategy. Instead of
extracting features on every frame, the CNN backbone (NDNetGaming) operates at
2 frame per second.

###  II-D Quality Evaluation

After obtaining all of the spatial, temporal, and CNN-based features within
each one-second chunk, they are concatenated into a 2180-dimensional feature
vector. By average-pooling the vectors within each video chunk, a vector of
features is obtained for the entire video and then trained with a shallow or
deep regressor head. In our implementation, we used a support vector machine
regressor (SVR) to map the features to predicted video quality scores [3, 7,
6, 35].

Figure 5: Box plots of PLCC, SRCC, and KRCC of evaluated BVQA algorithms on
the LIVE-Meta MCG dataset over 1000 splits. For each box, the median is
indicated by the center line, while the box edges represent the 25th and 75th
percentiles, and outliers are indicated by red circles.

##  III Performance evaluation

###  III-A The LIVE-Meta MCG Database

To evaluate the performance of our model against other methods, we used the
recently created LIVE-Meta MCG database [24]. This database contains mean
opinion scores (MOS) of 600 landscape and portrait gaming videos generated
from 30 pristine source sequences obtained from 16 different games using 20
different resolution-bitrate pairs to compress each pristine video. The
resolution ranged from 360p to 720p, while the bitrates range from 250 kbps to
2 mbps.

The subjective human study was conducted on a Google Pixel 5 mobile device.
All of the videos were upscaled to fit the mobile screen size (1080p) using
FFMPEG’s default bicubic interpolation function following decoding to native
videos. To map features to the MOS that was obtained from users viewing a
1080p Pixel 5 display, we applied all of the compared algorithms on the
display resolution.

###  III-B Evaluation Framework

####  III-B1 Compared Methods

We evaluated the performances of several popular NR IQA and VQA models: NIQE
[2], BRISQUE [3], TLVQM [35], VIDEVAL [34], RAPIQUE [10], VSFA [18], and two
gaming VQA models: NDNet-Gaming [14] and GAME-VQP [36]. NIQE is an
unsupervised model that pools predicted frame quality scores to generate video
quality predictions. Supervised VQA methods like BRISQUE, TLVQM, VIDEVAL,
RAPIQUE, and GAME-VQP operate by training an SVR to learn feature-to-score
mappings. The deep learning-based model VSFA uses a Resnet-50 [32] CNN
backbone to obtain quality-aware features, then maps them to MOS using a
single FC layer and a Gated Rectified Unit (GRU). NDNet-Gaming regresses video
quality predictions using a DenseNet-121 [33] backbone.

TABLE I: Median SRCC, KRCC, PLCC and RMSE on the LIVE-Meta MCG database over 1000 train-test splits. The underlined and boldfaced entries represent the best and top three performers. Metrics |  SRCC($\uparrow$) |  KRCC($\uparrow$) |  PLCC($\uparrow$) |  RMSE($\downarrow$)
---|---|---|---|---
NIQE |  -0.3900 |  -0.2795 |  0.4581 |  16.5475
BRISQUE |  0.7319 |  0.5395 |  0.7394 |  12.5618
TLVQM |  0.6553 |  0.4777 |  0.6889 |  13.5413
VIDEVAL |  0.7621 |  0.5756 |  0.7763 |  11.7520
RAPIQUE |  0.8740 |  0.6964 |  0.9039 |  8.0242
GAME-VQP |  0.8709 |  0.6885 |  0.8882 |  8.5960
NDNet-Gaming |  0.8382 |  0.6485 |  0.8200 |  10.5757
VSFA |  0.9143 |  0.7484 |  0.9264 |  7.1316
GAMIVAL |  0.9441 |  0.7963 |  0.9524 |  5.7683

####  III-B2 Evaluation Method

We randomly split the dataset into training and test sets (80%/20%), by
content, over 1000 iterations. The training set was further split to conduct
five-fold cross-validation. When splitting the training and validation sets,
we also ensured that the contents were mutually disjoint. We optimized the SVR
parameters $(C,\gamma)$ using grid search on the training and validation sets.
All of the evaluated supervised VQA methods were trained and tested using the
previous mentioned split strategy. NIQE is an unsupervised model, hence was
not trained. NDNet-Gaming is based on a pretrained model and was evaluated on
1000 test splits without training. While testing VSFA, the training and
validation sets were used to optimize the best-performing FC-GRU model
weights. Four performance metrics were used to evaluate algorithm performance:
the Spearman’s Rank-Order Correlation Coefficient (SRCC), the Kendall Rank
Correlation Coefficient (KRCC), Pearson’s Linear Correlation Coefficient
(PLCC), and the Root Mean Square Error (RMSE). The median values of these four
metrics against MOS are reported.

###  III-C Evaluation Results of NR-IQA and VQA Models

Table I shows the model performances on the LIVE-Meta MCG database. It may be
observerd that GAMIVAL achieved the best performance, while VSFA and RAPIQUE
ranked second and third respectively. It is worth noting that the two gaming
VQA models (NDNet-Gaming and GAME-VQP) also yielding fairly good fair
performance. For better visualization, Fig. 5 shows box plots of the SRCC,
KRCC, and PLCC values. It may be noticed that the GAMIVAL values are more
tightly grouped, indicating its stable performance and low variance.

Section V-B includes an ablation study that analyzes the contributions of each
GAMIVAL feature set. We further summarize the complexity and runtime
comparison of the NR-VQA models in Section V-C.

##  IV Conclusion and future work

We have developed a dual path gaming video quality assessment model that
deploys distortion-sensitive natural scene features along one path, and
distortion and semantically aware deep features along the other path. In order
to better regularize the distributions of the space-time MSCN video
coefficients, especially on extremely smooth or constant regions often found
in gaming content videos, we applied an additive “neural noise” mechanism
which led to much improved prediction performance. Evaluations on a recent
large-scale MCG video database show that the new model, GAMIVAL, achieves
state-of-the-art quality prediction accuracy with low computational complexity
as compared with leading conventional and deep learning based VQA models.

##  V Appendix

TABLE II: Median SRCC, KRCC, PLCC and RMSE on the LIVE-Meta MCG database over 100 train-test splits. $\sigma_{W_{s}},\sigma_{W_{t}}$ |  SRCC($\uparrow$) |  KRCC($\uparrow$) |  PLCC($\uparrow$) |  RMSE($\downarrow$)
---|---|---|---|---
0 |  0.8949 |  0.7252 |  0.9108 |  7.5875
0.01 |  0.9371 |  0.7878 |  0.9496 |  5.9064
0.05 |  0.9376 |  0.7878 |  0.9490 |  5.8108
0.1 |  0.9374 |  0.7887 |  0.9520 |  5.6773
0.3 |  0.9407 |  0.7893 |  0.9521 |  5.6177
0.5 |  0.9427 |  0.7955 |  0.9550 |  5.7544
1 |  0.9392 |  0.7886 |  0.9505 |  5.7000
1.5 |  0.9439 |  0.7962 |  0.9526 |  5.6941
2 |  0.9366 |  0.7857 |  0.9493 |  5.8625
3 |  0.9387 |  0.7893 |  0.9494 |  5.7856

###  V-A Neural Noise

To study the parameters $\sigma_{W_{s}}$ and $\sigma_{W_{t}}$ of the noise
elements $W_{s}$ and $W_{t}$ in (2) and (3), we evaluted GAMIVAL on the LIVE-
Meta MCG Database while varying these parameters. As may be seen from Table
II, the performance of GAMIVAL was remarkably robust over a very wide range of
the two noise parameters, which we held equal, even over several orders of
magnitude. We selected the best-performing values
$\sigma_{W_{s}}=\sigma_{W_{t}}=1.5$, but GAMIVAL significantly outperformed
all other models for all nonzero parameter values.

###  V-B Ablation Study

To analyze the contribution of each GAMIVAL feature set, we conducted an
ablation study. Fig. 7 shows the increase in performance obtained as each
feature combination is added. It may be observed that the temporal “additive
noise” features improved performance more than did the spatial “additive
noise” features. The NDNetGaming component contributes significantly to the
performance, likely because it adds semantic significance to the quality
predictions.

###  V-C Complexity and Runtime Comparison

The experiments were performed in MATLAB R2021a and Python 3.6.10 under Ubuntu
20.04.4 LTS on a Desktop with an Intel Xeon CPU E5-2620<EMAIL_ADDRESS>processor,
and 64GB RAM. For a fair comparison, all algorithms were carried out on the
CPU. We used one 1080x2160 video from the LIVE-Meta MCG database to study the
computational efficiency of the NR-VQA models. Table III lists the execution
time and floating point operations, while Fig. 8 shows visual representations
of the trade-off between performance and complexity. It may be observed that
although VSFA delivers very good performance, it has the highest computational
complexity. The top-performing algorithms, RAPIQUE and GAMIVAL, are both
computationally efficient, making them promising options for real-time video
quality prediction applications.

(a) MOS=73.96, RAPIQUE=67.14, GAMIVAL=77.08 (b) MOS=73.14, RAPIQUE=66.38, GAMIVAL=74.05 |  (c) MOS=51.57, RAPIQUE=59.55, GAMIVAL=54.69 |  (d) MOS=40.10, RAPIQUE=56.78, GAMIVAL=48.39
---|---|---
Figure 6: Exemplar test frames of 720p gaming videos having varying bitrate
values, taken from the LIVE-Meta MCG Database, along with their MOS and
predicted quality score computed by RAPIQUE and GAMIVAL: (a) Hungry Dragon,
(b) State of Survival, (c) PGA, (d) Sonic. Figure 7: Results of ablation study
of GAMIVAL against three feature combinations:Noisy SpatialNSS (SNSS, Sec.
II-A), Noisy TemporalNSS (TNSS, Sec. II-B), and (NDNet) deep features (Sec.
II-C).
---
Figure 8: Scatter plots of SRCC of NR-VQA algorithms versus runtime on 1080p videos (right) and performance versus FLOPs (left). TABLE III: Model computation complexity on a 1080p video. Model | Platform |  Time (seconds) |  FLOPS ($\times 10^{9}$)
---|---|---|---
NIQE |  MATLAB |  728 |  1965
BRISQUE |  MATLAB |  205 |  241
TLVQM |  MATLAB |  588 |  283
VIDEVAL |  MATLAB |  959 |  2334
RAPIQUE |  MATLAB |  103 |  322
GAME-VQP |  MATLAB |  2053 |  11627
NDNet-Gaming |  Python, Tensorflow |  779 |  126704
VSFA |  Python, Pytorch |  2385 |  229079
GAMIVAL |  Python, Tensorflow, MATLAB |  201 |  8683

## References

  * [1] R. Shea, J. Liu, E. C.-H. Ngai, and Y. Cui, “Cloud gaming: architecture and performance,” _IEEE Network_ , vol. 27, no. 4, pp. 16–21, 2013.
  * [2] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,” _IEEE Signal Processing Letters_ , vol. 20, no. 3, pp. 209–212, 2012.
  * [3] A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” _IEEE Transactions on Image Processing_ , vol. 21, no. 12, pp. 4695–4708, 2012.
  * [4] D. Kundu, D. Ghadiyaram, A. C. Bovik, and B. L. Evans, “No-reference quality assessment of tone-mapped HDR pictures,” _IEEE Transactions on Image Processing_ , vol. 26, no. 6, pp. 2957–2971, 2017.
  * [5] W. Xue, X. Mou, L. Zhang, A. C. Bovik, and X. Feng, “Blind image quality assessment using joint statistics of gradient magnitude and laplacian features,” _IEEE Transactions on Image Processing_ , vol. 23, no. 11, pp. 4850–4862, 2014.
  * [6] D. Ghadiyaram and A. C. Bovik, “Perceptual quality prediction on authentically distorted images using a bag of features approach,” _Journal of Vision_ , vol. 17, no. 1, pp. 32–32, 2017.
  * [7] M. A. Saad, A. C. Bovik, and C. Charrier, “Blind prediction of natural video quality,” _IEEE Transactions on Image Processing_ , vol. 23, no. 3, pp. 1352–1365, 2014.
  * [8] J. P. Ebenezer, Z. Shang, Y. Wu, H. Wei, and A. C. Bovik, “No-reference video quality assessment using space-time chips,” in _2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP)_. IEEE, 2020, pp. 1–6.
  * [9] J. P. Ebenezer, Z. Shang, Y. Wu, H. Wei, S. Sethuraman, and A. C. Bovik, “Chipqa: No-reference video quality prediction via space-time chips,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 8059–8074, 2021.
  * [10] Z. Tu, X. Yu, Y. Wang, N. Birkbeck, B. Adsumilli, and A. C. Bovik, “RAPIQUE: Rapid and accurate video quality prediction of user generated content,” _IEEE Open Journal of Signal Processing_ , vol. 2, pp. 425–440, 2021.
  * [11] N. Barman, S. Schmidt, S. Zadtootaghaj, M. G. Martini, and S. Möller, “An evaluation of video quality assessment metrics for passive gaming video streaming,” in _Proceedings of the 23rd Packet Video Workshop_ , 2018, pp. 7–12.
  * [12] N. Barman, S. Zadtootaghaj, S. Schmidt, M. G. Martini, and S. Möller, “An objective and subjective quality assessment study of passive gaming video streaming,” _International Journal of Network Management_ , vol. 30, no. 3, p. e2054, 2020.
  * [13] S. Zadtootaghaj, S. Schmidt, S. S. Sabet, S. Möller, and C. Griwodz, “Quality estimation models for gaming video streaming services using perceptual video quality dimensions,” in _Proceedings of the 11th ACM Multimedia Systems Conference_ , 2020, pp. 213–224.
  * [14] M. Utke, S. Zadtootaghaj, S. Schmidt, S. Bosse, and S. Möller, “NDNetGaming-development of a no-reference deep CNN for gaming video quality prediction,” _Multimedia Tools and Applications_ , pp. 1–23, 2020\.
  * [15] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2015.
  * [16] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Commun. ACM_ , vol. 60, no. 6, p. 84–90, may 2017.
  * [17] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2014, pp. 580–587.
  * [18] D. Li, T. Jiang, and M. Jiang, “Quality assessment of in-the-wild videos,” in _Proceedings of the 27th ACM International Conference on Multimedia_ , 2019, pp. 2351–2359.
  * [19] Z. Ying, M. Mandal, D. Ghadiyaram, and A. Bovik, “Patch-VQ:’patching up’the video quality problem,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 14 019–14 029.
  * [20] N. Barman, S. Zadtootaghaj, S. Schmidt, M. G. Martini, and S. Möller, “GamingVideoSET: a dataset for gaming video streaming applications,” in _2018 16th Annual IEEE Workshop on Network and Systems Support for Games (NetGames)_ , 2018, pp. 1–6.
  * [21] N. Barman, E. Jammeh, S. A. Ghorashi, and M. G. Martini, “No-reference video quality estimation based on machine learning for passive gaming video streaming applications,” _IEEE Access_ , vol. 7, pp. 74 511–74 527, 2019\.
  * [22] S. Wen, S. Ling, J. Wang, X. Chen, Y. Jing, and P. L. Callet, “Subjective and objective quality assessment of mobile gaming video,” in _ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2022, pp. 1810–1814.
  * [23] X. Yu, Z. Tu, Z. Ying, A. C. Bovik, N. Birkbeck, Y. Wang, and B. Adsumilli, “Subjective quality assessment of user-generated content gaming videos,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2022, pp. 74–83.
  * [24] A. Saha, Y.-C. Chen, C. Davis, B. Qui, X. Wang, R. Gowda, I. Katsavounidis, and A. C. Bovik, “Study of subjective and objective quality assessment of mobile cloud gaming videos,” in _IEEE Transactions on Image Processing_ , in peer review.
  * [25] S. Zadtootaghaj, N. Barman, S. Schmidt, M. G. Martini, and S. Möller, “NR-GVQM: A no reference gaming video quality metric,” in _2018 IEEE International Symposium on Multimedia (ISM)_. IEEE, 2018, pp. 131–134.
  * [26] S. Göring, R. R. R. Rao, and A. Raake, “NOFU—a lightweight no-reference pixel based video quality model for gaming content,” in _2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX)_. IEEE, 2019, pp. 1–6.
  * [27] Z. Li, A. Aaron, I. Katsavounidis, A. Moorthy, and M. Manohara, “Toward a practical perceptual video quality metric,” _The Netflix Tech Blog_ , vol. 6, no. 2, 2016.
  * [28] S. J. Pan and Q. Yang, “A survey on transfer learning,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 22, no. 10, pp. 1345–1359, 2009.
  * [29] H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” _IEEE Transactions on Image Processing_ , vol. 15, no. 11, pp. 3440–3451, 2006.
  * [30] Y. Jin, T. Goodall, A. Patney, R. Webb, and A. Bovik, “A foveated video quality assessment model using space-variant natural scene statistics,” in _2021 IEEE International Conference on Image Processing (ICIP)_. IEEE, 2021, pp. 1419–1423.
  * [31] P. C. Madhusudana, N. Birkbeck, Y. Wang, B. Adsumilli, and A. C. Bovik, “ST-GREED: Space-time generalized entropic differences for frame rate dependent video quality prediction,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 7446–7457, 2021.
  * [32] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 770–778.
  * [33] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 4700–4708.
  * [34] Z. Tu, Y. Wang, N. Birkbeck, B. Adsumilli, and A. C. Bovik, “UGC-VQA: Benchmarking blind video quality assessment for user generated content,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 4449–4464, 2021.
  * [35] J. Korhonen, “Two-level approach for no-reference consumer video quality assessment,” _IEEE Transactions on Image Processing_ , vol. 28, no. 12, pp. 5923–5938, 2019.
  * [36] X. Yu, Z. Tu, N. Birkbeck, Y. Wang, B. Adsumilli, and A. C. Bovik, “Perceptual quality assessment of UGC gaming videos,” 2022.


∎

11institutetext: Inge S. Helland 22institutetext: Department of Mathematics,
University of Oslo
P.O. Box 1053 Blindern, N-0316 Oslo, Norway
Tel.: +47-93688918
22email<EMAIL_ADDRESS>
# The Bell experiment and the limitations of actors

Bell experiment

Inge S. Helland

(Received: date / Accepted: date)

###### Abstract

The well known Bell experiment with two actors Alice and Bob is considered.
First the simple deduction leading to the CHSH inequality under local realism
is reviewed, and some arguments from the literature are recapitulated. Then I
take up certain background themes before I enter a discussion of Alice’s
analysis of the situation. An important point is that her mind is limited by
the fact that her Hilbert space in this context is two-dimensional. General
statements about a mind’s limitation during a decision process are derived
from recent results on the reconstruction of quantum theory from conceptual
variables. These results apply to any decision situation. Let all the data
from the Bell experiment be handed over to a new actor Charlie, who performs a
data analysis. But his mind is also limited: He has a four-dimensional Hilbert
space in the context determined by the experiment. I show that this implies
that neither Alice nor Charlie can have the argument leading to the CHSH
inequality as a background for making decisions related to the experiment.
Charlie may be any data analyst, and he may communicate with any person. It is
argued that no rational person can be convinced by the CHSH argument when
making empirical decisions on the Bell situation.

###### Keywords:

Bell’s theorem CHSH inequality conceptual variables limitation quantum
foundation.

##  1 Introduction

Quantum mechanics is held by almost all physicists to be the most successfull
theory ever deviced (although this assertion has been challenged RefJ1 ).
Nevertheless there has been and still are serious discussions about the
interpretation of the theory. The relevant Wikipedia entry lists more than 16
different interpretations of quantum mechanics. Although some of these are
related, this is obviously not a satisfying situation.

Much of the present discussion has centered around the Bell theorem RefJ2 :
Quantum theory is inconsistent with local realism. In more concrete terms, the
Bell inequalities, in particular the CHSH inequality RefJ3 , which is derived
by a simple argument assuming local realism, can be violated by quantum
mechanics.

This has raised the question: Can the CHSH inequality also be violated by
Nature, regardless of whether or not quantum mechanics is seen as an all-
embracing true theory? Numerous experiments have been done to test this
question, the first by Aspect et al. RefJ4 . These experiments have been
criticized, and a list of possible loopholes have been identified RefJ0 .
Finally. in 2015 several loophole-free experiments were performed RefJ5  RefJ6
, and the conclusion seems to be clear: _There exist conditions under which
the CHSH inequality is violated in practice._

This has lead to new discussions: Should we abandon the hypothesis of
locality, which seems to contradict relativity theory? (Using seemingly
reasonable arguments, this is for instance claimed in RefJ7 .) Or should we in
some sense or other question realism as a universal assumption?

In the present article, I will argue for a version of non-realism: From a
general theorem on Hilbert space reconstruction it is proved that the mind of
any actor will be limited in a given context. In concrete terms he is not able
to have in his mind more than two relevant maximally accessible conceptual
variables when making a decision, if these are essentially different and both
related to his main thought. These terms are precisely defined. Applied to the
Bell experiment, this is shown to imply that no actor is able to have all the
assumptions behind the CHSH inequality in his mind in a context where he is to
make decisions related to the experiment.

My arguments will rely on a general epistemic interpretation of quantum
theory, advocated in the book RefB1 . More details will be given below.

##  2 The Bell experiment and the CHSH inequality

Two observers Alice and Bob are spacelikely separated at the moment when they
observe. Midways between them is a source of entangled spin 1/2 particles, one
particle in a pair is sent towards Alice, the other towards Bob. In concrete
terms, the joint state of the two particles is given by

$|\psi^{0}\rangle=\frac{1}{\sqrt{2}}(|1+\rangle|2-\rangle-|1-\rangle|2+\rangle).$
(1)

Here and in the following the spin component in any direction is normalized to
$\pm 1$. In (1) $|1u\rangle$ means that the spin component of particle 1 in
some fixed ($z$) direction is $u$, while $|2v\rangle$ means that the spin
component of particle 2 in the $z$-direction is $v$. This state expresses that
the total spin of the two particles is 0. One can imagine that these particles
previously have been together in some bound state with spin 0.

Alice is given the choice between measuring the spin component of her particle
in one of two directions $a$ or $a^{\prime}$. If she measures in the
$a$-direction, her response ($\pm 1$) is called $A$, and if she measures in
the $a^{\prime}$ direction, her response is called $A^{\prime}$. Similarly,
Bob can measure in one of two directions $b$ (giving a response $B$) or
$b^{\prime}$ (giving a response $B^{\prime}$). The whole procedure is repeated
$n$ times with different entangled particle pairs and with different
directions/ settings chosen by Alice and Bob.

We now temporarily take the point of departure that all these response
variables exist in some sense. This can be seen as an assumption on realism.
At the very least, we will assume that this point of departure is meaningful
for some observer or for a group of communicating observers.

Since all responses then are $\pm 1$, we then have the inequality

$AB+A^{\prime}B+AB^{\prime}-A^{\prime}B^{\prime}=A(B+B^{\prime})+A^{\prime}(B-B^{\prime})\leq
2.$ (2)

The argument for this is simply: $B$ and $B^{\prime}$ are either equal to one
another or unequal. In the first case, $B-B^{\prime}=0$ and $B+B^{\prime}=\pm
2$; in the last case $B-B^{\prime}=\pm 2$ and $B+B^{\prime}=0$. Therefore,
$AB+A^{\prime}B+AB^{\prime}-A^{\prime}B^{\prime}$ is equal to either $A$ or
$A^{\prime}$, both of these being $\pm 1$, multiplied by $\pm 2$. All
possibilities lead to $AB+A^{\prime}B+AB^{\prime}-A^{\prime}B^{\prime}=\pm 2$.

From this, a statistician will argue: Assume that we can consider
$A,A^{\prime},B$ and $B^{\prime}$ as random variables, defined on the same
probability space $(\Omega,\mathcal{F},P)$. Then by taking expectations over
the terms in (2), we find

$E(AB)+E(A^{\prime}B)+E(AB^{\prime})-E(A^{\prime}B^{\prime})\leq 2.$ (3)

A physicist will have a related argument: Assume that there is a hidden
variable $\lambda$ such that
$A=A(\lambda),A^{\prime}=A^{\prime}(\lambda),B=B(\lambda)$ and
$B^{\prime}=B^{\prime}(\lambda)$. The assumption that such a hidden variable
exists, is called local realism in the physical literature. By integrating
over the probability distribution $\rho$ of $\lambda$, this gives

$E(AB)=\int A(\lambda)B(\lambda)\rho(\lambda)d\lambda$ (4)

etc.. Thus by integrating term for term in (2), we again find (3).

Of course the above two arguments are equivalent; it is just a question of
using either the notation $(\omega,P)$ or $(\lambda,\rho)$. There are
different traditions here. These arguments are reviewed and discussed in
detail by Richard Gill Ref18 .

The inequality (3) is called the CHSH inequality after the authors of RefJ3 ,
and has been the source of much controversy. First, it is known that if we use
quantum mechanics to model the above experiment, one can find settings such
that the CHSH inequality is violated. Secondly, recent loophole-free
experiments RefJ5  RefJ6  have shown that the CHSH inequality may be violated
in practice.

Thus the simple assumptions sketched above for (3) cannot hold.

##  3 Briefly on the literature

There is a large physical literature around these questions. First, various
authors have used rather advanced arguments claiming that Bell’s theorem is
wrong, and these arguments have each time been countered by Gill and
collaborators (see for instance Ref12 ). Much of the literature has recently
been reviewed by Kupczynski Ref13 , and Marian Kupczynski has also arrived at
his own conclusions there. I agree with him that a joint probability
distribution of $(A,A^{\prime},B,B^{\prime})$ does not exist, hence that a
joint probability distribution of the 4 variables
$(AB,AB^{\prime},A^{\prime}B,A^{\prime}B^{\prime})$ does not exist. In a
physical setting, Justo Pastor Lambare Ref17  has argued that this should
imply that 4 different hidden variables $\lambda_{i}$ should be chosen in the
equations corresponding to (4).

Another interesting argument has been put forward by Ilja Schmelzer RefJ7 . He
argues that the key formula

$E(AB|a,b)=\int A(a,b,\lambda)B(a,b,\lambda)\rho(\lambda)d\lambda$ (5)

follows from the logic of plausible reasoning (the objective Bayesian
interpretation of probability theory) taken alone, and therefore that the
violation of the CHSH inequality has as a consequence that Einstein causality
(in essence the assumption of nonlocality) has been violated.

To counter this last argument, we must go somewhat into the logic of plausible
reasoning. Applied statistics is based upon a large number of propositions
about parameters, and we may assume that the set of all these propositions
form a Boolean algebra. The tradition in statistics is to associate such
Boolean algebras with set theory, and in fact this association can be made
precise. Mathematically, a Stone space is a compact totally disconnected
Hausdorff space; these details are not too important. But Stone’s
representation theorem Ref14  says that every Boolean algebra $B$ is connected
to a Stone space $S(B)$ in the following sense: The topology on $S(B)$ is
generated by a (closed) basis consisting of all sets of the form $\\{x\in
S(B)|b\in x\\}$. Then every Boolean algebra $B$ is isomorphic to the algebra
of subsets of its Stone space $S(B)$ that are both closed and open.

The problem is to which extent one can associate probabilities to all such
propositions about parameters. Here the answer depends on which school in
statistical inference you belong to. An extreme Bayesian will be willing to
assign probabilities to all propostions. Most statisticians find themselves in
an in-between position, for some statements they can associate probabilities,
epistemic probabilities, for some statements they can not. Sometimes, like
when the problem in question has some symmetry and we can associate a prior to
the right invariant measure of a transitive group $G$, one can act as a
Bayesian, but often this attitude is not possible. In this article I will also
support the frequentist tradition in statistics, where prior probabilities of
propositions are not necessarily assumed to be available. Taking such an
attitude, statements like (5) are not automatically true. In fact, we will
argue below that all expectations and every statistical analysis should be
taken from the point of view of an observer/ actor or from the point of view
of a group of communicating actors. This point of departure is central for all
the different arguments in the book RefB1 , and it will be crucial for the
discussion that I will make below.

##  4 An epistemic approach towards quantum theory

In the literature, many different interpretations of quantum theory are given.
Some of these emphasize the epistemological aspect of the quantum state, which
I will also do here. Among these, one could mention QBism Ref2 , founded by
Christopher Fuchs, and Carlo Rovelli’s Relational Quantum Mechanics Ref3 . The
latter relies upon 2 hypotheses: 1) All systems for describing systems
relative to an observer are equivalent. 2) Quantum mechanics provides a
complete scheme for description of the physical world. A general physical
theory is a theory about the state that physical systems have, relative to
each other. In particular, an observer may be such a system.

In RefB1  a general epistemic view upon quantum theory is advocated. The basis
is a set of conceptual variables connected to some agent/observer or shared by
a group of communicating observers. Some of these variables may be given
numerical values through some experiment; these are called accessible
variables or epistemic conceptual variables (e-variables). An example may be
the spin component $\theta^{a}$ of a particle in some given direction $a$. (In
the Bell experiment setting above, I have used the notation $A=\theta^{a}$
etc.) Other variables are inaccessible, can not be given values. An example
may be the full spin vector $\phi$ of a particle. Variables such as
$\theta^{a}$ which can not be extended without loosing their accessability
property, are called maximally accessible. Based upon this view, ontological
aspects of the quantum state may also be considered Ref71 .

One must distinguish sharply between conceptual variables attached to a single
actor and conceptual variables attached to a group of actors. According to
Zwirn RefZ , see below, only the first variables have a primary role to play
in our description of the world. But when making decisions, and when arriving
at joint descriptions after having communicated, both kinds of variables will
be important.

In the views developed in RefB1 , quantum theory may be based upon
concentrating on such conceptual variables. In agreement with N. David Mermin
Ref25  the only ‘real things’ in physics are events. In the discrete case an
event is given in some context by ‘$\theta=u$’, where $\theta$ is a conceptual
variable, and $u$ is one of its values. It is argued in RefB1  that these
variables either should be connected to the mind of a single actor or the
joint minds of a group of communicating actors. And they should always be
associated with some context and with a concrete physical situation.

It is important that my conceptual variables are connected to a concrete
context. By the well-known Kochen-Specker theorem it is impossible to assign
simultaneously, noncontextual definite values to all (of a finite set of)
quantum mechanical observables in a consistent manner.

The following result is developed in RefB1  and improved in Ref50 : For every
maximally accessible discrete e-variable $\theta$, varying on some space
$\Omega_{\theta}$, on which a transitive group $G$ can be defined, there
corresponds under weak condition a unique operator $A^{\theta}$ defined on
some common Hilbert space $\mathcal{H}$, and to every question ‘What is the
value of $\theta$ if measured?’ together with a sharp answer ‘$\theta=u$’
there is a vector, an eigenvector of $A^{\theta}$ with eigenvalue $u$.

More precisely, in Ref50  (cp. also RefB1 ) the following is proved:

Theorem 1. Consider a situation where there are two maximally accessible
conceptual variables $\theta$ and $\eta$ in the mind of an actor or in the
minds of a communicating group of actors. Make the following assumptions:

(i) On one of these variables, $\theta$, there can be defined group actions
from a transitive group $G$ with a trivial isotropy group and with a left-
invariant measure $\rho$ on the space $\Omega_{\theta}$.

(ii) There exists a unitary irreducible representation $U(\cdot)$ of the group
$G$ defined on $\theta$ such that the coherent states $U(g)|\theta_{0}\rangle$
are in one-to-one correspondence with the values of $g\in G$, and hence with
the values of $\theta$.

(iii) The two maximally accessible variables $\theta$ and $\eta$ can both be
seen as functions of an underlying inaccessible variable
$\phi\in\Omega_{\phi}$. There is a transformation $k$ acting on
$\Omega_{\phi}$ such that $\eta(\phi)=\theta(k\phi)$.

Then there exists a Hilbert space $\mathcal{H}$ connected to the situation,
and to every accessible conceptual variable there can be associated a unique
symmetric operator on $\mathcal{H}$.

A simple assumption implying the technical condition (ii) is given in Ref50 ,
and this technical condition can be shown to hold in the spin 1/2 case.

Condition (iii) is of particular interest. Two conceptual variables satisfying
this condition are said to be _related_. When it is impossible to find an
underlying variable $\phi$ such that this condition holds, we say that
$\theta$ and $\eta$ are _essentially different_.

In particular, these conditions hold for pairs of components $\theta^{a}$ and
$\theta^{a^{\prime}}$ of a spin 1/2 particle, where $\phi$ is the inaccessible
spin vector, and $k$ is a particular rotation of $\phi$. (We can take $k$ as a
$180^{o}$ rotation around the midline between the directions $a$ and
$a^{\prime}$.) Then the corresponding Hilbert space $\mathcal{H}$ is two-
dimensional, and every question ‘What is the value of $\theta^{a}$?’ together
with an answer ‘$\theta^{a}=u$’, where $u=\pm 1$, corresponds to a unique unit
vector in $\mathcal{H}$. In fact, here, every unit vector in $\mathcal{H}$ has
an interpretation in the form of a question-and-answer pair. This has recently
been generalized to the case of several such questions by Höhn Ref30  and Höhn
and Wever Ref20 .

The assumption that there can be defined a transitive group acting upon
$\theta$ is crucial. It can easily be satisfied when the range of $\theta$ is
finite or is the whole line ${\mathbb{R}}^{1}$, but may be more difficult when
$\theta$ is a vector.

Thus Theorem 1 can be used in a new foundation of parts of quantum theory, but
this foundation may then in practice first be limited to finite-valued or
scalar variables, and where these are maximally accessible. After this,
operators for other accessible conceptual variables can be found by looking at
them as functions of a maximal variable, and then making use of the spectral
theorem for the operator corresponding to this variable. Note also that in a
situation where we have several independent (scalar) accessible conceptual
variables, an operator corresponding to the vector composed of all these may
be defined by taking tensor products.

Experiments to measure an accessible conceptual variable $\theta$ are mostly
seen as perfect in the quantummechanical literature, measurement apparata are
seen as completely accurate. In practice, the apparata give inaccurate data
$x$ which are connected to $\theta$, and this inaccuracy may be modeled by a
statistical model $p(x|\theta)$. The observer/ experimentalist may then give
an estimate $\widehat{\theta}$ from the data $x$. From the point of view of
Convivial Solipsism RefJ8 , this estimate may be seen as the result of
measurement perceived by the consciousness of the observer, taken with respect
to later decisions as the true resulting value of $\theta$ if the current
experiment is accurate enough.

When the result of the experiment is discrete, as in the Bell experiment case,
it is often not problematic to regard the estimate from the data as the ‘true’
value in some sense (but, and this is important, again connected to an actor
or to a group of communicating actors). This will be assumed in the following.
In one run, $\theta^{a}=A$ may be seen as a fixed value, but seen from the
point of view of many repeated runs, they are random variables. But they are
random variables in an epistemic sense, their probablity distributions are
epistemic probabilities. (For a discussion of this concept in a statistical
setting, see RefB1 ). And, I repeat: The epistemic conceptual variables are
connected to an actor, for instance Alice in the Bell experiment, or to a
group of communicating actors. Later I will let Alice communicate with a more
knowledgeable actor Charlie.

##  5 The conditionality principle

The theory of statistical inference relies on certain principles, one of these
is the conditionality principle. The principle was first proposed by David Cox
Ref15  in 1958, based on a very simple example: Suppose that I, as a chemist
has made measurements on some material, and I want my measurements to be
analysed by a laboratory before I make a simple statistical treatment of the
results. I have the choice between two laboratories, one in New York and one
in San Fransisco. I decide to toss an unbiased coin to make a decision between
the labs.

In principle one might imagine that my final statistical analysis should be
based on the whole epistemic process, including the coin tossing. But David
Cox made the very reasonable assumption that one should condition the
statistical analysis on the result of this coin toss.

He then made a very bold generalization of this example: Let a data variable
$z$ have a distribution that is independent of the parameters of the
experiment that we want to analyze. Then $z$ is called ancillary. The general
conditionality principle then says: _One should condition the statistical
analysis on the value of any ancillary data variable._

One can discuss the principle in this generality; in fact, I have done so
Ref16  several years ago. But my point is now: The conditionality principle
should be used on the settings chosen by Alice and Bob in the Bell experiment
in their own statistical analysis of their data. The $z$ is then, for each of
them, the settings chosen (which in some discussions are thought about as the
results of some coin tosses.)

##  6 Convivial solipsism

A new philosophy, convivial solipsism, which may help to understand the basis
of quantum mechanics, a philosophy which also may be linked to my own ideas,
was recently proposed by Zwirn RefZ .

In general, solipsism is a philosophy with many variants. It is based on the
view that everything that we can know for sure by our mind is connected to
this mind. My mind is an autonomous separate world. The convivial variant also
recognizes that other people have their minds, and thus have sure statements
connected to their minds. And communication between different people is
possible. People that have communicated and agreed on certain questions may be
seen as a new unit, a new world, with respect to these questions. In a
macroscopic context this may be linked to a theory of making decisions.

##  7 Born’s formula

In RefB1  the Born rule is formulated as follows: Assume in general two
maximally accessible conceptual variables $\theta^{a}$ and $\theta^{b}$. Let,
for some observer or group of observers the event $\theta^{a}=u$ correspond to
the ket vector $\psi^{a}$ and the event $\theta^{b}=v$ correspond to the ket
vector $\psi^{b}$ in some common Hilbert space $\mathcal{H}$. Then

$P(\theta^{b}=v|\theta^{a}=u)=|\langle\psi^{b}|\psi^{a}\rangle|^{2}.$

In the Bell experiment, if $A=\theta^{a}$ is Alice’s response and $B=\eta^{b}$
is Bob’s response, and assuming an actor for which both $A$ and $B$ are
meaningful, can be related to the same Hilbert space, we find from this

$P(B=\pm 1|A=+1)=(1\pm\mathrm{cos}(a,b))/2,$

and assuming in addition that $P(A=-1)=P(A=+1)=1/2$, this gives
$E(AB)=-\mathrm{cos}(a,b)$. From this again follows that according to quantum
mechanics, the CHSH inequality may be violated, for instance $a\sim 0^{o}$,
$a^{\prime}\sim 90^{o}$, $b\sim 225^{o}$ and $b^{\prime}\sim 135^{o}$ gives

$E(AB)+E(AB^{\prime})+E(A^{\prime}B)-E(A^{\prime}B^{\prime})=2\sqrt{2}.$ (6)

In RefB1  the Born rule was derived by making three assumptions which
qualitatively may be formulated as: 1) The variable $\theta^{a}$ (here $A$) is
maximally accessible; 2) The so-called likelihood principle from statistics
holds; 3) The observer is either himself perfectly rational, or has ideals
which can be thought of in terms of an artificial, perfectly rational actor,
where rationality is given by the Dutch book principle. Of course an
additional assumption behind (6) is that each of the conceptual variables
$AB$, $AB^{\prime}$, $A^{\prime}B$ and $A^{\prime}B^{\prime}$, taken
separately, make sense to the actor(s) in question.

##  8 A mathematical theory of human decisions

Let the person A be in a decision process. He has the choice between the
prospects $\pi_{1},...,\pi_{r}$. Introduce a decision variable $\xi$ taking
$r$ values: $\xi=k$ if and only if $\pi_{k}$ is to be realized ($k=1,...,r$).
This is a variable in the mind of A.

In general, A is in some context at the time t when he is to take his
decision. In this context and at this time he can have several variables in
his mind: $\theta,\xi,\lambda,...$. I will call these conceptual variables. If
a variable $\lambda$ can be given some value at a future time, I will say that
$\lambda$ is accessible.

Say that a conceptual variable $\theta$ is ‘less than or equal to’ the
conceptual variable $\lambda$ if $\theta=f(\lambda)$ for some function $f$.
This defines a partial ordering both among all conceptual variables and also
among the accessible conceptual variables.

I will assume that if $\lambda$ is accessible, and $\theta=f(\lambda)$, then
$\theta$ is also accessible.

Now to a main assumption of my model: I will assume that all of the conceptual
variables in the mind of A, or some of them, can be seen as functions of an
underlying inaccessible $\phi$, belonging to the subconsciousness of A. As
such, $\phi$ can never be known by A, nor by any other person. Some
intelligent person, knowing A, having observed him over some period, and
knowing some practical psychology, may perhaps find a rough estimate of
$\phi$.

In the case where the accessible conceptual variables are spin components of a
particle, we can let $\phi$ be the inaccessible spin vector.

In general from this and from Zorn’s lemma, maximally accessible conceptual
variables (according to this partial ordering) always exist, since $\phi$ can
be seen as an upper bound for a set of accessible conceptual variables.
Trivially, the spin components of a particle are maximally accessible.

##  9 The data analysis made by Alice

Assume that a Bell experiment has been done. Before she has any contact with
Bob, Alice has a list of $n$ data from herself, settings $a$ or $a^{\prime}$
and corresponding responses $A$ or $A^{\prime}$.

Now by the conditionality principle, her analysis should be conditional, given
her setting, either $a$ or $a^{\prime}$. Assume that she first concentrates on
the runs with setting $a$ and a corresponding response $A$. For example one
may concentrate on the runs where $A=+1$, an event which corresponds to a
unique ket vector $\psi^{a}$ in her two-dimensional Hilbert space
$\mathcal{H}$. In fact the argument below will hold for any state of knowledge
about the responce $A$. We will be particularly interested in the mixed state
determined by $P(A=-1)=P(A=+1)=1/2$.

By Theorem 1, the Hilbert space describing Alice’s mind in the situation can
be constructed from two maximally accessible conceptual variables.

With this as a background, one can imagine two scenarios. First, she may also
have in mind the possible response $A^{\prime}$ from her other setting
$a^{\prime}$. Then from a general theory of a mind’s limitation, discussed in
the next section, she is not able to think of any more conceptual variables.
In particular, the possible responses made by Bob can not be addressed by her
in this state, and to her then, the argument leading to (2) and (3) does not
make any sense. Thus she has no opinion about the validity or not of the CHSH
inequality.

The other scenario is that she does not think of $A^{\prime}$ at all, but
concentrates her mind on the possible responses made by Bob. Let us assume
that she knows some quantum mechanics, in particular the Born rule. Then she
may use this rule to calculate $E(B|A=+1)$ and $E(B^{\prime}|A=+1)$ from the
known settings $b$ and $b^{\prime}$, and assuming $P(A=-1)=P(A=+1)=1/2$ she
can calculate $E(AB)$ and $E(AB^{\prime})$, that is, the first and the third
term in (3). But there is no way in which she can get any information on the
second and fourth term. Thus Alice is not able to give any meaning to the left
hand side of the CHSH inequality, and this inequality might well be violated
if we only are allowed to take into account the information posessed by Alice.
Again she can have no opinion on the CHSH inequality.

This conclusion is of course the same if the setting chosen by Alice is
$a^{\prime}$. And a completely similar discussion can be made seen from Bob’s
point of view. The conclusion is that the simple reasoning leading to the CHSH
inequality can not be made meaningful to either of these observers at a stage
where they only know their own responses.

For practical experiments, not assuming quantum theory, one might perhaps
imagine some other probability model doing the job that Born’s formula did
above, but the problem with missing information about two terms on the
lefthand side of the CHSH inequality is the same.

Note that in spite of all this, Alice may well be very intelligent. Her
Hilbert space relevant also to some specific other context may be of the form
$\mathcal{H}\otimes\mathcal{K}$, where $\mathcal{H}$ is her two-dimensional
Hilbert space connected to the Bell experiment, and $\mathcal{K}$ is a fairly
big Hilbert space connected to the other context.

##  10 A general theory of a mind’s limitation

Take as a point of departure any experimental situation or decision situation
and an observer/ actor $O$ in this situation. $O$ will have in his mind
several conceptual variables connected to the situation. Assume that two of
these are maximally accessible. I will assume that the conditions of Theorem 1
hold.

According to Theorem 1 the situation can then be described by a Hilbert space
$\mathcal{H}$. In RefB1  the corresponding theorem was proved with an extra
condition connected to an epistemic process, but it was shown in Ref50  that
this extra condition is in fact unnecessary. This is an important observation.
It implies that the conclusion of the theorem applies to any decision
situation, and that the conceptual variables involved may be decision
variables or underlying variables that somehow influence decisions.

Another qualification is the condition (iii) in Theorem 1. If $\theta$ and
$\eta$ together satisfy this condition, I will say that they are _related_. It
$\theta$ and $\eta$ can not be related in this way, we say that they are
_essentially different_.

In this Section I will assume that the technical conditions behind Theorem 1
are satisfied. For simplicity assume first also that the Hilbert space has a
finite dimension $d$. Look at one of $O$’s maximally accessible variables
$\theta$, say taking the values $u_{1},...,u_{d}$. This has a unique operator
$A^{\theta}$ connected to it. First, it follows from Theorem 4.5 i RefB1  that
the eigenspaces of $A^{\theta}$ are one-dimensional, and it follows from
Theorem 4.4 in RefB1  that the eigenvalues of $A^{\theta}$ are just
$u_{1},...,u_{d}$. And each event $\theta=u_{i}$ corresponds to a unique unit
vector $|\psi^{i}\rangle$ in $\mathcal{H}$, the eigenvector of $A^{\theta}$
giving the eigenvalue $u_{i}$.

It is relevant to look at a theory of decisions as described by Yukolov and
Sornette Ref35 . This reference is only one of a series of papers written on
Quantum Decision Theory by the same authors. Similar conclusions may be made
by taking as points of departure cognitive models, as developed in Ref21  and
Ref23 .

In agreement with Ref35  and RefB1  let us assume for simplicity that the
current state of mind of $O$ is given by a ket vector $|\psi\rangle$ in
$\mathcal{H}$. He is going to make a decision, and his possible prospects
$\pi_{j}$ are each represented by ket vectors $|\pi_{j}\rangle$ in
$\mathcal{H}$. Then, according to the Born rule, his probability of making
decision $\pi_{j}$ is given by

$p(\pi_{j})=|\langle\psi|\pi_{j}\rangle|^{2}.$ (7)

More generally, if his current state is given by a density matrix $\rho$, the
probability of making decision $\pi_{j}$ is given by
$p(\pi_{j})=\langle\pi_{j}|\rho\pi_{j}\rangle$.

The concept of permissibility is defined in the Appendix.

Theorem 2 Assume that the individual $O$ has two related maximally accesible
variables $\theta$ and $\eta$ in his mind. Then $\eta(\phi)=\theta(k\phi)$ for
an inaccessible variable $\phi$ and a transformation $k$ of $\Omega_{\phi}$.
Assume that a group $K$ of transformations of $\Omega_{\phi}$ can be found
such that $k\in K$ and $\theta(\cdot)$ is permissible with respect to $K$.

In this situation $O$ can not simultaneously have in mind any other maximally
accessible variable which is related to $\theta$, but essentially different
from $\eta$.

Proof. According to Theorem 1, two different related maximally accessible
variables, say $\theta$ and $\eta$ in the mind of $O$ will determine his
Hilbert space $\mathcal{H}$ in the given context. And from this, all other
conceptual variables in his mind will be associated with selfadjoint
operators. Assume that one of these, say $\eta^{\prime}$, essentially
different from $\eta$, is maximally accessible. Then a different alternative
theory could have been developed from $\theta$ and $\eta^{\prime}$, giving a
Hilbert space $\mathcal{H}^{\prime}$. We will show that this leads to a
contradiction.

Since $\theta$ and $\eta^{\prime}$ are related, there is a transformation
$k^{\prime}$ such that $\eta^{\prime}(\phi)=\theta(k^{\prime}\phi)$. Extend if
necessary $K$ to a group $K^{\prime}$ such that $k^{\prime}\in K^{\prime}$. We
need the following:

Lemma 1 In this situation $\theta(\cdot)$ is permissible with respect to the
group $K^{\prime}$.

Proof of Lemma 1. We have that $\theta(\phi_{1})=\theta(\phi_{2})$ implies
$\theta(k^{\prime}\phi_{1})=\theta(k^{\prime}\phi_{2})$, and further
$\theta(h\phi_{1})=\theta(h\phi_{2})$ for all $h\in K$. It follows from this
that $\theta(hk^{\prime}\phi_{1})=\theta(hk^{\prime}\phi_{2})$ and
$\theta(k^{\prime}h\phi_{1})=\theta(k^{\prime}h\phi_{2})$ for all $h\in K$. A
similar property holds for all group elements that can be written as a finite
product of $k^{\prime}$ and elements in $K$. But these products generate
$K^{\prime}$.

Proof of Theorem 2, continued. From Theorem A1 in the Appendix it follows that
$A^{\theta}=V(k)A^{\eta}V(k)^{\dagger}$ and
$A^{\theta}=V(k^{\prime})A^{\eta^{\prime}}V(k^{\prime})^{\dagger}$. But this
implies that
$A^{\eta}=V(k^{-1}k^{\prime})A^{\eta^{\prime}}V(k^{-1}k^{\prime})^{\dagger}$,
and using Theorem A1 again, we conclude that $\eta$ and $\eta^{\prime}$ are
related, which leads to a contradiction with the assumptions made.

∎

Theorem 2 also holds for continuous variables if one can assume that the
technical condition (ii) of Theorem 1 holds.

Go back to the previous section, the situation of the actor Alice. Her Hilbert
space is two-dimensional, and her current state $|\psi\rangle$ can be imagined
to be determined by the event of the type $A=+1$. Her Hilbert space may be
reconstructed by considering in addition one other binary variable $\eta$. As
discussed there, this variable may be either her other response $A^{\prime}$,
or one of Bob’s responses $B$ or $B^{\prime}$. There are also other
possibilities. But she is not able to, when making decisions, have other,
essentially different variables in her mind. To prove this from Theorem 2, we
concentrate on the case $A=\theta^{a}$ and $B=\theta^{b}$, where we need the
following:

Lemma 2 Let $K$ be the group of rotations of the spin vector in the plane
spanned by $a$ and $b$. Then both the components $\theta^{a}$ and $\theta^{b}$
are permissible with respect to $K$, and $\theta^{a}$ and $\theta^{b}$ are
related.

Proof. Permissibility follows essentially as in Proposition 2 in Ref50 .
$\theta^{a}$ can obviously be transfered into $\theta^{b}$ by a suitable
rotation in this plane.

∎

##  11 The data analysis made by Charlie

Assume that Alice and Bob meet after the experiment and share their
information on all the runs with a new observer Charlie. In particular, all
the settings, $a$ or $a^{\prime}$, respectively $b$ or $b^{\prime}$ are known,
and according to the conditionality principle, all expectations should be
calculated conditionally, given these settings. In concrete terms, Charlie has
the following data: Settings for Alice in each run, $x_{i}=a\ \mathrm{or}\
a^{\prime}$, Settings for Bob, $y_{i}=b\ \mathrm{or}\ b^{\prime}$, responses
$X_{i}$ for Alice ($A$ or $A^{\prime}$) and $Y_{i}$ for Bob ($B$ or
$B^{\prime}$), $i=1,...,n$. Charlie wants to do a statistical analysis, and by
the conditionality principle he will condition this analysis on all the
$x_{i}$’s and $y_{i}$’s. His analysis should be concentrated on the following
parameters, corresponding to the 4 parts of the data sets that he has
received:

$l_{1}=E(XY|x=a,y=b)=E(AB),$ (8)
$l_{2}=E(XY|x=a^{\prime},y=b)=E(A^{\prime}B),$ (9)
$l_{3}=E(XY|x=a,y=b^{\prime})=E(AB^{\prime}),$ (10)
$l_{4}=E(XY|x=a^{\prime},y=b^{\prime})=E(A^{\prime}B^{\prime}).$ (11)

Let us assume a knowledgeable observer Charlie, knowing both statistics and
some quantum theory. As any observer, his mind at a given moment can be
described by some unit vector in a Hilbert space, which must be taken as big
enough to be able to absorb the setting $a/a^{\prime}$ and $b/b^{\prime}$, and
the corresponding observations $X$ and $Y$, made by Alice and Bob. This can be
accomplished by a four-dimensional Hilbert space, concentrating on the four
possible conditional joint distributions of $X$ and $Y$. He should also
believe in no-signalling: $E(Z|a,b)=E(Z|a)$ for any variable $Z$ solely
connected to Alice.

Put in another way, in order that Charlie should be able to describe any pair
$(A=\theta^{a},B=\eta^{b})$ etc., his Hilbert space $\mathcal{H}^{\prime}$
must be four-dimensional. We will claim that his state when analysing the data
should be seen as an eigenstate of the operator in $\mathcal{H}^{\prime}$
corresponding to his conceptual variable
$\delta=\theta^{x}\eta^{x}+\theta^{y}\eta^{y}+\theta^{z}\eta^{z}$ the dot
product of the two inaccessible spin vectors, one belonging to Alice and one
to Bob. Even though these spin vectors are inaccessible, $\delta$ is
accessible to Charlie.

This can be seen as follows. As analysed in detail in Ref19 , the operator
corresponding to $\delta$ in $\mathcal{H}^{\prime}$ has two eigenvalues -3 and
-1, the single eigenvector corresponding to $\delta=-3$ is given by (1), while
the eigenspace corresponding to $\delta=-1$ is three-dimensional. Looking at
the definition of $\delta$, and the fact that $\theta^{x}$ and $\eta^{x}$ etc.
all take values $\pm 1$, the value $\delta=-3$ is only possible if
$\theta^{x}\eta^{x}=\theta^{y}\eta^{y}=\theta^{z}\eta^{z}=-1$, that is
$\eta^{x}=-\theta^{x}$ and so on, which implies $\eta^{a}=-\theta^{a}$ for all
fixed directions $a$. This is just a manification of the fact that Charlie
knows that the spin vectors associated with Alice and Bob are equal, but
opposite.

Let us assume that before doing any data analysis, Charlie tries to make a
probability model for the relevant variables. From the discussion of the
previous section, there is a limitation to how much Charlie is able to think
of at some given time when making this model. His maximally accessible
variables are the different pairs $\zeta=(\theta,\eta)$, where $\theta$ is
connected to Alice and $\eta$ is connected to Bob, and both $\theta$ and
$\eta$ are binary.

In particular, consider the four pairs $C=(A,B)$, $D=(A,B^{\prime})$,
$E=(A^{\prime},B)$ and $F=(A^{\prime},B^{\prime})$. Every pair corresponds to
one of the 4 parts of the data sets that he is going to analyse. If he thinks
hard, he can for instance put up a joint probability model for $C$ and $D$,
but this is the maximum of what he is able to do. These two variables are
related, so from these two maximally accessible variables he is, according to
Theorem 1, able to reconstruct a Hilbert space. Similarly, he is able to
construct a Hilbert space from $C$ and $E$; these are related. But the pairs
$D$ and $E$ have no relationship to each other; they are essentially
different. So we are in the situation of Theorem 2: Charlie is not able to
have in his mind all three pairs $C$, $D$ and $E$ when making decisions about
the experiment.

We have to verify that the conditions of Theorem 2 hold. Let $\theta=C=(A,B)$
and $\eta=D=(A,B^{\prime})$. Both are functions of $(\phi,\psi)$, where $\phi$
is the inaccessible spin vector for Alice’s particle, and $\psi$ is the
inaccessible spin vector for Bob’s particle. As was discussed above, in
Charlie’s context, each component of $\psi$ is opposite to the corresponding
component of $\phi$. This can be written $\psi=-\phi$, and thus both $C$ and
$D$ can be seen as functions of $\phi$. It follows then from Lemma 2 that the
vector $C=(A,B)$ is permissible with respect to one group, and the vector
$D=(A,B^{\prime})$ is permissible with respect to another group, both seen as
functions of $\phi$. The argument needed for finding a group with respect to
which both are permissible, goes essentially like the proof of Lemma 1.

From this, the conditions of Theorem 2 hold for $\theta=C$ and $\eta=D$. This
implies that Charlie is not simultaneously able to hold in his mind the
variable $E$, which is related to $C$, but essentially different from $D$.

Charlie is thus in particular not able to put up a joint probability model for
these 3 pairs. As a consequence, he is not able to put up a joint probability
model for the 4 binary variables $AB,AB^{\prime},A^{\prime}B^{\prime}$ and
$A^{\prime}B^{\prime}$.

Assume now, tentatively, that Charlie is able to put up a joint probability
model for his four variables $A,A^{\prime},B$ and $B^{\prime}$. Then he would
be able to deduce from this model also a joint probability model for the
variables $AB,AB^{\prime},A^{\prime}B$ and $A^{\prime}B^{\prime}$. Thus, from
what has just been said, this thesis is impossible. Charlie is not in any way
able to think of a joint probability model for his four basic variables. In
fact, he is not able to have in his mind all these four variables when making
decisions related to the experiment.

This has an important consequence: From this point of view, Charlie is simply
not able to follow the arguments leading to (2) and then to (3) when making
his decisions. Thus he can not then follow the arguments leading to the CHSH
inequality, and must see the validity of this inequality either as an
empirical question or a question that can be resolved by his knowledge about
quantum mechanics.

From his data, Charlie can compute natural estimates:
$\widehat{l_{1}}=\overline{AB}$, $\widehat{l_{2}}=\overline{A^{\prime}B}$,
$\widehat{l_{3}}=\overline{AB^{\prime}}$ and
$\widehat{l_{4}}=\overline{A^{\prime}B^{\prime}}$. Let us further assume that
the settings are such that, by the Born formula, which gives
$E(AB)=-\mathrm{cos}(a,b)$, the CHSH inequality is violated. (Again, one
choice, in some sense the optimal one, is $a\sim 0^{o}$, $a^{\prime}\sim
90^{o}$, $b\sim 225^{o}$ and $b^{\prime}\sim 135^{o}$.) Then, if the number
$n$ of runs is large enough, Charlie will find by using Born’s formula before
looking at his data, that with high probability from this:

$\overline{AB}+\overline{A^{\prime}B}+\overline{AB^{\prime}}-\overline{A^{\prime}B^{\prime}}>2.$
(12)

This may convince him that the CHSH inequality is _not_ valid, and he will be
surprised if his estimates do not satisfy (12). He will also predict that for
a future series of runs, if the number of runs is large enough and the
settings are as before, then (12) will hold with large probability.

Parts of this discussion does not depend on Charlie’s possible knowledge about
quantum theory, but it is all the time assumed that he is in a decision
situation which, as described in Ref21  can be connected to a finite number of
prospects.

In his modelling effort Charlie might be able to find separate numbers in
separate models for $l_{1}=E(AB)$, $l_{2}=E(A^{\prime}B)$,
$l_{3}=E(AB^{\prime})$ and $l_{4}=E(A^{\prime}B^{\prime})$. These numbers may
be compared either to the prediction made by quantum theory, or to the
estimates he found from the data from Alice and Bob. In either case, given
suitable settings $a,a^{\prime},b$ and $b^{\prime}$, he may be convinced that
the CHSH inequality may be violated.

Thus, by Bell’s theorem he believes that the assumption of local realism must
be violated. He may be convinced about the validity of Einstein’s relativity
theory, and from this he may deduce that the locality assumption should hold.
Hence his only option is to reject the universal assumption of realism.
Charlie may be convinced of the statement advocated in Ref25 : The only ‘real
things’ in physics are events, and any theory, any set of questions to be
answered, should be connected to the perception of events made by an actor or
by a group of communicating actors. As argued here, Charlie as an actor is
limited.

##  12 Charlie, Alice and others

Assume that Alice and Bob plan to make a new Bell experiment with $n$ runs
together. Assume also that Alice and Charlie meet and talk together after
Charlie has done a data analysis, but before any new series of runs. The issue
of their talk is the predicton (12) for the new experiment. Charlie may be
convinced about this prediction, but Alice is still unsure.

Another situation might be that Charlie chooses to market his prediction to
all people that he knows, telling them about his arguments behind this
prediction. This must be seen as a form of information processing. In a recent
paper Ref24  a model for joint decisions made by a group of people is based on
an extensive exchange of information. Such an analysis may then consider
possible future joint decisions made either by Charlie and Alice together or
by Charlie and his friends together.

I will not go into details here, but concentrate on the following: The
discussion between Alice and Charlie focuses on the single binary variable
$\zeta=\mathrm{sign}(\overline{AB}+\overline{A^{\prime}B}+\overline{AB^{\prime}}-\overline{A^{\prime}B^{\prime}}-2)$
for the new experiment. According to RefJ8 , Alice’s question to Charlie on
this variable may be seen as a measurement. If she should accept Charlie’s
answer, she will enter a new state partly given by $\zeta=+1$ for the relevant
set of settings $a,a^{\prime},b$ and $b^{\prime}$ in the future Bell
experiment. She will believe that an empirical version of the CHSH inequality
will be violated in the new experiment if $n$ is large enough. And she may be
convinced by Charlie’s arguments around local realism.

The discussion with other people may be more complicated. But if Charlie’s
arguments are strong enough, both theoretical arguments from quantum mechanics
and empirical results, most of his friends will probably enter a state partly
given by $\zeta=+1$: Thus they will with some part of their mind believe that
the CHSH inequality may be violated under suitable conditions. Hence in the
light of Bell’s theorem they will not be convinced that local realism, made
precise in some way, will always hold.

The setting for this last discussion must be such that the friends can
communicate with Charlie. This may be argued to imply that all the friends
have some relation to a four-dimensional Hilbertspace in connection to one run
of the Bell experiment, and that their mental state here then is given by (1),
that is, corresponding to $\delta=-3$. Then, by the above arguments, neither
of them will then have the possibility to, at the same time, have all the
variables $A,A^{\prime},B$ and $B^{\prime}$ in their mind, and thus they will
not be convinced of the argument leading to (2).

##  13 Discussion

We are all limited. Like Alice and Charlie neither of us can always answer all
questions, even simple ones that require a yes/no answer. This is of course
obvious, but one aspect of this may not be clear to everybody: Our mind is
limited by how many conceptual variables we can think of at the same time when
making a decision. Like Alice, we can sometimes seek answers from people
having more insight.

Decisions may be made by single a actor or by groups of communicating actors.
In some situations these decisions may be related to measurements that we are
about to make, and these measurements can be formulated by focused questions
to nature involving accessible conceptual variables. This is the point of
departure for the approach to quantum mechanics given in RefB1 .

Going back to Sect. 11, look at Charlie’s efforts to make a probability model
over his variables. In the language of statisticians RefB1 , these
probabilities may be called epistemic probabilities. He is able to make joint
probability models over pairs of variables, but not over all 4 variables. This
must mean that these epistemic models in general are different than ordinary
probability models. The fact that quantum probabilities behave differently
than ordinary probabilities is well known Ref26 .

##  14 Conclusions

The discussions around Bell’s theorem and the assumptions of local realism may
perhaps continue. In my view the paradoxes around this issue may be resolved
by considering an actor like Charlie discussed above. After analysing his Bell
experiment data, he is convinced that the assumption of local realism cannot
hold in general. He has two arguments for this: Empirical results and a belief
on the general validity of quantum mechanics. At the same time he is not able
to follow the simple arguments leading to the CHSH inequality. As I see it,
this may be said to be so because of his limitation: He is simply not able to
keep enough variables at the same time in his mind when making his decisions.

My arguments in this paper has partly rested on the epistemic process approach
towards quantum theory RefB1 . However, the arguments concentrated on the
actor Charlie also seem to have some universal validity. It must be concluded
that local realism is not longer a universally convincing position, given
these arguments. And this can be highlighted by focusing on the world as seen
by any specific actor or by a group of different, communicating, actors, all
being limited in the specific sense discussed in Sect. 12 and Sect. 13 above.

The process of making decisions may in certain situations be a difficult one.
In this paper I have argued that individual decisions are dependent on
conceptual variables in the mind of the person who makes the decisions. Other
decisions are made more rutinely, being determined by our upbringing and by
the social context that we live in. In some of our decisions, we are inspired
by other people. In our Western culture, religious faith or related, deep
issues, may alo play an important part. My own thoughts around this is given
in Ref81 .

In practice many decisions are made jointly by groups of people that
communicate. Also in the latter case a common philosophy and through this,
common conceptual variables play a role. Decisions may be initiated by persons
that the group look up to. In this world, really cruel and disastrous
decisions have also been made in this way. In a civilizest society such
decisions should be countered. In an idealized world, all decisions should be
made in a way that is rational and at the same time has a high ethical
standard. Also, in this ideal world, science in all its variants should lead
the way here.

###### Acknowledgements.

I am grateful to Chris Ennis and to Peter Morgan for discussions.

## References

  * (1) Nikulov, A.: Could ordinary quantum mechanics be just fine for all practical purposes? Quantum Stud.: Math. Found. 3, 41-55 (2016)
  * (2) Bell, J.S.: On the Einstein-Podolsky-Rosen paradox. Physics 1, 195-200 (1964)
  * (3) Clauser, J.F., Horne, M.A., Shimony, A. and Holt, R.A.: Proposed experiment to test local hidden-variables theories. Phys. Rev. Lett. 23, 880-884 (1969)
  * (4) Aspect, A., Grangier, P. and Roger, G.: Experimental realization of Einstein-Podolsky-Rosen-Bohm Gedankenexperiment: a new violation of Bell’s inequalities. Phys. Rev. Lett. 49. 91-94 (1982)
  * (5) Larsson, J.-Å.: Loopholes in Bell inequality tests of local realism. J. Phys. A: Math. Theor. 47, 424003 (2014)
  * (6) Giustina, M., Versteegh, M.A.M, Wengerowsky, S. et al. : Significant-loophole-free test of Bell’s theorem with entangled photons. Phys. Rev. Lett. 115, 250401 (2015)
  * (7) Shalm, L.K., Meyer-Scott, E. Christensen, B.G. Bierhorst, P. et al.: Strong loophole-free test of local realism. Phys. Rev. Lett. 115, 250402 (2015)
  * (8) Schmelzer, I.: EPR-Bell realism as a part of logic. arXiv: 1712.04334v2 [physics.gen-ph] (2018)
  * (9) Helland, I.S.: Epistemic Processes. A Basis for Statistics and Quantum Theory. Springer, Berlin. Revised edition (2021)
  * (10) Gill, R.D.: Statistics, causality and Bell’s theorem. Statistical Science 20 (4), 512-528 (2014)
  * (11) Gill, R.D., Weih, G., Zeilinger, A. and Zukowski, M.: Comment on ‘Exclusion of time theorem of Bell’ by K. Hess and W. Philipp. Europhys. Lett. 61, 282-283 (2003)
  * (12) Kupczynski, M.: Is the moon there if nobody looks: Bell inequalities and physical reality. Found. Phys. 8, 273 (2020)
  * (13) Lambare, J.P.: On the CHSH form of Bell’s inequalities. Found. Phys. 47, 321-326 (2017)
  * (14) Stone, M. H.: The Theory of Representations of Boolean Algebras. Transactions of the American Mathematical Society 40, 37-111 (1936)
  * (15) Fuchs, C.A.: QBism, the perimeter of Quantum Bayesianism. arXiv: 1003.5209 [quant-ph] (2010)
  * (16) Rovelli, C.: Relational quantum mechanics. Int. J, Theor. Phys. 35, 1637-1657 (1996)
  * (17) Zwirn, H.: The measurement problem: Decoherence and convivial solipsism. Found. Phys. 46, 635-667 (2016)
  * (18) Mermin, N. D.: What’s bad about this habit? Physics Today 62, 8-9 (2009)
  * (19) Helland, I.S.: On reconstructing parts of quantum theory from two related maximal conceptual variables. Intern. J. Theor. Phys. 61, 69 (2022a)
  * (20) Höhn, P.A.: Quantum theory from rules on information acquisition. Entropy 19 (3). 98 (2017)
  * (21) Höhn, P.A.: Toolbox for reconstructing quantum theory from rules on information acquisition. Quantum 1, 38 (2017)
  * (22) Höhn, P.A. and Wever, C.S.P.: Quantum theory from questions. Phys. Rev. A 95, 012102 (2017)
  * (23) Zwirn, H.: The measurement problem: Decoherence and Convivial Solipsism. Found. Phys. 46, 635-667 (2016)
  * (24) Schweder, T. and Hjort, N.L.: Confidence, Likelihood, Probability. Statistical Inference with Confidence Distributions. Cambridge University Press (2016)
  * (25) Cox, D.: Some problems connected with statistical inference. Annals of Statistics 29, 357-372 (1958)
  * (26) Helland, I.S.: Simple counterexamples against the conditionality principle. The American Statistician 49, 351-356 (1995)
  * (27) Yukalov, V.I. and Sornette D.: How brains make decisions. Springer Proceedings in Physics 150, 37-53 (2014)
  * (28) Busemeyer, J. R., & Bruza, P.: Quantum models of cognition and decision. Cambridge: Cambridge University Press (2012)
  * (29) Pothos, E. M., & Busemeyer, J. R.: Can quantum probability provide a new direction for cognitive modeling? With discussion. Behavioral and Brain Sciences, 36, 255–327 (2013)
  * (30) Pothos, E.M. & Busemeyer, J.R.: Quantum cognition. Ann. Rev. Psychol. 73, 749-778 (2022).
  * (31) Susskind, L. and Friedman, A: Quantum Mechanics. The Theoretical Minimum. Basic Books, New York (2014)
  * (32) Yukalov, V.I., Yukalova, E.P. and Sornette, D.: Information processing by networks of quantum decision makers. Physica A: Statistcal Mechanics and its Applications 492, 747-768 (2018)
  * (33) Khrennikov, A.: Ubiquitous quantum structure. Berlin: Springer (2010)
  * (34) Perelomov, A.: Generalized Coherent States and Their Applications. Springer, Berlin (1986)
  * (35) Helland, I.S.: Steps Towards a Unified Basis for Scientific Models and Methods. Singapore: World Scientific (2010)
  * (36) Helland, I.S.: Epistemological and ontological aspects of quantum theory. arXiv: 2112.10484 [quant-ph] (2022b)
  * (37) Helland, I.S.: On religious faith, Christianity, and the foundation of quantum mechanics. Europ. J. Theol. Philos. 2 (1), 10-17 (2022c)

Appendix. Operators and their properties.

In Ref50  I took as a point of departure Chapter 2 in Perelomov Ref60 , which
discusses coherent states for arbitrary Lie groups. Let $G$ be a transitive
group acting on the space $\Omega_{\theta}$ associated with some conceptual
variable $\theta$ and $U(g)$ its unitary irreducible representation acting on
the Hilbert space $\mathcal{H}$. I will assume that $G$ has a trivial isotropy
group, so that the elements $g$ of $G$ are in one-to-one correspondence with
the values of $\theta$.

As in Ref50  (and in RefB1 ) take a fixed vector $|\theta_{0}\rangle$ in
$\mathcal{H}$, and consider the set $\\{|\theta\rangle\\}$, where
$|\theta\rangle=U(g)|\theta_{0}\rangle$ with $g$ corresponding to the value
$\theta$. It is not difficult to see that two vectors $|\theta^{1}\rangle$ and
$|\theta^{2}\rangle$ correspond to the same state, i.e., differ by a phase
factor ($|\theta^{1}\rangle=\mathrm{exp}(i\alpha)|\theta^{2}\rangle$,
$|\mathrm{exp}(i\alpha)|=1$), only if
$U(g^{2,-1}g^{1})|\theta_{0}\rangle=\mathrm{exp}(i\alpha)|\theta_{0}\rangle$,
where $g^{1}$ corresponds to $\theta^{1}$ and $g^{2}$ corresponds to
$\theta^{2}$. Suppose $E=\\{e\\}$ is a subgroup of the group $G$, such that
its elements have the property

$U(e)|\theta_{0}\rangle=\mathrm{exp}[i\alpha(e)]|\theta_{0}\rangle.$ (13)

When the subgroup $E$ is maximal, it will be called the isotropy subgroup for
the state $|\theta_{0}\rangle$. More precisely, it is the isotropy subgroup of
the group $U(G)$ corresponding to this state.

The construction shows that the vectors $|\theta\rangle$ corresponding to a
value $\theta$ and thus to an element $g\in G$, for all the group elements $g$
belonging to a left coset class of $G$ with respect to the subgroup $E$,
differ only in a phase factor and so determine the same state. Choosing a
representative $g(x)$ in any equivalence class $x$, one gets a set of states
$\\{|\theta_{g(x)}\rangle\\}$, where $x\in X=G/E$. Again, using the
correspondence between $g$ and $\theta$, I will write these states as
$\\{|\theta(x)\rangle\\}$, or in a more concise form $\\{|x\rangle\\}$,
$|x\rangle\in\mathcal{H}$.

Definition A1 The system of states
$\\{|\theta\rangle=U(g)|\theta_{0}\rangle\\}$, where $g$ corresponds to
$\theta$ as above, is called the coherent-state system
$\\{U,|\theta_{0}\rangle\\}$. Let $E$ be the isotropy subgroup for the state
$|\theta_{0}\rangle$. Then the coherent state $|\theta(g)\rangle$ is
determined by a point $x=x(g)$ in the coset space $G/E$ corresponding to $g$
and to $|\theta(g)\rangle$ is defined by
$|\theta(g)\rangle=\mathrm{exp}(i\alpha)|x\rangle$,
$|\theta_{0}\rangle=|0\rangle$.

Remark. The states corresponding to the vector $|x\rangle$ may also be
considered as a one-dimensional subspace in $\mathcal{H}$, or as a projector
$\Pi_{x}=|x\rangle\langle x|$, $\mathrm{dim}\Pi_{x}=1$, in $\mathcal{H}$. Thus
the system of coherent states, as defined above, determines a set of one-
dimensional subspaces in $\mathcal{H}$, parametrized by points of the
homogeneous space $X=G/E$.

In Ref50  this theory is generalized to the case with two maximally accessible
conceptual variables $\theta$ and $\eta$ connected through an underlying
inaccessible variable $\phi$ by a transformation $k$ of the underlying space
$\Omega_{\phi}$ by $\eta(\phi)=\theta(k\phi)$. For this case define
$\psi=(\theta,\eta)$. It is shown in Ref50  that a group $N$ can be defined on
$\psi$ given by the group $G$ above and its isomorphic ‘copy’ $H$ acting on
$\eta$. Let $M$ be the isotropy subgroup of $N$ corresponding to $E$ above,
and define the coset $Z=N/M$. Then it is shown in Ref50  that there is an
irreducible representation $W(\cdot)$ acting on $N$, formed by the
representation $U(\cdot)$ of $G$ and the corresponding representation
$V(\cdot)$ of $H$. These are acting on the same Hilbert space $\mathcal{H}$,
and this implied states $|\psi(\phi)\rangle=W(n)|\psi_{0}\rangle$ constructed
as above, defined on the same Hilbert space, and that they satisfy a
resolution of the identity

$\int|\psi\rangle\langle\psi|\nu(d\psi)=I$ (14)

for a left-invariant measure $\nu$ on $\Psi=\\{\psi\\}$.

Furthermore, it is shown that $z=(x,y)$, where $x$ is an element of $X$ and
$y$ is an element of the corresponding coset $Y=H/F$. Here $F$ is the subgroup
of $H$ corresponding to the subgroup $E$ of $G$.

In fact, the situation here is completely symmetric between $(\theta,G,E,X)$
on the one side and $(\eta,H,F,Y)$ on the other side. Furthermore, the
transformations $N$ on $\psi=(\theta,\eta)$ are constructed from independent
transformations in $G$ on $\theta$ and transformations in $H$ on $\theta$.
Then it is reasonable to assume that the measure $\nu$ in (14) can be written
as $\nu(d\psi)=\rho(dx)\rho(dy)$ for some marginal measure $\rho$.

From this, the operators corresponding to $\theta$ and $\eta$ can be defined
by

$P(x)=\int_{Y}|\psi\rangle\langle\psi|\rho(dy),$ (15)

and

$A^{\theta}=\int_{X}\theta(x)P(x)\rho(dx).$ (16)

Similarly:

$Q(y)=\int_{X}|\psi\rangle\langle\psi|\rho(dx),$ (17)

and

$A^{\eta}=\int_{Y}\xi(y)Q(y)\rho(dy).$ (18)

Here, to recall, $y$ is defined as an element of the homogeneous space
$Y=H/F$, where $H$ is a transitive group, isomorphic to $G$, acting on the
space $\Omega_{\eta}$ on which $\eta$ varies, and $F$ is the subgroup of $H$
corresponding to $E$ of $G$.

For conceptual variables $\xi$ that are not maximally accessible, we can
always write $\xi=f(\theta)$ for some maximal $\theta$, and an operator for
$\xi$ can be found by using the spectral theorem on the operator $A^{\theta}$
(cp. equation (4.30) in RefB1 ).

To illustrate the theory, I will prove an important result, a correction/
precision of Theorem 4.2 in RefB1 . Then first I have to define the notion of
permissibility.

Definition A2 The function $\theta(\cdot)$ on a space $\Omega_{\phi}$ upon
which a group of transformations $K$ is defined, is said to be permissible if
the following holds: $\theta(\phi_{1})=\theta(\phi_{2})$ implies
$\theta(k\phi_{1})=\theta(k\phi_{2})$ for all $k\in K$.

This notion is studied thoroughly in Ref61 . The main conclusion is that if
$\theta(\cdot)$ is permissible, then there is a group $G$ acting on the image
space $\Omega_{\theta}$ such that $g(\theta(\phi))$ is defined as
$\theta(k\phi)$; $k\in K$. The mapping from $K$ to $G$ is an homomorphism. If
$K$ is transitive on $\Omega_{\phi}$, then $G$ is transitive on
$\Omega_{\theta}$. (Lemma 4.3 in RefB1 .) It is easy to show that $G$ has a
trivial isotropy group if $K$ has a trivial isotropy group.

Theorem A1 Assume that the function $\theta(\cdot)$ is permissible with
respect to a group $K$ acting on $\Omega_{\phi}$. Assume that $K$ is
transitive and has a trivial isotropy group. Let $T(\cdot)$ be a unitary
representation of $K$ such that the coherent states $T(k)|\psi_{0}\rangle$ are
in one-to-one correspondence with $k$. For any transformation $t\in K$ and any
such unitary representation $T$ of $K$, the operator
$T(t)^{\dagger}A^{\theta}T(t)$ is the operator corresponding to
$\theta^{\prime}$ defined by $\theta^{\prime}(\phi)=\theta(t\phi)$.

Proof. By (16) we have

$T(t^{-1})A^{\theta}T(t)=\int_{X}\theta(x)P_{t}(x)\rho(dx),$ (19)

where

$P_{t}(x)=\int_{Y}|\psi(t^{-1}\phi)\rangle\langle\psi(t^{-1}\phi)|\rho(dy).$
(20)

To show this, we need to prove that
$T(t^{-1})|\psi(\phi)\rangle=|\psi(t^{-1}\phi)\rangle$.

Note that $t^{-1}\phi$, permissibility and $\theta=\theta(\phi)$,
$\eta=\eta(\phi)$ induces from $t$ a new transformation $s$ acting on
$\psi=(\theta,\eta)$. Consider $s^{-1}(\theta,\eta)=s^{-1}(g,h)$ from the one-
to-one correspondence between $\theta$ and $g$ and between $\eta$ and $h$. Let
$S(s^{-1})=T(t^{-1})$ under this correspondence. Then
$T(t^{-1})|\psi(\phi)\rangle=S(s^{-1})|\psi\rangle$, where
$|\psi\rangle=T(k)|\psi_{0}\rangle$ is the vector i $\mathcal{H}$ which is in
one-to-one correspondence with the group element $k$, which generates $g=g(k)$
and hence $\theta$ by permissibility. Now the group $H$, which corresponds to
a copy of $G$, can also be seen as generated by $K$, so $h=h(k)$. This implies
that $(g,h)$ is a function of $k$, and by permissibility also
$\psi=(\theta,\eta)$ is a function of $k$. Since the two maximally accessible
variables $\theta$ and $\eta$ determine the Hilbert space, this function must
be one-to-one.

Write the group elements $s^{-1}(g,h)$ as $(g^{\prime},h^{\prime})$, new
members of groups $G$ and $H$ acting on $\theta$ and $\eta$, respectively. Let
again $E$ be the subgroup of $G$ constructed as in (13), and let $F$ be the
corrsponding subgroup of $H$. Write $X=G/E$ and $Y=H/F$. The new elements of
these cosets may be defined as $x^{\prime}=t^{-1}x$ and $y^{\prime}=t^{-1}y$,
respectively. This gives an element $z^{\prime}=(x^{\prime},y^{\prime})$, and
the corresponding state in $\mathcal{H}$ as $|z^{\prime}\rangle$, which as in
Definition A1 also, modulus a phase factor, can be written as
$|\psi(g^{\prime},h^{\prime})\rangle=|\psi(t^{-1}\phi)\rangle$.

Hence (19) and (20) can be written as

$T(t^{-1})A^{\theta}T(t)=\int_{X}\theta(x)P_{t}(x)\rho(dt^{-1}x),$ (21)

where

$P_{t}(x)=\int_{Y}|\psi(t^{-1}\phi)\rangle\langle\psi(t^{-1}\phi)|\rho(dt^{-1}y).$
(22)

Now make a change of variables from $(x,y)$ to
$(x^{\prime},y^{\prime})=(tx,ty)$ in these integrals. Since
$\nu(d\psi)=\rho(dx)\rho(dy)$ is left invariant, the corresponding $\rho$ may
be taken to be left invariant. Therefore the last integral may be written

$P_{t}(x)=\int_{Y}|\psi(\theta(t^{-1}\phi),\eta(\phi^{\prime})\rangle\langle\psi(\theta(t^{-1}\phi),\eta(\phi^{\prime})|\rho(dy^{\prime}),$
(23)

and we can write $P_{t}(x)=P(t^{-1}x)$. This is inserted into (21), and using
left-invariance of the measure again, this gives that the operator
$T(t^{-1})A^{\theta}T(t)$ is associated with the conceptual variable
$\theta(tx)$, which also may be written as $\theta(t\phi)$. ∎

By using this result in the same way as Theorem 4.2 is used in RefB1 , a rich
theory follows. I will limit me here to the case where $\theta$ is a discrete
conceptual variable. Then one can show:

1) The eigenvalues of $A^{\theta}$ coincide with the values of $\theta$.

2) The variable $\theta$ is maximally accessible if and only if the
eigenvalues of $A^{\theta}$ are non-degenerate.

3) For the maximal case the following holds in a given context: a) For a fixed
$\theta$ each question ‘What is the value of $\theta$?’ together with a sharp
answer ‘$\theta=u$’ can be associated with a normalized eigenvector of the
corresponding $A^{\theta}$. b) If there in the context are a set
$\\{\theta^{a};a\in\mathcal{A}\\}$ of maximally accessible conceptual
variables (these must by the results of Section 8 be related to each other)
one can consider all ket vectors that are normalized eigenvectors of some
operator $A^{\theta^{a}}$. Then each of these may be associated with a unique
question-and-answer as above.