# Designing with Non-Finite Output Dimension
via Fourier Coefficients of Neural Waveforms

Jonathan S. Kent
University of Illinois
<EMAIL_ADDRESS>
###### Abstract

> Ordinary Deep Learning models require having the dimension of their outputs
> determined by a human practitioner prior to training and operation. For
> design tasks, this places a hard limit on the maximum complexity of any
> designs produced by a neural network, which is disadvantageous if a greater
> allowance for complexity would result in better designs.
>
> In this paper, we introduce a methodology for taking outputs of non-finite
> dimension from neural networks, by learning a “neural waveform,” and then
> taking as outputs the coefficients of its Fourier series representation. We
> then present experimental evidence that neural networks can learn in this
> setting on a toy problem.

## Introduction

It is taken as read that Deep Learning and neural networks possess incredible
power to solve problems that are otherwise intractable. Recent advances have
lead to them being used in chip design (?), vehicle design (?), and
manufacturing (?). But in certain cases, their capabilities remain limited by
their architectures. Among these limitations, as will be addressed in this
paper, is that neural networks are designed with a finite number of outputs.
Given a choice from 1 to 9, a neural network can never choose 10, even if that
might be optimal, for example as a number of batteries or axes of motion.

Despite an enormous amount of effort in automatic neural network architecture
optimization (?; ?; ?; ?; ?), certain traits of these networks still need to
be pre-determined. And yet, the actual number of hyperparameters necessary to
specify by hand has been decreasing. It is now possible to automatically learn
the width of kernels in CNNs (?; ?; ?) and the effective $\Delta t$ in neural
ODEs (?). It is also possible, over time, to programatically adjust network
depth (?), and most famously the effective learning rate in gradient descent
(?).

Additionally, it is possible to use RNNs to output sequences of a length
determined by the model itself (?; ?), meaning that the output dimension of
the network is learned over time. However, optimizing an RNN for the later
dimensions of the output space would require significantly more computation,
as well as extra requirements for modeling long-term dependencies, a classic
weakness of recurrent architectures.

Attempts to allow for models to operate in an infinite-dimensional space have
included the use of Reproducing Kernel Hilbert Spaces (?) and quantum
computation (?). This means that these approaches are poorly suited to design
tasks for which ordinary neural networks are entirely appropriate. In this
paper, we will introduce an approach using neural networks as they exist
currently, fully capable of being accelerated by modern frameworks, for taking
a non-finite number of dimensions as the output of a learned function,
enabling models to make design decisions that were not thought of by their
human operators.

## Method

Figure 1: The proposed methodology; turning an input $x$ into a “neural
waveform” $s_{x}$, and takings its Fourier coefficients $a_{x\omega}$.

This method consists of two components: generating a neural waveform, and
calculating its Fourier coefficients.

### Neural Waveform

What we’re calling a “neural waveform” is a periodic function $s_{x}$, which
is itself the output of a neural network $\mathcal{S}$, given by
$s_{x}(t)=\mathcal{S}(\theta;x,t)$. Here, $\theta$ is the vector of learned
parameters for the network, $x$ is the model input, and $t$ is an analogue for
time. 111This is confusing notation, as $\theta,\ x,$ and $t$ all have
multiple, overlapping traditional meanings between the contexts of Machine
Learning and Harmonic Analysis. However, it has been chosen in an attempt to
maximize the over-all legibility of this manuscript. Computationally, $s_{x}$
takes the form of a set of time-value pairs, sampled using the following
method.

Over the interval $[-\pi,\pi]$, and with an appropriately large integer $N$,
we get a time-step $\Delta t=\frac{2\pi}{N}$, and from there
$t_{n}=-\pi+n\Delta t$, for the integers $0\leq n\leq N\in\mathbb{Z}$. We can
now compute $s_{x}=\\{(t_{n},\mathcal{S}(\theta;x,t_{n})|0\leq n\leq
N\in\mathbb{Z}\\}$, written in a functional form as $s_{x}(t)=v|(t,v)\in
s_{x}$. As an additional note, in order to ensure that $s_{x}$ is periodic,
i.e. that $s_{x}(t)=s_{x}(t+2n\pi)$, during implementation, $\mathcal{S}$ may
instead take $\sin(t)$ and $\cos(t)$ as a pair of inputs, rather than $t$
itself.

### Fourier Coefficients

For a given positive integer angular frequency $\omega$, the Fourier cosine
coefficient $a_{x\omega}$ of the waveform $s_{x}$ is given by

$a_{x\omega}=\frac{1}{\pi}\int_{-\pi}^{\pi}s_{x}(t)\cdot\cos(\omega t)\partial
t$

and the sine coefficient $b_{x\omega}$ by

$b_{x\omega}=\frac{1}{\pi}\int_{-\pi}^{\pi}s_{x}(t)\cdot\sin(\omega t)\partial
t$

(?). Because an integral is equal to its mean value times its width,
calculating these coefficients is computationally easy. For example,

$a_{x\omega}\ \ $ = | $\frac{1}{\pi}\int_{-\pi}^{\pi}s_{x}(t)\cdot\cos(\omega t)\partial t$
---|---
= | $\frac{1}{\pi}\sum_{n=0}^{N}s_{x}(t_{n})\cdot\cos(\omega t_{n})\Delta t$
= | $\frac{2}{N}\sum_{n=0}^{N}s_{x}(t_{n})\cdot\cos(\omega t_{n})$

with a similar procedure for $b_{x\omega}$. These Fourier coefficients are
then taken as the outputs of $\mathcal{S}(\theta;x,\cdot)$ for the input $x$.
However, because the formulae for $a_{x\omega}$ and $b_{x\omega}$ are valid
for any number values of $\omega$, the model $\mathcal{S}$ can be queried
along any number of output dimensions, regardless of the limitations of its
architecture.

## Experiments

A toy problem was created to test this kind of model architecture.
Specifically, whether or not it can learn to produce waveforms satisfying the
conditions placed on them by a loss function that uses the Fourier
coefficients as model outputs. This toy problem was simple, given an input $x$
as an integer, produce a waveform $s_{x}$ such that $a_{x\omega}=1$ for
$x=\omega$ and $a_{x\omega}=0$ for $x\neq\omega$, measured using Mean Squared
Error, and sampling both $x$ and $\omega$ from $0\leq x,\omega\leq 15$.

A complete implementation of this experiment, including the configuration and
hyperparameters, will be included in a Colaboratory notebook. See Figure 2 for
the waveforms output by a trained model given $x\in[0,4]$, and Figure 3 for
those given $x\in[11,15]$. It is clear that the model is absolutely capable of
learning to produce waveforms of some kind. But Figure 4 makes it clear that
these waveforms satisfy the conditions placed on them nearly flawlessly: where
$x=\omega,\ a_{x\omega}\approx 1$, and where $x\neq\omega,\ a_{x\omega}\approx
0$.

Figure 2: Neural waveforms on the toy problem, given $x\in[0,4]$. Figure 3:
Neural waveforms on the toy problem, given $x\in[11,15]$. Figure 4: Fourier
coefficients on the toy problem, for all inputs and frequencies. Error from
the identity matrix is not great enough to appear visually.

## Analyses, Conclusions, and Future Work

This methodology more or less involves taking the inner products between
$s_{x}(t)$ and sinusoidal functions $\sin(\omega t)$ and $\cos(\omega t)$.
Attention mechanisms in Transformers and the like (?) involves taking the
inner products between the outputs of attention heads and hidden states. As a
result, this method is analogous to taking attention weights, where what is
being attended to are sine waves. This provides an inroads to applying more
results from the work on Attention mechanisms to the problem of taking non-
finite outputs from neural networks.

This method, as it stands, looks to present an interesting capability, which
may see use in AI/ML-aided design programs. This will require that it be
coupled with some adaptive mechanism to check particular regions of
frequencies, in order to sample the frequencies the model is attempting to
output in with a finite amount of compute.

Future work will, of course, involve integration into Computer-Aided Design
programs and particular domain areas, like circuit and mechanical design, as
well as significant training and testing experimentation on both the neural
waveform method, and the coupled adaptive frequency selection mechanism.

## References

  * [Carvalho, Ramos, and Chaves 2011] Carvalho, A. R.; Ramos, F. M.; and Chaves, A. A.  2011\.  Metaheuristics for the feedforward artificial neural network (ann) architecture optimization problem.  Neural Computing and Applications 20(8):1273–1284.
  * [Chang et al. 2017] Chang, B.; Meng, L.; Haber, E.; Tung, F.; and Begert, D.  2017\.  Multi-level residual networks from dynamical systems view.  arXiv preprint arXiv:1710.10348.
  * [Dai et al. 2017] Dai, J.; Qi, H.; Xiong, Y.; Li, Y.; Zhang, G.; Hu, H.; and Wei, Y.  2017\.  Deformable convolutional networks.  In Proceedings of the IEEE international conference on computer vision, 764–773.
  * [Dorf and Tallarida 2018] Dorf, R. C., and Tallarida, R. J.  2018\.  Pocket book of electrical engineering formulas.  CRC Press.
  * [Hasani et al. 2020] Hasani, R.; Lechner, M.; Amini, A.; Rus, D.; and Grosu, R.  2020\.  Liquid time-constant networks.  arXiv preprint arXiv:2006.04439.
  * [Idrissi et al. 2016] Idrissi, M. A. J.; Ramchoun, H.; Ghanou, Y.; and Ettaouil, M.  2016\.  Genetic algorithm for neural network architecture optimization.  In 2016 3rd International Conference on Logistics Operations Management (GOL), 1–4.  IEEE.
  * [Khailany et al. 2020] Khailany, B.; Ren, H.; Dai, S.; Godil, S.; Keller, B.; Kirby, R.; Klinefelter, A.; Venkatesan, R.; Zhang, Y.; Catanzaro, B.; et al.  2020\.  Accelerating chip design with machine learning.  IEEE Micro 40(6):23–32.
  * [Kingma and Ba 2014] Kingma, D. P., and Ba, J.  2014\.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980.
  * [Kužnar et al. 2012] Kužnar, D.; Možina, M.; Giordanino, M.; and Bratko, I.  2012\.  Improving vehicle aeroacoustics using machine learning.  Engineering Applications of Artificial Intelligence 25(5):1053–1061.
  * [Laforgue et al. 2020] Laforgue, P.; Lambert, A.; Brogat-Motte, L.; and d’Alché Buc, F.  2020\.  Duality in rkhss with infinite dimensional outputs: Application to robust losses.  In International Conference on Machine Learning, 5598–5607.  PMLR.
  * [Lau et al. 2017] Lau, H.-K.; Pooser, R.; Siopsis, G.; and Weedbrook, C.  2017\.  Quantum machine learning over infinite dimensions.  Physical review letters 118(8):080501.
  * [Luo et al. 2018] Luo, R.; Tian, F.; Qin, T.; Chen, E.; and Liu, T.-Y.  2018\.  Neural architecture optimization.  arXiv preprint arXiv:1808.07233.
  * [Miikkulainen et al. 2019] Miikkulainen, R.; Liang, J.; Meyerson, E.; Rawal, A.; Fink, D.; Francon, O.; Raju, B.; Shahrzad, H.; Navruzyan, A.; Duffy, N.; et al.  2019\.  Evolving deep neural networks.  In Artificial intelligence in the age of neural networks and brain computing. Elsevier.  293–312.
  * [Mikolov et al. 2010] Mikolov, T.; Karafiát, M.; Burget, L.; Cernockỳ, J.; and Khudanpur, S.  2010\.  Recurrent neural network based language model.  In Interspeech, volume 2, 1045–1048.  Makuhari.
  * [Pintea et al. 2021] Pintea, S. L.; Tomen, N.; Goes, S. F.; Loog, M.; and van Gemert, J. C.  2021\.  Resolution learning in deep convolutional networks using scale-space theory.  arXiv preprint arXiv:2106.03412.
  * [Ramchoun et al. 2016] Ramchoun, H.; Idrissi, M. A. J.; Ghanou, Y.; and Ettaouil, M.  2016\.  Multilayer perceptron: Architecture optimization and training.  Int. J. Interact. Multim. Artif. Intell. 4(1):26–30.
  * [Romero et al. 2021] Romero, D. W.; Bruintjes, R.-J.; Tomczak, J. M.; Bekkers, E. J.; Hoogendoorn, M.; and van Gemert, J. C.  2021\.  Flexconv: Continuous kernel convolutions with differentiable kernel sizes.  arXiv preprint arXiv:2110.08059.
  * [Sundermeyer, Schlüter, and Ney 2012] Sundermeyer, M.; Schlüter, R.; and Ney, H.  2012\.  Lstm neural networks for language modeling.  In Thirteenth annual conference of the international speech communication association.
  * [Vaswani et al. 2017] Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I.  2017\.  Attention is all you need.  In Advances in neural information processing systems, 5998–6008.
  * [Wuest et al. 2016] Wuest, T.; Weimer, D.; Irgens, C.; and Thoben, K.-D.  2016\.  Machine learning in manufacturing: advantages, challenges, and applications.  Production & Manufacturing Research 4(1):23–45.


# TransRUPNet for Improved Out-of-Distribution Generalization in Polyp
Segmentation

Debesh Jha, Nikhil Kumar Tomar, Ulas Bagci  Machine and Hybrid Intelligence
Lab, Department of Radiology, Northwestern University, Chicago, USA

###### Abstract

Out-of-distribution (OOD) generalization is a critical challenge in deep
learning. It is specifically important when the test samples are drawn from a
different distribution than the training data. We develop a novel real-time
deep learning based architecture, TransRUPNet that is based on a Transformer
and residual upsampling network for colorectal polyp segmentation to improve
OOD generalization. The proposed architecture, TransRUPNet, is an encoder-
decoder network that consists of three encoder blocks, three decoder blocks,
and some additional upsampling blocks at the end of the network. With the
image size of $256\times 256$, the proposed method achieves an excellent real-
time operation speed of 47.07 frames per second with an average mean dice
coefficient score of 0.7786 and mean Intersection over Union of 0.7210 on the
out-of-distribution polyp datasets. The results on the publicly available
PolypGen dataset (OOD dataset in our case) suggest that TransRUPNet can give
real-time feedback while retaining high accuracy for in-distribution dataset.
Furthermore, we demonstrate the generalizability of the proposed method by
showing that it significantly improves performance on OOD datasets compared to
the existing methods.

###### Index Terms:

Computer aided diagnosis, out-of-distribution polyp segmentation, transformer,
colonoscopy, residual network

##  I Introduction

Colonoscopy is widely considered as the gold standard for the diagnosis of
colon cancer. Early detection of polyps is important as even a small increase
in adenoma detection rate can significantly decrease interval colorectal
cancer incidence [1]. Studies suggest a polyps miss rate of around 22-28% [2].
There are several reasons for polyp miss-rate in colonoscopy, for example, the
skill of endoscopists, bowel preparation quality, fast withdrawal time,
visibility, and differences in polyps characteristics. Deep learning-based
algorithms have emerged as a promising approach to improve diagnostic
performance by highlighting the presence of precancerous tissue in the colon
and reducing the clinical burden.

OOD detection and generalization is essential towards developing computer-
aided diagnostic support systems in colonoscopy. It is critical to ensure the
reliability and safety of deep learning systems. In this study, we introduce
the novel deep learning architecture, TransRUPNet, to address the critical
need for clinical integration of polyp segmentation routine, which is real-
time time and retains high accuracy. Most existing deep learning models are
trained based on closed-world assumption, where the test dataset is assumed to
be drawn from the same distribution as the training data, known as in-
distribution (ID). However, the models are deployed in an open-world scenario,
test samples can be out-of-distribution (OOD) and therefore should be handled
in caution. The distributional shifts can be caused by semantic shift (e.g.,
OOD samples are drawn from different classes), or covariate shift (e.g., OOD
samples from a different domain). The detection of semantic distribution shift
(e.g., due to the occurrence of new classes) is the focal point of OOD
detection tasks, where the label space can be different between ID and OOD
data and hence the model should not make any prediction. In addition to OOD
detection and generalization, several problems adopt the “open-world”
assumption and have a similar goal of identifying OOD examples for developing
robust systems. The main contribution are as follows:

  1. 1.

We propose TransRUPNet, an encoder-decoder architecture specifically designed
for accurate, real-time and improved out-of-distribution polyp segmentation.

  2. 2.

We compared the performance of TransRUPNet with the existing state-of-the-art
(SOTA) methods in four different polyp datasets (one in-distribution and three
OOD datasets) to show the method’s superiority.

  3. 3.

Our proposed architecture has strong generalization performance when compared
with eight SOTA methods.

Figure 1: Overall architecture of the TransRUPNet

##  II Related Work

Recently, there has been a significant advancement in the development of
models for polyp segmentation. While U-Net based architectures have been
widely used, several other approaches have also been proposed that focus on
capturing boundary details and leveraging the camouflage property of polyps.
One such architecture is PraNet [3], which incorporates reverse attention
modules to incorporate boundary cues. It combines a global feature map
obtained using a parallel partial decoder. Another approach proposed by [4]
introduces a boundary constraint network that utilizes a bilateral boundary
extraction module to analyze polyp and non-polyp regions. Polyp-PVT [5] takes
a different approach by introducing a camouflage identification module with a
pyramid vision transformer (PVT) encoder. This module aims to capture polyp
cues that are concealed in low-level features. The success of transformer-
based approaches in polyp segmentation has led to the development of more
similar works in the field. ColonFormer [6] proposes a hierarchical
transformer combined with a hierarchical pyramid network, incorporating a
residual axial attention module for efficient polyp segmentation. Overall,
these research demonstrates a wide range of architectural variations and
techniques used for polyp segmentation, inspiring further research for the
computer aided diagnosis system for colon polyp segmentation.

##  III Method

Figure 1 shows the block diagram of the proposed TransRUPNet architecture. It
is an encoder-decoder network that begins with a Pyramid Vision Transformer
(PVT) [7] as a pre-trained encoder. We extract three different feature maps
from the encoder and pass them through a series of $1\times 1$ Conv, Batch
Normalization and ReLU activation for reducing the number of feature channels
to $64$. The reduced feature maps are then passed to the up block and the
decoder blocks. Within the up block, the input feature map is first passed
through a bilinear upsampling to upscale the feature map’s height and width to
that of the original input image. Next, the upsampled feature map is passed
through a residual block to learn a more robust representation.

In Figure 2, each component of TransRUPNet is illustrated. The decoder block
also begins with a bilinear upsampling layer to increase the spatial
dimensions by a factor of $2$ and then concatenated with the reduced feature
from the encoder. Next, the concatenated feature map is passed through a
residual block to learn more robust semantic features which help to generate a
fine-quality segmentation mask. The output from the first decoder block is
passed to the next decoder block, which is further followed by an up block. We
concatenate the output from all four up blocks into a single feature
representation. After that, the concatenated feature map is followed by a
residual block, $1\times 1$ convolution and a sigmoid activation to generate
the final segmentation mask.

Figure 2: Components of the TransRUPNet TABLE I: Quantitative results on the Kvasir-SEG test dataset. Method | Backbone | mIoU | mDSC | Recall | Precision | F2 | FPS
---|---|---|---|---|---|---|---
U-Net [8] | - | 0.7472 | 0.8264 | 0.8504 | 0.8703 | 0.8353 | 106.88
U-Net++ [9] | - | 0.7420 | 0.8228 | 0.8437 | 0.8607 | 0.8295 | 81.34
ResU-Net++ [10] | - | 0.5341 | 0.6453 | 0.6964 | 0.7080 | 0.6576 | 43.11
HarDNet-MSEG [11] | HardNet68 | 0.7459 | 0.8260 | 0.8485 | 0.8652 | 0.8358 | 34.80
ColonSegNet [12] | - | 0.6980 | 0.7920 | 0.8193 | 0.8432 | 0.7999 | 73.95
DeepLabV3+ [13] | ResNet50 | 0.8172 | 0.8837 | 0.9014 | 0.9028 | 0.8904 | 67.88
PraNet [3] | Res2Net | 0.8296 | 0.8942 | 0.9060 | 0.9126 | 0.8976 | 31.89
TGANet [14] | ResNet50 | 0.8330 | 0.8982 | 0.9132 | 0.9123 | 0.9029 | 36.58
TransRUPNet (Ours) | PVT | 0.8445 | 0.9005 | 0.9195 | 0.9170 | 0.9048 | 47.07
TABLE II: Quantitative results on the Kvasir-SEG test dataset. Method | Backbone | mIoU | mDSC | Recall | Precision | F2
---|---|---|---|---|---|---
Training dataset: Kvasir-SEG – Test dataset: PolypGen (C6)
U-Net [8] | - | 0.5384 | 0.6126 | 0.7054 | 0.7508 | 0.6362
U-Net++ [9] | - | 0.5355 | 0.6163 | 0.7340 | 0.7230 | 0.6564
ResU-Net++ [10] | - | 0.2816 | 0.3684 | 0.6220 | 0.3526 | 0.4326
HarDNet-MSEG [11] | HardNet68 | 0.5548 | 0.6341 | 0.7197 | 0.7722 | 0.6487
ColonSegNet [12] | - | 0.4410 | 0.5290 | 0.6199 | 0.6403 | 0.5424
DeepLabV3+ [13] | ResNet50 | 0.7031 | 0.7629 | 0.7773 | 0.8693 | 0.7674
PraNet [3] | Res2Net | 0.6691 | 0.7307 | 0.7612 | 0.8755 | 0.7378
TGANet | ResNet50 | 0.6750 | 0.7382 | 0.7692 | 0.8887 | 0.7391
TransRUPNet (Ours) | PVT | 0.7210 | 0.7786 | 0.8522 | 0.8175 | 0.7929
Training dataset: Kvasir-SEG – Test dataset: CVC-ClinicDB
U-Net [8] | - | 0.5433 | 0.6336 | 0.6982 | 0.7891 | 0.6563
U-Net++ [9] | - | 0.5475 | 0.6350 | 0.6933 | 0.7967 | 0.6556
ResU-Net++ [10] | - | 0.3585 | 0.4642 | 0.5880 | 0.5770 | 0.5084
HarDNet-MSEG [11] | HardNet68 | 0.6058 | 0.6960 | 0.7173 | 0.8528 | 0.7010
ColonSegNet [12] | - | 0.5090 | 0.6126 | 0.6564 | 0.7521 | 0.6246
DeepLabV3+ [13] | ResNet50 | 0.7388 | 0.8142 | 0.8331 | 0.8735 | 0.8198
PraNet [3] | Res2Net | 0.7286 | 0.8046 | 0.8188 | 0.8968 | 0.8077
TGANet | ResNet50 | 0.7444 | 0.8196 | 0.8290 | 0.8879 | 0.8207
TransRUPNet (Ours) | PVT | 0.7765 | 0.8539 | 0.8736 | 0.8870 | 0.8590
Training dataset: Kvasir-SEG – Test dataset: BKAI-IGH
U-Net [8] | - | 0.5686 | 0.6347 | 0.6986 | 0.7882 | 0.6591
U-Net++ [9] | - | 0.5592 | 0.6269 | 0.6900 | 0.7968 | 0.6493
ResU-Net++ [10] | - | 0.3204 | 0.4166 | 0.6979 | 0.3922 | 0.5019
HarDNet-MSEG [11] | HardNet68 | 0.5711 | 0.6502 | 0.7420 | 0.7469 | 0.6830
ColonSegNet [12] | - | 0.4910 | 0.5765 | 0.7191 | 0.6644 | 0.6225
DeepLabV3+ [13] | ResNet50 | 0.6589 | 0.7286 | 0.7919 | 0.8123 | 0.7493
PraNet [3] | Res2Net | 0.6609 | 0.7298 | 0.8007 | 0.8240 | 0.7484
TGANet | ResNet50 | 0.6612 | 0.7289 | 0.7740 | 0.8184 | 0.7412
TransRUPNet (Ours) | PVT | 0.7218 | 0.7945 | 0.8497 | 0.8337 | 0.8072
Figure 3: Qualitative example showing polyp segmentation

##  IV Experiment

###  IV-A Dataset

We use four publicly available colonoscopy polyp segmentation datasets. We
consider Kvasir-SEG [15] as the in-distribution dataset and other datasets
such as PolypGen [16], BKAI-IGH [17], and CVC-ClinicDB [18] as the OOD
dataset. PolypGen dataset is collected from 6 medical centers from Norway,
Italy, France, United Kingdom, and Egypt, incorporating more than 300
patients. This dataset is complex as it contains diverse samples from
different cohort populations from different countries. Therefore, we use the
Kvasir-SEG for in-distribution testing and PolypGen, BKAI-IGH, and CVC-
ClinicDB for OOD generalization.

###  IV-B Experiment setup and configuration

We select Kvasir-SEG [15] dataset for training all the models. It contains
1000 images and mask pair. We use 880 images and masks for training our method
and the rest for validation and testing. In addition, we perform extensive
data augmentation to increase the size of training samples. All the
experiments are implemented using with PyTorch framework. We run all the
experiments on an NVIDIA RTX 3090 GPU system. We use Adam optimizer with a
learning rate of 1e-4 and a batch size of 8. Additionally, we use a combined
binary cross-entropy and dice loss for training our models.

##  V Result

Comparison with SOTA on in-distrubtion data:  Table I shows the result of the
TransRUPNet. It obtained an mean dice coefficient of 0.9005, mIoU of 0.8445,
recall of 0.9195, precision of 0.9170 and F2-score of 0.9048. With the image
resolution of $256\times 256$, TransRUPNet obtained a real-time processing
speed of 47.07 frames per second (FPS). The most competitive network to
TransRUPNet was TGANet, to whom our architecture outperformed by 1.15% in mIoU
and 0.23% in DSC. The processing speed of our network is almost 1.5 times
TGANet.

Comparison with SOTA on OOD data:  We have evaluated the performance of
TransRUPNet on three OOD datasets. For this, we train different models on
Kvasir-SEG dataset and test it on PolypGen (Center 6). Kindly note that this
is the experimental setup for EndoCV 2021 Challenge [19]. We obtained an
improvement of 4.6% in mIoU and 4.04% in mDSC. Similarly, we obtained an
improvement of 3.21% in mIoU and 3.43% in mDSC on CVC-ClinicDB datasets.
Additionally, we obtained an improvement of 6.06% in mIoU and 6.56% in mDSC
for the TransRUPNet when tested on BKAI-IGH datasets as compared to the SOTA
TGANet [14].

Figure 3 shows the effectiveness of TransRUPNet in qualitative results. As
evidenced by the Figure, TransRUPNet avoids the issues such as over-
segmentation or under-segmentation, which is observed in the case of SOTA
TGANet and PRANet. Additionally, TransRUPNet accurately segments one or more
polyps within the frames, even under challenging conditions. This highlights
the robustness of TransRUPNet in handling complex scenarios and its ability to
delineate the boundaries of polyps correctly. The performance drop of
TransRUPNet compared to the in-distribution datasets is observed because there
are insufficiently cleaned images in datasets, such as PolypGen (C6), that
show elongated black regions on the left side, leading to distorted resizing
and decreased OOD performance. Additionally, there are also huge variations
among the training dataset and OOD datasets. For instance, BKAI-IGH also
contains images from FICE (Flexible spectral Imaging Color Enhancement), BLI
(Blue Light Imaging), and LCI (Linked Color Imaging), in addition to WLI
(White Light Imaging) which are not present in the training datasets. In case
of CVC-ClinicDB, it is a video sequence dataset, whereas our model is trained
on still frames which might have affected the performance. However, the
performance for all the datasets is satisfactory considering the OOD nature of
the experiment.

##  VI Conclusion

In this study, we proposed TransRUPNet architecture by leveraging a pre-
trained Pyramid Vision Transformer (PVT) as an encoder and incorporating a
simple residual block for accurate polyp segmentation. The experimental
results on various in-distribution and OOD datasets demonstrate that
TransRUPNet can provide real-time feedback with high accuracy and perform
significantly well on OOD datasets compared to the existing methods. By
addressing the challenge of OOD generalization and providing reliable polyp
segmentation results, TransRUPNet can be the strong benchmark for developing
computer-aided diagnostic support systems in colonoscopy.

## References

  * [1] G. Urban, P. Tripathi, T. Alkayali, M. Mittal, F. Jalali, W. Karnes, and P. Baldi, “Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy,” _Gastroenterology_ , vol. 155, no. 4, pp. 1069–1078, 2018.
  * [2] A. Leufkens, M. Van Oijen, F. Vleggaar, and P. Siersema, “Factors influencing the miss rate of polyps in a back-to-back colonoscopy study,” _Endoscopy_ , vol. 44, no. 05, pp. 470–475, 2012.
  * [3] D.-P. Fan, G.-P. Ji, T. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao, “Pranet: Parallel reverse attention network for polyp segmentation,” in _Proceedings of the International conference on medical image computing and computer-assisted intervention (MICCAI)_ , 2020, pp. 263–273.
  * [4] G. Yue, W. Han, B. Jiang, T. Zhou, R. Cong, and T. Wang, “Boundary constraint network with cross layer feature integration for polyp segmentation,” _IEEE Journal of Biomedical and Health Informatics_ , 2022.
  * [5] B. Dong, W. Wang, D.-P. Fan, J. Li, H. Fu, and L. Shao, “Polyp-PVT: polyp segmentation with pyramid vision transformers,” _arXiv preprint arXiv:2108.06932_ , 2021.
  * [6] N. T. Duc, N. T. Oanh, N. T. Thuy, T. M. Triet, and V. S. Dinh, “Colonformer: An efficient transformer based method for colon polyp segmentation,” _IEEE Access_ , vol. 10, pp. 80 575–80 586, 2022.
  * [7] W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” in _Proceedings of the IEEE/CVF international conference on computer vision (ICCV)_ , 2021, pp. 568–578.
  * [8] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in _Proceedings of the International Conference on Medical image computing and computer-assisted intervention (MICCAI)_ , 2015, pp. 234–241.
  * [9] Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “UNet++: a nested u-net architecture for medical image segmentation,” in _Deep learning in medical image analysis and multimodal learning for clinical decision support_ , 2018, pp. 3–11.
  * [10] D. Jha, P. H. Smedsrud, M. A. Riegler, D. Johansen, T. De Lange, P. Halvorsen, and H. D. Johansen, “Resunet++: An advanced architecture for medical image segmentation,” in _Proceedings of the International Symposium on Multimedia (ISM)_ , 2019, pp. 225–2255.
  * [11] C.-H. Huang, H.-Y. Wu, and Y.-L. Lin, “HarDNet-MSEG A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS,” _arXiv preprint arXiv:2101.07172_ , 2021.
  * [12] D. Jha, S. Ali, N. K. Tomar, H. D. Johansen, D. Johansen, J. Rittscher, M. A. Riegler, and P. Halvorsen, “Real-time polyp detection, localization and segmentation in colonoscopy using deep learning,” _IEEE Access_ , vol. 9, pp. 40 496–40 510, 2021.
  * [13] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 801–818.
  * [14] N. K. Tomar, D. Jha, U. Bagci, and S. Ali, “Tganet: text-guided attention for improved polyp segmentation,” in _Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2022)_ , 2022, pp. 151–160.
  * [15] D. Jha, P. H. Smedsrud, M. A. Riegler, P. Halvorsen, T. d. Lange, D. Johansen, and H. D. Johansen, “Kvasir-SEG: a segmented polyp dataset,” in _Proceedings of the International Conference on Multimedia Modeling (MMM)_ , 2020, pp. 451–462.
  * [16] S. Ali, D. Jha, N. Ghatwary, S. Realdon, R. Cannizzaro, O. E. Salem, D. Lamarque, C. Daul, M. A. Riegler, K. V. Anonsen _et al._ , “A multi-centre polyp detection and segmentation dataset for generalisability assessment,” _Scientific Data_ , vol. 10, no. 1, p. 75, 2023.
  * [17] P. N. Lan, N. S. An, D. V. Hang, D. Van Long, T. Q. Trung, N. T. Thuy, and D. V. Sang, “NeoUNet: towards accurate colon polyp segmentation and neoplasm detection,” _arXiv preprint arXiv:2107.05023_ , 2021.
  * [18] J. Bernal, F. J. Sánchez, G. Fernández-Esparrach, D. Gil, C. Rodríguez, and F. Vilariño, “WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians,” _Computerized Medical Imaging and Graphics_ , vol. 43, pp. 99–111, 2015.
  * [19] S. Ali, N. Ghatwary, D. Jha, E. Isik-Polat, G. Polat, C. Yang, W. Li, A. Galdran, M.-Á. G. Ballester, V. Thambawita _et al._ , “Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge,” _arXiv preprint arXiv:2202.12031_ , 2022.


# Collective Intelligence in Human-AI Teams: A Bayesian Theory of Mind
Approach

Samuel Westby1, Christoph Riedl2

###### Abstract

We develop a network of Bayesian agents that collectively model the mental
states of teammates from the observed communication. Using a generative
computational approach to cognition, we make two contributions. First, we show
that our agent could generate interventions that improve the collective
intelligence of a human-AI team beyond what humans alone would achieve.
Second, we develop a real-time measure of human’s theory of mind ability and
test theories about human cognition. We use data collected from an online
experiment in which 145 individuals in 29 human-only teams of five communicate
through a chat-based system to solve a cognitive task. We find that humans (a)
struggle to fully integrate information from teammates into their decisions,
especially when communication load is high, and (b) have cognitive biases
which lead them to underweight certain useful, but ambiguous, information. Our
theory of mind ability measure predicts both individual- and team-level
performance. Observing teams’ first 25% of messages explains about 8% of the
variation in final team performance, a 170% improvement compared to the
current state of the art.

##  1 Introduction

The reliance on teamwork in organizations (Wuchty, Jones, and Uzzi 2007),
coupled with remarkable recent progress in artificial intelligence, have
supercharged the vision to develop collaborative Human-AI teams (Malone and
Bernstein 2015; O’Neill et al. 2020). Human-AI teams promise to overcome human
biases and information processing limitations, reaching performance higher
than human-only teams could (Brynjolfsson, Rock, and Syverson 2018). Despite
some recent advances (e.g., Bansal et al. 2019b; Pynadath et al. 2022; Seraj
et al. 2022) there remain significant difficulties in developing agents that
interact with multiple, heterogeneous humans working on cognitive tasks
engaged in cooperative communication in an ad-hoc team. Here, we draw on
research of cognitive processes to develop Human-AI teams and explain
collaborative decision making.

To communicate efficiently, humans infer the beliefs, opinions, knowledge, and
related states of mind of other people (Nickerson 1999; Call and Tomasello
2008). This is referred to as social perceptiveness or theory of mind (ToM;
Premack and Woodruff 1978). Recent research on collective intelligence, has
provided a wide range of empirical evidence suggesting that ToM (and related
processes governing collective memory, attention, and reasoning) is a
significant predictor of human collective intelligence (Woolley et al. 2010;
Riedl et al. 2021; Woolley et al. 2022; Engel et al. 2014). Indeed, ToM is
especially beneficial for interdependent cognitive tasks that benefit when
teams leverage their members’ expertise (Lewis 2003). Some work suggests that
ToM is the mechanism that allows collectives to use more performance-relevant
information from their environment than a single individual without such
connections could, for example, by facilitating a balance between diversity
and cognitive efficiency (Riedl and Woolley 2017; Hong and Page 2004). As
human civilization shifts further toward knowledge work (Autor 2014) where the
most value is realized if members fully use and integrate their unique
expertise, this ability is increasingly important.

Figure 1: Framework of human-AI teaming with Theory of Mind (ToM). a) Nested
layers of ToM agents. Agents model ego networks of the individual they shadow.
b) Every human team member is paired with an AI agent. Humans send messages to
others through a shared environment. The ToM agent infers beliefs for both own
ideas (Ego Model), and ideas of others (one Alter Model per network neighbor).
Ego Model is updated with initial information and new knowledge generated by
the human through self actualization. Alter Models are updated based on
incoming messages from teammates through partner actualization. Agents combine
information from the ego and alter models with weighting determined $\alpha$
denoting ToM ability.

Recent work has started to develop a formal account of collective intelligence
to explain the relationship between individual interaction and collective
performance using (approximate or variational) Bayesian inference (or free
energy; Friston 2010, 2013; Heins et al. 2022). The free energy principle is a
mathematical framework for multiscale behavioral processes that suggests a
system of self-similar agents self-organizes by minimizing variational free
energy in its exchanges with the environment (Fig. 1a; Friston, Kilner, and
Harrison 2006). Recent extensions have applied the framework to explain human
communication (Vasil et al. 2020). A key advantage of this approach is that
free energy minimization can be translated into a generative, agent-based
process theory (Friston et al. 2017; Kaufmann, Gupta, and Taylor 2021). This
generative theory provides a computational approach to cognition (Tenenbaum et
al. 2011; Griffiths 2015) that allows us to simultaneously (a) build agents
for Human-AI teams that are highly explainable but also (b) test theories
about human cognitive processes and measure human theory of mind ability in
real time. This promises to advance our understanding of a key process of
human collective intelligence. The current state of the art to measure theory
of mind—the Reading the Mind in the Eyes test (Baron-Cohen et al. 2001)—is a
static, indirect, survey-based instrument, which typically explains about 3%
of the variation (Riedl et al. 2021).

In this paper, we develop a Bayesian theory of mind agent that can form ad-hoc
mental models about its teammates, based exclusively on observations drawn
from human communication (Fig. 1b). We use data collected from a large, IRB
approved human-subject experiment in which 145 individuals in 29 teams of
five, randomly assigned to different social networks controlling the team’s
communication topology, communicate through a chat-based system to solve a
Hidden Profile task (Stasser and Titus 1985). We then simulate artificial
teams in which each human is shadowed by an agent. The agent observes the same
incoming and outgoing messages as the human did in the experiment. Modeling
human behavior with our generative AI model allows us to test whether people
do indeed form mental models of what their teammates know, how effectively
they do so, and whether this ability is predictive of team performance. In a
last step, we perform a counterfactual simulation to demonstrate how our
Bayesian agent could trigger effective interventions that would increase
Human-AI team performance over the observed human-only teams.

Our work provides a framework that expands theory of mind (and collective
intelligence more broadly) from a static construct to a dynamical one that may
vary according to situational factors, for example, due to changes in arousal,
anxiety, and motivation with dynamically changing task requirements, time
pressure, and recognition (Qin et al. 2022; Balietti and Riedl 2021). We
contribute to a body of research that has so far mostly used toy models—often
using only a single agent—with an application to real data from multi-human
communication (Vasil et al. 2020; Kaufmann, Gupta, and Taylor 2021; Albarracin
et al. 2022; Heins et al. 2022). Our work generates important cognitive
insights into how humans communicate and reason to uncover hidden profiles. In
summary, we make four main contributions.

  1. 1.

We develop a networked Bayesian agent that models beliefs using human-human
communication. We apply the agent to data from a human team experiment and
demonstrate how the agent can monitor theory of mind in real time, predict
both correct and human answers, and intervene to raise human performance.

  2. 2.

We find the model accurately captures the decisions made by humans, varying in
predictable ways with experimental stimuli like network position and task
difficulty. Comparing model fits with simpler “lesioned” ToM models shows the
value contributed by each component.

  3. 3.

We develop two real-time measures for human theory of mind ability. The first,
based on observed human communication and decisions, explains 51% variation in
final team performance. The second, based on communication alone, explains 8%
variation in final team performance after observing just the first quarter of
communication, a 170% improvement compared to the current state of the art,
the Reading the Mind in the Eyes test. Simulations of artificial human-AI
teams suggest a significant 4% performance increase from AI triggered
interventions.

  4. 4.

We contribute to cognitive theory by presenting empirical evidence that
cognitive biases explain the shortfall of human performance, such as a
tendency to under-weight ambiguous information and failure to fully integrate
information provided by others. We explain temporal patterns showing that high
functioning teams send the most useful information early before converging on
common beliefs.

##  2 Related Work

### Human-Agent teaming.

A long history of Human-Agent teaming has evolved alongside technological
developments. Early examples such as Vannevar Bush’s Memex (Bush et al. 1945)
demonstrate a longstanding fascination with augmenting human performance.
Recently, work has specialized in many sub-fields including understanding
mental models and related constructs like team situational awareness (Chen and
Barnes 2014; Converse, Cannon-Bowers, and Salas 1993; Glikson and Woolley
2020). For example, effective Human-AI interaction has been shown to rely
critically on the ability to form mental models about what AI teammates are
doing (Bansal et al. 2019b; Paleja et al. 2021; Bansal et al. 2019a; Gero et
al. 2020; Alipour et al. 2021).

Significantly less work has focused on how AI can form “mental models” of
humans. Fügener et al. (2022) highlight this disparity by identifying
situations where humans having mental models of the AI are not helpful, while
AI having “mental models” of humans are. Given the challenges of designing
multi-agent systems, human-AI teaming work has often focused on studying pairs
of one agent and one human (e.g., Bansal et al. 2019a, b; Baker, Saxe, and
Tenenbaum 2011; Fügener et al. 2022; Alipour et al. 2021). Furthermore, past
work has often side-stepped challenges posed by language-based communication
by constraining the scope to spatial or highly stylized tasks (Kaufmann,
Gupta, and Taylor 2021; Baker et al. 2017; Khalvati et al. 2019). Others use
Wizard of Oz techniques (Schelble et al. 2022; Hohenstein et al. 2022) to
facilitate communication-based human-AI teaming interaction. To build an
autonomous AI teammate that improves human-only team performance, one must
build agents that overcome these obstacles. This becomes more challenging in a
team of unfamiliar teammates, without a priori knowledge, while learning
dynamically from language-based communication (Stone et al. 2010).

### Multi-agent Bayesian models

Multi-agent Bayesian models have been used to study coordination (Khalvati et
al. 2019; Wu et al. 2021), opinion dynamics (Albarracin et al. 2022),
efficient information fusion (Pavlin et al. 2010), and theory of mind (Baker
et al. 2017). This can be modeled as a partially observable Markov decision
process (POMDP) for each agent where the states are the set of other agents’
beliefs and observations are dependent on other agents’ actions (Smith,
Friston, and Whyte 2022).

##  3 Hidden Profile & Human Subject Data

A primary advantage of teams over lone individuals when solving complex
problems is their ability to expand the pool of available information, thereby
enabling teams to reach higher quality solutions (Mesmer-Magnus and DeChurch
2009). The Hidden Profile task (Stasser and Titus 1985) is a research task
designed to mimic this decision-making scenario in which individuals hold
private knowledge (Stone et al. 2010). In the task, some information is
commonly held among all team members while each individual is also endowed
with unique private information. Subjects do not know what information is
shared or private. Information sharing is the central process through which
teammates collectively solve the task (Mesmer-Magnus and DeChurch 2009);
conversely, failing to share all available information causes them to come to
incorrect conclusions. Despite the importance of information sharing for team
performance, past research has shown teams often deviate from the optimal use
of information (Stasser and Titus 1985). Discussions tend to reinforce
information held in common, rather than share information held uniquely by one
team member (Nickerson 1999). One reason for this is that individuals impute
their own knowledge on others and hence assume that private information is
already shared (Nickerson 1999).

This gives rise to the “hidden profile” and directly points to avenues in
which AI may improve team performance: identifying which information is
uniquely held by each teammate and encouraging them to share it. Specifically,
an agent may detect if their own mental model diverges from the inferred
mental model of another (“I know something that I believe you don’t know”)
indicating a window of opportunity for an effective intervention. It also
provides the basis for our measure of theory of mind ability. Individuals who
form more precise mental models of their teammates (and who impute less of
their own knowledge on others) will be more efficient communicators who share
more useful information in a more targeted manner.

We use data from an IRB approved online experiment conducted on the Volunteer
Science platform (Radford et al. 2016) in which 145 individuals in 29 human-
only teams of five solved a Hidden Profile task (data and code available at
https://github.com/riedlc/HumanAITeamsAndCI). The task is framed as a crime
investigation: the team needs to pool clues to answer questions about the
target, culprit, and time of an art heist. There are six clues for each
question. When combined, the correct answer out of five possible options is
obvious. One randomly selected clue is given to every teammate (the public
clue) and each individual receives one of the remaining five clues (the
private clue, also randomly selected). Teams were randomly assigned to a
communication topology using all 21 possible connected five node graphs (e.g.,
star, ring, chain; see Fig. 1a for one example). Teams then communicate via
text-based chat where each message is sent to all neighboring teammates. After
a five minute discussion phase, each individual submits answers for the
culprit’s identity, the target, and the day of the art heist. Subjects were
recruited from Amazon Mechanical Turk (Paolacci, Chandler, and Ipeirotis 2010)
and paid a $0.75 flat fee for participation as well as a $0.25 performance
bonus for each correct answer. The entire task took about seven minutes to
complete. Subjects are blind to the network topology they have been assigned
to as well as the total number of individuals on their team. For simplicity of
exposition and analysis, we rely only on the culprit dimension of the task. We
compute individual performance as $\\{0,1\\}$ depending on whether the culprit
guess is correct and team performance as the average individual performance
(majority voting does not substantively change results). On average,
individuals received $3.1$ (SD $1.9$) chat messages from each partner.

To make initial clues and communication machine interpretable, we manually
code the content as strong no (SN), maybe no (MN), maybe yes (MY), and strong
yes (SY) for each of the five answer options (the inferred states). This
creates a set of 20 possible observations. We translate messages into
likelihoods for the inferred states using fixed values, either estimated from
the data using maximum likelihood estimation (MLE) from a grid search or using
untrained intuitive values. For example, the message,“it might be #4”, would
be coded as maybe yes with a likelihood of $1.4$ for $\\#3$, leaving the
likelihoods for the states not mentioned in the message unaffected. Ambiguous
statements and messages related to team coordination were coded as “neutral”
and dropped from the analysis. Notice that agents form beliefs solely based on
the observed human communication, even if humans make certain statements about
wrong facts (e.g., “strong yes #4” when the correct answer is #3), or
ambiguous statements about correct facts. Agents can thus form wrong beliefs
(Albarracin et al. 2022).

##  4 Bayesian Multi-Agent Model

We create a networked Bayesian agent for each individual. Each agent “shadows”
one human, observing the same messages (both are inside the same Markov
blanket; Fig. 1a) and infers human beliefs and derive the answer to the Hidden
Profile task. That is, our Bayesian system is a model of beliefs about hidden
states of their environment. Ideally, the state inferred after the
communication phase is identical to the correct answer to the Hidden Profile
task. The resulting model has five parameters: four information weights SN,
MN, MY, SY determining the likelihood distribution of observations under
inferred beliefs, and the theory of mind ability $\alpha_{D}$ which modulates
the relative weighting of the self vs. partner beliefs which we describe in
more detail below.

Figure 2: Bayesian model of Theory of Mind. Agent 1, shadowing Player 1,
models all teammates in Player 1’s ego network (Markov blanket). In Agent 1’s
generative model, Alter3 corresponds to Agent 1’s beliefs of Player 3’s
beliefs. At time $t$ Player 3 says, “It might be #4”, which is coded as a MY
for answer 4. Given this new observation, Agent 1 uses Equation 1 to update
its beliefs about Player 3’s beliefs.

### Mental Models.

We use Bayesian inference to infer a posterior distribution
$p(\textbf{s}\mid\textbf{o})$ over states s (five answer options), from a set
of observations o (messages sent between players in the recorded chat). Since
there are five discrete states, we can compute posteriors directly without the
need to approximate them. More complicated environments may require the use of
approximate inference methods like free energy/active inference (Friston,
Kilner, and Harrison 2006).

Agents are comprised of one Ego Model and one Alter Model for each neighbor
(Fig. 1b). That is, the model follows a multi-agent paradigm with independent
mental models nested within an agent. All models hold a posterior distribution
of inferred states, but differ in how they are initialized and updated. Ego
Models are initialized with priors derived from the public and private clues
assigned to the player (paper stack icons in Fig. 1b) and updated with
outgoing messages from a player (self actualization). Alter Models are
initialized with uniform priors and updated with incoming messages of the
corresponding (partner actualization).

Mental models are updated by accounting for the surprise of the observation.
Surprise-weighting thus encodes a preference for posteriors with low surprise.
That is, the effect of new observations is diminished relative to the Ego or
Alter model’s existing posterior using

$p_{i}(s\mid o^{i}_{1:t})\propto p_{i}(s\mid
o^{i}_{1:t-1})p_{i}(o^{i}_{t}|s)^{\overbrace{\scriptstyle-\log{p_{i}(s\mid
o^{i}_{1:t-1})}}^{\text{surprise}}}$ (1)

where $s$ is a state and $o^{i}_{t}$ is an observation (message) sent by
player $i$ at time $t$. The likelihood is raised to the negative log of the
previous time step’s posterior.

| Performance | Human-Agent |  | Model Comparison |
---|---|---|---|---|---
| (% Correct) | Agreement | LogLik | (Likelihood Ratio test) | $\alpha_{D}$
Human | 66.2% |  |  |  |
Random | 19.6 $\pm$ 3.1% | 20.0 $\pm$ 3.0% | -231.759 |  |
Prior only | 48.8 $\pm$ 3.1% | 46.6 $\pm$ 2.9% | -215.906 | vs. Random $p<0.0001$ |
ToM (self-actualization only, MLE) | 63.6 $\pm$ 1.8% | 73.0 $\pm$ 1.9% | -146.372 | vs. Prior-only $p<0.0001$ | 0
ToM (partner-actualization only, MLE) | 76.7 $\pm$ 1.0% | 66.3 $\pm$ 1.0% | -133.653 | vs. Self-only $p<0.0001$ | 1
ToM (MLE) | 72.8 $\pm$ 0.8% | 75.1 $\pm$ 1.0% | -106.640 | vs. Partner-only $p<0.0001$ | 0.95
ToM (max performance) | 77.2 $\pm$ 0.8% | 71.3 $\pm$ 0.9% | -109.826 | vs. MLE $p=0.012$ | 0.95
ToM (max agreement) | 71.8 $\pm$ 0.7% | 79.7 $\pm$ 0.8% | -118.464 | vs. MLE $p<0.0001$ | 0.45
With random intervention | 79.0 $\pm$ 1.8% | 70.8 $\pm$ 1.6% |  |  | 0.95
With intervention |  82.1 $\pm$ 0.7% | 70.0 $\pm$ 0.6% |  | vs. Rand. int. $p<0.0001$ | 0.95
Table 1:  Model evaluation results and comparison with human behavior.
P-values based on likelihood ratio test. We calculate standard deviations over
100 trials. Information weights (SN, MN, SY, MY) are learned from the data
through a grid search: Prior (0.05, 0.05, 2, 2), self-act. (0.05, 0.05, 1.5,
2), partner-act. (0.15, 1, 1.55, 2), MLE (0.1, 1, 1.45, 2), max perf. (0.35,
0.85, 1.95, 2), max agg. (0.05, 0.75, 1.25, 1.95). Interventions use the same
parameters as max performance and $t$-test for comparison.

### Agent.

The agent is the hierarchical coordinator and aggregator of its mental models.
The ToM ability parameter $\alpha_{D}$ modulates the relative weight with
which the agent combines its Ego and Alter models. We conceptualize
$\alpha_{D}$ as the ability to accurately infer beliefs of other agents and
pay attention to them (Apperly and Butterfill 2009). It represents the
relative weighting between the agent’s own Ego Model and its Alter Models.
When $\alpha_{D}=0$, the Alter posterior is uniform and has no effect and the
final prediction is based only on the Ego Model. When $\alpha_{D}=1$, the
Alter Models are weighted equally to the Ego Model. Agent $i$ aggregates its
mental models into a final posterior distribution using

$p_{i}(s\mid\textbf{M}_{i})\propto
p_{i}\left(s\mid\textbf{o}^{i}\right)\prod_{\begin{subarray}{c}m\in\textbf{M}_{i}\\\
m\neq i\end{subarray}}p_{m}\left(s\mid\textbf{o}^{m}\right)^{\alpha_{D}}$ (2)

where $\textbf{M}_{i}$ is Agent $i$’s set of mental models and
$p_{m}\left(s\mid\textbf{o}^{m}\right)$ is posterior of state $s$ for the
mental model of player $m$ over $\textbf{o}^{m}$, the set of $i$’s
observations of $m$.

### Alternative Models.

To test whether the full representational capacity of theory of mind with both
self-actualization and partner-actualization loops are necessary to understand
human mental states, we formulate two alternative models that “lesion” one or
both of the updating loops. This allows us to test whether it is possible to
explain human inferences about their teammates without appealing to a fully
developed theory of mind. We compute $p$-values from likelihood ratio tests
comparing the models.

##  5 Results

Figure 3: Human performance varies with task difficulty and number of
communication partners. From left to right: a) Human performance decreases
with task difficulty and is outperformed by AI agent in most cases. b) Agent
improves over human performance especially when communicating with many
teammates. c) Agents with many communication partners benefit most from high
ToM ability $\alpha_{D}$. (parameters: $\alpha_{D}=0.95$,
$SN=0.35;MN=0.85;MY=1.95;SY=2$).

### Model Evaluation.

We find strong support for the hypothesis that humans use Bayesian inference
to model the minds of their teammates and communicate and make decisions
according to those models (Table 1). Compared to a model using only prior
information (the clues distributed in the experiment), a model capturing
humans’ ability to update their own beliefs (self-actualization only) fits the
data significantly better. A model allowing humans to update beliefs about
their teammates (partner-actualization only) fits significantly better still.
Finally, a model including the capability to update both own and partner
beliefs has the highest fit. Higher values for $\alpha_{D}$ generally lead to
more peaked posterior distributions. This explains why the parameter values
that produce the highest likelihood differ slightly from those of the highest
accuracy ($\alpha_{D}^{\mathit{MLE}}=0.95$ vs.
$\alpha_{D}^{\mathit{maxacc}}=0.45$). In summary, the comparative fit analysis
provides reliable evidence for the mutually inferred alignment of attention
(cf., mental states) among teammates.

Our model accurately captures the judgments of human participants, varying in
predictable ways with random experimental manipulation of task difficulty and
the number of communication partners. We measure the task difficulty faced by
each individual based on how much information the individual can draw about
the correct answer from the two clues they initially received. This captures
how difficult it is for an individual to guess the correct answer before
communicating with other players (this is a somewhat noisy measure as it
ignores team-level effects of the clue distributions). Not surprisingly, human
performance decreases with task difficulty, suggesting that humans suffer from
cognitive overload (Fig. 3a). Our agent achieves high accuracy predicting
human’s incorrect answers under high task difficulty (high true-negative
rate).

Human performance varies with the number of communication partners (Fig. 3b).
Given the nature of the task, access to more communication partners should be
beneficial as this guarantees access to more information. Humans, however,
perform worse with more communication partners while our ToM agent achieves
its highest performance when placed in the most central network position
(agent is 20% better than human with four partners). This suggests that humans
struggle to integrate information when communicating with many teammates. This
picture becomes even clearer when contrasting this with ToM ability
$\alpha_{D}$ (Fig. 3c). Higher levels of ToM ability $\alpha_{D}$ have the
highest benefit on performance in central network positions, yet $\alpha_{D}$
hardly matters when connected to just a single teammate.

### Analysis of Human Decision Biases.

The ToM model predicts with high accuracy instances in which humans provide
the correct answer as well as those in which they provide the wrong answer
(48% true-negative accuracy). Comparing the information weighting parameters
for optimal performance with those for the highest model fit with data from
the human subject experiment from the MLE estimates, we can directly see why
human performance falls short. Humans pay not enough attention to information
ruling out alternatives (optimal information weighting for strong no $0.25$
vs. MLE fit $0.05$). The difference is even more pronounced for ambiguous
information (optimal information weighting for maybe no $0.9$ vs. MLE fit
$0.05$): humans undervalue information that is ambiguous, yet crucial in
arriving at the correct answer. Because this information is ambiguous, humans
may attempt to make sense of it by imputing their own understanding (i.e.,
resorting to their own prior) instead of updating their beliefs in the
direction of the ambiguous message. A similar weighting difference for maybe
yes statements suggests that humans communicate strong yes information in
vague ways (maybe-ing their statements) and could significantly improve their
performance by placing higher weight on such statements (or communicating them
more forcefully).

### Measuring Theory of Mind.

We propose two measures of human theory of mind ability: $\alpha_{D}$ and
$\alpha_{C}$. The first, $\alpha_{D}$, is based on an individual’s ability to
form and integrate accurate mental models of others when making decisions and
corresponds directly to our model parameter that governs the relative
weighting of the Ego vs. Alter Models. The second, $\alpha_{C}$, captures an
individual’s ability to communicate the most useful information. We perform
maximum likelihood estimate using a grid search over the relevant parameter
space (Balietti, Klein, and Riedl 2021). Then, we fix the maximum likelihood
estimate of the nuisance parameters for information weighting (SN, MN, MY, SY)
but consider the marginal of all values of $\alpha_{D}$. Instead of then
picking the global best fitting value for the entire data set, we pick the
maximum likelihood estimate of $\alpha_{D}$ separately for each individual.
That is, we use the model’s inner ToM working to estimate which value of
individual $i^{\prime}s$ $\alpha_{D}$ produces the highest likelihood of the
observed decision. For the second measure $\alpha_{C}$, we consider outgoing
messages sent by each individual and compute the expected surprise that this
message should produce for the recipient, relative to ego’s Alter Model of the
recipient. Notice that we compute this internally within the Markov blanket of
an agent. We do not use information about how surprising the message is for
the recipient but rather how useful the sender thinks it should be relative to
what they think the recipient knows. Intuitively, individuals who possess a
high theory of mind ability, will be better at sending the right message to
the right person compared to those with lower ToM ability. Both measures
capture social perceptiveness: how much attention an individual pays to what
others in the team know.

We find that individual-level ToM ability $\alpha_{D}$ is a strong predictor
of individual-level performance ($\beta=0.59;p<0.001;R^{2}=0.26$). Aggregating
to the team level, we find that average ToM ability
$\alpha_{D}^{\mathit{team}}$ is a strong predictor of final team performance
(Fig. 4a). We find that the effect of ToM ability is moderated by average
betweenness centrality suggesting team performance increases most when high-
ToM ability $\alpha_{D}$ individuals occupy high betweenness network positions
($\beta=0.39;p=0.04$). The amount of communication sent within a team,
notably, is not a significant predictor of team performance
($\beta=-0.00;p=0.265$).

Figure 4: Theory of Mind ability predicts team performance. a) Team average
ToM $\alpha_{D}^{\mathit{team}}$ is a strong predictor of the final team
performance. b) Communication ToM $\alpha_{D}^{\mathit{team}}$ serves as a
real-time measure of collective intelligence. Only about the first 25% of team
messages are necessary to make significant predictions of final team
performance. c) High- and low-performing teams have markedly different
temporal patterns of ToM $\alpha_{C}$.

Turning to our analysis of theory of mind communication ability
$\alpha_{C}^{\mathit{team}}$, we find that it is a strong predictor of team-
level performance ($\beta=0.47;p=0.019$). Given that we can measure
$\alpha_{C}$ on the message level, it can serve as a real-time measure of
theory of mind. We find that after observing only the first 25% of a team’s
messages, $\alpha_{C}^{\mathit{team}}$ is a significant predictor of final
team performance (Fig. 4b). We analyze the temporal pattern in which high- vs.
low-performing teams communicate (Fig. 4c). High-performing teams send
messages with high information content (high surprise) early during the team
task but then send consolidating, low information content messages at the end
to facilitate convergence (low surprise). We illustrate this in the example
below (∗’s indicate high surprise messages with novel content).

> Human 1: It will not happen on Tuesday∗
> Human 3: No Wednesday or Friday∗
> Human 2: Monday or Thursday∗
> Human 2: Did you get a no Thursday info?
> Human 5: Yeah I got no Thursday∗
> Human 3: I got no Thursday
> Human 5: So it must be Monday

This suggests that team cognition is not static but instead emerges
dynamically and that high-performing teams have the collective capacity to
modulate shared cognition dynamically to achieve both efficient information
transfer and convergence during different periods of the task. Low-performing
teams on the other hand fail to send high information content messages and
also fail to achieve convergence sending more surprising messages late during
the task. This pattern illustrates that high information content alone is not
desirable. Instead, convergence and joint attention to team consensus are
crucial (Woolley et al. 2022).

The current standard to predict social perceptiveness is the Reading the Mind
in the Eyes (RME) test (Baron-Cohen et al. 2001; Almaatouq et al. 2021). Using
data from a large meta analysis (Riedl et al. 2021) of 5,279 individuals in
1,356 groups, we find RME explains between 0% and 3% of the variation in team
performance (depending on the task). Our $\alpha_{C}^{\mathit{team}}$ measure
explains 8% after observing only 25% of team communication, an improvement of
about 170%. Our proposed measure captures social perceptiveness passively and
in real time which can be used to interpret a team’s current status and
determine opportunities for interventions. Furthermore, RME captures social
perceptiveness of a single individual, while our measure is group based. Our
work also extends previous measures of “information diversity” (Riedl and
Woolley 2017). It thus captures aspects of collective attention and memory
(Gupta and Woolley 2020).

### Human-Agent Team Performance.

So far, our AI agent was only passively shadowing its assigned human,
reasoning about the mental states of that human and its connected teammates.
In this section, we extend this reasoning to allow the agent to trigger
interventions that could be deployed in human-AI teams and quantify what
performance improvement this might yield. We perform a counterfactual
simulation in which we allow each AI agent to identify and send one message to
each network neighbor. Each agent compares its Ego Model against its Alter
Models to identify divergence in inferred beliefs. Each agent then draws from
the set of messages it received during the team discussion and chooses a
message to send to each neighbor. To do this, the agent calculates the effect
of sharing one of its available messages on the Alter Models and shares the
message that results in the lowest KL divergence, defined as
$D_{\text{KL}}(Q\,||\,P)=\sum_{i}Q(i)\ln\frac{Q(i)}{P(i)}$, between the Ego
and Alteri posteriors over all five possible answer options. If no message
lowers the KL divergence, the agent shares no message. This is summarized as
taking the action $a_{ij}$ for each agent $i$ and neighbor $j$ where

$a_{ij}=\underset{m\in\textbf{o}}{\textit{arg\,min}}\,D_{\text{KL}}\left(p_{\text{ego}_{i}}(\textbf{s
}|\textbf{ o})\,||\,p_{\text{alter}_{j}}(\textbf{s }|\textbf{ o'},m)\right)$
(3)

$m$ is selected from the set of messages o that agent $i$ sent or received, s
is a vector of the five possible answers, and o’ is the set of messages agent
$i$ received from agent $j$. To establish a baseline intervention, we let
$a_{ij}$ be a random message in $\textbf{o}\cup\\{\text{no message\\}}$. Here,
performance improves to 79.0 $\pm$ 1.8% averaged over 100 trials. For the
targeted intervention, performance improves 4.9% to 82.1 $\pm$ 0.7% which is
significantly higher ($t$-test $p<0.0001$) than the random intervention.
Notice that this intervention would not be possible without our ToM-based
multi-agent model. Without it, we could not determine which message to send to
which alter.

##  6 Discussion

We develop a framework that combines theory of mind, Bayesian inference, and
collective intelligence into a generative computational model. Our model
accurately captures the decisions made by human participants in a cognitive
decision-making experiment, varying in predictable ways with experimental
manipulation of task difficulty and network position. Our results suggest that
humans use Bayesian inference and Theory of Mind to model their own beliefs
and those of their teammates and communicate and make decisions according to
those models. We provide empirical evidence that humans do not do this
perfectly but suffer from cognitive biases. Nonetheless, our Bayesian agent is
robust and achieves high performance even when fed biased and incorrect
information, providing a pathway to implement high-performing human-AI teams.
Notably, our agent works in ad hoc teams with heterogeneous partners without
any pretraining. In such human-AI teams, our AI could augment humans’ limited
cognitive memory, attention, and reasoning abilities to increase collective
intelligence.

We show empirical evidence that the collective dynamics of Bayesian agents
updating probabilities of hypotheses using observations, collectively predict
the performance at the team level. This provides the basis for a real-time
measure of theory of mind ability, and maybe even collective intelligence more
broadly (Heins et al. 2022). The better the mental models of the team members
align—the less surprising observations drawn from communication become—the
higher the team’s collective intelligence. Our implementation of direct
surprise weighting could be extended with a fuller implementation of the free
energy principle that would allow agents to learn asymmetric beliefs about the
reliability of their partners’ signals. Taken together, this is a framework to
capture the emergence of collective memory, attention, and reasoning in real
time (Luria 1973; Gupta and Woolley 2020).

## Acknowledgements

This work was supported by the Army Research Laboratory [Grant
W911NF-19-2-0135].

## References

  * Albarracin et al. (2022) Albarracin, M.; Demekas, D.; Ramstead, M. J.; and Heins, C. 2022.  Epistemic communities under active inference.  _Entropy_ , 24(4): 476.
  * Alipour et al. (2021) Alipour, K.; Ray, A.; Lin, X.; Cogswell, M.; Schulze, J. P.; Yao, Y.; and Burachas, G. T. 2021.  Improving users’ mental model with attention-directed counterfactual edits.  _Applied AI Letters_ , 2(4): e47.
  * Almaatouq et al. (2021) Almaatouq, A.; Alsobay, M.; Yin, M.; and Watts, D. J. 2021.  Task complexity moderates group synergy.  _Proceedings of the National Academy of Sciences_ , 118(36): e2101062118.
  * Apperly and Butterfill (2009) Apperly, I. A.; and Butterfill, S. A. 2009.  Do humans have two systems to track beliefs and belief-like states?  _Psychological Review_ , 116(4): 953.
  * Autor (2014) Autor, D. H. 2014.  Skills, education, and the rise of earnings inequality among the “other 99 percent”.  _Science_ , 344(6186): 843–851.
  * Baker, Saxe, and Tenenbaum (2011) Baker, C.; Saxe, R.; and Tenenbaum, J. 2011.  Bayesian theory of mind: Modeling joint belief-desire attribution.  In _Proceedings of the Thirty-Third Annual Meeting of the Cognitive Science Society_ , 2469–2474.
  * Baker et al. (2017) Baker, C. L.; Jara-Ettinger, J.; Saxe, R.; and Tenenbaum, J. B. 2017.  Rational quantitative attribution of beliefs, desires and percepts in human mentalizing.  _Nature Human Behaviour_ , 1(4): 1–10.
  * Balietti, Klein, and Riedl (2021) Balietti, S.; Klein, B.; and Riedl, C. 2021.  Optimal design of experiments to identify latent behavioral types.  _Experimental Economics_ , 24(3): 772–799.
  * Balietti and Riedl (2021) Balietti, S.; and Riedl, C. 2021.  Incentives, competition, and inequality in markets for creative production.  _Research Policy_ , 50(4): 104212.
  * Bansal et al. (2019a) Bansal, G.; Nushi, B.; Kamar, E.; Lasecki, W. S.; Weld, D. S.; and Horvitz, E. 2019a.  Beyond accuracy: The role of mental models in human-AI team performance.  In _Proceedings of the Seventh AAAI Conference on Human Computation and Crowdsourcing_ , 2–11.
  * Bansal et al. (2019b) Bansal, G.; Nushi, B.; Kamar, E.; Weld, D. S.; Lasecki, W. S.; and Horvitz, E. 2019b.  Updates in human-AI teams: Understanding and addressing the performance/compatibility tradeoff.  In _Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence_ , 2429–2437.
  * Baron-Cohen et al. (2001) Baron-Cohen, S.; Wheelwright, S.; Hill, J.; Raste, Y.; and Plumb, I. 2001.  The “Reading the Mind in the Eyes” Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism.  _The Journal of Child Psychology and Psychiatry and Allied Disciplines_ , 42(2): 241–251.
  * Brynjolfsson, Rock, and Syverson (2018) Brynjolfsson, E.; Rock, D.; and Syverson, C. 2018.  Artificial intelligence and the modern productivity paradox: A clash of expectations and statistics.  In _The Economics of Artificial Intelligence: An Agenda_ , 23–57. Chicago, IL: University of Chicago Press.
  * Bush et al. (1945) Bush, V.; et al. 1945.  As we may think.  _The Atlantic Monthly_ , 176(1): 101–108.
  * Call and Tomasello (2008) Call, J.; and Tomasello, M. 2008.  Does the chimpanzee have a theory of mind? 30 years later.  _Trends in Cognitive Sciences_ , 12(5): 187–192.
  * Chen and Barnes (2014) Chen, J. Y.; and Barnes, M. J. 2014.  Human–agent teaming for multirobot control: A review of human factors issues.  _IEEE Transactions on Human-Machine Systems_ , 44(1): 13–29.
  * Converse, Cannon-Bowers, and Salas (1993) Converse, S.; Cannon-Bowers, J.; and Salas, E. 1993.  Shared mental models in expert team decision making.  _Individual and group decision making: Current issues_ , 221: 221–46.
  * Engel et al. (2014) Engel, D.; Woolley, A. W.; Jing, L. X.; Chabris, C. F.; and Malone, T. W. 2014.  Reading the mind in the eyes or reading between the lines? Theory of mind predicts collective intelligence equally well online and face-to-face.  _PloS One_ , 9(12): e115212.
  * Friston (2010) Friston, K. 2010.  The free-energy principle: a unified brain theory?  _Nature Reviews Neuroscience_ , 11(2): 127–138.
  * Friston (2013) Friston, K. 2013.  Life as we know it.  _Journal of the Royal Society Interface_ , 10(86): 20130475.
  * Friston et al. (2017) Friston, K.; FitzGerald, T.; Rigoli, F.; Schwartenbeck, P.; and Pezzulo, G. 2017\.  Active inference: a process theory.  _Neural Computation_ , 29(1): 1–49.
  * Friston, Kilner, and Harrison (2006) Friston, K.; Kilner, J.; and Harrison, L. 2006.  A free energy principle for the brain.  _Journal of Physiology_ , 100(1-3): 70–87.
  * Fügener et al. (2022) Fügener, A.; Grahl, J.; Gupta, A.; and Ketter, W. 2022.  Cognitive challenges in human–artificial intelligence collaboration: investigating the path toward productive delegation.  _Information Systems Research_ , 33(2): 678–696.
  * Gero et al. (2020) Gero, K. I.; Ashktorab, Z.; Dugan, C.; Pan, Q.; Johnson, J.; Geyer, W.; Ruiz, M.; Miller, S.; Millen, D. R.; Campbell, M.; et al. 2020.  Mental models of AI agents in a cooperative game setting.  In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ , 1–12.
  * Glikson and Woolley (2020) Glikson, E.; and Woolley, A. W. 2020.  Human trust in artificial intelligence: Review of empirical research.  _Academy of Management Annals_ , 14(2): 627–660.
  * Griffiths (2015) Griffiths, T. L. 2015.  Manifesto for a new (computational) cognitive revolution.  _Cognition_ , 135: 21–23.
  * Gupta and Woolley (2020) Gupta, P.; and Woolley, A. W. 2020.  The emergence of collective intelligence behavior.  In _Proceedings of the Paper presented at the 8th ACM Collective Intelligence (CI) Conference, Virtual Event, Zurich, Switzerland_.
  * Heins et al. (2022) Heins, C.; Klein, B.; Demekas, D.; Aguilera, M.; and Buckley, C. 2022.  Spin glass systems as collective active inference.  _arXiv preprint arXiv:2207.06970_.
  * Hohenstein et al. (2022) Hohenstein, J.; Larson, L. E.; Hou, Y. T.-Y.; Harris, A. M.; Schecter, A.; Dechurch, L.; Contractor, N.; and Jung, M. F. 2022.  Vero: A Method for Remotely Studying Human-AI Collaboration.  In _Proceedings of the 55th Hawaii International Conference on System Sciences_.
  * Hong and Page (2004) Hong, L.; and Page, S. E. 2004.  Groups of diverse problem solvers can outperform groups of high-ability problem solvers.  _Proceedings of the National Academy of Sciences_ , 101(46): 16385–16389.
  * Kaufmann, Gupta, and Taylor (2021) Kaufmann, R.; Gupta, P.; and Taylor, J. 2021.  An active inference model of collective intelligence.  _Entropy_ , 23(7): 830.
  * Khalvati et al. (2019) Khalvati, K.; Park, S. A.; Mirbagheri, S.; Philippe, R.; Sestito, M.; Dreher, J.-C.; and Rao, R. P. 2019.  Modeling other minds: Bayesian inference explains human choices in group decision-making.  _Science Advances_ , 5(11): eaax8783.
  * Lewis (2003) Lewis, K. 2003.  Measuring transactive memory systems in the field: scale development and validation.  _Journal of Applied Psychology_ , 88(4): 587.
  * Luria (1973) Luria, A. R. A. R. 1973.  _The working brain; an Introduction to Neuropsychology_.  New York, NY: Basic Books.
  * Malone and Bernstein (2015) Malone, T. W.; and Bernstein, M. S. 2015.  _Handbook of Collective Intelligence_.  MIT Press.
  * Mesmer-Magnus and DeChurch (2009) Mesmer-Magnus, J. R.; and DeChurch, L. A. 2009.  Information sharing and team performance: a meta-analysis.  _Journal of Applied Psychology_ , 94(2): 535.
  * Nickerson (1999) Nickerson, R. S. 1999.  How we know—and sometimes misjudge—what others know: Imputing one’s own knowledge to others.  _Psychological Bulletin_ , 125(6): 737.
  * O’Neill et al. (2020) O’Neill, T.; McNeese, N.; Barron, A.; and Schelble, B. 2020.  Human–autonomy teaming: A review and analysis of the empirical literature.  _Human Factors_ , 64: 904–938.
  * Paleja et al. (2021) Paleja, R.; Ghuy, M.; Ranawaka Arachchige, N.; Jensen, R.; and Gombolay, M. 2021\.  The utility of explainable ai in ad hoc human-machine teaming.  _Advances in Neural Information Processing Systems (NeurIPS)_ , 34: 610–623.
  * Paolacci, Chandler, and Ipeirotis (2010) Paolacci, G.; Chandler, J.; and Ipeirotis, P. G. 2010.  Running experiments on amazon mechanical turk.  _Judgment and Decision making_ , 5(5): 411–419.
  * Pavlin et al. (2010) Pavlin, G.; de Oude, P.; Maris, M.; Nunnink, J.; and Hood, T. 2010.  A multi-agent systems approach to distributed bayesian information fusion.  _Information Fusion_ , 11(3): 267–282.
  * Premack and Woodruff (1978) Premack, D.; and Woodruff, G. 1978.  Does the chimpanzee have a theory of mind?  _Behavioral and Brain Sciences_ , 1(4): 515–526.
  * Pynadath et al. (2022) Pynadath, D. V.; Dilkina, B.; Jeong, D. C.; John, R. S.; Marsella, S. C.; Merchant, C.; Miller, L. C.; and Read, S. J. 2022.  Disaster world.  _Computational and Mathematical Organization Theory_ , in press.
  * Qin et al. (2022) Qin, Y.; Zhang, W.; Lee, R.; Sun, X.; and Sajda, P. 2022.  Predictive Power of Pupil Dynamics in a Team Based Virtual Reality Task.  In _2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops_ , 592–593. IEEE.
  * Radford et al. (2016) Radford, J.; Pilny, A.; Reichelmann, A.; Keegan, B.; Welles, B. F.; Hoye, J.; Ognyanova, K.; Meleis, W.; and Lazer, D. 2016.  Volunteer science: An online laboratory for experiments in social psychology.  _Social Psychology Quarterly_ , 79(4): 376–396.
  * Riedl et al. (2021) Riedl, C.; Kim, Y. J.; Gupta, P.; Malone, T. W.; and Woolley, A. W. 2021.  Quantifying collective intelligence in human groups.  _Proceedings of the National Academy of Sciences_ , 118(21): e2005737118.
  * Riedl and Woolley (2017) Riedl, C.; and Woolley, A. W. 2017.  Teams vs. crowds: A field test of the relative contribution of incentives, member ability, and emergent collaboration to crowd-based problem solving performance.  _Academy of Management Discoveries_ , 3(4): 382–403.
  * Schelble et al. (2022) Schelble, B. G.; Flathmann, C.; McNeese, N. J.; Freeman, G.; and Mallick, R. 2022\.  Let’s Think Together! Assessing Shared Mental Models, Performance, and Trust in Human-Agent Teams.  _Proceedings of the ACM on Human-Computer Interaction_ , 6(GROUP): 1–29.
  * Seraj et al. (2022) Seraj, E.; Wang, Z.; Paleja, R.; Martin, D.; Sklar, M.; Patel, A.; and Gombolay, M. 2022.  Learning efficient diverse communication for cooperative heterogeneous teaming.  In _Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems_ , 1173–1182.
  * Smith, Friston, and Whyte (2022) Smith, R.; Friston, K. J.; and Whyte, C. J. 2022.  A step-by-step tutorial on active inference and its application to empirical data.  _Journal of Mathematical Psychology_ , 107: 102632.
  * Stasser and Titus (1985) Stasser, G.; and Titus, W. 1985.  Pooling of unshared information in group decision making: Biased information sampling during discussion.  _Journal of Personality and Social Psychology_ , 48(6): 1467.
  * Stone et al. (2010) Stone, P.; Kaminka, G. A.; Kraus, S.; and Rosenschein, J. S. 2010.  Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination.  In _Proceedings of the Twenty-Fourth Conference on Artificial Intelligence_.
  * Tenenbaum et al. (2011) Tenenbaum, J. B.; Kemp, C.; Griffiths, T. L.; and Goodman, N. D. 2011.  How to grow a mind: Statistics, structure, and abstraction.  _Science_ , 331(6022): 1279–1285.
  * Vasil et al. (2020) Vasil, J.; Badcock, P. B.; Constant, A.; Friston, K.; and Ramstead, M. J. 2020.  A world unto itself: human communication as active inference.  _Frontiers in Psychology_ , 11: 417.
  * Woolley et al. (2010) Woolley, A. W.; Chabris, C. F.; Pentland, A.; Hashmi, N.; and Malone, T. W. 2010\.  Evidence for a collective intelligence factor in the performance of human groups.  _Science_ , 330(6004): 686–688.
  * Woolley et al. (2022) Woolley, A. W.; Chow, R. M.; Mayo, A. T.; Riedl, C.; and Chang, J. W. 2022.  Collective Attention and Collective Intelligence: The Role of Hierarchy and Team Gender Composition.  _Organization Science_ , in press.
  * Wu et al. (2021) Wu, S. A.; Wang, R. E.; Evans, J. A.; Tenenbaum, J. B.; Parkes, D. C.; and Kleiman-Weiner, M. 2021.  Too Many Cooks: Bayesian Inference for Coordinating Multi-Agent Collaboration.  _Topics in Cognitive Science_ , 13(2): 414–432.
  * Wuchty, Jones, and Uzzi (2007) Wuchty, S.; Jones, B. F.; and Uzzi, B. 2007.  The increasing dominance of teams in production of knowledge.  _Science_ , 316(5827): 1036–1039.