# Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun
Rebus Art Understanding

Tuo Zhang111footnotemark: 1 , Tiantian Feng111footnotemark: 1, Yibin
Ni211footnotemark: 1, Mengqin Cao311footnotemark: 1,
Ruying Liu1, Katharine Butler4, Yanjun Weng6
Mi Zhang5, Shrikanth S. Narayanan1, Salman Avestimehr1
1University of Southern California, 2Shanghai International Studies
University,
3Independent Researcher, 4The Butler Museum, 5The Ohio State University
6Jingdezhen Imperial Kiln Institute These authors contribute equally

###### Abstract

Large vision-language models (VLMs) have demonstrated remarkable abilities in
understanding everyday content. However, their performance in the domain of
art, particularly culturally rich art forms, remains less explored. As a pearl
of human wisdom and creativity, art encapsulates complex cultural narratives
and symbolism. In this paper, we offer the Pun Rebus Art Dataset, a multimodal
dataset for art understanding deeply rooted in traditional Chinese culture. We
focus on three primary tasks: identifying salient visual elements, matching
elements with their symbolic meanings, and explanations for the conveyed
messages. Our evaluation reveals that state-of-the-art VLMs struggle with
these tasks, often providing biased and hallucinated explanations and showing
limited improvement through in-context learning. By releasing the Pun Rebus
Art Dataset, we aim to facilitate the development of VLMs that can better
understand and interpret culturally specific content, promoting greater
inclusiveness beyond English-based corpora.

##  1 Introduction

Each culture develops its unique symbolic systems of visual elements, which
are conventionally understood within that culture to convey specific meanings.
For example, to viewers unfamiliar with Chinese arts and linguistics, the
combination of a monkey and a horse might seem nonsensical. However, in
Chinese culture, "a monkey lying on top of the horse" is described as a pun on
"马上封侯" (mǎ shàng fēng hóu)111The notation is in Pinyin, the official
romanization pronunciation system for Standard Mandarin Chinese, representing
the wish for promotion. This form of wordplay is prevalent in Chinese
decorative arts, appearing in various art formats throughout Chinese history,
from the emperor’s court to the commoners’ kitchen, transcending boundaries of
power, wealth, education, and media. As an example, in Figure 1, we
demonstrate a Chinese pun rebus painting with "a monkey lying on top of the
horse," which indicates the wish for promotion by connecting homophonically
similar Chinese characters of "horse-马(mǎ)," "on top of-上(shàng)," combined to
form ‘mashang’ also meaning ‘right away’, and "monkey-猴(hóu), sounding similar
to 侯(hóu) for ‘marquis."

In this work, we propose the Pun Rebus Art Dataset, which is rooted in
traditional Chinese culture. We focus on Chinese Pun Rebus art for three major
reasons: 1) creating a pun rebus artwork involves combining textual meanings
with corresponding visual representations, making it naturally multimodal; 2)
pun rebus is prevalent in Chinese art, rarely seen in other cultures such as
western painting [19]; 3) pun rebus art remains widespread in contemporary
Chinese culture, demonstrating its enduring impact and lasting value to
preserve cultural identity while engaging new generations.

We introduce three sequential tasks that reflect the underlying chain-of-
thought process of experts in decoding Chinese pun rebuses. Our goal is to
benchmark the capability of large vision-language models (VLMs) in
recognizing, interpreting, and comprehending these rich and cultural-specific
meanings across vision and language: 1) identifying the salient and relevant
visual elements in art; 2) matching the visual elements with symbolic meaning;
and 3) generating an explanation to express why an artwork convey certain
messages. To the best of our knowledge, this is one of the first datasets to
test AI’s abilities in handling cultural-specific art expression, particularly
focusing on the accurate identification and interpretation of visual
signifiers within Chinese pun rebus art.

Our results highlight the inherent challenges by both AI and non-expert humans
in understanding Chinese pun rebus arts compared to experts. In the visual
element identification task, even the best VLM captures only about 30% of key
elements, slightly outperforming non-expert humans. Moreover, most VLMs
struggle to match the symbolic meanings associated with Chinese culture, with
GPT-4o achieving the highest accuracy of 42% in a 7-way multiple-choice
question. In comparison, non-expert humans manage to reach a 55% accuracy in
this task. Finally, experts note that the explanations generated by VLMs in
expression understanding often involve biases and hallucinations, underscoring
the current VLMs’ limitation in understanding Chinese art and potentially
other culturally specific contents. We hope that our effort in curating,
releasing, and benchmarking the Chinese Pun Rebus Arts dataset will facilitate
the development of VLMs in understanding cross-cultural content other than
English-based corpus, thereby promoting greater inclusiveness.

Figure 1: The illustration of the chain of thought on understanding the
Chinese pun rebus. The example artwork uses a horse and a monkey to construct
the pun "马上封侯" (mǎ shàng fēng hóu), which means "May you instantly become a
marquis" in English.

##  2 General Framework for Pun Rebus Understanding

A pun rebus in Chinese culture leverages visual elements to indicate an
underlying expression, metaphor, or meaning that is seemingly unrelated to the
given image [19, 20]. The fundamental mechanism of pun rebuses hinges on the
interplay between the imagery composed, on the one hand, and the semantic and
phonetic components of the Chinese logographs used to express a message,
usually auspicious. Specifically, the interpretation of pun rebuses relies on
homophonic associations between the names of the depicted images (or their
interactions) and the Chinese characters (logographs) used to express the
concepts that form the intended message, either partially or fully. The names
of the objects in a pun rebus are often homophonically similar to, or even
identical with, the cued expression, analogous to using the English string
‘eye—can—sea—ewe’ to express ‘I can see you’. A pun rebus design is intended
to initiate a cognitive translation process of "image-sound-sound-meaning,"
contrasting sharply with the more direct and straightforward ‘text-meaning’
decoding typically observed in pure verbal understanding. Because the process
is not only culturally but also linguistically specific, it is extremely
challenging for an uninformed viewer to perceive and decipher any underlying
meanings of this art form. These artworks are composed for aesthetic or
attention-attracting purposes.

Generally, the chain of thought on understanding the pun rebus is composed of
three sequential steps: (1) spotting the salient visual elements within the
artwork; (2) utilizing these identified elements to formulate the underlying
pun; (3) understanding the intended message or wish conveyed by the pun rebus.
We present a visualized example in Figure 1 as an illustration of pun rebus
understanding.

##  3 Pun Rebus Art Dataset

###  3.1 Data Collection

The Pun Rebus Art dataset is designed as a comprehensive benchmark for
exploring the intersection of image analysis, morphological variation, and
phonological elements within the context of Chinese linguistics and cultural
artifacts. This dataset is the result of extensive efforts to curate a diverse
array of historical artwork documents. Initiated in 1987 by Dr. Ni Yibin, a
co-author of this paper, the dataset’s preparation involved meticulous
collection, annotation, and verification processes that require expert
knowledge of Chinese art, literature, history, and linguistics. The corpus
comprises 1,011 captioned images sourced predominantly from globally-renowned
Chinese-art-collecting institutions, including the Palace Museum, the
Metropolitan Museum of Art, and the British Museum. The images in this dataset
are subject to the Creative Commons Zero (CC0) license. Spanning over two
millennia, from the Han Dynasty (206 BCE – 220 CE) to the 20th century, the
dataset encompasses a rich diversity of more than ten different media types,
including paintings, ceramics, bronzes, sculptures, jade, Cloisonné,
lacquerware, and embroidery. The collection of the Pun Rebus Art dataset is
ongoing as we continue to curate it with additional artworks to enhance its
representational diversity.

###  3.2 Data Annotation

Each entry has been meticulously annotated by human experts with knowledge of
Chinese linguistics, art, and history. Figure 2 exemplifies the structured
content in the Pun Rebus Art dataset. Each entry comprises the following
components: (1) the original artwork without its caption; (2) the articulated
pun rebus, presented bilingually to encompass both the original Chinese script
and its English counterpart; (3) the salient elements that constitute the
pun’s design; and (4) an analysis delineating the relationship between the
visual representation and the intended pun rebus. To ensure high-quality
annotations, we implement a strict three-round validation check after the
initial annotating process.

Figure 2: An example data sample and category distribution of the Pun Rebus
Art dataset. We offer both English and Chinese versions of the data annotation
in the proposed dataset. The dataset querying system is available on
http://niyibin.org/punrebus/punrebus_main_en.php.

##  4 Task Setups

Based on the characteristics of the Chinese pun rebus artwork, we present
three primary and progressive tasks in this paper: Element Identification,
Symbolic Matching, and Expression Understanding. We want to highlight that the
researchers are highly encouraged to explore additional applications and
analyses tailored to their specific interests and needs using this dataset. In
the following, we describe the details of each task and the corresponding
evaluation metrics.

###  4.1 Task Design

Element Identification. In the initial task, we aim to explore: What catches
the model’s attention most in the artwork? Artworks are complex composites of
features such as texture, shape, color, and other painting elements. However,
not all these features are essential for constructing the pun embedded within
the artwork. This task seeks to determine which elements the model prioritizes
from its perspective. For instance, consider the artwork shown in Figure 3: a
ceramic jar made in the Yongzheng period of the Qing Dynasty (1732 - 1735).
This jar exhibits numerous features, including its egg-like shape, the white-
color clay body, the flowers at the top, and the colorful rock at the bottom.
However, only the narcissus flowers, the red berries of nandina, and the
lingzhi mushrooms depicted on the jar are crucial to its implied wishes. In
Chinese, the sound of ’narcissus’ echoes the phrase for ’heavenly immortals
and fairies,’ and the sound of ’nandina’ echoes ’heaven,’ while ’lingzhi
mushrooms’ are traditionally associated with longevity. Their combined
presence suggests the wish ’May you enjoy a long life as immortals.’ In
contrast, other elements like the jar’s shape or the rock at the bottom, while
visually striking, do not contribute significantly to the articulated wish.

Symbolic Matching. In the second task, we investigate the question: What does
the model understand after reading the artwork? Drawing upon expertise in
Chinese iconographic art history and cultural studies, we have classified the
auspicious expressions depicted in the datasets into seven categories, as the
example shown in Figure 3. The category distribution is presented in Figure 2.
We ask the model to make a selection that best aligns with the conveyed
meaning behind the given artwork images among the eight options. This task
serves as a direct evaluation of the model’s ability to comprehend the pun
rebus reasoning embedded within each artwork.

Figure 3: Illustration of the three evaluation tasks using an 18th-century
Chinese ceramic as an example. Bold marks the salient elements. Symbolic
Matching probes the model’s understanding of the artwork’s implied meanings.
Element Identification asks what catches the model’s attention most in the
artwork. Expression Understanding delves into the rationale behind the model’s
interpretations.

Expression Understanding. Finally, we want to see Why does the model interpret
the artwork as it does? This task is designed to delve into the reasoning
behind the model’s decisions, providing insights into its interpretative
process. By understanding the justifications for the model’s choices, we can
assess how closely it aligns with human understanding of cultural and symbolic
meanings.

###  4.2 Evaluation Metrics

Element Identification. For the element identification, we report the absolute
score and the similarity score. The absolute score represents the overlap
between key elements in the model’s output and those in the ground truth. Let
$G=\\{g_{1},g_{2},\dots,g_{n}\\}$ represent the set of elements in the ground
truth description, and $P=\\{p_{1},p_{2},\dots,p_{m}\\}$ denote the set of
elements identified by the language model. The absolute score for a single
instance, $S$, is calculated as follows:

$S_{Abs}(G,P)=\frac{|G\cap P|}{|G|}$ (1)

It quantifies the extent to which essential elements are captured in the
model’s output, normalized by the total number of elements in the ground
truth. For the overall performance across the dataset, we report the average
score, $\overline{S_{Abs}}$, computed as the mean of individual scores across
all test instances.

Apart from the absolute score, we introduce the similarity score to account
for semantic equivalence, which considers synonyms and semantically related
terms that align with the ground truth. We map both ground truth answers $G$
and generated answers $P$ to word embedding using the pre-trained Sentence-
BERT [13]. For each test instance, we measure word-wise cosine similarity
between each element in $G$ and all elements in $P$, recording the highest
similarity score for each element in $G$. The similarity score for each
instance is the average of these maximum scores for all elements in $G$:

$S_{Sim}(G,P)=\frac{1}{|G|}\sum_{g\in G}\max_{p\in P}\cos(emb(g),emb(p))$ (2)

where $emb(x)$ denotes the embedding of the element $x$, and $\cos$ denotes
the cosine similarity function. We report the average score
$\overline{S_{Sim}}$ for the overall performance of the dataset.

Symbolic Matching. For the symbolic matching, we evaluate using the accuracy.
It is worth noting that certain artworks may convey multiple implied meanings
among the options provided. An answer is considered correct if it includes at
least one implied meaning specified in the ground truth.

Expression Understanding. We conduct human evaluation to judge the expression
understanding. The panel of human judges consists of five individuals: three
authors of this paper and two independent experts with educational and
professional backgrounds in the field of art history. We ask each judge to
grade the model-generated explanations on a scale from 1 to 10. A score of 10
represents a perfect explanation, indicating that the human judge cannot
distinguish whether the answer is from the machine or a human expert. A score
of 1 signifies that the response is completely incorrect and irrelevant. We
also list our findings and hypothesis from human evaluations in Section 5.3.

##  5 Experiments

We evaluate the performance of various widely used VLMs using the Pun Rebus
Art dataset. Our evaluation is conducted under both zero-shot and five-shot
settings to examine the inherent ability without fine-tuning specific to our
dataset. Specifically, we aim to probe the ingrained knowledge and reasoning
processes of these models, exploring their potential limitations or biases in
interpreting objects and concepts related to Chinese culture. This is
particularly pertinent to ensure the inclusiveness of VLMs given that most
models are predominantly trained on English-based resources, which may affect
their performance on culturally specific tasks [21, 8]. We use the unified
prompt for each task across all models, which are listed in Appendix. We
sample with default hyperparameters in all cases. All experiments are
conducted with NVIDIA A100 GPUs.

###  5.1 Baselines

VLMs. Our selection prioritizes the largest, most recent, and highest-
performing VLMs currently available. Our selection comprises: (1) The GPT-4
model family [1]. We include both GPT-4o and GPT-4V in our benchmark. (2) The
Gemini 1.0 Pro Vision [17] is the only multimodality LMM available for public
usage among the Gemini model family. (3) The Claude 3 model family [2]. Our
benchmark includes Claude 3 Opus, Sonnet, and Haiku. (4) The Qwen-VL family
[3]. It is worth noting that its training incorporates the Chinese image-text
data corpus, making it more relevant to benchmark our dataset. We incorporate
both Qwen-VL-Plus and Qwen-VL-Max in our evaluation. For all models listed, we
utilize the latest model checkpoint available at the time of writing this
paper. Detailed checkpoint information and version specifics are provided in
Appendix.

Human Performance Estimates. Following the previous study [5], we include an
evaluation of human performance to compare with the VLMs. Unlike the expert
panel described in Section 4.2, we enlist crowd-workers who lack a specialized
background in Chinese art, representing the general population’s
understanding. Specifically, the panel consists of 3 bilingual individuals,
all native Chinese speakers who are also fluent in English. Each participant
will review 50 artworks and respond to the questions related to symbolic
matching and element identification. These artworks are randomly selected in
the full dataset with the same label distribution. We report the average
scores across participants as the human performance estimates. We want to note
that the human performance in this paper should not be considered as an upper
bound for VLMs. Instead, it is used to measure how well ordinary people raised
in contemporary Chinese society understand traditional Chinese arts.

| Symbolic Matching | Element Identification
---|---|---
| Accuracy ($\uparrow$) |  $\overline{S_{Abs}}$ ($\uparrow$) |  $\overline{S_{Sim}}$ ($\uparrow$)
Random Choice | 14.29% | \- - | \- -
GPT-4o | 40.40% | 0.3145 | 0.5688
$\Lsh$ Five-shot |  $\Lsh$ 42.18% |  $\Lsh$ 0.3499 |  $\Lsh$ 0.5851
GPT-4V | 26.53% | 0.2616 | 0.5003
Gemini Pro | 27.92% | 0.3398 | 0.5003
Claude 3 Opus | 22.47% | 0.2405 | 0.4983
$\Lsh$ Five-shot |  $\Lsh$ 19.77% |  $\Lsh$ 0.2623 |  $\Lsh$ 0.5127
Claude 3 Sonnet | 20.55% | 0.1767 | 0.4030
Claude 3 Haiku | 21.91% | 0.1713 | 0.4350
Qwen-VL-Max | 37.77% | 0.2453 | 0.4786
$\Lsh$ Five-shot |  $\Lsh$ 21.45% |  $\Lsh$ 0.0327 |  $\Lsh$ 0.3406
Qwen-VL-Plus | 28.88% | 0.2545 | 0.4131
Human Estimate | 55.33% | 0.2483 | 0.4615

Table 1: Evaluation results for the symbolic matching and element
identification tasks among various VLMs. Bold results are best for zero-shot
evaluation in each category. Right: sample results by GPT-4o, Gemini Pro,
Claude 3 Opus and Qwen-VL-Max over a matching/identification instance.

###  5.2 Main Results

####  5.2.1 Evaluation under Zero-shot Settings

In this section, we compare different VLMs through a zero-shot evaluation of
the Pun Rebus Art benchmark, as detailed in Table 1. We make five key
observations:

(1) The challenging nature of the Pun Rebus Art dataset. We observe that the
highest accuracy in symbolic matching achieved under the zero-shot setting is
around 40% for all models. Notably, the human estimation also averages only
around 55%, underscoring the difficulty of understanding the symbolic meaning
in the art. As we stated in Section 3, the Pun Rebus Art dataset spans artwork
ranging over 2000 years, where many visual representations or underlying
narratives may have lost their prominence in contemporary Chinese culture.
Moreover, to correctly understand an artwork, VLMs must first identify the key
elements and then connect these elements into a coherent story.

(2) The Pun Rebus dataset extends beyond the knowledge scope of VLMs. The
relatively low scores observed in the element identification reveal that the
tested VLMs fail to understand the Pun Rebus artworks, leading to 50% of the
key elements being missed in the recognition. The even lower accuracy in the
symbolic matching reflects VLMs’ sparse knowledge of Pun Rebus-related
content, demonstrating their lack of sufficient knowledge and reasoning
ability to transfer the identified key elements into the conveyed meanings.
The substantial historical span of the dataset, combined with the struggling
performance observed in our evaluations, indicates that the cultural and
linguistic content within these artworks extends beyond the training knowledge
of the tested models.

(3) Element recognition versus cultural interpretation limits VLMs. The VLM
with high symbolic matching accuracy, such as GPT-4o, also scores well in
element identification. However, VLMs like Claude 3 Opus score high in element
recognition but struggle with symbolic understanding. For example, as we
showed in Table 1, Claude 3 Opus identifies bok choy in the artwork but fails
to link it to moral integrity, a symbol in Chinese culture due to its similar
pronunciation with ’incorruptible.’ This highlights a critical aspect of VLM’s
performance: translating visual recognition into meaningful cultural
interpretation. In Appendix, we detail 12 distinct mechanisms used in Chinese
culture to derive symbolic meanings from visual elements, including puns,
shapes, numerals, and aliases.

(4) GPT-4o demonstrates superior performance compared to other models.
Notably, GPT-4o largely outperforms other models in our evaluation, including
GPT-4V. This improvement is partly due to enhanced visual recognition
abilities, as evidenced by higher element identification scores achieved by
GPT-4o compared to GPT-4v. Other factors, such as the integration of end-to-
end multimodal learning techniques in GPT-4o, may also lead to a more
effective interpretation of complex visual and textual information. Despite
these notable improvements, the precise factors contributing to the improved
performance of GPT-4o remain unclear to us.

(5) The impact of using Chinese image-text data corpus in the Pre-training of
VLMs. Among the tested models, only the Qwen-VL family publicly announced
substantial Chinese data in their training corpus. Our Pun Rebus dataset is
naturally bilingual, with content rooted in Chinese culture and questions
posed in English. The Qwen-VL-Max achieved the second-highest accuracy in
symbolic matching, only below GPT-4o. Examination of Qwen model responses in
element identification showed that 18.99% of Qwen-VL-Max and 17.90% of Qwen-
VL-Pro responses were in Chinese characters. This language mismatch
contributed to their relatively low scores in element identification, as the
ground truth answers were in English. Appendix includes examples of these
responses with corresponding artwork images. Human inspection further found
that Chinese responses predominantly occurred with images deeply embedded in
Chinese culture, such as traditional ink paintings or fable stories. We
speculate that the Qwen models were exposed to Chinese culture-related image-
text pairs without English translations during their pre-training.
Consequently, the models defaulted to Chinese responses instead of English
when encountering similar elements.

####  5.2.2 Evaluation under Few-shot Settings

We also evaluate the in-context learning ability of models using a 5-shot
prompt on the Pun Rebus dataset. Specifically, we select the best-performing
models from each model family: GPT-4o, Claude 3 Opus, and Qwen-VL-Max. We do
not include Gemini Pro because the currently publicly available API only
supports interleaved images as a few shot prompts but not the multiple image
input as the other model. The results are presented in Table 1. We make two
key observations:

(1) Marginal improvements with five-shot prompting. With five-shot prompting,
we observed slight increases in the symbolic matching performance of GPT-4o
and in the element identification performance of both Claude 3 Opus and
GPT-4o. The prompt directly illustrates what the elements look like and
highlights which elements are important to the conveyed meaning, leading to
improved performance in element identification. However, element
identification is inherently simpler and requires less reasoning compared to
symbolic matching. Symbolic matching is more complex, as the model must
identify the mechanisms to integrate the spotted elements into coherent
stories. The prompts provided answers but did not explain the underlying
mechanisms, resulting in minimal improvement in symbolic matching. In some
cases, the performance is even lower compared to zero-shot settings, as the
model could not understand the reasoning behind the prompts.

(2) Hallucination and Shortcuts Exploitation to the in-context examples. With
Qwen-VL-Max, we observe the performance decreases in all tasks under the five-
shot settings. With human inspection of the element identification responses,
we found that the word "Pheasant" appeared 317 times, approximately 31.35% of
all answers. In our provided prompt, we included an example labeled "Quail."
Biologically, quails belong to the pheasant family. We speculate that the
behavior of the large VLMs is associated with the "lazy learners" phenomenon
discussed in [16], where the VLMs frequently exploit shortcuts in in-context
examples for downstream tasks. This leads the model to incorrectly identify
various elements as "Pheasant," regardless of whether the artwork depicted
humans, flowers, or other animals. These observations suggest that the VLMs
tend to exploit shortcuts to the in-context examples, resulting in the
generation of hallucinated answers that are close to the few-shot examples,
which could be the major reason for the decline in performance.

###  5.3 Human Evaluation and Error Analysis

Figure 4: An example of expression understanding generated by GPT-4o and
Gemini Pro, including the expert review and the expert-provided answer for
this artwork. Errors are highlighted in red.

####  5.3.1 Expert Review on Expression Understanding

Our expert judges reviewed the expression understanding generated by GPT-4o
and Gemini Pro. We randomly selected 50 responses from each VLM, ensuring the
samples maintained the same category distribution as the full dataset. An
example review and the judges’ explanations are shown in Figure 4. Overall,
GPT-4o received an average score of 3.47, while Gemini Pro received an average
score of 3.01 from the expert judges. The expert judges make two key
observations from the reviews:

(1) Reasons of errors in expression understanding. The primary issue is
incorrect recognition or missing salient elements. For example, both models
failed to recognize a persimmon in one artwork, mistakenly identifying it as a
peach, reflecting the challenges in element identification shown in Table 1.
Secondly, even when VLMs correctly identified elements, they often
misunderstood the conveyed meaning. As shown in Figure 4, Gemini recognized
fish but completely missed its pun. Also, Gemini tends to fabricate things
that do not appear in the pun rebus designs. Lastly, in some cases, the VLMs
achieved an expert-level understanding but selected an incorrect option.

(2) Potential bias in VLMs. Experts noted potential bias in the generated
answers. When VLMs fail to recognize an element, they tend to link it to
common symbols in Chinese culture, specifically bats, peaches, pine trees, and
rocks, which are frequently used to represent longevity and good luck. They
often defaulted to associating uncertain elements with these four elements
based on shape similarity. For example, they might interpret long, tree-shaped
elements as pine trees and round-shaped elements as peaches based on shape
similarity. Additionally, VLMs frequently associate the artwork with positive
themes such as happiness, longevity, or wealth. Consequently, both VLMs
performed poorly when interpreting artworks intended to express themes related
to moral integrity or societal harmony.

####  5.3.2 Error Analysis

In this section, we conduct a deeper analysis of the key observations made by
the experts. Our discussion addresses the following three questions:

(1) Is computer vision a bottleneck for understanding artworks? We evaluated
the models on text-only questions, providing only the story name conveyed by
each artwork for symbolic matching. Detailed answers are listed in Appendix.
Each model achieved over 80% accuracy. However, when images were included,
accuracy dropped to below 45% for all models. These results suggest that while
the models can understand the meaning of the story, they struggle to visualize
what the story looks like or is composed of when interpreting the actual
artwork.

(2) What is the model’s preference in understanding? We analyzed the label-
wise performance and the confusion matrix for incorrect symbolic matching
answers for GPT-4o, as detailed in Appendix. GPT-4o achieves the lowest
performance on options related to moral integrity and societal harmony, with
accuracies around 20%, mirroring expert observations. The confusion matrix
shows that the model tends to favor option D, which relates to fecundity,
among the erroneous choices.

(3) Would fine-tuning help? We create a custom GPT-4V to explore. We compiled
68 pieces of artwork with their annotations into a single document and
uploaded it to the ChatGPT web page to build a customized Pun Rebus GPT-4V
model. Since the model could only be accessed through the web page, we
conducted a small-scale evaluation with 120 different artworks. The customized
GPT-4V achieved a symbolic matching accuracy of 66%, demonstrating the
potential benefits of fine-tuning. The model can be accessed through this
link, and we encourage readers to give it a try.

##  6 Related Works

###  6.1 Multi-modal Multicultural Understanding

Recent advancements in VLMs have spurred interest in enabling models to
interpret culturally rich content. Researchers have begun to evaluate cultural
commonsense [14], culturally diverse facts [9, 7], and cultural moral norms
[12] in LLMs. These works discover LLMs have limited culturally specific
knowledge and frequently output culturally biased responses to human prompts.
Some studies on multicultural visual recognition have explored improving
recognition performances for food [11], heritage [4], and clothing [6] in
culturally diverse contexts. However, these works primarily focus on enhancing
cultural understanding within a single modality. A more relevant effort to our
proposed dataset is the MaRVL dataset [10], aiming to evaluate multicultural
reasoning abilities in VLMs.

###  6.2 Computational Pun and Pun Rebus Understanding

Computational pun understanding has been extensively studied in NLP in the
last decade, with efforts made to design language models for pun detection and
comprehension [22, 15]. More recently, researchers have investigated the
abilities of LLMs in understanding puns [18], demonstrating their capability
to recognize and explain puns, although generating humorous puns remains
challenging. However, the understanding of pun rebus, which requires both
visual recognition and language reasoning, has not been extensively studied in
evaluating VLMs. To the best of our knowledge, the closest work related to our
proposed dataset is the humor understanding from the image presented in [5],
which shows that VLMs struggle to recognize the humorous elements of the
visual content.

##  7 Limitations

While our step-by-step error analysis provides valuable insights into the
performance of VLMs on pun rebus understanding, it lacks an in-depth
examination regarding the nuanced mechanisms within pun rebuses that may
influence model performance. For example, we have not analyzed how the
attribution of the elements (e.g., quantities, positions, etc.) in the artwork
affects the models’ reasoning abilities. We plan to continue collaborating
with art historians to annotate each sample in the dataset with mechanism
details and address this analysis in future studies. Additionally, our
database contains a substantial collection of ceramic arts, which are 3D
objects. However, we have only used the front image for testing, thereby
ignoring their 3D characteristics. Addressing this limitation is crucial for a
comprehensive understanding of these artworks. We plan to incorporate the 3D
aspects of these objects in our future studies. Moreover, the expression
understanding results were primarily reviewed by expert judges. While this
ensures a high level of expertise, it is worth incorporating more
crowdsourcing efforts to evaluate VLM’s explanations to understand how
different groups perceive VLM answers. This would further help identify
discrepancies in understandings between experts and non-experts, shedding
light on potential biases in VLM outputs.

##  8 Conclusions

In this work, we offer the Pun Rebus Art dataset and evaluate whether state-
of-the-art VLMs can interpret Chinese culture and artworks. Our findings
reveal that: 1) Current VLMs struggle to spot the salient visual elements in
the Chinese Pub Rebus Arts, though they outperform ordinary humans; 2) Due to
the knowledge gap in cultural understanding, VLMs face challenges in
transferring the spotted elements into their underlying auspicious meaning or
matching the symbolic meanings; 3) We also observe substantial limitations in
VLM’s ability to provide coherent explanations for interpreting Chinese Pun
Rebus Arts. The responses provided by these VLMs often exhibit biases towards
fixed objects and include significant hallucinations; 4) In-context learning
does not effectively guide VLMs to improve their performance in pun rebus art
understanding.

In the future, a promising area of research will be developing effective data
curation to incorporate more diverse and cross-cultural knowledge into the
training and evaluation processes of VLMs. This approach holds promise for
making VLMs more inclusive and universally beneficial, enhancing their ability
to understand and interpret various cultures.

## References

  * [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  arXiv preprint arXiv:2303.08774, 2023.
  * [2] AI Anthropic.  The claude 3 model family: Opus, sonnet, haiku.  Claude-3 Model Card, 2024.
  * [3] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou.  Qwen-vl: A frontier large vision-language model with versatile abilities.  arXiv preprint arXiv:2308.12966, 2023.
  * [4] Federico Becattini, Pietro Bongini, Luana Bulla, Alberto Del Bimbo, Ludovica Marinucci, Misael Mongiovì, and Valentina Presutti.  Viscounth: a large-scale multilingual visual question answering dataset for cultural heritage.  ACM Transactions on Multimedia Computing, Communications and Applications, 19(6):1–20, 2023.
  * [5] Jack Hessel, Ana Marasović, Jena D Hwang, Lillian Lee, Jeff Da, Rowan Zellers, Robert Mankoff, and Yejin Choi.  Do androids laugh at electric sheep? humor “understanding” benchmarks from the new yorker caption contest.  In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 688–714, 2023.
  * [6] Wei-Lin Hsiao and Kristen Grauman.  From culture to clothing: Discovering the world events behind a century of fashion images.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1066–1075, 2021.
  * [7] Xuming Hu, Junzhe Chen, Xiaochuan Li, Yufei Guo, Lijie Wen, Philip S. Yu, and Zhijiang Guo.  Do large language models know about facts?  ArXiv, abs/2310.05177, 2023.
  * [8] Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei.  Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting.  In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12365–12394, 2023.
  * [9] Amr Keleg and Walid Magdy.  Dlama: A framework for curating culturally diverse facts for probing the knowledge of pretrained language models.  arXiv preprint arXiv:2306.05076, 2023.
  * [10] Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott.  Visually grounded reasoning across languages and cultures.  arXiv preprint arXiv:2109.13238, 2021.
  * [11] Weiqing Min, Zhiling Wang, Yuxin Liu, Mengjiang Luo, Liping Kang, Xiaoming Wei, Xiaolin Wei, and Shuqiang Jiang.  Large scale visual food recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  * [12] Aida Ramezani and Yang Xu.  Knowledge of cultural moral norms in large language models.  arXiv preprint arXiv:2306.01857, 2023.
  * [13] Nils Reimers and Iryna Gurevych.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019.
  * [14] Siqi Shen, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Soujanya Poria, and Rada Mihalcea.  Understanding the capabilities and limitations of large language models for cultural commonsense.  arXiv preprint arXiv:2405.04655, 2024.
  * [15] Jiao Sun, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Tagyoung Chung, Jing Huang, Yang Liu, and Nanyun Peng.  ExPUNations: Augmenting puns with keywords and explanations.  In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4590–4605. Association for Computational Linguistics, December 2022.
  * [16] Ruixiang Tang, Dehan Kong, Longtao Huang, and Hui Xue.  Large language models can be lazy learners: Analyze shortcuts in in-context learning.  arXiv preprint arXiv:2305.17256, 2023.
  * [17] Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al.  Gemini: a family of highly capable multimodal models.  arXiv preprint arXiv:2312.11805, 2023.
  * [18] Zhijun Xu, Siyu Yuan, Lingjie Chen, and Deqing Yang.  " a good pun is its own reword": Can large language models understand puns?  arXiv preprint arXiv:2404.13599, 2024.
  * [19] Ni Yibin.  The anatomy of rebus in chinese decorative arts.  Oriental art, 49(3):12–23, 2003.
  * [20] Ni Yibin.  Kan Tu Shuo Ci (Speaking of Ceramics through Pictures).  Zhonghua Book Company, 2008.
  * [21] Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak.  Don’t trust ChatGPT when your question is not in English: A study of multilingual abilities and types of LLMs.  In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7915–7927, Singapore, December 2023. Association for Computational Linguistics.
  * [22] Yichao Zhou, Jyun-Yu Jiang, Jieyu Zhao, Kai-Wei Chang, and Wei Wang.  “the boating store had its best sail ever”: Pronunciation-attentive contextualized pun recognition.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 813–822, Online, July 2020. Association for Computational Linguistics.

##  Appendix A Appendix

###  A.1 Datasheets for Pun Rebus Art Dataset

Motivation of the Dataset. The Pun Rebus Art dataset is designed as a
comprehensive benchmark for exploring the intersection of image analysis,
morphological variation, and phonological elements within the context of
Chinese linguistics and cultural artifacts. This dataset is the result of
extensive efforts to curate a diverse array of historical artwork documents.

Creator of the Dataset. The Pun Rebus Art Dataset was created and collected by
Dr. Ni Yibin, a co-author of this paper.

Composition of the Dataset. Initiated in 1987 by Dr. Ni Yibin, a co-author of
this paper, the dataset’s preparation involved meticulous collection,
annotation, and verification processes that require expert knowledge of
Chinese art, literature, history, and linguistics. The corpus comprises 1,011
captioned images sourced predominantly from globally-renowned Chinese-art-
collecting institutions, including the Palace Museum, the Metropolitan Museum
of Art, and the British Museum. Spanning over two millennia, from the Han
Dynasty (206 BCE – 220 CE) to the 20th century, the dataset encompasses a rich
diversity of more than ten different media types, including paintings,
ceramics, bronzes, sculptures, jade, Cloisonné, lacquerware, and embroidery.
The images of these artworks are stored in the dataset in the Joint
Photographic Experts Group (JPEG) format.

Distribution of the Dataset. The Pun Rebus Art dataset could be accessed via
http://niyibin.org/punrebus/punrebus_main_en.php. The code for reproducing the
results of this paper is available on https://github.com/zhang-tuo-pdf/Pun-
Rebus-Art-Benchmark/tree/main. It is worth noting that the category
information for each data sample is stored in the GitHub link.

Maintenance of the Dataset. The collection of the Pun Rebus Art dataset is
ongoing as we continue to curate it with additional artworks to enhance its
representational diversity. We welcome researchers and enthusiasts interested
in this program to join us in expanding and improving this valuable resource.

Licence of the Dataset. The images and their annotation in this dataset are
subject to the Creative Commons Zero (CC0) license.

###  A.2 Symbolic Imagery Mechanism

In this section, we briefly describe the mechanisms behind pun rebus in visual
artworks. Through our investigation, we have identified and summarized 12
distinct mechanisms that form a symbolic imagery as followings:

Symbolic. Using the images of people/objects in the artwork as symbols.

Pun. Using homophones of names of people/objects in the artwork.

Shape. Using the shape attributes of objects in the artwork.

Length/Size. Using the length or size attributes of objects in the artwork.

Color. Using the color attributes of objects in the artwork.

Figure. Using the names of people/objects in the artwork.

Alias. Using aliases and polyphonic characters for people/objects in the
artwork.

Numeral. Using the quantity of visual elements in the artwork.

Verb. Using verbs triggered by specific actions in the artwork events.

Preposition. Using prepositions triggered by spatial relationships in the
artwork events.

Character. Using pictographic Chinese characters appearing in the artwork.

Loanword. Using borrowed Chinese characters or radicals from the names of
people/objects appearing in the artwork (names of people/objects in the
artwork sound the same and share characters with the intended meaning)

In our future work, we plan to label each sample with its corresponding
mechanism and further investigate the sensitivity of VLMs to each specific
mechanism.

###  A.3 Experiment Details

LVM API Checkpoints. For all models listed in this work, we utilize the latest
model checkpoint available at the time of writing this paper. Specifically,
for GPT-4o, we used gpt-4o-2024-05-13 model; for GPT-4V, we used gpt-4-vision-
preview model. For Gemini model, we used Gemini 1.0 Pro Vision. For Claude 3
model family, we used claude-3-opus-20240229, claude-3-sonnet-20240229, and
claude-3-haiku-20240307. For Qwen-VL model family, we used qwen-vl-plus and
qwen-vl-max.

Computing Infrastructure. All experiments are performed on two computing
servers with ten GPUs. The server is equipped with AMD EPYC 7502 32-Core
Processor and 1024G memory. The GPU is NVIDIA RTX A100. For models with API
access, we just run the inference with CPUs.

Evaluation Prompts. For symbolic matching task, we used the following prompt
for all models:

⬇

This is a traditional Chinese artwork that likely conveys its ideas, thoughts,
or wishes through symbolic, punning, shape, color, figure, numeral, verb,
preposition, character, loanword or alias through the artwork. \

Carefully analyze the visual elements present in the artwork and select the
option from the list below that best aligns with its conveyed meaning: \n \

A. Longevity and Good Health \n \

B. Happiness,Joy, Good Luck \n \

C. Prestige, Promotion, and Good Exam Results \n \

D. Fecundity, Harmonious Relationship and Family \n \

E. Wealth or Prosperity \n \

F. Moral Integrity, Eremitism \n \

G. Peace and Protection from Evil, Societal Harmony \n \

You must make a selection using the option above in your response. Your
response should start with the chosen letter that best matches the word’s
meaning based on a precise and sound justification for your selection. Please
do not include your justification in your response.

For element identification, we used the following prompt for all models:

⬇

Please analyze the provided image carefully to identify key visual elements.
Focus on components that traditionally have symbolic meaning in the cultural
context from which the artwork originates.\

Look for elements that might represent ideas, virtues, or wishes, especially
those commonly found in nature or historical motifs.\

For instance, in Chinese culture, certain animals and plants are known to
symbolize specific messages when depicted in art. \

Based on these principles, identify the primary visual elements in the image
that are likely used to convey a message or a wish.\

Please list the discernible elements present in the image, excluding any
assumptions about elements not clearly visible.\

Pleas answer the question in one line with the following format strictly: name
of element A, name of element B, etc

For expression understanding, we used the following prompt for all models:

⬇

This is a traditional Chinese artwork that likely conveys its ideas, thoughts,
or wishes through symbolic, punning, shape, color, figure, numeral, verb,
preposition, character, loanword or alias through the artwork. \

Carefully analyze the visual elements present in the artwork and select the
option from the list below that best aligns with its conveyed meaning: \n \

A. Longevity and Good Health \n \

B. Happiness,Joy, Good Luck \n \

C. Prestige, Promotion, and Good Exam Results \n \

D. Fecundity, Harmonious Relationship and Family \n \

E. Wealth or Prosperity \n \

F. Moral Integrity, Eremitism \n \

G. Peace and Protection from Evil, Societal Harmony \n \

You must make a selection using the option above in your response. Your
response should start with the chosen letter that best matches the word’s
meaning, followed by a precise and sound justification for your selection.

For text-only understanding evaluation, we used the following prompt for all
models:

⬇

f"What does the word \"{chinese_word}\" want to represent in Chinese culture?
Please select the option from the list below that best aligns with its
conveyed meaning: \n \

A. Longevity and Good Health \n \

B. Happiness,Joy, Good Luck \n \

C. Prestige, Promotion, and Good Exam Results \n \

D. Fecundity, Harmonious Relationship and Family \n \

E. Wealth or Prosperity \n \

F. Moral Integrity, Eremitism \n \

G. Peace and Protection from Evil, Societal Harmony \n \

You must make a selection using the option above in your response. Your
response should start with the chosen letter that best matches the word’s
meaning, followed by a precise and sound justification for your selection.

To make sure the output answers are in a unified format for scoring, we have
to made some slightly changes in words for the prompt that we used in Qwen
model family. The detailed prompt that we used in experiment are listed in our
GitHub link.

Figure 5: The example questionnaire for an artwork image to the crowd-workers.
The first question is related to the symbolic matching task, and the second
question is related to the element identification task.

Questions to the Crowd-workers. Figure 5 shows an example questionnaire for an
artwork image to our recruited crowd-workers. We do not record any crowd-
worker IDs in our experiment records. The average time of for each human
evaluation is around 90 minutes, and we pay each crowd-worker $30 each hour.
Crowdworking studies involving standard computer vision corpora (with no
personal disclosures) do not require IRB review according to our institution’s
guidelines. Although we are not legal experts and this is not legal advice,
this opinion is based on United States federal regulation 45 CFR 46, under
which this study qualifies as exempt.

###  A.4 Further Analysis on Experiment Results

Text-only Evaluation Performance. We evaluated the models on text-only
questions, providing only the story name conveyed by each artwork for symbolic
matching. We used the accuracy as the evaluation metrics, the same as we used
for symbolic matching task with artwork images in the main paper. The
evaluation results are listed in Table 2.

| Text-only Symbolic Matching
---|---
| Accuracy ($\uparrow$)
Random Choice | 14.29%
GPT-4o | 88.55%
GPT-4V | 87.47%
Gemini Pro | 85.06%
Claude 3 Opus | 85.87%
Claude 3 Sonnet | 85.60%
Claude 3 Haiku | 86.93%
Qwen-VL-Max | 84.00%
Qwen-VL-Plus | 81.87%

Table 2: Evaluation results for the text-only symbolic matching tasks among
various VLMs. Bold results are best for zero-shot evaluation.

Error Examples by Qwen-VL. As we mentioned in the Section 5.2, we observed the
language mismatch in the response from Qwen-VL model family. We also observed
the hallucination in the responses from Qwen-VL Max model under the 5-shot
settings. In Figure 6, we provide several error examples to illustrate them.

Figure 6: Several element identification examples by Qwen-VL model family. Red
text color indicates the wrong identification results, and Blue text color
indicates language mismatch responses.

Detailed Analysis on GPT-4o Results. We analyzed the category-wise accuracy
performance and the confusion matrix of incorrect symbolic matching answers
for GPT-4o, as shown in Figure 7 and Figure 8, respectively. The results
indicates that GPT-4o has the most confidence on the option D, which is
related to fecundity, when reading the pun rebus artwork, as it achieves the
highest accuracy for this category and frequently mislabeling other answers as
option D. Also, GPT-4o made its lowest accuracy on the option F and G, which
are related to the moral integrity and societal harmony. The confusion matrix
suggests that GPT-4o has very sparse knowledge regarding option F, as the
error distribution for this category is nearly uniform compared to the errors
for other options.

Figure 7: The category-wise accuracy performance of symbolic matching answers
for GPT-4o. Figure 8: The confusion matrix of incorrect symbolic matching
answers for GPT-4o.


# High-sensitivity low-noise photodetector using large-area silicon
photomultiplier

Takahiko Masuda***These authors contributed equally to this work. Research
Institute for Interdisciplinary Science, Okayama University, Okayama 700-8530,
Japan  Ayami Hiramoto∗ Research Institute for Interdisciplinary Science,
Okayama University, Okayama 700-8530, Japan  Daniel G. Ang  Department of
Physics, Harvard University, Cambridge, Massachusetts 02138, USA  Cole
Meisenhelder  Department of Physics, Harvard University, Cambridge,
Massachusetts 02138, USA  Cristian D. Panda  Department of Physics, University
of California, Berkeley, California 94720, USA  Noboru Sasao  Research
Institute for Interdisciplinary Science, Okayama University, Okayama 700-8530,
Japan  Satoshi Uetake  Research Institute for Interdisciplinary Science,
Okayama University, Okayama 700-8530, Japan  Xing Wu  Department of Physics,
Harvard University, Cambridge, Massachusetts 02138, USA  James Franck
Institute and Department of Physics, University of Chicago, Chicago, Illinois
60637, USA  David P. DeMille  James Franck Institute and Department of
Physics, University of Chicago, Chicago, Illinois 60637, USA  John M. Doyle
Research Institute for Interdisciplinary Science, Okayama University, Okayama
700-8530, Japan  Department of Physics, Harvard University, Cambridge,
Massachusetts 02138, USA  Gerald Gabrielse  Center for Fundamental Physics,
Northwestern University, Evanston, Illinois 60208, USA  Koji Yoshimura
Research Institute for Interdisciplinary Science, Okayama University, Okayama
700-8530, Japan

###### Abstract

The application of silicon photomultiplier (SiPM) technology for weak-light
detection at a single photon level has expanded thanks to its better photon
detection efficiency in comparison to a conventional photomultiplier tube
(PMT). SiPMs with large detection area have recently become commercially
available, enabling applications where the photon flux is low both temporarily
and spatially. On the other hand, several drawbacks exist in the usage of
SiPMs such as a higher dark count rate, many readout channels, slow response
time, and optical crosstalk; therefore, users need to carefully consider the
trade-offs. This work presents a SiPM-embedded compact large-area photon
detection module. Various techniques are adopted to overcome the disadvantages
of SiPMs so that it can be generally utilized as an upgrade from a PMT. A
simple cooling component and recently developed optical crosstalk suppression
method are adopted to reduce the noise which is more serious for larger-area
SiPMs. A dedicated readout circuit increases the response frequency and
reduces the number of readout channels. We favorably compare this design with
a conventional PMT and obtain both higher photon detection efficiency and
larger-area acceptance.

##  1 Introduction

Optical sensors are essential devices in a wide range of scientific research.
Various characteristics are required for different experimental systems. In
the case of advanced researches, weak-light detection at a single photon level
is often required, such as fluorescence due to a relatively small number of
atoms and molecules. Among various highly sensitive photon sensors with
single-photon detection capability, silicon photomultipliers (SiPMs) have
recently emerged as high-performance alternative, with significant development
and expanding list of applications[1, 2, 3, 4, 5, 6]. Avalanche photodiodes
(APDs) have high quantum efficiency but too much noise for low photon counts,
photomultiplier tubes (PMTs) have low noise but low quantum efficiency. SiPMs
fill in the gap.

A SiPM consists of an array of pixelated microcells of silicon APD. All
microcells are reverse biased over the diode breakdown voltage. The excess of
the bias voltage from the breakdown voltage which is called overvoltage
determine the SiPM performance such as avalanche gain and dark count rate
(DCR). Since each microcell is sensitive to a single photon and has well-
uniform avalanche gain among the microcells, the sum signal of all microcells
contains photon number information. In particular, SiPMs are attractive
alternatives to PMTs in many cases due to their higher photon detection
efficiency (PDE), lower operation voltage, and magnetic-field
insensitiveness[7, 8, 9]. Furthermore, SiPMs have extended the spectral range
from near-infrared[10], visible, to ultraviolet[11]; currently it covers a
similar spectral region as PMTs.

Since only small SiPMs were available at the beginning of their development,
the main applications were those that did not need a large sensitive area or
those that required compact detectors. Examples include scintillation
detectors with fine grain size[12, 13], or fiber readouts[14, 15]. Recent
developments have expanded the sensitive area and now SiPMs as large as or
even larger than PMTs are available. This enables us to use a SiPM for more
general applications such as a replacement of a PMT. This is especially useful
for spectroscopy of atoms and molecules where the emission signal tends to
have a large etendue and it is difficult to secure sufficient geometrical
acceptance with small sensors. In comparison to PMTs, however, SiPMs have a
number of disadvantages that need to be addressed: larger DCR[16], many
readout channels, slower response time, and optical crosstalk (OCT)[17, 18].

We develop a photon detector based on a large-area SiPM, overcoming the
disadvantages of SiPMs, as a direct replacement for a conventional PMT for the
ACME experiment[19, 20]. The experiment aims to measure the permanent electric
dipole moment of the electron[21], by performing spin precession in a
molecular beam, where the phase of the precesion is read out using laser-
induced fluorescence (LIF) which emits photons at 512 nm. The goal is to
develop a SiPM based photon detector with higher visible PDE and improve
photon number resolution. These lead to a higher efficiency in measuring
fluorescence photons, increasing experiment sensitivity.

Here, we describe the design and performance of the developed module. We
utilized a large-area array-type SiPM to maximize the sensitive area. To
reduce the higher DCR, the SiPM sensor is cooled below the freezing point of
water while the module casing is kept at room temperature. While small SiPM
chips or arrays that equip similar cooling mechanisms are commercially
available or reported[22], our design enables good cooling performance for a
large-area SiPM with a simplified cooling design. Furthermore, custom optical
filters are used to minimize the OCT, and a dedicated readout circuit has been
developed to extend the response frequency and reduce the number of readout
channels.

We did a comprehensive characterization of the modules and performed a test
experiment using an actual molecular beam. As well as the high PDE, the
modules show good uniform single photoelectron gain among over 100 channels
and noise levels almost consistent with the shot noise limit; all these
parameters exceed those of conventional PMTs. Sufficient cooling power is
achieved using a simple mechanism which enables cooling the module down to
-20∘C, where the DCR is less than 3% of that at room temperature. Finally,
long-term stability of these parameters was confirmed in a high photon-yield
application.

In this paper, we present the design of the module in section 2. Section 3
reports the detailed characterization of the module on a test bench, then the
molecular fluorescence measurement is described in Section 4. Section 5
presents the summary and conclusion of the study.

##  2 Instruments

###  2.1 Module design

The SiPM array chip used is a commercial surface mount 16-channel array
(Hamamatsu Photonics K.K. S13361-6075NE-04). It was selected because of its
high PDE (45% for wavelengths around 500 nm) and the large sensor area. The 16
channels are aligned as a $4\times 4$ matrix with a gap between channels of
0.2 mm, covering a total of $25\times 25$ mm square area. Each channel has
$6\times 6$ mm square sensitive area that consists of 6336 $75\times 75$
$\mu$m microcells. The SiPM has a DCR of typical $\sim$2 M count per second
(cps) for one channel at room temperature. This is quite higher than a PMT
whose typical dark count is $\sim$kcps for a similar sensitive area[23]. The
OCT in this SiPM is typical 13%.

Figure 1: a) Cutout view of the module. Electrical cables between PCBs are
omitted. b) Picture of the inside the vacuum chamber. c) Front view of the
mounted module.

A schematic and pictures of the module are shown in Fig. 1. To mitigate the
DCR, the SiPM is housed in an aluminum vacuum chamber and is cooled down below
the freezing point of water using a thermoelectric cooler element (TEC,
Thorlabs, Inc. TECH4). The SiPM is soldered onto a 2.0-mm-thick printed
circuit board (PCB) made of aluminum instead of a usual glass epoxy to improve
the thermal conductance between the SiPM and the TEC. Both surfaces of the TEC
are attached to the PCB and the chamber wall using a thin layer of grease
(Apiezon N). The SiPM and TEC heat load is transferred to a water-cooled
heatsink on the outside of the chamber through the chamber wall itself. The
water temperature is stabilized by a thermoelectric chiller to $\sim
20\,^{\circ}\mathrm{C}$ sufficiently high to avoid dew condensation around the
chamber. One negative temperature coefficient thermistor is mounted on the PCB
and the PCB temperature is stabilized at a level of $0.01\,^{\circ}\mathrm{C}$
by a temperature controller (Arroyo instruments 7154-05-12). The PCB
temperature is also monitored by two additional out-of-loop platinum
temperature sensors on the PCB and a multimeter (Keithley DAQ6510 with 7710).

Three wavelength-selective optical filters are placed along the photon path to
maximize the signal to noise ratio (SNR) and minimize OCT. Due to crosstalk
(including OCT and delayed crosstalk[17]) and afterpulse, a SiPM usually
counts more than just the incident photons with a certain probability $p$. It
effectively increases the single photoelectron gain from one pixel discharge
and degrades the photon-counting resolution. The degradation is expressed as
an excess noise factor. It is known that the gain is increased by a factor of
$1+p$ and the excess noise factor is also $1+p$ by ignoring small $p^{2}$
terms[24, 25]. We have described the ideal filter configuration for the OCT
suppression[26], and we use a similar setup here. Since the signal wavelength
is 512 nm in our case, we use bandpass filters (BPFs) whose transmission peaks
are around that wavelength: an interference green BPF (Semrock FF01-520/70)
whose center wavelength is 520 nm and whose full width at half maximum is 70
nm, a 2-mm-thick absorptive green BPF (SCHOTT BG39), and a 1-mm-thick
absorptive green BPF (SCHOTT BG40) from the outside to the inside. The
innermost filter is directly glued on the SiPM surface using transparent
silicone potting (Momentive Performance Materials TSE3032). The resulting $p$
was $\sim 0.1$ including crosstalk and afterpulse.

On the backside of the vacuum chamber, we placed bias and readout PCBs. Since
the heat load of the readout circuit is not negligible ($\sim$5 W), the
readout circuit board is also attached by a thermal conductive sheet (3M
5580H) to the backplate which is directly in contact with the heatsink. Signal
and bias lines are connected via thin 50-$\Omega$ coaxial cables (Hirose,
U.FL) from these PCBs to the SiPM PCB through the vacuum feedthrough PCB. All
electronics are contained in an aluminum housing.

###  2.2 Readout circuit

For effective use of large-area array type SiPMs, the readout circuit needs to
be carefully considered for response speed and the number of readout channels.
In particular, the response speed would be an issue for a SiPM with a large
sensor area because its capacitance is generally proportional to its surface
area. There are several methods to solve these issues.

The number of readout channels can be reduced by electrically ganging several
channels into one output. Suitable ways of ganging depend on requirements and
constraints such as response speed, SNR, and cost. Passively connecting SiPMs
in parallel is the simplest way but the response speed becomes slow because
the net capacitance increases linearly with the number of SiPMs. Series
connection of SiPMs can reduce the net capacitance as the inverse of the
number of connected SiPMs; however, it can only be used for pulse-like signal
detection and not for DC-like signals due to the AC coupling. It also requires
higher bias voltage proportional to the number of SiPMs, and the output signal
charge is reduced. Recently a hybrid connection readout has been proposed and
used for some experiments[27, 6]. The hybrid connection mitigates the high
bias voltage in the series connection but still cannot be used for DC-like
signals; thus, this method is effective for sparse pulse-like signals such as
scintillation detectors. Another type of compensation method for the slow
response speed is a pole-zero cancellation[28] (PZC). This method uses a
linear filter for shortening the slow tail by summing the original signal and
the derivative pulse in time domain. Although this scheme needs careful
parameter tuning to cancel the poles of the SiPM, it can shorten the slow tail
component while keeping the DC-like signals intact.

We adopted the PZC technique to extend the response speed. We also developed
an active summing circuit connected to the PZC directly. The circuit is shown
in Fig. 2. The circuit is for one 16-channel array. Since the breakdown
voltage of each channel is uniform enough, we simply use one DC supply
(Keysight E36105B) to supply bias voltage for all channels. The individual
channel output is transmitted to the backside amplifier board through coaxial
cables and the vacuum feedthrough board, and then is simply terminated by a 51
$\Omega$ resistor. The voltage signal across the termination resistor is
buffered and sent to the PZC part.

The PZC outputs from all 16 channels are summed up by one operational
amplifier to reduce the number of readout channels. The summed signal is also
shaped by a following three-pole Bessel filter with a -3 dB bandwidth of 5 MHz
to smooth the output signal so that the SNR is improved and the digitizing
sampling rate can be reduced.

For diagnostic purposes, we extract signals from individual channels before
the PZC part and send the one-channel signal through a multiplexer. The bias
current is monitored at the shunt resistor and an instrumentation amplifier.

Figure 2: Circuit diagram including a bias supply and readout. The power
supply lines, bypass capacitors for operational amplifiers, and other less
relevant components are omitted. $\mathrm{V_{b}}$: Bias supply; M1: LMH6629;
M2: ADA4899; M3: AD620; M4: AD820.

Single photoelectron pulse shapes at an overvoltage of 3.0 V are shown in Fig.
3. The signal shape before the PZC part has a long tail whose decay constant
is $\sim$100 ns. The PZC extends the -3 dB bandwidth of the response frequency
from 2 MHz to 8 MHz. This value is better than the intrinsic response
frequency of the SiPM. Since the SiPM’s internal capacitance and resistance
were measured in a separate setup to be 1.4 nF and 30 $\Omega$, respectively,
it implies that even with an ideal transimpedance amplifier with zero input
impedance, the resulting response frequency would be only 3.8 MHz. As shown in
Fig. 3b, the single photoelectron pulse can also be clearly seen after the PZC
even though 16 channels are summed and so the noise is also overlayed. The
pulse shape is clearly shortened by the PZC; the decay constant is around 25
ns. The SNR is improved after the 5-MHz Bessel filter as shown in Fig. 3c.
This circuit enables a pulse counting method as well as a pulse integration
method owing to the shortened pulse shape. Since our required bandwidth is 5
MHz, we tuned the PZC parameter as a bit faster than 5 MHz and the pulse is
shaped by the Bessel filter. One can change the PZC parameter to make it much
faster. Since it increases the noise level, one needs to tune based on the
required response speed and noise level.

Figure 3: Single photoelectron pulse shapes at three points of the readout
circuit. Each offset is subtracted to zero. a) individual channel output
before the PZC part. b) summed signal of 16 channels. c) Bessel filter output.

##  3 Characterization

In this section, we describe the basic characterization of the SiPM module on
a test bench. We constructed 10 modules in total. With these modules, we
measured 1) the single photoelectron gain as an integral area of a single
photoelectron pulse, 2) the temperature dependence of the breakdown voltage,
3) the temperature dependence of the DCR, 4) the linearity of the output
signal, and 5) the excess noise factor and the relative PDE. We also measured
the long term stability of the SiPM module while continuously injecting
photons. The module was set in a light-tight box to block ambient light, and a
thermo-stabilized 515 nm laser diode (OSRAM PLT5) was used as a light source
for the linearity, excess noise factor, and long term stability measurements.
A scroll pump was used for keeping vacuum inside the SiPM housing around 1 Pa,
the water temperature of the heatsink was controlled to be 20∘C, and the TEC
was used in tandem with a PID controller to keep the thermally isolated parts
to be a nominal temperature of -15∘C, unless otherwise stated. In the
following measurements, a 2-channel oscilloscope (NI PXIe-5164) was used to
record waveforms with a sampling rate of 100 MHz and a -3 dB bandwidth of 20
MHz. The breakdown voltage of each SiPM array was measured before the
characterization test. We applied 3.0 V overvoltage, which is the recommended
value by the manufacture[29] to obtain enough gain factor while keeping the
affects from the afterpulse, OCT, and DCR acceptable.

Single photoelectron gain
Dark count noise was recorded to obtain a single photoelectron waveform of the
summed and filtered signal. A single-photon area is defined as an integral of
the waveform (Fig. 3(c)) between -100 ns and 200 ns from the leading edge, and
it is obtained in a unit of pV$\cdot$s. A single-photon area distribution is
obtained from a 12.5 ms trace, where the DCR at -15∘C is roughly 20 kcps for
each channel. The typical value is measured to be around 175 pVs for the
summed signal.

We also measured the single-photon area of an individual SiPM channel in a
similar manner using the multiplexer output. A Gaussian fitting of the single-
photon area distribution gives the mean value. Figure 4(a) shows a
distribution of the single-photon area of each channel. Although the SiPM
module does not have a bias adjustment system for each channel of the SiPM
array, the deviation of the single-photon area is only 3% owing to very
similar breakdown voltages among channels. The effect of the deviation to the
photon counting resolution is less than 0.1%, and it is negligible compared
with the effects of the crosstalk and the afterpulse ($\sim$7%), which are
expressed in the measurement of the excess noise factor.

Temperature dependence of the breakdown voltage
Figure 4(b) shows the measured breakdown voltage as a function of temperature
at the SiPM-mounted PCB, where a clear linear relationship is observed. Since
the breakdown voltage is known to be linear to the SiPM chip temperature[29],
this result suggests that the SiPM chip is also cooled down as a linear
function of the temperature at the PCB down to -20∘C.

Figure 4: a) Single photoelectron area of all the channels. The values
$\mu_{G}$ and $\sigma_{G}$ are the mean and the standard deviation obtained by
a Gaussian fitting, respectively. The effect of the deviation to the photon
counting resolution is less than 0.1%. b) Relation between temperature at the
SiPM-mounted PCB and the breakdown voltage fit by a linear function, where the
parameters $p_{0}$ and $p_{1}$ are the intercept and the slope, respectively.

Dark count rate
The DCR of the individual channels is measured by changing the TEC operation
temperature every 5∘C between -20∘C and 0∘C. A 12.5 ms trace was recorded for
each channel using the multiplexer output. We set the threshold as 1.5 mV over
the baseline, and peaks exceeding the threshold are counted as dark counts.
Figure 5 shows the distribution of the DCR of all channels at -15∘C and the
temperature dependency of the DCR, which can be approximated by an exponential
function[29]. In this figure, the mean value of the DCR of all the channels is
plotted. The DCR at -15∘C is suppressed to 3% of that at room temperature,
which is extrapolated using a function shown in Fig. 5.

Figure 5: a) Histogram showing DCR of all the channels at -15∘C. b)
Temperature dependency of the DCR. The DCR at low temperature is suppressed as
expected.

Signal linearity
The output signal linearity was measured using the laser diode. Pulsed light
was injected repeatedly by modulating the laser current using an arbitrary
waveform generator (siglent SDG2042X) to measure the deviation of the measured
number of photons. The light from the laser diode is reduced by more than
three orders of magnitude and adjusted using ND filters. We injected a 400 kHz
square pulse whose envelope is a triangle as shown in Fig. 6. Each 2.5 $\mu$s
cycle is called a bin, and the number of photons in each bin is measured. The
number of photons in a bin was around 8000 photons at the peak of the triangle
envelope.

Figure 7 shows a comparison of the number of photons with and without an ND16
filter in front of the SiPM module. Each point in the plot corresponds to the
number of photons at each bin. A linear fit is applied to a low yield region,
and the residual from the fit result is shown at the bottom of Fig. 7. The
linearity is within $\pm 1\%$ up to around 6000 photons, and we observe a
saturation of the output signal above it. This saturation is not due to the
SiPM but comes from the saturation of gain in the readout circuit. Therefore,
we can reach better linearity by reducing the readout circuit gain, if
necessary, while the linearity is good enough for our use since the expected
number of photons is around 1000 at the maximum.

Figure 6: a) Overall envelope of the square pulses created by a 515 nm laser
diode. b) Zoomed view of the pulse structure corresponding to the range from
5.00 ms to 5.02 ms in the left figure. Figure 7: a) Comparison of the number
of photons at each bin, with and without an ND16 filter in front of the SiPM
module. b) Residual from a linear fitting. The linearity is within $\pm 1\%$
up to around 6000 photons.

Excess noise factor and the relative PDE
We can measure the excess noise factor using the same setup with the
measurement of the signal linearity. When the injected light has a Poisson
distribution, it is known that the Fano factor of the number of measured
photons taking into account the effective gain also becomes $1+p$, which is
the same expression as the excess noise factor[25, 30]. Here, the effective
gain is calculated by multiplying the single-photoelectron gain by $1+p$, and
we assumed $p=0.1$ in the following measurement. The Fano factor ($F$) can be
measured as $F=\sigma_{N}^{2}/\mu_{N}$, where $\sigma_{N}$ and $\mu_{N}$ are
the standard deviation and mean value of the number of photons obtained from
1000-shot waveforms (Fig. 6), respectively. The number of photons is
calculated by dividing the waveform integral by the effective gain at each
bin. The number of photons in a bin was set to be 1000 photons at the peak.

Figure 8 shows an example of the measured Fano factor of a module. The average
value of $F$ for the 10 modules is $F\sim 1.07$. This result is consistent
with the assumption of $p\sim 0.1$, and the excess noise factor obtained from
this measurement is smaller than those of typical PMTs, $\sim$1.2[31]

Figure 8: Example of the Fano factor at each bin. The average value for the 10
modules is $F\sim 1.07$. The gray plot shows the signal envelope. The Fano
factor in the bins without light injection are not estimated properly because
they are strongly affected by fluctuation of the signal baseline.

Since we injected a certain number of photons repeatedly in the above
measurement, the relative PDE can be obtained by comparing the number of
measured photons between the 10 modules. We found that the relative photon
detection efficiencies are consistent within $\pm$5%.

Long term operation
Long-term measurements of detector performance were carried out over seven
weeks with short breaks due to other tests. The purpose of this operation is
to test the stability of the single photoelectron gain and the PDE of the SiPM
module. In this measurement, we used a 4-channel oscilloscope (Tektronix
MSO64) with a sampling rate of 125MHz and a -3 dB bandwidth of 20 MHz, and a
10 ms waveform trace was recorded every one minute. The number of photons in a
bin at the peak was set to 1000. Figure 9 shows the single photoelectron area
and the signal yield during the long term operation. The signal yield is
obtained as the total number of photoelectrons in 10 ms. They are stable
within 1% over seven weeks, while the integrated number of detected photons is
about 1.7$\times 10^{14}$. In the case of PMTs, output current could change by
around 10% after detecting this amount of photons due to damage of the last
dinode[23], while the deviation of the SiPM module efficiency remained within
$\pm$1% for the entire duration of the test.

Figure 9: a) Stability of the single photoelectron gain during the long term
operation. b) Signal yield stability. Each red band corresponds to a $\pm 1\%$
region, and they are stable within 1% over seven weeks.

##  4 Molecular fluorescence measurement

In this section, we show an example of the SiPM module application to LIF
measurements of molecules. Specifically, we used the setup of the ACME II
experiment[20] and observed the fluorescence of thorium monoxide (ThO)
molecules. ThOs initially prepared in the Q state were excited with a laser
resonant with the Q-I transition. We use the detector to count fluorescence
photons with a wavelength of 512 nm from the decay of the I to X state.

The experimental setup is shown in Fig. 10. The ThO molecules were produced in
the leftmost chamber. A ThO2 pellet was mounted inside the chamber. A pulsed
YAG laser ablated the pellet and ThO molecule gas was formed. The gas was
cooled down by 17 K neon buffer gas, forming a cryogenic molecular beam mostly
in the ground (X) electronic state[32]. The molecules in the beam were then
excited to the Q state using Stimulated Raman Adiabatic Passage (STIRAP) via
X-C-Q transitions [33] and were collimated by an electrostatic molecular lens
to the readout region downstream[34]. The molecules in the Q state were
resonantly excited to the I state by a 746 nm continuous-wave laser (M Squared
SolsTis), which decays quickly (lifetime $115\pm 4$ ns[35]) to the ground
state X, emitting 512 nm photons. The photons were collected by an optical
lens doublet[36] and guided to the SiPM or PMT (Hamamatsu photonics K.K.
R7600-300U) through a 16-mm-diameter cylindrical quartz light pipe.

Figure 10: Schematic view of the ThO beam line setup. ThO molecules are
produced in the cryogenic buffer gas cell at the leftmost part of the beam
line, then the molecules go rightward.

We replaced the SiPM with the PMT to compare the absolute signal detection
rate. The PMT had the same interference bandpass filter in front of the
window. The molecular beam intensity was monitored by another equivalent PMT
set which was mounted on another light pipe. The PMT output was amplified by a
factor of 25 through a preamplifier (Stanford Research System SR445A) twice
and was shaped by a 5 MHz low pass filter. Both signals were digitized by an
oscilloscope (NI PXIe-5162) with a 100 MHz sampling rate. The SiPM was
operated at an overvoltage of 3.0 V at the temperature of -20 ∘C. The applied
high voltage of the PMT was -800 V. The gain of both the SiPM and the PMT were
calibrated just before the measurement.

The signal count rates are shown in Fig. 11. Both signals were averaged 100
shots and smoothed with a sliding time window of 100 ns. The module has a
higher detection rate than the PMT by a factor of more than 3. Note that the
PMT detection rate was scaled by the beam intensity monitor so that both
detection rates can be directly compared. The improvement of the photo
detection efficiency itself is at a level of a factor of $\sim$2.3. The PMT
has an effective area of 18 $\times$ 18 mm square, and our SiPM chip has an
effective area of 24 $\times$ 24 mm square. The remaining factor seems to be
covered by the effective area and alignment, since the SiPM and PMT surfaces
were placed slightly far from the lightpipe end. This demonstrates an
important improvement for the next generation of the ACME experiment.

Figure 11: Molecular signal count rate. The PMT count rate is scaled by the
beam intensity monitor. The spike noises at 250 $\mu$s and 5250 $\mu$s are
likely correlated to 200 Hz flashlamp operation in the ablation YAG laser.

##  5 Summary

The recent development of a large-area silicon photomultiplier array enables
an upgrade from a conventional photomultiplier tube to improve the photon
detection efficiency. We developed a SiPM-array embedded photon detector that
mitigates the disadvantages of SiPMs. It uses cooling of the SiPM array to
suppress DCR. The readout circuit reduces the number of channels from 16 to 1
with low enough baseline noise to distinguish single photoelectron pulses. It
also provides frequency response from DC to above the intrinsic bandwidth of
the SiPM.

We performed a comprehensive characterization of the modules. The uniformity
of the single photoelectron gain and PDE are better than 3% and 5%,
respectively. The module has a good cooling capability and the resultant DCR
at -15∘C is successfully suppressed to 3% of that at room temperature. We also
confirmed that the output signal linearity is within $\pm$1% up to around 6000
photons per $\sim$1 $\mu$s. The excess noise factor was measured to be $F\sim
1.07$. Finally, the long-term stability test shows the single photoelectron
gain and the PDE were stable within 1% over seven weeks.

These studies demonstrate that a replacing PMT with large-area SiPM can be
done in a robust way that takes advantage of the increased PDE for general
applications such as spectroscopy of atoms and molecules as shown in this
research.

## Funding

Okayama University (RECTOR program); Japan Society for the Promotion of
Science (KAKENHI JP20KK0068, JP21H01113, JP21J01252); Matsuo Foundation;
National Science Foundation; Gordon and Betty Moore Foundation; Alfred P.
Sloan Foundation.

## Acknowledgments

We would like to thank Z. Lasner for initial considerations of using SiPMs and
discussion about the characteristics, and N.R. Hutzler for the suggestion of
the optical crosstalk suppression. We also appreciate T. Kobayakawa, D.
Lascar, Z. Han, P. Hu, and S. Liu for helping the development.

## References

  * [1] Eva Vilella and Angel Diéguez.  Dynamic range extension of SiPM detectors with the time-gated operation.  Optics Express, 22(10):12007, 2014.
  * [2] Tianqi Zhao, Yu Peng, Quanlong Miao, Baicheng Li, Kun Liang, Ru Yang, and Dejun Han.  One-dimensional single-photon position-sensitive silicon photomultiplier and its application in Raman spectroscopy.  Optics Express, 25(19):22820, 2017.
  * [3] Mehrab N. Modi, Kayvon Daie, Glenn C Turner, and Kaspar Podgorski.  Two-photon imaging with silicon photomultipliers.  Optics Express, 27(24):35830, 2019.
  * [4] Frank Simon.  Silicon photomultipliers in particle and nuclear physics.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 926:85–100, 2019.
  * [5] Stefan Gundacker and Arjan Heering.  The silicon photomultiplier: fundamentals and applications of a modern solid-state photon detector.  Physics in Medicine & Biology, 65(17):17TR01, 2020.
  * [6] Chanhoon Chung, Theresa Backes, Clemens Dittmar, Waclaw Karpinski, Thomas Kirn, Daniel Louis, Georg Schwering, Michael Wlochal, and Stefan Schael.  The Development of SiPM-Based Fast Time-of-Flight Detector for the AMS-100 Experiment in Space.  Instruments, 6(1):14, 2022.
  * [7] Kei Wagatsuma, Kenta Miwa, Muneyuki Sakata, Keiichi Oda, Haruka Ono, Masashi Kameyama, Jun Toyohara, and Kenji Ishii.  Comparison between new-generation SiPM-based and conventional PMT-based TOF-PET/CT.  Physica Medica, 42(October):203–210, oct 2017.
  * [8] Claudio Piemonte and Alberto Gola.  Overview on the main parameters and technology of modern Silicon Photomultipliers.  Nuclear Instruments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 926(August 2018):2–15, 2019.
  * [9] Michael G. Giacomelli.  Evaluation of silicon photomultipliers for multiphoton and laser scanning microscopy.  Journal of Biomedical Optics, 24(10):1, oct 2019.
  * [10] Yusei Tamura, Yoshihito Suzuki, Kenji Makino, Takuya Fujita, Shigeyuki Nakamura, Koei Yamamoto, Tomokazu Kurabayashi, and Takashi Baba.  Development of InGaAs MPPC for NIR photon counting applications.  In Michel J. Digonnet and Shibin Jiang, editors, Optical Components and Materials XV, page 35. SPIE, feb 2018.
  * [11] Fabio Acerbi, Giovanni Paternoster, Massimo Capasso, Marco Marcante, Alberto Mazzi, Veronica Regazzoni, Nicola Zorzi, and Alberto Gola.  Silicon Photomultipliers: Technology Optimizations for Ultraviolet, Visible and Near-Infrared Range.  Instruments, 3(1):15, 2019.
  * [12] Jung Yeol Yeom, Ruud Vinke, Nikolai Pavlov, Stephen Bellis, Liam Wall, Kevin O’Neill, Carl Jackson, and Craig S. Levin.  Fast Timing Silicon Photomultipliers for Scintillation Detectors.  IEEE Photonics Technology Letters, 25(14):1309–1312, jul 2013.
  * [13] M.T. Lucchini, W. Chung, S.C. Eno, Y. Lai, L. Lucchini, M. Nguyen, and C.G. Tully.  New perspectives on segmented crystal calorimeters for future colliders.  Journal of Instrumentation, 15(11):P11005–P11005, 2020.
  * [14] A. Izmaylov, S. Aoki, J. Blocki, J. Brinson, A. Dabrowska, I. Danko, M. Dziewiecki, B. Ellison, L. Golyshkin, R. Gould, T. Hara, B. Hartfiel, J. Holeczek, M. Khabibullin, A. Khotjantsev, D. Kielczewska, J. Kisiel, T. Kozlowski, Yu. Kudenko, R. Kurjata, T. Kutter, J. Lagoda, J. Liu, J. Marzec, W. Metcalf, P. Mijakowski, O. Mineev, Yu. Musienko, D. Naples, M. Nauman, D. Northacker, J. Nowak, V. Paolone, M. Posiadala, P. Przewlocki, J. Reid, E. Rondio, A. Shaykhiev, M. Sienkiewicz, D. Smith, J. Sobczyk, M. Stodulski, A. Straczek, R. Sulej, A. Suzuki, J. Swierblewski, T. Szeglowski, M. Szeptycka, T. Wachala, D. Warner, N. Yershov, T. Yano, A. Zalewska, K. Zaremba, and M. Ziembicki.  Scintillator counters with WLS fiber/MPPC readout for the side muon range detector (SMRD) of the T2K experiment.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 623(1):382–384, nov 2010.
  * [15] D. Naito, Y. Maeda, N. Kawasaki, T. Masuda, H. Nanjo, T. Nomura, M. Sasaki, N. Sasao, S. Seki, K. Shiomi, and Y. Tajima.  Development of a low-mass and high-efficiency charged-particle detector.  Progress of Theoretical and Experimental Physics, 2016:023C01, 2016\.
  * [16] Woo-Suk Sul, Chae-Hun Lee, and Gyu-Seong Cho.  Influence of Guard-Ring Structure on the Dark Count Rates of Silicon Photomultipliers.  IEEE Electron Device Letters, 34(3):336–338, 2013.
  * [17] Ferenc Nagy, Massimo Mazzillo, Lucio Renna, Giuseppina Valvo, Delfo Sanfilippo, Beatrice Carbone, Angelo Piana, Giorgio Fallica, and József Molnár.  Afterpulse and delayed crosstalk analysis on a STMicroelectronics silicon photomultiplier.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 759:44–49, 2014\.
  * [18] J. Rosado, V.M. Aranda, F. Blanco, and F. Arqueros.  Modeling crosstalk and afterpulsing in silicon photomultipliers.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 787:153–156, 2015.
  * [19] J. Baron, W. C. Campbell, D. DeMille, J. M. Doyle, G. Gabrielse, Y. V. Gurevich, P. W. Hess, N. R. Hutzler, E. Kirilov, I. Kozyryev, B. R. O’Leary, C. D. Panda, M. F. Parsons, E. S. Petrik, B. Spaun, A. C. Vutha, and A. D. West.  Order of Magnitude Smaller Limit on the Electric Dipole Moment of the Electron.  Science, 343(6168):269–272, 2014.
  * [20] V. Andreev, D. G. Ang, D. DeMille, J. M. Doyle, G. Gabrielse, J. Haefner, N. R. Hutzler, Z. Lasner, C. Meisenhelder, B. R. O’Leary, C. D. Panda, A. D. West, E. P. West, and X. Wu.  Improved limit on the electric dipole moment of the electron.  Nature, 562(7727):355–360, 2018.
  * [21] Cari Cesarotti, Qianshu Lu, Yuichiro Nakai, Aditya Parikh, and Matthew Reece.  Interpreting the electron EDM constraint.  Journal of High Energy Physics, 2019(5):59, may 2019.
  * [22] Eli Slenders, Eleonora Perego, Mauro Buttafava, Giorgio Tortarolo, Enrico Conca, Sabrina Zappone, Agnieszka Pierzynska-Mach, Federica Villa, Enrica Maria Petrini, Andrea Barberis, Alberto Tosi, and Giuseppe Vicidomini.  Cooled SPAD array detector for low light-dose fluorescence laser scanning microscopy.  Biophysical Reports, 1(2):100025, dec 2021.
  * [23] HAMAMATSU PHOTONICS K.K.  Photomultiplier tubes and assemblies for scintillation counting and high energy physics, 2017.
  * [24] M. Teich, K. Matsuo, and B. Saleh.  Excess noise factors for conventional and superlattice avalanche photodiodes and photomultiplier tubes.  IEEE Journal of Quantum Electronics, 22(8):1184–1193, August 1986\.
  * [25] S. Vinogradov, T. Vinogradova, V. Shubin, D. Shushakov, and K. Sitarsky.  Probability distribution and noise factor of solid state photomultiplier signals with cross-talk and afterpulsing.  In 2009 IEEE Nuclear Science Symposium Conference Record (NSS/MIC). IEEE, October 2009.
  * [26] Takahiko Masuda, Daniel G. Ang, Nicholas R. Hutzler, Cole Meisenhelder, Noboru Sasao, Satoshi Uetake, Xing Wu, David DeMille, Gerald Gabrielse, John M. Doyle, and Koji Yoshimura.  Suppression of the optical crosstalk in a multi-channel silicon photomultiplier array.  Optics Express, 29(11):16914, may 2021.
  * [27] K. Ieki, T. Iwamoto, D. Kaneko, S. Kobayashi, N. Matsuzawa, T. Mori, S. Ogawa, R. Onda, W. Ootani, R. Sawada, K. Sato, and R. Yamada.  Large-area MPPC with enhanced VUV sensitivity for liquid xenon scintillation detector.  Nuclear Instruments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 925:148–155, 2019.
  * [28] A. Gola, C. Piemonte, and A. Tarolli.  Analog Circuit for Timing Measurements With Large Area SiPMs Coupled to LYSO Crystals.  IEEE Transactions on Nuclear Science, 60(2):1296–1302, 2013.
  * [29] HAMAMATSU PHOTONICS K.K.  Mppc techinical note, 2021.
  * [30] Sergey Vinogradov.  Perfomance of silicon photomultipliers in photon number and time resolution.  In Proceedings of International Conference on New Photo-detectors — PoS(PhotoDet2015). Sissa Medialab, June 2016\.
  * [31] HAMAMATSU PHOTONICS K.K.  Head-on pmt product selection, 2018.
  * [32] Nicholas R. Hutzler, Hsin-I Lu, and John M. Doyle.  The Buffer Gas Beam: An Intense, Cold, and Slow Source for Atoms and Molecules.  Chemical Reviews, 112(9):4803–4827, 2012.
  * [33] Xing Wu, Zhen Han, James Chow, Daniel Gordon Ang, Cole Meisenhelder, Cristian Daniel Panda, Elizabeth Petrik West, Gerald Gabrielse, John M. Doyle, and David DeMille.  The metastable Q ${}^{3}\Delta_{2}$ state of ThO: A new resource for the ACME electron EDM search.  New Journal of Physics, 22(2):023013, nov 2020.
  * [34] Xing Wu, Peiran Hu, Zhen Han, Daniel Gordon Ang, Cole Meisenhelder, Gerald Gabrielse, John M. Doyle, and David DeMille.  Electrostatic focusing of cold and heavy molecules for the ACME electron EDM search.  New Journal of Physics, 24(7):073043, jul 2022.
  * [35] Damian L. Kokkin, Timothy C. Steimle, and David Demille.  Branching ratios and radiative lifetimes of the U, L, and I states of thorium oxide.  Physical Review A - Atomic, Molecular, and Optical Physics, 90(6):1–10, 2014.
  * [36] CRISTIAN DANIEL PANDA.  Order of magnitude improved limit on the electric dipole moment of the electron.  Ph.d thesis, Harvard University, 2018.