#  LM-Debugger: An Interactive Tool for
Inspection and Intervention in Transformer-Based Language Models

Mor Geva1 Avi Caciularu2, Guy Dar3 Paul Roit2 Shoval Sadde1
Micah Shlain1 Bar Tamir4 Yoav Goldberg1,2
1Allen Institute for AI 2Bar-Ilan University
3Tel Aviv University 4The Hebrew University of Jerusalem
<EMAIL_ADDRESS>Work done during an internship at AI2.

###### Abstract

The opaque nature and unexplained behavior of transformer-based language
models (LMs) have spurred a wide interest in interpreting their predictions.
However, current interpretation methods mostly focus on probing models from
outside, executing behavioral tests, and analyzing salience input features,
while the internal prediction construction process is largely not understood.
In this work, we introduce LM-Debugger, an interactive debugger tool for
transformer-based LMs, which provides a fine-grained interpretation of the
model’s internal prediction process, as well as a powerful framework for
intervening in LM behavior. For its backbone, LM-Debugger relies on a recent
method that interprets the inner token representations and their updates by
the feed-forward layers in the vocabulary space. We demonstrate the utility of
LM-Debugger for single-prediction debugging, by inspecting the internal
disambiguation process done by GPT2. Moreover, we show how easily LM-Debugger
allows to shift model behavior in a direction of the user’s choice, by
identifying a few vectors in the network and inducing effective interventions
to the prediction process. We release LM-Debugger as an open-source tool and a
demo over GPT2 models.

Figure 1: Illustration of the main capabilities of LM-Debugger. Our tool
interprets dominant changes in the output distribution induced by the feed-
forward layers across the network (self-attention layers are not shown), and
enables configuring interventions for shifting the prediction in directions of
the user’s choice.

##  1 Introduction

Transformer-based language models (LMs) are the backbone of modern NLP models
Bommasani et al. (2021), but their internal prediction construction process is
opaque. This is problematic to end-users that do not understand why the model
makes specific predictions, as well as for developers who wish to debug or fix
model behaviour.

Recent work Elhage et al. (2021); Geva et al. (2022) suggested that the
construction process of LM predictions can be viewed as a sequence of updates
to the token representation. Specifically, Geva et al. (2022) showed that
updates by the feed-forward network (FFN) layers, one of the building blocks
of transformers Vaswani et al. (2017), can be decomposed into weighted
collections of sub-updates, each induced by a FFN parameter vector, that can
be interpreted in the vocabulary space.

In this work, we make a step towards LM transparency by employing this
interpretation approach to create LM-Debugger, a powerful tool for inspection
and intervention in transformer LM predictions. LM-Debugger provides three
main capabilities for single-prediction debugging and model analysis
(illustrated in Figure 1). First, for a given input (e.g. “My wife is working
as a”), it interprets the model’s prediction at each layer in the network, and
the major changes applied to it by FFN layers. This is done by projecting the
token representation before and after the FFN update as well as the major FFN
sub-updates at any layer to the output vocabulary. Second, it allows
intervening in the prediction by changing the weights of specific sub-updates,
e.g. increasing (decreasing) a sub-update that promotes music-related
(teaching-related) concepts, which results in a modified output. Last, for a
given LM, LM-Debugger interprets all the FFN parameter vectors across the
network and creates a search index over the tokens they promote. This allows
an input-independent analysis of the concepts encoded by the model’s FFN
layers, and enables configuring general and effective interventions.

We demonstrate the utility of LM-Debugger for two general use-cases. In the
context of prediction debugging, we use the fine-grained tracing of LM-
Debugger to inspect the internal disambiguation process performed by the
model. Furthermore, we demonstrate how our tool can be used to configure a few
powerful interventions that effectively control different aspects in text
generation.

We release LM-Debugger as an open-source tool at
https://github.com/mega002/lm-debugger and host a demo of GPT2 Brown et al.
(2020) at https://lm-debugger.apps.allenai.org. This to provide better
understanding of the inner workings of transformer-based LMs and to facilitate
future work in analyzing and controlling NLP models.

Figure 2: The prediction view of LM-Debugger, showing the prediction trace for
a given input (main panel), allowing to configure interventions (lower panel)
and interpret sub-updates to the output distribution (right panel).

##  2 Underlying Interpretation Method

LM-Debugger establishes a framework for interpreting a token’s representation
and updates applied to it at each layer in the network. This framework builds
upon recent findings by Geva et al. (2022), who viewed the token
representation as a changing distribution over the output vocabulary, and the
output from each FFN layer as a collection of weighted sub-updates to that
distribution, which are often interpretable to humans. We next elaborate on
the findings we rely on at this work.

Consider a transformer LM with $L$ layers and an embedding matrix
$E\in\mathbb{R}^{d\times|\mathcal{V}|}$ of hidden dimension $d$, over a
vocabulary $\mathcal{V}$. Let $\mathbf{w}=w_{1},...,w_{t}$ s.t. $\forall
i=1,...,t:\;w_{i}\in\mathcal{V}$ be an input sequence of tokens, then at each
layer $\ell=1,...,L$, the hidden representation $\mathbf{x}_{i}^{\ell}$ of the
$i$-th token is being processed and updated by a FFN layer through a residual
connection He et al. (2016):111Layer normalization is omitted Geva et al.
(2022).

$\mathbf{\tilde{\text{$\mathbf{x}$}}}_{i}^{\ell}=\mathbf{x}_{i}^{\ell}+\texttt{FFN}^{\ell}(\mathbf{x}_{i}^{\ell}),$

where $\mathbf{x}_{i}^{\ell}$ is the output from the preceding multi-head
self-attention layer, and $\mathbf{\tilde{\text{$\mathbf{x}$}}}_{i}^{\ell}$ is
the updated token representation Vaswani et al. (2017). Geva et al. (2022)
proposed an interpretation method for these updates in terms of the
vocabulary, which we employ as the backbone of LM-Debugger and describe in
detail next.

#### Token Representation as a Distribution Over the Output Vocabulary.

The token representation before ($\mathbf{x}_{i}^{\ell}$) and after
($\mathbf{\tilde{\text{$\mathbf{x}$}}}_{i}^{\ell}$) the FFN update at any
layer $\ell$ is interpreted by projecting it to the vocabulary space and
converting it to a distribution:

$\mathbf{p}_{i}^{\ell}=\text{softmax}(E\mathbf{x}_{i}^{\ell})\;\;;\;\;\mathbf{\tilde{\text{$\mathbf{p}$}}}_{i}^{\ell}=\text{softmax}(E\mathbf{\tilde{\text{$\mathbf{x}$}}}_{i}^{\ell})$

The final model output is defined by
$\mathbf{y}=\mathbf{\tilde{\text{$\mathbf{p}$}}}_{i}^{L}$.

#### The FFN Output as a Weighted Collection of Sub-Updates.

Each FFN layer is defined with two parameter matrices
$K^{\ell},V^{\ell}\in\mathbb{R}^{d_{m}\times d}$, where $d_{m}$ is the
intermediate hidden dimension, and a non-linearity function $f$ (bias terms
are omitted):

$\displaystyle\texttt{FFN}^{\ell}(\mathbf{x}^{\ell})=f\left(K^{\ell}\mathbf{x}^{\ell}\right)V^{\ell}$
(1)

Geva et al. (2022) interpreted the FFN output by (a) decomposing it into sub-
updates, each induced by a single FFN parameter vector, and (b) projecting
each sub-update to the vocabulary space. Formally, Eq. 1 can be decomposed as:

$\texttt{FFN}^{\ell}(\mathbf{x}^{\ell})=\sum_{i=1}^{d_{m}}f(\mathbf{x}^{\ell}\cdot\mathbf{k}_{i}^{\ell})\mathbf{v}_{i}^{\ell}=\sum_{i=1}^{d_{m}}m_{i}^{\ell}\mathbf{v}_{i}^{\ell}.$

where $\mathbf{k}_{i}^{\ell}$ is the $i$-th row of $K^{\ell}$,
$\mathbf{v}_{i}^{\ell}$ is the $i$-th column of $V^{\ell}$, and
$m_{i}^{\ell}:=f(\mathbf{x}^{\ell}\cdot\mathbf{k}_{i}^{\ell})$ is the
activation coefficient of $\mathbf{v}_{i}^{\ell}$ for the given input. Each
term in this sum is interpreted as a sub-update to the output distribution, by
inspecting the top-scoring tokens in its projection to the vocabulary, i.e.
$E\mathbf{v}_{i}^{\ell}$.

In the rest of the paper, we follow Geva et al. (2022) and refer to columns of
$V^{\ell}$ as “value vectors” and to their weighted input-dependent form as
“sub-updates”. Importantly, value vectors are static parameter vectors that
are independent on the input sequence, while sub-updates are dynamic as they
are weighted by input-dependent coefficients. For a model with $L$ layers and
a hidden dimension $d_{m}$, there are $L*d_{m}$ static value vectors, which
induce $L*d_{m}$ corresponding sub-updates when running an input through the
model.

##  3 LM-Debugger

LM-Debugger leverages both static and dynamic analysis of transformer FFN
layers and the updates they induce to the output distribution for debugging
and intervention in LM predictions. These capabilities are provided in two
main views, which we describe next.

###  3.1 Prediction View

This view, shown in Figure 2, is designed for per-example debugging. It allows
running inputs through the model to generate text in an auto-regressive
manner, while tracing the dominant sub-updates in every layer and applying
interventions.

#### Prediction Trace

(Figure 2, main panel). The user enters an input for the model, for which a
detailed trace of the prediction across the network is provided. For each
layer, it shows the top-tokens in the output distribution, before and after
the FFN update, and the 10 most dominant FFN sub-updates. For every sub-update
$m_{i}\mathbf{v}_{i}^{\ell}$ we show an identifier L$[\ell]$D$[i]$ of its
corresponding value vector and the coefficient for the given input (e.g.
L17D4005 and $9.79$).222The layer and dimension in the identifier use zero-
index. The top distribution tokens and sub-updates are sorted by the token
probability/sub-update coefficient from left (highest) to right (lowest). A
small arrow next to each sub-update allows setting an intervention on its
corresponding value vector.

#### Interventions

(Figure 2, lower panel). Beyond tracing the output distribution, LM-Debugger
also allows intervening in the prediction process by setting the coefficients
of any vector values in the network, thus, inducing sub-updates of the user’s
choice. To set an intervention for a specific value vector, the user should
enter its identifier to the panel and choose whether to “turn it on or off”,
that is, setting its coefficient to the value of the coefficient of the most
dominant sub-update in that layer, or to zero, respectively. When running an
input example, all interventions in the panel will be effective, for the
entire generation process.

#### Value Vector Information

(Figure 2, right panel). A natural question that arises is how to choose
meaningful interventions. LM-Debugger provides two complementary approaches
for this. A bottom-up approach is to observe the dominant sub-updates for
specific examples, and apply interventions on them. A sub-update can be
interpreted by inspecting the top-tokens in the projection of its
corresponding value vector to the vocabulary Geva et al. (2022). For
convenience, we let the user assign names to value vectors. Another way to
find meaningful interventions is by a top-down approach of searching for value
vectors that express concepts of the user’s interest. We provide this
capability in the exploration view of LM-Debugger, which is described next.

###  3.2 Exploration View

This view allows static exploration of value vectors, primarily for analyzing
which concepts are encoded in the FFN layers, how concepts are spread over
different layers, and identifying groups of related value vectors.

#### Keyword Search

(Figure 3). Value vectors are interpreted by the top tokens they promote. By
considering these sets of tokens as textual documents, LM-Debugger allows
searching for concepts encoded in value vectors across the layers. This is
enabled by a search index that LM-Debugger holds in the background, which
stores the projections of all value vectors to the vocabulary, and allows
executing simple queries against them using the BM25 Robertson et al. (1995)
algorithm.

#### Cluster Visualization

(Figure 4). Assuming the user is interested in locating a specific concept in
the network and that she has found a relevant value vector, either from
debugging an example in the prediction view or by the keyword search. A
natural next step is to find similar value vectors that promote related
tokens. To this end, LM-Debugger provides a clustering of all value vectors in
the network, which allows mapping any value vector to a cluster of similar
vectors in the hidden space Geva et al. (2022). The interface displays a
random sample of vectors from the cluster, as well as an aggregation of their
top tokens as a word cloud, showing the concepts promoted by the cluster.

##  4 Debugging LM Predictions by Tracing FFN Updates

In this section, we demonstrate the utility of LM-Debugger for interpreting
model behaviour upon a given example. As an instructive example, we will
consider the case of sense disambiguation.

When generating text, LMs often need to perform sense disambiguation and
decide on one plausible continuation. For example, the word “for” in the input
“The book is for” has two plausible senses of purpose (e.g. “reading”) and
person (e.g. “him”) Karidi et al. (2021). We will now inspect the prediction
by GPT2 Brown et al. (2020) and track the internal sense disambiguation
process for this example. To this end, we enter the input in the prediction
view and click Trace, which provides a full trace of the prediction across
layers.

Table 1 displays a part of this trace from selected layers, showing a gradual
transition from purpose to person sense. Until layer 11 (out of 24), the top-
tokens in the output distribution are mostly related to sale/example purposes.
Starting from layer 12, the prediction slowly shifts to revolve about the
audience of the book, e.g. anyone and ages, until layer 18 where sale is
eliminated from the top position. In the last layers, tokens become more
specific, e.g. beginners and adults.

To examine the major updates through which the prediction has formed, we can
click on specific sub-updates in the trace to inspect the top-scoring tokens
in their projections. We observe that in early layers, tokens are often
related to purpose sense (e.g. instance in L2D1855 and buyers in L12D659), in
intermediate layers tokens are a mix of both senses (readers in L16D3026 and
preschool in L17D2454, and sale/free in L16D1662), and mostly person sense in
the last layers (users in L18D685, people in L20D3643, and those in L21D2007).

Figure 3: Keyword search in the exploration view of LM-Debugger, which matches
user queries against the tokens promoted by value vectors of the model. Figure
4: Cluster visualization in the exploration view of LM-Debugger, which maps a
given value vector to its cluster of similar value vectors in the network.
Layer: 4 Sense: purpose
---
Before: example, the, instance, purposes
After: example, the, instance, all
Layer: 10 Sense: purpose
Before: the, sale, example, a
After: the, sale, a, example
Layer: 15 Sense: purpose/person
Before: sale, the, anyone, use
After: sale, anyone, the, ages
Layer: 20 Sense: person
Before: beginners, anyone, adults, sale
After: anyone, beginners, adults, readers
Table 1: Partial prediction trace of GPT2 for the input “This book is for”,
showing the internal disambiguation process from purpose to person sense
across layers.

##  5 Configuring Effective Interventions for Controlled Text Generation

Beyond interpretability, LM-Debugger enables to intervene in LM predictions.
We show this by finding value vectors that promote specific concepts and
applying simple and effective interventions.

#### Controlling Occupation Prediction.

Consider the input “My wife is working as a”. When running it through GPT2,
the final prediction from the last layer has the top tokens nurse, teacher,
waitress. We would like to intervene in the prediction in order to change its
focus to occupations related to software engineering, which in general are
less associated with women De-Arteaga et al. (2019). To this end, we will use
the exploration view of LM-Debugger to search for value vectors promoting
software-related concepts.

Searching the keywords “software”, “developer”, and “engineer” brings up two
value vectors with coherent concepts: L10D3141 and L17D115 (Figure 3). Now, we
will add these value vectors to the intervention panel in the prediction view,
and run the example again. Our intervention, that only involved two (0.002%)
vectors in the network, dramatically changed the prediction to software,
programmer, consultant, developer, effectively shifting it in the direction we
wanted. This demonstrates the power of LM-Debugger to change model behaviour
and fix undesirable predictions.

Input | Interven. |  Continuation
---|---|---
“Service in this place is” | - |  a bit of a mess. I’m not sure
$\uparrow$ Positive |  a good place to make the right efforts to make
$\uparrow$ Negative |  a waste of a bunch of crap that is too
“I have been to this restaurant twice and was” | - |  both times I was disappointed. The first time I
$\uparrow$ Positive |  have been served excellent food and good service. The
$\uparrow$ Negative |  have been disappointed. The food is over processed and
“We went on a weeknight. Place was” | - |  packed. We had to wait for the bus
$\uparrow$ Positive |  good, good food, good staff, good people
$\uparrow$ Negative |  too far for us to get lost. We were
“Went for breakfast on 6/16/14. We” | - |  had a great time. We had a great time
$\uparrow$ Positive |  have a good team of people who are able to
$\uparrow$ Negative |  were too heavy for the wrong type of food that
Table 2: Continuations (limited to 10 tokens) generated by GPT2 for different
inputs from the Yelp dataset, with and without interventions for “turning on”
sub-updates for positive and negative sentiment.

#### Controlling the Sentiment of Generated Text.

The previous example focused on next-token prediction. We now take this one
step further and configure powerful and general interventions that influence
various texts generated by the model. For our experimental setting, we will
attempt to control the sentiment in generated reviews by GPT2, for inputs
taken from the Yelp dataset Asghar (2016).

We choose our interventions independently of the inputs, with two easy steps.
First, we use the keyword search (Figure 3) to identify “seed” value vectors
that promote positive and negative adjectives/adverbs, using the queries
“terrible, mediocre, boring” and “spacious, superb, delicious”. Then, we take
one value vector for each polarity and, using the cluster visualization
(Figure 4), expand it to a diverse set of vectors from its corresponding
cluster, that promote similar concepts. Overall, we select 5-6 value vectors
for each polarity (details in Appendix A.1), to which we apply interventions.

Table 2 presents the texts generated by GPT2 (each limited to 10 tokens) for
multiple inputs, with and without applying interventions. Clearly, across all
the examples, our intervention in the prediction successfully leads to the
desired effect, turning the sentiment of the generated text to be positive or
negative, according to the configured sub-updates.

##  6 Implementation Details

The prediction view is implemented as a React web application with a backend
Flask server that runs an API for executing models from the Transformers
library by HuggingFace Wolf et al. (2020). The exploration view is a Streamlit
web application, which (a) sends user search queries to an Elasticsearch index
with the top tokens of all vector value projections, and (b) visualize
clusters of value vectors created with the scikit-learn package Pedregosa et
al. (2011). Our current implementation supports any GPT2 model from
HuggingFace, and other auto-regressive models can be plugged-in with only a
few local modifications (e.g. translating the relevant layer names). More
details and instructions for how to deploy and run LM-Debugger are provided at
https://github.com/mega002/lm-debugger.

##  7 Related Work

Interpreting single-predictions and the general behavior of LMs is a growing
research area that attracted immense attention in recent years Belinkov et al.
(2020); Choudhary et al. (2022). LM-Debugger is a the first tool to interpret
and intervene in the prediction construction process of transformer-based LMs
based on FFN updates.

Existing interpretation and analysis frameworks mostly rely on methods for
behavioral analysis Ribeiro et al. (2020) by probing models with adversarial
Wallace et al. (2019) or counterfactual examples Tenney et al. (2020), input
saliency methods that assign importance scores to input features Wallace et
al. (2019); Tenney et al. (2020), and analysis of the attention layers Hoover
et al. (2020); Vig and Belinkov (2019).

More related to LM-Debugger, other tools analyze patterns in neuron
activations Rethmeier et al. (2020); Dalvi et al. (2019); Alammar (2021).
Unlike these methods, we focus on interpreting the model parameters and on
intervening in their contribution to the model’s prediction.

The functionality of LM-Debugger is mostly related to tools that trace hidden
representations across layers. Similarly to LM-Debugger, Alammar (2021);
Nostalgebraist (2020) interpret the token representation in terms of the
output vocabulary. We take this one step further and interpret the FFN updates
to the representation, allowing to observe not only the evolution of the
representation but also the factors that induce changes in it.

Our intervention in FFN sub-updates relates to recent methods for locating and
editing knowledge in the FFN layers of LMs Meng et al. (2022); Dai et al.
(2021). Different from these methods, LM-Debugger aims to provide a
comprehensive and fine-grained interpretation of the prediction construction
process across the layers.

##  8 Conclusion

We introduce LM-Debugger, a debugger tool for transformer-based LMs, and the
first tool to analyze the FFN updates to the token representations across
layers. LM-Debugger provides a fine-grained interpretation of single-
predictions, as well as a powerful framework for intervention in LM
predictions.

## Acknowledgements

We thank the REVIZ team at the Allen Institute for AI, particularly Sam
Skjonsberg and Sam Stuesser. This project has received funding from the
Computer Science Scholarship granted by the Séphora Berrebi Foundation, the
PBC fellowship for outstanding PhD candidates in Data Science, and the
European Research Council (ERC) under the European Union’s Horizon 2020
research and innovation programme, grant agreement No. 802774 (iEXTRACT).

## References

  * Alammar (2021) J Alammar. 2021.  Ecco: An open source library for the explainability of transformer language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations_ , pages 249–257, Online. Association for Computational Linguistics.
  * Asghar (2016) Nabiha Asghar. 2016.  Yelp dataset challenge: Review rating prediction.  _arXiv preprint arXiv:1605.05362_.
  * Belinkov et al. (2020) Yonatan Belinkov, Sebastian Gehrmann, and Ellie Pavlick. 2020.  Interpretability and analysis in neural NLP.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts_ , pages 1–5, Online. Association for Computational Linguistics.
  * Bommasani et al. (2021) Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen Creel, Jared Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas F. Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, O. Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir P. Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Benjamin Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, J. F. Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher R’e, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishna Parasuram Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei A. Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2021.  On the opportunities and risks of foundation models.  _ArXiv_ , abs/2108.07258.
  * Brown et al. (2020) Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Proceedings of Neural Information Processing Systems (NeurIPS)_.
  * Choudhary et al. (2022) Shivani Choudhary, Niladri Chatterjee, and Subir Kumar Saha. 2022.  Interpretation of black box nlp models: A survey.  _arXiv preprint arXiv:2203.17081_.
  * Dai et al. (2021) Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. 2021.  Knowledge neurons in pretrained transformers.  _arXiv preprint arXiv:2104.08696_.
  * Dalvi et al. (2019) Fahim Dalvi, Avery Nortonsmith, Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, and James Glass. 2019.  NeuroX: A toolkit for analyzing individual neurons in neural networks.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33(01):9851–9852.
  * De-Arteaga et al. (2019) Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019.  Bias in bios: A case study of semantic representation bias in a high-stakes setting.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , FAT* ’19, page 120–128, New York, NY, USA. Association for Computing Machinery.
  * Elhage et al. (2021) Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2021.  A mathematical framework for transformer circuits.  _Transformer Circuits Thread_.  Https://transformer-circuits.pub/2021/framework/index.html.
  * Geva et al. (2022) Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. 2022.  Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space.  _arXiv preprint arXiv:2203.14680_.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition.  In _Proceedings of the conference on computer vision and pattern recognition (CVPR)_.
  * Hoover et al. (2020) Benjamin Hoover, Hendrik Strobelt, and Sebastian Gehrmann. 2020.  exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_ , pages 187–196, Online. Association for Computational Linguistics.
  * Karidi et al. (2021) Taelin Karidi, Yichu Zhou, Nathan Schneider, Omri Abend, and Vivek Srikumar. 2021\.  Putting words in BERT’s mouth: Navigating contextualized vector spaces with pseudowords.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10300–10313, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Meng et al. (2022) Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022.  Locating and editing factual knowledge in gpt.  _arXiv preprint arXiv:2202.05262_.
  * Nostalgebraist (2020) Nostalgebraist. 2020.  interpreting GPT: the logit lens.
  * Pedregosa et al. (2011) Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011.  Scikit-learn: Machine learning in python.  _the Journal of machine Learning research_ , 12:2825–2830.
  * Rethmeier et al. (2020) Nils Rethmeier, Vageesh Kumar Saxena, and Isabelle Augenstein. 2020.  Tx-ray: Quantifying and explaining model-knowledge transfer in (un-)supervised nlp.  In _Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)_ , volume 124 of _Proceedings of Machine Learning Research_ , pages 440–449. PMLR.
  * Ribeiro et al. (2020) Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020.  Beyond accuracy: Behavioral testing of NLP models with CheckList.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4902–4912, Online. Association for Computational Linguistics.
  * Robertson et al. (1995) Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, and Mike Gatford. 1995.  et almbox. 1995. okapi at trec-3.  _Nist Special Publication Sp_ , 109:109.
  * Tenney et al. (2020) Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan. 2020.  The language interpretability tool: Extensible, interactive visualizations and analysis for NLP models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 107–118, Online. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems (NIPS)_ , pages 5998–6008.
  * Vig and Belinkov (2019) Jesse Vig and Yonatan Belinkov. 2019.  Analyzing the structure of attention in a transformer language model.  In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pages 63–76, Florence, Italy. Association for Computational Linguistics.
  * Wallace et al. (2019) Eric Wallace, Jens Tuyls, Junlin Wang, Sanjay Subramanian, Matt Gardner, and Sameer Singh. 2019.  AllenNLP interpret: A framework for explaining predictions of NLP models.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations_ , pages 7–12, Hong Kong, China. Association for Computational Linguistics.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.

##  Appendix A Appendix

###  A.1 Details on Interventions to Control Generated Text Sentiment

Table 3 lists all the value vectors selected for our interventions described
in §5, and examples for top-scoring tokens in their projections. These vectors
were found with the exploration view of LM-Debugger (§3.2), using both keyword
search and clustering visualisation. All the interventions were configured to
“turn on” these vectors, namely, setting their coefficients to be maximal for
the corresponding layer. This is following the observation by Geva et al.
(2022) that FFN updates operate in a token promotion mechanism (rather than
elimination).

Sentiment | Value Vector |  Example Top-scoring Tokens
---|---|---
Positive | L13D1763 |  properly, appropriately, adequate, truthful, humane, fulfil, inclusive, timely, patiently, sustainable
L13D2011 |  clean, Proper, secure, flawless, safest, graceful, smooth, calmly
L14D944 |  peacefully, graceful, respectful, careful, generous, patiently, calm, tolerant, fair
L15D74 |  Excellence, superb, trustworthy, marvelous, terrific, awesome, Amazing
L20D988 |  successful, optimal, perfect, satisfactory, welcome, helpful, fulfilling, healthy
Negative | L11D4 |  outdated, inadequate, stale, lousy, dull, mediocre, boring, wasteful
L14D2653 |  trivial, dismiss, rigid, unsupported, only, prejud, obfusc, pretend, dispar, slander
L16D974 |  inappropriately, poorly, disrespect, unreliable, unhealthy, insecure, improperly, arrogance
L17D3790 |  inappropriate, improper, wrong, bad, harmful, unreasonable, defective, disturbance, errors
L18D91 |  confused, bizarre, unfairly, horrible, reckless, neglect, misplaced, strange, nasty, mistakenly
L18D3981 |  wrong, incorrect, insufficient, misleading, premature, improperly, unrealistic, outdated, unfair
Table 3: Value vectors used for controlling sentiment in generated text, that
promote positive and negative adjectives/adverbs. For each vector, we show
example top-scoring tokens from its projection to the vocabulary, as presented
in the exploration view of LM-Debugger.


# Shadows and photon rings of a quantum black hole

Jing-Peng Ye 1 Zhi-Qing He1 Ai-Xu Zhou2 Zi-Yang Huang2 Jia-Hui Huang2,3
<EMAIL_ADDRESS>1 School of Information and Optoelectronic Science and
Engineering, South China Normal University, Guangzhou 510631, China
2 Key Laboratory of Atomic and Subatomic Structure and Quantum Control
(Ministry of Education), Guangdong Basic Research Center of Excellence for
Structure and Fundamental Interactions of Matter, School of Physics, South
China Normal University, Guangzhou 510006, China
3 Guangdong Provincial Key Laboratory of Quantum Engineering and Quantum
Materials, Guangdong-Hong Kong Joint Laboratory of Quantum Matter, South China
Normal University, Guangzhou 510006, China

###### Abstract

Recently, a black hole model in loop quantum gravity has been proposed by
Lewandowski, Ma, Yang and Zhang (Phys. Rev. Lett. 130, 101501 (2023)). The
metric tensor of the quantum black hole (QBH) is a suitably modified
Schwarzschild one. In this paper, we calculate the radius of light ring and
obtain the linear approximation of it with respect to the quantum correction
parameter $\alpha$: $r_{l}\simeq 3M-\frac{\alpha}{9M}$. We then assume the QBH
is backlit by a large, distant plane of uniform, isotropic emission and
calculate the radius of the black hole shadow and its linear approximation:
$r_{s}=3\sqrt{3}M-\frac{\alpha}{6\left(\sqrt{3}M\right)}$. We also consider
the photon ring structures in the shadow when the impact parameter $b$ of the
photon approaches to a critical impact parameter $b_{\textrm{c}}$, and obtain
a formula for estimating the deflection angle, which is
$\varphi_{\textrm{def}}=-\frac{\sqrt{2}}{\omega
r_{l}^{2}}\log{\left(b-b_{c}\right)+\widetilde{C}(b)}$. We also numerically
plot the images of shadows and photon rings of the QBH in three different
illumination models and compare them with that of a Schwarzschild in each
model. It is found that we could distinguish the quantum black hole with a
Schwarzschild black hole by the shadow images in certain specific illumination
model.

##  1 Introduction

Although the gravitational force can now be well described by classical
general relativity, there exist some problems for general relativity.
Singularity is one of the most fundamental ones. Penrose proposed the first
version of the singularity theorem [1], and a more general one was proved by
Hawking and Penrose [2], which states that the existence of spacetime
singularity is inevitable under certain common physical conditions. How should
one treat spacetime singularities? We may expect that a quantum theory of
gravity could cure spacetime singularities. One of the candidate theories of
quantum gravity is loop quantum gravity (LQG), which is a background-
independent and nonperturbative scheme [3, 4, 5, 6, 7, 8, 9, 10]. The
cosmological big-bang singularity was resolved theoretically and numerically
in the context of loop quantum cosmology(LQC) [11, 12, 13, 14, 15]. For the
singularity of a Schwarzschild black hole (BH), some attempts aimed to solve
it by quantizing the interior of the BH with the techniques developed in LQG
[24, 23, 22, 21, 20, 19, 18, 17, 16]. In addition, the LQG corrections to BH
formation or gravitational collapse in different models have also been studied
[33, 32, 31, 30, 29, 28, 27, 26, 25, 35, 34].

Recently, the quantum effects of LQG have been taken into account in studying
the gravitational collapse of spherically symmetric dust matter [27, 35, 34].
The dust collapse was studied in the context of LQC model. The collapse
stopped when the energy density of the dust reached the Plank scale, and the
collapsing dust then turned to bounce. By matching metrics at the boundary
between the interior and exterior of the collapsing dust, the LQG corrected
exterior vacuum metric was derived. In the collapsing stage, a quantum-
corrected Schwarzschild metric appeared in the exterior spacetime, while, in
the late bouncing stage, something like a white hole appeared [35].

Significant progresses have also been made for observation of black holes via
radio waves in recent years. The first image of the supermassive black hole
$\text{M}87^{*}$ was provided by Event Horizon Telescope in 2019 [37, 38]. In
2022, the image of the supermassive black hole Sagittarius $A^{*}$ in our
Galactic Center was also released [39, 40]. Two important concepts related to
these images are the light ring and shadow of a BH. A light ring is a circular
photon orbit around a BH with a radius larger than the event horizon [41, 42,
43, 44]. A shadow is a dark area casted by a black hole for an observer at
asymptotic infinity. The radius of a BH shadow is closely related to the
radius of the light ring [45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57,
58, 59, 60, 61]. Usually, the shadow is not totally dark and there are some
tiny bright structures in it, which is dubbed photon rings [47]. Obviously,
the geometric information of a BH can be decoded from the structures of the
bright photon rings and shadow of the BH.

In this paper, we will investigate the shadow and the structure of the bright
photon rings of the quantum corrected black hole (QBH) proposed in [34, 35].
In section II, we calculate the radius of the light ring and the critical
impact parameter for the QBH case, discuss their deviations from the
Schwarzschild BH case and provide the leading term of the deviation with
respect to the quantum correction parameter $\alpha$. In section III, we
consider a model where the QBH is backlit by a large, distant screen of
uniform, isotropic emission [47], obtain the radius of shadow under the
background light source, and discuss the relation of deflection angle and the
impact parameter. In section IV, we plot the shadows and photon rings of the
QBH under several illumination conditions, and compare them with that of a
Schwarzschild black hole (SBH). We summarize our work in section V.

##  2 Radius of light ring

In this section, we discuss the radius of the unstable light ring of the QBH
spacetime proposed in [35, 36] with the metric described by

$\displaystyle
ds^{2}=-f(r)dt^{2}+f(r)^{-1}dr^{2}+r^{2}(d\theta^{2}+\sin^{2}\theta
d\varphi^{2}),~{}f(r)=1-\frac{2M}{r}+\frac{\alpha M^{2}}{r^{4}}.$ (1)

Here we use the natural units, $c=G=\hbar=1$. $\alpha=16\sqrt{3}\pi\gamma^{3}$
is the quantum correction parameter. $\gamma\approx 0.2375$ is the Barbero-
Immirzi parameter [65, 66]. There is a lower bound $M_{\textrm{min}}$ for the
mass parameter $M$ of the above BH in order that the horizons exist. In the
natural units, the value of this lower bound is

$M_{\textrm{min}}=\frac{16\gamma\sqrt{\pi\gamma}}{3\sqrt[4]{3}}\approx 0.8314$
(2)

which is a little smaller than the Plank mass $M_{P}=1$. In this paper we will
focus on $M>M_{\textrm{min}}$, there are two different horizons, one inner and
one outer horizon, which are denoted by $r_{-}$ and $r_{+}$ respectively.

Due to the spherical symmetry of the QBH, a planar null geodesic motion is
equivalent to an equatorial null geodesic motion. The null trajectory equation
in the equatorial plane is given by [68, 67]

$\frac{1}{r^{4}}\left(\frac{dr}{d\varphi}\right)^{2}=\frac{E^{2}}{L^{2}}-U_{\textrm{MS}}(r)=\left(\frac{1}{b}\right)^{2}-U_{\textrm{MS}}(r),$
(3)

where $E$ and $L$ are conserved energy and orbital angular momentum
respectively. $b=\frac{L}{E}$ is the impact parameter. $U_{\textrm{MS}}(r)$ is
the effective potential

$U_{\textrm{MS}}(r)=\frac{1}{r^{2}}f(r)=\frac{1}{r^{2}}\left(1-\frac{2M}{r}+\frac{\alpha
M^{2}}{r^{4}}\right).$ (4)

The possible values of the radii of the light rings are determined by the
extremal values of the effective potential $U_{\textrm{MS}}$, i.e. the roots
of the following equation

$\frac{dU_{\textrm{MS}}}{dr}=0$ (5)

Since the above equation is a quartic polynomial equation, all the four roots
can be given in analytic forms. It is found that when
$M>\frac{2^{4}}{3^{3}}\sqrt{\alpha}\approx 0.64$ ,two real roots exist for the
equation. So we have two real roots and two complex roots when the mass $M$ is
greater than $M_{\textrm{min}}\approx 0.8314$. The smaller real root is
located inside the outer horizon $r_{+}$, so the radius of the light ring is

$\displaystyle
r_{l}=\frac{1}{4}\left(3M+2\xi+\sqrt{\frac{\left(3M-\xi\right)\left(3M+2\xi\right)^{2}}{\xi}}\right),$
(6)

where

$\xi=\sqrt{\frac{9M^{2}}{4}+\sigma},$ (7)

and

$\displaystyle\sigma=\frac{4M^{2}\alpha}{\sqrt[3]{\frac{27M^{4}\alpha}{2}-\frac{1}{2}\sqrt{M^{4}\alpha^{2}(729M^{4}-256M^{2}\alpha)}}}+\sqrt[3]{\frac{27M^{4}\alpha}{2}-\frac{1}{2}\sqrt{M^{4}\alpha^{2}(729M^{4}-256M^{2}\alpha)}}.$
(8)

In order for comparison with SBH, we choose the mass parameter $M=0.84$, which
is a little larger than $M_{\textrm{min}}$, in all numerical figures in this
section. The effective potentials of both the QBH and SBH are shown in (1). It
is obviously that the radius of the light ring of the QBH is a little smaller
than that of the SBH. Explicitly, the radius of the light ring of QBH in this
case is $r_{l}\approx 2.323$ and the radius of the light ring of SBH in this
case is $r_{\textrm{Schw}}=3M\approx 2.52$.

Figure 1: The red curve $U_{\textrm{Schw}}$ is the effective potential of a
SBH, and the blue one $U_{\textrm{MS}}$ is the effective potential of the QBH.

The deviation of the radius of a QBH light ring from that of a SBH (with the
same mass) decreases with the increase of the BH mass. This is shown in
FIG.(2).

.

Figure 2: The deviation of light-ring radii of the QBH and SBH. $\Delta
r=r_{l}-r_{\textrm{Schw}}$.

Although we have obtained the exact relation between the radius of the light
ring of QBH and the quantum correction parameter $\alpha$ in Eq.(2), it is
easy to see in FIG.(2) that the quantum correction is much less than 1 even
for a BH with Plank mass. Then we present a linear approximation for the
light-ring radius with respect to the quantum correction parameter $\alpha$.
Define two dimensionless parameter $y$ and $z$ as

$\displaystyle y=\frac{r}{2M},~{}~{}z=\frac{\alpha}{(2M)^{2}}.$ (9)

For the dimensionless light ring radius $y_{\textrm{MS}}$, we have
$y_{\textrm{MS}}=\frac{r_{l}}{2M}$. $y_{\textrm{MS}}$ and $z$ satisfy the
following dimensionless equation

$z=-\frac{2}{3}\left(2y_{\textrm{MS}}^{4}-3y_{\textrm{MS}}^{3}\right).$ (10)

We plot this relation in FIG.(3). When $z=0(\alpha=0)$ , we have a nonzero
solution $y_{\textrm{MS}}=\frac{3}{2}(r_{l}=3M)$, and this is just the case of
a SBH. When $z>0$, there are two positive real roots and the larger one is the
light-ring radius.

Figure 3: The function of $z$ and $y_{\textrm{MS}}$.

Since the quantum correction is much less than 1, we approximate
$y_{\textrm{MS}}$ around $\frac{3}{2}$, and obtain

$z\simeq-\frac{9}{2}\left(y_{\textrm{MS}}-\frac{3}{2}\right),$ (11)

i.e.

$r_{l}\simeq 3M-\frac{\alpha}{9M}.$ (12)

Next, we consider the critical impact parameter $b_{c}$ with which a light ray
from spatial infinity approaches to the light ring of the QBH and has an
arbitrarily large deflection angle. This critical impact parameter is
determined by the following equation

$\left(\frac{1}{b_{c}}\right)^{2}-\frac{1}{r_{l}^{2}}\left(1-\frac{2M}{r_{l}}+\frac{\alpha
M^{2}}{r_{l}^{4}}\right)=0,$ (13)

and we have

$b_{c}=\frac{r_{l}}{\sqrt{\left(1-\frac{2M}{r_{l}}+\frac{\alpha
M^{2}}{r_{l}^{4}}\right)}}$ (14)

Taking a linear approximation with respect to the quantum correction parameter
$\alpha$, we can obtain

$b_{c}=3\sqrt{3}M-\frac{\alpha}{6\left(\sqrt{3}M\right)}+\mathcal{O}(\alpha^{2}).$
(15)

The first term is just the critical impact parameter of a SBH with mass $M$.

##  3 Fine structure of photon rings

In this section, we use the light-tracing method to derive the radius of the
shadow of QBH in our model and discuss the bright photon ring structure in the
shadow. The light rays with different impact parameters have different
deflection angles. Here the deflection angle is defined as the total change in
orbital plane azimuthal angle of a null geodesic. The deflection angle of a
“straight-line motion” is $\pi$. When the impact parameter $b$ is large
enough, the deflection angle is a little larger than $\pi$, and the light
trajectory intersects with the background light source. This kind of light
form a bright region in the view of a distant observer. See the green lines
outside the blue-line region in Fig.(4). When the impact parameters become
smaller and the values of the deflection angles are between $\frac{3}{2}\pi$
and $\frac{5}{2}\pi$, the trajectories of the light doesn’t intersect with the
background light source, and the relevant impact parameters correspond the
first shadow region on the screen of a distant observer. See the blue lines in
Fig.(4). When the values of the deflection angles are between $\frac{5}{2}\pi$
and $\frac{7}{2}\pi$, the trajectories of the light intersect with the
background light source, and the relevant impact parameters correspond a
bright ring region inside the shadow on the screen of a distant observer. See
the blue lines in Fig.(4).

Figure 4: The shadow and photon ring. The red circle stands for the light
ring, the yellow region on the left stands for the background light source and
the purple region on the right stands for screen of the observer. The green
lines indicate trajectories of light that can be observed on the screen and
the blue lines indicate trajectories of light that can’t be observed.

As we mentioned before, when the impact parameter $b$ approaches to $b_{c}$,
the deflection angles can be arbitrarily large. For $b<b_{c}$, the
trajectories of the light fall into the inside region of the light ring and
don’t intersect with the background light source. These correspond to a shadow
region on the screen of the distant observer. For $b>b_{c}$, the trajectories
with deflection angles belonging to the intervals
$(\frac{3}{2}\pi+2k\pi,\frac{5}{2}\pi+2k\pi),k=0,1,2,...$ correspond to a
$k^{\textrm{th}}$ order shadow region on the screen of the distant observer,
and the trajectories with deflection angles belonging to the intervals
$(\frac{5}{2}\pi+2k\pi,\frac{7}{2}\pi+2k\pi),k=0,1,2,...$ correspond to a
$k^{\textrm{th}}$ order bright ring on the screen of the distant observer. In
fact, except the $0^{\textrm{th}}$ order bright ring, the width of all higher
order bright rings is tiny. This can be seen in the following Table.1 and in
Fig.(6(d)). The trajectories with typical deflection angles,
$\varphi_{\textrm{def}}=\frac{3}{2}\pi,\frac{5}{2}\pi,\frac{7}{2}\pi,\frac{9}{2}\pi$,
are shown in Fig.5.

$\varphi_{\textrm{def}}$ | $3/2\ \pi$ | $5/2\ \pi$ | $7/2\ \pi$ | $9/2\ \pi$
---|---|---|---|---
b | 5.121380595 | 4.248622455 | 4.208645804 | 4.205903279
$\varphi_{\textrm{def}}$ | $11/2\ \pi$ | $13/2\ \pi$ | $15/2\ \pi$ | $17/2\ \pi$
b | 4.205903279 | 4.205891983 | 4.205891240 | 4.205891190
Table 1: Values of $b$ for some specific deflection angles.

(a) $\varphi_{\textrm{def}}=\frac{3}{2}\pi$

(b) $\varphi_{\textrm{def}}=\frac{5}{2}\pi$

(c) $\varphi_{\textrm{def}}=\frac{7}{2}\pi$

(d) $\varphi_{\textrm{def}}=\frac{9}{2}\pi$

Figure 5: Typical trajectories with different deflection angles.

The shadows on the screen of a distant observer of the QBH case and the SBH
case are shown in Fig.(6(a)) and Fig.(6(b)) respectively, and the comparison
between them is shown in Fig.(6(c)). One can see the radii of the shadow and
the bright ring of the QBH are both a little smaller than that of the SBH. A
demonstration of the $1^{\textrm{st}}$ and $2^{\textrm{nd}}$ order bright
rings are in Fig.(6(d)).

(a) The shadow and photon ring of the QBH.

(b) The shadow and photon ring of the SBH.

(c) The comparison of the shadows and photon rings between the QBH and SBH.

(d) A zoomed-in image from Fig.(6(a)) shows the fine structures of the photon
ring of the QBH. One can see the $0^{\textrm{th}}$ order photon ring is much
wider than the $1^{\textrm{st}}$ order photon ring.

Figure 6: The fine structure of the photon rings.

Now let’s consider the relation between the deflection angle
$\varphi_{\textrm{def}}$ and impact parameter $b$ when $b$ approaches to
$b_{c}$. From Eq.(3), we can express the deflection angle of the light as a
function of $b$,

$\varphi_{\textrm{def}}=2\int_{r_{\textrm{turn}}}^{\infty}\frac{dr}{r^{2}\sqrt{\frac{1}{b^{2}}-U_{\textrm{MS}}(r)}},$
(16)

where $r_{\textrm{turn}}$ is the shortest distance from a trajectory to the
BH. $r_{\textrm{turn}}$ is determined by the following equation

$\frac{1}{b^{2}}=U_{\textrm{MS}}(r)|_{r=r_{\textrm{turn}}}.$ (17)

When $b=b_{c}$, $r_{\textrm{turn}}=r_{\textrm{MS}}$. Then we present an
approximation of the above relation when $b\rightarrow b_{c}^{+}$. We
introduce a cut-off $\delta$ ($\delta\ll r_{\textrm{turn}}$) to divide the
above integral into two parts:

$\displaystyle\int_{r_{\textrm{turn}}}^{\infty}\frac{dr}{r^{2}\sqrt{\frac{1}{b^{2}}-U_{\textrm{MS}}(r)}}=\int_{r_{\textrm{turn}}}^{r_{\textrm{turn}}+\delta}{\frac{dr}{r^{2}\sqrt{\frac{1}{b^{2}}-U_{\textrm{MS}}(r)}}}+\int_{r_{\textrm{turn}}+\delta}^{\infty}{\frac{dr}{r^{2}\sqrt{\frac{1}{b^{2}}-U_{\textrm{MS}}(r)}}}.$
(18)

The second part gives a finite function $C_{\delta}(b)$ of $b$ even when
$b=b_{\textrm{c}}$. When $b\rightarrow b_{c}^{+}$, the first part gives the
divergent contribution, so we just need to consider the approximation of the
first part. When $b\rightarrow b_{c}^{+}$, we have
$r_{\textrm{turn}}\rightarrow r_{l}$, and we can approximate
$U_{\textrm{MS}}(r)$ as

$U_{\textrm{MS}}(r)\simeq\frac{1}{b_{\textrm{c}}^{2}}-\frac{\omega^{2}}{2}{(r-r_{l})}^{2},$
(19)

where $\omega^{2}=-U^{{}^{\prime\prime}}_{\textrm{MS}}(r_{l})$. Define a
parameter $\epsilon=1-\frac{b_{c}^{2}}{b^{2}}(b>b_{c})$ to measure the
closeness to $b_{c}$ of $b$. Then the turning point $r_{\textrm{turn}}$ is
determined by the following equation

$\frac{\omega^{2}}{2}{(r_{\textrm{turn}}-r_{l})}^{2}=\frac{\epsilon}{b_{\textrm{c}}^{2}}.$
(20)

Considering $r_{\textrm{turn}}>r_{l}$, we have

$r_{\textrm{turn}}=r_{l}+\frac{\sqrt{2\epsilon}}{\omega b_{\textrm{c}}}.$ (21)

The first part of Eq.(18) can be estimated as

$\displaystyle\int_{r_{\textrm{turn}}}^{r_{\textrm{turn}}+\delta}\frac{dr}{r^{2}\sqrt{\frac{1}{b^{2}}-U_{\textrm{MS}}(r)}}\simeq\frac{1}{r_{l}^{2}}\int_{r_{\textrm{turn}}}^{r_{\textrm{turn}}+\delta}\frac{dr}{\sqrt{\frac{1}{b^{2}}-U_{\textrm{MS}}(r)}}\simeq-\frac{1}{\sqrt{2}\omega
r_{l}^{2}}\zeta,$ (22)

where

$\zeta=\ln{\frac{\left(b_{\textrm{c}}\omega\delta-\sqrt{b_{\textrm{c}}\omega\delta(b_{\textrm{c}}\omega\delta+2\sqrt{2\epsilon})}+\sqrt{2\epsilon}\right)^{2}}{2\epsilon}}$
(23)

Expand $\zeta$ with respect to $\epsilon$, we could obtain an approximation of
Eq.(22)

$\int_{r_{\textrm{turn}}}^{\delta}\frac{dr}{r^{2}\sqrt{\frac{1}{b^{2}}-U_{\textrm{MS}}(r)}}\simeq-\frac{1}{\sqrt{2}\omega
r_{l}^{2}}\log{(\epsilon)}.$ (24)

By substituting $\epsilon=1-\frac{b_{\textrm{c}}^{2}}{b^{2}}$ into the above
equation and considering the splitting in Eq.(18), we have

$\varphi_{\textrm{def}}=-\frac{\sqrt{2}}{\omega
r_{l}^{2}}\log{\left(b-b_{\textrm{c}}\right)+\widetilde{C}_{\delta}(b)}.$ (25)

Thus, we obtain a log-divergence for $\varphi_{\textrm{def}}$ when
$b\rightarrow b_{c}^{+}$. The function $\widetilde{C}_{\delta}(b)$ is the
remaining finite contribution even when $b=b_{c}$. In Fig.(7), we plot
$\dfrac{\varphi_{\textrm{def}}}{\pi}$ with respect to $\log(b-b_{\textrm{c}})$
based on the accurate integration Eq.(16), and show that it is a linear curve
when $b-b_{\textrm{c}}\to 0^{+}$, which is consistent with the approximation.

Figure 7: The numerical plot of $\varphi_{\textrm{def}}$ with respect to
$\log(b-b_{\textrm{c}})$ when $b$ tends to $b_{\textrm{c}}$.

Finally, let’s consider the coefficient $-\frac{\sqrt{2}}{\omega
r_{l}^{2}}\equiv h$ of the log-divergence term in the above approximation
(25). According to the expression of the effective potential, Eq.(4), and the
definition of the dimensionless parameters, Eq.(2), we can get

$U_{\textrm{MS}}^{\prime\prime}(r)|_{r_{l}}=\frac{9-8y_{\textrm{MS}}}{16M^{4}y_{\textrm{MS}}^{5}}.$
(26)

Since $U^{\prime\prime}(r_{l})=-\omega^{2}$, we have

$\omega=\sqrt{-\frac{9-8y_{\textrm{MS}}}{16M^{4}y_{\textrm{MS}}^{5}}}.$ (27)

Substitute the linear approximation of $z$ in Eq.(11) into the above equation,
we obtain

$\omega=81\sqrt{\frac{-54+32z}{M^{4}(-27+4z)^{5}}}.$ (28)

Together with $r_{l}=2My_{\textrm{MS}}$, we expand the coefficient
$h=-\frac{\sqrt{2}}{\omega r_{l}^{2}}$ with respect to $z$,

$h=h\left(0\right)+z\left.\frac{dh}{dz}\right|_{z=0}+\mathcal{O}(z^{2}),$ (29)

and we obtain the explicit expression for the coefficient $h$ at first-order
quantum correction

$h=1+\frac{2}{9}z+\mathcal{O}(z^{2})=1+\frac{\alpha}{18M^{2}}+\mathcal{O}(\alpha^{2}).$
(30)

When the quantum correction $\alpha=0$, we have $\left.h\right|_{\alpha=0}=1$
which is the corresponding coefficient in Schwarzschild case that was studied
in [47].

##  4 Shadows of QBHs under various illumination backgrounds

In this section, we consider the shadows and photon rings of QBHs under three
illumination models, and compare them with that of the Schwarzschild BH cases.
Model 1 and 2 are similar to the models considered in the first and third rows
in Fig.(6) in reference [47]. A relatively thick accretion disk is considered
in Model 3.

In Model 1, the background light is emitted from an optically thin but
geometrically thick disk region near the black hole. The emission disk extends
from the BH horizon to spatial infinity. The thickness of the disk is chosen
as three times of the radius of the light ring radius, i.e. $3r_{l}$. We
assume the disk emits isotropically with uniform brightness. It is also
considered that the screen is infinitely far away from the BH and is
perpendicular to the rotationally symmetric axis of the disk.

In Model 2, the background light is emitted from a spherical cloud near the
black hole. The emission region extends from the photon sphere to the exterior
region of the BH. The emission in this model is locally isotropic, but the
emitted density is exponentially decaying with respect to the radial
coordinate, i.e. the emission intensity $\rho$ is chosen as
$\rho=\rho_{0}\exp{(-r^{2}/M^{2})},r\geq r_{l}$.

In Model 3, the background light is emitted from a relatively thick accretion
disk. The thickness of the disk is chosen as $1$ in the unit of black hole
mass $M$. The emission profile is chosen as $\rho=\rho_{0}(r-2M)^{-3},r\geq
r_{l}$.

Now we consider the brightness of the image of a BH and the surrounding
emitting matter. One should integrate the emissivity along the light ray that
reaches the screen [47]. To simplify the calculations, we ignore the redshift
effects since the relative brightness of different regions of the image is
more important. Quantitatively, the brightness of a point in the image with
light ray $\mathcal{C}$ is proportional to

$\displaystyle B_{\mathcal{C}}=\int_{\mathcal{C}}\rho dl.$ (31)

Since the rotational symmetry of our models, the above integral can be reduced
to an 2-dimensional integral. We choose $(r,\varphi)$ as the polar
coordinates. With the equation of motion of photon, the brightness corresponds
to light a path with impact parameter $b$ is proportional to

$\displaystyle
B(b)=\frac{1}{b}\int_{0}^{\varphi_{\text{def}}(b)}\rho(r,\varphi)r^{2}f^{-1/2}d\varphi,$
(32)

where $r$ is a function of $\varphi$ determined by Eq.(3).

(a) Model 1 : Thick disk

(b) Model 2 : Spherical cloud

(c) Model 3 : Accretion disk

(d) Comparism of images between the LQG (LHS) and Schwarzschild’s (RHS) metric
with model 1.

(e) Comparism of images between the LQG (LHS) and Schwarzschild’s (RHS) metric
with model 2.

(f) Comparism of images between the LQG (LHS) and Schwarzschild’s (RHS) metric
with model 3.

Figure 8: Shadows and photon rings of the QBH from LQG model in three
illumination models and comparisons with that of the SBH.

In Fig.(8), the three plots in the first row show the brightness with respect
to the impact parameter $b$ (different paths) in the three illumination
models. We also compare the brightness of the images of the QBH from LQG and
SBH in each illumination model. The blue line is for the QBH and the orange
line for the SBH. In Model 1 and Model 2, the differences between the two BHs
are both qualitatively and quantitatively small. In Model 3, the difference is
quantitatively large. This is interesting since it means we can distinguish
the QBH from the SBH under certain specific conditions.

The second row shows the images of shadows and photon rings of the QBH in
three illumination models and the comparisons with that of the SBH cases. The
photon rings are clear in Model 1 and Model 3 in the disklike emission cases.
However, Model 2 has a spherical emission and the photon-ring feature is not
obvious.

##  5 Conclusion

We study the shadows and the photon ring structures of a quantum black hole
arising from a loop quantum cosmology model. We calculate the radius of the
light ring of the quantum black hole, and give an approximation of it,
$r_{l}\simeq 3M-\frac{\alpha}{9M}$, at the first order of the quantum
correction parameter $\alpha$. Considering a simple model where we assume the
quantum black hole is backlit by a large and distant plane of uniform and
isotropic emission, we calculate the radius of the shadow, which is
$r_{s}=3\sqrt{3}M-\frac{\alpha}{6\left(\sqrt{3}M\right)}$ at the first order
of the quantum correction parameter. We then study the structure of the photon
ring, and compare the shadow and photon ring of the quantum black hole with
that of the SBH. The quantum correction term makes the radius of the shadow
smaller than that of the Schwarzschild black hole, which is consistent with
the claim in [49]. We also estimate the relationship between the deflection
angle $\varphi_{\textrm{def}}$ and the impact parameter of a light ray. When
$b$ is approaching to the critical value $b_{c}$, we obtain a logarithmic
relationship $\varphi_{\textrm{def}}\simeq\frac{\sqrt{2}}{\omega
r_{l}^{2}}\log\left(b-b_{\textrm{c}}\right)$.

We then plot the images of the shadows and photon rings of the QBH under three
illumination conditions. And we also compare the images of QBH with that of
the SBH in each illumination models. In Model 1 and Model 2, there are only
small quantitative difference between the images of the two kinds of BHs. In
Model 3, the difference of the images is quantitatively large. This means we
could distinguish the QBH and the SBH under some specific conditions from the
shadow images.

## Acknowledgments

The authors thank Jingbo Yang, Zhan-Feng Mai and Cong Zhang for useful
discussions and communications. This work is partially supported by Guangdong
Major Project of Basic and Applied Basic Research (No.2020B0301030008).

## References

  * [1] R. Penrose, Phys. Rev. Lett. 14, 57-59 (1965).
  * [2] S. W. Hawking and R. Penrose, Proc. Roy. Soc. Lond. A 314, 529-548 (1970).
  * [3] C. Rovelli, Quantum Gravity (Cambridge University Press, Cambridge, England, 2004).
  * [4] T. Thiemann, Modern Canonical Quantum General Relativity (Cambridge University Press, Cambridge, England, 2007).
  * [5] T. Thiemann, Lect. Notes Phys. 631, 41 (2003).
  * [6] A. Ashtekar, J. Lewandowski, Class. Quant. Grav. 21, R53 (2004).
  * [7] M. Han, Y. Ma, W. Huang, Int. J. Mod. Phys. D 16, 1397 (2007).
  * [8] K. Giesel, H. Sahlmann, Proc. Sci., QGQGS2011, 002 (2011).
  * [9] C. Rovelli, Proc. Sci., QGQGS2011, 003 (2011).
  * [10] A. Perez, Living Rev. Relativity 16, 3 (2013).
  * [11] M. Bojowald, Phys. Rev. Lett. 86, 5227-5230 (2001).
  * [12] A. Ashtekar, M. Bojowald, J. Lewandowski, Adv. Theor. Math. Phys. 7, 233 (2003).
  * [13] A. Ashtekar, T. Pawlowski, P. Singh, Phys. Rev. Lett. 96, 141301 (2006).
  * [14] A. Ashtekar, T. Pawlowski and P. Singh, Phys. Rev. D 73, 124038 (2006).
  * [15] A. Ashtekar, T. Pawlowski, P. Singh, Phys. Rev. D 74, 084003 (2006).
  * [16] A. Ashtekar, M. Bojowald, Class. Quant. Grav. 23, 391 (2006).
  * [17] C. G. Boehmer and K. Vandersloot, Phys. Rev. D 76, 104030 (2007).
  * [18] L. Modesto, Int. J. Theor. Phys. 49, 1649-1683 (2010).
  * [19] A. Corichi and P. Singh, Class. Quant. Grav. 33, 055006 (2016).
  * [20] A. Ashtekar, J. Olmedo and P. Singh, Phys. Rev. Lett. 121, 241301 (2018).
  * [21] A. Ashtekar, J. Olmedo and P. Singh, Phys. Rev. D 98, 126003 (2018).
  * [22] N. Bodendorfer, F. M. Mele and J. Münch, Class. Quant. Grav. 36, 195015 (2019).
  * [23] C. Zhang, Y. Ma, S. Song and X. Zhang, Phys. Rev. D 102, 041502(2020).
  * [24] C. Zhang, Y. Ma, S. Song and X. Zhang, Phys. Rev. D 105, 024069 (2022).
  * [25] M. Bojowald, R. Goswami, R. Maartens, P. Singh, Phys. Rev. Lett. 95, 091302 (2005).
  * [26] M. Bojowald, J.D. Reyes, R. Tibrewala, Phys. Rev. D 80, 084002 (2009).
  * [27] J.G. Kelly, R. Santacruz, E. Wilson-Ewing, Class. Quant. Grav. 38, 04LT01 (2021).
  * [28] J. Ben Achour, S. Brahma, J.P. Uzan, J. Cosmol. Astropart. Phys. 03, 041 (2020).
  * [29] J. Ben Achour, S. Brahma, S. Mukohyama, J.P. Uzan, J. Cosmol. Astropart. Phys. 09, 020 (2020).
  * [30] J. Münch, Class. Quant. Grav. 38, 175015 (2021).
  * [31] J. Münch, Phys. Rev. D 104, 046019 (2021).
  * [32] V. Husain, J.G. Kelly, R. Santacruz, E. Wilson-Ewing, Phys. Rev. Lett. 128, 121301 (2022).
  * [33] K. Giesel, B.F. Li, P. Singh, Phys. Rev. D 104, 106017 (2021).
  * [34] J. Lewandowski, Y. Ma, J. Yang and C. Zhang, Phys. Rev. Lett. 130, no.10, 101501 (2023).
  * [35] J. Yang, C. Zhang and Y. Ma, Eur. Phys. J. C 83, no.7, 619 (2023).
  * [36] C. Zhang, Y. Ma and J. Yang, Phys. Rev. D 108, no.10, 104004 (2023).
  * [37] K. Akiyama et al. [Event Horizon Telescope], Astrophys. J. Lett. 875, L1 (2019).
  * [38] K. Akiyama et al. [Event Horizon Telescope], Astrophys. J. Lett. 875, no.1, L4 (2019).
  * [39] K. Akiyama et al. [Event Horizon Telescope], Astrophys. J. Lett. 930, no.2, L12 (2022).
  * [40] K. Akiyama et al. [Event Horizon Telescope], Astrophys. J. Lett. 930, no.2, L14 (2022).
  * [41] P. V. P. Cunha and C. A. R. Herdeiro, Phys. Rev. Lett. 124, no.18, 181101 (2020).
  * [42] M. Guo and S. Gao, Phys. Rev. D 103, no.10, 104031 (2021).
  * [43] S. W. Wei, Phys. Rev. D 102, no.6, 064039 (2020).
  * [44] R. Ghosh and S. Sarkar, Phys. Rev. D 104, no.4, 044019 (2021).
  * [45] P. V. P. Cunha, C. A. R. Herdeiro and E. Radu, Phys. Rev. D 96, no.2, 024039 (2017).
  * [46] P. V. P. Cunha, C. A. R. Herdeiro, E. Radu and H. F. Runarsson, Phys. Rev. Lett. 115, no.21, 211102 (2015).
  * [47] S. E. Gralla, D. E. Holz and R. M. Wald, Phys. Rev. D 100, no.2, 024018 (2019).
  * [48] M. Cvetic, G. W. Gibbons and C. N. Pope, Phys. Rev. D 94, no.10, 106005 (2016).
  * [49] H. Lu and H. D. Lyu, Phys. Rev. D 101, no.4, 044059 (2020).
  * [50] L. Ma and H. Lu, Phys. Lett. B 807, 135535 (2020).
  * [51] R. Q. Yang and H. Lu, Eur. Phys. J. C 80, no.10, 949 (2020).
  * [52] M. Wielgus, Phys. Rev. D 104, no.12, 124058 (2021).
  * [53] M. Guo and P. C. Li, Eur. Phys. J. C 80, no.6, 588 (2020).
  * [54] A. Das, A. Saha and S. Gangopadhyay, Eur. Phys. J. C 80, no.3, 180 (2020).
  * [55] A. Övgün and İ. Sakallı, Class. Quant. Grav. 37, no.22, 225003 (2020).
  * [56] R. Kumar, S. G. Ghosh and A. Wang, Phys. Rev. D 101, no.10, 104001 (2020).
  * [57] R. Shaikh, Phys. Rev. D 100, no.2, 024028 (2019).
  * [58] S. W. Wei, Y. X. Liu and R. B. Mann, Phys. Rev. D 99, no.4, 041303 (2019).
  * [59] Z. Zhong, Z. Hu, H. Yan, M. Guo and B. Chen, Phys. Rev. D 104, no.10, 104028 (2021).
  * [60] P. C. Li, M. Guo and B. Chen, Phys. Rev. D 101, no.8, 084041 (2020).
  * [61] R. Shaikh, P. Kocherlakota, R. Narayan and P. S. Joshi, Mon. Not. Roy. Astron. Soc. 482, no.1, 52-64 (2019).
  * [62] H. Huang, J. Kunz, J. Yang and C. Zhang, Phys. Rev. D 107, no.10, 104060 (2023).
  * [63] A. Ashtekar, J. Olmedo and P. Singh, Phys. Rev. Lett. 121, no.24, 241301 (2018).
  * [64] C. Zhang, Phys. Rev. D 104, no.12, 126003 (2021).
  * [65] K. A. Meissner, Class. Quant. Grav. 21, 5245-5252 (2004).
  * [66] M. Domagala and J. Lewandowski, Class. Quant. Grav. 21, 5233-5244 (2004).
  * [67] S. Chandrasekhar, The Mathematical Theory of Black Holes, Oxford University Press, p123, 1983.
  * [68] Y. X. Chen, J. H. Huang and H. Jiang, Phys. Rev. D 107, no.4, 044066 (2023).
  * [69] G. S. Bisnovatyi-Kogan, O. Y. Tsupko and V. Perlick, PoS MULTIF2019, 009 (2019).
  * [70] G. S. Bisnovatyi-Kogan and O. Y. Tsupko, Phys. Rev. D 105, no.6, 064040 (2022).