# ASTRO: An AST-Assisted Approach for Generalizable Neural Clone Detection

Yifan Zhang<EMAIL_ADDRESS>Vanderbilt University1400 18th St.
SNashvilleTN37212USA ,  Junwen Yang<EMAIL_ADDRESS>Meta, Inc.1 Hacker
WayMenlo ParkCA94025USA ,  Haoyu Dong<EMAIL_ADDRESS>Duke University2424
Erwin RdDurhamNC27705USA ,  Qingchen Wang<EMAIL_ADDRESS>University of Hong
KongPokfulam RdHong Kong SAR37212 ,  Huajie Shao<EMAIL_ADDRESS>College of
William & MaryMcGlothlin-Street Hall 126 251 Jamestown
RdWilliamsburgVA23185USA ,  Kevin Leach<EMAIL_ADDRESS>Vanderbilt
University1400 18th St. SNashvilleTN37212USA and  Yu Huang
<EMAIL_ADDRESS>Vanderbilt University1400 18th St.
SNashvilleTN37212USA

(2022)

###### Abstract.

Neural clone detection has attracted the attention of software engineering
researchers and practitioners. However, most neural clone detection methods do
not generalize beyond the scope of clones that appear in the training dataset.
This results in poor model performance, especially in terms of model recall.
In this paper, we present an Abstract Syntax Tree (AST) assisted approach for
geneRalizable neural clone detectiOn, or ASTRO, a framework for finding clones
in codebases reflecting industry practices. We present three main components:
(1) an AST-inspired representation for source code that leverages program
structure and semantics, (2) a global graph representation that captures the
context of an AST among a corpus of programs, and (3) a graph embedding for
programs that, in combination with extant large-scale language models,
improves state-of-the-art code clone detection. Our experimental results show
that ASTRO improves state-of-the-art neural clone detection approaches in both
recall and F-1 scores.

††copyright: acmcopyright††journalyear: 2022††doi:
XXXXXXX.XXXXXXX††conference: Automated Software Engineering Industrial
Showcase; 2022; Ann Arbor, Michigan††price: 15.00††isbn:
978-1-4503-XXXX-X/18/06

##  1\. Introduction

Code Clones refer to syntactically or functionally similar code fragments
within or between software systems (Sajnani, 2016). Having clones in
industrial codebases can substantially increase maintenance costs (Jnanamurthy
et al., 2019). For example, 22.3% of operating systems’ defects were
introduced by code cloning (Zhang et al., 2021). Moreover, clones can also
lead to intellectual property violations and security problems (Duan et al.,
2017). With the increasing scale of modern software, clone detection (CD)
techniques have been developed to automatically detect clones within corpora
of code. Traditional CD approaches aim to find similar code patterns by
leveraging abstract representations, including tokens (Lee and Jeong, 2005;
Wang et al., 2018), Abstract Syntax Trees (AST) (Baxter et al., 1998; Jiang et
al., 2007), Program Dependency Graphs (PDG) (Zou et al., 2020), and other
intermediate representations. However, these methods frequently focus on one
representation but ignore others, leading to a failure to generalize across
diverse software corpora.

With the rise of neural networks, CD techniques have leveraged neural
architectures that learn program semantics and structure (Lei et al., 2022;
Wang et al., 2020). For example, White et al. (White et al., 2016) learns both
lexical and syntactic embeddings by sequence modeling, Zhang et al. (Zhang et
al., 2019) incorporates code structural information within a Long Short Term
Memory (LSTM) to improve detection performance, and Yu et al. (Yu et al.,
2019) applies machine learning to this task. These neural CD approaches are
based on learning patterns of semantics in the dataset and follow common
practices in deep learning. While these models achieve high precision, their
applications are restricted to the domain of semantics present in the programs
that comprise the training data. That is, when code fragments from other
domains are placed as input to such models, the detection performance as
measured by recall is close to zero. This inability to generalize across
software domains can be problematic due to the large scale of industrial
software codebases.

Figure 1. Diagram of ASTRO. It consists of three main components: (a) AST
parsing and truncation 3.1, (b) global AST graph learning and matching 3.2 and
(c) pretrained and subgraph embedding extraction 3.3. The global AST graph in
(b) is built from the training data. In practice, Code A and Code B would be
snippets of source that are not part of the training set.

In this paper, we propose ASTRO, a novel framework that combines neural
network architectures, Abstract Syntax Trees, and context-aware program
structure to quickly and accurately detect code clones. Our approach is based
on two key insights derived from domain knowledge in software engineering: (1)
identical code snippets share identical AST structures. Even though large
programs can have large ASTs, there is a limited number of node types and
topologies of ASTs for a given programming language. This insight allows us to
capture the key structural characteristics of a program and detect code clones
that only differ on detailed design (i.e., variable names, adding or deleting
statements, etc.). (2) Unseen code snippets can be represented and retrieved
with a combination of previously-seen (sub-) ASTs. With this insight, we build
a global graph with training data (i.e., programs) that captures structural
and co-occurrence information of nodes in source ASTs (see Section 3.2).
Previous work such as ASTNN (Zhang et al., 2019) have also considered neural
networks and graph embeddings of ASTs as monolithic, end-to-end systems.
However, our approach can be used as a plug-in to existing pretrained models
(such as CodeBERT, RoBERTa, and ASTNN) to enhance the performance and
generalizability of code clone detection.

At a high level, ASTRO consists of three components, illustrated in Figure 1.
First, we parse all training programs into ASTs with $k$-hierarchy truncation
and build a global AST graph accordingly. Then, each pair of code snippets to
be considered for clone detection are queried from the global AST graph with
their truncated AST representations. Lastly, we introduce a novel strategy to
generate merged subtree embeddings for each code pair as plug-in to their
embeddings extracted from a pretrained model, and input the concatenated
embedding to our non-linear detection head to improve the generalizability of
clone detection. We elaborate on the design of ASTRO in Section 3.

We evaluate ASTRO using Big Clone Bench (Svajlenko and Roy, 2015) and compare
against typical CD methods (Jiang et al., 2007) and more recent neural CD
methods (White et al., 2016; Sajnani et al., 2016). Our results show that,
while ASTRO sacrifices precision by less than 0.19, it substantially improves
the recall by 0.85 and F-1 score by 0.80. This indicates that ASTRO is capable
of more readily generalizing clone detection across input programs across
software domains. The main contributions of this paper are:

  * $\bullet$

ASTRO, a graph-based and AST-guided neural network model for code clone
detection.

  * $\bullet$

A novel graph-based approach to capture inherent information of code snippets
for generalization.

  * $\bullet$

A novel approach to leverage domain knowledge in SE to improve the large-scale
pretrained models in AI.

  * $\bullet$

A systematic evaluation of ASTRO on neural clone detection and pretrained
model enhancement. ASTRO improves the recall and F1 score of neural CD
methods, and enhances the generalizability of large-scale pretrained models.

The rest of the paper is structured as follows: Section 2 highlights previous
related work, Section 3 describes the technical approach behind the ASTRO
framework, Section 4 presents our evaluation and discussion for ASTRO, and
Section 5 concludes the paper.

##  2\. Background and Related Work

Code clones are widespread and prevalent since developers frequently reuse
code via copy-paste to reduce programming effort. Previous studies indicate
that 12% of Linux code was identified as code clones (Kapser and Godfrey,
2003), and 19% of the X Window system functionality entails code clones
(Baker, 1995).

Unfortunately, code clones increase the likelihood of errors and software
defects. Many studies have indicated serious negative impacts of code clones
in software evolution, including hidden bug propagation (Mondal et al., 2017),
unintentional inconsistencies (Göde and Koschke, 2011), and high instability
(Mondal et al., 2012). Substantial research has been conducted to detect code
clones and related defects.

Traditional code clone detection. Before the renaissance of neural networks,
code clone detection was categorized in three ways: string-based, in which
substrings of program source code were directly compared (Baker, 1995), token-
based, in which lexical program token sequences were directly compared (Kamiya
et al., 2002), and parser-based, in which graph structures like ASTs and PDGs
were directly compared (Baxter et al., 1998; Jiang et al., 2007). However,
these methods often overfit or miss complex cases in which code may be
semantically similar but structurally different (e.g., inlined functions,
renamed variables, or nested loops in different order).

Neural code clone detection More recently, several works based on neural
networks have been explored for code clone detection, which have focused on
learning abstract representations of code such as ASTs (Zhang et al., 2019),
PDGs (Zou et al., 2020), and CFGs (Fang et al., 2020). Although these methods
can achieve high precision (i.e., given a source code input, they can identify
a low number of clones), they fail to generalize when subtle changes exist in
code clones, resulting in poor recall (i.e., given a source code input, these
models cannot identify all clones of that input).

Figure 2. Illustration of the global AST graph construction and learning.
Subfigure (a) shows an example program and AST, Subfigure (b) shows how inter-
and intra-hierarchy edges are modeled in the global graph, and Subfigure (c)
shows how each node embedding is initialized and trained to form the global
graph.

##  3\. Approach

ASTRO consists of three main components, shown in Figure 1. Our task is to
identify whether two input code snippets are clones, thus it is a supervised
binary classification task. Given two source code snippets as input, we first
parse them into ASTs, then derive an embedding based upon fixed-depth walks of
each subtree in each AST (Section 3.1). Then, we generate a _global graph_
that captures the relationships that exist between subtrees of an AST (e.g.,
methods are defined in classes, thus method subtrees are related to class
subtrees). Our global graph construction allows for the creation of a merged
embedding that captures both structure (by matching ASTs and subtrees between
snippets) and semantics (by capturing the context in which various subtrees
appear) (Section 3.2). Finally, we use a non-linear detection head to
determine whether the two input snippets are clones (Section 3.3). We discuss
each step below.

###  3.1. AST Parsing and Truncation

Each pair of code for clone detection is first parsed into ASTs. In this
paper, we consider Java programs for our evaluation, we parse using the
JavaLang parser111JavaLang is available at https://github.com/c2nes/Javalang.
JavaLang’s abstract grammar contains 19 node types. As ASTs are recursive
structures, they can be an unlimited height depending on the size and
complexity of a given program. However, we hypothesize that very deep nodes in
the AST are less important for conveying generalizable program semantics than
more shallow nodes. Thus, we derive an embedding of an input program’s AST by
truncating the tree to a fixed height $k$. We refer to each layer of sibling
nodes as a _hierarchy_ of nodes kept within the truncated AST. In our
experiments, we set $k=5$, meaning we keep all nodes of height 5 or less from
the input AST.

###  3.2. Global AST Graph Learning and Subgraph Query

To address the challenge of generalizability, we construct a _global AST
graph_ using truncated ASTs obtained from the previous step (Section 3.1). We
leverage two pieces of information from all programs in a training corpus to
build the global AST graph: (1) the types of nodes in the ASTs, and (2) edges
between nodes within each AST. Both are available at each level of each AST.
An indicative example is shown in Figure 2. The input program is parsed into
an AST, from which $k$ hierarchies of nodes are kept (Figure 2 (a)). When
quantifying the co-occurrences of different nodes within an AST, we model
_inter-hierarchy_ edges (e.g., ForStatement $\rightarrow$ Assignment) and
_intra-hierarchy_ edges (e.g., Assignment $\rightarrow$ MethodInvocation) in
the construction of the global AST graph (Figure 2 (b)). Each AST node is
represented by an embedding (256 dimensions in our experiments), which
reflects an average of embeddings of each child (Figure 2 (c)).

To build the global AST graph, we apply unsupervised graph representation
learning with the GraphSAGE algorithm (Hamilton et al., 2017). GraphSAGE is a
graph representation learning method that learns the embeddings of nodes in
the graph based on graph structure and node features. Before training the
global graph representation, we use Flair word embeddings (Akbik et al., 2019)
to initialize the word embedding for each name of each node type (e.g.,
“ForControl”) in the AST with a dimension of 256 (adopted from GraphSAGE). For
sequence aggregation in GraphSAGE, we apply the default mean aggregator and
obtain a 256-dimension embedding vector for every node in the global graph.
Due to the large scale of edge modeling in the global AST graph, we set the
sampling ratio of all edges to 0.1% to mitigate the risk of the over smoothing
problem.

With the global AST graph trained by the training corpus of programs, to
compare whether two code snippets are clones, we make queries of their ASTs
against the global graph to obtain their corresponding node representations
(i.e., a subgraph from the global graph), as shown in Figure 1 (a) and (b). In
the query process, we finalize the representation for the subgraph
(corresponding to the given AST) using all its node embeddings based on its
tree structure. Specifically, starting from the bottom of the AST, we take the
average of all embeddings of each node’s children, then repeat the process
until we reach the top of the tree. The final subgraph embedding of the AST is
a single 256-dimensional vector that reflects an average of all of the nodes
in the AST.

###  3.3. Pretrained and Subgraph Embedding Extraction

Finally, we concatenate the subgraph embedding (Section 3.2) with a pretrained
embedding (of the source code tokens) as our final ASTRO embedding to be used
for clone detection, shown in Figure 1 (c). In ASTRO, the pretrained embedding
can be adopted from any state-of-the-art language model, such as RoBERTa (Liu
et al., 2019) or CodeBERT (Feng et al., 2020). Since the AST representation
only reflects program syntax and structure, we further introduce program
semantics by adopting pretrained code models to our detection framework. For
example, by adopting RoBERTa, the final code embedding concatenates a
768-dimension pretrained embedding with our 256-dimension subgraph embedding.
This strategy improves the overall representation of the input code snippets
for the clone detection task.

To make a final prediction, we integrate a non-linear detection head for clone
detection in ASTRO, following common practice. Specifically, for our
experiments, we adopt the detection head in CodeBERT (Feng et al., 2020) and
simplify the structure to three $\mathrm{tanh}$-activated non-linear layers
with 512, 256, and 128 hidden nodes, respectively, to test the effectiveness
of our ASTRO plug-in. We use a learning rate of 1e-3, dropout rate of 0.1,
batch size of 512, and 50 epochs during training.

##  4\. Empirical Evaluation

###  4.1. Dataset

We evaluate ASTRO on Big Clone Bench (BCB) (Svajlenko and Roy, 2015), a
widely-used dataset for clone detection. The training, validation, and testing
splits follow the state-of-the-art work: 901,028 training code pairs, and
415,416 validation/testing code pairs.

###  4.2. Evaluation Results

We evaluate the performance of ASTRO with three metrics: precision, recall,
and F1 score. We compare ASTRO with multiple traditional and neural CD models.
As shown in Table 1, in the clone classification task, Deckard (Jiang et al.,
2007), DLC (White et al., 2016), and SourcererCC (Sajnani et al., 2016) models
have relatively higher precision, but very low total recall, indicating their
ineffectiveness in revealing structurally similar but semantically diverse
clones. In contrast, ASTRO sacrifices a trivial amount of precision (- 0.19)
compared with DLC, but achieves much higher recall (+ 0.85) and F1 (+ 0.80),
demonstrating effectiveness in revealing structurally similar clones.

Table 1. Quantitative evaluation of ASTRO and other neural clone detection methods using precision, recall and F1 score. Results of Deckard, DLC and SourcererCC are from (Wei and Li, 2017). Approaches | Precision | Recall | F1
---|---|---|---
Deckard (Jiang et al., 2007) | 0.93 | 0.02 | 0.03
DLC (White et al., 2016) | 0.95 | 0.01 | 0.01
SourcererCC (Sajnani et al., 2016) | 0.88 | 0.02 | 0.03
ASTRO (Ours) | 0.76 | 0.86 | 0.81

###  4.3. Ablation Studies

Table 2. Ablation studies of ASTRO (Graph). (A) removes embedding merging in the subgraph, (B) removes edge sampling before training global graph representation, and (C) removes the whole graph learning and uses word embeddings. Approaches | Precision | Recall | F1 | Recall Diff
---|---|---|---|---
ASTRO (Graph) | 0.16 | 0.79 | 0.25 | N/A
No Merged Embedding (A) | 0.16 | 0.59 | 0.25 |  - 25.3%
No Edge Sampling (B) | 0.16 | 0.65 | 0.25 |  - 17.7%
No Graph Learning (C) | 0.16 | 0.51 | 0.24 |  - 35.4%

In this section, we study the contributions of each proposed module in the
subgraph embeddding of ASTRO as input into detection head without
concatenating the pretrained embedding, denoted as ASTRO (Graph). Experimental
results of three ablation studies are shown in Table 2.

In the first row, ASTRO (Graph) indicates the use of subgraph embeddings
alone, without any additional concatenation to other embeddings. In (A), we
remove the upward merging process of AST hierarchies and use average
embeddings of all AST nodes. It shows a decrease of 0.20 in recall, indicating
that merged embeddings maintain critical hierarchical information of program
code. In (B), due to the large quantity of edges, we increase the sampling
rate from 0.1% to 10% to imitate the learning of GraphSAGE without edge
sampling. This shows a decrease of 0.14 in recall, indicating possible over-
smoothing in graph representation learning. In (C), we directly apply 2148
dimensional Flair word embedding from the names of node types. This leads to a
reduction in recall by 0.28, demonstrating the necessity of use graph
embedding to improve node representation quality.

###  4.4. Discussion

We further discuss the generalizability of ASTRO with respect to pretrained
models by comparing performance with and without our embedding in combination
with a pretrained embedding. As shown in Table 3, when input to the same non-
linear detection head, concatenating our embedding to RoBERTa and CodeBERT
embeddings improves F1 scores by 2.8% and 6.6%, respectively, along with much
higher recall. This demonstrates that our approach improves generalizability
to additional code clones previously undetected by state-of-the-art models.

Table 3. Discussion of generalizability of pretrained models. *The RoBERTa and CodeBERT models in this table are fine-tuned with our simplified non-linear detection head.  Approaches | Precision | Recall | F1 | F1 Dif
---|---|---|---|---
RoBERTa∗(Liu et al., 2019) | 0.69 | 0.73 | 0.72 | N/A
CodeBERT∗(Feng et al., 2020) | 0.70 | 0.84 | 0.76 | N/A
ASTRO (RoBERTa) | 0.65 | 0.86 | 0.74 |  + 2.8%
ASTRO (CodeBERT) | 0.76 | 0.86 | 0.81 |  + 6.6%

##  5\. Conclusion

In conclusion, we present a novel approach that leverages structural
information from ASTs combined with pretrained models to build a code clone
detection framework suitable for industrial codebases. Our results demonstrate
an improvement on generalizability, and can be used in tandem with existing
language models to improve recall and F1 scores overall. We hope this work can
inspire additional work addressing generalizability issues in neural clone
detection, especially with respect to improving recall for software across
domains, as in real-life industrial applications.

## References

  * (1)
  * Akbik et al. (2019) Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019\.  FLAIR: An easy-to-use framework for state-of-the-art NLP. In _NAACL 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_. 54–59.
  * Baker (1995) Brenda S Baker. 1995\.  On finding duplication and near-duplication in large software systems. In _Proceedings of 2nd Working Conference on Reverse Engineering_. IEEE, 86–95.
  * Baxter et al. (1998) Ira D Baxter, Andrew Yahin, Leonardo Moura, Marcelo Sant’Anna, and Lorraine Bier. 1998\.  Clone detection using abstract syntax trees. In _Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272)_. IEEE, 368–377.
  * Duan et al. (2017) Ruian Duan, Ashish Bijlani, Meng Xu, Taesoo Kim, and Wenke Lee. 2017.  Identifying open-source license violation and 1-day security risk at large scale. In _Proceedings of the 2017 ACM SIGSAC Conference on computer and communications security_. 2169–2185.
  * Fang et al. (2020) Chunrong Fang, Zixi Liu, Yangyang Shi, Jeff Huang, and Qingkai Shi. 2020.  Functional code clone detection with syntax and semantics fusion learning. In _Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis_. 516–527.
  * Feng et al. (2020) Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020\.  Codebert: A pre-trained model for programming and natural languages.  _arXiv preprint arXiv:2002.08155_ (2020).
  * Göde and Koschke (2011) Nils Göde and Rainer Koschke. 2011.  Frequency and risks of changes to clones. In _Proceedings of the 33rd International Conference on Software Engineering_. 311–320.
  * Hamilton et al. (2017) Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017\.  Inductive representation learning on large graphs.  _Advances in neural information processing systems_ 30 (2017).
  * Jiang et al. (2007) Lingxiao Jiang, Ghassan Misherghi, Zhendong Su, and Stephane Glondu. 2007.  Deckard: Scalable and accurate tree-based detection of code clones. In _29th International Conference on Software Engineering (ICSE’07)_. IEEE, 96–105.
  * Jnanamurthy et al. (2019) HK Jnanamurthy, Raoul Jetley, Frans Henskens, David Paul, Mark Wallis, and Sithu D Sudarsan. 2019\.  Analysis of industrial control system software to detect semantic clones. In _2019 IEEE International Conference on Industrial Technology (ICIT)_. IEEE, 773–779.
  * Kamiya et al. (2002) Toshihiro Kamiya, Shinji Kusumoto, and Katsuro Inoue. 2002\.  CCFinder: A multilinguistic token-based code clone detection system for large scale source code.  _IEEE Transactions on Software Engineering_ 28, 7 (2002), 654–670.
  * Kapser and Godfrey (2003) Cory Kapser and Michael W Godfrey. 2003.  Toward a taxonomy of clones in source code: A case study.  _Evolution of large scale industrial software architectures_ 16 (2003), 107–113.
  * Lee and Jeong (2005) Seunghak Lee and Iryoung Jeong. 2005.  SDD: high performance code clone detection system for large scale source code. In _Companion to the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications_. 140–141.
  * Lei et al. (2022) Maggie Lei, Hao Li, Ji Li, Namrata Aundhkar, and Dae-Kyoo Kim. 2022.  Deep learning application on code clone detection: A review of current knowledge.  _Journal of Systems and Software_ 184 (2022), 111141.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Mondal et al. (2012) Manishankar Mondal, Chanchal K Roy, Md Saidur Rahman, Ripon K Saha, Jens Krinke, and Kevin A Schneider. 2012.  Comparative stability of cloned and non-cloned code: An empirical study. In _Proceedings of the 27th Annual ACM Symposium on Applied Computing_. 1227–1234.
  * Mondal et al. (2017) Manishankar Mondal, Chanchal K Roy, and Kevin A Schneider. 2017\.  Bug propagation through code cloning: An empirical study. In _2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)_. IEEE, 227–237.
  * Sajnani (2016) Hitesh Sajnani. 2016\.  _Large-scale code clone detection_.  University of California, Irvine.
  * Sajnani et al. (2016) Hitesh Sajnani, Vaibhav Saini, Jeffrey Svajlenko, Chanchal K Roy, and Cristina V Lopes. 2016\.  Sourcerercc: Scaling code clone detection to big-code. In _Proceedings of the 38th International Conference on Software Engineering_. 1157–1168.
  * Svajlenko and Roy (2015) Jeffrey Svajlenko and Chanchal K Roy. 2015.  Evaluating clone detection tools with bigclonebench. In _2015 IEEE international conference on software maintenance and evolution (ICSME)_. IEEE, 131–140.
  * Wang et al. (2018) Pengcheng Wang, Jeffrey Svajlenko, Yanzhao Wu, Yun Xu, and Chanchal K Roy. 2018.  CCAligner: a token based large-gap clone detector. In _Proceedings of the 40th International Conference on Software Engineering_. 1066–1077.
  * Wang et al. (2020) Wenhan Wang, Ge Li, Bo Ma, Xin Xia, and Zhi Jin. 2020.  Detecting code clones with graph neural network and flow-augmented abstract syntax tree. In _2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER)_. IEEE, 261–271.
  * Wei and Li (2017) Huihui Wei and Ming Li. 2017.  Supervised Deep Features for Software Functional Clone Detection by Exploiting Lexical and Syntactical Information in Source Code.. In _IJCAI_. 3034–3040.
  * White et al. (2016) Martin White, Michele Tufano, Christopher Vendome, and Denys Poshyvanyk. 2016.  Deep learning code fragments for code clone detection. In _2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)_. IEEE, 87–98.
  * Yu et al. (2019) Hao Yu, Wing Lam, Long Chen, Ge Li, Tao Xie, and Qianxiang Wang. 2019\.  Neural detection of semantic code clones via tree-based convolution. In _2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)_. IEEE, 70–80.
  * Zhang et al. (2019) Jian Zhang, Xu Wang, Hongyu Zhang, Hailong Sun, Kaixuan Wang, and Xudong Liu. 2019\.  A novel neural source code representation based on abstract syntax tree. In _2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)_. IEEE, 783–794.
  * Zhang et al. (2021) Weiwei Zhang, Shengjian Guo, Hongyu Zhang, Yulei Sui, Yinxing Xue, and Yun Xu. 2021\.  Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations.  _arXiv preprint arXiv:2111.10793_ (2021).
  * Zou et al. (2020) Yue Zou, Bihuan Ban, Yinxing Xue, and Yun Xu. 2020\.  CCGraph: a PDG-based code clone detector with approximate graph matching. In _2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE)_. IEEE, 931–942.


# A Bigger Cloud 9? New HI Observations of the RELHIC Candidate M94-Cloud 9

Ananthan Karunakaran Department of Astronomy & Astrophysics, University of
Toronto, Toronto, ON M5S 3H4, Canada  Dunlap Institute for Astronomy and
Astrophysics, University of Toronto, Toronto ON, M5S 3H4, Canada  Kristine
Spekkens Department of Physics and Space Science, Royal Military College of
Canada P.O. Box 17000, Station Forces Kingston, ON K7K 7B4, Canada  Department
of Physics, Engineering Physics and Astronomy, Queen’s University, Kingston,
ON K7L 3N6, Canada

###### Abstract

We present new H I observations of the REionization-Limited H I Cloud (RELHIC)
candidate, M94-CL9, detected around M94 by Zhou et al. using the Five-hundred-
meter Aperture Spherical Telescope (FAST). M94-CL9’s H I properties as
detected by FAST are consistent with a RELHIC as noted by Benitez-Llambay &
Navarro. Our observations with the Robert C. Byrd Green Bank Telescope (GBT)
detect greater H I emission in M94-CL9 and result in H I properties that are
larger (corrected velocity width, $W_{50,c,t}=35.7\pm
0.6\,{\mathrm{km\,s^{-1}}}$; and integrated flux, $\int\mathrm{Sdv}=0.28\pm
0.04\,\mathrm{Jy}\cdot{\mathrm{km\,s^{-1}}}$) than those found by Zhou et al.
but that match those from the FAST All-Sky H I (FASHI) survey. These larger
properties do not preclude M94-CL9 from being a RELHIC, but the wider spectral
extent and spectral asymmetry reported here may be in tension with predictions
of RELHIC properties.

Dwarf galaxies (416), Neutral hydrogen clouds (1099)

††facilities: GBT††software: GBTIDL (Marganian et al., 2006), Numpy (Harris et
al., 2020), Matplotlib (Hunter, 2007)

##  1 Introduction

Within the $\Lambda$CDM paradigm, low-mass dark matter haloes and the galaxies
that are embedded within them are the building blocks of large-scale
structures in the Universe. Some low-mass haloes ($M_{200}\sim
10^{9}M_{\odot}$), despite having significant H I reservoirs, may have had
their potential for significant star formation halted by reionization. Using
the APOSTLE cosmological simulations, Benítez-Llambay et al. (2017) postulate
that there exist large quantities of low-halo mass ($3\times
10^{8}<M_{200}/M_{\odot}<5\times 10^{9}$) systems with significant H I
reservoirs that correlate well with their halo masses yet have minimal stellar
components. In simulations, the H I components of these REionization-Limited H
I Clouds (RELHICs) are generally circular and symmetric in their
distributions, have low velocity widths ($W_{50}\sim
20\,{\mathrm{km\,s^{-1}}}$), are well-separated from other massive haloes, and
are in approximate hydrostatic and thermal equilibrium with their dark matter
halo and the ionizing UV background, respectively. The implications of
discovering and characterizing such objects are significant for $\Lambda$CDM.

Using the Five-hundred-meter Aperture Spherical radio Telescope (FAST), Zhou
et al. (2023, hereafter Z23) observed the environment of the Messier 94 (M94)
galaxy, revealing extended H I emission, including three new H I clouds. One
of these clouds, M94-Cloud 9 (M94-CL9), has no obvious optical counterpart and
its H I properties from FAST are comparable to those of RELHICs in the APOSTLE
simulations: it has a narrow velocity width ($W_{50}\sim
20{\mathrm{km\,s^{-1}}}$), it is relatively isolated from M94 ($D_{proj}\sim
109\,\mathrm{kpc}\sim 1.3$ degrees), and its H I distribution is fairly
circular ($\theta_{CL9}\sim 6^{\prime}$, axial ratio $\mathrm{b/a}>0.8$ at a
column density of $1\times 10^{18}\mathrm{cm^{-2}}$; $\theta_{FAST}\sim
2.9^{\prime}$). More recently, Benítez-Llambay & Navarro (2023) presented a
comparison of the FAST H I distribution of M94-CL9 measured by Z23 with RELHIC
models from Benítez-Llambay et al. (2017) and find consistency between
observations and simulations.

Here, we present a new H I profile of M94-CL9 using the Robert C. Byrd Green
Bank Telescope (GBT). The larger, 9’ L-band beam of the GBT relative to the
2.9’ one for FAST should encompass the entire FAST detection presented by Z23
and provides a useful comparison spectrum.

##  2 Observations and Results

We conducted 42 minutes of position-switched observations of M94-CL9 with the
GBT (program AGBT23B-099). Along with the L-band receiver, we used a narrow
bandpass configuration (bandpass width $\sim$ 11.72
MHz/2500$\,{\mathrm{km\,s^{-1}}}$, spectral resolution $\sim$ 0.36
kHz/0.7$\,{\mathrm{km\,s^{-1}}}$) of the Versatile GBT Astronomical
Spectrometer (VEGAS). We reduced the data using the standard getps procedure
within GBTIDL111http://gbtidl.nrao.edu/.

The calibrated GBT spectrum222We note that we have scaled the flux output by
getps by a factor of 1.2 following the results of Goddy et al. (2020) to
account for offsets in noise diode calibration values. is shown in Figure 1 in
blue, and has been smoothed to a spectral resolution of
$4.88\,{\mathrm{km\,s^{-1}}}$ to match that of the FAST observations from Z23
in gray. The representative noise of the GBT spectrum at this resolution is
$\sim 1.2$ mJy. We show the velocity range over which Z23 consider the H I
emission of M94-CL9 in the FAST data (see their table 1) with the vertical
gray lines.

We derive the distance-independent properties (i.e. systemic velocity,
$V_{sys}$; uncorrected velocity width, $W_{50}$; integrated flux,
$\int\mathrm{Sdv}$) of the H I emission following Karunakaran et al. (2020).
We find $V_{sys}=293.8\pm 0.4\,{\mathrm{km\,s^{-1}}}$,
$W_{50}=37.6\,{\mathrm{km\,s^{-1}}}$, and $\int\mathrm{Sdv}=0.28\pm
0.04\,\mathrm{Jy}\cdot{\mathrm{km\,s^{-1}}}$. The velocity width after
applying corrections for instrumental broadening, redshift broadening, and
turbulence is $W_{50,c,t}=35.7\pm 0.6\,{\mathrm{km\,s^{-1}}}$. The resulting H
I mass, $M_{HI}$, derived using the standard equation (e.g. Haynes &
Giovanelli, 1984) and assuming the M94 distance of 4.66 Mpc333We believe they
have assumed a distance uncertainty of 0.5 Mpc and for consistency adopt this
as well. as in Z23, is $M_{HI}=(1.4\pm 0.4)\times 10^{6}M_{\odot}$.

Figure 1: GBT H I spectrum of M94-CL9 shown in blue and the FAST spectrum from
Z23 shown in gray. The velocity range of the H I emission noted by Z23 is
shown with the vertical dashed lines. We highlight the difference in H I
emission as the hatched and shaded regions.

##  3 Discussion

From Figure 1 we can see that the H I profiles obtained with FAST444We note
that the spectra shown in figure 4 of Z23 should have units of flux density
(as plotted here) instead of antenna temperature since the peak fluxes not
only match here but also match the listed peak flux value
($9.03\,\mathrm{mJy}\cdot{\mathrm{km\,s^{-1}}}$) in the FASHI survey catalog.
(gray) and the GBT (blue) are consistent with one another. Our integrated flux
($0.28\pm 0.04\,\mathrm{Jy}\cdot{\mathrm{km\,s^{-1}}}$) is comparable to that
listed in table 1 of Z23, $0.239\pm
0.01\,\mathrm{Jy}\cdot{\mathrm{km\,s^{-1}}}$, but a factor of two larger than
the value listed in the text of their section $4.2$, $0.14\pm
0.02\,\mathrm{Jy}\cdot{\mathrm{km\,s^{-1}}}$, the latter of which is used in
Benítez-Llambay & Navarro (2023). The former FAST value is listed in the FASHI
(FAST All-Sky H I Zhang et al., 2024) survey catalog (ID_FASHI = 20230020346,
Name = J125153.61+401759.1) along with a velocity width
($W_{50,FASHI}=32.54\pm 1.92{\mathrm{km\,s^{-1}}}$) that is commensurate with
ours. While the GBT spectrum appears to have an excess of flux at lower
heliocentric velocities (shaded and hatched region between
$\mathrm{265\lesssim V_{helio}\lesssim 290\,{\mathrm{km\,s^{-1}}}}$) relative
to the FAST spectrum from Z23, the spectrum of CL9 from FASHI (J. Wang, priv.
comm.) is similar in shape. Both the GBT and FASHI spectra of CL9 show signs
of an asymmetric H I profile with a steeper receding edge and a shallower
approaching edge. The asymmetry in both the GBT and FASHI spectra suggest that
it is real and not just a result of instrumental differences. While these new
results do not preclude M94-CL9 from being classified as a RELHIC, it does
bring it in some tension with the predicted properties of RELHICs which
typically expect a narrow ($W_{50}\sim 20{\mathrm{km\,s^{-1}}}$), Gaussian-
like H I distribution. We remind the reader that our GBT detection is
spatially unresolved, and does not constrain the morphology of the H I within
the beam. Higher spatial resolution H I data of M94-CL9 will be informative in
this regard.

We thank Ming Zhu, Ruilei Zhou, Alejandro Benítez-Llambay, and Jing Wang for
useful discussions. AK acknowledges support from the Natural Sciences and
Engineering Research Council of Canada (NSERC), the University of Toronto Arts
& Science Postdoctoral Fellowship program, and the Dunlap Institute. KS
acknowledges support from NSERC. The Green Bank Observatory is a facility of
the National Science Foundation operated under cooperative agreement by
Associated Universities, Inc.

## References

  * Benítez-Llambay & Navarro (2023) Benítez-Llambay, A., & Navarro, J. F. 2023, ApJ, 956, 1, doi: 10.3847/1538-4357/acf767
  * Benítez-Llambay et al. (2017) Benítez-Llambay, A., Navarro, J. F., Frenk, C. S., et al. 2017, MNRAS, 465, 3913, doi: 10.1093/mnras/stw2982
  * Goddy et al. (2020) Goddy, J., Stark, D. V., & Masters, K. L. 2020, Research Notes of the American Astronomical Society, 4, 3, doi: 10.3847/2515-5172/ab66bd
  * Harris et al. (2020) Harris, C. R., Millman, K. J., van der Walt, S. J., et al. 2020, Nature, 585, 357, doi: 10.1038/s41586-020-2649-2
  * Haynes & Giovanelli (1984) Haynes, M. P., & Giovanelli, R. 1984, AJ, 89, 758, doi: 10.1086/113573
  * Hunter (2007) Hunter, J. D. 2007, Computing in Science and Engineering, 9, 90, doi: 10.1109/MCSE.2007.55
  * Karunakaran et al. (2020) Karunakaran, A., Spekkens, K., Zaritsky, D., et al. 2020, ApJ, 902, 39, doi: 10.3847/1538-4357/abb464
  * Marganian et al. (2006) Marganian, P., Garwood, R. W., Braatz, J. A., Radziwill, N. M., & Maddalena, R. J. 2006, in Astronomical Society of the Pacific Conference Series, Vol. 351, Astronomical Data Analysis Software and Systems XV, ed. C. Gabriel, C. Arviset, D. Ponz, & S. Enrique, 512
  * Zhang et al. (2024) Zhang, C.-P., Zhu, M., Jiang, P., et al. 2024, Science China Physics, Mechanics, and Astronomy, 67, 219511, doi: 10.1007/s11433-023-2219-7
  * Zhou et al. (2023) Zhou, R., Zhu, M., Yang, Y., et al. 2023, ApJ, 952, 130, doi: 10.3847/1538-4357/acdcf5


LLVMs4Protest: Harnessing the Power of Large Language and Vision Models for
Deciphering Protests in the News

Yongjun Zhang1,2

1 Department of Sociology and Institute for Advanced Computational Science,
Stony Brook University, Stony Brook, New York, United States

2 Version 1, Updated on Nov 29, 2023

*<EMAIL_ADDRESS>
## Abstract

Large language and vision models have transformed how social movements
scholars identify protest and extract key protest attributes from multi-modal
data such as texts, images, and videos. This article documents how we fine-
tuned two large pretrained transformer models, including longformer and swin-
transformer v2, to infer potential protests in news articles using textual and
imagery data. First, the longformer model was fine-tuned using the Dynamic of
Collective Action (DoCA) Corpus. We matched the New York Times articles with
the DoCA database to obtain a training dataset for downstream tasks. Second,
the swin-transformer v2 models was trained on UCLA-protest imagery data. UCLA-
protest project contains labeled imagery data with information such as
protest, violence, and sign. Both fine-tuned models will be available via
https://github.com/Joshzyj/llvms4protest. We release this short technical
report for social movement scholars who are interested in using LLVMs to infer
protests in textual and imagery data.

## Introduction

The rapid advancement of large language and vision models has revolutionized
the way social movement scholars analyze protest events using multimodal data
such as text, images, and videos. Particularly, OpenAI’s GPT-4 has empowered
scholars with limited computational skills to intuitively examine textual and
visual data. However, there are obstacles in adopting these AI tools. One is
the cost of accessing advanced data analytic models from OpenAI, which can be
prohibitive for analyzing vast quantities of data. An alternative is to use
open-source LLMs like Meta’s LLaMa 2, but this requires substantial machine
learning knowledge and hardware resources. To address these challenges, we
fine-tuned large-scale language and vision models for specific tasks,
including protest event identification and key attribute extraction. These
models are easily operable on standard computing devices.

Next, we will briefly summarize the major development in social movement
literature that uses machine learning techniques to automatically identify
protests. Then, we will delve into some technical details on training data and
transfer learning to build our models. Finally, we will discuss some potential
issues and limitations.

## The Machine Learning Approach to Studying Social Movements

Identifying protest events is one of the primary tasks in the field of social
movement studies Earl . (2004). One of the early endeavours made by social
movement scholars was to systematically, manually extract protest information
from newspaper articles. For instance, the Dynamics of Collective Action
project identified protest events in 1955-1995 from the New York Times Soule
Earl (2005). This human labeling strategy was both time consuming and
expensive. Thus, many scholars attempted to automate the whole process using
machine learning techniques. For instance, Alex Hanna developed the Machine-
learning Protest Event Data System (MPEDS) to identify protest events and
extract protest attributes from news articles Hanna (2017). Other scholars
such as Han Zhang and Jennifer Pan adopted Convolutional Neural Networks to
classify multimodal social media data including texts and images to build
Chinese protest database H. Zhang Pan (2019). In recent years, several groups
in computer science have also devoted efforts to detect sociopolitical events
on social media data. For instance, the shared tasks for Socio-political and
Crisis Events Detection have attracted numerous social and computer scientists
to develop deep learning algorithms to identify multilingual protests, fine-
grained socio-political events, and Black Lives Matter protests in news
Hürriyetoğlu . (2019). Still, limited scholars have fine-tuned and shared
large-scale language and vision models to identify protest events in news and
social media data. One exception is that davidson2023start and
zhang2023generative propose to use generative AI tools, specifically GPT-4V to
extract protest information from images and texts.

Instead of relying on the recent advancement of GPT-4 models, in this article,
we adopt the transfer learning approach to fine-tune existing pretrained large
language and vision models for our downstream tasks in the field of social
movement studies, specifically protest identification and attribute
extraction. One of the underlying logic is that we attempt to take advantage
of existing labeled training datasets both in textual and imagery format,
which allows us to achieve the state-of-the-art performance.

## Training Data Preparation

### Dynamics of Collective Action

The widely used training dataset to classify protest events in newspaper
articles is Dynamics of Collective Action Earl . (2004); Hanna (2017). The
DoCA project annotated news articles in the New York Times from 1960 to 1995.
It is an event database, consisting of detailed article- and event-level
information, such as reporting year, article title, protest size, claims,
social movement organizations, locations, timing, etc (see
https://web.stanford.edu/group/collectiveaction/). Unfortunately, DoCA does
not contain the original articles for these events, thus we collected these
textual data from ProQuest Database by searching article titles. We managed to
obtain 16,700 articles that match the DoCA records. We further excluded
articles containing multiple protest events in DoCA, which yielded 11,902
positive news articles in our training dataset.

Given that one of the primary tasks to identify whether a news article is
protest related, we further retrieved negative articles from the New York
Times Annotated Corpus via the Linguistic Data Consortium. We added 27,000
news articles in several categories that are not likely to contain protest
related articles such as entertainment, book reviews, business, classified,
finance, sports, real estate, leisure, and obituary.

### UCLA-Protest Image Dataset

We rely on the UCLA-Protest Image dataset to fine-tune our large vision model.
UCLA protest database contains 40,764 images, including 11,659 protest images,
with various annotations of visual attributes such as violence and police Won
. (2017). We are also working on our own protest image dataset retrieving from
Global Database of Events, Languages, and Tones (GDELT).

## Fine-tune Large Language and Vision Models

### The Longformer Model

Instead of utilizing the pretrained BERT model, we fine-tune the longformer
model, which is specifically used for the long documents Beltagy . (2020). We
fine-tune the longformer-base-4096 model with our training dataset, since it
supports sequences of length up to 4096. Note that longformer-base-4096 is a
BERT-like model started from the RoBERTa checkpoint and pretrained for marked
language modeling on long documents. For more technical details, please check
beltagy2020longformer’s paper.

### The Swin-Transformer V2 Model

Instead of training CNN models like VGG or ResNet, we fine-tune the Swin-
transformer v2 model Liu . (2021, 2022). Swin Transformer V2 has been shown
the state-of-art performance in computer vision tasks. For more technical
details, please check liu2021swin,liu2022swin’s articles. We fine-tuned the
swinv2-base-patch4-window8-256 model pretrained by the Microsoft research
group.

Note that the accuracy rate for both models is over 94%. We will release more
details on evaluation metrics when we finalize our models for scholarly
community. The baseline models are available to our scholarly community.

## Discussion

### AI for Social Movement Studies

In this short essay, we attempt to document some of our ongoing work on fine-
tuning large language and vision models to classify protest events and extract
protest attributes from news articles and images. In recent years, social
activists, movement organizations, and bystanders are increasingly sharing
protest activities via social media platforms with multimodal data and more
visual information is included in news reports. Because of the recent
advancement in generative AIs and deep learning techniques, scholars can take
advantage of these large language and vision tools to analyze multimodal data
to extract event information to address social science questions. Social
movement scholars can benefit from adopting these tools to advance movement
data collection and theory testing as well. Our work is one of these attempts
by fine-tuning the state-of-the-art Longformer and Swin Transformer models to
identify protests.

### Limitations and Future Directions

There are several limitations that readers should be aware of. First, fine-
tuning transformer models requires scholars to have some basic knowledge of
deep learning and programming skills. Fortunately, the swift advancement of
generative AIs have lowered the barriers to computational social science Y.
Zhang (2023). Scholars can use ChatGPT to generate, annotate, and debug codes.
Second, the training data used to fine-tune longformer model has time
constraints. As most of these positive news articles are from 1960-1995 and it
might limit the potential of inferring protests in recent years. This warrants
further studies. Third, our work still needs more cross-validation. For
instance, a comparison between these fine-tuned models and GPT-4 annotation
can offer social movement scholars more nuances in terms of choosing
generative AIs or transfer learning with pretrained models. Lastly, our
classifiers focus solely on protest identification and some limited number of
protest attributes. We are still working on expanding protest attributes.
These new features will be released soon once we finalize our fine-tuning
processes.

## Acknowledgments

I acknowledge the support from the Institute for Advanced Computational
Science for access to high performance computing systems and OpenAI APIs and
ChatGPT.

## References

  * Beltagy . (2020) beltagy2020longformerBeltagy, I., Peters, ME. Cohan, A.  2020\.  Longformer: The long-document transformer Longformer: The long-document transformer.  arXiv preprint arXiv:2004.05150.
  * Davidson (2023) davidson2023startDavidson, T.  2023\.  Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research Start generating: Harnessing generative artificial intelligence for sociological research.
  * Earl . (2004) earl2004useEarl, J., Martin, A., McCarthy, JD. Soule, SA.  2004\.  The use of newspaper data in the study of collective action The use of newspaper data in the study of collective action.  Annu. Rev. Sociol.3065–80.
  * Hanna (2017) hanna2017mpedsHanna, A.  2017\.  Mpeds: Automating the generation of protest event data Mpeds: Automating the generation of protest event data.
  * Hürriyetoğlu . (2019) hurriyetouglu2019overviewHürriyetoğlu, A., Yörük, E., Yüret, D., Yoltar, Ç., Gürel, B., Duruşan, F.Akdemir, A.  2019\.  Overview of clef 2019 lab protestnews: Extracting protests from news in a cross-context setting Overview of clef 2019 lab protestnews: Extracting protests from news in a cross-context setting.  Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10 Experimental ir meets multilinguality, multimodality, and interaction: 10th international conference of the clef association, clef 2019, lugano, switzerland, september 9–12, 2019, proceedings 10 ( 425–432).
  * Liu . (2022) liu2022swinLiu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y.others  2022\.  Swin transformer v2: Scaling up capacity and resolution Swin transformer v2: Scaling up capacity and resolution.  Proceedings of the IEEE/CVF conference on computer vision and pattern recognition Proceedings of the ieee/cvf conference on computer vision and pattern recognition ( 12009–12019).
  * Liu . (2021) liu2021swinLiu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z.Guo, B.  2021\.  Swin transformer: Hierarchical vision transformer using shifted windows Swin transformer: Hierarchical vision transformer using shifted windows.  Proceedings of the IEEE/CVF international conference on computer vision Proceedings of the ieee/cvf international conference on computer vision ( 10012–10022).
  * Soule Earl (2005) soule2005movementSoule, S. Earl, J.  2005\.  A movement society evaluated: Collective protest in the United States, 1960-1986 A movement society evaluated: Collective protest in the united states, 1960-1986.  Mobilization: An International Quarterly103345–364.
  * Won . (2017) won2017protestWon, D., Steinert-Threlkeld, ZC. Joo, J.  2017\.  Protest activity detection and perceived violence estimation from social media images Protest activity detection and perceived violence estimation from social media images.  Proceedings of the 25th ACM international conference on Multimedia Proceedings of the 25th acm international conference on multimedia ( 786–794).
  * H. Zhang Pan (2019) zhang2019casmZhang, H. Pan, J.  2019\.  Casm: A deep-learning approach for identifying collective action events with text and image data from social media Casm: A deep-learning approach for identifying collective action events with text and image data from social media.  Sociological Methodology4911–57.
  * Y. Zhang (2023) zhang2023generativeZhang, Y.  2023\.  Generative AI has lowered the barriers to computational social sciences Generative ai has lowered the barriers to computational social sciences.  arXiv preprint arXiv:2311.10833.


# Modeling of Chromospheric Features and Dynamics in Solar Plage

Sanja Danilovic<EMAIL_ADDRESS>Institute for Solar Physics, Dept. of
Astronomy, Stockholm University, Albanova University Center, 10691 Stockholm,
Sweden

###### Abstract

The chromosphere is a dynamic and complex layer where all the relevant
physical processes happen on very small spatio-temporal scales. A few spectral
lines that can be used as chromospheric diagnostics give us convoluted
information that is hard to interpret without realistic theoretical models.
What are the key ingredients that these models need to contain? The magnetic
field has a paramount effect on chromospheric structuring. This is obvious
from the ubiquitous presence of chromospheric dynamic fibrilar structures
visible on the solar disk and at the limb. The numerical experiments presented
in this manuscript illustrate the present state of modeling. They showcase to
what extent our models reproduce various chromospheric features and their
dynamics. The publication describes the effect different ingredients have on
chromospheric models and provides a recipe for building one-to-one models.
Combining these models with observations will provide insight into the
physical processes that take place in the solar atmosphere.

###### keywords:

solar chromosphere , MHD simulations

††journal: Advances in Space Research

##  1 Introduction

The solar atmosphere is complex, non-homogeneous and highly dynamic. It
encompasses plasma widely ranging in density, temperature, ionization and
magnetodynamic properties which complicates the modeling. The density drops
exponentially with height by 10 magnitudes from near-surface convection layers
to the corona. This affects the state of the matter which we aim to describe
as realistically as possible in our numerical models (Leenaarts, 2020).

A prime agent causing non-uniformity is magnetism. Looking at the
chromosphere, e.g. in H$\alpha$ line core, the solar surface is covered with
opaque long fibrils (Rutten, 2007) that emanate from large magnetic features
organized at mesogranular to supergranular scales (Stein & Nordlund, 2012).
Their appearance varies depending on which chromospheric line is used for
observations and what band-pass range is sampled (Cauzzi et al., 2008; Kianfar
et al., 2020). Although the long fibrils tend to outline the direction of the
magnetic field in the chromosphere (Vissers et al., 2022), their appearance
largely depends on the unsigned flux density of the region. So, on the one
hand, in active regions and around strong network structures, they are solid,
opaque and show large contrast. In quiet Sun regions, on the other hand, they
appear scarcer and extend less far from the network where they originate.

Fig. 1: Vertically emergent H$\alpha$ intensity at blue wing corresponding to
v${}_{\rm dopp}=-36$ km s-1 (top) and the line core (middle) and formation
height of H$\alpha$ line core (bottom). A movie of the H$\alpha$ line-core
emergent intensity is available as supplementary material.[Movie] Fig. 2: PDF
of the column mass at the H$\alpha$ line-core $\tau=1$ height as a function of
the $\tau=1$ height.

Apart from long fibrils, the chromosphere shows dynamic structures originating
at fibril footpoints in the magnetic network (Tsiropoula et al., 2012). The
state-of-the-art observations show that we can find flows, waves, and shocks
in the chromosphere (Jess et al., 2015; Molnar et al., 2021; Morosin et al.,
2022). These phenomena contribute to the heating of the upper layers.
Furthermore, the magnetic field and its interaction with convective flows
determine how much energy is available and how it is transported further up.
As a consequence, we measure different responses of different layers in the
solar atmosphere depending on the observed region. This is mirrored in a
varying relationship between the magnetic field and emission in diagnostics at
different spectral ranges, from EUV to millimeter range (Loukitcheva et al.,
2009; Barczynski et al., 2018). Particularly, the chromospheric temperature
seems to correlate the most with the horizontal component of the chromospheric
magnetic field (Leenaarts et al., 2018).

This publication does not attempt to provide a review but to reflect the
author’s opinion on the modeling of the solar chromosphere and serves as a
follow-up to a talk given at the 16th European Solar Physics Meeting. The main
focus is on data-inspired models that self-consistently consider the near-
surface layers of the convection zone where the actual driving of most
chromospheric phenomena occurs. These models aim to reproduce the observed
evolution of the magnetic field by replicating the overall magnetic field
topology and flux emergence. As such, they allow studying the energy transport
between subsurface layers, photosphere, chromosphere, and corona.

Fig. 3: Vertical magnetic field at temperature minumum (top) and the field
inclination at z$=2.7$ Mm (bottom).  Fig. 4: Volume rendering of the Alfvén
wave proxi $f_{\rm alf}$ (in red and blue). The horizontal plane shows the
vertical component of the magnetic field in the photosphere. The green lines
outline the direction of the magnetic field lines. The figure is produced with
VAPOR (Li et al., 2019).

##  2 Magnetic field configuration

Features visible at the photospheric layers, at any moment in time, from small
magnetic concentrations to sunspots, show different amounts of magnetic flux
per unit area. Their small-scale appearance and short-temporal dynamics are a
result of different regimes of so-called magneto-convection (e.g. Schüssler,
2013). Consequently, the energy input to the higher layers differs from one
region to the next.

Figure 1 shows a snapshot from a numerical experiment that aims to reproduce
the magnetic properties of the observed plage quantitatively. The model is
made by using the MURaM code (Vögler et al., 2005; Rempel, 2017). It contains
only the minimum physics required to be called ‘comprehensive’. That is:
single-fluid MHD, 3D grey LTE radiative transfer, a tabulated LTE equation of
state, Spitzer heat conduction, and optically thin radiative losses in the
corona based on CHIANTI (Landi et al., 2012). The model is built in phases
starting from the hydrodynamic convection simulation that includes photosphere
and near surface convection zone down to $-8$ Mm below the average $\tau_{\rm
500}=1$ height. This number is a compromise. On the one hand, the domain
should be deep enough to allow magnetic flux to converge toward the downflow
lanes of large-scale convective cells (Nordlund et al., 2009; Stein &
Nordlund, 2012). On the other hand, the horizontal extension of the
computational box should be $4$ times its depth to provide enough volume for
the development of a large-scale flow pattern (Schüssler, 2013), but also to
allow a small enough grid spacing so that small scales are reasonably
resolved. In the second phase, a bipolar uniform vertical field of $200$ G is
added in two halves to the fully-developed nonmagnetic convection simulation.
The initial mean unsigned flux density of $200$ G was enough to eventually
form pore aggregation and magnetic structures that extended $>5$ Mm in radius.
In the third phase, the potential field extrapolation is used to extend the
computational domain to $14$ Mm above the average $\tau_{\rm 500}=1$ height
and include the upper solar atmosphere. At the bottom boundary, a horizontal
field at roughly equipartition field strengths is allowed to emerge into the
domain (Rempel, 2014). At the upper boundary, the field is set to be vertical.
Also, the upper boundary is open to outflows but closed to inflows. The
horizontal boundaries are periodic. In the last phases, the model is first run
at a lower resolution and then again at the double resolution until a relaxed
state is achieved. The final grid spacing is $39$ and $21$ km in the
horizontal and vertical directions, respectively.

The middle panel in Fig. 1 shows emergent intensity in the H$\alpha$ line
core. The synthetic images are generated with the 3D radiative transfer code
Multi3D (Leenaarts & Carlsson, 2009). The same H I model atom as in Bjørgen et
al. (2019) is used with the LTE electron densities. The H$\alpha$ synthetic
image in Fig. 1 shows long fibrils that spread everywhere, connecting the
large features of opposite polarity outlined in the top panel of Fig. 3. The
middle and bottom panels in Fig. 1 can be compared with Fig.7 in Leenaarts et
al. (2012). The difference between the model used here and the model used in
Leenaarts et al. (2012) is manifold. First, the domain of the simulations
presented in this article is almost twice the size of the model used by
Leenaarts et al. (2012). This allows fibrils of longer horizontal extension to
be formed. The mean unsigned field here is $180$ G versus $30$ G in the other
study. The resulting appearance of fibrils agrees with observations that show
dependence on unsigned magnetic flux density. Finally, Leenaarts et al. (2012)
used a model with a simple bipolar configuration, while the present model has
a more complex field allowing the opposite polarities to be connected in
multiple directions and fibrils to cover almost the whole horizontal extent of
the computational domain. The comparison of the two figures reveals that the
H$\alpha$ spectral line forms in the same way in both models. The line-core
image shows that the intensity is anti-correlated to the average formation
height. This means that the darker fibrils are, the higher they are formed.

Figure 2 shows that the $\tau=1$ height has a constant column mass of roughly
$3\times 10^{-5}$. This trend is the same as reported in Leenaarts et al.
(2012). Compared to Fig. 12 in the same article, the plot also shows branches
toward the higher column masses and the higher heights. The number of
occurrences in these branches is very low compared to the main cloud that is
confined between $1-5\times 10^{-5}$ g cm-2. The main cloud is shifted toward
higher heights and extended to nearly $5$ Mm. This clearly shows that the same
column masses are reached at significantly higher heights, meaning that much
more material is lifted higher than in the QS case presented in Leenaarts et
al. (2012).

The synthetic H$\alpha$ image in Fig. 1 shows more than just long fibrils.
This particular snapshot shows two surges at $[12,25]$ Mm and $[30,33]$ Mm and
a jet at $[11,1]$ Mm. These features are visible as protrusions in the bottom
panel of Fig. 1 where the formation height reaches $7$ Mm or so. The
preliminary analysis suggests that surges appear during magnetic field
reconfiguration without a bulk flux emergence or through a mechanism similar
to the one described in Iijima & Yokoyama (2017). The model also shows
features that resemble short and long dynamic fibrils. Short dynamic fibrils
can be visible in the region with less flux, in the lower-left quadrant of the
domain where the field is mainly vertical, as the bottom panel of Fig. 3
shows. Long dynamic fibrils are visible around $[7,25]$ Mm, where the field is
inclined and forms a slope. This agrees well with observations of the dynamic
fibrils (De Pontieu et al., 2007). Finally, the same model shows the
ubiquitous presence of so-called rapid blue-shifted excursions (RBE, Rouppe
van der Voort et al., 2009) visible in the top panel of Fig. 1.

##  3 Small-scales dynamics

The second important ingredient in chromospheric modeling is the small-scale
dynamics. On the smallest scales, the observations show signatures of
turbulent dynamo action (Danilovic et al., 2010) that results in mixed-
polarities. Such structures can undergo a magnetic reconnection that can
trigger mass loading into the chromospheric fibrils. Furthermore, the
turbulent motion of the plasma sets the properties of the photospheric driver
that excites magnetic footpoints. This generates different types of waves that
can transport the energy to the chromosphere and higher. For example, vortex
flows can serve as drivers of torsional Alfvén waves (Shelyag et al., 2013;
Battaglia et al., 2021). Depending on the amount of magnetic flux, the
observations show horizontal (Steiner et al., 2010) and vertical kind (e.g.
Bonet et al., 2010) of vortex flows present everywhere in the photosphere and
chromosphere (Wedemeyer-Böhm et al., 2012; Dakanalis et al., 2022). Their
presence can be recognized in specific heating signatures generated in the
upper photosphere (Moll et al., 2012). Their abundance indicates that their
contribution to coronal heating might be significant.

Fig. 5: Proxies for Alfvén $f_{\rm alf}$ (left), fast $f_{\rm fast}$ (middle)
and slow magneto-acoustic $f_{\rm long}$ (right) waves averaged over the the
formation height of the H$\alpha$ line core. [Movie] The red rectangles mark
the regions that are used for power spectra averaging shown in Fig. 6.

The question remains to what extent the present simulation can resolve and
reproduce plasma dynamics on the smallest scales. When it comes to waves, in
the complex simulations like the ones presented in the previous section,
identifying and following different wave modes is difficult. To separate them
and analyze their spatial and temporal characteristics, the following
quantities are calculated after Khomenko et al. (2018):

$f_{\rm alf}=\hat{e_{\parallel}}\cdot\bigtriangledown\times\mathbf{v}$

$f_{\rm
fast}=\bigtriangledown\cdot\left(\mathbf{v}-\hat{e_{\parallel}}v_{\parallel}\right)$

$f_{\rm
long}=\hat{e_{\parallel}}\cdot\bigtriangledown\left(\mathbf{v}\cdot\hat{e_{\parallel}}\right)$

where $\mathbf{v}$ is the velocity vector and $\hat{e_{\parallel}}$ the field-
aligned unit vector. The quantities represent proxies for three wave modes
slow $f_{\rm long}$ and fast $f_{\rm fast}$ magneto-acoustic and Alfvén
$f_{\rm alf}$. Figure 4 shows distribution of the quantity $f_{\rm alf}$ for
the snapshot shown in Fig. 1. Logically, the quantity gets larger with height
and follows the magnetic field orientation. To visualize where the wave power
for each mode is located, each quantity is averaged within the layer where
column mass ranges $-5<\log_{10}(m_{\rm c})<-4.5$. As Fig. 1 shows, this is
the chromosphere as we see it in the H$\alpha$ line core. The resulting maps
are shown in Fig. 5. The quantity $f_{\rm alf}$ is predominant in regions with
the horizontal field. High $f_{\rm alf}$ quantities can also be found in
surges and the strong magnetic feature at $[32,13]$ Mm. The other two
quantities are not as strong. They have a morphology similar to $f_{\rm alf}$
along the closed loops. In regions with open or inclined field, the
distribution of $f_{\rm alf}$ and $f_{\rm fast}$ is similar. The distribution
of $f_{\rm long}$, unlike the other two quantities, shows increased value
along the rims between open and closed field regions, i.e. at the loop
footpoints. The supplementary movie shows the temporal evolution of these
quantities over $635$ s with the cadence of $\sim 10$ s. Rapid changes are
visible in all three quantities. In most places, the quantities show abrupt
changes in sign and magnitude. The changes seem faster in regions with the
closed field. In some places, an apparent propagation can be identified.
Proxies of all three wave modes are detectable whenever a more explosive event
happens i.e. onset of a small flare at the end of the time series at $[27,22]$
Mm and $[38,36]$ Mm.

The three outlined regions are used to produce averaged power spectra shown in
Fig. 6. The red quadrant outlines a region with open/inclined field. The two
rectangles cover high canopy and, when summed, outline region of equal size.
Averaged power spectra are multiplied with the corresponding average density
profile and plotted as a function of height and frequency.

Fig. 6: Power spectra of $f_{\rm long}$ (left), $f_{\rm fast}$ (middle) and
$f_{\rm alf}$ (right) as a function of height and frequency in regions with
horizontal (top) and vertical (bottom) field.

The power maps for two field configurations are significantly different. The
differences are evident in all three quantities. The power over all
frequencies decreases less rapidly for horizontal field configuration (top row
of Fig. 6) and stays constant in the range of heights between $4-6$ Mm where
chromospheric canopy coincides (see bottom panel of Fig. 1). Only $f_{\rm
alf}$ shows higher power extending further up to $11$ Mm height and only at
frequencies lower than $20$ mHz. In the case of a more vertical field
configuration (bottom row of Fig. 6) the power drop off is gradual with
height. All three quantities show a higher power in the frequency range
between $2$ and $10$ mHz all the way up to the corona. The leakage towards
higher frequencies is present for both $f_{\rm long}$ and $f_{\rm alf}$, but
over different height ranges.

The power distribution shown in Fig. 6 is in agreement with several
observational studies. There are many reports on the detection of oscillations
with periods around $5$ minutes in solar corona (De Pontieu et al., 2005, and
references therein). They are mostly interpreted as upward-propagating
longitudinal magnetoacoustic waves, but there are reports that state detection
of Alfvén (Tomczyk et al., 2007) or transverse kink waves (Van Doorsselaere et
al., 2008) which should be fast magnetoacoustic waves (De Moortel &
Nakariakov, 2012). Detecting oscillations at high frequencies is, however,
harder primarily due to the limited number of photons available in the UV/EUV
part of the solar spectrum, as well as in strong chromospheric lines. There
are, however, a few studies based on the H$\alpha$ narrow-band filtergrams.
Shetye et al. (2021) analyzed the sudden appearance and disappearance of dark
features in H$\alpha$ wings. They found high-frequency oscillations with an
average frequency between $10-63$ mHz which they interpreted as fast
magnetoacoustic waves. Similarly, Okamoto & De Pontieu (2011) and more
recently Bate et al. (2022) studied transverse oscillations in spicules,
features visible off the disk. They detected high-frequency waves with periods
down to $10$ s named Alfvénic, which allows them to be both Alfvén or fast
magnetoacoustic waves in nature.

##  4 Effect of the emerging flux

The third important ingredient for reproducing chromospheric features and
dynamics is taking into account the flux emergence. Emerging flux has a
significant role in filling the Sun’s atmosphere with hot plasma. The flux
emergence happens on a wide range of spatial scales (Thornton & Parnell, 2011;
Gošić et al., 2016; Smitha et al., 2017). Although most of the flux emerging
on granular scales gets pulled back (Danilovic et al., 2010), some percentage
rise and may contribute to the heating of higher atmospheric layers (Martínez
González & Bellot Rubio, 2009). Reconnection of the newly emerging with the
pre-existing magnetic field may be an important source of (1) the heating and
filling of the coronal loops with mass (Chitta et al., 2017) (2) generation of
jets on different spatial scales (Yokoyama & Shibata, 1996) (3) generation of
spicules (Samanta et al., 2019) and so on. Localized transient heating events
are most likely always related to emerging magnetic flux which brings
substantial magnetic and kinetic energy at short time scales. It is suggested
by observations (Peter et al., 2014) and also confirmed with numerical models
(da Silva Santos et al., 2022) that transition region and coronal temperatures
can be reached at much lower heights in the atmosphere when violent events
take place. Finally, on the larger scales, observations show high correlations
of the temperature with the strength of the horizontal magnetic field in the
low chromosphere of the emerging regions (Leenaarts et al., 2018).

The following numerical experiments illustrate how the different properties of
emerging flux result in different heating rates in the chromosphere and the
formation of different chromospheric features. The two experiments are a
continuation of the run presented in section $2$. They start with the same
field configuration but differ in properties of the bipolar flux system that
is advected through the bottom boundary within an area with an elliptical
shape (Cheung et al., 2019). In the first case, flux emergence run 1 (FER1),
the emergence is limited within an ellipsoidal area with a major and minor
axis $(a,b)=(10,2)$ Mm and field strength of $5000$ G. In the second, FER2,
the emerging flux is confined within $(a,b)=(3,1)$ Mm and field strength of
$8000$ G. So FER1 has less flux advected over a larger area and FER2 has the
opposite. As a result, the emerging flux in two numerical experiments reaches
the photosphere at different times. Due to the difference in properties of the
emerging flux, the two experiments lead to different magnetic field evolution
as visible in the top row of Fig. 7 and in supplementary movies. Time $t=0$ is
arbitrarily chosen for both experiments and marks the moment when emerging
flux reaches the photosphere.

The FER1 was inspired by the observations analyzed in Leenaarts et al. (2018).
The aim was to reproduce the orientation of the emerging with respect to the
preexisting field. The major feature of this run is a peacock jet that gets
generated as the emerging flux pushes into the large negative polarity at
$[35,23]$ Mm and forms a positive intrusion. Observations that inspired the
numerical experiment also show a peacock jet whose properties are greatly
reproduced in the model. The peeling of magnetic field lines due to
reconnection with the emerging field as well as subsequent collimation of the
ejected material differs from the scenarios presented in Yokoyama & Shibata
(1996) and Moreno-Insertis & Galsgaard (2013).

The FER2 is partly analyzed in da Silva Santos et al. (2022). That analysis
focuses on the snapshot at $t=36.6$ min that produced the greatest brightness
excess in the synthetic $3$ mm emission. As that study shows, the 3 mm
emission largely outlines dissipation in current sheets that are generated in
the interaction region between emerging and pre-existing systems. Some $10$
min after the analyzed moment, there is a flare onset.

Bottom panels of Fig. 7 show the sum of the viscous and resistive heating
rates integrated over the height that corresponds to column mass in the range
$-5<\log_{10}(m_{\rm c})<-4.5$. The difference between the runs is largest
just above the emergence region $[12-27,15-30]$ Mm. In the case of FER1, the
heating patterns are mostly in form of stripes in the direction of overlying
field lines. The heating is of the order of $20$ kWm-2 in these cases. There
are cases of localized heating that exceed $80$ kWm-2 and these events
resemble the UV bursts (Peter et al., 2014). In the case of FER2, localized
heating events are larger and volume-filling, with heating rates always larger
than $100$ kWm-2. This is in line with values extracted from observations
(Díaz Baso et al., 2021). With the onset of the flare, the heating of $>100$
kWm-2 covers the whole area above the emerging flux.

Fig. 7: Snapshots from two runs that differ in emergence flux advected into
the computational domain (left- FER1 and right - FER2). Top row - vertical
component of the magnetic field. Bottom row - Sum of resistive and viscous
heating terms summed over column masses $-5<\log_{10}(m_{\rm c})<-4.5$. Movies
are available as supplementary material.[FER1 movie] [FER2 movie]

##  5 Computationally expensive physics

Great efforts are made in building models that go beyond the single fluid
description. A multi-fluid treatment that separates ion and neutral species is
necessary for some atmospheric processes (Ballester et al., 2018). This comes
from the fact that the ionization fraction increases rapidly with height which
makes the chromosphere essentially collisionally weakly coupled. The easy way
to extend the single-fluid model to incorporate the two-fluid plasma effects
is by adding the two additional terms in generalized Ohm’s law. These are
ambipolar diffusion (AD) with the Hall effect (e.g. Rempel & Przybylski,
2021). However, as shown by Nóbrega-Siverio et al. (2020), the inclusion of
these effects does not make sense without also including the non-equilibrium
ionization of hydrogen. The two must be taken into account together because
including only AD with the assumption of LTE underestimates the ionization
fraction and as a result, overestimates the effect of ambipolar diffusion.

Although the numerical experiments presented in previous sections do not
account for any computationally expensive physics, they do manage to reproduce
many chromospheric features and their dynamics to a large extent. This is in
disagreement with previous studies that state that the generation of features
observed in the chromosphere, i.e. long fibrils and spicules is not possible
without the inclusion of non-equilibrium ionization of hydrogen and ambipolar
diffusion (Martínez-Sykora et al., 2017; Rutten et al., 2019; Carlsson et al.,
2019). The models presented here show even quantitative agreement with
observed values (Withbroe & Noyes, 1977) at the spatial resolution presently
available (da Silva Santos et al., 2022). This comes as a surprise since
numerical diffusivity and viscosity terms are larger than their real physical
values. However, the numerical experiments show that the Poynting flux input
in the photosphere determines the total energy dissipation rate while
variations in numerical diffusivity and viscosity terms have only a marginal
effect (Galsgaard & Nordlund, 1996; Rempel, 2017).

Including both non-equilibrium ionization of hydrogen and the ion-neutral
interaction effects are important for several reasons. The dynamics and
temperature in the upper chromosphere and transition region will be largely
affected. The resulting temperature variation will be finer and small-scale
pockets with higher density will be formed (Nóbrega-Siverio et al., 2020;
Martínez-Sykora et al., 2020). As a result, also the appearance of
chromospheric features from long fibrils to spicules and dynamic fibrils will
be altered. For example, it is demonstrated that H$\alpha$ line core intensity
outlines density ridges and the line-core width correlates with temperature
(Cauzzi et al., 2009; Leenaarts et al., 2012). The aforementioned changes in
density and temperature distribution would lead to larger variations in
H$\alpha$ line intensity and this may result in e.g. apparently less opaque
long fibrils than shown in Fig. 1. Furthermore, the heating through ion-
neutral collisions will affect the reconnection rate and may give more
homogenous heating in the emergence regions (Leake & Arber, 2006). Finally,
the inclusion of ambipolar diffusion and the Hall effect will further help the
production of Poynting flux in the form of Alfvén waves (Khomenko et al.,
2021).

##  6 Summary

As Withbroe & Noyes (1977) write: ’The configuration of the magnetic field is
an important factor in controlling the energy losses.’. So to be able to
properly model the solar atmosphere and with it the most complex layer,
chromosphere, we have to start from there. The final recipe for building ideal
atmospheric models of solar plage starts with reproducing magnetic field
configuration. The first phase would be reproducing photospheric magnetic
features e.g. their size and the field strength. We can adjust this by
choosing correctly the strength and configuration of the initial field and
setting the lower boundary of the computational domain sufficiently deep so
that the magnetic flux can converge and create extended magnetic features as
we observe in the Sun. By choosing all these parameters right we are self-
consistently reproducing the correct regime of magneto-convection and ensuring
that the available energy input is correct. The second phase is choosing the
correct time to extend the domain to the chromosphere. The correct overall
magnetic field configuration then provides channels for energy and mass to be
transferred to upper layers the same way as it might happen on the Sun. The
final phase is introducing the emerging flux if we see its signatures in the
observations.

Once the overall magnetic field configuration matches the observations, we can
add computationally expensive physics: non-equilibrium ionization of hydrogen
and ambipolar diffusion with the Hall effect. This final step would possible
close the gap between observation and models. The only ingredient that is
missing then is the effect of non-thermal particles. Although efforts are made
to include the effect of the non-thermal particles into realistic MHD
simulations (Nordlund et al., 2018; Frogner et al., 2020; Ruan et al., 2020),
these models are still computationally expensive to be used for one-to-one
comparison with observations on the larger scales. By building one-to-one
models, we can identify the exact location where the effect of non-thermal
particles may be significant. This will be visible in the differences between
observed and synthetic observables. In this way, one-to-one models will
complement RADYN models (Carlsson & Stein, 1992) that do not take into account
the magnetic field and its forces acting on plasma.

##  7 Acknowledgments

S.D. thanks Matthias Rempel, Johan Bjørgen and Jorrit Leenaarts for sharing
their codes which allowed these results to be obtained. This project has
received funding from Swedish Research Council (2021-05613), Swedish National
Space Agency (2021-00116) and the Knut and Alice Wallenberg Foundation. This
research data leading to the results obtained has been supported by SOLARNET
project that has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement no 824135. The
calculations were performed on resources provided by the Swedish National
Infrastructure for Computing (SNIC) at the National Supercomputer Centre (NSC)
at Linköping University and the PDC Centre for High Performance Computing
(PDC-HPC) at the Royal Institute of Technology in Stockholm.

## References

  * Ballester et al. (2018) Ballester, J. L., Alexeev, I., Collados, M., et al. 2018, Space Sci. Rev., 214, 58
  * Barczynski et al. (2018) Barczynski, K., Peter, H., Chitta, L. P., et al. 2018, A&A, 619, A5
  * Bate et al. (2022) Bate, W., Jess, D. B., Nakariakov, V. M., et al. 2022, ApJ, 930, 129
  * Battaglia et al. (2021) Battaglia, A. F., Canivete Cuissa, J. R., Calvo, F., et al. 2021, A&A, 649, A121
  * Bjørgen et al. (2019) Bjørgen, J. P., Leenaarts, J., Rempel, M., et al. 2019, A&A, 631, A33
  * Bonet et al. (2010) Bonet, J. A., Márquez, I., Sánchez Almeida, J., et al. 2010, ApJ, 723, L139
  * Carlsson et al. (2019) Carlsson, M., De Pontieu, B., & Hansteen, V. H. 2019, ARA&A, 57, 189
  * Carlsson & Stein (1992) Carlsson, M. & Stein, R. F. 1992, ApJ, 397, L59
  * Cauzzi et al. (2009) Cauzzi, G., Reardon, K., Rutten, R. J., et al. 2009, A&A, 503, 577
  * Cauzzi et al. (2008) Cauzzi, G., Reardon, K. P., Uitenbroek, H., et al. 2008, A&A, 480, 515
  * Cheung et al. (2019) Cheung, M. C. M., Rempel, M., Chintzoglou, G., et al. 2019, Nature Astronomy, 3, 160
  * Chitta et al. (2017) Chitta, L. P., Peter, H., Solanki, S. K., et al. 2017, ApJS, 229, 4
  * da Silva Santos et al. (2022) da Silva Santos, J. M., Danilovic, S., Leenaarts, J., et al. 2022, A&A, 661, A59
  * Dakanalis et al. (2022) Dakanalis, I., Tsiropoula, G., Tziotziou, K., et al. 2022, A&A, 663, A94
  * Danilovic et al. (2010) Danilovic, S., Beeck, B., Pietarila, A., et al. 2010, ApJ, 723, L149
  * Danilovic et al. (2010) Danilovic, S., Schüssler, M., & Solanki, S. K. 2010, A&A, 513, A1
  * De Moortel & Nakariakov (2012) De Moortel, I. & Nakariakov, V. M. 2012, Philosophical Transactions of the Royal Society of London Series A, 370, 3193
  * De Pontieu et al. (2007) De Pontieu, B., Hansteen, V. H., Rouppe van der Voort, L., et al. 2007, ApJ, 655, 624
  * De Pontieu et al. (2005) De Pontieu, B., Erdélyi, R., & De Moortel, I. 2005, ApJ, 624, L61
  * Díaz Baso et al. (2021) Díaz Baso, C. J., de la Cruz Rodríguez, J., & Leenaarts, J. 2021, A&A, 647, A188
  * Frogner et al. (2020) Frogner, L., Gudiksen, B. V., & Bakke, H. 2020, A&A, 643, A27
  * Galsgaard & Nordlund (1996) Galsgaard, K. & Nordlund, Å. 1996, J. Geophys. Res., 101, 13445
  * Gošić et al. (2016) Gošić, M., Bellot Rubio, L. R., del Toro Iniesta, J. C., et al. 2016, ApJ, 820, 35
  * Iijima & Yokoyama (2017) Iijima, H. & Yokoyama, T. 2017, ApJ, 848, 38
  * Jess et al. (2015) Jess, D. B., Morton, R. J., Verth, G., et al. 2015, Space Sci. Rev., 190, 103
  * Khomenko et al. (2021) Khomenko, E., Collados, M., Vitas, N., et al. 2021, Philosophical Transactions of the Royal Society of London Series A, 379, 20200176
  * Khomenko et al. (2018) Khomenko, E., Vitas, N., Collados, M., et al. 2018, A&A, 618, A87
  * Kianfar et al. (2020) Kianfar, S., Leenaarts, J., Danilovic, S., et al. 2020, A&A, 637, A1
  * Landi et al. (2012) Landi, E., Del Zanna, G., Young, P. R., et al. 2012, ApJ, 744, 99
  * Leake & Arber (2006) Leake, J. E. & Arber, T. D. 2006, A&A, 450, 805
  * Leenaarts (2020) Leenaarts, J. 2020, Living Reviews in Solar Physics, 17, 3
  * Leenaarts & Carlsson (2009) Leenaarts, J. & Carlsson, M. 2009, The Second Hinode Science Meeting: Beyond Discovery-Toward Understanding, 415, 87
  * Leenaarts et al. (2012) Leenaarts, J., Carlsson, M., & Rouppe van der Voort, L. 2012, ApJ, 749, 136
  * Leenaarts et al. (2018) Leenaarts, J., de la Cruz Rodríguez, J., Danilovic, S., et al. 2018, A&A, 612, A28
  * Li et al. (2019) Li, S., Jaroszynski, S., Pearse, S., et al. 2019, Atmosphere, 10, 488
  * Loukitcheva et al. (2009) Loukitcheva, M., Solanki, S. K., & White, S. M. 2009, A&A, 497, 273
  * Martínez González & Bellot Rubio (2009) Martínez González, M. J. & Bellot Rubio, L. R. 2009, ApJ, 700, 1391
  * Martínez-Sykora et al. (2020) Martínez-Sykora, J., Leenaarts, J., De Pontieu, B., et al. 2020, ApJ, 889, 95
  * Martínez-Sykora et al. (2017) Martínez-Sykora, J., De Pontieu, B., Hansteen, V. H., et al. 2017, Science, 356, 1269
  * Moll et al. (2012) Moll, R., Cameron, R. H., & Schüssler, M. 2012, A&A, 541, A68
  * Molnar et al. (2021) Molnar, M. E., Reardon, K. P., Cranmer, S. R., et al. 2021, ApJ, 920, 125
  * Moreno-Insertis & Galsgaard (2013) Moreno-Insertis, F. & Galsgaard, K. 2013, ApJ, 771, 20
  * Morosin et al. (2022) Morosin, R., de la Cruz Rodríguez, J., Díaz Baso, C. J., et al. 2022, arXiv:2203.01688
  * Nóbrega-Siverio et al. (2020) Nóbrega-Siverio, D., Moreno-Insertis, F., Martínez-Sykora, J., et al. 2020, A&A, 633, A66
  * Nordlund et al. (2009) Nordlund, Å., Stein, R. F., & Asplund, M. 2009, Living Reviews in Solar Physics, 6, 2
  * Nordlund et al. (2018) Nordlund, Å., Ramsey, J. P., Popovas, A., et al. 2018, MNRAS, 477, 624
  * Okamoto & De Pontieu (2011) Okamoto, T. J. & De Pontieu, B. 2011, ApJ, 736, L24
  * Peter et al. (2014) Peter, H., Tian, H., Curdt, W., et al. 2014, Science, 346, 1255726
  * Rempel (2014) Rempel, M. 2014, ApJ, 789, 132
  * Rempel (2017) Rempel, M. 2017, ApJ, 834, 10
  * Rempel & Przybylski (2021) Rempel, M. & Przybylski, D. 2021, ApJ, 923, 79
  * Rouppe van der Voort et al. (2009) Rouppe van der Voort, L., Leenaarts, J., de Pontieu, B., et al. 2009, ApJ, 705, 272
  * Ruan et al. (2020) Ruan, W., Xia, C., & Keppens, R. 2020, ApJ, 896, 97
  * Rutten (2007) Rutten, R. J. 2007, The Physics of Chromospheric Plasmas, 368, 27
  * Rutten et al. (2019) Rutten, R. J., Rouppe van der Voort, L. H. M., & De Pontieu, B. 2019, A&A, 632, A96
  * Samanta et al. (2019) Samanta, T., Tian, H., Yurchyshyn, V., et al. 2019, Science, 366, 890
  * Schüssler (2013) Schüssler, M. 2013, Solar and Astrophysical Dynamos and Magnetic Activity, 294, 95
  * Shelyag et al. (2013) Shelyag, S., Cally, P. S., Reid, A., et al. 2013, ApJ, 776, L4
  * Shetye et al. (2021) Shetye, J., Verwichte, E., Stangalini, M., et al. 2021, ApJ, 921, 30
  * Smitha et al. (2017) Smitha, H. N., Anusha, L. S., Solanki, S. K., et al. 2017, ApJS, 229, 17
  * Stein & Nordlund (2012) Stein, R. F. & Nordlund, Å. 2012, ApJ, 753, L13
  * Steiner et al. (2010) Steiner, O., Franz, M., Bello González, N., et al. 2010, ApJ, 723, L180
  * Thornton & Parnell (2011) Thornton, L. M. & Parnell, C. E. 2011, Sol. Phys., 269, 13
  * Tomczyk et al. (2007) Tomczyk, S., McIntosh, S. W., Keil, S. L., et al. 2007, Science, 317, 1192
  * Tsiropoula et al. (2012) Tsiropoula, G., Tziotziou, K., Kontogiannis, I., et al. 2012, Space Sci. Rev., 169, 18
  * Van Doorsselaere et al. (2008) Van Doorsselaere, T., Nakariakov, V. M., Young, P. R., et al. 2008, A&A, 487, L17
  * Vissers et al. (2022) Vissers, G. J. M., Danilovic, S., Zhu, X., et al. 2022, A&A, 662, A88
  * Vögler et al. (2005) Vögler, A., Shelyag, S., Schüssler, M., et al. 2005, A&A, 429, 335
  * Withbroe & Noyes (1977) Withbroe, G. L. & Noyes, R. W. 1977, ARA&A, 15, 363
  * Wedemeyer-Böhm et al. (2012) Wedemeyer-Böhm, S., Scullion, E., Steiner, O., et al. 2012, Nature, 486, 505
  * Yokoyama & Shibata (1996) Yokoyama, T. & Shibata, K. 1996, PASJ, 48, 353