# SimPal: Towards a Meta-Conversational Framework to Understand Teacher’s
Instructional Goals for K-12 Physics

Effat Farhana<EMAIL_ADDRESS>Vanderbilt UniversityNashville,
TennesseeUSA ,  Souvika Sarkar<EMAIL_ADDRESS>Auburn UniversityAuburn,
AlabamaUSA ,  Ralph Knipper<EMAIL_ADDRESS>Auburn UniversityAuburn,
AlabamaUSA ,  Indrani Dey<EMAIL_ADDRESS>University of Wisconsin-
MadisonMadison, WisconsinUSA ,  Hari Narayanan<EMAIL_ADDRESS>Auburn
UniversityAuburn, AlabamaUSA ,  Sadhana Puntambekar
<EMAIL_ADDRESS>University of Wisconsin-MadisonMadison,
WisconsinUSA and  Santu Karmaker<EMAIL_ADDRESS>Auburn UniversityAuburn,
AlabamaUSA

(2024; 2024)

###### Abstract.

Simulations are widely used to teach science in grade schools. These
simulations are often augmented with a conversational artificial intelligence
(AI) agent to provide real-time scaffolding support for students conducting
experiments using the simulations. AI agents are highly tailored for each
simulation, with a predesigned set of Instructional Goals (IGs), making it
difficult for teachers to adjust IGs as the agent may no longer align with the
revised IGs. Additionally, teachers are hesitant to adopt new third-party
simulations for the same reasons. In this research, we introduce SimPal, a
Large Language Model (LLM) based meta-conversational agent, to solve this
misalignment issue between a pre-trained conversational AI agent and the
constantly evolving pedagogy of instructors. Through natural conversation with
SimPal, teachers first explain their desired IGs, based on which SimPal
identifies a set of relevant physical variables and their relationships to
create symbolic representations of the desired IGs. The symbolic
representations can then be leveraged to design prompts for the original AI
agent to yield better alignment with the desired IGs. We empirically evaluated
SimPal using two LLMs, ChatGPT-3.5 and PaLM 2, on 63 Physics simulations from
PhET and Golabz. Additionally, we examined the impact of different prompting
techniques on LLM’s performance by utilizing the TELeR taxonomy to identify
relevant physical variables for the IGs. Our findings showed that SimPal can
do this task with a high degree of accuracy when provided with a well-defined
prompt.

Large Language Models, Conversational AI, Meta-Conversation, K-12 Science

††journalyear: 2024††journalyear: 2024††copyright: rightsretained††conference:
Proceedings of the Eleventh ACM Conference on Learning @ Scale; July 18–20,
2024; Atlanta, GA, USA††booktitle: Proceedings of the Eleventh ACM Conference
on Learning @ Scale (L@S ’24), July 18–20, 2024, Atlanta, GA, USA††doi:
10.1145/3657604.3664695††isbn: 979-8-4007-0633-2/24/07††ccs: Computing
methodologies Natural language processing††ccs: Applied computing Education

##  1\. Introduction

Simulations are widely used in science education, and prior research shows
that using simulations in science education can enhance students’
comprehension of scientific concepts (Rutten et al., 2012; Kollöffel and De
Jong, 2013). However, students often need guidance and scaffolding when
conducting experiments with simulations (Graesser et al., 2006; González-Cruz
et al., 2003), and it is challenging for one teacher to provide real-time
support to multiple students simultaneously (Falloon, 2019). Recent
advancements in Large Language Models (LLMs) (Brown et al., 2020) have
revolutionized conversational AI agents as a plausible solution to provide
real-time support to students. But LLM-powered conversational AI agents also
present unique challenges. First, existing AI agents are highly customized for
a specific simulation with a predesigned set of Instructional Goals (IGs)
(Fischer and Dershimer, 2020). Therefore, teachers often struggle to edit
these predesigned IGs or redesign the IGs because the AI agent will no longer
be aligned with the revised IGs. Second, middle or high school science
teachers lack the technical expertise to customize AI agents (Park et al.,
2023). This leads to the use of pre-existing, non-customizable agents or
third-party software, which requires more time and resources for simulations.
For similar reasons, teachers also hesitate to integrate new/other third-party
(closed-source) simulations into their instructional materials.

How can we empower teachers to integrate any third-party (open or closed-
source) simulation into their instruction materials such that they can I)
freely design their own Instructional Goals (IGs) and II) quickly customize a
conversational AI agent to better align with their IGs? More importantly, how
can we achieve this goal without requiring teachers to understand the
technical details of Large Language Models (LLMs) like GPT-4 (Achiam et al.,
2023) and PaLM (Chowdhery et al., 2023; Anil et al., 2023)? While LLMs are
trained on vast internet text data and can aid in language comprehension tasks
like answering questions (Li et al., 2021) and facilitating human
conversations (Sundar and Heck, 2023), adapting LLMs to domain-specific tasks
is still challenging due to a lack of proper knowledge grounding in that
particular domain. It is also unrealistic to expect school teachers to learn
knowledge-grounding techniques that require in-depth machine learning or deep
learning knowledge.

This paper introduces SimPal, a meta-conversational agent that can assist
school teachers in adopting any existing physics simulation into their lesson
plan while allowing them to custom-design their own IGs and customize a
general-purpose LLM that aligns with those custom IGs, facilitating
instruction at scale. SimPal achieves this ambitious goal through meta-
conversation, which is essentially a conversation with the teacher about
structuring future conversations with students for simulation-based physics
experiments. Through natural (meta-)conversation with SimPal, teachers first
explain their desired IGs, based on which SimPal identifies a set of relevant
physical variables and their relationships to create symbolic representations
of the desired IGs. The symbolic representations can then be leveraged to
design prompts for the original AI agent to yield better alignment with the
desired IGs.

Figure 1. SimPal’s high-level overview: The teacher converses with SimPal,
discussing their simulation of interest and corresponding IG. As the
conversation progresses, SimPal extracts useful information from the
conversation to infer a computational representation of the teacher’s IG. That
internal representation is then communicated back to the teacher so they can
make any necessary adjustments.

Figure 1 presents an overview of SimPal’s interaction with the teacher. The
teacher conveys their IGs to SimPal, and then SimPal creates symbolic
representations of IGs by identifying relevant physical characteristics and
their interactions. Accurately identifying relevant physical variables is
crucial, as the IGs are encoded in terms of these variables and will guide
student interactions. SimPal’s architecture allows a teacher to tailor their
lesson plan by I) modifying the variables and relations of a simulation
through natural conversation and II) integrating any third-party simulation.

A challenging first step toward achieving this goal is to have the LLM
accurately identify variables from the simulation selected by a teacher that
best matches their IGs. In this paper, we empirically evaluate this task’s
accuracy on 63 physics simulations from PhET and Golabz using two LLMs:
ChatGPT-3.5 (Brown et al., 2020) and PaLM 2 (Anil et al., 2023). By employing
the recently introduced TELeR taxonomy, we examined the impact of different
prompting strategies on LLM’s ability to identify the physical variables
relevant to the IGs. Our findings demonstrated that SimPal can perform this
task with a high degree of accuracy when provided with an appropriately
crafted prompt.

##  2\. Background and Related Work

Conversational Agents in K-12 Science. Conversational agents, like Betty’s
Brain (Kinnebrew and Biswas, 2012; Kinnebrew et al., 2013) and MetaTutor
(Bouchet et al., 2012; Azevedo et al., 2009) have been used to foster
students’ learning. In Betty’s Brain (Kinnebrew and Biswas, 2012; Kinnebrew et
al., 2013), students learn science and mathematics concepts by teaching a
virtual agent, Betty. MetaTutor is a hypermedia-based biology learning
environment where teachers set learning goals and students choose
metacognitive processes, with occasional pedagogical agent prompts. All of the
aforementioned frameworks support students’ learning, whereas SimPal offers a
conversational AI assistant for teachers to develop simulation-based science
lesson plans.

LLMs and K-12 Education. LLMs have recently been increasingly used to enhance
student learning. Zhang et al. utilized LLMs in solving arithmetic math word
problems (Zhang et al., 2023). Prihar et al. (Prihar et al., 2023) utilized
GPT-3 with few shot learning to generate middle school math explanations on
ASSISTments. They found that GPT-3, primarily trained on English text,
generated explanations that were significantly inferior to teacher-authored
ones. Lately, Khan Academy has introduced a GPT-4 (Achiam et al., 2023)
powered tutoring system, Khanmigo (Khan, 2024), to assist teachers in planning
their lessons and providing feedback on students writing. Our proposed
approach, SimPal, is similar to Khanmigo in terms of assisting teachers in
planning their lessons. However, SimPal differs from Khanmigo in that it
allows teachers to integrate any third-party simulations into their lesson
plans.

Grounding LLMs to Unseen Tasks. LLMs, which represent vast amounts of
information, still require adaptation to specific tasks. Traditionally, task-
specific supervised data is used to fine-tune an LLM and adapt it to new
natural language processing (NLP) applications (Dai and Le, 2015; Howard and
Ruder, 2018; Radford et al., 2019; Hu et al., 2023). However, fine-tuning
faces two major challenges: insufficient training data and a lack of computing
resources and expertise. Few-shot learning is another approach that uses
prompt engineering (Gao et al., 2021; Chen et al., 2022) and domain-specific
examples (Brown et al., 2020). However, few-shot learning may be challenging
for lesson planning due to teachers’ individual teaching styles and
preferences. Reinforcement learning (RL) from human feedback (RLHF) employs RL
to optimize human preferences during LLM training (Ouyang et al., 2022).
However, it can incur significant exploration costs in RL. In contrast, our
approach, known as meta-conversation, uses natural conversation to infer a
human preference, i.e., the teacher’s lesson plan.

Prompt Taxonomy for LLM. As LLM’s prompt impacts the output accuracy of LLMs,
a recent study proposed a taxonomy, TELeR (Santu and Feng, 2023), to design
and evaluate prompting techniques systematically. TELeR taxonomy has seven
levels of prompts. We only explain the four prompt levels [Level 1- Level 4]
used in our study in Table 1.

##  3\. Instruction Goals and SimPal

We formulate a teacher’s IG in terms of variables and relationships among
variables. Consider a toy example where the teacher’s instructional goal is to
teach inversely proportional relationships in Newton’s Second Law of Motion in
a PhET simulation (Lecerio, 2019). As demonstrated in Figure 1, the teacher
conveys their IGs (e.g., inversely proportional relationships Newton’s Second
Law of Motion) to SimPal. Then, SimPal generates relevant topics (e.g., force,
acceleration) for the lab and asks the teacher to review those. Upon receiving
the teacher’s feedback, SimPal then identifies a set of relevant variables and
their relationships to create symbolic representations of the desired IGs
based on the teacher’s feedback.

The scope of our study is variable extraction in Physics simulations, with the
task described as follows.

Problem Definition. Given an IG of a simulation topic, SimPal uses LLMs to
generate variables. The task is to assess LLM’s accuracy of generated
variables given a natural language description of the IG.

Table 1. TELeR Taxonomy for LLM Prompting Level (L) | Definition
---|---
L1 | One sentence describing the high-level task goal
L2 | Multi-sentence prompt describing the high-level
| goals and sub-tasks
L3 | Prompt describing the high-level goals
| and sub-tasks in bulleted style.
L4 | Prompt specifying high-level goals, sub-tasks, and
| output evaluation criteria (e.g., few-shot examples)

##  4\. Experimental Design

###  4.1. Underlying LLM of SimPal

Table 2 lists three LLMs that we assessed in our preliminary analysis.

Table 2. LLMs Evaluated in this work. Model | Creator | # Parameters
---|---|---
ChatGPT-3.5 (gpt-3.5-turbo-0613, (Brown et al., 2020)) | OpenAI | 175B
PaLM 2 (chat-bison-001, (Anil et al., 2023)) | Google | 340B
LLaMA-2 (Llama-2-70b-chat-hf, (Touvron et al., 2023)) | Meta | 70B

###  4.2. Prompt Design with SimPal

We used Level 1 to Level 4 following the TELeR taxonomy in Table 1. Example
Level 1, 2, 3, and 4 prompts are given below.

  * •

Level 1 Identify and list the variables associated with these topics and the
description, along with their corresponding symbols.

  * •

Level 2 You are a physics teacher in a high school, and you are preparing a
lesson plan on related concepts. You have a list of topics and descriptions.
Your task is to Level 1 Prompt Text
Please provide the variables and symbols in the following JSON format. The key
would be the “Name” of the variable and the value would be the “Symbol”.
Include symbols and strictly follow the JSON format.
Do not print topics and descriptions; only variable names and corresponding
symbols are used.

  * •

Level 3 Level 2 Prompt Text
Please provide the variables and symbols in the following JSON format: [
“Name”: ” ”, “Symbol”: ” ” ]
\- List down all the relevant variables and their symbols.

  * •

Level 4 Level 3 Prompt Text
You are given a GUIDELINES_PROMPT to show an example but do not include the
variables from the GUIDELINES_PROMPT in the response if they are not relevant.

###  4.3. Simulation Dataset

Our dataset includes simulations from PhET (Wieman, 2002) and Golabz
(University of Twente, 2012). PhET hosts free math and science simulations.
Golabz hosts online science labs to promote inquiry learning at scale. We
performed preliminary analysis on five PhET simulations (Section 4.4) and
final evaluation on 32 PhET and 31 Golabz simulations (Section 5).

###  4.4. Preliminary Experiments and Insights

We investigated the output of three LLMs on five PhET simulations using the
TELeR taxonomy prompting levels [Level 1– Level 4]. Table 3 shows that all
three LLMs’ F1-scores fall with Level-4 prompting. Observing the format
accuracy of Levels 2 and 3, we conclude that ChatGPT-3.5 and PaLM 2 generate
output in the desired format. Based on the results in Table 3, we selected two
LLMs, ChatGPT-3.5 and PaLM 2, with Level 2 and Level 3 prompting levels.

Table 3. LLM Performance and Prompting Levels as per the TeLER Taxonomy. Format Accuracy = (0) 1, if LLM-generated Results (Do not) Follow the Prompt’s Format Specification. The Highest of each Metric per Prompt Level is in Bold Model | Format Accuracy | Precision | Recall | F1 Score
---|---|---|---|---
Level 1 |  |  |
ChatGPT-3.5 | 0 | 0.923 | 0.923 | 0.923
PaLM 2 | 0 | 0.923 | 0.958 | 0.94
LLaMA-2 (70B) | 0 | 0.929 | 1 | 0.963
Level 2 |  |  |
ChatGPT-3.5 | 1 | 0.78 | 0.729 | 0.754
PaLM 2 | 1 | 0.881 | 0.835 | 0.857
LLaMA-2 (70B) | 0 | 0.876 | 0.897 | 0.887
Level 3 |  |  |
ChatGPT-3.5 | 1 | 0.898 | 0.877 | 0.887
PaLM 2 | 1 | 0.853 | 0.848 | 0.851
LLaMA-2 (70B) | 0.4 | 0.755 | 0.767 | 0.761
Level 4 |  |  |
ChatGPT-3.5 | 1 | 0.732 | 0.691 | 0.711
PaLM 2 | 1 | 0.96 | 0.712 | 0.818
LLaMA-2 (70B) | 0 | 0.82 | 0.761 | 0.7894

##  5\. Final Case Study and Evaluation

Dataset. We evaluated SimPal’s performance in 63 Physics simulations,
including 32 from PhET and 31 from Golabz, as depicted in Table 4. For each
simulation, we designed two prompting levels (Level 2 and Level 3) using two
LLMs: ChatGPT-3.5 and PaLM 2.

Table 4. Dataset Statistics. L2 = Level 2, L3 = Level 3, #Prompts = Total Prompts by Level 2 and Level 3 | ChatGPT-3.5 | PaLM 2
---|---|---
| L2 | L3 | #Prompts | L2 | L3 | #Prompts
Golabz | 32 | 32 | 64 | 32 | 32 | 64
PhET | 31 | 31 | 62 | 31 | 31 | 62

Evaluation. We created prompts by extracting IGs and topics from lab web
pages. The IGs in PhET and Golabz are the learning goals and lab descriptions,
respectively. To identify gold standard variables for a lab, we identified
topics from the lab webpage and added additional terms from the Teacher
Resources section. Finally, we cross-referenced the relevant terms with an
open-source CK-12 Physical Science textbook (CK-12, 2024), aligned to the Next
Generation Science Standards (NGSS) (Council et al., 2013) to determine the
final gold standards and manually compared SimPal’s outputs to the gold
standards.

Metric. For each simulation, the LLM inferred variables are com- pared against
the list of gold standard variables to compute the true positive, false
positive, true negative, and false negative statistics. Then, all such
statistics in a dataset were aggregated to compute the final Precision,
Recall, and micro-averaged F1 score.

Table 5. An Example Annotation Scheme and SimPal’s Output Evaluation on a Lab Titled Wave on a String Topics | LLM Output | Gold Standard
---|---|---
| ””Name””: ””Wavelength””,” | frequency
| , ””Symbol””: ””0̆3BB”” |
Frequency | ””Name””: ””Frequency””, | amplitude
| ””Symbol””: ””f”” |
Amplitude | ””Name””: ””Period”” , | wavelength
| ””Symbol””: ””T”” |
Damping | ””Name””: ””Amplitude”” , | period
| ””Symbol””: ””A”” |
| ””Name””: ””Speed”” , |
| ””Symbol””: ””v”” |
| ””Name””: ””Damping Coefficient””, |
| ””Symbol””: ””0̆3B2”” |

Table 5 presents an example of SimPal’s output evaluation in a lab. We
calculated true positive values (TP) by comparing the number of matched LLM
outputs to the gold standard, resulting in four true positives. We calculated
false positives (FP) by subtracting the number of LLM outputs from the true
positives, yielding two false positives. Further, we calculated the false
negatives (FN) by subtracting true positives from the number of gold standard
outputs, resulting in zero false negatives in the given example.

###  5.1. Results and Discussion

Table 6 presents our evaluation results of SimPal.

TELeR Prompting Levels and SimPal Performance. Level 3 prompting resulted in
higher F1 scores for both LLMs than Level 2 in Golabz simulations. In PhET
simulations, Level 2 prompting produced a higher recall score than Level 3 in
PaLM 2.

LLM Family and SimPal Performance. ChatGPT-3.5 outperformed PaLM 2 in
F1-scores in both Golabz and PhET simulations with Level 3 prompting.
ChatGPT-3.5 also achieved a higher F1 score than PaLM 2 for Level 2 prompting
in Golabz simulations.

Simulation Source and SimPal Performance. Golabz simulations resulted in a
higher F1-score in both Level 2 and Level 3 prompting than PhET in
ChatGPT-3.5. In PaLM 2, Golabz simulations outperformed PhET in F1-score in
only Level 3 prompting.

The differences in F1 scores between Golabz and PhET simulations may be due to
content alignment differences. Golabz simulations may have been more aligned
with curriculum standards. Additionally, PhET simulations may contain more
complex or detailed information, resulting in the generation of extraneous
outputs.

Table 6. SimPal’s Performance with TeLER Prompt Levels 2 and 3 for LLM
Families and Simulation Sources in Table 4 ChatGPT-3.5
---
| Precision | Recall | F1 | Precision | Recall | F1
| Level 3 | Level 2
Golabz | 0.590 | 0.713 | 0.60 | 0.525 | 0.627 | 0.541
PhET | 0.560 | 0.654 | 0.581 | 0.523 | 0.519 | 0.539
PaLM 2
| Level 3 | Level 2
Golabz | 0.607 | 0.639 | 0.568 | 0.555 | 0.591 | 0.525
PhET | 0.512 | 0.584 | 0.547 | 0.529 | 0.628 | 0.547

##  6\. Future Work

We plan to extend SimPal to provide support to students via meta-conversation.
This includes feedback on writings, answered questions, and hint generation.
Additionally, we plan to use SimPal’s student interaction data to generate
recommendations for teachers, such as identifying high-performing and
struggling students.

##  7\. Conclusion

In this study, we present SimPal, an LLM-based meta-conversational framework
for simulation-based science labs, allowing teachers to include third-party
(open or closed-source) simulations into lesson plans, facilitating
instruction at scale. We assessed SimPal’s variable generation capabilities
with two LLMs: ChatGPT-3.5 and PaLM 2 on 63 Physics simulations from PhET and
Golabz, experimenting with different prompts following the TELeR prompting
taxonomy. Our findings showed that I) SimPal can provide a meaningful variable
list tailored to the lab and instruction goal, and II) the LLM prompting level
impacts SimPal’s performance. Furthermore, we observed that Golabz simulations
outperformed PhET in the F1 score. It is important to note a limitation in our
evaluation; our gold standard outputs may lack the subject matter expertise of
real school teachers, potentially leading to disparities in F1 scores. Future
work will involve incorporating feedback from teachers and subject matter
experts to improve the accuracy and relevance of LLM outputs.

###### Acknowledgements.

This work was funded in part by the National Science Foundation through Grant
2302974.

## References

  * (1)
  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023\.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ (2023).
  * Anil et al. (2023) Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023\.  Palm 2 technical report.  _arXiv preprint arXiv:2305.10403_ (2023).
  * Azevedo et al. (2009) Roger Azevedo, Amy M Witherspoon, Arthur C Graesser, Danielle S McNamara, Amber Chauncey, and Emily Siler. 2009.  MetaTutor: Analyzing Self-Regulated Learning in a Tutoring System for Biology. AIED.
  * Bouchet et al. (2012) François Bouchet, Roger Azevedo, John S Kinnebrew, and Gautam Biswas. 2012.  Identifying Students’ Characteristic Learning Behaviors in an Intelligent Tutoring System Fostering Self-Regulated Learning.  _International Educational Data Mining Society_ (2012).
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language models are few-shot learners.  _Advances in neural information processing systems_ 33 (2020), 1877–1901.
  * Chen et al. (2022) Xiang Chen, Lei Li, Ningyu Zhang, Xiaozhuan Liang, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022.  Decoupling knowledge from memorization: Retrieval-augmented prompt learning.  _Advances in Neural Information Processing Systems_ 35 (2022), 23908–23922.
  * Chowdhery et al. (2023) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023\.  Palm: Scaling language modeling with pathways.  _Journal of Machine Learning Research_ 24, 240 (2023), 1–113.
  * CK-12 (2024) CK-12. 2024.  CK-12 Physical Science for Middle School.  https://flexbooks.ck12.org/cbook/ck-12-middle-school-physical-science-flexbook-2.0/.
  * Council et al. (2013) National Research Council et al. 2013\.  Next generation science standards: For states, by states.  (2013).
  * Dai and Le (2015) Andrew M Dai and Quoc V Le. 2015.  Semi-supervised sequence learning.  _Advances in neural information processing systems_ 28 (2015).
  * Falloon (2019) Garry Falloon. 2019.  Using simulations to teach young students science concepts: An Experiential Learning theoretical analysis.  _Computers & Education_ 135 (2019), 138–159.
  * Fischer and Dershimer (2020) Christian Fischer and R Charles Dershimer. 2020.  Preparing teachers to use educational games, virtual experiments, and interactive science simulations for engaging students in the practices of science.  (2020).
  * Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021.  Making Pre-trained Language Models Better Few-shot Learners. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. 3816–3830.
  * González-Cruz et al. (2003) Javier González-Cruz, Rogelio Rodríguez-Sotres, and Mireya Rodríguez-Penagos. 2003.  On the convenience of using a computer simulation to teach enzyme kinetics to undergraduate students with biological chemistry-related curricula.  _Biochemistry and Molecular Biology Education_ 31, 2 (2003), 93–101.
  * Graesser et al. (2006) Arthur C Graesser, G Tanner Jackson, Hyun-Jeong Joyce Kim, and Andrew Olney. 2006.  AutoTutor 3-D Simulations: Analyzing Users’ Actions and Learning Trends. In _Proceedings of the Annual Meeting of the Cognitive Science Society_ , Vol. 28.
  * Howard and Ruder (2018) Jeremy Howard and Sebastian Ruder. 2018.  Universal Language Model Fine-tuning for Text Classification. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 328–339.
  * Hu et al. (2023) Nathan Hu, Eric Mitchell, Christopher D Manning, and Chelsea Finn. 2023.  Meta-Learning Online Adaptation of Language Models. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_. 4418–4432.
  * Khan (2024) Sal Khan. 2024.  Khanmigo.  https://www.khanacademy.org/khan-labs.
  * Kinnebrew and Biswas (2012) John S Kinnebrew and Gautam Biswas. 2012.  Identifying Learning Behaviors by Contextualizing Differential Sequence Mining with Action Features and Performance Evolution.  _International Educational Data Mining Society_ (2012).
  * Kinnebrew et al. (2013) John S Kinnebrew, Kirk M Loretz, and Gautam Biswas. 2013.  A contextualized, differential sequence mining method to derive students’ learning behavior patterns.  _JEDM— Journal of Educational Data Mining_ 5, 1 (2013), 190–219.
  * Kollöffel and De Jong (2013) Bas Kollöffel and Ton De Jong. 2013.  Conceptual Understanding of electrical circuits in secondary vocational engineering education: Combining traditional instruction with inquiry learning in a virtual lab.  _Journal of engineering education_ 102, 3 (2013), 375–393.
  * Lecerio (2019) Leah L. Lecerio. 2019.  Newton’s Second Law of Motion.  https://phet.colorado.edu/en/contributions/view/5092.
  * Li et al. (2021) Alexander Hanbo Li, Patrick Ng, Peng Xu, Henghui Zhu, Zhiguo Wang, and Bing Xiang. 2021.  Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. 4078–4088.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _Advances in neural information processing systems_ 35 (2022), 27730–27744.
  * Park et al. (2023) Joonhyeong Park, Tang Wee Teo, Arnold Teo, Jina Chang, Jun Song Huang, and Sengmeng Koo. 2023.  Integrating artificial intelligence into science lessons: teachers’ experiences and views.  _International Journal of STEM Education_ 10, 1 (2023), 61.
  * Prihar et al. (2023) Ethan Prihar, Morgan Lee, Mia Hopman, Adam Tauman Kalai, Sofia Vempala, Allison Wang, Gabriel Wickline, Aly Murray, and Neil Heffernan. 2023.  Comparing different approaches to generating mathematics explanations using large language models. In _International Conference on Artificial Intelligence in Education_. Springer, 290–295.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Rutten et al. (2012) Nico Rutten, Wouter R Van Joolingen, and Jan T Van Der Veen. 2012.  The learning effects of computer simulations in science education.  _Computers & education_ 58, 1 (2012), 136–153.
  * Santu and Feng (2023) Shubhra Kanti Karmaker Santu and Dongji Feng. 2023.  TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks. In _Findings of the Association for Computational Linguistics: EMNLP_. 14197––14203.
  * Sundar and Heck (2023) Anirudh S Sundar and Larry Heck. 2023.  cTBLS: Augmenting Large Language Models with Conversational Tables. In _Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023)_. 59–70.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023\.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ (2023).
  * University of Twente (2012) the Netherlands University of Twente. 2012.  Global Online Science Labs for Inquiry Learning in Schools.  https://premium.golabz.eu/about/go-lab-initiative.
  * Wieman (2002) Carl Wieman. 2002.  PhET.  https://phet.colorado.edu/.
  * Zhang et al. (2023) Mengxue Zhang, Zichao Wang, Zhichao Yang, Weiqi Feng, and Andrew Lan. 2023.  Interpretable Math Word Problem Solution Generation Via Step-by-step Planning. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics_. 6858––6877.


# Mitigating a discrete sign problem with extreme learning machines

Scott Lawrence<EMAIL_ADDRESS>Department of Physics, University of
Colorado, Boulder, CO 80309, USA  Los Alamos National Laboratory Theoretical
Division T-2, Los Alamos, NM 87545, USA  Yukari Yamauchi<EMAIL_ADDRESS>Institute for Nuclear Theory, University of Washington, Seattle, WA 98195, USA

###### Abstract

An extreme learning machine is a neural network in which only the weights in
the last layer are changed during training; for such networks training can be
performed efficiently and deterministically. We use an extreme learning
machine to construct a control variate that tames the sign problem in the
classical Ising model at imaginary external magnetic field. Using this control
variate, we directly compute the partition function at imaginary magnetic
field in two and three dimensions, yielding information on the positions of
Lee-Yang zeros.

††preprint: LA-UR-23-29892††preprint: INT-PUB-23-037

In seminal papers by Lee and Yang [1, 2], phase transitions in many-body
systems are investigated by studying the location of zeros of the partition
function in the complex plane of a control parameter, such as the temperature
or an external field. For example, the temperature at which a phase transition
occurs can be extracted by examining the location of the zeros as the
thermodynamic limit is taken. Given such a connection between the locations of
zeros and properties of phase transitions, the location of the Lee-Yang zeros
has been studied in a variety of systems in both theory [3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14] and experiment [15, 16, 17, 18]. Of particular interest to
this Letter, the location of the Lee-Yang zeros of the classical Ising model
has been studied by the high cumulants of thermodynamic observables [6, 10],
tensor networks [7], and on a complete and random graph [9].

Direct computation of the Ising partition function at imaginary external field
is obstructed by the presence of a sign problem, broadly similar to those that
prevent the simulation of real-time quantum dynamics via the path integral and
the computation of the nuclear equation of state at finite fermion density. A
great collection of methods has been developed over the past few decades to
treat sign problems that occur in lattice simulations; among them complex
Langevin [19], the density of states method [20], canonical methods [21, 22],
reweighting methods [23], series expansions in the chemical potential [24],
fermion bags [25], and analytic continuation from imaginary chemical
potentials [26], and contour deformation methods [27]. Contour deformations
methods in particular have recently been combined with machine learning
approaches [28, 29, 30, 31, 32, 33] including for theories with complex
couplings [34]; unfortunately, these methods are unable to treat systems with
discrete degrees of freedom.

In this Letter we introduce a machine learning approach for treating a lattice
sign problem, inspired by previous work on control variates [35, 36, 37, 38],
which does not depend on the ability to analytically continue the action and
observables. (See [39] for another approach to generalizing contour
deformation methods to spin systems.) We demonstrate the method in studying
the Lee-Yang zeros of the classical Ising model on a lattice $\Lambda$, whose
partition function is given by

$Z(J;h)=\sum_{s\in\\{0,1\\}^{|\Lambda|}}\exp\Bigg{\\{}J\sum_{\langle
x,y\rangle}s_{x}s_{y}+h\sum_{x}s_{x}\Bigg{\\}}\text{,}$ (1)

where the first sum in the exponential is taken over all pairs of neighboring
lattice sites. We will use both a two-dimensional square lattice and a three-
dimensional cubic lattice, each with periodic boundary conditions.

The partition function is polynomial in fugacity $e^{h}$, and at fixed $J$ is
therefore determined up to a constant factor by the location of its zeros. The
Lee-Yang theorem states that all zeros lie on the imaginary $h$ axis, on which
the partition function is purely real. At larger volumes the zeros become more
dense as a function of $-ih$, and when $J$ corresponds to the second-order
phase transition this line of zeros reaches the real axis in the infinite-
volume limit.

To find the zeros, we will operate at fixed $J$ and compute the ratio

$\frac{Z(h;J)}{Z_{Q}(J)}=\bigg{\langle}\exp\Big{\\{}h\sum_{x}s_{x}\Big{\\}}\bigg{\rangle}_{Q}\equiv\langle\mathcal{O}\rangle_{Q}\text{,}$
(2)

where the “quenched” expectation value of $\mathcal{O}$ is computed relative
to the Boltzmann distribution at $h=0$. This expectation value has a signal-
to-noise problem related to the configuration-dependent phase of
$\mathcal{O}\equiv e^{h\sum s}$: the variance of the phase is always of order
unity, while the signal, as a ratio of partition functions, falls
exponentially with the volume.

We increase the signal-to-noise ratio by constructing an appropriate control
variate, as was done in [40, 41, 37] for other signal-to-noise problems, and
in [35, 36] for various sign problems. We will find a function $f(s)$—termed
the _control variate_ —with vanishing expectation value, so that
$\langle\mathcal{O}\rangle=\langle\mathcal{O}-f\rangle$. The variance of the
new observable is

$\mathrm{Var}(\mathcal{O}-f)=\langle\mathcal{O}^{2}\rangle-2\langle\mathcal{O}f\rangle+\langle
f^{2}\rangle+\langle\mathcal{O}\rangle^{2}\text{.}$ (3)

Thus when $f$ is strongly correlated with $\mathcal{O}$, the new function
$\tilde{\mathcal{O}}\equiv\mathcal{O}-f$ has a smaller variance than the
original observable $\mathcal{O}$. We will refer to $\tilde{\mathcal{O}}$ as
the variance-reduced observable.

To avoid introducing any difficult-to-control bias in the Monte Carlo
calculation, we will construct $f(s)$ in such a way as to guarantee that its
expectation value vanishes exactly. Defining a discrete differentiation
operator by

$\nabla_{x}g(\vec{s})\equiv g(s)-g(s)|_{s_{x}\rightarrow-s_{x}}\text{,}$ (4)

we begin with a function $g(s)$ and construct the control variate $f(s)$
according to

$f(s)e^{-S}\equiv\sum_{x}\nabla_{x=x_{0}}g(T_{x}s)\text{.}$ (5)

Here the sum is taken over all lattice sites $x$ and $T_{x}$ is the
translation operator. The most general translationally invariant control
variate $f(s)$ can be expressed in this way—translational invariance is
desirable in this case as the observable of interest, $\mathcal{O}=e^{-ih\sum
s}$, is also translationally invariant. The choice of differentiation site
$x_{0}$ is irrelevant due to the sum over translations.

Any choice of $g(s)$ will yield a valid control variate, but not all will
improve the signal-to-noise ratio. As in [37], we begin the optimization
process by noting that given a basis of candidate control variates $F_{i}$,
the optimal linear combination may be determined by measuring the correlations
$M_{ij}=\langle F_{i}F_{j}\rangle$ and $v_{i}=\langle\mathcal{O}F_{i}\rangle$,
and computing

$c=M^{-1}v\text{.}$ (6)

The optimal control variate (within the chosen basis) is now given by
$f=\sum_{i}c_{i}F_{i}$.

To improve on the performance of a directly constructed basis of control
variates, we take inspiration from _extreme learning machines_ [42]. An
extreme learning machine (henceforth ELM) is a neural network in which only
the final layer is trained; all other weights are left equal to their
pseudorandomly initialized values. The learning process is now linear
regression, which is both efficient and deterministic. The loss in
expressivity from the fact that most weights are fixed, is at least partially
compensated by the ability to have a far larger network for the same
computational cost.

Figure 1: Ratio of partition functions on an $8\times 8$ lattice at $J=0.4$.
The bottom panel shows the deviation from the exact result, given by the
transfer matrix method.

In this Letter we define $g(s)$ via an ELM with a single hidden layer. The
inputs are $N$ functions $h_{j}(s)$, detailed below. The hidden layer is of
width $WL^{d}$, where $L^{d}$ is the spacetime volume of the lattice and $W$
is a tuneable width scaling factor. We use the CELU function [43] for the
activation layer. Thus the ELM can be written in the form

$g(x)=c_{i}\sigma_{\mathrm{CELU}}(A_{ij}x_{j})\text{.}$ (7)

Only the parameters in the vector $c$ are optimized. The $N\times WL^{d}$
matrix $A$ is left in its randomly initialized state for the entire procedure:
each component of $A$ is drawn independently from the uniform distribution on
the interval $[-L^{-d},L^{-d}]$. As an implementation detail, the
differentiation and averaging defined by Eq. (5) are performed before
multiplication by $c$. This allows the optimization of the coefficients $c$ to
be performed by directly solving a linear system, just as in [37].

In principle the spin configuration might be directly used as an input to the
ELM. In practice, as is often the case in machine learning tasks, a large
boost in performance is seen when the inputs are augmented with hand-crafted
functions of the spin configuration. Here, we select inputs to the ELM by
trial and error.

First, from the spin configuration $s$ we construct a ‘scaled’ version

$\tilde{s}_{x}=e^{-D(x)/D_{0}}s_{x}\text{,}$ (8)

where the distance function $D(x)$ is a measure of the distance from $x$ to
the origin:

$D(x)=\left[\sum_{k=1}^{d}2-2\cos\left(\frac{2\pi
x_{k}}{L}\right)\right]^{1/2}\text{.}$ (9)

This construction has the effect of encouraging the ELM to focus on short-
distance physics. We take $D_{0}=0.5$ throughout.

The input to the ELM consists of a total of $2+3L^{d}$ elements. The scaled
spin configuration $\tilde{s}$ accounts for $L^{d}$ of those. We also include
the real and imaginary parts of the phase of the Boltzmann factor (that is,
$\cos\mathrm{Im}\,S(s)$ and $\sin\mathrm{Im}\,S(s)$). Finally, we include two
scaled copies of $\tilde{s}$: $\tilde{s}\cos\mathrm{Im}\,S(s)$ and
$\tilde{s}\sin\mathrm{Im}\,S(s)$.

The detailed training procedure is as follows. The weights of the ELM are
independently drawn from the uniform distribution specified above. We collect
$K$ samples from the Ising model Boltzmann factor at some $J$ (but $h=0$), and
these samples are split into two sets. The first set, of size
$K_{\mathrm{learn}}$, is used only in fitting the optimal weights of the ELM,
while the second (of size $K_{\mathrm{est}}=K-K_{\mathrm{learn}}$) is used for
evaluating expectation values. This separation is necessary to avoid bias in
the final computed expectation values. Throughout this letter the two sets
will be chosen to be of equal size.

Figure 2: The performance of control variates constructed from an ELM across
lattice sizes and ELM widths, as measured by the ratio of the magnitude of the
expectation value of $\mathcal{O}$ to the variance of the estimator. All
calculations are performed at $J=0.2$ and an external field of $h=0.1i$, with
$5\times 10^{4}$ samples used to train each ELM and $5\times 10^{4}$ samples
per data point.

On each sample the ELM gives $WL^{d}$ outputs, which we will name $g_{i}(s)$.
Each output is differentiated with respect to the origin according to the
finite differencing operator defined above and summed over possible
translations, defining a basis $f_{i}(s)$ of possible control variates. The
correlations $M_{ij}$ and $c_{i}$ are measured on the $K_{\mathrm{learn}}$
samples, and from those measured values the optimal coefficients $c_{i}$
estimated. This defines the control variate to be used, and the improved
observable

$\tilde{\mathcal{O}}\equiv\mathcal{O}-\sum_{i}c_{i}f_{i}$ (10)

is measured on the remaining samples.

One additional technical detail must be treated: the correlation matrix $M$ is
typically ill-conditioned, with a condition number that rises rapidly with the
width parameter $W$. We regularize $M$ by adding a small multiple of the
identity matrix ($10^{-10}$ in this Letter).

To verify the correctness of this method, we first work with the model in two
dimensions. At modest values of $L$, the partition function may be computed
exactly by means of the transfer matrix. Three calculations of the partition
function on an $8\times 8$ lattice are shown in Fig. 1. The data points are
the Monte Carlo estimate of the ratio in Eq. (2) with or without the variance
reduction method applied. Each calculation is done with
$K_{\mathrm{learn}}=5\times 10^{3}=K_{\mathrm{est}}$ samples, and a width
scaling of $W=3$. Both data agree with the exact result, while the errors from
the calculation with the variance reduction are seen to be substantially
smaller than those without the reduction. The coupling of $J=0.4$ is chosen to
be slightly hotter than the critical coupling in two dimensions of
$J_{c}\approx 0.441$ [44].

Fig 2 shows the performance of the ELM, measured by the variance of the
estimator, as a function of the size of the lattice. We select a high
temperature and weak magnetic field, of $J=0.2$ and $h=0.1$, to avoid zeros of
the partition function and make this ratio meaningful. In additional to the
unimproved estimator, two computations are shown, corresponding to ELM widths
of $L^{2},3L^{2},$ and $8L^{2}$. We see that for any fixed size of ELM, there
is no exponential improvement in the variance, only a factor which is fixed or
decaying as $L$ increases. The typical improvement seen, a factor of $\sim
10$, corresponds to an advantage of $10^{2}$ in computational time when high
precision is desired.

Finally, in Fig. 3 we show the partition function at imaginary magnetic field
on a three-dimensional lattice, at lattice sizes of $L=4,5,6$. The largest
lattice size, $6^{3}$, is beyond what can be computed via the transfer matrix
with reasonable computational resources. We take $J=0.2$ to again be slightly
hotter than the phase transition (which sits at $J_{c}\approx 0.22$ [45]).
Each calculation is done with $K_{\mathrm{learn}}=2\times
10^{4}=K_{\mathrm{est}}$ samples, and a width scaling of $W=5$.

Figure 3: The partition function of the Ising model, as a function of complex
magnetic field strength, for different volumes in three spacetime dimensions
($4^{3}$, $5^{3}$, and $6^{3}$). The increasing density of the zeros at larger
volumes is clearly visible. At larger volumes and magnetic field strengths,
the raw data is insufficient to distinguish the partition function from $0$,
making it impossible to localize zeros further from the real axis; the use of
an ELM improves the situation somewhat, enabling the location of zeros to be
further constrained. Data points within two standard deviations of $0$ are
hidden for readability.

We have detailed a practical algorithm for mitigating volume-scaling sign
problems in lattice field theory. In the context in which we have tested the
method—the Ising model at imaginary external magnetic field—it consistently
yields a speedup of two orders of magnitude over the naive approach. This
approach is directly applicable to spin systems and other models in which the
degrees of freedom in the path integral are discrete, a marked advantage over
previous machine learning approaches that make use of contour deformations;
however, at this stage we have not attained an exponential improvement in the
average phase. This deficiency may be expected to be a fruitful direction for
future work.

All results in this Letter make use of the JAX library Equinox [46] for the
implementation of the ELM. S.L. is grateful to Frederic Koehler for originally
suggesting extreme learning as an technique of interest.

S.L. was supported at the beginning of this work by the U.S. Department of
Energy under Contract No. DE-SC0017905, and subsequently by a Richard P.
Feynman fellowship from the LANL LDRD program. LANL is operated by Triad
National Security, LLC, for the National Nuclear Security Administration of
U.S. Department of Energy (Contract No. 89233218CNA000001). Y.Y. is supported
by the INT’s U.S. Department of Energy grant No. DE-FG02-00ER41132.

## References

  * Yang and Lee [1952] C. N. Yang and T. D. Lee, Phys. Rev. 87, 404 (1952).
  * Lee and Yang [1952] T. D. Lee and C. N. Yang, Phys. Rev. 87, 410 (1952).
  * Biskup  _et al._ [2000] M. Biskup, C. Borgs, J. T. Chayes, L. J. Kleinwaks, and R. Kotecký, Phys. Rev. Lett. 84, 4794 (2000).
  * Alves  _et al._ [2002] N. A. Alves, J. P. N. Ferrite, and U. H. E. Hansmann, Phys. Rev. E 65, 036110 (2002).
  * Lee [2013] J. Lee, Phys. Rev. Lett. 110, 248101 (2013).
  * Deger  _et al._ [2018] A. Deger, K. Brandner, and C. Flindt, Phys. Rev. E 97, 012115 (2018).
  * García-Saez and Wei [2015] A. García-Saez and T.-C. Wei, Phys. Rev. B 92, 125132 (2015).
  * Gnatenko  _et al._ [2017] K. P. Gnatenko, A. Kargol, and V. M. Tkachuk, Phys. Rev. E 96, 032116 (2017).
  * Krasnytska  _et al._ [2016] M. Krasnytska, B. Berche, Y. Holovatch, and R. Kenna, Journal of Physics A: Mathematical and Theoretical 49, 135001 (2016).
  * Deger  _et al._ [2020] A. Deger, F. Brange, and C. Flindt, Physical Review B 102, 174418 (2020).
  * Connelly  _et al._ [2020] A. Connelly, G. Johnson, F. Rennecke, and V. Skokov, Phys. Rev. Lett. 125, 191602 (2020), arXiv:2006.12541 [cond-mat.stat-mech] .
  * Rennecke and Skokov [2022] F. Rennecke and V. V. Skokov, Annals Phys. 444, 169010 (2022), arXiv:2203.16651 [hep-ph] .
  * Johnson  _et al._ [2023] G. Johnson, F. Rennecke, and V. V. Skokov, Phys. Rev. D 107, 116013 (2023), arXiv:2211.00710 [hep-ph] .
  * Clarke  _et al._ [2023] D. A. Clarke, K. Zambello, P. Dimopoulos, F. Di Renzo, J. Goswami, G. Nicotra, C. Schmidt, and S. Singh, PoS LATTICE2022, 164 (2023), arXiv:2301.03952 [hep-lat] .
  * Binek [1998] C. Binek, Phys. Rev. Lett. 81, 5644 (1998).
  * Wei and Liu [2012] B.-B. Wei and R.-B. Liu, Phys. Rev. Lett. 109, 185701 (2012).
  * Peng  _et al._ [2015] X. Peng, H. Zhou, B.-B. Wei, J. Cui, J. Du, and R.-B. Liu, Phys. Rev. Lett. 114, 010601 (2015).
  * Brandner  _et al._ [2017] K. Brandner, V. F. Maisi, J. P. Pekola, J. P. Garrahan, and C. Flindt, Phys. Rev. Lett. 118, 180601 (2017).
  * Aarts and Stamatescu [2008] G. Aarts and I.-O. Stamatescu, JHEP 09, 018, arXiv:0807.1597 [hep-lat] .
  * Langfeld and Lucini [2016] K. Langfeld and B. Lucini,  _Proceedings, International Meeting Excited QCD 2016: Costa da Caparica, Portugal, March 6-12, 2016_ , Acta Phys. Polon. Supp. 9, 503 (2016), arXiv:1606.03879 [hep-lat] .
  * Alexandru  _et al._ [2005] A. Alexandru, M. Faber, I. Horvath, and K.-F. Liu, Phys. Rev. D72, 114513 (2005), arXiv:hep-lat/0507020 [hep-lat] .
  * de Forcrand and Kratochvila [2006] P. de Forcrand and S. Kratochvila,  _Hadron physics, proceedings of the Workshop on Computational Hadron Physics, University of Cyprus, Nicosia, Cyprus, 14-17 September 2005_ , Nucl. Phys. Proc. Suppl. 153, 62 (2006), [,62(2006)], arXiv:hep-lat/0602024 [hep-lat] .
  * Fodor and Katz [2002] Z. Fodor and S. D. Katz, Phys. Lett. B534, 87 (2002), arXiv:hep-lat/0104001 [hep-lat] .
  * Allton  _et al._ [2002] C. R. Allton, S. Ejiri, S. J. Hands, O. Kaczmarek, F. Karsch, E. Laermann, C. Schmidt, and L. Scorzato, Phys. Rev. D66, 074507 (2002), arXiv:hep-lat/0204010 [hep-lat] .
  * Chandrasekharan [2013] S. Chandrasekharan, Eur. Phys. J. A49, 90 (2013), arXiv:1304.4900 [hep-lat] .
  * de Forcrand and Philipsen [2007] P. de Forcrand and O. Philipsen, JHEP 01, 077, arXiv:hep-lat/0607017 [hep-lat] .
  * Alexandru  _et al._ [2022] A. Alexandru, G. Basar, P. F. Bedaque, and N. C. Warrington, Rev. Mod. Phys. 94, 015006 (2022), arXiv:2007.05436 [hep-lat] .
  * Alexandru  _et al._ [2017] A. Alexandru, P. F. Bedaque, H. Lamm, and S. Lawrence, Phys. Rev. D96, 094505 (2017), arXiv:1709.01971 [hep-lat] .
  * Ohnishi  _et al._ [2019] A. Ohnishi, Y. Mori, and K. Kashiwa, JPS Conf. Proc. 26, 024011 (2019).
  * Kashiwa  _et al._ [2019] K. Kashiwa, Y. Mori, and A. Ohnishi, Phys. Rev. D 99, 114005 (2019), arXiv:1903.03679 [hep-lat] .
  * Alexandru  _et al._ [2018a] A. Alexandru, P. F. Bedaque, H. Lamm, and S. Lawrence, Phys. Rev. D97, 094510 (2018a), arXiv:1804.00697 [hep-lat] .
  * Alexandru  _et al._ [2018b] A. Alexandru, P. F. Bedaque, H. Lamm, S. Lawrence, and N. C. Warrington, Phys. Rev. Lett. 121, 191602 (2018b), arXiv:1808.09799 [hep-lat] .
  * Lawrence and Yamauchi [2021] S. Lawrence and Y. Yamauchi, Phys. Rev. D 103, 114509 (2021), arXiv:2101.05755 [hep-lat] .
  * Lawrence  _et al._ [2022] S. Lawrence, H. Oh, and Y. Yamauchi, Phys. Rev. D 106, 114503 (2022), arXiv:2205.12303 [hep-lat] .
  * Lawrence [2020] S. Lawrence, Phys. Rev. D 102, 094504 (2020), arXiv:2009.10901 [hep-lat] .
  * Lawrence and Yamauchi [2023] S. Lawrence and Y. Yamauchi, Phys. Rev. D 107, 114505 (2023), arXiv:2212.14606 [hep-lat] .
  * Bhattacharya  _et al._ [2023] T. Bhattacharya, S. Lawrence, and J.-S. Yoo, Control variates for lattice field theory (2023), arXiv:2307.14950 [hep-lat] .
  * Bedaque and Oh [2023] P. F. Bedaque and H. Oh, Leveraging neural control variates for enhanced precision in lattice field theory (2023), arXiv:2312.08228 [hep-lat] .
  * Kashiwa  _et al._ [2023] K. Kashiwa, Y. Namekawa, A. Ohnishi, and H. Takase, Application of the path optimization method to a discrete spin system (2023), arXiv:2309.06018 [hep-lat] .
  * Fernandez and Martin-Mayor [2009] L. Fernandez and V. Martin-Mayor, Physical Review E 79, 051109 (2009).
  * Weigel and Janke [2010] M. Weigel and W. Janke, Physical Review E 81, 066701 (2010).
  * Huang  _et al._ [2006] G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, Neurocomputing 70, 489 (2006).
  * Barron [2017] J. T. Barron, arXiv preprint arXiv:1704.07483 (2017).
  * Onsager [1944] L. Onsager, Phys. Rev. 65, 117 (1944).
  * Talapov and Blöte [1996] A. Talapov and H. Blöte, Journal of Physics A: Mathematical and General 29, 5727 (1996).
  * Kidger and Garcia [2021] P. Kidger and C. Garcia, Differentiable Programming workshop at Neural Information Processing Systems 2021 (2021).