The MicroBooNE Collaboration ††thanks<EMAIL_ADDRESS>
# First simultaneous measurement of differential muon-neutrino charged-current
cross sections on argon for final states with and without protons using
MicroBooNE data

P. Abratenko  Tufts University, Medford, MA, 02155, USA  O. Alterkait  Tufts
University, Medford, MA, 02155, USA  D. Andrade Aldana  Illinois Institute of
Technology (IIT), Chicago, IL 60616, USA  L. Arellano  The University of
Manchester, Manchester M13 9PL, United Kingdom  J. Asaadi  University of
Texas, Arlington, TX, 76019, USA  A. Ashkenazi  Tel Aviv University, Tel Aviv,
Israel, 69978  S. Balasubramanian  Fermi National Accelerator Laboratory
(FNAL), Batavia, IL 60510, USA  B. Baller  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  G. Barr  University of Oxford,
Oxford OX1 3RH, United Kingdom  D. Barrow  University of Oxford, Oxford OX1
3RH, United Kingdom  J. Barrow  Massachusetts Institute of Technology (MIT),
Cambridge, MA, 02139, USA  University of Minnesota, Minneapolis, MN, 55455,
USA  Tel Aviv University, Tel Aviv, Israel, 69978  V. Basque  Fermi National
Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  O. Benevides Rodrigues
Illinois Institute of Technology (IIT), Chicago, IL 60616, USA  S. Berkman
Fermi National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  Michigan
State University, East Lansing, MI 48824, USA  A. Bhanderi  The University of
Manchester, Manchester M13 9PL, United Kingdom  A. Bhat  University of
Chicago, Chicago, IL, 60637, USA  M. Bhattacharya  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  M. Bishai  Brookhaven National
Laboratory (BNL), Upton, NY, 11973, USA  A. Blake  Lancaster University,
Lancaster LA1 4YW, United Kingdom  B. Bogart  University of Michigan, Ann
Arbor, MI, 48109, USA  T. Bolton  Kansas State University (KSU), Manhattan,
KS, 66506, USA  J. Y. Book  Harvard University, Cambridge, MA 02138, USA  M.
B. Brunetti  University of Warwick, Coventry CV4 7AL, United Kingdom  L.
Camilleri  Columbia University, New York, NY, 10027, USA  Y. Cao  The
University of Manchester, Manchester M13 9PL, United Kingdom  D. Caratelli
University of California, Santa Barbara, CA, 93106, USA  F. Cavanna  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  G. Cerati
Fermi National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  A.
Chappell  University of Warwick, Coventry CV4 7AL, United Kingdom  Y. Chen
SLAC National Accelerator Laboratory, Menlo Park, CA, 94025, USA  J. M. Conrad
Massachusetts Institute of Technology (MIT), Cambridge, MA, 02139, USA  M.
Convery  SLAC National Accelerator Laboratory, Menlo Park, CA, 94025, USA  L.
Cooper-Troendle  University of Pittsburgh, Pittsburgh, PA, 15260, USA  J. I.
Crespo-Anadón  Centro de Investigaciones Energéticas, Medioambientales y
Tecnológicas (CIEMAT), Madrid E-28040, Spain  R. Cross  University of Warwick,
Coventry CV4 7AL, United Kingdom  M. Del Tutto  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  S. R. Dennis  University of
Cambridge, Cambridge CB3 0HE, United Kingdom  P. Detje  University of
Cambridge, Cambridge CB3 0HE, United Kingdom  A. Devitt  Lancaster University,
Lancaster LA1 4YW, United Kingdom  R. Diurba  Universität Bern, Bern CH-3012,
Switzerland  Z. Djurcic  Argonne National Laboratory (ANL), Lemont, IL, 60439,
USA  R. Dorrill  Illinois Institute of Technology (IIT), Chicago, IL 60616,
USA  K. Duffy  University of Oxford, Oxford OX1 3RH, United Kingdom  S. Dytman
University of Pittsburgh, Pittsburgh, PA, 15260, USA  B. Eberly  University of
Southern Maine, Portland, ME, 04104, USA  P. Englezos  Rutgers University,
Piscataway, NJ, 08854, USA  A. Ereditato  University of Chicago, Chicago, IL,
60637, USA  Fermi National Accelerator Laboratory (FNAL), Batavia, IL 60510,
USA  J. J. Evans  The University of Manchester, Manchester M13 9PL, United
Kingdom  R. Fine  Los Alamos National Laboratory (LANL), Los Alamos, NM,
87545, USA  O. G. Finnerud  The University of Manchester, Manchester M13 9PL,
United Kingdom  W. Foreman  Illinois Institute of Technology (IIT), Chicago,
IL 60616, USA  B. T. Fleming  University of Chicago, Chicago, IL, 60637, USA
D. Franco  University of Chicago, Chicago, IL, 60637, USA  A. P. Furmanski
University of Minnesota, Minneapolis, MN, 55455, USA  F. Gao  University of
California, Santa Barbara, CA, 93106, USA  D. Garcia-Gamez  Universidad de
Granada, Granada E-18071, Spain  S. Gardiner  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  G. Ge  Columbia University, New
York, NY, 10027, USA  S. Gollapinni  Los Alamos National Laboratory (LANL),
Los Alamos, NM, 87545, USA  E. Gramellini  The University of Manchester,
Manchester M13 9PL, United Kingdom  P. Green  University of Oxford, Oxford OX1
3RH, United Kingdom  H. Greenlee  Fermi National Accelerator Laboratory
(FNAL), Batavia, IL 60510, USA  L. Gu  Lancaster University, Lancaster LA1
4YW, United Kingdom  W. Gu  Brookhaven National Laboratory (BNL), Upton, NY,
11973, USA  R. Guenette  The University of Manchester, Manchester M13 9PL,
United Kingdom  P. Guzowski  The University of Manchester, Manchester M13 9PL,
United Kingdom  L. Hagaman  University of Chicago, Chicago, IL, 60637, USA  O.
Hen  Massachusetts Institute of Technology (MIT), Cambridge, MA, 02139, USA
C. Hilgenberg  University of Minnesota, Minneapolis, MN, 55455, USA  G. A.
Horton-Smith  Kansas State University (KSU), Manhattan, KS, 66506, USA  Z.
Imani  Tufts University, Medford, MA, 02155, USA  B. Irwin  University of
Minnesota, Minneapolis, MN, 55455, USA  M. S. Ismail  University of
Pittsburgh, Pittsburgh, PA, 15260, USA  C. James  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  X. Ji  Nankai University, Nankai
District, Tianjin 300071, China  J. H. Jo  Brookhaven National Laboratory
(BNL), Upton, NY, 11973, USA  R. A. Johnson  University of Cincinnati,
Cincinnati, OH, 45221, USA  Y.-J. Jwa  Columbia University, New York, NY,
10027, USA  D. Kalra  Columbia University, New York, NY, 10027, USA  N. Kamp
Massachusetts Institute of Technology (MIT), Cambridge, MA, 02139, USA  G.
Karagiorgi  Columbia University, New York, NY, 10027, USA  W. Ketchum  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  M. Kirby
Brookhaven National Laboratory (BNL), Upton, NY, 11973, USA  Fermi National
Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  T. Kobilarcik  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  I. Kreslo
Universität Bern, Bern CH-3012, Switzerland  M. B. Leibovitch  University of
California, Santa Barbara, CA, 93106, USA  I. Lepetic  Rutgers University,
Piscataway, NJ, 08854, USA  J.-Y. Li  University of Edinburgh, Edinburgh EH9
3FD, United Kingdom  K. Li  Wright Laboratory, Department of Physics, Yale
University, New Haven, CT, 06520, USA  Y. Li  Brookhaven National Laboratory
(BNL), Upton, NY, 11973, USA  K. Lin  Rutgers University, Piscataway, NJ,
08854, USA  B. R. Littlejohn  Illinois Institute of Technology (IIT), Chicago,
IL 60616, USA  H. Liu  Brookhaven National Laboratory (BNL), Upton, NY, 11973,
USA  W. C. Louis  Los Alamos National Laboratory (LANL), Los Alamos, NM,
87545, USA  X. Luo  University of California, Santa Barbara, CA, 93106, USA
C. Mariani  Center for Neutrino Physics, Virginia Tech, Blacksburg, VA, 24061,
USA  D. Marsden  The University of Manchester, Manchester M13 9PL, United
Kingdom  J. Marshall  University of Warwick, Coventry CV4 7AL, United Kingdom
N. Martinez  Kansas State University (KSU), Manhattan, KS, 66506, USA  D. A.
Martinez Caicedo  South Dakota School of Mines and Technology (SDSMT), Rapid
City, SD, 57701, USA  S. Martynenko  Brookhaven National Laboratory (BNL),
Upton, NY, 11973, USA  A. Mastbaum  Rutgers University, Piscataway, NJ, 08854,
USA  I. Mawby  Lancaster University, Lancaster LA1 4YW, United Kingdom  N.
McConkey  University College London, London WC1E 6BT, United Kingdom  V.
Meddage  Kansas State University (KSU), Manhattan, KS, 66506, USA  J. Micallef
Massachusetts Institute of Technology (MIT), Cambridge, MA, 02139, USA  Tufts
University, Medford, MA, 02155, USA  K. Miller  University of Chicago,
Chicago, IL, 60637, USA  A. Mogan  Colorado State University, Fort Collins,
CO, 80523, USA  T. Mohayai  Fermi National Accelerator Laboratory (FNAL),
Batavia, IL 60510, USA  Indiana University, Bloomington, IN 47405, USA  M.
Mooney  Colorado State University, Fort Collins, CO, 80523, USA  A. F. Moor
University of Cambridge, Cambridge CB3 0HE, United Kingdom  C. D. Moore  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  L. Mora Lepin
The University of Manchester, Manchester M13 9PL, United Kingdom  M. M.
Moudgalya  The University of Manchester, Manchester M13 9PL, United Kingdom
S. Mulleriababu  Universität Bern, Bern CH-3012, Switzerland  D. Naples
University of Pittsburgh, Pittsburgh, PA, 15260, USA  A. Navrer-Agasson  The
University of Manchester, Manchester M13 9PL, United Kingdom  N. Nayak
Brookhaven National Laboratory (BNL), Upton, NY, 11973, USA  M. Nebot-Guinot
University of Edinburgh, Edinburgh EH9 3FD, United Kingdom  J. Nowak
Lancaster University, Lancaster LA1 4YW, United Kingdom  N. Oza  Columbia
University, New York, NY, 10027, USA  O. Palamara  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  N. Pallat  University of Minnesota,
Minneapolis, MN, 55455, USA  V. Paolone  University of Pittsburgh, Pittsburgh,
PA, 15260, USA  A. Papadopoulou  Argonne National Laboratory (ANL), Lemont,
IL, 60439, USA  V. Papavassiliou  New Mexico State University (NMSU), Las
Cruces, NM, 88003, USA  H. B. Parkinson  University of Edinburgh, Edinburgh
EH9 3FD, United Kingdom  S. F. Pate  New Mexico State University (NMSU), Las
Cruces, NM, 88003, USA  N. Patel  Lancaster University, Lancaster LA1 4YW,
United Kingdom  Z. Pavlovic  Fermi National Accelerator Laboratory (FNAL),
Batavia, IL 60510, USA  E. Piasetzky  Tel Aviv University, Tel Aviv, Israel,
69978  I. Pophale  Lancaster University, Lancaster LA1 4YW, United Kingdom  X.
Qian  Brookhaven National Laboratory (BNL), Upton, NY, 11973, USA  J. L. Raaf
Fermi National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  V.
Radeka  Brookhaven National Laboratory (BNL), Upton, NY, 11973, USA  A.
Rafique  Argonne National Laboratory (ANL), Lemont, IL, 60439, USA  M.
Reggiani-Guzzo  University of Edinburgh, Edinburgh EH9 3FD, United Kingdom
The University of Manchester, Manchester M13 9PL, United Kingdom  L. Ren  New
Mexico State University (NMSU), Las Cruces, NM, 88003, USA  L. Rochester  SLAC
National Accelerator Laboratory, Menlo Park, CA, 94025, USA  J. Rodriguez
Rondon  South Dakota School of Mines and Technology (SDSMT), Rapid City, SD,
57701, USA  M. Rosenberg  Tufts University, Medford, MA, 02155, USA  M. Ross-
Lonergan  Los Alamos National Laboratory (LANL), Los Alamos, NM, 87545, USA
C. Rudolf von Rohr  Universität Bern, Bern CH-3012, Switzerland  I. Safa
Columbia University, New York, NY, 10027, USA  G. Scanavini  Wright
Laboratory, Department of Physics, Yale University, New Haven, CT, 06520, USA
D. W. Schmitz  University of Chicago, Chicago, IL, 60637, USA  A. Schukraft
Fermi National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  W.
Seligman  Columbia University, New York, NY, 10027, USA  M. H. Shaevitz
Columbia University, New York, NY, 10027, USA  R. Sharankova  Fermi National
Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  J. Shi  University of
Cambridge, Cambridge CB3 0HE, United Kingdom  E. L. Snider  Fermi National
Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  M. Soderberg  Syracuse
University, Syracuse, NY, 13244, USA  S. Söldner-Rembold  The University of
Manchester, Manchester M13 9PL, United Kingdom  J. Spitz  University of
Michigan, Ann Arbor, MI, 48109, USA  M. Stancari  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  J. St. John  Fermi National
Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  T. Strauss  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  A. M. Szelc
University of Edinburgh, Edinburgh EH9 3FD, United Kingdom  W. Tang
University of Tennessee, Knoxville, TN, 37996, USA  N. Taniuchi  University of
Cambridge, Cambridge CB3 0HE, United Kingdom  K. Terao  SLAC National
Accelerator Laboratory, Menlo Park, CA, 94025, USA  C. Thorpe  Lancaster
University, Lancaster LA1 4YW, United Kingdom  The University of Manchester,
Manchester M13 9PL, United Kingdom  D. Torbunov  Brookhaven National
Laboratory (BNL), Upton, NY, 11973, USA  D. Totani  University of California,
Santa Barbara, CA, 93106, USA  M. Toups  Fermi National Accelerator Laboratory
(FNAL), Batavia, IL 60510, USA  Y.-T. Tsai  SLAC National Accelerator
Laboratory, Menlo Park, CA, 94025, USA  J. Tyler  Kansas State University
(KSU), Manhattan, KS, 66506, USA  M. A. Uchida  University of Cambridge,
Cambridge CB3 0HE, United Kingdom  T. Usher  SLAC National Accelerator
Laboratory, Menlo Park, CA, 94025, USA  B. Viren  Brookhaven National
Laboratory (BNL), Upton, NY, 11973, USA  M. Weber  Universität Bern, Bern
CH-3012, Switzerland  H. Wei  Louisiana State University, Baton Rouge, LA,
70803, USA  A. J. White  University of Chicago, Chicago, IL, 60637, USA  S.
Wolbers  Fermi National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA
T. Wongjirad  Tufts University, Medford, MA, 02155, USA  M. Wospakrik  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  K. Wresilo
University of Cambridge, Cambridge CB3 0HE, United Kingdom  W. Wu  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  University of
Pittsburgh, Pittsburgh, PA, 15260, USA  E. Yandel  University of California,
Santa Barbara, CA, 93106, USA  T. Yang  Fermi National Accelerator Laboratory
(FNAL), Batavia, IL 60510, USA  L. E. Yates  Fermi National Accelerator
Laboratory (FNAL), Batavia, IL 60510, USA  H. W. Yu  Brookhaven National
Laboratory (BNL), Upton, NY, 11973, USA  G. P. Zeller  Fermi National
Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  J. Zennamo  Fermi
National Accelerator Laboratory (FNAL), Batavia, IL 60510, USA  C. Zhang
Brookhaven National Laboratory (BNL), Upton, NY, 11973, USA

(February 29 2024)

###### Abstract

We report the first double-differential neutrino-argon cross section
measurement made simultaneously for final states with and without protons for
the inclusive muon neutrino charged-current interaction channel. The proton
kinematics of this channel are further explored with a differential cross
section measurement as a function of the leading proton’s kinetic energy that
extends across the detection threshold. These measurements utilize data
collected using the MicroBooNE detector from 6.4$\times 10^{20}$ protons on
target from the Fermilab Booster Neutrino Beam with a mean neutrino energy of
$\sim$0.8 GeV. Extensive data-driven model validation utilizing the
conditional constraint formalism is employed. This motivates enlarging the
uncertainties with an empirical reweighting approach to minimize the
possibility of extracting biased cross section results. The extracted nominal
flux-averaged cross sections are compared to widely used event generator
predictions revealing severe mismodeling of final states without protons for
muon neutrino charged-current interactions, possibly from insufficient
treatment of final state interactions. These measurements provide a wealth of
new information useful for improving event generators which will enhance the
sensitivity of precision measurements in neutrino experiments.

††preprint: APS/123-QED

Neutrino experiments measure flavor oscillations as a function of neutrino
energy in order to determine neutrino mixing parameters and search for new
physics beyond the Standard Model [1, 2, 3, 4, 5]. This requires precise
mapping between reconstructed and true neutrino energy. The inclusive muon
neutrino charged current ($\nu_{\mu}$CC) interaction channel,
$\nu_{\mu}\text{N}\rightarrow\mu^{-}X$, where N is the struck nucleus and $X$
is the hadronic final state, is important for these measurements because it
identifies the neutrino flavor with high purity and efficiency due to the sole
requirement of detecting the muon.

A number of these neutrino experiments utilize liquid argon time projection
chambers (LArTPCs) [6, 5]. These tracking calorimeters have low detection
thresholds and excellent particle identification (PID) capabilities [7, 8, 9,
10, 11]. LArTPCs enable the inclusive $\nu_{\mu}$CC channel to be divided into
subchannels based on the composition of the final state, each having a
different mapping between true and reconstructed neutrino energy. This
improves the energy reconstruction and increases the sensitivity of precision
measurements [12].

Maximizing the physics reach of LArTPCs requires neutrino-argon interaction
modeling capable of describing all final state particles. Existing models are
unable to describe data with such detail, necessitating large interaction
modeling uncertainties [12, 13]. This is unsurprising; theoretical models
attempting to describe experimental observables must simultaneously account
for multiple scattering mechanisms [14], in-medium nuclear modifications to
the fundamental neutrino interactions [15, 16], and final-state interactions
(FSI) involving the hadronic reaction products as they exit the nucleus [17].
The prominence of nuclear effects grows with the size of the target nucleus,
further complicating the modeling of scattering for heavy nuclei like argon.

Efforts to simulate $\nu_{\mu}$CC interactions benefit from measurements that
simultaneously probe the leptonic and hadronic kinematics. Building on
previous MicroBooNE work [18, 19, 20], we report a double-differential
measurement of the muon energy, $E_{\mu}$, and muon scattering angle with
respect to the neutrino beam, $\cos\theta_{\mu}$, for the $\nu_{\mu}$CC
channel split into final states with one or more protons (“Np” where N$\geq
1$) and without protons (“0p”). An event is only included in the Np signal if
the leading proton exiting the nucleus has kinetic energy above 35 MeV, which
roughly corresponds to the proton tracking threshold in MicroBooNE [21]. The
proton kinematics are further explored with a differential cross section
measurement of the inclusive $\nu_{\mu}$CC channel (“Xp” where X$\geq 0$) as a
function of the leading proton’s kinetic energy, $K_{p}$, that extends across
the tracking threshold via the inclusion of a 0-35 MeV bin that includes
events without a final state proton.

Additional motivation for these measurements comes from the fact that LArTPCs
utilize the gap between the neutrino and shower vertices to differentiate
electrons from photons. The absence of additional vertex activity, usually
from protons, makes it difficult to determine if a gap is present. This
impacts $\nu_{e}$CC selections, which are the signal in many oscillation
measurements, through lower efficiencies and purities for $\nu_{e}$CC 0p
events than $\nu_{e}$CC Np events [22, 23]. Since the $\nu_{\mu}$CC channel is
essential in constraining the $\nu_{e}$CC prediction, improved modeling of 0p
and Np final states for $\nu_{\mu}$CC is important. This need is highlighted
by prior MicroBooNE results [20], which observed an excess of $\nu_{\mu}$CC
events at low reconstructed neutrino energies, potentially indicative of poor
0p cross section modeling.

We utilize data collected with the MicroBooNE detector [24] from an exposure
of $6.4\times 10^{20}$ protons on target (POT) from the Booster Neutrino Beam
(BNB) at a mean neutrino energy of $\sim$0.8 GeV [25]. The detector is
comprised of a TPC volume with an active mass of 85 tonnes of liquid argon,
and an array of 32 photomultiplier tubes (PMTs). When an interaction occurs in
the detector, scintillation light and ionization electrons are produced by the
charged particles emanating from the interaction. The light is recorded by the
PMTs, providing ns-scale timing used to reject background cosmic ray events
that are out of time with the beam. The ionization electrons drift in a 273
V/cm electric field to three wire readout planes which record charge
distributions used for calorimetry and three-dimensional (3D) mm-scale
imaging.

Event reconstruction, calorimetry, and PID are performed with the Wire-Cell
topographical 3D image processing algorithm [26]. Wire charge distributions
are first deconvolved from the detector response by a TPC signal processing
algorithm [27, 28, 29]. Wire-Cell uses the deconvolved readouts to reconstruct
3D images without topological assumptions about the source of activity as
“tracks” or “showers” [30]. A many-to-many TPC-charge to PMT-light matching
algorithm is used for further cosmic ray rejection [31].

Particle identification starts with finding kinks in the selected cluster to
identify tracks [30]. Candidate neutrino vertices and particle flow trees are
formed concurrently based on $dQ/dx$, topology, and particle flow
relationships. A final neutrino vertex is chosen by a SparseConvNet deep
neural network [32]. Proton and muon candidates are distinguished based on
characteristic differences in their $dQ/dx$ profile [21] using a test
statistic constructed from a Kolmogorov–Smirnov shape comparison score [33]
and the normalization to the median $dQ/dx$ of protons.

Two methods are used to calculate the energy of track-like particles; range
and summation of $dE/dx$. Between the two methods, the energy resolution is
$\sim$10% for muons at all energies and $\sim$8% for protons below 200 MeV,
above this the resolution degrades to $\sim$25% due to the increased
probability of re-scattering [21]. The range method is used for stopping
tracks and is based on the NIST PSTAR database [34]. This method estimates the
energy of tracks without bias [21]. Summation of $dE/dx$ is used for tracks
that are shorter than 4 cm, exit the active volume, have a “wiggled” topology
[30], or emit $\delta$ rays. To calculate the kinetic energy, this method
converts $dQ/dx$ to $dE/dx$ with an effective recombination model then sums
$dE/dx$ for each $\sim$6 mm segment of the track. The $dE/dx$ method
underestimates the energy by $\sim$10% [30, 35], but this is incorporated into
the detector model and appears consistent in data and simulation [20].

The $\nu_{\mu}$CC event selection is identical to that of previous MicroBooNE
work [20, 18, 19, 21]. It utilizes the “generic neutrino selection” [36] as a
preselection, which reduces cosmic ray contamination down to 15%. Backgrounds
are further rejected with a boosted decision tree (BDT) trained using the
XGBoost library [26] on a set of background taggers. The efficiency (purity)
of the fully inclusive $\nu_{\mu}$CC selection is 68% (92%), with backgrounds
predominantly coming from neutral current $\pi^{\pm}$ events. The selection is
further divided into 0p and Np selections based on the reconstructed leading
proton’s kinetic energy, $K_{p}^{rec}$. The Np selection contains events in
which there is at least one proton with $K_{p}^{rec}>35$ MeV. The 0p selection
contains all other events. This threshold is motivated by MicroBooNE’s ability
to detect tracks $>1$ cm, which corresponds to 35 MeV for protons [21]. True
Np events are thus defined as having a proton with true $K_{p}>35$ MeV and
true 0p events are defined as having either zero final-state protons, or no
proton with $K_{p}>35$ MeV. The Np selection has a 49% efficiency for true Np
events and a high purity of 95% due to low contamination from
non-$\nu_{\mu}$CC and 0p events. The 0p selection has an efficiency of 53% for
true 0p events. The larger number of Np events, which outnumber 0p events
$\sim$7:1, increases the prominence of Np events in the 0p selection, reducing
the fraction of true 0p events in the 0p selection to 32% [21].

The 0p and Np cross sections are extracted simultaneously. This allows the
number of true Np events in the 0p selection to be predicted based on the
observation of the Np selection (and vice versa). The same strategy is
employed in other MicroBooNE work [21, 23]. Simultaneous extraction requires
unfolding

$\begin{pmatrix}M_{0p}\\\
M_{Np}\end{pmatrix}=\begin{pmatrix}R_{0p0p}&R_{0pNp}\\\
R_{Np0p}&R_{NpNp}\end{pmatrix}\cdot\begin{pmatrix}S_{0p}\\\
S_{Np}\end{pmatrix}+\begin{pmatrix}B_{0p}\\\ B_{Np}\end{pmatrix},$ (1)

where $M$ is the reconstructed distribution, $S$ is the differential cross
section to be extracted, $B$ is the distribution of background events that are
not part of the inclusive $\nu_{\mu}$CC channel, and $R$ is the response
matrix describing the mapping between the true and reconstructed
distributions. The first (second) index on $R$ corresponds to the
reconstructed (true) proton multiplicity.

The Wiener-SVD unfolding technique [37] is used to extract nominal flux-
averaged cross section results [38]. This method returns a regularized
unfolded cross section and corresponding covariance matrix, $V_{S}$, which
describes the uncertainties and bin-to-bin correlations on the result. The
form of Eq. (1) allows the unfolding to account for correlations between the
0p and Np channels [21]. An additional smearing matrix, $A_{C}$, that captures
the bias induced by regularization is also obtained in the unfolding. Cross
section predictions should be multiplied by $A_{C}$ when compared to the
unfolded result. The extracted cross sections, $A_{C}$, and $V_{S}$ which is
obtained via blockwise unfolding to preserve inter-variable correlations [39,
21], can be found in the Supplemental Material.

Monte Carlo (MC) simulations are used to estimate $R$ and part of $B$. The
neutrino flux is modeled with the Geant4 simulation of the BNB from MiniBooNE
[40, 25]. Neutrino-argon interactions are simulated with the G18_10a_02_11a
configuration of the GENIE v3.0.6 event generator [41] tuned to CC0$\pi$ data
from T2K [42] by reweighting based on two CCQE and CCMEC parameters [43]. The
resulting prediction is referred to as the “MicroBooNE Tune”. Final state
particles are propagated through a detector simulation using the Geant4
toolkit v4_10_3_03c [40] and processed using the LArSoft [44] framework. The
simulated TPC and PMT waveforms are overlaid on beam-off data to provide an
accurate description of cosmic ray activity. These overlaid MC samples are
processed like real data and used to estimate $R$ and $B$.

Uncertainties are estimated with covariance matrices calculated from a multi-
universe approach as in [21]. The total covariance matrix,
$V^{\mathrm{sys}}=V_{\mathrm{flux}}+V_{\mathrm{reint}}+V_{\mathrm{xs}}+V_{\mathrm{det}}+V_{\mathrm{MC}}^{\mathrm{stat}}+V_{\mathrm{dirt}}+V_{\mathrm{POT}}+V_{\mathrm{Target}}+V_{\mathrm{rw}}$,
is given by the sum of the covariance matrices calculated for each systematic
uncertainty described below.

Uncertainty in the neutrino flux [25] is contained in $V_{\mathrm{flux}}$. The
flux contributes 5-10% uncertainty to the cross section results and is often
the dominant systematic for the Np cross section measurements. Neutrino-argon
cross section uncertainties are accounted for in $V_{\mathrm{xs}}$ [43] and
contribute $\sim$5$\%$ uncertainty. In $V_{\mathrm{reint}}$, uncertainties on
reinteractions of final state particles outside the nucleus are accounted for.
These are estimated using the Geant4Reweight [45] package and are relatively
small except at high $K_{p}$ where reinteractions occur for $>75\%$ of
protons. The flux, cross section, and re-interaction uncertainties are
estimated with the multi-sim technique [46].

Detector response uncertainties [47] are contained in $V_{\mathrm{det}}$.
These are the biggest sources of uncertainty for 0p events, typically ranging
from 5-15% compared to $\sim$5% for Np, and are larger at high energies and
backward scattering angles. To evaluate these uncertainties, a detector model
parameter is varied by $1\sigma$ and bootstrapping is used to estimate the
impact of this variation and form $V_{\mathrm{det}}$ [20, 21]. A Gaussian
process regression smoothing algorithm [48, 49, 50] is implemented to prevent
an overestimation of detector systematics due to statistical fluctuations [19,
21].

Flat 50%, 2% and 1% uncertainties on neutrino interactions outside the
detector, POT counting, and the number of target nuclei are contained in
$V_{\mathrm{dirt}}$, $V_{\mathrm{POT}}$ and $V_{\mathrm{Target}}$,
respectively. Their impact on the extracted cross sections is small.

A data-driven model validation procedure is employed to detect mismodeling
that may bias the extracted cross section results. As described in [21], this
relies on the conditional constraint formalism [51] to increase the stringency
of the validation. These constraints leverage correlations between variables
and channels arising from shared physics modeling to update the model
prediction and reduce its uncertainties based on data observations. The
unfolding does not utilize these constraints; they are only employed for model
validation. To be validated, the model is required to describe the data at the
$2\sigma$ level. This is evaluated with $\chi^{2}$ goodness of fit (GoF) tests
interpreted by using the number of degrees of freedom, $ndf$, which
corresponds to the number of bins, to obtain $p$-values.

A kinematic variable that is relatively well understood and reconstructed can
be validated by directly comparing the model prediction to the data in the
phase space relevant to the unfolding. The muon kinematics for events fully
contained (FC) within the detector fit this criteria, and are validated with a
GoF test on the $E_{\mu}^{rec}$ distributions in $\cos\theta_{\mu}^{rec}$
slices used for the cross section extraction. These tests yield $p$-values of
0.45 and 0.98 for the 0p and Np distributions, respectively. The model passes
validation in these tests. For the partially contained (PC) muon kinematics,
the modeling of activity outside the active detector volume, which cannot be
reconstructed, must also be validated. This is done by using the 0p and Np FC
muon kinematics to constrain the PC distributions. These tests, found in the
Supplemental Material of [21], result in $p$-values of 0.84 for 0p and 0.99
for Np. This indicates that the overall model adequately describes the 0p and
Np $E_{\mu}^{rec}$ distributions in $\cos\theta_{\mu}^{rec}$ slices for PC
events.

(a)

(b)

(c)

Figure 1: The unfolded double-differential $\cos\theta_{\mu}$ and $E_{\mu}$
cross section for the $\nu_{\mu}$CC channel. The bins are in angular slices
indicated by the downwards ticks on the x-axis; their edges are
$\\{-1,-0.5,0,0.3,0.5,0.7,0.8,0.9,1\\}$. The bins increase in energy along
each slice. Binning details are in the Supplemental Material. The inclusive Xp
result is shown in (a) and the simultaneously extracted 0p and Np results are
shown in (b) and (c), respectively. The inner (outer) error bars represent the
data statistical (total uncorrelated) uncertainty. Generator predictions
correspond to the colored lines with their $\chi^{2}/ndf$ displayed in the
legend.

When the $K_{p}^{rec}$ distribution is constrained by the 0p and Np muon
kinematics the resulting $p$-value is $3\times 10^{-5}$, indicating that the
model is unable to describe the observed proton kinematics. The discrepancy
appears at low $K_{p}^{rec}$ where modeling becomes challenging due to the
prominence of FSI and nuclear effects. Appropriate uncertainties on
$K_{p}^{rec}$ are important for the $K_{p}$ differential cross section
measurement and the split into 0p and Np subchannels. Thus, this shortcoming
of the model motivates expanding the uncertainties to mitigate the possibility
of extracting biased results. The additional uncertainty was obtained
empirically by estimating a $K_{p}$ distribution through unfolding the
$K_{p}^{rec}$ data using the constrained prediction and only statistical
uncertainties. The ratio of the unfolded distribution to the constrained
signal prediction defines a reweighting function. This is applied to the MC
and treated as a $1\sigma$ deviation from the original prediction due to a
cross section effect allowing $V_{\mathrm{rw}}$ to be calculated identically
to $V_{\mathrm{xs}}$ [21]. Including $V_{\mathrm{rw}}$ reduces the $p$-value
for the $K_{p}^{rec}$ distribution to 0.82 after constraint, enabling the
extraction of the desired cross sections.

The extracted nominal flux-averaged cross sections are compared to event
generator predictions from GENIE v3.0.6 G18_10a_02_11a (GENIE) [41], the
“MicroBooNE tune” GENIE configuration ($\mu\texttt{BooNE}$ tune) [43], NuWro
21.02 (NuWro) [52], GiBUU 2023 (GiBUU) [53], and NEUT 5.4.0.1 (NEUT) [54].
These were processed with NUISANCE [55], do not include theoretical
uncertainties, and are smeared with $A_{C}$ obtained from unfolding. Agreement
between the data and each prediction is quantified by $\chi^{2}/ndf$ values
calculated with uncertainties according to $V_{S}$ and $ndf$ corresponding to
the number of bins.

The double-differential $\cos\theta_{\mu}$ and $E_{\mu}$ cross section results
are shown in Fig. 1 as a function of bin index, which are in angular slices
ranging from backwards on the left to forward on the right and increasing in
energy along each slice. Binning details are in the Supplemental Material. In
all three channels, the generators tend to underpredict the peak of the
$E_{\mu}$ distribution. Figure 1(a) shows the fully inclusive Xp result. The
$\chi^{2}$ values indicate that NEUT best describes this data. The 0p result
is shown in Fig. 1(b). At forward angles, GiBUU describes this result better
than the other generators, which significantly underpredict the cross section.
Agreement at backwards angles is more comparable, but GiBUU still has the
lowest $\chi^{2}/ndf$ over all bins. NEUT, though offering the best
description of the inclusive channel, shows the largest discrepancy with the
0p data. Figure 1(c) shows the Np result. The $\chi^{2}$ values for Np are
comparable between the different generators. Simultaneously extracting the 0p
and Np cross sections allows the results to be examined concurrently with a
$\chi^{2}$ calculated across all bins. These $\chi^{2}/ndf$ values are:
287.5/69 for the $\mu\texttt{BooNE}$ tune, 266.3/69 for GENIE, 263.7/69 for
NuWro, 298.8/69 for NEUT, and 249.8/69 for GiBUU. These indicate that GiBUU
best describes the data when the proton content of the hadronic final state is
examined in more detail.

The differential cross section for the inclusive channel as a function of
$K_{p}$ is shown in Fig. 2. The first bin extends from 0-35 MeV and includes
events without a final state proton; it is equivalent to the 0p signal
definition. The generator predictions diverge at low energies, particularly
for the 0p bin where only GiBUU is able to describe the data. This gives GiBUU
the lowest $\chi^{2}$ despite its underprediction of the data at moderate-to-
high energies. Similar underprediction is seen for GENIE and NuWro in this
region. NEUT and the $\mu\texttt{BooNE}$ tune describe moderate-to-high
energies well, but NEUT also significantly underpredicts the 0p bin,
increasing its $\chi^{2}$.

Comparing the 0p and Np results to the Xp result in Fig. 1 demonstrates how a
model that does well for inclusive scattering may not also be able to describe
the hadronic final state. NEUT stands out in its relatively good description
of Xp but not of 0p due to a significant underprediction of the 0p cross
section. This is possibly attributable to the way NEUT treats binding energy
for nucleon FSI. NEUT assigns nucleons an effective mass when propagating them
through the nucleus and only allows interactions if the total energy is twice
the energy of the free nucleon mass [54]. This reduces the strength of FSI for
low energy nucleons [17] leading to a sharp drop-off in the cross section at
low $K_{p}$ and the low 0p cross section prediction that agrees poorly with
the data.

[70]7(0,0)(0,142)

Figure 2: The unfolded differential cross section for the inclusive
$\nu_{\mu}$CC channel as a function of the leading proton’s kinetic energy.
The dashed line indicates the 35 MeV tracking threshold, below which is a
single bin that includes events without protons. The inset magnifies the last
three bins. See Fig. 1 for style details.

Unlike NEUT, GiBUU describes the data relatively well when the $\nu_{\mu}$CC
channel is split into final states with and without protons. This can possibly
be attributed to its implementation of FSI with a transport model, which
includes a binding potential that treats “target” and “ejected” nucleons
identically. The “ejected” nucleons are propagated on realistic trajectories
through the residual nucleus according to a potential consistent with the
initial interaction [56, 53]. This is absent in other generators, which
propagate “ejected” nucleons on straight lines and decouple the initial
interaction from the FSI. After the initial interaction, an outgoing nucleon
may repeatedly collide with other nucleons, depleting the initial nucleon of
its energy and shifting the $K_{p}$ distribution towards smaller values [17,
57, 58, 59, 56]. It is plausible that the more self-consistent treatment of
FSI in GiBUU better captures this effect. This hypothesis is consistent with
the way GiBUU better describes the 0p final states in Fig. 1(b), and has the
only prediction that mirrors the sharp peak in the data at the lowest $K_{p}$
in Fig. 2. Additional 0p and Np results in [21] and measurements of transverse
kinematic imbalance variables [60] sensitive to FSI modeling in [61, 62]
support this hypothesis.

In summary, we report differential cross-section measurements of the
$\nu_{\mu}$CC channel that probe the phase space of lepton and hadronic
kinematics. An underprediction of the cross section for final states without
protons is observed for all event generator predictions except GiBUU, which
offers a significantly better description of the data possibly due to its more
sophisticated treatment of final state interactions. These measurements
provide new information to stimulate further improvement of models and
generators to match the precision required for future neutrino oscillation
measurements and beyond the Standard Model searches.

###### Acknowledgements.

This document was prepared by the MicroBooNE collaboration using the resources
of the Fermi National Accelerator Laboratory (Fermilab), a U.S. Department of
Energy, Office of Science, HEP User Facility. Fermilab is managed by Fermi
Research Alliance, LLC (FRA), acting under Contract No. DE-AC02-07CH11359.
MicroBooNE is supported by the following: the U.S. Department of Energy,
Office of Science, Offices of High Energy Physics and Nuclear Physics; the
U.S. National Science Foundation; the Swiss National Science Foundation; the
Science and Technology Facilities Council (STFC), part of the United Kingdom
Research and Innovation; the Royal Society (United Kingdom); the UK Research
and Innovation (UKRI) Future Leaders Fellowship; and the NSF AI Institute for
Artificial Intelligence and Fundamental Interactions. Additional support for
the laser calibration system and cosmic ray tagger was provided by the Albert
Einstein Center for Fundamental Physics, Bern, Switzerland. We also
acknowledge the contributions of technical and scientific staff to the design,
construction, and operation of the MicroBooNE detector as well as the
contributions of past collaborators to the development of MicroBooNE analyses,
without whom this work would not have been possible. For the purpose of open
access, the authors have applied a Creative Commons Attribution (CC BY) public
copyright license to any Author Accepted Manuscript version arising from this
submission.

## References

  * Tanabashi  _et al._ [2018] M. Tanabashi _et al._ (Particle Data Group), Review of Particle Physics, Phys. Rev. D 98, 030001 (2018).
  * Qian and Vogel [2015] X. Qian and P. Vogel, Neutrino Mass Hierarchy, Prog. Part. Nucl. Phys. 83, 1 (2015), arXiv:1505.01891 [hep-ex] .
  * Acero  _et al._ [2022] M. A. Acero _et al._ (NO$\nu$A Collaboration), Improved measurement of neutrino oscillation parameters by the NO$\nu$A experiment, Phys. Rev. D 106, 032004 (2022), arXiv:2108.08219 [hep-ex] .
  * Abe  _et al._ [2021] K. Abe _et al._ (T2K Collaboration), Improved constraints on neutrino mixing from the T2K experiment with $3.13\times 10^{21}$ protons on target, Phys. Rev. D 103, 112008 (2021), arXiv:2101.03779 [hep-ex] .
  * Abi  _et al._ [2020] B. Abi _et al._ (DUNE Collaboration), Long-baseline neutrino oscillation physics potential of the DUNE experiment, Eur. Phys. J. C 80, 978 (2020), arXiv:2006.16043 [hep-ex] .
  * Acciarri  _et al._ [2015] R. Acciarri _et al._ , A Proposal for a Three Detector Short-Baseline Neutrino Oscillation Program in the Fermilab Booster Neutrino Beam (2015), arXiv:1503.01520 [physics.ins-det] .
  * Rubbia [1977] C. Rubbia,  _The liquid-argon time projection chamber: a new concept for neutrino detectors_, Tech. Rep. CERN-EP-INT-77-8 (1977).
  * Chen  _et al._ [1976] H. H. Chen _et al._ ,  _A Neutrino detector sensitive to rare processes. I. A Study of neutrino electron reactions_ , Tech. Rep. FERMILAB-PROPOSAL-0496 (1976).
  * Willis and Radeka [1974] W. Willis and V. Radeka, Liquid-argon ionization chambers as total-absorption detectors, Nucl. Instrum. Meth. A 120, 221 (1974).
  * Nygren [1974] D. R. Nygren,  _The Time Projection Chamber: A New 4 $\pi$ Detector for Charged Particles_, Tech. Rep. PEP-0144 (SLAC, 1974).
  * Cavanna  _et al._ [2018] F. Cavanna, A. Ereditato, and B. T. Fleming, Advances in liquid argon detectors, Nucl. Instrum. Meth. A 907, 1 (2018).
  * Alvarez-Ruso  _et al._ [2018] L. Alvarez-Ruso _et al._ , NuSTEC White Paper: Status and challenges of neutrino–nucleus scattering, Prog. Part. Nucl. Phys. 100, 1 (2018).
  * Nagu  _et al._ [2020] S. Nagu _et al._ , Impact of cross-sectional uncertainties on DUNE sensitivity due to nuclear effects, Nucl. Phys. B 951, 114888 (2020).
  * Formaggio and Zeller [2012] J. A. Formaggio and G. P. Zeller, From eV to EeV: Neutrino cross sections across energy scales, Rev. Mod. Phys 84, 1307 (2012).
  * Nieves  _et al._ [2004] J. Nieves, J. E. Amaro, and M. Valverde, Inclusive quasielastic charged-current neutrino-nucleus reactions, Phys. Rev. C 70, 055503 (2004).
  * Van Orden and Donnelly [2019] J. W. Van Orden and T. W. Donnelly, Nuclear theory and event generators for charge-changing neutrino reactions, Phys. Rev. C 100, 044620 (2019).
  * Dytman  _et al._ [2021] S. Dytman _et al._ , Comparison of validation methods of simulations for final state interactions in hadron production experiments, Phys. Rev. D 104, 053006 (2021), arXiv:2103.07535 [hep-ph] .
  * Abratenko  _et al._ [2022a] P. Abratenko _et al._ (MicroBooNE Collaboration), First Measurement of Energy-Dependent Inclusive Muon Neutrino Charged-Current Cross Sections on Argon with the MicroBooNE Detector, Phys. Rev. Lett. 128, 151801 (2022a).
  * Abratenko  _et al._ [2023a] P. Abratenko _et al._ (MicroBooNE Collaboration), Measurement of triple-differential inclusive muon-neutrino charged-current cross section on argon with the MicroBooNE detector (2023a), arXiv:2307.06413 [hep-ex] .
  * Abratenko  _et al._ [2022b] P. Abratenko _et al._ (MicroBooNE Collaboration), Search for an anomalous excess of inclusive charged-current ${\nu}_{e}$ interactions in the MicroBooNE experiment using Wire-Cell reconstruction, Phys. Rev. D 105, 112005 (2022b).
  * Abratenko  _et al._ [2024] P. Abratenko _et al._ (MicroBooNE Collaboration), Inclusive cross section measurements in final states with and without protons for charged-current $\nu_{\mu}$-ar scattering in MicroBooNE, arXiv preprint (2024), arXiv:XXXXXXXX [hep-ex] .
  * Abratenko  _et al._ [2022c] P. Abratenko _et al._ (MicroBooNE Collaboration), Search for an anomalous excess of charged-current ${\nu}_{e}$ interactions without pions in the final state with the MicroBooNE experiment, Phys. Rev. D 105, 112004 (2022c).
  * Abratenko  _et al._ [2022d] P. Abratenko _et al._ (MicroBooNE Collaboration), Differential cross section measurement of charged current ${\nu}_{e}$ interactions without final-state pions in MicroBooNE, Phys. Rev. D 106, L051102 (2022d).
  * Acciarri  _et al._ [2017a] R. Acciarri _et al._ (MicroBooNE Collaboration), Design and construction of the MicroBooNE detector, JINST 12 (02), P02017.
  * Aguilar-Arevalo  _et al._ [2009] A. A. Aguilar-Arevalo _et al._ (MiniBooNE Collaboration), Neutrino flux prediction at MiniBooNE, Phys. Rev. D 79, 072002 (2009).
  * Qian  _et al._ [2018] X. Qian _et al._ , Three-dimensional Imaging for Large LArTPCs, JINST 13 (05), P05032, arXiv:1803.04850 [physics.ins-det] .
  * Acciarri  _et al._ [2017b] R. Acciarri _et al._ (MicroBooNE Collaboration), Noise Characterization and Filtering in the MicroBooNE Liquid Argon TPC, JINST 12 (08), P08003, arXiv:1705.07341 [physics.ins-det] .
  * Adams  _et al._ [2018a] C. Adams _et al._ (MicroBooNE Collaboration), Ionization electron signal processing in single phase LArTPCs. Part I. Algorithm Description and quantitative evaluation with MicroBooNE simulation, JINST 13 (07), P07006, arXiv:1802.08709 [physics.ins-det] .
  * Adams  _et al._ [2018b] C. Adams _et al._ (MicroBooNE Collaboration), Ionization electron signal processing in single phase LArTPCs. Part II. Data/simulation comparison and performance in MicroBooNE, JINST 13 (07), P07007, arXiv:1804.02583 [physics.ins-det] .
  * Abratenko  _et al._ [2022e] P. Abratenko _et al._ (MicroBooNE Collaboration), Wire-cell 3D pattern recognition techniques for neutrino event reconstruction in large LArTPCs: algorithm description and quantitative evaluation with MicroBooNE simulation, JINST 17 (01), P01037.
  * Abratenko  _et al._ [2021a] P. Abratenko _et al._ (MicroBooNE Collaboration), Neutrino event selection in the MicroBooNE liquid argon time projection chamber using Wire-Cell 3D imaging, clustering, and charge-light matching, JINST 16 (06), P06043.
  * Graham  _et al._ [2018] B. Graham, M. Engelcke, and L. van der Maaten, 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks, in  _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (2018) pp. 9224–9232.
  * Smirnov [1939] N. V. Smirnov, Estimate of deviation between empirical distribution functions in two independent samples, Bulletin Moscow University 2, 3 (1939).
  * [34] PSTAR at NIST: https://physics.nist.gov/PhysRefData/Star/Text/PSTAR.html.
  * Adams  _et al._ [2020] C. Adams _et al._ (MicroBooNE Collaboration), Calibration of the charge and energy loss per unit length of the MicroBooNE liquid argon time projection chamber using muons and protons, JINST 15 (03), P03022.
  * Abratenko  _et al._ [2021b] P. Abratenko _et al._ (MicroBooNE Collaboration), Cosmic Ray Background Rejection with Wire-Cell LArTPC Event Reconstruction in the MicroBooNE Detector, Phys. Rev. Appl. 15, 064071 (2021b).
  * Tang  _et al._ [2017] W. Tang _et al._ , Data Unfolding with Wiener-SVD Method, JINST 12 (10), P10002, arXiv:1705.03568 [physics.data-an] .
  * Koch and Dolan [2020] L. Koch and S. Dolan, Treatment of flux shape uncertainties in unfolded, flux-averaged neutrino cross-section measurements, Phys. Rev. D 102, 113012 (2020).
  * Gardiner [2024] S. Gardiner, Mathematical methods for neutrino cross-section extraction, arXiv preprint (2024), arXiv:2401.04065 [hep-ex] .
  * Agostinelli  _et al._ [2003] S. Agostinelli _et al._ , Geant4—a simulation toolkit, Nucl. Instrum. Meth. A 506, 250 (2003).
  * Alvarez-Ruso  _et al._ [2021] L. Alvarez-Ruso _et al._ (GENIE Collaboration), Recent highlights from GENIE v3, Eur. Phys. J. ST 230, 4449 (2021).
  * Abe  _et al._ [2016] K. Abe _et al._ (T2K Collaboration), Measurement of double-differential muon neutrino charged-current interactions on C8H8 without pions in the final state using the T2K off-axis beam, Phys. Rev. D 93, 112012 (2016).
  * Abratenko  _et al._ [2022f] P. Abratenko _et al._ (MicroBooNE Collaboration), New $\mathrm{CC}0\pi$ GENIE model tune for MicroBooNE, Phys. Rev. D 105, 072001 (2022f).
  * Snider and Petrillo [2017] E. Snider and G. Petrillo, LArSoft: toolkit for simulation, reconstruction and analysis of liquid argon TPC neutrino detectors, J. Phys. Conf. Ser 898, 042057 (2017).
  * Calcutt  _et al._ [2021] J. Calcutt, C. Thorpe, K. Mahn, and L. Fields, Geant4Reweight: a framework for evaluating and propagating hadronic interaction uncertainties in Geant4, JINST 16, P08042 (2021).
  * Roe [2007] B. P. Roe, Statistical errors in monte carlo estimates of systematic errors, Nucl. Instrum. Meth. A 570, 159 (2007).
  * Abratenko  _et al._ [2022g] P. Abratenko _et al._ (MicroBooNE Collaboration), Novel approach for evaluating detector-related uncertainties in a LArTPC using MicroBooNE data, Eur. Phys. J. C 82, 454 (2022g).
  * Frate  _et al._ [2017] M. Frate _et al._ , Modeling Smooth Backgrounds and Generic Localized Signals with Gaussian Processes (2017), arXiv:1709.05681 [physics.data-an] .
  * Li  _et al._ [2020] L. Li, N. Nayak, J. Bian, and P. Baldi, Efficient neutrino oscillation parameter inference using Gaussian processes, Phys. Rev. D 101, 012001 (2020).
  * Rasmussen and Williams [2005] C. E. Rasmussen and C. K. I. Williams,  _Gaussian Processes for Machine Learning_ (The MIT Press, 2005).
  * Eaton [1983] M. L. Eaton,  _Multivariate Statistics: a Vector Space Approach_ (John Wiley and Sons, 1983) pp. 116–117.
  * Golan  _et al._ [2012] T. Golan, J. Sobczyk, and J. Żmuda, NuWro: the Wrocław Monte Carlo Generator of Neutrino Interactions, Nucl. Phys. B Proc. Suppl. 229-232, 499 (2012).
  * Buss  _et al._ [2012] O. Buss _et al._ , Transport-theoretical description of nuclear reactions, Phys. Rep. 512, 1 (2012).
  * Hayato and Pickering [2021] Y. Hayato and L. Pickering, The NEUT neutrino interaction simulation program library, Eur. Phys. J. ST 230, 4469 (2021).
  * Stowell  _et al._ [2017] P. Stowell _et al._ , NUISANCE: a neutrino cross-section generator tuning and comparison framework, JINST 12, P01016 (2017).
  * Mosel [2019] U. Mosel, Neutrino event generators: foundation, status and future, J. Phys. G 46, 113001 (2019).
  * Mosel  _et al._ [2014] U. Mosel, O. Lalakulich, and K. Gallmeister, Reaction mechanisms at $\mathrm{MINER}\nu\mathrm{A}$, Phys. Rev. D 89, 093003 (2014).
  * Niewczas and Sobczyk [2019] K. Niewczas and J. T. Sobczyk, Nuclear transparency in Monte Carlo neutrino event generators, Phys. Rev. C 100, 015505 (2019).
  * Lalakulich  _et al._ [2012] O. Lalakulich, K. Gallmeister, and U. Mosel, Many-body interactions of neutrinos with nuclei: Observables, Phys. Rev. C 86, 014614 (2012).
  * Lu and other [2016] X. G. Lu and other, Measurement of nuclear effects in neutrino interactions with minimal dependence on neutrino energy, Phys. Rev. C 94, 015503 (2016), arXiv:1512.05748 [nucl-th] .
  * Abratenko  _et al._ [2023b] P. Abratenko _et al._ (MicroBooNE Collaboration), Multidifferential cross section measurements of ${\nu}_{\mu}$-argon quasielasticlike reactions with the MicroBooNE detector, Phys. Rev. D 108, 053002 (2023b).
  * Abratenko  _et al._ [2023c] P. Abratenko _et al._ , First Double-Differential Measurement of Kinematic Imbalance in Neutrino Interactions with the MicroBooNE Detector, Phys. Rev. Lett. 131, 101802 (2023c).


# RLSF: Reinforcement Learning via
Symbolic Feedback

David S. Hippocampus
Department of Computer Science
Cranberry-Lemon University
Pittsburgh, PA 15213
<EMAIL_ADDRESS>
Use footnote for providing further information about author (webpage,
alternative address)— _not_ for acknowledging funding agencies. Piyush Jha
Georgia Tech
<EMAIL_ADDRESS>
&Prithwish Jana
Georgia Tech
<EMAIL_ADDRESS>
&Arnav Arora
Georgia Tech
<EMAIL_ADDRESS>
&Vijay Ganesh
Georgia Tech
<EMAIL_ADDRESS>

###### Abstract

In recent years, large language models (LLMs) have had a dramatic impact on
various sub-fields of AI, most notably on natural language understanding
tasks. However, there is widespread agreement that the logical reasoning
capabilities of contemporary LLMs are, at best fragmentary (i.e., may work
well on some problem instances but fail dramatically on others). While
traditional LLM fine-tuning approaches (e.g., those that use human feedback)
do address this problem to some degree, they suffer from many issues,
including unsound black-box reward models, difficulties in collecting
preference data, and sparse scalar reward values.

To address these challenges, we propose a new training/fine-tuning paradigm we
refer to as Reinforcement Learning via Symbolic Feedback (RLSF), which is
aimed at enhancing the reasoning capabilities of LLMs. In the RLSF setting,
the LLM that is being trained/fine-tuned is considered as the RL agent, while
the environment is allowed access to reasoning or domain knowledge tools
(e.g., solvers, algebra systems). Crucially, in RLSF, these reasoning tools
can provide feedback to the LLMs via poly-sized certificates (e.g., proofs),
that characterize errors in the LLM-generated object with respect to some
correctness specification. The ability of RLSF-based training/fine-tuning to
leverage certificate-generating symbolic tools enables sound fine-grained
(token-level) reward signals to LLMs, and thus addresses the limitations of
traditional reward models mentioned above.

Via extensive evaluations, we show that our RLSF-based fine-tuning of LLMs
outperforms traditional approaches on two different applications, namely,
program synthesis from natural language pseudo-code to programming language
(C++), and solving the Game of 24. For the former, RLSF-tuned LLMs show a
significant improvement (e.g., +52.64% in compilation accuracy and +31.43% in
functional correctness for Google’s CodeGemma-2b) compared to supervised fine-
tuning. Further, CodeGemma-2b gets +34.82% in compilation accuracy and +17.01%
in functional correctness compared to ChatGPT (100$\boldsymbol{\times}$
larger). Similarly, for the Game of 24, we observed +25% success rate using
Meta’s Llama2-7b compared to traditional methods, and +7% success rate
compared to ChatGPT (25$\boldsymbol{\times}$ larger). A takeaway is that fine-
tuning via RLSF enables relatively smaller LLMs to significantly outperform
orders of magnitude larger models (e.g., ChatGPT).

##  1 Introduction

In recent years, Large Language Models (LLMs) have had a dramatic impact on
many sub-fields of AI [bommasani2021opportunities]. Tasks that seemed
impossible just a few years ago, are now routinely solved by LLMs. Examples
include language translation [qin2024multilingual, min2023recent], text-to-
image generation [zhang2023text], coding assistants [liang2024large], and
open-ended text generation [achiam2023gpt].

Despite their impressive performance, these models often struggle with tasks
that require complex reasoning [mao2024champ, stechly2024self,
zhang2022paradox]. This limitation has sparked a growing interest in exploring
methods that can enhance the reasoning capabilities of LLMs, particularly
through the incorporation of corrective feedback loops between learning and
reasoning processes [ganesh2022machine, kambhampati2024llms]. For example,
Kambhampati et al. [kambhampati2024llms] state that "LLMs cannot plan
themselves but can play a variety of constructive roles in solving planning
tasks–especially as approximate knowledge sources and candidate plan
generators in the so-called LLM-Modulo Frameworks in conjunction with external
sound model-based verifiers."

Figure 1: Contrasting RLHF with RLSF: The image depicts two distinct fine-
tuning paradigms. (Left) RLHF operates within an environment governed by a
black-box reward model, typically offering scalar feedback. (Right) By
contrast, the environment in RLSF leverages sound symbolic reasoning tools and
also provides fine-grained token-level vector feedback that is, in turn, based
on poly-sized certificates produced by these symbolic tools.

The concept of incorporating reasoning tools into machine learning in general,
and LLMs in particular, is rooted in the idea of combining the strengths of
these two sub-fields of AI [hitzler2023compendium, ganesh2022machine,
kambhampati2024llms, bouraoui2019shallow]. While LLMs excel at capturing
statistical patterns and generating fluent text, they can fail to perform
sound reasoning and generate text that is logically coherent. In fact, the
logical or code errors introduced by LLMs in the objects they generate can be
very subtle, and not immediately obvious upon inspection. This motivates the
need to use reasoning and verification tools at different stages of an LLM’s
life cycle (from data curation, training, fine-tuning, and inference). For
instance, using program analysis tools during inference [agrawal2024monitor],
and integrating solvers into neural network layers [wang2024grounding] or
during gradient descent [ashok2022solver+] has shown promising results in
terms of faster convergence and efficient data utilization.

By contrast to LLMs, symbolic reasoning systems, such as theorem provers and
constraint solvers 111We define the term symbolic reasoning systems broadly to
include solvers, provers, computer algebra systems, program analysis tools,
knowledge bases, and simulators. The only requirement is that they
analyze/solve inputs that are formally defined, and can produce poly-sized
certificates of their analysis., are adept at handling sound logical
reasoning, perform symbolic mathematical operations and maintain coherence,
but they do not seem to possess the impressive generative capabilities of
LLMs. By integrating these two approaches, a hybrid system can be created that
can leverage the strengths of both paradigms, potentially leading to more
robust and capable AI systems.

One popular approach to fine-tuning LLMs is Reinforcement Learning from Human
Feedback (RLHF) [ouyang2022training, stiennon2020learning], which relies on
manually collecting correct/incorrect cases and creating an approximate
(possibly unsound) black-box reward model. Such an approach can be expensive,
error-prone, and may not accurately capture the nuances of a reasoning task.
Moreover, the reward signal thus generated can be sparse and scalar in nature.
Such sparse reward signals can fall short of fully capturing those aspects of
an object generated by an LLM that may be incorrect with respect to a well-
defined specification (or inconsistent with domain knowledge).

To address these challenges, we propose a new training/fine-tuning paradigm we
refer to as Reinforcement Learning via Symbolic Feedback (RLSF) that is aimed
at enhancing the reasoning capabilities of LLMs as well as making them
compliant with domain knowledge. Figure 1 highlights the differences between
RLHF and RLSF. In the RLSF setting, the LLM is considered as the RL agent to
be trained/fine-tuned, while the environment is allowed access to reasoning
tools, that can generate poly-sized certificates of their analyses.

In the forward direction (LLM to environment), formal objects generated (such
as programs, proofs, circuits, molecules, or theories) by the LLM are fed to
the RLSF environment, which leverages reasoning tools to perform analysis or
verification of such objects against specifications or domain knowledge. In
the reverse direction (environment to LLM), any certificate (that identifies
what is wrong with the LLM-generated object) produced by symbolic tools is
used as part of a reward function to provide corrective feedback to the LLM.
Leveraging symbolic tools to generate poly-sized certificates 222Examples of
poly-sized certificates include unsatisfiability proofs, compiler feedback,
equivalence testing, etc. Our approach is not limited to any one type of
symbolic tool, as long as these certificates can be converted into appropriate
rewards. It is possible that the reasoning problem addressed by these symbolic
tools is NP-hard, and therefore, in general, we cannot always expect
certificates that are polynomial in the input size. Having said that, many of
these tools, such as compilers, computer algebra systems, or solvers, work
well in practice., which provide sound, fine-grained (token-level) feedback to
LLMs, eliminates the need for manual preference data collection and addresses
limitations of traditional reward models.

Contributions. In this paper, we present the RLSF paradigm and evaluate the
effectiveness of RLSF-based fine-tuning methodologies on LLMs across two
distinct application scenarios that present unique challenges to their
reasoning capabilities. Our evaluation encompasses comprehensive experiments
conducted in the domains of program synthesis from natural language pseudo-
code to a programming language (C++) and solving the Game of 24 (a
standardized benchmark to test the logical reasoning capabilities of LLMs).
Through our extensive experiments, we demonstrate the efficacy of RLSF in
significantly enhancing the reasoning capabilities of LLMs across these
diverse tasks.

Our results show a significant improvement over the supervised fine-tuning
(SFT) approach for widely-used code LLMs (such as stable-code-instruct-3b
[pinnaparaju2024stable], deepseek-coder-1.3b-base [guo2024deepseek] and
Google’s code-gemma-2b [google2024codegemma]) in the program synthesis task,
against zero-shot and two traditional fine-tuning methods. For example, our
RLSF-tuned code-gemma-2b shows an improvement of +52.64% in compilation
accuracy (CompAcc) and +31.43% in functional correctness accuracy (FCorrAcc)
over SFT. Further, the RLSF-tuned code-gemma-2b achieve superior results
(+34.82% in CompAcc and +17.01% in FCorrAcc) compared to ChatGPT (100x larger)
[achiam2023gpt]. For the Game of 24, we perform an extensive empirical
evaluation of RLSF against different prompting and traditional fine-tuning
methods. We observe significant improvement using RLSF on popular LLMs –
Google’s Gemma-2b-it [team2024gemma] and Meta’s Llama2-7b-chat
[touvron2023llama] with +15% and +25% success respectively, compared to
traditional methods. Notably, post RLSF fine-tuning, Llama2-7b-chat also
outperforms (+7%) ChatGPT ($\sim$25$\boldsymbol{\times}$ larger). This
underscores the importance of RLSF fine-tuning that facilitates relatively
smaller LLMs to significantly outperform models, such as ChatGPT, which are
orders of magnitude larger.

##  2 Related Work

Neurosymbolic Reinforcement Learning (NRL): We refer the reader to the
Compendium of Neurosymbolic AI for a rich and recent literature survey on
combinations of learning and reasoning systems [hitzler2023compendium]. There
has been considerable work in NRL [acharya2023neurosymbolic]. However, all the
NRL work that we are aware of focuses on combining symbolic reasoning tools
with the RL agent, as opposed to our RLSF approach, where we combine the
symbolic tools with the environment. Further, in the RLSF paradigm, a key
innovation is that we leverage the poly-sized certificates generated (which
characterize why the object generated by the RL agent is inconsistent with the
specification or domain knowledge) by the environment’s symbolic tools to
provide a fine-grained reward signal to the RL agent. This allows for a
modular framework with more expressive and interpretable feedback during the
training process.

By contrast, Garnelo et al. [garnelo2016towards] uses a typical NRL approach
that advocates generating an embedding using neural networks and then learning
a mapping from the embedding to a symbolic representation. However, they do
not use the symbolic methods as part of the environment to generate
certificates and use them to create appropriate reward signals. Further, these
methods need to modify the underlying agent and are limited by the agent’s
capability for handling large symbolic representations.

Combinations of LLMs with Symbolic Tools: Previous research has explored the
integration of program analysis tools during inference [agrawal2024monitor].
While existing efforts have often relied on binary or scalar signals for
utilizing symbolic feedback during training or fine-tuning [chen2020program,
jana2023attention, jha2024bertrlfuzzer, zou2024fgeo], the RLSF paradigm
leverages richer token-level (vector) symbolic feedback, thus enabling
significantly more effective reasoning capabilities. Kambhampati et al.
[kambhampati2024llms] introduced the concept of LLM-Modulo Frameworks, which
proposes the utilization of external sound symbolic reasoning tools. We
concretize their philosophical position via the RLSF paradigm.

LLM-based Feedback: Another line of work that has explored using LLMs as part
of a corrective feedback loop [shinn2024reflexion, madaan2024self,
paul2023refiner, kim2024language, chen2023teaching] where the feedback is
usually provided as in-context (not RL), under the assumption that LLMs are
better at verification compared to generation. However, recent studies have
challenged this assumption, suggesting that LLMs may not be as effective at
verification/self-critiquing tasks as previously thought [mao2024champ,
stechly2024self, valmeekam2023can, huang2023large, stechly2023gpt]. By
contrast, in our work, we use sound symbolic systems to provide corrective
feedback and do so via an RL approach.

##  3 Reinforcement Learning via Symbolic Feedback (RLSF)

Algorithm 1 Reinforcement Learning via Symbolic Feedback (RLSF)

Number of epochs $N_{epochs}$, pre-trained model $Model$, symbolic reasoning
tool $SymbolicReasoner$, reward function $RewardFunc$, prompt dataset $D$

Fine-tuned model $Model^{\prime}$

for $N_{epochs}$ do

for $batch_{i}$ in $D$ do

$response_{i}\sim Model(batch_{i})$ $\triangleright$ Generate a response

$cert_{i}\leftarrow SymbolicReasoner(batch_{i},response_{i})$ $\triangleright$
Compute the certificate

$vector_{i}\leftarrow RewardFunc(cert_{i})$ $\triangleright$ Compute the
vector feedback

$Model^{\prime}\leftarrow ppo\\_step(Model,batch_{i},response_{i},cert_{i})$
$\triangleright$ Update model using PPO

$Model\leftarrow Model^{\prime}$

end for

end for

In this section, we introduce the Reinforcement Learning via Symbolic Feedback
(RLSF) algorithm. The algorithm incorporates fine-grained, token-level
feedback generated by reasoning or domain knowledge tools, thereby addressing
the limitations of traditional reward-based methods.

###  3.1 Algorithm Description

The RLSF algorithm, outlined in Algorithm 1, fine-tunes a pre-trained language
model $Model$ using reinforcement learning (RL) with the help of the fine-
grained certificate provided by a symbolic reasoning tool $SymbolicReasoner$.
The framework leverages a reward function ($RewardFunc$) to compute vector
feedback ($vector_{i}$) based on the certificate generated by the symbolic
reasoning tool. This feedback aligns with the shape of the response generated
by the language model, facilitating fine-grained adjustments during fine-
tuning. The algorithm operates over a specified number of epochs $N_{epochs}$,
iterating through a dataset $D$.

Inputs and Outputs. The algorithm takes as input the pre-trained model (RL
agent to be fine-tuned) $Model$, the symbolic reasoning tool
$SymbolicReasoner$ and the reward function $RewardFunc$ (to be used as the RL
environment), and the dataset $D$ that consists of input prompts for $Model$.
The output is a fine-tuned model $Model^{\prime}$. The algorithm performs the
following steps:

Epoch Iteration: For each epoch from $1$ to $N_{epochs}$, the algorithm
iterates through $D$.

Batch Processing: For each batch $batch_{i}$ in the dataset $D$, the algorithm
performs the following:

  * •

Response generation ($response_{i}$): The $Model$ generates a response
$response_{i}$ given the input prompt $batch_{i}$.

  * •

Certificate computation ($cert_{i}$): The symbolic reasoning tool
$SymbolicReasoner$ computes a certificate $cert_{i}$ corresponding to the
response $response_{i}$.

  * •

Vector feedback computation ($vector_{i}$): The reward function ($RewardFunc$)
calculates a vector feedback ($vector_{i}$) based on the certificate
$cert_{i}$. This certificate typically includes fine-grained error messages or
other relevant information extracted from the symbolic analysis of the prompt-
response pair. The reward function processes this certificate to generate
token-level vector feedback. This token-level feedback provides detailed
guidance to the language model during fine-tuning, facilitating precise
adjustments to its logical reasoning capabilities. The vector feedback has the
same shape as the response tokens (computed using the tokenizer provided by
$Model$). Figures 2 and 3 give a concrete example of such a vector feedback
for the two tasks.

  * •

Model update ($Model^{\prime}$): The $Model$ is updated to $Model^{\prime}$
using the Proximal Policy Optimization (PPO) algorithm, using the input prompt
$batch_{i}$, response $response_{i}$, and certificate $cert_{i}$.

###  3.2 Implementation, TRL Library, and Hardware Details

The RLSF algorithm is implemented using a modified version of the Transformer
Reinforcement Learning (TRL) library [vonwerra2022trl], a comprehensive
framework integrated with Huggingface transformers [wolf-
etal-2020-transformers] designed for training transformer language models with
reinforcement learning (RL). We use the Proximal Policy Optimization (PPO)
algorithm, commonly used in Reinforcement Learning from Human Feedback (RLHF)
[yang2024harnessing, ouyang2022training, stiennon2020learning], for fine-
tuning the language model. However, TRL only allows for scalar reward signals
during the RL fine-tuning process. We modify the library to support vector
reward signals, allowing fine-grained feedback at the token level. This
enhancement enables RLSF to leverage symbolic feedback effectively during the
RL process. All our experiments were conducted on a high-performance CentOS V7
cluster equipped with Intel E5-2683 v4 Broadwell processors running at 2.10
GHz and 64 GiB of memory. We used 4 NVIDIA V100 GPUs for the task in Section 4
and 1 NVIDIA V100 GPU for the task in Section LABEL:subsec:g24.

Figure 2: RLSF for translation of NL pseudo-code to code: In this setting, the
symbolic environment utilizes the g++ compiler to check whether the generated
C++ code compiles or not. If it compiles, the vectorized feedback has the
value $p=1+r$ (higher reward indicating the positive case) corresponding to
every token. Otherwise, a reward value of $n=0$ (lower reward indicating the
negative case) is returned for every token representing an erroneous line, and
$p$ for all other tokens. The special tokens <BOS> and <EOS> are both $p$ for
the first case, and both $n$ for the second. Figure 3: RLSF for the Game of
24: In this illustration, the symbolic environment utilizes the Computer
Algebra System (CAS) library SymPy [sympy] to generate a vectorized reward as
feedback. Each element in the vector corresponds to a token in the response,
where erroneous tokens are penalized with a value of 0 and correct ones are
assigned 1. The last element of the vector reward (corresponding to the <EOS>
token) is 1 only if the entire response is correct, otherwise, it is 0.

##  4 Reasoning Task A: Natural Language Pseudo-code to Code (C++)

Automated code synthesis from natural language (NL) descriptions is a crucial
task in software engineering. It has garnered significant attention over the
years, and recently, researchers [poesia2022synchromesh, ugare2024improving]
have been attempting to address it using LLMs. Motivated by this, we evaluate
how the RLSF paradigm enhances LLM-based NL to programming language (PL)
generation. Specifically, we focus on translating a pseudo-code expressed in
NL to C++ with the following properties: the generated code must be (a)
syntactically correct w.r.t. g++ compiler and (b) functionally correct (or
equivalent to) the pseudo-code w.r.t a test suite 333Note that in general the
task of determining whether a C++ code is functionally equivalent to pseudo-
code (even when it is specified in formal logic) is undecidable. Hence, we
consider a weaker functional equivalence property, namely, that a generated
C++ code is deemed functionally correct if it passes all test cases
corresponding to a given pseudo-code..

###  4.1 Benchmark and RLSF setup

We evaluate using the SPoC dataset [kulal2019spoc], which comprises a
collection of contest problems from the competitive programming website
Codeforces [codeforces]. Kulal et al. [kulal2019spoc] gathered 16,326 C++
programs that were accepted as successful solutions to 677 problems hosted on
Codeforces. Each problem is accompanied by a test suite containing multiple
test cases. Additionally, they employed 59 crowd-workers from Amazon
Mechanical Turk to write NL pseudo-code corresponding to each line of the C++
code, ensuring a line-level granularity of text description. Each program
comprises an average of 14.7 lines and the number of lines ranges in
$[1,457]$. In this paper, we use the TestP partition of the SPoC dataset,
which set aside 158 out of 677 problems ($\sim$23.34%) for testing, resulting
in 1,778 pairs of pseudo-code and C++ code. The remaining data, consisting of
14,548 pairs across 519 problems ($\sim$76.66%), was used to train the LLMs.

Please refer to Figure 2 for an illustration of the RLSF setup used in code
generation from pseudo-code. Let us assume that in the supervised learning
setup, we have a training dataset
$\mathcal{D}=\left\\{\left({pc}_{i},c_{i},{TS}_{i}\right)\right\\}$ with
triplets of pseudo-code ${pc}_{i}$, gold-standard code $c_{i}$, and a test-
suite $TS_{i}$. In each training step, the LLM is first provided with the
prompt shown in Figure 2. The LLM responds by generating code
$\widehat{c_{i}}$, which may or may not be syntactically correct. This code is
then passed through the target language compiler (in this case, g++), which
identifies the set of line numbers ($E$) responsible for any syntactic errors.
Further, if $\widehat{c_{i}}$ compiles, it is run on ${TS}_{i}$. Accordingly,
the pass rate ($r\in[0,1]$) is calculated which represents the fraction of
test cases in ${TS}_{i}$ exhibiting the expected input-output behavior. Note
that, $r=0$ if $\widehat{c_{i}}$ does not compile. Using the vectorized
feedback, we aim to assign a high scalar reward $p$ (indicating a positive
case) to any token corresponding to a line in $\widehat{c_{i}}$ that is free
of syntactic errors. Conversely, we want to assign a low scalar reward $n$
(indicating a negative case) to any token corresponding to a line in
$\widehat{c_{i}}$ containing a syntactic error. Considering $p=1+r$ and $n=0$,
the vectorized feedback is thereby represented as:

$\begin{split}&RewardFunc\left(SymbolicReasoner\left(\left({pc}_{i},c_{i},{TS}_{i}\right),\texttt{<BOS>}\widehat{c_{i}}\texttt{<EOS>}\right)\right)\\\
&=\begin{cases}\mathbf{p}_{\left|\widehat{c_{i}}\right|+2}\;\;\text{i.e.}\;\;\left[p,p,\ldots,p\right]\in\mathbb{R}^{\left|\widehat{c_{i}}\right|+2}&\text{,
if $\widehat{c_{i}}$ compiles}\\\
\left[n\right]\mathbin{\|}\left[\text{rwd}(x)\middle|\;\forall
x\in\widehat{c_{i}},\text{rwd}(x)\leftarrow n\;\text{if $\text{line}(x)\in E$,
else $\text{rwd}(x)\leftarrow p$}\right]\mathbin{\|}\left[n\right]&\text{,
otherwise}\end{cases}\end{split}$ (1)

where $\left|\widehat{c_{i}}\right|$ denotes the number of tokens in
$\widehat{c_{i}}$, $x$ represents a token in $\widehat{c_{i}}$, and
$\mathbin{\|}$ indicates concatenation. Note that, the token-wise reward is
$0$ for tokens in syntactically erroneous lines, $1$ for lines that compile in
otherwise incorrect code, and between $[1,2]$ for lines in syntactically
correct code. The rewards for <BOS> and <EOS> tokens are $p$ when
$\widehat{c_{i}}$ compiles and $n$ otherwise. This vectorized symbolic
feedback refines the LLM through Proximal Policy Optimization (Algorithm 1).

###  4.2 Evaluation Metrics

Compilation Accuracy (CompAcc). The percentage of generated C++ codes that are
syntactically correct, indicating that it compiles without errors using a g++
compiler.

Functional Correctness Accuracy (FCorrAcc). The percentage of test cases
demonstrating the expected input-output behavior for each generated C++ code,
averaged across all the generated codes. If a generated code contains
syntactical errors, FCorrAcc is zero for that code.

Note that, the codes in the SPoC dataset do not contain any ‘#include’
preprocessor directives or ‘using namespace’ lines. As a result, the LLMs are
not trained to generate these directives in their output. To address this, as
a post-processing step, we prepend each code with a fixed and sufficiently
comprehensive set of 21 preprocessor directives, followed by a ‘using
namespace std;’ line. This is done before calculating CompAcc, FCorrAcc, and
the vectorized feedback, to ensure that an LLM-generated code using standard
user-defined libraries compiles with the g++ compiler.

###  4.3 Comparative Models Used

We perform RLSF fine-tuning over three recent open-sourced code LLMs from
Huggingface [huggngface] namely, Google’s CodeGemma-2b [google2024codegemma],
StabilityAI’s Stable-Code-Instruct-3b [pinnaparaju2024stable], and
DeepseekAI’s Deepseek-Coder-1.3b-base [guo2024deepseek]. We also include
results against the closed-source ChatGPT model, i.e., OpenAI’s
gpt-3.5-turbo-0301 [achiam2023gpt]. We use the OpenAI API [chatgpt] to access
this model. To ensure reproducibility, the temperature and top_p parameters
are both set to $0$. It is worth noting that the three open-sourced models
have 2 billion, 2.7 billion, and 1.3 billion parameters, respectively. In
contrast, GPT-3.5 (gpt-3.5-turbo-0301) is rumored to have around 175 billion
parameters, which is about 100 times more.

Table 1: Natural Language Pseudo-code to Code Translation Results: Performance
comparison of fine-tuning methodologies over different LLMs