# Search for a $CP$-odd light Higgs boson in $J/\psi\to\gamma A^{0}$

M. Ablikim1, M. N. Achasov10,b, P. Adlarson68, S. Ahmed14, M. Albrecht4, R.
Aliberti28, A. Amoroso67A,67C, M. R. An32, Q. An64,50, X. H. Bai58, Y. Bai49,
O. Bakina29, R. Baldini Ferroli23A, I. Balossino24A, Y. Ban39,h, K.
Begzsuren26, N. Berger28, M. Bertani23A, D. Bettoni24A, F. Bianchi67A,67C, J.
Bloms61, A. Bortone67A,67C, I. Boyko29, R. A. Briere5, H. Cai69, X. Cai1,50,
A. Calcaterra23A, G. F. Cao1,55, N. Cao1,55, S. A. Cetin54A, J. F. Chang1,50,
W. L. Chang1,55, G. Chelkov29,a, D. Y. Chen6, G. Chen1, H. S. Chen1,55, M. L.
Chen1,50, S. J. Chen35, X. R. Chen25, Y. B. Chen1,50, Z. J Chen20,i, W. S.
Cheng67C, G. Cibinetto24A, F. Cossio67C, X. F. Cui36, H. L. Dai1,50, X. C.
Dai1,55, A. Dbeyssi14, R. E. de Boer4, D. Dedovich29, Z. Y. Deng1, A. Denig28,
I. Denysenko29, M. Destefanis67A,67C, F. De Mori67A,67C, Y. Ding33, C. Dong36,
J. Dong1,50, L. Y. Dong1,55, M. Y. Dong1,50,55, X. Dong69, S. X. Du72, Y. L.
Fan69, J. Fang1,50, S. S. Fang1,55, Y. Fang1, R. Farinelli24A, L. Fava67B,67C,
F. Feldbauer4, G. Felici23A, C. Q. Feng64,50, J. H. Feng51, M. Fritsch4, C. D.
Fu1, Y. Gao64,50, Y. Gao39,h, Y. G. Gao6, I. Garzia24A,24B, P. T. Ge69, C.
Geng51, E. M. Gersabeck59, A Gilman62, K. Goetzen11, L. Gong33, W. X.
Gong1,50, W. Gradl28, M. Greco67A,67C, L. M. Gu35, M. H. Gu1,50, C. Y
Guan1,55, A. Q. Guo25, A. Q. Guo22, L. B. Guo34, R. P. Guo41, Y. P. Guo9,f, A.
Guskov29,a, T. T. Han42, W. Y. Han32, X. Q. Hao15, F. A. Harris57, K. L.
He1,55, F. H. Heinsius4, C. H. Heinz28, Y. K. Heng1,50,55, C. Herold52, M.
Himmelreich11,d, T. Holtmann4, G. Y. Hou1,55, Y. R. Hou55, Z. L. Hou1, H. M.
Hu1,55, J. F. Hu48,j, T. Hu1,50,55, Y. Hu1, G. S. Huang64,50, L. Q. Huang65,
X. T. Huang42, Y. P. Huang1, Z. Huang39,h, T. Hussain66, N Hüsken22,28, W.
Ikegami Andersson68, W. Imoehl22, M. Irshad64,50, S. Jaeger4, S. Janchiv26, Q.
Ji1, Q. P. Ji15, X. B. Ji1,55, X. L. Ji1,50, Y. Y. Ji42, H. B. Jiang42, X. S.
Jiang1,50,55, J. B. Jiao42, Z. Jiao18, S. Jin35, Y. Jin58, M. Q. Jing1,55, T.
Johansson68, N. Kalantar-Nayestanaki56, X. S. Kang33, R. Kappert56, M.
Kavatsyuk56, B. C. Ke44,1, I. K. Keshk4, A. Khoukaz61, P. Kiese28, R. Kiuchi1,
R. Kliemt11, L. Koch30, O. B. Kolcu54A,m, B. Kopf4, M. Kuemmel4, M. Kuessner4,
A. Kupsc37,68, M. G. Kurth1,55, W. Kühn30, J. J. Lane59, J. S. Lange30, P.
Larin14, A. Lavania21, L. Lavezzi67A,67C, Z. H. Lei64,50, H. Leithoff28, M.
Lellmann28, T. Lenz28, C. Li40, C. H. Li32, Cheng Li64,50, D. M. Li72, F.
Li1,50, G. Li1, H. Li64,50, H. Li44, H. B. Li1,55, H. J. Li15, H. N. Li48,j,
J. L. Li42, J. Q. Li4, J. S. Li51, Ke Li1, L. K. Li1, Lei Li3, P. R. Li31,k,l,
S. Y. Li53, W. D. Li1,55, W. G. Li1, X. H. Li64,50, X. L. Li42, Xiaoyu Li1,55,
Z. Y. Li51, H. Liang64,50, H. Liang1,55, H. Liang27, Y. F. Liang46, Y. T.
Liang25, G. R. Liao12, L. Z. Liao1,55, J. Libby21, C. X. Lin51, D. X. Lin25,
T. Lin1, B. J. Liu1, C. X. Liu1, D. Liu14,64, F. H. Liu45, Fang Liu1, Feng
Liu6, G. M. Liu48,j, H. M. Liu1,55, Huanhuan Liu1, Huihui Liu16, J. B.
Liu64,50, J. L. Liu65, J. Y. Liu1,55, K. Liu1, K. Y. Liu33, Ke Liu17,n, L.
Liu64,50, M. H. Liu9,f, P. L. Liu1, Q. Liu55, Q. Liu69, S. B. Liu64,50, T.
Liu1,55, W. M. Liu64,50, X. Liu31,k,l, Y. Liu31,k,l, Y. B. Liu36, Z. A.
Liu1,50,55, Z. Q. Liu42, X. C. Lou1,50,55, F. X. Lu51, H. J. Lu18, J. D.
Lu1,55, J. G. Lu1,50, X. L. Lu1, Y. Lu1, Y. P. Lu1,50, C. L. Luo34, M. X.
Luo71, P. W. Luo51, T. Luo9,f, X. L. Luo1,50, X. R. Lyu55, F. C. Ma33, H. L.
Ma1, L. L. Ma42, M. M. Ma1,55, Q. M. Ma1, R. Q. Ma1,55, R. T. Ma55, X. X.
Ma1,55, X. Y. Ma1,50, F. E. Maas14, M. Maggiora67A,67C, S. Maldaner4, S.
Malde62, Q. A. Malik66, A. Mangoni23B, Y. J. Mao39,h, Z. P. Mao1, S.
Marcello67A,67C, Z. X. Meng58, J. G. Messchendorp56, G. Mezzadri24A, T. J.
Min35, R. E. Mitchell22, X. H. Mo1,50,55, N. Yu. Muchnoi10,b, H. Muramatsu60,
S. Nakhoul11,d, Y. Nefedov29, F. Nerling11,d, I. B. Nikolaev10,b, Z. Ning1,50,
S. Nisar8,g, S. L. Olsen55, Q. Ouyang1,50,55, S. Pacetti23B,23C, X. Pan9,f, Y.
Pan59, A. Pathak1, A. Pathak27, P. Patteri23A, M. Pelizaeus4, H. P. Peng64,50,
K. Peters11,d, J. Pettersson68, J. L. Ping34, R. G. Ping1,55, S. Pogodin29, R.
Poling60, V. Prasad64,50, H. Qi64,50, H. R. Qi53, M. Qi35, T. Y. Qi9, S.
Qian1,50, W. B. Qian55, Z. Qian51, C. F. Qiao55, J. J. Qin65, L. Q. Qin12, X.
P. Qin9, X. S. Qin42, Z. H. Qin1,50, J. F. Qiu1, S. Q. Qu36, K. H. Rashid66,
K. Ravindran21, C. F. Redmer28, A. Rivetti67C, V. Rodin56, M. Rolo67C, G.
Rong1,55, Ch. Rosner14, M. Rump61, H. S. Sang64, A. Sarantsev29,c, Y.
Schelhaas28, C. Schnier4, K. Schoenning68, M. Scodeggio24A,24B, W. Shan19, X.
Y. Shan64,50, J. F. Shangguan47, M. Shao64,50, C. P. Shen9, H. F. Shen1,55, X.
Y. Shen1,55, H. C. Shi64,50, R. S. Shi1,55, X. Shi1,50, X. D Shi64,50, J. J.
Song15, J. J. Song42, W. M. Song27,1, Y. X. Song39,h, S. Sosio67A,67C, S.
Spataro67A,67C, K. X. Su69, P. P. Su47, F. F. Sui42, G. X. Sun1, H. K. Sun1,
J. F. Sun15, L. Sun69, S. S. Sun1,55, T. Sun1,55, W. Y. Sun27, X Sun20,i, Y.
J. Sun64,50, Y. Z. Sun1, Z. T. Sun1, Y. H. Tan69, Y. X. Tan64,50, C. J.
Tang46, G. Y. Tang1, J. Tang51, J. X. Teng64,50, V. Thoren68, W. H. Tian44, Y.
T. Tian25, I. Uman54B, B. Wang1, C. W. Wang35, D. Y. Wang39,h, H. J.
Wang31,k,l, H. P. Wang1,55, K. Wang1,50, L. L. Wang1, M. Wang42, M. Z.
Wang39,h, Meng Wang1,55, S. Wang9,f, W. Wang51, W. H. Wang69, W. P. Wang64,50,
X. Wang39,h, X. F. Wang31,k,l, X. L. Wang9,f, Y. Wang51, Y. D. Wang38, Y. F.
Wang1,50,55, Y. Q. Wang1, Y. Y. Wang31,k,l, Z. Wang1,50, Z. Y. Wang1, Ziyi
Wang55, Zongyuan Wang1,55, D. H. Wei12, F. Weidner61, S. P. Wen1, D. J.
White59, U. Wiedner4, G. Wilkinson62, M. Wolke68, L. Wollenberg4, J. F.
Wu1,55, L. H. Wu1, L. J. Wu1,55, X. Wu9,f, X. H. Wu27, Z. Wu1,50, L. Xia64,50,
H. Xiao9,f, S. Y. Xiao1, Z. J. Xiao34, X. H. Xie39,h, Y. G. Xie1,50, Y. H.
Xie6, T. Y. Xing1,55, C. J. Xu51, G. F. Xu1, Q. J. Xu13, W. Xu1,55, X. P.
Xu47, Y. C. Xu55, F. Yan9,f, L. Yan9,f, W. B. Yan64,50, W. C. Yan72, H. J.
Yang43,e, H. X. Yang1, L. Yang44, S. L. Yang55, Y. X. Yang12, Yifan Yang1,55,
Zhi Yang25, M. Ye1,50, M. H. Ye7, J. H. Yin1, Z. Y. You51, B. X. Yu1,50,55, C.
X. Yu36, G. Yu1,55, J. S. Yu20,i, T. Yu65, C. Z. Yuan1,55, L. Yuan2, X. Q.
Yuan39,h, Y. Yuan1, Z. Y. Yuan51, C. X. Yue32, A. A. Zafar66, X. Zeng Zeng6,
Y. Zeng20,i, A. Q. Zhang1, B. X. Zhang1, Guangyi Zhang15, H. Zhang64, H. H.
Zhang51, H. H. Zhang27, H. Y. Zhang1,50, J. J. Zhang44, J. L. Zhang70, J. Q.
Zhang34, J. W. Zhang1,50,55, J. Y. Zhang1, J. Z. Zhang1,55, Jianyu Zhang1,55,
Jiawei Zhang1,55, L. M. Zhang53, L. Q. Zhang51, Lei Zhang35, S. Zhang51, S. F.
Zhang35, Shulei Zhang20,i, X. D. Zhang38, X. Y. Zhang42, Y. Zhang62, Y. T.
Zhang72, Y. H. Zhang1,50, Yan Zhang64,50, Yao Zhang1, Z. Y. Zhang69, G. Zhao1,
J. Zhao32, J. Y. Zhao1,55, J. Z. Zhao1,50, Lei Zhao64,50, Ling Zhao1, M. G.
Zhao36, Q. Zhao1, S. J. Zhao72, Y. B. Zhao1,50, Y. X. Zhao25, Z. G. Zhao64,50,
A. Zhemchugov29,a, B. Zheng65, J. P. Zheng1,50, Y. H. Zheng55, B. Zhong34, C.
Zhong65, L. P. Zhou1,55, Q. Zhou1,55, X. Zhou69, X. K. Zhou55, X. R.
Zhou64,50, X. Y. Zhou32, A. N. Zhu1,55, J. Zhu36, K. Zhu1, K. J. Zhu1,50,55,
S. H. Zhu63, T. J. Zhu70, W. J. Zhu36, W. J. Zhu9,f, Y. C. Zhu64,50, Z. A.
Zhu1,55, B. S. Zou1, J. H. Zou1 (BESIII Collaboration) 1 Institute of High
Energy Physics, Beijing 100049, People’s Republic of China
2 Beihang University, Beijing 100191, People’s Republic of China
3 Beijing Institute of Petrochemical Technology, Beijing 102617, People’s
Republic of China
4 Bochum Ruhr-University, D-44780 Bochum, Germany
5 Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA
6 Central China Normal University, Wuhan 430079, People’s Republic of China
7 China Center of Advanced Science and Technology, Beijing 100190, People’s
Republic of China
8 COMSATS University Islamabad, Lahore Campus, Defence Road, Off Raiwind Road,
54000 Lahore, Pakistan
9 Fudan University, Shanghai 200443, People’s Republic of China
10 G.I. Budker Institute of Nuclear Physics SB RAS (BINP), Novosibirsk 630090,
Russia
11 GSI Helmholtzcentre for Heavy Ion Research GmbH, D-64291 Darmstadt, Germany
12 Guangxi Normal University, Guilin 541004, People’s Republic of China
13 Hangzhou Normal University, Hangzhou 310036, People’s Republic of China
14 Helmholtz Institute Mainz, Staudinger Weg 18, D-55099 Mainz, Germany
15 Henan Normal University, Xinxiang 453007, People’s Republic of China
16 Henan University of Science and Technology, Luoyang 471003, People’s
Republic of China
17 Henan University of Technology, Zhengzhou 450001, People’s Republic of
China
18 Huangshan College, Huangshan 245000, People’s Republic of China
19 Hunan Normal University, Changsha 410081, People’s Republic of China
20 Hunan University, Changsha 410082, People’s Republic of China
21 Indian Institute of Technology Madras, Chennai 600036, India
22 Indiana University, Bloomington, Indiana 47405, USA
23 INFN Laboratori Nazionali di Frascati , (A)INFN Laboratori Nazionali di
Frascati, I-00044, Frascati, Italy; (B)INFN Sezione di Perugia, I-06100,
Perugia, Italy; (C)University of Perugia, I-06100, Perugia, Italy
24 INFN Sezione di Ferrara, (A)INFN Sezione di Ferrara, I-44122, Ferrara,
Italy; (B)University of Ferrara, I-44122, Ferrara, Italy
25 Institute of Modern Physics, Lanzhou 730000, People’s Republic of China
26 Institute of Physics and Technology, Peace Ave. 54B, Ulaanbaatar 13330,
Mongolia
27 Jilin University, Changchun 130012, People’s Republic of China
28 Johannes Gutenberg University of Mainz, Johann-Joachim-Becher-Weg 45,
D-55099 Mainz, Germany
29 Joint Institute for Nuclear Research, 141980 Dubna, Moscow region, Russia
30 Justus-Liebig-Universitaet Giessen, II. Physikalisches Institut, Heinrich-
Buff-Ring 16, D-35392 Giessen, Germany
31 Lanzhou University, Lanzhou 730000, People’s Republic of China
32 Liaoning Normal University, Dalian 116029, People’s Republic of China
33 Liaoning University, Shenyang 110036, People’s Republic of China
34 Nanjing Normal University, Nanjing 210023, People’s Republic of China
35 Nanjing University, Nanjing 210093, People’s Republic of China
36 Nankai University, Tianjin 300071, People’s Republic of China
37 National Centre for Nuclear Research, Warsaw 02-093, Poland
38 North China Electric Power University, Beijing 102206, People’s Republic of
China
39 Peking University, Beijing 100871, People’s Republic of China
40 Qufu Normal University, Qufu 273165, People’s Republic of China
41 Shandong Normal University, Jinan 250014, People’s Republic of China
42 Shandong University, Jinan 250100, People’s Republic of China
43 Shanghai Jiao Tong University, Shanghai 200240, People’s Republic of China
44 Shanxi Normal University, Linfen 041004, People’s Republic of China
45 Shanxi University, Taiyuan 030006, People’s Republic of China
46 Sichuan University, Chengdu 610064, People’s Republic of China
47 Soochow University, Suzhou 215006, People’s Republic of China
48 South China Normal University, Guangzhou 510006, People’s Republic of China
49 Southeast University, Nanjing 211100, People’s Republic of China
50 State Key Laboratory of Particle Detection and Electronics, Beijing 100049,
Hefei 230026, People’s Republic of China
51 Sun Yat-Sen University, Guangzhou 510275, People’s Republic of China
52 Suranaree University of Technology, University Avenue 111, Nakhon
Ratchasima 30000, Thailand
53 Tsinghua University, Beijing 100084, People’s Republic of China
54 Turkish Accelerator Center Particle Factory Group, (A)Istanbul Bilgi
University, HEP Res. Cent., 34060 Eyup, Istanbul, Turkey; (B)Near East
University, Nicosia, North Cyprus, Mersin 10, Turkey
55 University of Chinese Academy of Sciences, Beijing 100049, People’s
Republic of China
56 University of Groningen, NL-9747 AA Groningen, The Netherlands
57 University of Hawaii, Honolulu, Hawaii 96822, USA
58 University of Jinan, Jinan 250022, People’s Republic of China
59 University of Manchester, Oxford Road, Manchester, M13 9PL, United Kingdom
60 University of Minnesota, Minneapolis, Minnesota 55455, USA
61 University of Muenster, Wilhelm-Klemm-Str. 9, 48149 Muenster, Germany
62 University of Oxford, Keble Rd, Oxford, UK OX13RH
63 University of Science and Technology Liaoning, Anshan 114051, People’s
Republic of China
64 University of Science and Technology of China, Hefei 230026, People’s
Republic of China
65 University of South China, Hengyang 421001, People’s Republic of China
66 University of the Punjab, Lahore-54590, Pakistan
67 University of Turin and INFN, (A)University of Turin, I-10125, Turin,
Italy; (B)University of Eastern Piedmont, I-15121, Alessandria, Italy;
(C)INFN, I-10125, Turin, Italy
68 Uppsala University, Box 516, SE-75120 Uppsala, Sweden
69 Wuhan University, Wuhan 430072, People’s Republic of China
70 Xinyang Normal University, Xinyang 464000, People’s Republic of China
71 Zhejiang University, Hangzhou 310027, People’s Republic of China
72 Zhengzhou University, Zhengzhou 450001, People’s Republic of China
a Also at the Moscow Institute of Physics and Technology, Moscow 141700,
Russia
b Also at the Novosibirsk State University, Novosibirsk, 630090, Russia
c Also at the NRC ”Kurchatov Institute”, PNPI, 188300, Gatchina, Russia
d Also at Goethe University Frankfurt, 60323 Frankfurt am Main, Germany
e Also at Key Laboratory for Particle Physics, Astrophysics and Cosmology,
Ministry of Education; Shanghai Key Laboratory for Particle Physics and
Cosmology; Institute of Nuclear and Particle Physics, Shanghai 200240,
People’s Republic of China
f Also at Key Laboratory of Nuclear Physics and Ion-beam Application (MOE) and
Institute of Modern Physics, Fudan University, Shanghai 200443, People’s
Republic of China
g Also at Harvard University, Department of Physics, Cambridge, MA, 02138, USA
h Also at State Key Laboratory of Nuclear Physics and Technology, Peking
University, Beijing 100871, People’s Republic of China
i Also at School of Physics and Electronics, Hunan University, Changsha
410082, China
j Also at Guangdong Provincial Key Laboratory of Nuclear Science, Institute of
Quantum Matter, South China Normal University, Guangzhou 510006, China
k Also at Frontiers Science Center for Rare Isotopes, Lanzhou University,
Lanzhou 730000, People’s Republic of China
l Also at Lanzhou Center for Theoretical Physics, Lanzhou University, Lanzhou
730000, People’s Republic of China
m Currently at Istinye University, 34010 Istanbul, Turkey
n Henan University of Technology, Zhengzhou 450001, People’s Republic of China

###### Abstract

Using $J/\psi$ radiative decays from 9.0 billion $J/\psi$ events collected by
the BESIII detector, we search for di-muon decays of a $CP$-odd light Higgs
boson ($A^{0}$), predicted by many new physics models beyond the Standard
Model, including the Next-to-Minimal Supersymmetric Standard Model. No
evidence for the $CP$-odd light Higgs production is found, and we set $90\%$
confidence level upper limits on the product branching fraction
$\mathcal{B}(J/\psi\to\gamma A^{0})\times\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})$
in the range of $(1.2-778.0)\times 10^{-9}$ for $0.212\leq m_{A^{0}}\leq 3.0$
GeV/$c^{2}$. The new measurement is a 6-7 times improvement over our previous
measurement, and is also slightly better than the BaBar measurement in the
low-mass region for $\tan\beta=1$.

The origin of mass is one of the most important questions in physics. The
masses of the fundamental particles are generated through spontaneous breaking
of electroweak symmetry by the Higgs mechanism higgs . The Higgs mechanism
implies the existence of at least one new scalar particle, the Higgs boson,
which was the last missing Standard Model (SM) particle. It was discovered by
the Large Hadron Collider experiments at CERN Aad  in July 2012 and has a
profound effect on our fundamental understanding of matter.

Many models beyond the SM, such as the Next-to-Minimal Supersymmetric Standard
Model (NMSSM) Hiller ; Dermisek ; Steggermann , extend the Higgs sector to
include additional Higgs fields. The NMSSM adds an additional singlet chiral
superfield to the Minimal Supersymmetric Standard Model (MSSM) MSSM  to
alleviate the so-called “little hierarchy problem” Delgado . It contains three
$CP$-even, two $CP$-odd, and two charged Higgs bosons Hiller ; Dermisek . The
mass of the lightest Higgs boson, $A^{0}$, may be smaller than twice the mass
of the charmed quark, thus making it accessible via $J/\psi\to\gamma A^{0}$
Wilczek .

The branching fraction of $V\to\gamma A^{0}$ ($V=\Upsilon,J/\psi$) is
expressed as Wilczek ; Mangano ; Nason

$\frac{\mathcal{B}(V\to\gamma A^{0})}{\mathcal{B}(V\to
l^{+}l^{-})}=\frac{G_{F}m_{q}^{2}g_{q}^{2}C_{\rm
QCD}}{\sqrt{2}\pi\alpha}\left(1-\frac{m_{A^{0}}^{2}}{m_{V}^{2}}\right),$ (1)

where $\alpha$ is the fine structure constant, $G_{F}$ is the Fermi coupling
constant, $l\equiv e$ or $\mu$, $m_{q}$ is the quark mass, $C_{\rm QCD}$
includes the leptonic width of $\mathcal{B}(V\to l^{+}l^{-})$ Barbieri ;
Beneke  as well as $m_{A^{0}}$ dependent QCD and relativistic corrections to
$\mathcal{B}(V\to\gamma A^{0})$ Nason , and $g_{q}$ is the effective Yukawa
coupling to the Higgs field to the up- or down-type quark-pair. In the NMSSM,
$g_{c}=\cos\theta_{A}/\tan\beta$ ($q=c$) for the charm quark and
$g_{b}=\cos\theta_{A}\tan\beta$ ($q=b$) for the bottom quark, where
$\tan\beta$ is the ratio of up- and down-type Higgs doublets, and
$\cos\theta_{A}$ is the fraction of the nonsinglet component of the $A^{0}$
Dermisek1 ; fayet . The value of $\cos\theta_{A}$ is zero for a pure $A^{0}$
singlet state fayet1 . The branching fraction of $J/\psi\to\gamma A^{0}$ is
predicted to be in the range of $10^{-9}-10^{-7}$ depending upon the $A^{0}$
mass and the NMSSM parameters Dermisek . The branching fraction of
$A^{0}\to\mu^{+}\mu^{-}$ is predicted to be much larger for $\tan\beta\geq 1$
Dermisek1 . An experimental study of such a low-mass Higgs boson is desirable
to test the SM Lisanti  and to look for new physics beyond the SM Hiller ;
Dermisek ; fayet2 .

The BaBar slacR1008 , CLEO cleo , and CMS cms  experiment have searched for
di-muon decays of $A^{0}$, and placed a strong exclusion upper limit on
$g_{b}$. On the other hand, the BESIII measurements, sensitive on $g_{c}$, is
complementary to those by considering $g_{b}$. The recent BESIII measurement
bes3-2 , based on 225 million $J/\psi$ events, is slightly lower than the
BaBar measurement slacR1008  in the low-mass region for $\tan\beta\leq 0.6$.
The combined measurements of the BESIII and BaBar have revealed that the
$A^{0}$ is mostly singlet in nature because of obtained upper limit on
$\cos\theta_{A}(=|\sqrt{g_{b}g_{c}}|)\times\sqrt{\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})}$,
independent of $\tan\beta$, is very close to zero especially in the low-mass
region bes3-2 . However, this BESIII limit bes3-2  is still an order of
magnitude above the theoretical predictions Dermisek . BESIII has recently
accumulated about 39 times more $J/\psi$ events in comparison to the previous
measurement bes3-2 , and these can be utilized to discover the $A^{0}$ or
exclude parameter space of the NMSSM bes3jps .

This paper describes the search for di-muon decays of a $CP$-odd light Higgs
boson in radiative decays of $J/\psi$ using 9 billion $J/\psi$ events
collected by the BESIII detector in 2009, 2018, and 2019 bes3jps . Because
muon particle identification (PID) was not available for the $J/\psi$ data
collected in 2012, we exclude this data sample for the $A^{0}$ search.

##  I BESIII Detector and Monte Carlo Simulation

The BESIII detector bes3det  records symmetric $e^{+}e^{-}$ collisions
provided by the BEPCII storage ring Yu:IPAC2016-TUYA01 , which operates with a
peak luminosity of $1\times 10^{33}$ cm-2s-1 in the center-of-mass (CM) energy
range from 2.0 to 4.95 GeV. BESIII has collected large data samples in this
energy region Ablikim:2019hff . The cylindrical core of the BESIII detector
covers 93% of the full solid angle and consists of a helium-based multilayer
drift chamber (MDC), a plastic scintillator time-of-flight system (TOF), and a
CsI(Tl) electromagnetic calorimeter (EMC), which are all enclosed in a
superconducting solenoidal magnet providing a 1.0 T magnetic field. The
solenoid is supported by an octagonal flux-return yoke with resistive plate
counter muon identification modules interleaved with steel. The MDC measures
the momentum of charged particles with a resolution of $0.5\%$ at 1 GeV/$c$.
The EMC measures the photon energies with a resolution of $2.5\%$ ($5\%$) at 1
GeV in the barrel (end-cap) region. The time resolution of the TOF in the
barrel region is 68 ps. The time resolution of the TOF in the endcap region
was 110 ps before 2015 and was improved to be 60 ps after upgrading with the
multi-gap resistive plate chambers. Muons with momentum above 0.5 GeV/c are
identified by the iron flux return of the magnet instrumented with about 1272
$m^{2}$ of resistive plate muon counters (MUC) arranged in nine (eight) layers
in the barrel (endcaps).

Simulated Monte Carlo (MC) events based on Geant4 geant4  are used to optimize
the event selection criteria, to study the potential backgrounds, and to
determine the detector acceptance. A MC sample of 9.0 billion inclusive
$J/\psi$ events is used for the background studies with the generic TopoAna
tool topo . The known $J/\psi$ decay modes are generated by the EvtGen
generator evtgen  with branching fractions taken from the Particle Data Group
(PDG) pdg , and the remaining unknown decay modes by LUNDCHARM lundcharm . The
final state radiation corrections are included in the MC simulation using
PHOTOS photos . The production of the $J/\psi$ resonance through $e^{+}e^{-}$
annihilation including the beam-energy spread and the initial-state-radiation
(ISR) is simulated by the KKMC kkmc . A $2.93$ fb-1 $\psi(3770)$ data sample
psipp ; psipp1  is used to study the background from the quantum
electrodynamics (QED) process of $e^{+}e^{-}\to\gamma\mu^{+}\mu^{-}$. To
compute the detection efficiency, we generate 0.12 million simulated signal MC
events at 23 different Higgs mass points ranging from 0.212 to 3.0 GeV/$c^{2}$
with a phase-space model for the $A^{0}\to\mu^{+}\mu^{-}$ decay and a $P$-wave
model for the $J/\psi\to\gamma A^{0}$ decay evtgen .

##  II Data Analysis

We select events with two oppositely charged tracks and at least one photon
candidate. A photon candidate, reconstructed with clusters of energy deposited
in the EMC, is selected with a minimum energy of 25 MeV in the barrel region
($|\cos\theta|<0.8$) or 50 MeV in the end-cap region
($0.86<|\cos\theta|<0.92$). The energy deposited in the nearby TOF is included
to improve the energy resolution and reconstruction efficiency. The angle
between a photon and the nearest extrapolated track in the EMC is required to
be larger than 10 degrees to remove bremsstrahlung photons. The EMC timing is
required to be within 700 ns relative to the event start time to suppress
electronic noise and energy deposits unrelated to the signal events.

Charged tracks are reconstructed from the ionization signals measured by the
MDC and are required to be in the MDC detection acceptance region of
$|\cos\theta|<0.93$, where $\theta$ is the angle of the charged track with the
$z$ axis, which is the axis of the MDC. Further, their points of closest
approach to the $z$-axis must be within $\pm 10$ cm from the interaction point
along the $z$ direction and within $\pm 1$ cm in the plane perpendicular to
$z$. To suppress contamination by electrons and pions, both charged tracks are
required to satisfy the following selection criteria: 1) $E_{\rm
cal}^{\mu}/p<0.9$ $c$, 2) $0.1<E_{\rm cal}^{\mu}<0.3$ GeV, and 3) the absolute
value of the time difference between the TOF and expected muon time ($\Delta
t^{\rm TOF}$) must be less than 0.26 ns. Here, $E_{\rm cal}^{\mu}$ is the
energy deposited in the EMC by the $\mu^{+}/\mu^{-}$ particle, and $p$ is the
momentum of the charged muon track. To further improve the purity of muons,
one of the charged tracks is required to have its penetration depth in the MUC
be greater than ($-40.0+70\times p/({\rm GeV}/c)$) cm for $0.5\leq p\leq 1.1$
GeV/$c$ and 40 cm for $p>1.1$ GeV/$c$.

The two muon tracks are required to originate from a common vertex by
performing a vertex fit to form an $A^{0}$ candidate. A four-constraint (4C)
kinematic fit is performed with the two charged tracks and one of the photon
candidates in order to improve the mass resolution of the $A^{0}$ candidate.
If there is more than one $\gamma\mu^{+}\mu^{-}$ candidate, the candidate with
the minimum value of the $\chi^{2}$ from the 4C kinematic fit ($\chi_{4\rm
C}^{2}$) is selected, and the $\chi_{4\rm C}^{2}$ is required to be less than
40 to reject backgrounds from $J/\psi\to\pi^{+}\pi^{-}\pi^{0}$ and
$e^{+}e^{-}\to\gamma\pi^{+}\pi^{-}\pi^{0}$. We reject fake photons by
requiring the di-muon invariant mass ($m_{\mu^{+}\mu^{-}}$) obtained from the
4C kinematic fit to be less than 3.04 GeV/$c^{2}$. To suppress backgrounds
from $e^{+}e^{-}\to\gamma\mu^{+}\mu^{-}$ and
$J/\psi\to\mu^{+}\mu^{-}(\gamma)$, the absolute value of the cosine of the
muon helicity angle ($\cos\theta_{\mu}^{\rm hel}$), defined as the angle
between the direction of one of the muons and the direction of the $J/\psi$ in
the $A^{0}$ rest frame, is required to be less than 0.92.

After the above selection criteria, we determine the signal yield as a
function of $m_{A^{0}}$ in the interval of $0.212\leq m_{A^{0}}\leq 3.0$
GeV/$c^{2}$ by performing a series of one-dimensional unbinned extended
maximum likelihood (ML) fits to the reduced mass, $m_{\rm
red}=\sqrt{m_{\mu^{+}\mu^{-}}^{2}-4m_{\mu}^{2}}$ distribution of surviving
events. Fig. 1 shows the $m_{\rm red}$ distribution of surviving events
together with the background predictions from various simulated MC samples and
$2.93$ fb-1 of $\psi(3770)$ data psipp ; psipp1 . We use $m_{\rm red}$ rather
than $m_{\mu^{+}\mu^{-}}$ because it is easier to model the non-peaking
background across the entire $m_{A^{0}}$ region, in particular, the kinematic
threshold region $m_{\mu^{+}\mu^{-}}\approx 2m_{\mu}$ ($m_{\rm red}\approx
0$). The non-peaking background is dominated by
$e^{+}e^{-}\to\gamma\mu^{+}\mu^{-}$ and $J/\psi\to\mu^{+}\mu^{-}(\gamma)$, and
the peaking background by $J/\psi\to\rho/\omega\pi$ and $J/\psi\to\gamma f$
$(f=f_{2}(1270),f_{0}(1500),f_{0}(1710))$ decays, where both $\rho/\omega$ and
$f$ decay to $\pi^{+}\pi^{-}$. The $m_{\rm red}$ distribution of data is
generally well described by the background predictions, except in the low-mass
region, where KKMC kkmc  fails to reproduce the ISR events for the
$e^{+}e^{-}\to\gamma J/\psi,J/\psi\to\mu^{+}\mu^{-}$ process. This
disagreement has little impact on the search because the signal extraction
procedure does not depend on the background predictions.

Figure 1: The $m_{\rm red}$ distribution of data (black dot points with error
bars), together with the background predictions of the QED
$e^{+}e^{-}\to\gamma\mu^{+}\mu^{-}$ process from $\psi(3770)$ data (black
histogram) and $J/\psi\to\rho\pi$, $\mu^{+}\mu^{-}(\gamma)$, $\gamma f$
($f=f_{2}(1270),f_{0}(1500),f_{0}(1710)$) decays from MC samples of those
processes (gray pattern histogram). The dashed cyan histogram represents the
combined background.

The fit function includes the contributions of signal, continuum background
and peaking background components from $\rho/\omega$, $f_{2}(1270)$,
$f_{0}(1500)$, and $f_{0}(1710)$ mesons. Table 1 summarizes the $m_{\rm red}$
fit intervals for the various $m_{A^{0}}$ points used to handle both non-
peaking and peaking backgrounds smoothly.

Table 1: The $m_{\rm red}$ fit intervals for various $m_{A^{0}}$ points.  $m_{\rm red}$ fit interval (GeV/$c^{2}$) |  $m_{A^{0}}$ points (GeV/$c^{2}$) |  Order of Polynomial function
---|---|---
$0.002-0.45$ |  $0.212,0.4$ |  $5^{th}$
$0.3-0.65$ |  $0.401,0.6$ |  $4^{th}$
$0.4-1.06$ |  $0.601,1.0$ |  $3^{rd}$
$0.95-1.95$ |  $1.001,1.8$ |  $2^{nd}$
$1.7-2.5$ |  $1.802,2.4$ |  $5^{th}$
$2.3-2.7$ |  $2.402,2.6$ |  $4^{th}$
$2.5-2.9$ |  $2.602,2.848$ |  $5^{th}$
$2.75-3.0$ |  $2.85,2.90$ |  $6^{th}$
$2.85-3.032$ |  $2.902,3.0$ |  $5^{th}$

Simulated MC samples are used to develop the probability density functions
(PDFs) of signal and backgrounds. The $A^{0}$ is assumed to be a scalar or
pseudo-scalar particle with a very narrow decay width in comparison to the
experimental resolution Res . We describe the $m_{\rm red}$ distribution of
the signal PDF by the sum of two Crystal Ball (CB) functions CB  with a common
peak value and opposite side tails. The $m_{\rm red}$ resolution varies from
$2$ MeV/$c^{2}$ to $12$ MeV/$c^{2}$ while the signal efficiency varies between
27% and 53% depending upon the muon momentum values at different $A^{0}$ mass
points. We interpolate the signal efficiency and signal PDF parameters
linearly between the mass points of the generated signal MC events. The non-
peaking background PDF is described by a function
$\tanh(\sum_{l=1}^{5}p_{l}m_{\rm red}^{l})$ in the threshold mass region of
$0.212\leq m_{A^{0}}\leq 0.40$ GeV/$c^{2}$, where $p_{l}$ are the polynomial
coefficients. This function provides a threshold like behavior in the low-mass
region of the $m_{\rm red}$ distribution and passes through the origin when
$m_{\rm red}=0$. In the other mass regions, we use second, third, fourth,
fifth or sixth-order Chebyshev polynomial function to describe the non-peaking
background PDFs detailed in Table 1. We determine the initial parameters of
these background PDFs using a cocktail MC sample of all possible non-peaking
backgrounds to achieve better agreement between data and the fit models.

To take into account the well-known structure of the $\rho$-$\omega$
interference, we describe the peaking background PDF of the $m_{\rm red}$
distribution with the Gounaris and Sakurai (GS) function in the range of
$0.4\leq m_{\rm red}\leq 1.06$ GeV/$c^{2}$ gsmodel . The fit formula, detailed
in Ref. BaBar_model , is the same as that used previously by the BaBar
BaBar_model  and BESIII psipp1  experiments in the measurement of the
$e^{+}e^{-}\to\pi^{+}\pi^{-}$ cross-section in the $\rho/\omega$ mass region.
The amplitudes for the higher $\rho$ states, $\rho(1450)$, $\rho(1700)$, and
$\rho(2150)$, as well as the massses and widths of those states are taken from
Ref. BaBar_model . We fix the $\omega$ width according to the PDG pdg  value
and float the other parameters during the fit. We describe the peaking
background PDFs corresponding to $f_{2}(1270)$, $f_{0}(1500)$ and
$f_{0}(1710)$ resonances by the sum of the two CB functions CB  using the
parameters determined from MC samples of $J/\psi\to\gamma f$,
$f\to\pi^{+}\pi^{-}$ decays, where $f=f_{2}(1270)$, $f_{0}(1500)$, and
$f_{0}(1710)$ mesons.

The search for the $A^{0}$ narrow resonance is performed in steps of
approximately half the $m_{\rm red}$ resolution, i.e., 1 MeV/$c^{2}$ in the
mass range of $0.22\leq m_{A^{0}}\leq 1.5$ GeV/$c^{2}$ and 2.0 MeV/$c^{2}$ in
the other $m_{A^{0}}$ regions, with a total of 2,035 $m_{A^{0}}$ points. The
PDF parameters of the signal and peaking backgrounds of $J/\psi\to\gamma f$
are fixed while the non-peaking background PDF, and the numbers of the signal,
peaking, and non-peaking background events are floated. Plots of the fit to
the $m_{\rm red}$ distribution for two selected mass points are shown in Fig.
2.

Figure 2: Fits to the $m_{\rm red}$ distributions for (top) $m_{A^{0}}=0.221$
GeV/$c^{2}$ and (bottom) $m_{A^{0}}=0.696$ GeV/$c^{2}$. The corresponding
local significance values at these mass points are observed to be $3.3\sigma$
and $3.5\sigma$, respectively. Black dots with error bars represent the data,
the red long-dashed curve the non-peaking background, the pink dotted curve
the peaking background, the green dashed curve the signal PDF, and the solid
blue curve the total fit results. In the bottom figure, the well-known
$\rho-\omega$ interference is taken care of by describing the peaking
background PDF of the $m_{\rm red}$ distribution by a GS function gsmodel ;
BaBar_model , as described in the text.

The fit is repeated with alternative signal, peaking, and non-peaking
background PDFs to determine the systematic uncertainties for the numbers of
signal events associated with the corresponding PDFs at each $m_{A^{0}}$
point. The uncertainty associated with the signal PDF is studied by replacing
the sum of the two CB functions with a ‘Cruijff’ function Cruijff . The
uncertainty associated with the $\rho-\omega$ peak is evaluated by varying the
$\rho$ and $\omega$ contributions in the formula of Eq.(26) of Ref.
BaBar_model . The uncertainty due to the peaking background of
$J/\psi\to\gamma f$ is studied by replacing the sum of the two CB functions
with the simulated MC samples of the corresponding decay processes convolved
with a Gaussian function whose parameters are floated during the fit. The
uncertainty due to the non-peaking background PDF is studied by replacing the
$\tanh(\sum_{l=1}^{5}p_{l}m_{\rm red}^{l})$ and $n^{th}$ order Chebyshev
polynomial function with $\tanh(\sum_{l=1}^{6}p_{l}m_{\rm red}^{l})$ and
$(n+1)^{th}$ order Chebyshev polynomial functions, respectively, in the fit.
The one with the largest signal yield among these fit scenarios is considered
to produce the final result.

Figure 3: (a) The product branching fractions $\mathcal{B}(J/\psi\to\gamma
A^{0})\times\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})$ (BF) and (b) signal
significance ($\mathcal{S}$) obtained from the fit as a function of
$m_{A^{0}}$.

The product branching fraction of $J/\psi\to\gamma A^{0}$ and
$A^{0}\to\mu^{+}\mu^{-}$ as a function of $m_{A^{0}}$ is calculated as

$\mathcal{B}(J/\psi\to\gamma
A^{0})\times\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})=\frac{N_{sig}}{\epsilon\cdot
N_{J/\psi}},$ (2)

where $N_{sig}$ is the number of signal events, $\epsilon$ is the signal
selection efficiency, and $N_{J/\psi}=(8.998\pm 0.039)\times 10^{9}$ is the
number of $J/\psi$ events. Fig. 3 shows the plots of the product branching
fractions $\mathcal{B}(J/\psi\to\gamma
A^{0})\times\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})$ and the statistical
significance, defined as $\mathcal{S}={\rm sign}(N_{sig})\sqrt{2{\rm
ln}(\mathcal{L}_{\rm max}/\mathcal{L}_{0})}$, where $\mathcal{L}_{\rm
max}~{}(\mathcal{L}_{0})$ is the maximum likelihood value for a fit with
number of signal events being floated (fixed at zero). The largest upward
local significance value is determined to be $3.5\sigma$ at $m_{A^{0}}=0.696$
GeV/$c^{2}$. Based on a large ensemble of pseudo experiments slacR1008 , the
probability of observing a fluctuation of $\mathcal{S}\geq 3.5\sigma$ is
estimated to be $12\%$. The corresponding global significance value is
determined to be at the level of $1\sigma$. Thus, we conclude that no evidence
of Higgs production is found within the searched $m_{A^{0}}$ regions.

##  III Systematic Uncertainties

According to Eq. 2, the systematic uncertainties for the branching fraction
measurement include those from the number of signal events, the reconstruction
efficiency, and the number of $J/\psi$ events. The uncertainties associated
with the number of signal events originating from the PDF parameters of signal
and backgrounds are considered by performing alternative fits at each
$m_{A^{0}}$ point.

Pseudo experiments are utilized to test the reliability of the fit procedures
and compute the fit bias, which may appear due to imperfect signal and
background modeling. The same fit procedure is performed in each pseudo
experiment. The resultant average difference between the input and output
signal yields is determined to be 0.3 events. We consider it as an additive
systematic uncertainty ($\sigma_{\rm add}$), which may affect the significance
of any observation but does not scale with the reconstructed signal yield.

The uncertainties associated with the reconstruction efficiency and the number
of $J/\psi$ events don’t affect the significance of any observation. Thus, we
consider them as multiplicative systematic uncertainties ($\sigma_{\rm mult}$)
and scale with the number of reconstructed signal events. The uncertainty
associated with the reconstruction efficiency includes those from tracking,
PID, and the photon detection efficiency.

The uncertainty due to MDC tracking is determined to be $1\%$ per track using
the high statistics control samples of $J/\psi\to\rho\pi$ and $J/\psi\to
p\bar{p}\pi^{+}\pi^{-}$. A total of $2.0\%$ systematic uncertainty is assigned
for the two charged tracks in this analysis. The systematic uncertainty
associated with the photon reconstruction efficiency is determined using a
control sample of $e^{+}e^{-}\to\gamma\mu^{+}\mu^{-}$ in which the ISR photon
is predicted using the four momenta of the two charged tracks. This sample
also includes the dominant contribution from $J/\psi\to\gamma\pi^{+}\pi^{-}$
decay, including all the possible intermediate resonances. The relative
difference in efficiency between data and MC is found to be $0.2\%$, which is
considered as a systematic uncertainty.

A control sample of $J/\psi\to\mu^{+}\mu^{-}(\gamma)$ is used to evaluate the
systematic uncertainty due to the muon PID, $\cos\theta_{\mu}^{\rm hel}$, and
$\chi_{4\rm C}^{2}$ requirements. In this sample, one track is tagged with a
tight muon PID. The final uncertainty associated with the muon PID also takes
into account the fraction of events with one or two tracks identified as muons
obtained from the simulated signal MC sample. The corresponding uncertainties,
computed as the relative change in efficiency between data and MC, are
determined to be $(2.9-4.1)\%$, $0.8\%$ and $1.8\%$, respectively. The
systematic uncertainty due to the number of $J/\psi$ events is $0.3\%$ using
$J/\psi$ inclusive hadronic events. Table 2 summarizes the fit bias and
multiplicative sources of the systematic uncertainties, where we obtain the
total $\sigma_{\rm mult}$ by adding the individual ones in quadrature. The
total $\sigma_{\rm mult}$ varies between $4.1\%$ to $5.0\%$ depending on the
Higgs mass point. The final systematic uncertainty is calculated as
$\sqrt{\sigma_{\rm add}^{2}+(\sigma_{\rm mult}\times N_{sig})^{2}}$.

Table 2: The fit bias and multiplicative sources of the systematic uncertainties. The systematic uncertainties associated with the signal, peaking, and non-peaking background PDFs are taken as the largest difference of signal yield among the alternative fit scenarios at each $m_{A^{0}}$ point as described in Sect. II. Source | Uncertainty
---|---
Additive systematic uncertainties (events)
Fit Bias | 0.3
Total | 0.3
Multiplicative systematic uncertainties ($\%$)
Tracking | 2.0
Photon detection efficiency | 0.2
Depth in MUC | 2.9 – 4.1
$E_{cal}^{\mu}$ | 0.1
$\Delta t^{\rm TOF}$ | Negl.
Cos$\theta_{\mu}^{hel}$ | 0.8
$\chi_{4\rm C}^{2}$ | 1.8
$J/\psi$ counting | 0.7
Total | 4.1 – 5.0

##  IV Result

Since no evidence of Higgs production is found, we set $90\%$ confidence level
(C.L.) upper limits on the product branching fractions
$\mathcal{B}(J/\psi\to\gamma A^{0})\times\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})$
as a function of $m_{A^{0}}$ using a Bayesian method pdg  after incorporating
the systematic uncertainty by smearing the likelihood curve with a Gaussian
function having a width equal to the systematic uncertainty. The limits vary
in the range of $(1.2-778.0)\times 10^{-9}$ for the Higgs mass region of
$0.212\leq m_{A^{0}}\leq 3.0$ GeV/$c^{2}$ depending on the $m_{A^{0}}$ point,
as shown in Fig. 4. The new measurement has a 6-7 times improvement over the
previous BESIII measurement bes3-2 .

Figure 4: The $90\%$ C.L. upper limits on the product branching fractions
$\mathcal{B}(J/\psi\to\gamma A^{0})\times\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})$
versus $m_{A^{0}}$ including all the uncertainties, together with the expected
limits computed using a large number of pseudo experiments. The inner and
outer bands correspond to $68\%$ and $95\%$ of the expected limit values and
include the statistical uncertainties only.

To compare our results with the BaBar measurement slacR1008 , we also compute
$90\%$ C.L. upper limits on the effective Yukawa coupling of the Higgs fields
to the bottom-quark pair
$g_{b}(=g_{c}\tan^{2}\beta)\times\sqrt{\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})}$
as a function of $m_{A^{0}}$ for different values of $\tan\beta$ using Eq.(1)
as shown in Fig. 5. Our new measurement is slightly better than the BaBar
measurement slacR1008  in the low-mass region for $\tan\beta=1.0$.

Figure 5: The $90\%$ C.L. upper limits on the effective Yukawa coupling of the
Higgs field to the bottom-quark pair
$g_{b}(=g_{c}\tan^{2}\beta)\times\sqrt{\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})}$
as a function of $m_{A^{0}}$ for different values of $\tan\beta$ together with
the BaBar measurement. Our results are slightly better than the BaBar
measurement in the low mass region for $\tan\beta=1$.

##  V Summary

We search for di-muon decays of $A^{0}$ in $J/\psi\to\gamma A^{0}$ using $9.0$
billion $J/\psi$ events collected by the BESIII detector. No evidence of Higgs
production is found, and we set $90\%$ C.L. upper limits on product branching
fractions $\mathcal{B}(J/\psi\to\gamma
A^{0})\times\mathcal{B}(A^{0}\to\mu^{+}\mu^{-})$ in the range of
$(1.2-778.0)\times 10^{-9}$ for $0.212\leq m_{A^{0}}\leq 3.0$ GeV/$c^{2}$.
This result has an improvement by a factor of 6-7 over the previous BESIII
measurement bes3-2 , and is better than the BaBar measurement slacR1008  for
$m_{A^{0}}\leq 0.7$ GeV/$c^{2}$ for $\tan\beta=1$. Thus, our measurement is
more stringent for $m_{A^{0}}\leq 0.7$ GeV/$c^{2}$ over the existing
experimental results slacR1008 ; cleo ; cms ; bes3-2 ; bes3-1 . The new BESIII
limit is also lower than the theoretical prediction at the threshold Higgs
mass point of $0.212$ GeV/$c^{2}$, and thus constrains a large fraction of the
parameter space of the new physics models, including NMSSM Hiller ; Dermisek ;
fayet2 .

##  VI Acknowledgments

The BESIII collaboration thanks the staff of BEPCII, the IHEP computing center
and the supercomputing center of USTC for their strong support. This work is
supported in part by National Key R$\&$D Program of China under Contracts Nos.
2020YFA0406300, 2020YFA0406400; National Natural Science Foundation of China
(NSFC) under Contracts Nos. 11335008, 11625523, 11625523, 11635010, 11735014,
11822506, 11835012, 11935015, 11935016, 11935018, 11961141012, 12022510,
12025502, 12035009, 12035013, 12035013, 12061131003, 11705192, 11950410506,
12061131003; the Chinese Academy of Sciences (CAS) Large-Scale Scientific
Facility Program; Joint Large-Scale Scientific Facility Funds of the NSFC and
CAS under Contracts Nos. U1732263, U1832207, U1832103, U2032111; CAS Key
Research Program of Frontier Sciences under Contract No. QYZDJ-SSW-SLH040; 100
Talents Program of CAS; INPAC and Shanghai Key Laboratory for Particle Physics
and Cosmology; ERC under Contract No. 758462; European Union Horizon 2020
research and innovation programme under Contract No. Marie Sklodowska-Curie
grant agreement No 894790; German Research Foundation DFG under Contracts Nos.
443159800, Collaborative Research Center CRC 1044, FOR 2359, FOR 2359, GRK
214; Istituto Nazionale di Fisica Nucleare, Italy; Ministry of Development of
Turkey under Contract No. DPT2006K-120470; National Science and Technology
fund; Olle Engkvist Foundation under Contract No. 200-0605; STFC (United
Kingdom); The Knut and Alice Wallenberg Foundation (Sweden) under Contract No.
2016.0157; The Royal Society, UK under Contracts Nos. DH140054, DH160214; The
Swedish Research Council; U. S. Department of Energy under Contracts Nos. DE-
FG02-05ER41374, DE-SC-0012069.

## References

  * (1) P. W. Higgs, Phys. Lett. 12, 132 (1964); P. W. Higgs, Phys. Rev. Lett. 13, 508 (1964); P. W. Higgs, Phys. Rev. 145, 1156 (1966); F. Englert and R. Brout, Phys. Rev. Lett. 13, 321 (1964); G. S. Guralnik, C. R. Hagen and T. W. B. Kibble, Phys. Rev. Lett. 13, 585 (1964).
  * (2) G. Aad, et al. (Atlas Collaboration), Phys. Lett. B 716, 1–29 (2012); S. Chatrchyan, et al. (CMS Collaboration), Phys. Lett. B 716, 30–61 (2012).
  * (3) G. Hiller, Phys. Rev. D 70, 034018 (2004); R. Dermisek and J. F. Gunion, Phys. Rev. Lett. 95, 041801 (2005).
  * (4) R. Dermisek, J. F. Gunion, and B. McElrath, Phys. Rev. D 76, 051105 (2007).
  * (5) J. Steggermann, Ann. Rev. Nucl. Part. Sci. 70, 197 (2020).
  * (6) H. E. Haber and G. L. Kane, Phys. Rep. 117, 75 (1985).
  * (7) A. Delgado, C. Kolda, and A. D. Puente, Phys. Lett. B 710, 460 (2012).
  * (8) F. Wilczek, Phys. Rev. Lett. 39, 1304 (1977).
  * (9) M. L. Mangano and P. Nason, Mod. Phys. Lett. A 22, 1373 (2007).
  * (10) P. Nason, Phys. Lett. B 175, 223 (1986).
  * (11) R. Barbieri, R. Gatto, R. Kögerler, and Z. Kunszt, Phys. Lett. B 57, 455 (1975).
  * (12) M. Beneke, A. Signer, and V. A. Smirnov, Phys. Rev. Lett. 80, 2535 (1998).
  * (13) R. Dermisek and J. F. Gunion, Phys. Rev. D 81, 075003 (2010); F. Domingo, J. High Energy Phys. 04, 016 (2011).
  * (14) P. Fayet, Phys. Rev. D 75, 115017 (2007); Phys. Lett. B 675, 267 (2009).
  * (15) P. Fayet, Nucl. Phys. B 187, 184 (1981); H. B. Li and T. Luo, Phys. Lett. B 686, 249 (2010).
  * (16) M. Lisanti and J. G. Wacker, Phys. Rev. D 79, 115006 (2009).
  * (17) P. Fayet and M. Mezard, Phys. Lett. B 104, 104 (1981); P. Fayet, Phys. Rev. D 103, 035034 (2021).
  * (18) J. P. Lees et al. (BABAR Collaboration), Phys. Rev. D 87, 059903(R) (2013); V. Prasad, Ph.D. thesis [Report No. SLAC-R-1008, 2013 (unpublished)], arXiv:1307.4560 (2013).
  * (19) W. Love et al. (CLEO Collaboration), Phys. Rev. Lett. 101, 151802 (2008).
  * (20) S. Chatrchyan et al. (CMS Collaboration), Phys. Rev. Lett. 109, 121801 (2012).
  * (21) M. Ablikim et al. (BESIII Collaboration), Phys. Rev. D 93, 052005 (2016).
  * (22) http://bes3.ihep.ac.cn/doc/3313.html
  * (23) M. Ablikim et al. (BESIII Collaboration), Nucl. Instrum. Methods Phys. Res., Sect. A 614, 345 (2010).
  * (24) C. H. Yu et al., Proceedings of IPAC2016, Busan, Korea, 2016, doi:10.18429/JACoW-IPAC2016-TUYA01.
  * (25) M. Ablikim et al. [BESIII Collaboration], Chin. Phys. C 44, 040001 (2020).
  * (26) S. Agostinelli et al. (GEANT Collaboration), Nucl. Instrum. Methods Phys. Res., Sect. A 506, 250 (2003).
  * (27) X. Y. Zhou, S. X. Du, G. Li and C. P. Shen, Comput. Phys. Commun. 258, 107540 (2021).
  * (28) D. J. Lange, Nucl. Instrum. Methods Phys. Res., Sect. A 462, 152 (2001); R.G. Ping, Chin. Phys. C 32, 599 (2008).
  * (29) P.A. Zyla et al. (Particle Data Group), Prog. Theor. Exp. Phys. 2020, 083C01 (2020).
  * (30) J. C. Chen, G. S. Huang, X. R. Qi, D. H. Zhang, and Y. S. Zhu, Phys. Rev. D 62, 034003 (2000).
  * (31) E. Barberio and Z. Was, Comput. Phys. Commun. 79, 291 (1994).
  * (32) S. Jadach, B. F. L. Ward, and Z. Was, Phys. Rev. D 63, 113009 (2001).
  * (33) M. Ablikim et al. (BESIII Collaboration), Chin. Phys. C 37, 123001 (2013).
  * (34) M. Ablikim et al. (BESIII Collaboration), Phys. Lett. B 753, 629 (2016).
  * (35) E. Fullana and M. A. Sanchis-Lozano, Phys. Lett. B 653, 67 (2007).
  * (36) J. E. Gaiser, Ph.D. thesis [Report No. SLAC-R-255, 1982 (unpublished)]; M. J. Oreglia, Ph. D. thesis [Report No. SLAC- R-236, 1980 (unpublished)]; T. Skwarnicki, Ph.D. Thesis [Report No. DESY-F-31-86-02, 1986 (unpublished)].
  * (37) G. J. Gounaris and J. J. Sakurai, Phys. Rev. Lett. 21, 244 (1968).
  * (38) J. P. Lees et al. (BaBar Collaboration), Phys. Rev. D 86, 032013 (2012).
  * (39) The Cruijff function is an asymmetric Gaussian function having a common mean ($\mu$), different left-right resolutions ($\sigma_{L,R}$) and non-Gaussian tails ($\alpha_{L,R}$): $f(x)={\rm exp}((x-\mu)^{2}/(2\sigma_{L,R}^{2}+\alpha_{L,R}(x-\mu)^{2}))$.
  * (40) M. Ablikim et al. (BESIII Collaboration), Phys. Rev. D 85, 092012 (2012).


Preprint 1 0.9

# Evaluating Micro Parsons Problems as Exam Questions

Zihan Wu  University of MichiganAnn ArborMIUSA<EMAIL_ADDRESS>0000-0002-3161-2232 and  David H. Smith IV  University of IllinoisUrbanaILUSA
<EMAIL_ADDRESS>0000-0002-6572-4347

(2024)

###### Abstract.

Parsons problems are a type of programming activity that present learners with
blocks of existing code and requiring them to arrange those blocks to form a
program rather than write the code from scratch. Micro Parsons problems extend
this concept by having students assemble segments of code to form a single
line of code rather than an entire program. Recent investigations into micro
Parsons problems have primarily focused on supporting learners leaving open
the question of micro Parsons efficacy as an exam item and how students
perceive it when preparing for exams.

To fill this gap, we included a variety of micro Parsons problems on four
exams in an introductory programming course taught in Python. We use Item
Response Theory to investigate the difficulty of the micro Parsons problems as
well as the ability of the questions to differentiate between high and low
ability students. We then compare these results to results for related
questions where students are asked to write a single line of code from
scratch. Finally, we conduct a thematic analysis of the survey responses to
investigate how students’ perceptions of micro Parsons both when practicing
for exams and as they appear on exams.

Parsons Problems, CS1, Assessment, micro Parsons Problems

††journalyear: 2024††copyright: acmlicensed††conference: Proceedings of the
2024 Innovation and Technology in Computer Science Education V. 1; July 8–10,
2024; Milan, Italy††booktitle: Proceedings of the 2024 Innovation and
Technology in Computer Science Education V. 1 (ITiCSE 2024), July 8–10, 2024,
Milan, Italy††doi: 10.1145/3649217.3653583††isbn:
979-8-4007-0600-4/24/07††ccs: Social and professional topics Computing
education

##  1\. Introduction

Learning to program is difficult for beginners. Researchers have been working
to design pedagogical activities for learners that are engaging and welcoming
for novices. To improve student engagement, present models for good practice,
and provide immediate feedback, Parsons and Haden (Parsons and Haden, 2006)
designed Parsons problems as an alternative type of practice. Instead of
asking learners to write code from scratch, Parsons problems provide mixed-up
code blocks and require learners to identify the blocks needed in the correct
solution and rearrange them into the correct order. As a type of completion
problem, Parsons problems make programming practice less challenging for
beginners by reducing the problem space.

In traditional Parsons problems, each code block contains at least one line of
code. Micro Parsons problems extended this concept by focusing on a single
line of code. It provides small code fragments for learners and asks them to
form a code statement in one line. Prior work on micro Parsons problems has
studied its effect on learners as practice questions, and found that it
reduced the time needed for learners to complete the problems, encouraged more
learners to complete optional practice tasks, and resulted in comparable
learning gain as writing code from scratch (Wu et al., 2023).

For traditional Parsons problems, Denny et al. (2008) evaluated their
effectiveness as exam items, and found that students’ performances on these
problems are highly correlated with code writing tasks. As a fine-grained
variation of Parsons problems, micro Parsons problems also have the potential
to be used as exam items. However, existing literature on micro Parsons
problems focuses on their benefits for learners, leaving an unaddressed
question concerning their effectiveness as exam items.

This paper aims to bridge this gap by conducting an empirical investigation of
using micro Parsons problems within the context of exams and exam preparation
materials in an introductory programming course. We investigate the following
research questions:

  1. RQ1:

What are the psychometric properties (e.g., item difficulty and
discrimination) of micro Parsons problems and how do these compare to those of
single line code writing problems?

  2. RQ2:

What are students perceptions and preferences when comparing micro Parsons
problems to single line code writing problems for the purposes of studying and
when they appear on summative assessments?

##  2\. Background

###  2.1. Parsons and Micro Parsons Problems

Parsons problems, as introduced by Parsons and Haden (2006), are a programming
activity typically used in introductory courses. They provide a scaffolded
environment to construct programs, presenting students with blocks of code
that they must rearrange to form a solution rather than having them write that
code from scratch. This problem type has shown a variety of benefits for
learners such as improving learning efficiency (Ericson et al., 2018),
improving engagement (Parsons and Haden, 2006; Morin et al., 2020; Ericson et
al., 2015), reducing cognitive load (Bender et al., 2023; Fabic et al., 2019;
Morrison et al., 2016), improving the motivation of students (Kumar, 2017).
Researchers have designed many variants of Parsons problems and investigated
students’ perceptions (Morin and Kecskemety, 2021; Oyelere et al., 2019; Denny
et al., 2008).

A recently introduced variant of Parsons problems is micro Parsons problems
(Wu et al., 2021; Wu, 2023). Whereas in traditional Parsons problems students
are asked to vertically arrange blocks that each contain at least one line of
code, micro Parsons problems require students to arrange smaller code
fragments horizontally to form a single line of code. The motivation for this
variation is to transfer the positive impacts that have been found with
respect to traditional Parsons problems to other domains where the
construction of a single line of code or expression is the core learning
objective (e.g., regex, SQL). Much like prior investigations into traditional
Parsons problems, initial investigations into micro Parsons have also focused
on the impacts of utilizing them as a tool for learning (Wu et al., 2023).
Given the recency of this variety of Parsons problem, additional
investigations to fully explore the effectiveness of this problem type in both
formative and summative contexts are warranted.

###  2.2. Parsons Problems as Exam Items

Though both the original purpose of Parsons problems and much of the research
that has followed has focused on their use as a tool for learning (Ericson et
al., 2022), several studies have investigated their utility as an exam item.
Denny et al. (2008) provided the first of these investigations. They
identified that students’ performance in Parsons problems was highly
correlated with performance on code writing tasks, suggesting both items
measure a similar skill. Similarly, Poulsen et al. (2022) found that proof
writing questions presented in a format similar to Parsons Problems, termed
“Proof Blocks”, still provided ample information on students’ proof writing
abilities.

Denny et al. (2008) make additional recommendations on the design of Parsons
problems for exams suggesting that: (1) incorrect blocks of code (termed
distractors) should be visually grouped with their correct alternatives to
minimize cognitive load and (2) the inclusion of syntax that indicates
structure (e.g., curly braces, indentation) can be used to provide hints on
the correct response. With respect to the former design consideration
involving the use of distractors, more recent work has indicated that the
presence of distractors increasing the difficulty of the item compared to not
having distractors. However, their presence was shown to have a minimal impact
on Parsons problems quality as an exam item at the cost of students spending
substantially more time on the problem (Smith and Zilles, 2023; Smith et al.,
2023a; Smith IV, 2023; Smith et al., 2023b). Similar investigations informing
the design and utility of micro Parsons as an exam item have yet to be
performed.

###  2.3. Measurement Theory

Central to evaluating a question type as an exam item is selecting a tool set
for performing that evaluation. Additionally, the goal of the assessment must
be identified and in order to determine how its items can be improved to meet
that goal. There are multiple ways to evaluate the type of questions. In the
case of the study by Denny et al. (2008), one of the goals was to identify if
Parsons problems evaluated a skill similar to code writing. Other criteria
outside of this work include the difficulty of the item, and the ability of
the question to delineate between students who are adept at a given skill and
those who are not. By improving these characteristics of the items that make
up an exam we can improve the measurement ability of the exam as a whole (Ebel
and Frisbie, 1972).

A common approach for examining the characteristics of an item is Item
Response Theory (IRT). IRT models the probability of a student of a given
ability ($\theta$) to respond to a problem with a variety of estimated
parameters that characterize a given item. There are a variety of models in
the IRT family, each characterized by the number of parameters it includes.
For example, the 2 Parameter Logistic Model (2PL)

$P_{i}(\theta)=\frac{1}{1+e^{-(a_{i}\cdot(\theta-b_{i}))}}$

includes parameters for estimating item discrimination ($a$) and difficulty
($b$) (De Champlain, 2010; Birnbaum, 1968). Difficulty is characterized by the
ability level at which a student has a 50% chance of responding correctly to a
question and discrimination how sharply an item distinguishes between students
at a that given difficulty level. The 3PL model adds a third parameter
generally referred to as “pseudo-guessing” which estimates a lower asymptotic
bound and is interpreted as the probability of a low-ability student guessing
the correct solution (Birnbaum, 1968). The 4PL model adds a final fourth
parameter for estimating an upper asymptotic bound which is often referred to
as “carelessness” or “slip” and is interpreted as the probability of an
otherwise high ability student responding incorrectly to the question (Barton
and Lord, 1981).

##  3\. Methods

(a) Standard micro Parsons - The first variation of micro Parsons problems as
used on exams one and two.

(b) Fill-in-the-Blank micro Parsons - The second variation of Parsons problems
as used on exams three and four.

Figure 1. The two variations of Parsons problems used in the course.

###  3.1. Course Context and Data Collection

Figure 2. An example of a single-line programming question

To evaluate our research questions, we designed a variety of micro Parsons
problems and deployed them on exams in a large introductory Python course. The
course aims at undergraduate students with non-tech majors such as business
majors, and covers basic Python topics including Python Collections, loops,
and basic classes and objects. Students are given four proctored exams
throughout the semester along with a practice exam generator released a week
before each exam. These exam generators allow students to continuously
generate practice exams, which draw questions from a bank of questions defined
by the instructor.

The number of students sitting each exam varied throughout the semester due to
attrition, with 437 taking the first exam and 418 taking the final. All exams
are computer-based and delivered via an open-source assessment platform called
PrairieLearn (West et al., 2015). This platform enables students to receive
immediate feedback on the correctness of a submission and facilitates partial
credit by allowing students to make multiple attempts on a question for a
reduced number of points for each subsequent attempt.

Beyond micro Parsons problems, these exams contain the following other
question types: 1) code writing question, 2) traditional parsons problems, 3)
single-line code writing problems (Figure 2), 3) code fixing problems, 4)
short answer code comprehension questions, and 5) code tracing questions.
Collection of this data, as well as all other data collected in this course,
was approved by the institutional ethics review board at the institution where
the study was conducted.

###  3.2. Problem Design

Two variations of micro Parsons problems were deployed in the course. In the
first two exams the included problems asked students to arrange the given
blocks to form a single, independent line of code that accomplished a given
task (Figure 1(a)). In the remaining two exams, students were given an
existing segment of code with a single line of code removed. For these
problems, students were asked to arrange blocks to “fill in the blank” such
that the segment of code as a whole accomplished its given task (Figure 1(b)).
In both cases, the micro Parsons problems included jumbled distractors reduce
the possibility of guessing the correct solution. Additionally, students were
given partial credit for incorrect attempts based on the number of blocks they
positioned correctly.

###  3.3. Student Survey

In addressing RQ2, students were given an end-of-course survey on their
perceptions of the micro Parsons problems and how they compared to single-line
programming questions. First, students were asked to provide their general
thoughts in comparing the two question types in short answer format. Following
that, students were asked two multiple choice asking which question type they
preferred where the options were: (1) Micro Parsons, (2) Single Line Code
Writing, and (3) No Preference. The questions are as follows:

  * Q1:

Which question type do you prefer to appear on exams?

  * Q2:

Which question type do you prefer to appear on study materials (e.g., practice
exams)?

In association with each multiple choice question was a short answer question
which asked students to expand on their preference.

(a) Item difficulty statistics from the 2PL model for each of the micro
Parsons included on exams. Coefficients are interpreted as the ability level
at which a student has a 50% chance of answering correctly.

(b) Item discrimination statistics from 2PL model for each of the micro
Parsons included on exams. The thresholds for item discrimination (e.g., how
well an item distinguishes between students above and below a given ability
level) from Baker (2001) are presented as well.

Figure 3. The Item-Difficulty and Item-Discrimination statistics for the items
used in this study. Figure 4.  Comparison of the distributions of item-
difficulty and item-discrimination statistics for micro Parsons and single-
line code writing questions

##  4\. RQ1: Difficulty and Discrimination

To address RQ1 the scores students received on questions were dichotomized
based on the success of a student’s first attempt. This is done to enable
analysis with a 2PL IRT model, which requires the use of dichotomized scores.
The reason for fitting the model based on the correctness of a student’s first
attempt rather than their best attempt is, given students can attempt a
question multiple times, many questions were ultimately answered correctly by
all or a large majority of students. This would prevent those questions from
being included when fitting the model, as 2PL IRT does not support the
inclusion of questions with one response category (i.e., all correct or all
incorrect). We fit a 2PL IRT model for each exam to determine the item
difficulty and item discrimination statistics for the items that appear on
those exams. To aid in the interpretation of the item-discrimination
statistics, figures use the thresholds presented by Baker (2001) of: Very Low
($\leq 0.34$), Low ($0.35-0.64$), Moderate ($0.65-1.34$), High ($1.35-1.69$),
and Very High ($>1.7$).

For the comparison between single-line code writing problems and micro Parsons
problems, we selected a total of 17 single-line code writing questions that
appeared alongside the micro Parsons problems on exams. These questions cover
related topics (e.g., string slicing, boolean expressions, file opening) but
are not identical in format to the micro Parsons. As such, their results are
included to help contextualize the item statistics for micro Parsons – given
they are a related item testing similar concepts – rather than provide a one-
to-one comparison.

###  4.1. Results

Overall, the micro Parsons problems appeared to generally have low difficulty
(Figure 3(a)) coefficients with moderate discrimination (Figure 3(b)). These
results indicate that the problems, in general, appear to do a reasonable job
of differentiating between students is the lower ability range. Comparing the
distribution of difficulty and discrimination statistics for micro Parsons and
single-line programming questions we find that both appear to have similar
distributions of difficulty but single-line code writing problems have higher
item-discrimination (Figure 4). This indicates that on average the item
functions for students of lower ability and single-line code writing questions
provide a more sharp delineation between students at that ability level.

In looking at Figure 3(a), the one notable exception to this trend are the
questions relating to opening a file which appeared on exam three. These
questions had exceptionally high difficulty compared to micro Parsons which
appeared on other exams. These questions differed from their counterparts in
that the student was required to read the surrounding code and, based on the
indentation, determine whether to open the file using only the open() function
or using with-as. In looking at the incorrect submissions we find that 34% of
all incorrect submissions attempted to use with-as. This highlights a
potential connection exists between the distractors that are chosen in micro
Parsons and the provided code for which the students are intended to fill in
the blanks.

##  5\. RQ2: Student Perceptions and Preferences

In total, we collected survey responses from 390 students. Two researchers
independently coded the first 50 responses to develop an initial code book.
Upon iterating and finalizing the code book on the first 50 responses, the
researchers independently coded the next 50 to establish an interrater
agreement. The inter-rater agreement, Krippendorf’s alpha, was $\alpha=0.7$,
which was sufficient for preliminary conclusion (Krippendorff, 2018). One of
the researchers coded the remaining 290 responses according to the established
code book.

###  5.1. Results

We collected anonymous responses from 390 students asking which question types
they preferred, both in the context of exams and practice. When asked about
the preferred type of question to appear on the exam, 163 (41%) students chose
micro Parsons problems, 138 (35%) chose single-line code writing questions,
and 89 (23%) students chose no preference. For practice, however, the most
popular response was no preference (n = 160, 41%), while 134 (34%) students
preferred single-line code writing questions, and 96 (25%) students preferred
micro Parsons problems.

Several themes emerged from students’ responses in explaining their choices
and comparing the two types of problems.

####  5.1.1. Perceived Difficulty

When comparing the two types of problems, 83 students (21%) explicitly noted
that they thought micro Parsons problems were easier. Among them, 13 students
felt that micro Parsons problems provided hints compared to the single-line
code writing question, especially when they did not know where to start:
“micro parsons are good to give you a head start on the topic and help you out
a bit, [single-line code writing] questions kind of mess you up if you don’t
know where to start.” (P308) Fifteen students felt that micro Parsons problems
were easier in exams because they provided partial credit for incorrect
responses.

Contrary to prior within-subjects think-aloud studies on Parsons problems
where all participants find them easier than writing code from scratch (Haynes
and Ericson, 2021), in our survey responses, 50 students (13%) felt that micro
Parsons problems are more difficult than single-line code writing problems. “I
found that the micro Parsons were slightly more difficult as I had to try to
think through the code with what was given instead of trying to think it
through on my own.” (P23) In our study, only micro Parsons problems contained
the fill-in-the-blank type of problems, which can also contribute to the
perceived difficulty.

####  5.1.2. Assisting Learning

Most students reported that they found both types of problems beneficial for
learning. Meanwhile, 36 students (9%) specifically pointed out that micro
Parsons are helpful for learning, because they struggle less and can practice
specific parts of a single line. They felt that micro Parsons problems provide
examples of how the code lines should look like, provide better feedback on
which part of the solution was incorrect, and require less memorization: “it
is more in line with how actual coding is: you do not have to memorize a lot
of stuff, you should know the concept and the ways to look for the
information.” (P98) On the other hand, 32 (8%) students explicitly mentioned
that the single-line code-writing questions helped them learn. They felt these
questions involve “actual coding” and shows what they actually know. They felt
that having less help could force them memorize better or apply more high-
level thinking.

Interestingly, learners’ comments on which variation they think better helps
learning do not always align with their selected preferences for homework or
exams. Many students explicitly said they preferred the practice exams to
“have the same format of questions” that would appear on the exam. (P159) For
the exam questions, students chose micro Parsons problems because they provide
extra credits when the answers are incorrect.

####  5.1.3. Confusing Distractors

A total of 40 students (10%) mentioned that micro Parsons problems could be
confusing at times. Specifically, 10 students offered complaints about the
distractors in the micro Parsons problems. Some suggested removing all
distractors, or providing a “use all blocks” prompt for those problems that do
not contain distractors: “Usually with the micro Parsons problems I tend to
overthink my answer and end up getting the question wrong. All the extra
blocks are confusing.” (P215) Most of the time, the distractors contain minor
problems that novices tend to make, such as the use of square brackets instead
of parentheses, or ignoring quotation marks around string literals, and minor
logic errors (e.g., and vs or). Four students explicitly expressed their
annoyance with these distractors: “it is extremely annoying when micro parsons
problems include wrong blocks because it only feels like they are trying to
trip me up based on reading errors.” (P292)

##  6\. Discussion

From our results we highlight and expand on three takeaways: 1) the ability of
micro Parsons problems to provide partial credit and simplified feedback; 2)
commentary on how distractor selection can in some cases significantly impact
item difficulty; 3) the expertise reversal effect and its potential impact on
the design of micro Parsons; and 4) the relationship between the presence of
distractors and confusion students encountered on micro Parsons problems. Each
of these serves to inform the design and use of micro Parsons as an item used
on exams and practice materials alike.

###  6.1. Feedback and Partial Credit with Micro Parsons Problems

Figure 5.  The feedback and partial credit mechanism used for our
implementation of micro Parsons.

As noted by students in the qualitative results, one of the key differences
between micro Parsons problems and single-line code writing problems is the
limited solution space enforced by requiring students to arrange predefined
blocks. As noted by Denny et al. (2008) – though in the context of traditional
Parsons problems – this enables more straightforward scoring for instructors.
However, a notable difference between the micro Parsons used in this study and
those used in the investigations of Denny et al. (2008) is our study uses an
online platform to deliver the questions whereas their used a paper based
exam. The ability to receive instantaneous feedback and make multiple attempts
allows forwards these benefits onto students as well both during the exam and
while practicing.

As noted by many students, the use of single-line code writing problems and
micro Parsons alike was well received as many students viewed as questions as
testing them on and allowing them to practice elements of larger programs in
isolation. Though it may be the case that micro Parsons problems have less
discriminating power relative to single-line code writing questions there are
other considerations that instructors may have beyond these metrics. By
reducing the error space and providing feedback that is easier to interpret
for this class of programming problem it is likely that the students’
experience is improved. This is particularly a consideration in the context of
practice exams, such as the ones our students encountered, as it scaffolds
their studying process.

###  6.2. Distractor Selection for Fill-in-the-Blank

As discussed in the quantitative results (Section 4), the fill-in-the-blank
micro Parsons (see fig. 1(b)) had much higher item-difficulties any other
micro Parsons problem or single line code writing question. In these
questions, students were presented with all the blocks needed to open a file
either using with-as or simply storing the result of open() in a variable. The
intention was for students to infer from the code they were completing that
with-as was not appropriate given it requires any manipulation of the file to
be done underneath it in an indented block. This problem highlights a unique
connection that can be made between the distractors chosen and the context of
the code the student is required to complete in micro Parsons problems for
languages where whitespace is semantic.

###  6.3. Potential Expertise Reversal Effect

From the student survey, we discovered that some students perceived micro
Parsons problems as more difficult than writing code from scratch. This
finding differs from prior work that included within-subject studies for
students to compare micro Parsons problems or regular Parsons problems with
writing code from scratch (Wu et al., 2023; Haynes and Ericson, 2021). One
potential explanation is the expertise reversal effect. It refers to the
situation where some instructional methods designed for novices become
inefficient or have negative consequences for advanced learners (Kalyuga,
2009). According to cognitive load theory, the instructional methods for
novices usually contain extra information and guidance. However, expert
learners already possess schema-based knowledge, which provides internal
guidance. Expert learners need to resolve and reconcile the two different sets
of guidance, which can introduce extra burdens in working memory.

As pointed out by some students in the qualitative analysis, constructing
solutions only by given blocks can sometimes be even more difficult when they
already have a plain-code answer in mind. The hints, in this case, the blocks,
can become extra guidance that is not needed by expert learners and cause a
burden for them. This is particularly a consideration when selecting
distractor blocks as the inclusion of plausible alternatives – and thus more
blocks – may increase the prevalence and impact of this effect. This may be
considered a desirable difficulty in practice materials for illustrating
alternative solutions. However, in summative assessments this added difficulty
may stem from the question format rather than the content being tested. This
finding highlights the careful consideration that should be taken when
selecting the number of blocks, and distractors, that are included in a
solution space.

###  6.4. Confusing Distractors

In our qualitative results, we discovered that some students found micro
Parsons problems confusing. The major complaint is the distractors. Only a few
students elaborated on their confusion, explaining that they felt that the
distractors were too tricky. There are two interpretations behind students’
perception of the trickiness of the distractors. First, students felt that
sometimes it was too difficult to identify the correct blocks versus the
distractors; It is also possible that the complaints reflected that they deem
the difficulty unnecessary.

Although distractors received complaints from some students, prior work in the
context of traditional Parsons problems suggests they may be useful for
learning. Prior work on proof blocks, a type of Parsons problem for
mathematical proofs has shown that learners who completed proof blocks with
distractors performed better on the post-test than those who completed proof
blocks without distractors, though the difference was not statistically
significant (Poulsen et al., 2023). In a multi-institutional multi-national
study, learners who practiced with distractors were also found to have better
performances on fixing code that contains similar errors in the distractors
than those who did not (Ericson et al., 2023). Considering the importance of
debugging and fixing code, especially with the development of generative AI
tools, it is likely distractors will become increasingly important for
learning.

Meanwhile, the visual presentation of the distractors can also affect
students’ perceptions. Similar to previous research on micro Parsons problems,
we only used jumbled distractors in this study. Because micro Parsons problems
require learners to rearrange code in one line, the design of interfaces for
visual grouping distractors with their correct alternatives is not as
straightforward. In our study, no student provided concrete examples of
distractors that they found confusing. Future research should investigate the
attributes that make distractors confusing for learners, and examine the
effective ways to add distractors that can reduce confusion while maintaining
item difficulty and item discrimination.

##  7\. Limitations and Future Work

There are several limitations to the results of our study that can affect its
generalizability. First, only a small number of micro Parsons were used
throughout the semester. More questions and question versions would need to be
created to provide a more complete and accurate picture of how micro Parsons
function as an exam item and how they compare to related text-entry code
writing questions.

Second, a limitation of the quantitative analysis of this work is that the
comparison of item-discrimination statistics between micro Parsons problems
and single-line code writing problems was at a categorical level. That is, we
did not create pairs of questions with identical solutions and compare them.
As such, our results comparing the item discrimination and difficulty
distributions are more suited to contextualize the results for micro Parsons
using a related code writing exercise.

A final limitation is students had access to practice exams which contained
both the single line code writing problem generators used on exams and micro
Parsons that covered related topics to those on their exams. This likely had
the impact of lowering the difficulty of the items compared to if fully novel
questions were used on the exams.

##  8\. Conclusion

This study employed micro Parsons problems as exam items across four exams,
and adopted measurement theory to analyze its effectiveness. We calculated the
item difficulty and item discrimination of micro Parsons problems as well as
traditional single-line code writing problems across the semester. In this
study, we found that micro Parsons questions are comparable in difficulty but
slightly lower in discrimination compared to writing code from scratch. We
also found insights that are worth future exploration from our student survey.
Students displayed diverse preferences when it came to micro Parsons problems
and single-line code questions in exam, largely based on their perceived
potential to earn more points. We found a potential expertise reversal effect
from student response that considered micro Parsons problems harder than
writing code from scratch, which motivates further inquiries into the ideal
design for micro Parsons problems used in exams.

## References

  * (1)
  * Baker (2001) Frank B Baker. 2001.  _The basics of item response theory_.  ERIC.
  * Barton and Lord (1981) Mark A Barton and Frederic M Lord. 1981.  An upper asymptote for the three-parameter logistic item-response model.  _ETS Research Report Series_ 1981, 1 (1981), i–8.
  * Bender et al. (2023) Jeff Bender, Bingpu Zhao, Alex Dziena, and Gail Kaiser. 2023.  Integrating Parsons puzzles within Scratch enables efficient computational thinking learning.  _Research and Practice in Technology Enhanced Learning_ 18 (2023), 022–022.
  * Birnbaum (1968) Allan Birnbaum. 1968.  Some latent trait models and their use in inferring an examinee’s ability.  _Statistical theories of mental test scores_ (1968).
  * De Champlain (2010) André F De Champlain. 2010.  A primer on classical test theory and item response theory for assessments in medical education.  _Medical education_ 44, 1 (2010), 109–117.
  * Denny et al. (2008) Paul Denny, Andrew Luxton-Reilly, and Beth Simon. 2008.  Evaluating a new exam question: Parsons problems. In _Proceedings of the Fourth International Workshop on Computing Education Research_ (Sydney, Australia) _(ICER ’08)_. Association for Computing Machinery, New York, NY, USA, 113–124.  https://doi.org/10.1145/1404520.1404532
  * Ebel and Frisbie (1972) Robert L Ebel and David A Frisbie. 1972.  _Essentials of educational measurement_.
  * Ericson et al. (2022) Barbara J. Ericson, Paul Denny, James Prather, Rodrigo Duran, Arto Hellas, Juho Leinonen, Craig S. Miller, Briana B. Morrison, Janice L. Pearce, and Susan H. Rodger. 2022.  Parsons Problems and Beyond: Systematic Literature Review and Empirical Study Designs. In _Proceedings of the 2022 Working Group Reports on Innovation and Technology in Computer Science Education_ (¡conf-loc¿, ¡city¿Dublin¡/city¿, ¡country¿Ireland¡/country¿, ¡/conf-loc¿) _(ITiCSE-WGR ’22)_. Association for Computing Machinery, New York, NY, USA, 191–234.  https://doi.org/10.1145/3571785.3574127
  * Ericson et al. (2018) Barbara J. Ericson, James D. Foley, and Jochen Rick. 2018.  Evaluating the Efficiency and Effectiveness of Adaptive Parsons Problems. In _Proceedings of the 2018 ACM Conference on International Computing Education Research_ (Espoo, Finland) _(ICER ’18)_. Association for Computing Machinery, New York, NY, USA, 60–68.  https://doi.org/10.1145/3230977.3231000
  * Ericson et al. (2015) Barbara J. Ericson, Mark J. Guzdial, and Briana B. Morrison. 2015.  Analysis of Interactive Features Designed to Enhance Learning in an Ebook. In _Proceedings of the Eleventh Annual International Conference on International Computing Education Research_ (Omaha, Nebraska, USA) _(ICER ’15)_. Association for Computing Machinery, New York, NY, USA, 169–178.  https://doi.org/10.1145/2787622.2787731
  * Ericson et al. (2023) Barbara J. Ericson, Janice L. Pearce, Susan H. Rodger, Andrew Csizmadia, Rita Garcia, Francisco J. Gutierrez, Konstantinos Liaskos, Aadarsh Padiyath, Michael James Scott, David H. Smith, Jayakrishnan M. Warriem, and Angela Zavaleta Bernuy. 2023.  Multi-Institutional Multi-National Studies of Parsons Problems.  In _Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education_. Association for Computing Machinery, New York, NY, USA, 57–107.  https://doi.org/10.1145/3623762.3633498
  * Fabic et al. (2019) Geela Venise Firmalo Fabic, Antonija Mitrovic, and Kourosh Neshatian. 2019.  Evaluation of Parsons problems with menu-based self-explanation prompts in a mobile python tutor.  _International Journal of Artificial Intelligence in Education_ 29 (2019), 507–535.
  * Haynes and Ericson (2021) Carl C. Haynes and Barbara J. Ericson. 2021.  Problem-Solving Efficiency and Cognitive Load for Adaptive Parsons Problems vs. Writing the Equivalent Code. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ (¡conf-loc¿, ¡city¿Yokohama¡/city¿, ¡country¿Japan¡/country¿, ¡/conf-loc¿) _(CHI ’21)_. Association for Computing Machinery, New York, NY, USA, Article 60, 15 pages.  https://doi.org/10.1145/3411764.3445292
  * Kalyuga (2009) Slava Kalyuga. 2009.  The expertise reversal effect.  In _Managing cognitive load in adaptive multimedia learning_. IGI Global, 58–80.
  * Krippendorff (2018) Klaus Krippendorff. 2018.  _Content analysis: An introduction to its methodology_.  Sage publications.
  * Kumar (2017) Amruth N. Kumar. 2017.  The Effect of Providing Motivational Support in Parsons Puzzle Tutors. In _Artificial Intelligence in Education_ , Elisabeth André, Ryan Baker, Xiangen Hu, Ma. Mercedes T. Rodrigo, and Benedict du Boulay (Eds.). Springer International Publishing, Cham, 528–531.
  * Morin and Kecskemety (2021) Brooke Morin and Krista M Kecskemety. 2021.  Collaborative Parsons Problems in a Remote-learning First-year Engineering Classroom. In _2021 ASEE Virtual Annual Conference Content Access_.
  * Morin et al. (2020) Brooke C Morin, Krista M Kecskemety, Kathleen A Harper, and Paul Alan Clingan. 2020.  Work in Progress: Parsons Problems as a Tool in the First-Year Engineering Classroom. In _2020 ASEE Virtual Annual Conference Content Access_.
  * Morrison et al. (2016) Briana B. Morrison, Lauren E. Margulieux, Barbara Ericson, and Mark Guzdial. 2016.  Subgoals Help Students Solve Parsons Problems. In _Proceedings of the 47th ACM Technical Symposium on Computing Science Education_ (Memphis, Tennessee, USA) _(SIGCSE ’16)_. Association for Computing Machinery, New York, NY, USA, 42–47.  https://doi.org/10.1145/2839509.2844617
  * Oyelere et al. (2019) Solomon Sunday Oyelere, Friday Joseph Agbo, Ismaila Temitayo Sanusi, Abdullahi Abubakar Yunusa, and Kissinger Sunday. 2019.  Impact of puzzle-based learning technique for programming education in Nigeria context. In _2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT)_ , Vol. 2161. IEEE, 239–241.
  * Parsons and Haden (2006) Dale Parsons and Patricia Haden. 2006.  Parson’s programming puzzles: a fun and effective learning tool for first programming courses. In _Proceedings of the 8th Australasian Conference on Computing Education-Volume 52_. 157–163.
  * Poulsen et al. (2023) Seth Poulsen, Hongxuan Chen, Yael Gertner, Benjamin Cosman, Matthew West, and Geoffrey L Herman. 2023.  Measuring the Impact of Distractors on Student Learning Gains while Using Proof Blocks.  _arXiv preprint arXiv:2311.00792_ (2023).
  * Poulsen et al. (2022) Seth Poulsen, Mahesh Viswanathan, Geoffrey L Herman, and Matthew West. 2022.  Evaluating proof blocks problems as exam questions.  _ACM Inroads_ 13, 1 (2022), 41–51.
  * Smith et al. (2023a) David H. Smith, Max Fowler, and Craig Zilles. 2023a.  Investigating the Role and Impact of Distractors on Parsons Problems in CS1 Assessments. In _Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1_ (¡conf-loc¿, ¡city¿Turku¡/city¿, ¡country¿Finland¡/country¿, ¡/conf-loc¿) _(ITiCSE 2023)_. Association for Computing Machinery, New York, NY, USA, 417–423.  https://doi.org/10.1145/3587102.3588819
  * Smith and Zilles (2023) David H. Smith and Craig Zilles. 2023.  Discovering, Autogenerating, and Evaluating Distractors for Python Parsons Problems in CS1. In _Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1_ (¡conf-loc¿, ¡city¿Toronto ON¡/city¿, ¡country¿Canada¡/country¿, ¡/conf-loc¿) _(SIGCSE 2023)_. Association for Computing Machinery, New York, NY, USA, 924–930.  https://doi.org/10.1145/3545945.3569801
  * Smith et al. (2023b) David H. Smith, IV, Seth Poulsen, Max Fowler, and Craig Zilles. 2023b.  Comparing the Impacts of Visually Grouped and Jumbled Distractors on Parsons Problems in CS1 Assessments. In _Proceedings of the ACM Conference on Global Computing Education Vol 1_ (¡conf-loc¿, ¡city¿Hyderabad¡/city¿, ¡country¿India¡/country¿, ¡/conf-loc¿) _(CompEd 2023)_. Association for Computing Machinery, New York, NY, USA, 154–160.  https://doi.org/10.1145/3576882.3617927
  * Smith IV (2023) David H. Smith IV. 2023.  Useful Distractions? Investigating the Utility of Distractors in Parsons Problems. In _Proceedings of the 2023 ACM Conference on International Computing Education Research - Volume 2_ (Chicago, IL, USA) _(ICER ’23)_. Association for Computing Machinery, New York, NY, USA, 62–63.  https://doi.org/10.1145/3568812.3603463
  * West et al. (2015) Matthew West, Geoffrey L Herman, and Craig Zilles. 2015.  Prairielearn: Mastery-based online problem solving with adaptive scoring and recommendations driven by machine learning. In _2015 ASEE Annual Conference & Exposition_. 26–1238.
  * Wu (2023) Zihan Wu. 2023.  Investigating the Effectiveness of Variations of Micro Parsons Problems. In _Proceedings of the 2023 ACM Conference on International Computing Education Research - Volume 2_ (Chicago, IL, USA) _(ICER ’23)_. Association for Computing Machinery, New York, NY, USA, 120–122.  https://doi.org/10.1145/3568812.3603447
  * Wu et al. (2021) Zihan Wu, Barbara Ericson, and Christopher Brooks. 2021.  Regex Parsons: Using Horizontal Parsons Problems to Scaffold Learning Regex. In _Proceedings of the 21st Koli Calling International Conference on Computing Education Research_ (Joensuu, Finland) _(Koli Calling ’21)_. Association for Computing Machinery, New York, NY, USA, Article 31, 3 pages.  https://doi.org/10.1145/3488042.3489968
  * Wu et al. (2023) Zihan Wu, Barbara J. Ericson, and Christopher Brooks. 2023.  Using Micro Parsons Problems to Scaffold the Learning of Regular Expressions. In _Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1_ (¡conf-loc¿, ¡city¿Turku¡/city¿, ¡country¿Finland¡/country¿, ¡/conf-loc¿) _(ITiCSE 2023)_. Association for Computing Machinery, New York, NY, USA, 457–463.  https://doi.org/10.1145/3587102.3588853