Redondo0000-0003-3737-4121, D.D. Redondo Ferrero0000-0002-3463-0559, L.
Romero, S. Sánchez Navas0000-0001-6129-9059, L. Urda Gómez0000-0002-7865-5010,
J. Vazquez Escobar0000-0002-7533-2283, C. Willmott

Universidad Autónoma de Madrid, Madrid, Spain J.F. de
Trocóniz0000-0002-0798-9806

Universidad de Oviedo, Instituto Universitario de Ciencias y Tecnologías
Espaciales de Asturias (ICTEA), Oviedo, Spain B. Alvarez
Gonzalez0000-0001-7767-4810, J. Cuevas0000-0001-5080-0821, J. Fernandez
Menendez0000-0002-5213-3708, S. Folgueras0000-0001-7191-1125, I. Gonzalez
Caballero0000-0002-8087-3199, J.R. González Fernández0000-0002-4825-8188, E.
Palencia Cortezon0000-0001-8264-0287, C. Ramón Álvarez0000-0003-1175-0002, V.
Rodríguez Bouza0000-0002-7225-7310, A. Soto Rodríguez0000-0002-2993-8663, A.
Trapote0000-0002-4030-2551, C. Vico Villalba0000-0002-1905-1874, P.
Vischia0000-0002-7088-8557

Instituto de Física de Cantabria (IFCA), CSIC-Universidad de Cantabria,
Santander, Spain S. Bhowmik0000-0003-1260-973X, S. Blanco
Fernández0000-0001-7301-0670, J.A. Brochero Cifuentes0000-0003-2093-7856, I.J.
Cabrillo0000-0002-0367-4022, A. Calderon0000-0002-7205-2040, J. Duarte
Campderros0000-0003-0687-5214, M. Fernandez0000-0002-4824-1087, C. Fernandez
Madrazo0000-0001-9748-4336, G. Gomez0000-0002-1077-6553, C. Lasaosa
García0000-0003-2726-7111, C. Martinez Rivero0000-0002-3224-956X, P. Martinez
Ruiz del Arbol0000-0002-7737-5121, F. Matorras0000-0003-4295-5668, P. Matorras
Cuevas0000-0001-7481-7273, E. Navarrete Ramos0000-0002-5180-4020, J. Piedra
Gomez0000-0002-9157-1700, L. Scodellaro0000-0002-4974-8330, I.
Vila0000-0002-6797-7209, J.M. Vizan Garcia0000-0002-6823-8854

University of Colombo, Colombo, Sri Lanka M.K. Jayananda0000-0002-7577-310X,
B. Kailasapathy590000-0003-2424-1303, D.U.J. Sonnadara0000-0001-7862-2537,
D.D.C. Wickramarathna0000-0002-6941-8478

University of Ruhuna, Department of Physics, Matara, Sri Lanka W.G.D.
Dharmaratna600000-0002-6366-837X, K. Liyanage0000-0002-3792-7665, N.
Perera0000-0002-4747-9106, N. Wickramage0000-0001-7760-3537

CERN, European Organization for Nuclear Research, Geneva, Switzerland D.
Abbaneo0000-0001-9416-1742, C. Amendola0000-0002-4359-836X, E.
Auffray0000-0001-8540-1097, G. Auzinger0000-0001-7077-8262, J. Baechler, D.
Barney0000-0002-4927-4921, A. Bermúdez Martínez0000-0001-8822-4727, M.
Bianco0000-0002-8336-3282, B. Bilin0000-0003-1439-7128, A.A. Bin
Anuar0000-0002-2988-9830, A. Bocci0000-0002-6515-5666, E.
Brondolin0000-0001-5420-586X, C. Caillol0000-0002-5642-3040, T.
Camporesi0000-0001-5066-1876, G. Cerminara0000-0002-2897-5753, N.
Chernyavskaya0000-0002-2264-2229, D. d’Enterria0000-0002-5754-4303, A.
Dabrowski0000-0003-2570-9676, A. David0000-0001-5854-7699, A. De
Roeck0000-0002-9228-5271, M.M. Defranchis0000-0001-9573-3714, M.
Deile0000-0001-5085-7270, M. Dobson0009-0007-5021-3230, F. Fallavollita61, L.
Forthomme0000-0002-3302-336X, G. Franzoni0000-0001-9179-4253, W.
Funk0000-0003-0422-6739, S. Giani, D. Gigi, K. Gill0009-0001-9331-5145, F.
Glege0000-0002-4526-2149, L. Gouskos0000-0002-9547-7471, M.
Haranko0000-0002-9376-9235, J. Hegeman0000-0002-2938-2263, B. Huber, V.
Innocente0000-0003-3209-2088, T. James0000-0002-3727-0202, P.
Janot0000-0001-7339-4272, J. Kieseler0000-0003-1644-7678, S.
Laurila0000-0001-7507-8636, P. Lecoq0000-0002-3198-0115, E.
Leutgeb0000-0003-4838-3306, C. Lourenço0000-0003-0885-6711, B.
Maier0000-0001-5270-7540, L. Malgeri0000-0002-0113-7389, M.
Mannelli0000-0003-3748-8946, A.C. Marini0000-0003-2351-0487, M. Matthewman, F.
Meijers0000-0002-6530-3657, S. Mersi0000-0003-2155-6692, E.
Meschi0000-0003-4502-6151, V. Milosevic0000-0002-1173-0696, F.
Moortgat0000-0001-7199-0046, M. Mulders0000-0001-7432-6634, S. Orfanelli, F.
Pantaleo0000-0003-3266-4357, M. Peruzzi0000-0002-0416-696X, G.
Petrucciani0000-0003-0889-4726, A. Pfeiffer0000-0001-5328-448X, M.
Pierini0000-0003-1939-4268, D. Piparo0009-0006-6958-3111, H.
Qu0000-0002-0250-8655, D. Rabady0000-0001-9239-0605, G. Reales Gutiérrez, M.
Rovere0000-0001-8048-1622, H. Sakulin0000-0003-2181-7258, S.
Scarfi0009-0006-8689-3576, C. Schwick, M. Selvaggi0000-0002-5144-9655, A.
Sharma0000-0002-9860-1650, K. Shchelina0000-0003-3742-0693, P.
Silva0000-0002-5725-041X, P. Sphicas620000-0002-5456-5977, A.G. Stahl
Leiton0000-0002-5397-252X, A. Steen0009-0006-4366-3463, S.
Summers0000-0003-4244-2061, D. Treille0009-0005-5952-9843, P.
Tropea0000-0003-1899-2266, A. Tsirou, D. Walter0000-0001-8584-9705, J.
Wanczyk630000-0002-8562-1863, K.A. Wozniak640000-0002-4395-1581, P.
Zehetner0009-0002-0555-4697, P. Zejdl0000-0001-9554-7815, W.D. Zeuner

Paul Scherrer Institut, Villigen, Switzerland T.
Bevilacqua650000-0001-9791-2353, L. Caminada650000-0001-5677-6033, A.
Ebrahimi0000-0003-4472-867X, W. Erdmann0000-0001-9964-249X, R.
Horisberger0000-0002-5594-1321, Q. Ingram0000-0002-9576-055X, H.C.
Kaestli0000-0003-1979-7331, D. Kotlinski0000-0001-5333-4918, C.
Lange0000-0002-3632-3157, M. Missiroli650000-0002-1780-1344, L.
Noehte650000-0001-6125-7203, T. Rohe0009-0005-6188-7754

ETH Zurich - Institute for Particle Physics and Astrophysics (IPA), Zurich,
Switzerland T.K. Aarrestad0000-0002-7671-243X, K.
Androsov630000-0003-2694-6542, M. Backhaus0000-0002-5888-2304, A.
Calandri0000-0001-7774-0099, C. Cazzaniga0000-0003-0001-7657, K.
Datta0000-0002-6674-0015, A. De Cosa0000-0003-2533-2856, G.
Dissertori0000-0002-4549-2569, M. Dittmar, M. Donegà0000-0001-9830-0412, F.
Eble0009-0002-0638-3447, M. Galli0000-0002-9408-4756, K.
Gedia0009-0006-0914-7684, F. Glessgen0000-0001-5309-1960, C.
Grab0000-0002-6182-3380, D. Hits0000-0002-3135-6427, W.
Lustermann0000-0003-4970-2217, A.-M. Lyon0009-0004-1393-6577, R.A.
Manzoni0000-0002-7584-5038, M. Marchegiani0000-0002-0389-8640, L.
Marchese0000-0001-6627-8716, C. Martin Perez0000-0003-1581-6152, A.
Mascellani630000-0001-6362-5356, F. Nessi-Tedaldi0000-0002-4721-7966, F.
Pauss0000-0002-3752-4639, V. Perovic0009-0002-8559-0531, S.
Pigazzini0000-0002-8046-4344, M.G. Ratti0000-0003-1777-7855, M.
Reichmann0000-0002-6220-5496, C. Reissel0000-0001-7080-1119, T.
Reitenspiess0000-0002-2249-0835, B. Ristic0000-0002-8610-1130, F.
Riti0000-0002-1466-9077, D. Ruini, D.A. Sanz Becerra0000-0002-6610-4019, R.
Seidita0000-0002-3533-6191, J. Steggemann630000-0003-4420-5510, D.
Valsecchi0000-0001-8587-8266, R. Wallny0000-0001-8038-1613

Universität Zürich, Zurich, Switzerland C. Amsler660000-0002-7695-501X, P.
Bärtschi0000-0002-8842-6027, C. Botta0000-0002-8072-795X, D. Brzhechko, M.F.
Canelli0000-0001-6361-2117, K. Cormier0000-0001-7873-3579, R. Del Burgo, J.K.
Heikkilä0000-0002-0538-1469, M. Huwiler0000-0002-9806-5907, W.
Jin0009-0009-8976-7702, A. Jofrehei0000-0002-8992-5426, B.
Kilminster0000-0002-6657-0407, S. Leontsinis0000-0002-7561-6091, S.P.
Liechti0000-0002-1192-1628, A. Macchiolo0000-0003-0199-6957, P.
Meiring0009-0001-9480-4039, V.M. Mikuni0000-0002-1579-2421, U.
Molinatti0000-0002-9235-3406, I. Neutelings0009-0002-6473-1403, A.
Reimers0000-0002-9438-2059, P. Robmann, S. Sanchez Cruz0000-0002-9991-195X, K.
Schweiger0000-0002-5846-3919, M. Senger0000-0002-1992-5711, Y.
Takahashi0000-0001-5184-2265, R. Tramontano0000-0001-5979-5299

National Central University, Chung-Li, Taiwan C. Adloff67, C.M. Kuo, W. Lin,
P.K. Rout0000-0001-8149-6180, P.C. Tiwari410000-0002-3667-3843, S.S.
Yu0000-0002-6011-8516

National Taiwan University (NTU), Taipei, Taiwan L. Ceard, Y.
Chao0000-0002-5976-318X, K.F. Chen0000-0003-1304-3782, P.s. Chen, Z.g. Chen,
W.-S. Hou0000-0002-4260-5118, T.h. Hsu, Y.w. Kao, R. Khurana, G.
Kole0000-0002-3285-1497, Y.y. Li0000-0003-3598-556X, R.-S.
Lu0000-0001-6828-1695, E. Paganis0000-0002-1950-8993, A. Psallidas, X.f.
Su0009-0009-0207-4904, J. Thomas-Wilsker0000-0003-1293-4153, H.y. Wu, E.
Yazgan0000-0001-5732-7950

High Energy Physics Research Unit, Department of Physics, Faculty of Science,
Chulalongkorn University, Bangkok, Thailand C.
Asawatangtrakuldee0000-0003-2234-7219, N. Srimanobhas0000-0003-3563-2959, V.
Wachirapusitanand0000-0001-8251-5160

Çukurova University, Physics Department, Science and Art Faculty, Adana,
Turkey D. Agyel0000-0002-1797-8844, F. Boran0000-0002-3611-390X, Z.S.
Demiroglu0000-0001-7977-7127, F. Dolek0000-0001-7092-5517, I.
Dumanoglu680000-0002-0039-5503, E. Eskut0000-0001-8328-3314, Y.
Guler690000-0001-7598-5252, E. Gurpinar Guler690000-0002-6172-0285, C.
Isik0000-0002-7977-0811, O. Kara, A. Kayis Topaksu0000-0002-3169-4573, U.
Kiminsu0000-0001-6940-7800, G. Onengut0000-0002-6274-4254, K.
Ozdemir700000-0002-0103-1488, A. Polatoz0000-0001-9516-0821, B.
Tali710000-0002-7447-5602, U.G. Tok0000-0002-3039-021X, S.
Turkcapar0000-0003-2608-0494, E. Uslan0000-0002-2472-0526, I.S.
Zorbakir0000-0002-5962-2221

Middle East Technical University, Physics Department, Ankara, Turkey M.
Yalvac720000-0003-4915-9162

Bogazici University, Istanbul, Turkey B. Akgun0000-0001-8888-3562, I.O.
Atakisi0000-0002-9231-7464, E. Gülmez0000-0002-6353-518X, M.
Kaya730000-0003-2890-4493, O. Kaya740000-0002-8485-3822, S.
Tekten750000-0002-9624-5525

Istanbul Technical University, Istanbul, Turkey A. Cakir0000-0002-8627-7689,
K. Cankocak68,760000-0002-3829-3481, Y. Komurcu0000-0002-7084-030X, S.
Sen770000-0001-7325-1087

Istanbul University, Istanbul, Turkey O. Aydilek0000-0002-2567-6766, S.
Cerci710000-0002-8702-6152, V. Epshteyn0000-0002-8863-6374, B.
Hacisahinoglu0000-0002-2646-1230, I. Hos780000-0002-7678-1101, B.
Isildak790000-0002-0283-5234, B. Kaynak0000-0003-3857-2496, S.
Ozkorucuklu0000-0001-5153-9266, O. Potok0009-0005-1141-6401, H.
Sert0000-0003-0716-6727, C. Simsek0000-0002-7359-8635, D. Sunar
Cerci710000-0002-5412-4688, C. Zorbilmez0000-0002-5199-061X

Institute for Scintillation Materials of National Academy of Science of
Ukraine, Kharkiv, Ukraine A. Boyaryntsev0000-0001-9252-0430, B.
Grynyov0000-0003-1700-0173

National Science Centre, Kharkiv Institute of Physics and Technology, Kharkiv,
Ukraine L. Levchuk0000-0001-5889-7410

University of Bristol, Bristol, United Kingdom D. Anthony0000-0002-5016-8886,
J.J. Brooke0000-0003-2529-0684, A. Bundock0000-0002-2916-6456, F.
Bury0000-0002-3077-2090, E. Clement0000-0003-3412-4004, D.
Cussans0000-0001-8192-0826, H. Flacher0000-0002-5371-941X, M. Glowacki, J.
Goldstein0000-0003-1591-6014, H.F. Heath0000-0001-6576-9740, L.
Kreczko0000-0003-2341-8330, B. Krikler0000-0001-9712-0030, S.
Paramesvaran0000-0003-4748-8296, S. Seif El Nasr-Storey, V.J.
Smith0000-0003-4543-2547, N. Stylianou800000-0002-0113-6829, K. Walkingshaw
Pass, R. White0000-0001-5793-526X

Rutherford Appleton Laboratory, Didcot, United Kingdom A.H. Ball, K.W.
Bell0000-0002-2294-5860, A. Belyaev810000-0002-1733-4408, C.
Brew0000-0001-6595-8365, R.M. Brown0000-0002-6728-0153, D.J.A.
Cockerill0000-0003-2427-5765, C. Cooke0000-0003-3730-4895, K.V. Ellis, K.
Harder0000-0002-2965-6973, S. Harper0000-0001-5637-2653, M.-L.
Holmberg820000-0002-9473-5985, J. Linacre0000-0001-7555-652X, K. Manolopoulos,
D.M. Newbold0000-0002-9015-9634, E. Olaiya, D. Petyt0000-0002-2369-4469, T.
Reis0000-0003-3703-6624, G. Salvi0000-0002-2787-1063, T. Schuh, C.H. Shepherd-
Themistocleous0000-0003-0551-6949, I.R. Tomalin0000-0003-2419-4439, T.
Williams0000-0002-8724-4678

Imperial College, London, United Kingdom R. Bainbridge0000-0001-9157-4832, P.
Bloch0000-0001-6716-979X, C.E. Brown0000-0002-7766-6615, O. Buchmuller, V.
Cacchio, C.A. Carrillo Montoya0000-0002-6245-6535, G.S.
Chahal830000-0003-0320-4407, D. Colling0000-0001-9959-4977, J.S. Dancu, P.
Dauncey0000-0001-6839-9466, G. Davies0000-0001-8668-5001, J. Davies, M. Della
Negra0000-0001-6497-8081, S. Fayer, G. Fedi0000-0001-9101-2573, G.
Hall0000-0002-6299-8385, M.H. Hassanshahi0000-0001-6634-4517, A. Howard, G.
Iles0000-0002-1219-5859, M. Knight0009-0008-1167-4816, J.
Langford0000-0002-3931-4379, L. Lyons0000-0001-7945-9188, A.-M.
Magnan0000-0002-4266-1646, S. Malik, A. Martelli0000-0003-3530-2255, M.
Mieskolainen0000-0001-8893-7401, J. Nash840000-0003-0607-6519, M.
Pesaresi0000-0002-9759-1083, B.C. Radburn-Smith0000-0003-1488-9675, A.
Richards, A. Rose0000-0002-9773-550X, C. Seez0000-0002-1637-5494, R.
Shukla0000-0001-5670-5497, A. Tapper0000-0003-4543-864X, K.
Uchida0000-0003-0742-2276, G.P. Uttley0009-0002-6248-6467, L.H. Vage, T.
Virdee310000-0001-7429-2198, M. Vojinovic0000-0001-8665-2808, N.
Wardle0000-0003-1344-3356, D. Winterbottom0000-0003-4582-150X

Brunel University, Uxbridge, United Kingdom K. Coldham, J.E.
Cole0000-0001-5638-7599, A. Khan, P. Kyberd0000-0002-7353-7090, I.D.
Reid0000-0002-9235-779X

Baylor University, Waco, Texas, USA S. Abdullin0000-0003-4885-6935, A.
Brinkerhoff0000-0002-4819-7995, B. Caraway0000-0002-6088-2020, J.
Dittmann0000-0002-1911-3158, K. Hatakeyama0000-0002-6012-2451, J.
Hiltbrand0000-0003-1691-5937, A.R. Kanuganti0000-0002-0789-1200, B.
McMaster0000-0002-4494-0446, M. Saunders0000-0003-1572-9075, S.
Sawant0000-0002-1981-7753, C. Sutantawibul0000-0003-0600-0151, M.
Toms850000-0002-7703-3973, J. Wilson0000-0002-5672-7394

Catholic University of America, Washington, DC, USA R.
Bartek0000-0002-1686-2882, A. Dominguez0000-0002-7420-5493, C. Huerta
Escamilla, A.E. Simsek0000-0002-9074-2256, R. Uniyal0000-0001-7345-6293, A.M.
Vargas Hernandez0000-0002-8911-7197

The University of Alabama, Tuscaloosa, Alabama, USA R.
Chudasama0009-0007-8848-6146, S.I. Cooper0000-0002-4618-0313, S.V.
Gleyzer0000-0002-6222-8102, C.U. Perez0000-0002-6861-2674, P.
Rumerio860000-0002-1702-5541, E. Usai0000-0001-9323-2107, C.
West0000-0003-4460-2241, R. Yi0000-0001-5818-1682

Boston University, Boston, Massachusetts, USA A. Akpinar0000-0001-7510-6617,
A. Albert0000-0003-2369-9507, D. Arcaro0000-0001-9457-8302, C.
Cosby0000-0003-0352-6561, Z. Demiragli0000-0001-8521-737X, C.
Erice0000-0002-6469-3200, E. Fontanesi0000-0002-0662-5904, D.
Gastler0009-0000-7307-6311, S. Jeon0000-0003-1208-6940, J.
Rohlf0000-0001-6423-9799, K. Salyer0000-0002-6957-1077, D.
Sperka0000-0002-4624-2019, D. Spitzbart0000-0003-2025-2742, I.
Suarez0000-0002-5374-6995, A. Tsatsos0000-0001-8310-8911, S.
Yuan0000-0002-2029-024X

Brown University, Providence, Rhode Island, USA G. Benelli0000-0003-4461-8905,
X. Coubez26, D. Cutts0000-0003-1041-7099, M. Hadley0000-0002-7068-4327, U.
Heintz0000-0002-7590-3058, J.M. Hogan870000-0002-8604-3452, T.
Kwon0000-0001-9594-6277, G. Landsberg0000-0002-4184-9380, K.T.
Lau0000-0003-1371-8575, D. Li0000-0003-0890-8948, J. Luo0000-0002-4108-8681,
S. Mondal0000-0003-0153-7590, M.
Narain${}^{\textrm{\textdagger}}$0000-0002-7857-7403, N.
Pervan0000-0002-8153-8464, S. Sagir880000-0002-2614-5860, F.
Simpson0000-0001-8944-9629, M. Stamenkovic0000-0003-2251-0610, W.Y. Wong, X.
Yan0000-0002-6426-0560, W. Zhang

University of California, Davis, Davis, California, USA S.
Abbott0000-0002-7791-894X, J. Bonilla0000-0002-6982-6121, C.
Brainerd0000-0002-9552-1006, R. Breedon0000-0001-5314-7581, M. Calderon De La
Barca Sanchez0000-0001-9835-4349, M. Chertok0000-0002-2729-6273, M.
Citron0000-0001-6250-8465, J. Conway0000-0003-2719-5779, P.T.
Cox0000-0003-1218-2828, R. Erbacher0000-0001-7170-8944, F.
Jensen0000-0003-3769-9081, O. Kukral0009-0007-3858-6659, G.
Mocellin0000-0002-1531-3478, M. Mulhearn0000-0003-1145-6436, D.
Pellett0009-0000-0389-8571, W. Wei0000-0003-4221-1802, Y.
Yao0000-0002-5990-4245, F. Zhang0000-0002-6158-2468

University of California, Los Angeles, California, USA M.
Bachtis0000-0003-3110-0701, R. Cousins0000-0002-5963-0467, A.
Datta0000-0003-2695-7719, J. Hauser0000-0002-9781-4873, M.
Ignatenko0000-0001-8258-5863, M.A. Iqbal0000-0001-8664-1949, T.
Lam0000-0002-0862-7348, E. Manca0000-0001-8946-655X, W.A.
Nash0009-0004-3633-8967, D. Saltzberg0000-0003-0658-9146, B.
Stone0000-0002-9397-5231, V. Valuev0000-0002-0783-6703

University of California, Riverside, Riverside, California, USA R.
Clare0000-0003-3293-5305, M. Gordon, G. Hanson0000-0002-7273-4009, W.
Si0000-0002-5879-6326, S.
Wimpenny${}^{\textrm{\textdagger}}$0000-0003-0505-4908

University of California, San Diego, La Jolla, California, USA J.G.
Branson0009-0009-5683-4614, S. Cittolin0000-0002-0922-9587, S.
Cooperstein0000-0003-0262-3132, D. Diaz0000-0001-6834-1176, J.
Duarte0000-0002-5076-7096, L. Giannini0000-0002-5621-7706, J.
Guiang0000-0002-2155-8260, R. Kansal0000-0003-2445-1060, V.
Krutelyov0000-0002-1386-0232, R. Lee0009-0000-4634-0797, J.
Letts0000-0002-0156-1251, M. Masciovecchio0000-0002-8200-9425, F.
Mokhtar0000-0003-2533-3402, M. Pieri0000-0003-3303-6301, M.
Quinnan0000-0003-2902-5597, B.V. Sathia Narayanan0000-0003-2076-5126, V.
Sharma0000-0003-1736-8795, M. Tadel0000-0001-8800-0045, E.
Vourliotis0000-0002-2270-0492, F. Würthwein0000-0001-5912-6124, Y.
Xiang0000-0003-4112-7457, A. Yagil0000-0002-6108-4004

University of California, Santa Barbara - Department of Physics, Santa
Barbara, California, USA A. Barzdukas0000-0002-0518-3286, L.
Brennan0000-0003-0636-1846, C. Campagnari0000-0002-8978-8177, G.
Collura0000-0002-4160-1844, A. Dorsett0000-0001-5349-3011, J.
Incandela0000-0001-9850-2030, M. Kilpatrick0000-0002-2602-0566, J.
Kim0000-0002-2072-6082, A.J. Li0000-0002-3895-717X, P.
Masterson0000-0002-6890-7624, H. Mei0000-0002-9838-8327, M.
Oshiro0000-0002-2200-7516, J. Richman0000-0002-5189-146X, U.
Sarica0000-0002-1557-4424, R. Schmitz0000-0003-2328-677X, F.
Setti0000-0001-9800-7822, J. Sheplock0000-0002-8752-1946, D.
Stuart0000-0002-4965-0747, S. Wang0000-0001-7887-1728

California Institute of Technology, Pasadena, California, USA A.
Bornheim0000-0002-0128-0871, O. Cerri, A. Latorre, J.M.
Lawhorn0000-0002-8597-9259, J. Mao0009-0002-8988-9987, H.B.
Newman0000-0003-0964-1480, T. Q. Nguyen0000-0003-3954-5131, M.
Spiropulu0000-0001-8172-7081, J.R. Vlimant0000-0002-9705-101X, C.
Wang0000-0002-0117-7196, S. Xie0000-0003-2509-5731, R.Y.
Zhu0000-0003-3091-7461

Carnegie Mellon University, Pittsburgh, Pennsylvania, USA J.
Alison0000-0003-0843-1641, S. An0000-0002-9740-1622, M.B.
Andrews0000-0001-5537-4518, P. Bryant0000-0001-8145-6322, V.
Dutta0000-0001-5958-829X, T. Ferguson0000-0001-5822-3731, A.
Harilal0000-0001-9625-1987, C. Liu0000-0002-3100-7294, T.
Mudholkar0000-0002-9352-8140, S. Murthy0000-0002-1277-9168, M.
Paulini0000-0002-6714-5787, A. Roberts0000-0002-5139-0550, A.
Sanchez0000-0002-5431-6989, W. Terrill0000-0002-2078-8419

University of Colorado Boulder, Boulder, Colorado, USA J.P.
Cumalat0000-0002-6032-5857, W.T. Ford0000-0001-8703-6943, A.
Hassani0009-0008-4322-7682, G. Karathanasis0000-0001-5115-5828, E. MacDonald,
N. Manganelli0000-0002-3398-4531, F. Marini0000-0002-2374-6433, A.
Perloff0000-0001-5230-0396, C. Savard0009-0000-7507-0570, N.
Schonbeck0009-0008-3430-7269, K. Stenson0000-0003-4888-205X, K.A.
Ulmer0000-0001-6875-9177, S.R. Wagner0000-0002-9269-5772, N.
Zipper0000-0002-4805-8020

Cornell University, Ithaca, New York, USA J. Alexander0000-0002-2046-342X, S.
Bright-Thonney0000-0003-1889-7824, X. Chen0000-0002-8157-1328, D.J.
Cranshaw0000-0002-7498-2129, J. Fan0009-0003-3728-9960, X.
Fan0000-0003-2067-0127, D. Gadkari0000-0002-6625-8085, S.
Hogan0000-0003-3657-2281, J. Monroy0000-0002-7394-4710, J.R.
Patterson0000-0002-3815-3649, J. Reichert0000-0003-2110-8021, M.
Reid0000-0001-7706-1416, A. Ryd0000-0001-5849-1912, J.
Thom0000-0002-4870-8468, P. Wittich0000-0002-7401-2181, R.
Zou0000-0002-0542-1264

Fermi National Accelerator Laboratory, Batavia, Illinois, USA M.
Albrow0000-0001-7329-4925, M. Alyari0000-0001-9268-3360, O.
Amram0000-0002-3765-3123, G. Apollinari0000-0002-5212-5396, A.
Apresyan0000-0002-6186-0130, L.A.T. Bauerdick0000-0002-7170-9012, D.
Berry0000-0002-5383-8320, J. Berryhill0000-0002-8124-3033, P.C.
Bhat0000-0003-3370-9246, K. Burkett0000-0002-2284-4744, J.N.
Butler0000-0002-0745-8618, A. Canepa0000-0003-4045-3998, G.B.
Cerati0000-0003-3548-0262, H.W.K. Cheung0000-0001-6389-9357, F.
Chlebana0000-0002-8762-8559, G. Cummings0000-0002-8045-7806, J.
Dickinson0000-0001-5450-5328, I. Dutta0000-0003-0953-4503, V.D.
Elvira0000-0003-4446-4395, Y. Feng0000-0003-2812-338X, J.
Freeman0000-0002-3415-5671, A. Gandrakota0000-0003-4860-3233, Z.
Gecse0009-0009-6561-3418, L. Gray0000-0002-6408-4288, D. Green, A.
Grummer0000-0003-2752-1183, S. Grünendahl0000-0002-4857-0294, D.
Guerrero0000-0001-5552-5400, O. Gutsche0000-0002-8015-9622, R.M.
Harris0000-0003-1461-3425, R. Heller0000-0002-7368-6723, T.C.
Herwig0000-0002-4280-6382, J. Hirschauer0000-0002-8244-0805, L.
Horyn0000-0002-9512-4932, B. Jayatilaka0000-0001-7912-5612, S.
Jindariani0009-0000-7046-6533, M. Johnson0000-0001-7757-8458, U.
Joshi0000-0001-8375-0760, T. Klijnsma0000-0003-1675-6040, B.
Klima0000-0002-3691-7625, K.H.M. Kwok0000-0002-8693-6146, S.
Lammel0000-0003-0027-635X, D. Lincoln0000-0002-0599-7407, R.
Lipton0000-0002-6665-7289, T. Liu0009-0007-6522-5605, C.
Madrid0000-0003-3301-2246, K. Maeshima0009-0000-2822-897X, C.
Mantilla0000-0002-0177-5903, D. Mason0000-0002-0074-5390, P.
McBride0000-0001-6159-7750, P. Merkel0000-0003-4727-5442, S.
Mrenna0000-0001-8731-160X, S. Nahn0000-0002-8949-0178, J.
Ngadiuba0000-0002-0055-2935, D. Noonan0000-0002-3932-3769, V.
Papadimitriou0000-0002-0690-7186, N. Pastika0009-0006-0993-6245, K.
Pedro0000-0003-2260-9151, C. Pena890000-0002-4500-7930, F.
Ravera0000-0003-3632-0287, A. Reinsvold Hall900000-0003-1653-8553, L.
Ristori0000-0003-1950-2492, E. Sexton-Kennedy0000-0001-9171-1980, N.
Smith0000-0002-0324-3054, A. Soha0000-0002-5968-1192, L.
Spiegel0000-0001-9672-1328, S. Stoynev0000-0003-4563-7702, J.
Strait0000-0002-7233-8348, L. Taylor0000-0002-6584-2538, S.
Tkaczyk0000-0001-7642-5185, N.V. Tran0000-0002-8440-6854, L.
Uplegger0000-0002-9202-803X, E.W. Vaandering0000-0003-3207-6950, I.
Zoi0000-0002-5738-9446

University of Florida, Gainesville, Florida, USA C. Aruta0000-0001-9524-3264,
P. Avery0000-0003-0609-627X, D. Bourilkov0000-0003-0260-4935, L.
Cadamuro0000-0001-8789-610X, P. Chang0000-0002-2095-6320, V.
Cherepanov0000-0002-6748-4850, R.D. Field, E. Koenig0000-0002-0884-7922, M.
Kolosova0000-0002-5838-2158, J. Konigsberg0000-0001-6850-8765, A.
Korytov0000-0001-9239-3398, K.H. Lo, K. Matchev0000-0003-4182-9096, N.
Menendez0000-0002-3295-3194, G. Mitselmakher0000-0001-5745-3658, A.
Muthirakalayil Madhu0000-0003-1209-3032, N. Rawal0000-0002-7734-3170, D.
Rosenzweig0000-0002-3687-5189, S. Rosenzweig0000-0002-5613-1507, K.
Shi0000-0002-2475-0055, J. Wang0000-0003-3879-4873

Florida State University, Tallahassee, Florida, USA T.
Adams0000-0001-8049-5143, A. Al Kadhim0000-0003-3490-8407, A.
Askew0000-0002-7172-1396, N. Bower0000-0001-8775-0696, R.
Habibullah0000-0002-3161-8300, V. Hagopian0000-0002-3791-1989, R.
Hashmi0000-0002-5439-8224, R.S. Kim0000-0002-8645-186X, S.
Kim0000-0003-2381-5117, T. Kolberg0000-0002-0211-6109, G. Martinez, H.
Prosper0000-0002-4077-2713, P.R. Prova, O. Viazlo0000-0002-2957-0301, M.
Wulansatiti0000-0001-6794-3079, R. Yohay0000-0002-0124-9065, J. Zhang

Florida Institute of Technology, Melbourne, Florida, USA B. Alsufyani, M.M.
Baarmand0000-0002-9792-8619, S. Butalla0000-0003-3423-9581, T.
Elkafrawy540000-0001-9930-6445, M. Hohlmann0000-0003-4578-9319, R. Kumar
Verma0000-0002-8264-156X, M. Rahmani

University of Illinois Chicago, Chicago, USA, Chicago, USA M.R.
Adams0000-0001-8493-3737, C. Bennett, R. Cavanaugh0000-0001-7169-3420, S.
Dittmer0000-0002-5359-9614, R. Escobar Franco0000-0003-2090-5010, O.
Evdokimov0000-0002-1250-8931, C.E. Gerber0000-0002-8116-9021, D.J.
Hofman0000-0002-2449-3845, J.h. Lee0000-0002-5574-4192, D. S.
Lemos0000-0003-1982-8978, A.H. Merrit0000-0003-3922-6464, C.
Mills0000-0001-8035-4818, S. Nanda0000-0003-0550-4083, G.
Oh0000-0003-0744-1063, B. Ozek0009-0000-2570-1100, D.
Pilipovic0000-0002-4210-2780, T. Roy0000-0001-7299-7653, S.
Rudrabhatla0000-0002-7366-4225, M.B. Tonjes0000-0002-2617-9315, N.
Varelas0000-0002-9397-5514, X. Wang0000-0003-2792-8493, Z.
Ye0000-0001-6091-6772, J. Yoo0000-0002-3826-1332

The University of Iowa, Iowa City, Iowa, USA M. Alhusseini0000-0002-9239-470X,
D. Blend, K. Dilsiz910000-0003-0138-3368, L. Emediato0000-0002-3021-5032, G.
Karaman0000-0001-8739-9648, O.K. Köseyan0000-0001-9040-3468, J.-P. Merlo, A.
Mestvirishvili920000-0002-8591-5247, J. Nachtman0000-0003-3951-3420, O. Neogi,
H. Ogul930000-0002-5121-2893, Y. Onel0000-0002-8141-7769, A.
Penzo0000-0003-3436-047X, C. Snyder, E. Tiras940000-0002-5628-7464

Johns Hopkins University, Baltimore, Maryland, USA B.
Blumenfeld0000-0003-1150-1735, L. Corcodilos0000-0001-6751-3108, J.
Davis0000-0001-6488-6195, A.V. Gritsan0000-0002-3545-7970, L.
Kang0000-0002-0941-4512, S. Kyriacou0000-0002-9254-4368, P.
Maksimovic0000-0002-2358-2168, M. Roguljic0000-0001-5311-3007, J.
Roskes0000-0001-8761-0490, S. Sekhar0000-0002-8307-7518, M.
Swartz0000-0002-0286-5070, T.Á. Vámi0000-0002-0959-9211

The University of Kansas, Lawrence, Kansas, USA A. Abreu0000-0002-9000-2215,
L.F. Alcerro Alcerro0000-0001-5770-5077, J. Anguiano0000-0002-7349-350X, P.
Baringer0000-0002-3691-8388, A. Bean0000-0001-5967-8674, Z.
Flowers0000-0001-8314-2052, D. Grove0000-0002-0740-2462, J.
King0000-0001-9652-9854, G. Krintiras0000-0002-0380-7577, M.
Lazarovits0000-0002-5565-3119, C. Le Mahieu0000-0001-5924-1130, C. Lindsey, J.
Marquez0000-0003-3887-4048, N. Minafra0000-0003-4002-1888, M.
Murray0000-0001-7219-4818, M. Nickel0000-0003-0419-1329, M.
Pitt0000-0003-2461-5985, S. Popescu950000-0002-0345-2171, C.
Rogan0000-0002-4166-4503, C. Royon0000-0002-7672-9709, R.
Salvatico0000-0002-2751-0567, S. Sanders0000-0002-9491-6022, C.
Smith0000-0003-0505-0528, Q. Wang0000-0003-3804-3244, G.
Wilson0000-0003-0917-4763

Kansas State University, Manhattan, Kansas, USA B. Allmond0000-0002-5593-7736,
A. Ivanov0000-0002-9270-5643, K. Kaadze0000-0003-0571-163X, A.
Kalogeropoulos0000-0003-3444-0314, D. Kim, Y. Maravin0000-0002-9449-0666, K.
Nam, J. Natoli0000-0001-6675-3564, D. Roy0000-0002-8659-7762, G.
Sorrentino0000-0002-2253-819X

Lawrence Livermore National Laboratory, Livermore, California, USA F.
Rebassoo0000-0001-8934-9329, D. Wright0000-0002-3586-3354

University of Maryland, College Park, Maryland, USA E.
Adams0000-0003-2809-2683, A. Baden0000-0002-6159-3861, O. Baron, A.
Belloni0000-0002-1727-656X, A. Bethani0000-0002-8150-7043, Y.M.
Chen0000-0002-5795-4783, S.C. Eno0000-0003-4282-2515, N.J.
Hadley0000-0002-1209-6471, S. Jabeen0000-0002-0155-7383, R.G.
Kellogg0000-0001-9235-521X, T. Koeth0000-0002-0082-0514, Y.
Lai0000-0002-7795-8693, S. Lascio0000-0001-8579-5874, A.C.
Mignerey0000-0001-5164-6969, S. Nabili0000-0002-6893-1018, C.
Palmer0000-0002-5801-5737, C. Papageorgakis0000-0003-4548-0346, M.M. Paranjpe,
L. Wang0000-0003-3443-0626, K. Wong0000-0002-9698-1354

Massachusetts Institute of Technology, Cambridge, Massachusetts, USA J.
Bendavid0000-0002-7907-1789, W. Busza0000-0002-3831-9071, I.A.
Cali0000-0002-2822-3375, Y. Chen0000-0003-2582-6469, M.
D’Alfonso0000-0002-7409-7904, J. Eysermans0000-0001-6483-7123, C.
Freer0000-0002-7967-4635, G. Gomez-Ceballos0000-0003-1683-9460, M. Goncharov,
P. Harris, D. Hoang, D. Kovalskyi0000-0002-6923-293X, J.
Krupa0000-0003-0785-7552, L. Lavezzo0000-0002-1364-9920, Y.-J.
Lee0000-0003-2593-7767, K. Long0000-0003-0664-1653, C.
Mironov0000-0002-8599-2437, C. Paus0000-0002-6047-4211, D.
Rankin0000-0001-8411-9620, C. Roland0000-0002-7312-5854, G.
Roland0000-0001-8983-2169, S. Rothman0000-0002-1377-9119, Z.
Shi0000-0001-5498-8825, G.S.F. Stephans0000-0003-3106-4894, J. Wang, Z.
Wang0000-0002-3074-3767, B. Wyslouch0000-0003-3681-0649, T. J.
Yang0000-0003-4317-4660

University of Minnesota, Minneapolis, Minnesota, USA B.
Crossman0000-0002-2700-5085, B.M. Joshi0000-0002-4723-0968, C.
Kapsiak0009-0008-7743-5316, M. Krohn0000-0002-1711-2506, D.
Mahon0000-0002-2640-5941, J. Mans0000-0003-2840-1087, B.
Marzocchi0000-0001-6687-6214, S. Pandey0000-0003-0440-6019, M.
Revering0000-0001-5051-0293, R. Rusack0000-0002-7633-749X, R.
Saradhy0000-0001-8720-293X, N. Schroeder0000-0002-8336-6141, N.
Strobbe0000-0001-8835-8282, M.A. Wadud0000-0002-0653-0761

University of Mississippi, Oxford, Mississippi, USA L.M.
Cremaldi0000-0001-5550-7827

University of Nebraska-Lincoln, Lincoln, Nebraska, USA K.
Bloom0000-0002-4272-8900, M. Bryson, D.R. Claes0000-0003-4198-8919, C.
Fangmeier0000-0002-5998-8047, F. Golf0000-0003-3567-9351, G.
Haza0009-0001-1326-3956, J. Hossain0000-0001-5144-7919, C.
Joo0000-0002-5661-4330, I. Kravchenko0000-0003-0068-0395, I.
Reed0000-0002-1823-8856, J.E. Siado0000-0002-9757-470X, W.
Tabb0000-0002-9542-4847, A. Vagnerini0000-0001-8730-5031, A.
Wightman0000-0001-6651-5320, F. Yan0000-0002-4042-0785, D.
Yu0000-0001-5921-5231, A.G. Zecchinelli0000-0001-8986-278X

State University of New York at Buffalo, Buffalo, New York, USA G.
Agarwal0000-0002-2593-5297, H. Bandyopadhyay0000-0001-9726-4915, L.
Hay0000-0002-7086-7641, I. Iashvili0000-0003-1948-5901, A.
Kharchilava0000-0002-3913-0326, C. McLean0000-0002-7450-4805, M.
Morris0000-0002-2830-6488, D. Nguyen0000-0002-5185-8504, S.
Rappoccio0000-0002-5449-2560, H. Rejeb Sfar, A. Williams0000-0003-4055-6532

Northeastern University, Boston, Massachusetts, USA G.
Alverson0000-0001-6651-1178, E. Barberis0000-0002-6417-5913, Y.
Haddad0000-0003-4916-7752, Y. Han0000-0002-3510-6505, A.
Krishna0000-0002-4319-818X, J. Li0000-0001-5245-2074, M.
Lu0000-0002-6999-3931, G. Madigan0000-0001-8796-5865, D.M.
Morse0000-0003-3163-2169, V. Nguyen0000-0003-1278-9208, T.
Orimoto0000-0002-8388-3341, A. Parker0000-0002-9421-3335, L.
Skinnari0000-0002-2019-6755, A. Tishelman-Charny0000-0002-7332-5098, B.
Wang0000-0003-0796-2475, D. Wood0000-0002-6477-801X

Northwestern University, Evanston, Illinois, USA S.
Bhattacharya0000-0002-0526-6161, J. Bueghly, Z. Chen0000-0003-4521-6086, K.A.
Hahn0000-0001-7892-1676, Y. Liu0000-0002-5588-1760, Y.
Miao0000-0002-2023-2082, D.G. Monk0000-0002-8377-1999, M.H.
Schmitt0000-0003-0814-3578, A. Taliercio0000-0002-5119-6280, M. Velasco

University of Notre Dame, Notre Dame, Indiana, USA R. Band0000-0003-4873-0523,
R. Bucci, S. Castells0000-0003-2618-3856, M. Cremonesi, A.
Das0000-0001-9115-9698, R. Goldouzian0000-0002-0295-249X, M.
Hildreth0000-0002-4454-3934, K.W. Ho0000-0003-2229-7223, K. Hurtado
Anampa0000-0002-9779-3566, C. Jessop0000-0002-6885-3611, K.
Lannon0000-0002-9706-0098, J. Lawrence0000-0001-6326-7210, N.
Loukas0000-0003-0049-6918, L. Lutton0000-0002-3212-4505, J. Mariano, N.
Marinelli, I. Mcalister, T. McCauley0000-0001-6589-8286, C.
Mcgrady0000-0002-8821-2045, K. Mohrman0009-0007-2940-0496, C.
Moore0000-0002-8140-4183, Y. Musienko160009-0006-3545-1938, H.
Nelson0000-0001-5592-0785, M. Osherson0000-0002-9760-9976, R.
Ruchti0000-0002-3151-1386, A. Townsend0000-0002-3696-689X, M.
Wayne0000-0001-8204-6157, H. Yockey, M. Zarucki0000-0003-1510-5772, L.
Zygala0000-0001-9665-7282

The Ohio State University, Columbus, Ohio, USA A. Basnet0000-0001-8460-0019,
B. Bylsma, M. Carrigan0000-0003-0538-5854, L.S. Durkin0000-0002-0477-1051, C.
Hill0000-0003-0059-0779, M. Joyce0000-0003-1112-5880, A.
Lesauvage0000-0003-3437-7845, M. Nunez Ornelas0000-0003-2663-7379, K. Wei,
B.L. Winer0000-0001-9980-4698, B. R. Yates0000-0001-7366-1318

Princeton University, Princeton, New Jersey, USA F.M.
Addesa0000-0003-0484-5804, H. Bouchamaoui0000-0002-9776-1935, P.
Das0000-0002-9770-1377, G. Dezoort0000-0002-5890-0445, P.
Elmer0000-0001-6830-3356, A. Frankenthal0000-0002-2583-5982, B.
Greenberg0000-0002-4922-1934, N. Haubrich0000-0002-7625-8169, S.
Higginbotham0000-0002-4436-5461, G. Kopp0000-0001-8160-0208, S.
Kwan0000-0002-5308-7707, D. Lange0000-0002-9086-5184, A.
Loeliger0000-0002-5017-1487, D. Marlow0000-0002-6395-1079, I.
Ojalvo0000-0003-1455-6272, J. Olsen0000-0002-9361-5762, D.
Stickland0000-0003-4702-8820, C. Tully0000-0001-6771-2174

University of Puerto Rico, Mayaguez, Puerto Rico, USA S.
Malik0000-0002-6356-2655

Purdue University, West Lafayette, Indiana, USA A.S.
Bakshi0000-0002-2857-6883, V.E. Barnes0000-0001-6939-3445, S.
Chandra0009-0000-7412-4071, R. Chawla0000-0003-4802-6819, S.
Das0000-0001-6701-9265, A. Gu0000-0002-6230-1138, L. Gutay, M.
Jones0000-0002-9951-4583, A.W. Jung0000-0003-3068-3212, D.
Kondratyev0000-0002-7874-2480, A.M. Koshy, M. Liu0000-0001-9012-395X, G.
Negro0000-0002-1418-2154, N. Neumeister0000-0003-2356-1700, G.
Paspalaki0000-0001-6815-1065, S. Piperov0000-0002-9266-7819, V. Scheurer, J.F.
Schulte0000-0003-4421-680X, M. Stojanovic0000-0002-1542-0855, J.
Thieman0000-0001-7684-6588, A. K. Virdi0000-0002-0866-8932, F.
Wang0000-0002-8313-0809, W. Xie0000-0003-1430-9191

Purdue University Northwest, Hammond, Indiana, USA J.
Dolen0000-0003-1141-3823, N. Parashar0009-0009-1717-0413, A.
Pathak0000-0001-9861-2942

Rice University, Houston, Texas, USA D. Acosta0000-0001-5367-1738, A.
Baty0000-0001-5310-3466, T. Carnahan0000-0001-7492-3201, S.
Dildick0000-0003-0554-4755, K.M. Ecklund0000-0002-6976-4637, P.J. Fernández
Manteca0000-0003-2566-7496, S. Freed, P. Gardner, F.J.M.
Geurts0000-0003-2856-9090, A. Kumar0000-0002-5180-6595, W.
Li0000-0003-4136-3409, O. Miguel Colin0000-0001-6612-432X, B.P.
Padley0000-0002-3572-5701, R. Redjimi, J. Rotter0009-0009-4040-7407, E.
Yigitbasi0000-0002-9595-2623, Y. Zhang0000-0002-6812-761X

University of Rochester, Rochester, New York, USA A. Bodek0000-0003-0409-0341,
P. de Barbaro0000-0002-5508-1827, R. Demina0000-0002-7852-167X, J.L.
Dulemba0000-0002-9842-7015, C. Fallon, A. Garcia-Bellido0000-0002-1407-1972,
O. Hindrichs0000-0001-7640-5264, A. Khukhunaishvili0000-0002-3834-1316, P.
Parygin850000-0001-6743-3781, E. Popova850000-0001-7556-8969, R.
Taus0000-0002-5168-2932, G.P. Van Onsem0000-0002-1664-2337

The Rockefeller University, New York, New York, USA K.
Goulianos0000-0002-6230-9535

Rutgers, The State University of New Jersey, Piscataway, New Jersey, USA B.
Chiarito, J.P. Chou0000-0001-6315-905X, Y. Gershtein0000-0002-4871-5449, E.
Halkiadakis0000-0002-3584-7856, A. Hart0000-0003-2349-6582, M.
Heindl0000-0002-2831-463X, D. Jaroslawski0000-0003-2497-1242, O.
Karacheban290000-0002-2785-3762, I. Laflotte0000-0002-7366-8090, A.
Lath0000-0003-0228-9760, R. Montalvo, K. Nash, H. Routray0000-0002-9694-4625,
S. Salur0000-0002-4995-9285, S. Schnetzer, S. Somalwar0000-0002-8856-7401, R.
Stone0000-0001-6229-695X, S.A. Thayil0000-0002-1469-0335, S. Thomas, J.
Vora0000-0001-9325-2175, H. Wang0000-0002-3027-0752

University of Tennessee, Knoxville, Tennessee, USA H. Acharya, D.
Ally0000-0001-6304-5861, A.G. Delannoy0000-0003-1252-6213, S.
Fiorendi0000-0003-3273-9419, T. Holmes0000-0002-3959-5174, N.
Karunarathna0000-0002-3412-0508, L. Lee0000-0002-5590-335X, E.
Nibigira0000-0001-5821-291X, S. Spanier0000-0002-7049-4646

Texas A&M University, College Station, Texas, USA D. Aebi0000-0001-7124-6911,
M. Ahmad0000-0001-9933-995X, O. Bouhali960000-0001-7139-7322, M.
Dalchenko0000-0002-0137-136X, R. Eusebi0000-0003-3322-6287, J.
Gilmore0000-0001-9911-0143, T. Huang0000-0002-0793-5664, T.
Kamon970000-0001-5565-7868, H. Kim0000-0003-4986-1728, S.
Luo0000-0003-3122-4245, S. Malhotra, R. Mueller0000-0002-6723-6689, D.
Overton0009-0009-0648-8151, D. Rathjens0000-0002-8420-1488, A.
Safonov0000-0001-9497-5471

Texas Tech University, Lubbock, Texas, USA N. Akchurin0000-0002-6127-4350, J.
Damgov0000-0003-3863-2567, V. Hegde0000-0003-4952-2873, A.
Hussain0000-0001-6216-9002, Y. Kazhykarim, K. Lamichhane0000-0003-0152-7683,
S.W. Lee0000-0002-3388-8339, A. Mankel0000-0002-2124-6312, T. Mengke, S.
Muthumuni0000-0003-0432-6895, T. Peltola0000-0002-4732-4008, I.
Volobouev0000-0002-2087-6128, A. Whitbeck0000-0003-4224-5164

Vanderbilt University, Nashville, Tennessee, USA E. Appelt0000-0003-3389-4584,
S. Greene, A. Gurrola0000-0002-2793-4052, W. Johns0000-0001-5291-8903, R.
Kunnawalkam Elayavalli0000-0002-9202-1516, A. Melo0000-0003-3473-8858, F.
Romeo0000-0002-1297-6065, P. Sheldon0000-0003-1550-5223, S.
Tuo0000-0001-6142-0429, J. Velkovska0000-0003-1423-5241, J.
Viinikainen0000-0003-2530-4265

University of Virginia, Charlottesville, Virginia, USA B.
Cardwell0000-0001-5553-0891, B. Cox0000-0003-3752-4759, J.
Hakala0000-0001-9586-3316, R. Hirosky0000-0003-0304-6330, A.
Ledovskoy0000-0003-4861-0943, A. Li0000-0002-4547-116X, C.
Neu0000-0003-3644-8627, C.E. Perez Lara0000-0003-0199-8864

Wayne State University, Detroit, Michigan, USA P.E. Karchin0000-0003-1284-3470

University of Wisconsin - Madison, Madison, Wisconsin, USA A. Aravind, S.
Banerjee0000-0001-7880-922X, K. Black0000-0001-7320-5080, T.
Bose0000-0001-8026-5380, S. Dasu0000-0001-5993-9045, I. De
Bruyn0000-0003-1704-4360, P. Everaerts0000-0003-3848-324X, C. Galloni, H.
He0009-0008-3906-2037, M. Herndon0000-0003-3043-1090, A.
Herve0000-0002-1959-2363, C.K. Koraka0000-0002-4548-9992, A. Lanaro, R.
Loveless0000-0002-2562-4405, J. Madhusudanan Sreekala0000-0003-2590-763X, A.
Mallampalli0000-0002-3793-8516, A. Mohammadi0000-0001-8152-927X, S. Mondal, G.
Parida0000-0001-9665-4575, D. Pinna, A. Savin, V. Shang0000-0002-1436-6092, V.
Sharma0000-0003-1287-1471, W.H. Smith0000-0003-3195-0909, D. Teague, H.F.
Tsoi0000-0002-2550-2184, W. Vetens0000-0003-1058-1163, A.
Warden0000-0001-7463-7360

Authors affiliated with an institute or an international laboratory covered by
a cooperation agreement with CERN S. Afanasiev0009-0006-8766-226X, V.
Andreev0000-0002-5492-6920, Yu. Andreev0000-0002-7397-9665, T.
Aushev0000-0002-6347-7055, M. Azarkin0000-0002-7448-1447, A.
Babaev0000-0001-8876-3886, A. Belyaev0000-0003-1692-1173, V. Blinov98, E.
Boos0000-0002-0193-5073, V. Borshch0000-0002-5479-1982, D.
Budkouski0000-0002-2029-1007, V. Bunichev0000-0003-4418-2072, M.
Chadeeva980000-0003-1814-1218, V. Chekhovsky, R. Chistov980000-0003-1439-8390,
A. Dermenev0000-0001-5619-376X, T. Dimova980000-0002-9560-0660, D.
Druzhkin990000-0001-7520-3329, M. Dubinin890000-0002-7766-7175, L.
Dudko0000-0002-4462-3192, A. Ershov0000-0001-5779-142X, G.
Gavrilov0000-0001-9689-7999, V. Gavrilov0000-0002-9617-2928, S.
Gninenko0000-0001-6495-7619, V. Golovtcov0000-0002-0595-0297, N.
Golubev0000-0002-9504-7754, I. Golutvin0009-0007-6508-0215, I.
Gorbunov0000-0003-3777-6606, A. Gribushin0000-0002-5252-4645, Y.
Ivanov0000-0001-5163-7632, V. Kachanov0000-0002-3062-010X, L.
Kardapoltsev980009-0000-3501-9607, V. Karjavine0000-0002-5326-3854, A.
Karneyeu0000-0001-9983-1004, V. Kim980000-0001-7161-2133, M. Kirakosyan, D.
Kirpichnikov0000-0002-7177-077X, M. Kirsanov0000-0002-8879-6538, V.
Klyukhin0000-0002-8577-6531, O. Kodolova1000000-0003-1342-4251, D.
Konstantinov0000-0001-6673-7273, V. Korenkov0000-0002-2342-7862, A.
Kozyrev980000-0003-0684-9235, N. Krasnikov0000-0002-8717-6492, A.
Lanev0000-0001-8244-7321, P. Levchenko1010000-0003-4913-0538, N.
Lychkovskaya0000-0001-5084-9019, V. Makarenko0000-0002-8406-8605, A.
Malakhov0000-0001-8569-8409, V. Matveev980000-0002-2745-5908, V.
Murzin0000-0002-0554-4627, A. Nikitenko102,1000000-0002-1933-5383, S.
Obraztsov0009-0001-1152-2758, V. Oreshkin0000-0003-4749-4995, V.
Palichik0009-0008-0356-1061, V. Perelygin0009-0005-5039-4874, M. Perfilov, S.
Petrushanko0000-0003-0210-9061, S. Polikarpov980000-0001-6839-928X, V.
Popov0000-0001-8049-2583, O. Radchenko980000-0001-7116-9469, M.
Savina0000-0002-9020-7384, V. Savrin0009-0000-3973-2485, V.
Shalaev0000-0002-2893-6922, S. Shmatov0000-0001-5354-8350, S.
Shulha0000-0002-4265-928X, Y. Skovpen980000-0002-3316-0604, S.
Slabospitskii0000-0001-8178-2494, V. Smirnov0000-0002-9049-9196, D.
Sosnov0000-0002-7452-8380, V. Sulimov0009-0009-8645-6685, E.
Tcherniaev0000-0002-3685-0635, A. Terkulov0000-0003-4985-3226, O.
Teryaev0000-0001-7002-9093, I. Tlisova0000-0003-1552-2015, A.
Toropin0000-0002-2106-4041, L. Uvarov0000-0002-7602-2527, A.
Uzunian0000-0002-7007-9020, A. Vorobyev${}^{\textrm{\textdagger}}$, N.
Voytishin0000-0001-6590-6266, B.S. Yuldashev103, A.
Zarubin0000-0002-1964-6106, I. Zhizhin0000-0001-6171-9682, A.
Zhokin0000-0001-7178-5907

†: Deceased
1Also at Yerevan State University, Yerevan, Armenia
2Also at TU Wien, Vienna, Austria
3Also at Institute of Basic and Applied Sciences, Faculty of Engineering, Arab
Academy for Science, Technology and Maritime Transport, Alexandria, Egypt
4Also at Ghent University, Ghent, Belgium
5Also at Universidade Estadual de Campinas, Campinas, Brazil
6Also at Federal University of Rio Grande do Sul, Porto Alegre, Brazil
7Also at UFMS, Nova Andradina, Brazil
8Also at Nanjing Normal University, Nanjing, China
9Now at Henan Normal University, Xinxiang, China
10Now at The University of Iowa, Iowa City, Iowa, USA
11Also at University of Chinese Academy of Sciences, Beijing, China
12Also at China Center of Advanced Science and Technology, Beijing, China
13Also at University of Chinese Academy of Sciences, Beijing, China
14Also at China Spallation Neutron Source, Guangdong, China
15Also at Université Libre de Bruxelles, Bruxelles, Belgium
16Also at an institute or an international laboratory covered by a cooperation
agreement with CERN
17Now at British University in Egypt, Cairo, Egypt
18Now at Cairo University, Cairo, Egypt
19Also at Birla Institute of Technology, Mesra, Mesra, India
20Also at Purdue University, West Lafayette, Indiana, USA
21Also at Université de Haute Alsace, Mulhouse, France
22Also at Department of Physics, Tsinghua University, Beijing, China
23Also at The University of the State of Amazonas, Manaus, Brazil
24Also at Erzincan Binali Yildirim University, Erzincan, Turkey
25Also at University of Hamburg, Hamburg, Germany
26Also at RWTH Aachen University, III. Physikalisches Institut A, Aachen,
Germany
27Also at Isfahan University of Technology, Isfahan, Iran
28Also at Bergische University Wuppertal (BUW), Wuppertal, Germany
29Also at Brandenburg University of Technology, Cottbus, Germany
30Also at Forschungszentrum Jülich, Juelich, Germany
31Also at CERN, European Organization for Nuclear Research, Geneva,
Switzerland
32Also at Institute of Physics, University of Debrecen, Debrecen, Hungary
33Also at Institute of Nuclear Research ATOMKI, Debrecen, Hungary
34Now at Universitatea Babes-Bolyai - Facultatea de Fizica, Cluj-Napoca,
Romania
35Also at Physics Department, Faculty of Science, Assiut University, Assiut,
Egypt
36Also at HUN-REN Wigner Research Centre for Physics, Budapest, Hungary
37Also at Faculty of Informatics, University of Debrecen, Debrecen, Hungary
38Also at Punjab Agricultural University, Ludhiana, India
39Also at University of Hyderabad, Hyderabad, India
40Also at University of Visva-Bharati, Santiniketan, India
41Also at Indian Institute of Science (IISc), Bangalore, India
42Also at IIT Bhubaneswar, Bhubaneswar, India
43Also at Institute of Physics, Bhubaneswar, India
44Also at Department of Physics, Isfahan University of Technology, Isfahan,
Iran
45Also at Sharif University of Technology, Tehran, Iran
46Also at Department of Physics, University of Science and Technology of
Mazandaran, Behshahr, Iran
47Also at Helwan University, Cairo, Egypt
48Also at Italian National Agency for New Technologies, Energy and Sustainable
Economic Development, Bologna, Italy
49Also at Centro Siciliano di Fisica Nucleare e di Struttura Della Materia,
Catania, Italy
50Also at Università degli Studi Guglielmo Marconi, Roma, Italy
51Also at Scuola Superiore Meridionale, Università di Napoli ’Federico II’,
Napoli, Italy
52Also at Fermi National Accelerator Laboratory, Batavia, Illinois, USA
53Also at Università di Napoli ’Federico II’, Napoli, Italy
54Also at Ain Shams University, Cairo, Egypt
55Also at Consiglio Nazionale delle Ricerche - Istituto Officina dei
Materiali, Perugia, Italy
56Also at Riga Technical University, Riga, Latvia
57Also at Department of Applied Physics, Faculty of Science and Technology,
Universiti Kebangsaan Malaysia, Bangi, Malaysia
58Also at Consejo Nacional de Ciencia y Tecnología, Mexico City, Mexico
59Also at Trincomalee Campus, Eastern University, Sri Lanka, Nilaveli, Sri
Lanka
60Also at Saegis Campus, Nugegoda, Sri Lanka
61Also at INFN Sezione di Pavia, Università di Pavia, Pavia, Italy
62Also at National and Kapodistrian University of Athens, Athens, Greece
63Also at Ecole Polytechnique Fédérale Lausanne, Lausanne, Switzerland
64Also at University of Vienna Faculty of Computer Science, Vienna, Austria
65Also at Universität Zürich, Zurich, Switzerland
66Also at Stefan Meyer Institute for Subatomic Physics, Vienna, Austria
67Also at Laboratoire d’Annecy-le-Vieux de Physique des Particules,
IN2P3-CNRS, Annecy-le-Vieux, France
68Also at Near East University, Research Center of Experimental Health
Science, Mersin, Turkey
69Also at Konya Technical University, Konya, Turkey
70Also at Izmir Bakircay University, Izmir, Turkey
71Also at Adiyaman University, Adiyaman, Turkey
72Also at Bozok Universitetesi Rektörlügü, Yozgat, Turkey
73Also at Marmara University, Istanbul, Turkey
74Also at Milli Savunma University, Istanbul, Turkey
75Also at Kafkas University, Kars, Turkey
76Now at stanbul Okan University, Istanbul, Turkey
77Also at Hacettepe University, Ankara, Turkey
78Also at Istanbul University - Cerrahpasa, Faculty of Engineering, Istanbul,
Turkey
79Also at Yildiz Technical University, Istanbul, Turkey
80Also at Vrije Universiteit Brussel, Brussel, Belgium
81Also at School of Physics and Astronomy, University of Southampton,
Southampton, United Kingdom
82Also at University of Bristol, Bristol, United Kingdom
83Also at IPPP Durham University, Durham, United Kingdom
84Also at Monash University, Faculty of Science, Clayton, Australia
85Now at an institute or an international laboratory covered by a cooperation
agreement with CERN
86Also at Università di Torino, Torino, Italy
87Also at Bethel University, St. Paul, Minnesota, USA
88Also at Karamanoğlu Mehmetbey University, Karaman, Turkey
89Also at California Institute of Technology, Pasadena, California, USA
90Also at United States Naval Academy, Annapolis, Maryland, USA
91Also at Bingol University, Bingol, Turkey
92Also at Georgian Technical University, Tbilisi, Georgia
93Also at Sinop University, Sinop, Turkey
94Also at Erciyes University, Kayseri, Turkey
95Also at Horia Hulubei National Institute of Physics and Nuclear Engineering
(IFIN-HH), Bucharest, Romania
96Also at Texas A&M University at Qatar, Doha, Qatar
97Also at Kyungpook National University, Daegu, Korea
98Also at another institute or international laboratory covered by a
cooperation agreement with CERN
99Also at Universiteit Antwerpen, Antwerpen, Belgium
100Also at Yerevan Physics Institute, Yerevan, Armenia
101Also at Northeastern University, Boston, Massachusetts, USA
102Also at Imperial College, London, United Kingdom
103Also at Institute of Nuclear Physics of the Uzbekistan Academy of Sciences,
Tashkent, Uzbekistan


# STAR: Boosting Low-Resource Event Extraction by Structure-to-Text
Data Generation with Large Language Models

Mingyu Derek Ma† Xiaoxuan Wang† Po-Nien Kung†
P. Jeffrey Brantingham‡ Nanyun Peng† Wei Wang†
†Department of Computer Science ‡Department of Anthropology
University of California, Los Angeles
{ma, xw27, ponienkung, violetpeng<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Structure prediction tasks such as event extraction require an in-depth
understanding of the output structure and sub-task dependencies, thus they
still heavily rely on task-specific training data to obtain reasonable
performance. Due to the high cost of human annotation, low-resource event
extraction, which requires minimal human cost, is urgently needed in real-
world information extraction applications. We propose to synthesize data
instances given limited seed demonstrations to boost low-resource event
extraction performance. We propose Star, a structure-to-text data generation
method that first generates complicated event structures ($Y$) and then
generates input passages ($X$), all with Large Language Models. We design
fine-grained step-by-step instructions and the error cases and quality issues
identified through self-reflection can be self-refined. Our experiments
indicate that data generated by Star can significantly improve the low-
resource event extraction performance and they are even more effective than
human-curated data points in some cases.

##  1 Introduction

Structure prediction tasks aim to induce structured information from
unstructured text Ma et al. (2021); Lu et al. (2023). Event extraction (EE), a
kind of structure prediction task, is a crucial component for information
extraction systems to identify and extract event triggers, event types, and
their related attributes or participants Ma et al. (2023); Hsu et al. (2022).
Complicated structure prediction tasks constrained by output structure and
requiring in-depth task understanding, such as event extraction, can only
accomplish reasonable performance by fine-tuning a specialized model with
task-specific human-created training data Li et al. (2023). Since Large
Language Models (LLM) are not well trained to handle structured outputs and
sub-task dependencies, EE cannot be solved reasonably by prompting LLMs to
generate structured output as predictions as shown in existing works Gao et
al. (2023).

There are unlimited types of events buried in text from various sources in
real-world scenarios. Existing works have developed human-curated datasets to
cover dozens to hundreds of event types Parekh et al. (2022), which is
particularly costly for such a rule-intense task. Low-resource event
extraction, which only provides extremely limited data points for each event
type, is specifically needed to unbind the constraint of data resources and
cover more event types at a much lower cost.

Figure 1: The proposed Star data generation pipeline conducts inverse
structure-to-text generation and is used to train off-the-shelf event
extraction models.

Existing low-resource EE methods borrow supervision signals from other tasks,
such as AMR or SRL, via transfer learning Huang et al. (2018); Zhang et al.
(2021) or reformulate the EE task as data-rich tasks, such as NLI or QA, via
indirect supervision Sainz et al. (2022); Lyu et al. (2021); Xu et al. (2023);
Lu et al. (2022). Existing works show that the LLM, such as GPT-3 Brown et al.
(2020) and LLaMA Touvron et al. (2023), have achieved remarkable performance
in many NLP tasks such as question answering and text generation. Inspired by
the recent success of using LLM in data generation for tasks like sentiment
analysis Ye et al. (2022), question answering Gao et al. (2021), and relation
extraction Josifoski et al. (2023), we approach the data limitation issue of
the low-resource EE task by synthesizing data points. However, there are
several non-trivial challenges to performing data generation for the EE task.
First, existing methods perform poorly in event extraction due to a lack of
output structure understanding, thus synthetic data generated by creating
silver event structure labels are noisy and contain a significant amount of
errors. Second, EE dataset suffers from data sparsity and imbalance issues.
For example, the most data-rich event type has more than 600x training data
compared with the most data-hungry one; most training events only contain 1-2
arguments. Third, EE dataset has limited diversity and constrains the
generalizability of the model trained on it. For example, most events happened
in the past and much fewer are future events; most participants are associated
with male pronouns instead of female ones; few triggers dominate annotations
for a certain event type.

Figure 2: The Star inverse data generation strategy.

In this paper, we present Star, a Structure-to-Text DatA GeneRation pipeline
for event extraction. Our proposed techniques enable LLM to self-refine and
produce dependable data points for low-resource event extraction. Rather than
using predicted answers as silver labels derived from the input passage
($X\rightarrow Y$) to enrich the training data, we propose to generate data
points inversely by providing ground-truth answers and prompting the LLM to
generate an input passage that leads to the given answer ($Y\rightarrow X$).
This inverse design reformulates the data generation task from structure
induction to conditional text generation. The system contains three key
components: 1) structure generation: creating diverse and commonsensical event
structures to be served as targets of EE data instances for better
generalizability and comprehensiveness; 2) instruction-guided initial data
generation: using comprehensive and detailed instruction to prompt the LLM
about the fine-grained task definition and constraints, and querying the LLM
to produce input passages of EE data instances; 3) iterative self-refining
strategy with self-reflected error identification: automatically detecting
errors in the generated data point and providing hindsight natural language
intervention to revise the generated data.

Star tackles the challenges mentioned above as 1) we reformulate the synthetic
data generation task from structure induction, on which LLMs perform poorly,
to conditional text generation, on which LLMs shine. Such an inverse data
generation design utilizes the LLMs’ text generation capabilities based on an
output structure with high confidence, and it can generate data points with
much higher quality; 2) with the ability to customize target structures across
different settings and event types, we could reduce the data imbalance issue;
3) we could also improve the data diversity by introducing a more diverse set
of trigger and argument mentions.

We perform experiments on the ACE05 dataset using $k$ exemplar data points per
event type as the seed demonstrations for the data generation pipeline. The
results show that data instances generated by Star can significantly improve
the EE performance for multiple EE models across sequence tagging and
generation formulations. We further show that the generated points are of high
quality and are more effective than training with human-created data instances
when using DEGREE as the EE model.

Our contributions are three-fold: 1) We introduce Star, an inverse structure-
to-text data generation pipeline for event extraction; 2) We introduce a
feasible approach to improve EE performance by using LLM; 3) Our experimental
results show that the generated data instances are more effective to boost the
low-resource EE performance than human-curated data under certain settings.

##  2 Related Works

###  2.1 Information Extraction with LLM

Some researchers have explored the utilization of large language models for
event extraction tasks. Gao et al. (2023) conducted an experiment where they
provided several examples and task descriptions to ChatGPT. Subsequently, they
tested ChatGPT’s ability to extract event information such as triggers and
event types from new input passages. However, their experiments revealed that
ChatGPT’s performance was subpar and lacked robustness. In a different study,
Li et al. (2023) evaluated ChatGPT’s performance on various information
extraction tasks and conducted a systematic analysis of the results. The
findings indicated that ChatGPT excelled in faithfulness and explainability.
However, it struggled with calibration and performance in standard IE
settings. In contrast, our work does not use LLM directly to handle IE tasks,
and we envision the better approach to utilize LLM for closed IE tasks is to
use it for training data generation.

###  2.2 Data Augmentation with LLM

Our research is positioned within the realm of pseudo-dataset generation with
LLM. A significant body of literature has delved into the concept of data
augmentation via sequence-to-sequence transformations in text data. The
configuration involves producing analogous text data from the provided input
text data ($X\rightarrow X^{\prime}$). Pioneers such as Kumar et al. (2020)
and Lee et al. (2021) have fine-tuned generic models to produce novel
datapoints from existing sections of training data. Subsequent work by Schick
and Schütze (2021) has demonstrated data generation through pre-trained
language models (PLMs) in an unsupervised manner, using only task
descriptions. Further progression was seen in the work of He et al. (2021),
who deployed large language models to yield unlabeled examples, which were
subsequently annotated using pseudo labels via classifiers. Similarly, Ye et
al. (2022) exploited PLMs to generate data, which were then fed into an
extremely compact model tailored for specific tasks.

Recently, with the advancement and increasing power of LLM, certain studies
have begun to generate synthetic data by inputting the ground truth ($Y$ to
$X$). Both Meng et al. (2022) and Gao et al. (2021) have executed this reverse
task by feeding a pre-trained language model with ground truth label
descriptions, subsequently generating the input text data. In a similar vein,
Josifoski et al. (2023) employed the inverse task for data augmentation in
structured data settings such as relation extraction. However, these works
generate data for tasks with significantly simpler output structures compared
with event extraction. What’s more, they require existing ground-truth data to
perform data generation while our work aims to generate both $Y$ and
subsequently $X$.

##  3 Star Data Generation

In this section, we introduce the method design of the Star data generation
process. The goal is to create $N$ new data instances based on $k$
demonstration data instances for each event type. We intend to use the $N$
data instances as additional training resources to provide additional learning
signals for EE models about a specific event type $et$. Each data point for an
event type $et$ includes ($X^{et}_{i}$, $Y^{et}_{i}$), where $X^{et}_{i}$ is a
natural language passage containing event information, $Y^{et}_{i}$ are event
structures that include the following elements for each event: 1) event
trigger, 2) event type, 3) argument mentions, 4) roles for each argument
mention. Note that a sentence could contain 0 to an unlimited number of
events, and each event can only have one trigger and one event type. The
trigger must be a sub-sequence of the $X$ passage and the event type must be
one of the $n_{et}$ pre-defined types according to pre-defined event ontology.
For an event, there could be 0 to $n_{arg\\_roles}^{et}$ arguments. Each event
type holds a pool of $n_{arg\\_role}^{et}$ argument roles as defined in the
event ontology.

There are three steps: 1) structure generation to prepare $Y$ (Section 3.1),
2) instruction-guided initial data generation to generate $X_{0}$ (Section
3.2), and 3) self-refining with self-reflection to revise $X_{0}$ to $X_{t}$
where $t$ is the times of revision (Section 3.3).

###  3.1 Structure Generation

The output distribution of the structure prediction dataset largely determines
the generalizability and robustness of the fine-tuned model. We first generate
a pool of valid seed words for trigger of each event type, and for arguments
of each (event type, argument role) combination. We then create $Y$ structure.
During the process of generating output structures, we particularly employ
output distribution control to relieve the data imbalance issue and improve
the data diversity and comprehensiveness.

#### Trigger candidate generation.

We obtain trigger candidates by prompting the LLM with 1) a definition of the
selected event type; and 2) a few passages that contain event triggers of the
selected event type as demonstrations. We use special tags to wrap the trigger
word in the demonstration passages and prompt the LLM to continuously generate
more passages with trigger words wrapped within tag pairs. Then we could parse
the response and extract the trigger candidate words.

#### Argument candidate generation.

We find reasonable arguments for a certain pair of event type and argument
role by prompting LLM with the following information: 1) definition of the
argument role under this specific event type (e.g. definition of Person
argument under Life:Be-Born event type is “The person who is born.”); 2) the
entity type we are looking for (e.g. a vehicle). The prompt would be like
“Given the definition of Instrument argument as The device used to inflict the
harm, what are some possible vehicle names that can be used as Instrument?”
for Instrument argument of Life:Injure event type. The allowed entity types of
arguments are provided in the event ontology, and we merge generated word
pools returned from separate queries if there are multiple allowed entity
types for an argument role. For example, the Origin argument of the
Movement:Transport event type could be of types GPE (geopolitical entity), LOC
(location), or FAC (facility name).

#### Creating structure combination.

After obtaining the pools of legit and commonsensical trigger and argument
candidates, we randomly sample trigger candidates and argument candidates to
create event structures that could be served as $Y$.

#### Structure distribution control.

For data label distribution, many existing human-curated datasets suffer from
unbalanced distribution and dominant labels Zhou et al. (2022); Cao et al.
(2022); Zhao et al. (2018). In event extraction, a single dominant trigger
word eclipses other relevant terms, leading to an unbalanced representation
Tong et al. (2022). We relieve the imbalanced event type distribution and
event trigger selection by evenly generating data points for each event type
and creating a much larger pool for trigger selection. Additionally, we
balance the argument hallucination ratio in the generated data points. We make
sure the generated dataset contains events with many arguments and few
arguments by evenly replacing argument value with None spanning different
argument hallucination ratios. What’s more, we balance the event density in
$X_{i}$ passages by providing $Y_{i}$ event structures with zero to five
events. Such output distribution controls of different perspectives ensure the
EE models trained on the generated data points could learn features and
patterns of different event and argument densities and thus enhancing
robustness and generalizability.

###  3.2 Instruction-Guided Inverse Data Generation

We use task instruction for multiple task granularities to provide detailed
definitions and recipes for the LLM to generate passages containing the
structured event information. The instruction is appended to $k$ in-context
learning examples verbalized by our instance verbalizer. Finally, we provide
the verbalized event structure information for the target $Y_{i}$ to prompt
the LLM to generate the corresponding $X_{i}$. We provide a complete prompt
example in Appendix A.

#### Task-level instruction.

We provide task-related instruction following the annotation guideline curated
by experts to guide the human annotation process of the ACE05 dataset
Doddington et al. (2004).111ACE English Annotation Guidelines for values
(https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/english-values-
guidelines-v1.2.4.pdf) and for events
(https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/english-events-
guidelines-v5.4.3.pdf). Specifically, we provide: 1) a definition of “event”;
2) a definition of “trigger”; 3) a definition of “participant arguments”; 4) a
definition of “attribute arguments”; 5) an overall task requirement that the
goal is to generate a sentence containing the event trigger words and
arguments; 6) hallucination clarification that instructs the model not to
generate arguments of certain roles if we explicitly provide that “the
argument is None”.

#### Event type-level instruction.

In this segment of instruction, we introduce meta-information provided by pre-
defined event ontology for a specific event type. Particularly, we introduce
the following information: 1) the name of the event type; 2) a definition
sentence of the event type; 3) the name and definition of each possible
argument roles under this specific event type. We identify all the possible
arguments of a certain event type following the event ontology. For argument
information, we specifically provide whether the argument is a participant
argument, which is an entity involved in the event, or an attribute argument,
which is a value or property as part of the event. We loop through all
possible argument roles instead of the ones with existing values to ensure the
generated passage $X_{i}$ does not contain hallucinated arguments that should
not appear according to the output structure $Y_{i}$.

#### Instance-level verbalizer.

We use the instance-level verbalizer to verbalize exemplar data instances and
$Y$ structure for the target data point, for which we expect the LLM to
generate a corresponding $X$ passage. We verbalize the data instances to three
segments: 1) the first segment provides overview information about the number
of events in the passage; 2) the second segment provides event information for
a specific event, if there are multiple events we concatenate multiple event
information together; 3) the passage segment containing a prompt phrase and
the $X_{i}$ passage with tags.

For the event information segment, we provide event trigger, argument mention
and argument roles for all possible argument roles under this event type. For
the passage segment, we use special tags to wrap trigger and argument words to
explicitly hint to the LLM about the roles and positions of the keywords while
learning-from-analogy of the in-context demonstrations. For example,
“<Plaintiff>He</Plaintiff> threatened to <Trigger>sue</Trigger> if the company
did not refund his money.” could provide an explicit indicator to the LLM that
“he” is served as an Plaintiff argument for the event triggered by “sue”.
During the generation process of the passage $X_{i}$ containing our target
event information, the LLM would also generate the trigger and argument tags
to not only reiterate the role of the keywords, but also facilitate our
mapping process to identify the start and end indexes of the trigger and
argument words in the generated passage. The start and end indexes are needed
to train sequence tagging and classification-based EE models and perform
religious prediction matching for precise evaluation.

###  3.3 Self-refinement by Self-reflection

After the initial passage ($X_{0}$) is generated, the self-refinement
mechanism reflects the quality and potential errors and further improves the
initial generation results through several iterative updates Madaan et al.
(2023); Wang et al. (2022); Liu et al. (2023). In the $t$-th refinement
iteration, we first identify the potential errors and quality issues of
$X^{\prime}_{t-1}$ from a diverse set of quality dimensions, then the issues
are feedback to the LLM by providing a template-based natural language
intervention $h_{i}$ along with the generated passage of previous iteration
$X^{\prime}_{t-1}$, so that the LLM could refine the generation result and
produce $X_{t}$. Some exemplar hindsight messages include “The passage lacks
the information about concrete time, location and involved person” and “The
passage contains a hallucinated argument Date incorrectly”.

We define a set of quality dimensions. The overall quality dimensions include
1) the commonsense level of the generated passage; 2) the informativeness of
the generated passage. the task-specific quality dimensions include 3) whether
a trigger is used to initiate an occurrence; 4) whether an argument is used as
an event participant or attribute; 5) whether the argument is serving the
required argument role; 6) whether POS tags of the trigger/argument mentions
match the ones provided in the event structure $Y$; 7) whether
trigger/argument mention is a subsequence of the generated passage; 8) whether
the passage contains information that could be served as an argument which
should not appear.

For each quality dimension, we create a QA instance to query LLM with
questions like “Is Location A describing the event performing a Destination
role?”. We then fill in the error hindsight template, which is different for
each quality dimension, if the response indicates an error. Such a self-
reflection design makes the self-refinement process generalizable and robust
since the entire reflection and error identification process is done by the
LLM itself without external add-on components.

##  4 Experiments on Few-shot Event Extraction

# |  EE Model |  Data Gen |  $N$=0 | 10 | 20 | 30 | 40 | 50 |  $N$=0 | 10 | 20 | 30 | 40 | 50
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Trigger Identification | Trigger Classification
1 | OneIE | Star | 65.18 | +1.43 | +1.92 | +2.75 | +2.87 | +2.96 | 62.65 | +1.02 | +1.23 | +1.47 | +1.85 | +2.14
2 | OneIE | Human† | +1.75 | +2.14 | +2.88 | +2.96 | +3.75 | +0.67 | +0.74 | +1.85 | +2.28 | +3.56
3 | DEGREE | Star | 57.65 | +2.87 | +4.07 | +4.56 | +5.23 | +5.92 | 52.49 | +2.31 | +5.43 | +5.67 | +5.82 | +6.61
4 | DEGREE | Human† | +2.30 | +3.84 | +4.33 | +4.49 | +5.84 | +1.92 | +5.04 | +5.08 | +5.42 | +6.37
|  |  | Argument Identification | Argument Classification
1 | OneIE | Star | 42.23 | +4.32 | +5.77 | +7.59 | +8.14 | +8.46 | 40.02 | +4.09 | +4.82 | +6.27 | +7.51 | +7.93
2 | OneIE | Human† | +7.58 | +10.48 | +12.47 | +12.76 | +12.89 | +7.36 | +9.60 | +11.91 | +12.79 | +12.98
3 | DEGREE | Star | 42.29 | +5.71 | +8.12 | +9.27 | +9.96 | +10.77 | 40.08 | +5.65 | +7.74 | +9.08 | +9.94 | +10.89
4 | DEGREE | Human† | +5.10 | +7.55 | +8.86 | +9.37 | +10.18 | +5.08 | +7.10 | +8.52 | +9.21 | +10.01
Table 1:  Event extraction performance (F1, %) when the EE models are trained
on $N$ augmented training data on top of 10 data points ($k=10$) for each
event type. The augmented training data is generated by the proposed Star data
generation pipeline or randomly sampled from the human-curated ACE05 dataset.
As indicated by †, using additional ground-truth data points is not a
realistic setting and it’s not a fair comparison with the Star generated data.
We include the ground-truth baselines to compare the level of quality and
effectiveness of the data generated by Star with regard to costly human
curation results. We observe training on data generated by Star can
significantly improve the model performance by around 10 points on argument
classification, and data generated by Star is even more effective than human-
curated ones when using DEGREE as the EE model.

1 |  Event Trigger: "sue".
---|---
Event Argument: ["text": "he","role": "Plaintiff"].
Event Type:"Justice:Sue"
Generated Passage: He threatened to <Trigger>sue</Trigger> if the company did
not refund his money.
2 |  Event Trigger: "escorted".
Event Arguments: ["text": "convoy","role": "Vehicle"], ["text": "U.S","role":
"Origin"], ["text": "soldiers","role": "Agent"].
Event Type:"Movement:Transport"
Output Passage: The U.S. <Artifact>convoy</Artifact> was
<Trigger>escorted</Trigger> by soldiers
through the dangerous territory.
3 |  Event Trigger: "flee".
Event Arguments: ["text": "he","role": "Artifact"], ["text": "syria","role":
"Destination"].
Event Type:"Movement:Transport"
Generated Passage: After fighting for months, <Artifact>he</Artifact> finally
decided to <Trigger>flee</Trigger>
to <Destination>Syria</Destination> to seek refuge.
4 |  Event Trigger: "appeal"1, "conviction"2, "sentence"3.
Event Arguments: ["text": "court", "role": "Adjudicator"]1,2,3,["text": "Anwar
Ibrahim","role": "Plaintiff"]1,
["text": "Malaysia","role": "Place"]1,2,3,["text": "Anwar Ibrahim","role":
"Defendant"]2,3.
Event Type:"Justice:Appeal"1, "Justice:Convict"2, "Justice:Sentence"3.
Generated Passage: Former Malaysian deputy prime minister <Defendant>Anwar
Ibrahim</Defendant> has been granted
a final chance to appeal his <Trigger>conviction</Trigger> and five-year
<Trigger>sentence</Trigger> on sodomy charges,
after the country’s <Adjudicator>court</Adjudicator> ruled that he had a right
to challenge the verdict.
This is the latest development in a high-profile case that has attracted
international attention
and criticism of the <Place><Place>Malaysia</Place></Place>n government’s
treatment of political opponents.
5 |  Event Trigger: "were taken"1, "go inside"2.
Event Arguments: ["text": "journalists","role": "Artifact"]1,2, ["text":
"hospital","role": "Destination"]1,2.
Event Type:"Movement:Transport"1,2
Generated Passage: Several <Artifact>journalists</Artifact> <Trigger>were
taken</Trigger> to a
<Destination>hospital</Destination>after being attacked while trying to
<Trigger>go inside</Trigger> a government building.

Table 2: Qualitative analysis examples using event trigger, type, and
arguments ($Y$) to generate input passage containing the given event
information ($X$). Superscripts indicate the event trigger1, arguments1, and
event type1 all belong to the same event.

###  4.1 Experimental Setup

To determine the usefulness of the additional augmented data, we compare the
performance of the event extraction tasks while training with and without the
generated data on well-known baselines. We adopt a $k$-shot event extraction
setting to only use $k$ training instances as seed examples for each event
type. The $k$ instances are used as in-context demonstrations for the Star
data generation pipeline and a subset of the training data for the event
extraction models. During evaluation, we use the full set of dev/test set to
produce the experimental results. We use the event ontology and data instances
from the widely used sentence-level English event extraction dataset ACE05E
Doddington et al. (2004). We use GPT-3.5-turbo LLM Brown et al. (2020) to
perform data generation.222https://platform.openai.com/docs/models/gpt-3-5

#### Baselines.

We use two representative EE models to benchmark the EE performance while
using different amounts of training data. OneIE Lin et al. (2020) is a multi-
task sequence-tagging EE model trained with global features. We use RoBERTa-
large model as its backbone Liu et al. (2019). DEGREE Hsu et al. (2022) is a
prompt-based generative model that fills in event type-specific human written
templates and expects the text generation results to contain both information
about event triggers and arguments. We use BART-large model as its backbone
Lewis et al. (2020). Note that OneIE additionally performs entity recognition
on top of event detection and event argument extraction tasks, thus the
performances between the two are not directly comparable.

#### Evaluation Metrics.

We follow previous EE works and report F1 scores for the following four tasks.
1) Trigger Identification: identified trigger span is correct. 2) Trigger
Classification: identified trigger is correct and its predicted event type is
correct. 3) Argument Identification: identified argument span is correct. 4)
Argument Classification: identified event argument is correct and its
predicted argument role is also correct.

###  4.2 Experimental Results

Table 1 shows the event extraction performance corresponding to different few-
shot settings and amounts of augmented data points.

First, we observe training on additional data generated by Star leads to 7.93
and 10.89 points of argument classification F1 improvements when using OneIE
and DEGREE respectively.

Second, we observe data generated by Star is even more effective than human-
curated data in boosting low-resource EE performance for all subtasks, by an
average of 0.7 points. This indicates the error and noise introduced in the
data generation process of Star is very limited and the model could benefit
from the additional diversity introduced by Star instead of human annotation.

Third, we observe the larger amount of augmented data leads to more
significant performance gain across combinations of different EE models, and
additional data generation methods for all sub-tasks. We also observe the
performance gain is more significant on argument-related tasks compared with
trigger-related tasks. Particularly, the argument classification task
demonstrates a significant improvement, with approximately a 13 F1 point
increase in both classification and identification tasks.

###  4.3 Qualitative Analysis

In order to assess the quality of the data generated by LLM, we provide five
examples as shown in Table 2. We observe that some tags that are supposed to
wrap trigger or argument words are not correctly generated, such as “He”
should be “<Plaintiff>He</Plaintiff>”. These tags are expected to be removed,
and not generating them correctly is not a major problem. In examples 1, 2 and
3, we observe that LLM demonstrates high precision in incorporating event
information into the passage following our prompting design. However, errors
can arise when handling multiple events or trigger words. Example 4
incorrectly states that the event “go outside” is associated with the argument
“government building” instead of the correct destination “hospital” as
specified in the event structure generated first. Additionally, example 5
shows that the generated passage includes hallucinated details about a
Sentence argument of “five years” and a Crime argument of “sodomy charged”,
which should not be mentioned since they are not specified in the event
information. To be specific, Crime is one of the possible argument roles of
event type Justice:Appeal, Justice:Convict and Justice:Sentence, and Sentence
is a possible argument role of event type Justice:Sentence. Since both
arguments do not have value in the generated event structure $Y$, thus any
mentions that could served as Crime or Sentence should not appear in the
generated passage.

##  5 Conclusion and Future Work

We present Star, an inverse data generation pipeline designed for low-resource
event extraction that generates complicated output structure first and then
curates input passage containing structure content, all with LLMs. The
proposed pipeline includes task-specific step-by-step instructions to create a
diverse set of trigger and argument candidates $Y$ and further create an
initially generated passage $X$. The proposed method also contains self-
refinement capabilities to fix self-identified error cases for better
generation quality. Experimental results show that the generated data
instances could improve the argument classification performance by around 10
F1 score points and they are even more effective than human-curated data
instances to boost the low-resource event extraction performance.

## References

  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language Models are Few-Shot Learners.
  * Cao et al. (2022) Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang, Rahul Gupta, Varun Kumar, Jwala Dhamala, and Aram Galstyan. 2022.  On the intrinsic and extrinsic fairness evaluation metrics for contextualized language representations.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 561–570, Dublin, Ireland. Association for Computational Linguistics.
  * Doddington et al. (2004) George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004.  The automatic content extraction (ACE) program – tasks, data, and evaluation.  In _Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)_ , Lisbon, Portugal. European Language Resources Association (ELRA).
  * Gao et al. (2023) Jun Gao, Huan Zhao, Changlong Yu, and Ruifeng Xu. 2023.  Exploring the Feasibility of ChatGPT for Event Extraction.
  * Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021.  Making Pre-trained Language Models Better Few-shot Learners.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3816–3830, Online. Association for Computational Linguistics.
  * He et al. (2021) Xuanli He, Islam Nassar, Jamie Ryan Kiros, Gholamreza Haffari, and Mohammad Norouzi. 2021.  Generate, annotate, and learn: Generative models advance self-training and knowledge distillation.
  * Hsu et al. (2022) I-Hung Hsu, Kuan-Hao Huang, Elizabeth Boschee, Scott Miller, Prem Natarajan, Kai-Wei Chang, and Nanyun Peng. 2022.  DEGREE: A Data-Efficient Generation-Based Event Extraction Model.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1890–1908, Seattle, United States. Association for Computational Linguistics.
  * Huang et al. (2018) Lifu Huang, Heng Ji, Kyunghyun Cho, Ido Dagan, Sebastian Riedel, and Clare Voss. 2018.  Zero-Shot Transfer Learning for Event Extraction.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2160–2170, Melbourne, Australia. Association for Computational Linguistics.
  * Josifoski et al. (2023) Martin Josifoski, Marija Sakota, Maxime Peyrard, and Robert West. 2023.  Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction.
  * Kumar et al. (2020) Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020.  Data Augmentation using Pre-trained Transformer Models.  In _Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems_ , pages 18–26, Suzhou, China. Association for Computational Linguistics.
  * Lee et al. (2021) Kenton Lee, Kelvin Guu, Luheng He, Tim Dozat, and Hyung Won Chung. 2021.  Neural data augmentation via example extrapolation.  _arXiv preprint 2102.01335_.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Li et al. (2023) Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, and Shikun Zhang. 2023.  Evaluating ChatGPT’s Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness.
  * Lin et al. (2020) Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020.  A Joint Neural Model for Information Extraction with Global Features.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7999–8009, Online. Association for Computational Linguistics.
  * Liu et al. (2023) Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. 2023.  Chain of Hindsight Aligns Language Models with Feedback.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.
  * Lu et al. (2022) Keming Lu, I-Hung Hsu, Wenxuan Zhou, Mingyu Derek Ma, and Muhao Chen. 2022.  Summarization as Indirect Supervision for Relation Extraction.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 6575–6594, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Lu et al. (2023) Keming Lu, I-Hung Hsu, Wenxuan Zhou, Mingyu Derek Ma, and Muhao Chen. 2023.  Multi-hop evidence retrieval for cross-document relation extraction.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , Toronto, Canada. Association for Computational Linguistics.
  * Lyu et al. (2021) Qing Lyu, Hongming Zhang, Elior Sulem, and Dan Roth. 2021.  Zero-shot Event Extraction via Transfer Learning: Challenges and Insights.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 322–332, Online. Association for Computational Linguistics.
  * Ma et al. (2021) Mingyu Derek Ma, Jiao Sun, Mu Yang, Kung-Hsiang Huang, Nuan Wen, Shikhar Singh, Rujun Han, and Nanyun Peng. 2021.  EventPlus: A Temporal Event Understanding Pipeline.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations_ , pages 56–65, Online. Association for Computational Linguistics.
  * Ma et al. (2023) Mingyu Derek Ma, Alexander K. Taylor, Wei Wang, and Nanyun Peng. 2023.  Dice: Data-efficient clinical event extraction with generative models.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics_ , Toronto, Canada. Association for Computational Linguistics.
  * Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023.  Self-Refine: Iterative Refinement with Self-Feedback.
  * Meng et al. (2022) Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022.  Generating Training Data with Language Models: Towards Zero-Shot Language Understanding.
  * Parekh et al. (2022) Tanmay Parekh, I.-Hung Hsu, Kuan-Hao Huang, Kai-Wei Chang, and Nanyun Peng. 2022\.  GENEVA: Pushing the Limit of Generalizability for Event Argument Extraction with 100+ Event Types.
  * Sainz et al. (2022) Oscar Sainz, Itziar Gonzalez-Dios, Oier Lopez de Lacalle, Bonan Min, and Eneko Agirre. 2022.  Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 2439–2455, Seattle, United States. Association for Computational Linguistics.
  * Schick and Schütze (2021) Timo Schick and Hinrich Schütze. 2021.  Generating datasets with pretrained language models.  _arXiv 2104.07540_.
  * Tong et al. (2022) MeiHan Tong, Bin Xu, Shuai Wang, Meihuan Han, Yixin Cao, Jiangqi Zhu, Siyu Chen, Lei Hou, and Juanzi Li. 2022.  DocEE: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3970–3982, Seattle, United States. Association for Computational Linguistics.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023.  LLaMA: Open and Efficient Foundation Language Models.
  * Wang et al. (2022) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022.  Self-Instruct: Aligning Language Model with Self Generated Instructions.
  * Xu et al. (2023) Jiashu Xu, Mingyu Derek Ma, and Muhao Chen. 2023.  Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction?
  * Ye et al. (2022) Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong. 2022.  ZeroGen: Efficient Zero-shot Learning via Dataset Generation.
  * Zhang et al. (2021) Hongming Zhang, Haoyu Wang, and Dan Roth. 2021.  Zero-shot Label-Aware Event Trigger and Argument Classification.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 1331–1340, Online. Association for Computational Linguistics.
  * Zhao et al. (2018) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018\.  Gender bias in coreference resolution: Evaluation and debiasing methods.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics.
  * Zhou et al. (2022) Yi Zhou, Masahiro Kaneko, and Danushka Bollegala. 2022.  Sense embeddings are also biased – evaluating social biases in static and contextualised sense embeddings.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1924–1935, Dublin, Ireland. Association for Computational Linguistics.

##  Appendix A Detailed Prompt Example

#### Task-level instruction.

The instruction is the same for all event types.

An event is a specific occurrence involving participants. An event is
something that happens. An event can frequently be described as a change of
state. Event trigger is the word that most clearly expresses its occurence.
Participant arguments are the entities that are involved in the event.
Attributes arguments are the values or properties that are part of the event.
The task is to generate a sentence containing the event trigger words and
arguments. If the argument is None, argument of a certain role should not
appear in the passage.

#### Event type-level instruction.

This instruction would be different when we generate data points for different
event types. Here we use the Justice:Execute event type as an example.

Event type is Justice:Execute. The event is related to someone being executed
to death. Argument Person is a participant argument used to describe the
person executed. Argument Agent is a participant argument used to describe the
agent responsible for carrying out the execution. Argument Crime is a
attribute argument used to describe the crime for which the person argument is
being executed. Argument Time is a attribute argument used to describe when
the execution takes place. Argument Place is a attribute argument used to
describe where the execution takes place.

#### Verbalized in-context examples.

We verbalize each of the exempler data point, the following verbalization is
an example. We concatenate the verbalized data points.

There are 2 events in the passage.

Event information for event 1: Event Trigger is torture. Argument Agent is
None. Argument Victim is None. Argument Instrument is None. Argument Time is
None. Argument Place is None.

Event information for event 2: Event Trigger is execution. Argument Person is
None. Argument Agent is Saddam Hussein. Argument Crime is None. Argument Time
is None. Argument Place is Iraq.

Passage containing triggers and arguments is: <Agent>Saddam Hussein</Agent>
controlled <Place>Iraq</Place> through fear, <Trigger>torture</Trigger> and
<Trigger>execution</Trigger>.

#### Verbalized target query.

We verbalize the event structure $Y_{i}$ of the target data point to prompt
the LLM to generate the corresponding passage containing the event
information. Here is an example.

There are 4 events in the passage.

Event information for event 1: Event Trigger is convicted. Argument Defendant
is men. Argument Adjudicator is None. Argument Crime is None. Argument Time is
None. Argument Place is Indonesia.

Event information for event 2: Event Trigger is execution. Argument Person is
men. Argument Agent is None. Argument Crime is None. Argument Time is None.
Argument Place is Indonesia.

Event information for event 3: Event Trigger is death. Argument Agent is None.
Argument Victim is None. Argument Instrument is None. Argument Time is None.
Argument Place is None.

Event information for event 4: Event Trigger is murder. Argument Agent is men.
Argument Victim is None. Argument Instrument is None. Argument Time is None.
Argument Place is None.

Passage containing triggers and arguments is: