Salyer0000-0002-6957-1077, D. Sperka0000-0002-4624-2019, D.
Spitzbart0000-0003-2025-2742, I. Suarez0000-0002-5374-6995, A.
Tsatsos0000-0001-8310-8911, S. Yuan0000-0002-2029-024X, A.G.
Zecchinelli0000-0001-8986-278X

Brown University, Providence, Rhode Island, USA G. Benelli0000-0003-4461-8905,
X. Coubez26, D. Cutts0000-0003-1041-7099, M. Hadley0000-0002-7068-4327, U.
Heintz0000-0002-7590-3058, J.M. Hogan830000-0002-8604-3452, T.
Kwon0000-0001-9594-6277, G. Landsberg0000-0002-4184-9380, K.T.
Lau0000-0003-1371-8575, D. Li0000-0003-0890-8948, J. Luo0000-0002-4108-8681,
S. Mondal0000-0003-0153-7590, M.
Narain${}^{\textrm{\textdagger}}$0000-0002-7857-7403, N.
Pervan0000-0002-8153-8464, S. Sagir840000-0002-2614-5860, F.
Simpson0000-0001-8944-9629, M. Stamenkovic0000-0003-2251-0610, X.
Yan0000-0002-6426-0560, W. Zhang

University of California, Davis, Davis, California, USA S.
Abbott0000-0002-7791-894X, J. Bonilla0000-0002-6982-6121, C.
Brainerd0000-0002-9552-1006, R. Breedon0000-0001-5314-7581, H.
Cai0000-0002-5759-0297, M. Calderon De La Barca Sanchez0000-0001-9835-4349, M.
Chertok0000-0002-2729-6273, M. Citron0000-0001-6250-8465, J.
Conway0000-0003-2719-5779, P.T. Cox0000-0003-1218-2828, R.
Erbacher0000-0001-7170-8944, F. Jensen0000-0003-3769-9081, O.
Kukral0009-0007-3858-6659, G. Mocellin0000-0002-1531-3478, M.
Mulhearn0000-0003-1145-6436, D. Pellett0009-0000-0389-8571, W.
Wei0000-0003-4221-1802, Y. Yao0000-0002-5990-4245, F. Zhang0000-0002-6158-2468

University of California, Los Angeles, California, USA M.
Bachtis0000-0003-3110-0701, R. Cousins0000-0002-5963-0467, A.
Datta0000-0003-2695-7719, G. Flores Avila, J. Hauser0000-0002-9781-4873, M.
Ignatenko0000-0001-8258-5863, M.A. Iqbal0000-0001-8664-1949, T.
Lam0000-0002-0862-7348, E. Manca0000-0001-8946-655X, A. Nunez Del Prado, D.
Saltzberg0000-0003-0658-9146, V. Valuev0000-0002-0783-6703

University of California, Riverside, Riverside, California, USA R.
Clare0000-0003-3293-5305, J.W. Gary0000-0003-0175-5731, M. Gordon, G.
Hanson0000-0002-7273-4009, W. Si0000-0002-5879-6326, S.
Wimpenny${}^{\textrm{\textdagger}}$0000-0003-0505-4908

University of California, San Diego, La Jolla, California, USA J.G.
Branson0009-0009-5683-4614, S. Cittolin0000-0002-0922-9587, S.
Cooperstein0000-0003-0262-3132, D. Diaz0000-0001-6834-1176, J.
Duarte0000-0002-5076-7096, L. Giannini0000-0002-5621-7706, J.
Guiang0000-0002-2155-8260, R. Kansal0000-0003-2445-1060, V.
Krutelyov0000-0002-1386-0232, R. Lee0009-0000-4634-0797, J.
Letts0000-0002-0156-1251, M. Masciovecchio0000-0002-8200-9425, F.
Mokhtar0000-0003-2533-3402, S. Mukherjee0000-0003-3122-0594, M.
Pieri0000-0003-3303-6301, M. Quinnan0000-0003-2902-5597, B.V. Sathia
Narayanan0000-0003-2076-5126, V. Sharma0000-0003-1736-8795, M.
Tadel0000-0001-8800-0045, E. Vourliotis0000-0002-2270-0492, F.
Würthwein0000-0001-5912-6124, Y. Xiang0000-0003-4112-7457, A.
Yagil0000-0002-6108-4004

University of California, Santa Barbara - Department of Physics, Santa
Barbara, California, USA A. Barzdukas0000-0002-0518-3286, L.
Brennan0000-0003-0636-1846, C. Campagnari0000-0002-8978-8177, J.
Incandela0000-0001-9850-2030, J. Kim0000-0002-2072-6082, A.J.
Li0000-0002-3895-717X, P. Masterson0000-0002-6890-7624, H.
Mei0000-0002-9838-8327, J. Richman0000-0002-5189-146X, U.
Sarica0000-0002-1557-4424, R. Schmitz0000-0003-2328-677X, F.
Setti0000-0001-9800-7822, J. Sheplock0000-0002-8752-1946, D.
Stuart0000-0002-4965-0747, T.Á. Vámi0000-0002-0959-9211, S.
Wang0000-0001-7887-1728

California Institute of Technology, Pasadena, California, USA A.
Bornheim0000-0002-0128-0871, O. Cerri, A. Latorre, J. Mao0009-0002-8988-9987,
H.B. Newman0000-0003-0964-1480, G. Reales Gutiérrez, M.
Spiropulu0000-0001-8172-7081, J.R. Vlimant0000-0002-9705-101X, C.
Wang0000-0002-0117-7196, S. Xie0000-0003-2509-5731, R.Y.
Zhu0000-0003-3091-7461

Carnegie Mellon University, Pittsburgh, Pennsylvania, USA J.
Alison0000-0003-0843-1641, S. An0000-0002-9740-1622, M.B.
Andrews0000-0001-5537-4518, P. Bryant0000-0001-8145-6322, M. Cremonesi, V.
Dutta0000-0001-5958-829X, T. Ferguson0000-0001-5822-3731, A.
Harilal0000-0001-9625-1987, C. Liu0000-0002-3100-7294, T.
Mudholkar0000-0002-9352-8140, S. Murthy0000-0002-1277-9168, P.
Palit0000-0002-1948-029X, M. Paulini0000-0002-6714-5787, A.
Roberts0000-0002-5139-0550, A. Sanchez0000-0002-5431-6989, W.
Terrill0000-0002-2078-8419

University of Colorado Boulder, Boulder, Colorado, USA J.P.
Cumalat0000-0002-6032-5857, W.T. Ford0000-0001-8703-6943, A.
Hart0000-0003-2349-6582, A. Hassani0009-0008-4322-7682, G.
Karathanasis0000-0001-5115-5828, N. Manganelli0000-0002-3398-4531, A.
Perloff0000-0001-5230-0396, C. Savard0009-0000-7507-0570, N.
Schonbeck0009-0008-3430-7269, K. Stenson0000-0003-4888-205X, K.A.
Ulmer0000-0001-6875-9177, S.R. Wagner0000-0002-9269-5772, N.
Zipper0000-0002-4805-8020

Cornell University, Ithaca, New York, USA J. Alexander0000-0002-2046-342X, S.
Bright-Thonney0000-0003-1889-7824, X. Chen0000-0002-8157-1328, D.J.
Cranshaw0000-0002-7498-2129, J. Fan0009-0003-3728-9960, X.
Fan0000-0003-2067-0127, S. Hogan0000-0003-3657-2281, P. Kotamnives, J.
Monroy0000-0002-7394-4710, M. Oshiro0000-0002-2200-7516, J.R.
Patterson0000-0002-3815-3649, J. Reichert0000-0003-2110-8021, M.
Reid0000-0001-7706-1416, A. Ryd0000-0001-5849-1912, J.
Thom0000-0002-4870-8468, P. Wittich0000-0002-7401-2181, R.
Zou0000-0002-0542-1264

Fermi National Accelerator Laboratory, Batavia, Illinois, USA M.
Albrow0000-0001-7329-4925, M. Alyari0000-0001-9268-3360, O.
Amram0000-0002-3765-3123, G. Apollinari0000-0002-5212-5396, A.
Apresyan0000-0002-6186-0130, L.A.T. Bauerdick0000-0002-7170-9012, D.
Berry0000-0002-5383-8320, J. Berryhill0000-0002-8124-3033, P.C.
Bhat0000-0003-3370-9246, K. Burkett0000-0002-2284-4744, J.N.
Butler0000-0002-0745-8618, A. Canepa0000-0003-4045-3998, G.B.
Cerati0000-0003-3548-0262, H.W.K. Cheung0000-0001-6389-9357, F.
Chlebana0000-0002-8762-8559, G. Cummings0000-0002-8045-7806, J.
Dickinson0000-0001-5450-5328, I. Dutta0000-0003-0953-4503, V.D.
Elvira0000-0003-4446-4395, Y. Feng0000-0003-2812-338X, J.
Freeman0000-0002-3415-5671, A. Gandrakota0000-0003-4860-3233, Z.
Gecse0009-0009-6561-3418, L. Gray0000-0002-6408-4288, D. Green, A.
Grummer0000-0003-2752-1183, S. Grünendahl0000-0002-4857-0294, D.
Guerrero0000-0001-5552-5400, O. Gutsche0000-0002-8015-9622, R.M.
Harris0000-0003-1461-3425, R. Heller0000-0002-7368-6723, T.C.
Herwig0000-0002-4280-6382, J. Hirschauer0000-0002-8244-0805, L.
Horyn0000-0002-9512-4932, B. Jayatilaka0000-0001-7912-5612, S.
Jindariani0009-0000-7046-6533, M. Johnson0000-0001-7757-8458, U.
Joshi0000-0001-8375-0760, T. Klijnsma0000-0003-1675-6040, B.
Klima0000-0002-3691-7625, K.H.M. Kwok0000-0002-8693-6146, S.
Lammel0000-0003-0027-635X, D. Lincoln0000-0002-0599-7407, R.
Lipton0000-0002-6665-7289, T. Liu0009-0007-6522-5605, C.
Madrid0000-0003-3301-2246, K. Maeshima0009-0000-2822-897X, C.
Mantilla0000-0002-0177-5903, D. Mason0000-0002-0074-5390, P.
McBride0000-0001-6159-7750, P. Merkel0000-0003-4727-5442, S.
Mrenna0000-0001-8731-160X, S. Nahn0000-0002-8949-0178, J.
Ngadiuba0000-0002-0055-2935, D. Noonan0000-0002-3932-3769, V.
Papadimitriou0000-0002-0690-7186, N. Pastika0009-0006-0993-6245, K.
Pedro0000-0003-2260-9151, C. Pena850000-0002-4500-7930, F.
Ravera0000-0003-3632-0287, A. Reinsvold Hall860000-0003-1653-8553, L.
Ristori0000-0003-1950-2492, E. Sexton-Kennedy0000-0001-9171-1980, N.
Smith0000-0002-0324-3054, A. Soha0000-0002-5968-1192, L.
Spiegel0000-0001-9672-1328, S. Stoynev0000-0003-4563-7702, J.
Strait0000-0002-7233-8348, L. Taylor0000-0002-6584-2538, S.
Tkaczyk0000-0001-7642-5185, N.V. Tran0000-0002-8440-6854, L.
Uplegger0000-0002-9202-803X, E.W. Vaandering0000-0003-3207-6950, A.
Whitbeck0000-0003-4224-5164, I. Zoi0000-0002-5738-9446

University of Florida, Gainesville, Florida, USA C. Aruta0000-0001-9524-3264,
P. Avery0000-0003-0609-627X, D. Bourilkov0000-0003-0260-4935, L.
Cadamuro0000-0001-8789-610X, P. Chang0000-0002-2095-6320, V.
Cherepanov0000-0002-6748-4850, R.D. Field, E. Koenig0000-0002-0884-7922, M.
Kolosova0000-0002-5838-2158, J. Konigsberg0000-0001-6850-8765, A.
Korytov0000-0001-9239-3398, K. Matchev0000-0003-4182-9096, N.
Menendez0000-0002-3295-3194, G. Mitselmakher0000-0001-5745-3658, K.
Mohrman0009-0007-2940-0496, A. Muthirakalayil Madhu0000-0003-1209-3032, N.
Rawal0000-0002-7734-3170, D. Rosenzweig0000-0002-3687-5189, S.
Rosenzweig0000-0002-5613-1507, J. Wang0000-0003-3879-4873

Florida State University, Tallahassee, Florida, USA T.
Adams0000-0001-8049-5143, A. Al Kadhim0000-0003-3490-8407, A.
Askew0000-0002-7172-1396, S. Bower0000-0001-8775-0696, R.
Habibullah0000-0002-3161-8300, V. Hagopian0000-0002-3791-1989, R.
Hashmi0000-0002-5439-8224, R.S. Kim0000-0002-8645-186X, S.
Kim0000-0003-2381-5117, T. Kolberg0000-0002-0211-6109, G. Martinez, H.
Prosper0000-0002-4077-2713, P.R. Prova, M. Wulansatiti0000-0001-6794-3079, R.
Yohay0000-0002-0124-9065, J. Zhang

Florida Institute of Technology, Melbourne, Florida, USA B. Alsufyani, M.M.
Baarmand0000-0002-9792-8619, S. Butalla0000-0003-3423-9581, S.
Das0000-0001-6701-9265, T. Elkafrawy550000-0001-9930-6445, M.
Hohlmann0000-0003-4578-9319, R. Kumar Verma0000-0002-8264-156X, M. Rahmani, E.
Yanes

University of Illinois Chicago, Chicago, USA, Chicago, USA M.R.
Adams0000-0001-8493-3737, A. Baty0000-0001-5310-3466, C. Bennett, R.
Cavanaugh0000-0001-7169-3420, R. Escobar Franco0000-0003-2090-5010, O.
Evdokimov0000-0002-1250-8931, C.E. Gerber0000-0002-8116-9021, M. Hawksworth,
A. Hingrajiya, D.J. Hofman0000-0002-2449-3845, J.h. Lee0000-0002-5574-4192, D.
S. Lemos0000-0003-1982-8978, A.H. Merrit0000-0003-3922-6464, C.
Mills0000-0001-8035-4818, S. Nanda0000-0003-0550-4083, G.
Oh0000-0003-0744-1063, B. Ozek0009-0000-2570-1100, D.
Pilipovic0000-0002-4210-2780, R. Pradhan0000-0001-7000-6510, E. Prifti, T.
Roy0000-0001-7299-7653, S. Rudrabhatla0000-0002-7366-4225, M.B.
Tonjes0000-0002-2617-9315, N. Varelas0000-0002-9397-5514, Z.
Ye0000-0001-6091-6772, J. Yoo0000-0002-3826-1332

The University of Iowa, Iowa City, Iowa, USA M. Alhusseini0000-0002-9239-470X,
D. Blend, K. Dilsiz870000-0003-0138-3368, L. Emediato0000-0002-3021-5032, G.
Karaman0000-0001-8739-9648, O.K. Köseyan0000-0001-9040-3468, J.-P. Merlo, A.
Mestvirishvili880000-0002-8591-5247, J. Nachtman0000-0003-3951-3420, O. Neogi,
H. Ogul890000-0002-5121-2893, Y. Onel0000-0002-8141-7769, A.
Penzo0000-0003-3436-047X, C. Snyder, E. Tiras900000-0002-5628-7464

Johns Hopkins University, Baltimore, Maryland, USA B.
Blumenfeld0000-0003-1150-1735, L. Corcodilos0000-0001-6751-3108, J.
Davis0000-0001-6488-6195, A.V. Gritsan0000-0002-3545-7970, L.
Kang0000-0002-0941-4512, S. Kyriacou0000-0002-9254-4368, P.
Maksimovic0000-0002-2358-2168, M. Roguljic0000-0001-5311-3007, J.
Roskes0000-0001-8761-0490, S. Sekhar0000-0002-8307-7518, M.
Swartz0000-0002-0286-5070

The University of Kansas, Lawrence, Kansas, USA A. Abreu0000-0002-9000-2215,
L.F. Alcerro Alcerro0000-0001-5770-5077, J. Anguiano0000-0002-7349-350X, P.
Baringer0000-0002-3691-8388, A. Bean0000-0001-5967-8674, Z.
Flowers0000-0001-8314-2052, D. Grove0000-0002-0740-2462, J.
King0000-0001-9652-9854, G. Krintiras0000-0002-0380-7577, M.
Lazarovits0000-0002-5565-3119, C. Le Mahieu0000-0001-5924-1130, J.
Marquez0000-0003-3887-4048, N. Minafra0000-0003-4002-1888, M.
Murray0000-0001-7219-4818, M. Nickel0000-0003-0419-1329, M.
Pitt0000-0003-2461-5985, S. Popescu910000-0002-0345-2171, C.
Rogan0000-0002-4166-4503, C. Royon0000-0002-7672-9709, R.
Salvatico0000-0002-2751-0567, S. Sanders0000-0002-9491-6022, C.
Smith0000-0003-0505-0528, Q. Wang0000-0003-3804-3244, G.
Wilson0000-0003-0917-4763

Kansas State University, Manhattan, Kansas, USA B. Allmond0000-0002-5593-7736,
A. Ivanov0000-0002-9270-5643, K. Kaadze0000-0003-0571-163X, A.
Kalogeropoulos0000-0003-3444-0314, D. Kim, Y. Maravin0000-0002-9449-0666, J.
Natoli0000-0001-6675-3564, D. Roy0000-0002-8659-7762, G.
Sorrentino0000-0002-2253-819X

Lawrence Livermore National Laboratory, Livermore, California, USA F.
Rebassoo0000-0001-8934-9329, D. Wright0000-0002-3586-3354

University of Maryland, College Park, Maryland, USA A.
Baden0000-0002-6159-3861, A. Belloni0000-0002-1727-656X, Y.M.
Chen0000-0002-5795-4783, S.C. Eno0000-0003-4282-2515, N.J.
Hadley0000-0002-1209-6471, S. Jabeen0000-0002-0155-7383, R.G.
Kellogg0000-0001-9235-521X, T. Koeth0000-0002-0082-0514, Y.
Lai0000-0002-7795-8693, S. Lascio0000-0001-8579-5874, A.C.
Mignerey0000-0001-5164-6969, S. Nabili0000-0002-6893-1018, C.
Palmer0000-0002-5801-5737, C. Papageorgakis0000-0003-4548-0346, M.M. Paranjpe,
L. Wang0000-0003-3443-0626

Massachusetts Institute of Technology, Cambridge, Massachusetts, USA J.
Bendavid0000-0002-7907-1789, I.A. Cali0000-0002-2822-3375, M.
D’Alfonso0000-0002-7409-7904, J. Eysermans0000-0001-6483-7123, C.
Freer0000-0002-7967-4635, G. Gomez-Ceballos0000-0003-1683-9460, M. Goncharov,
G. Grosso, P. Harris, D. Hoang, D. Kovalskyi0000-0002-6923-293X, J.
Krupa0000-0003-0785-7552, L. Lavezzo0000-0002-1364-9920, Y.-J.
Lee0000-0003-2593-7767, K. Long0000-0003-0664-1653, A.
Novak0000-0002-0389-5896, C. Paus0000-0002-6047-4211, D.
Rankin0000-0001-8411-9620, C. Roland0000-0002-7312-5854, G.
Roland0000-0001-8983-2169, S. Rothman0000-0002-1377-9119, G.S.F.
Stephans0000-0003-3106-4894, Z. Wang0000-0002-3074-3767, B.
Wyslouch0000-0003-3681-0649, T. J. Yang0000-0003-4317-4660

University of Minnesota, Minneapolis, Minnesota, USA B.
Crossman0000-0002-2700-5085, B.M. Joshi0000-0002-4723-0968, C.
Kapsiak0009-0008-7743-5316, M. Krohn0000-0002-1711-2506, D.
Mahon0000-0002-2640-5941, J. Mans0000-0003-2840-1087, B.
Marzocchi0000-0001-6687-6214, S. Pandey0000-0003-0440-6019, M.
Revering0000-0001-5051-0293, R. Rusack0000-0002-7633-749X, R.
Saradhy0000-0001-8720-293X, N. Schroeder0000-0002-8336-6141, N.
Strobbe0000-0001-8835-8282, M.A. Wadud0000-0002-0653-0761

University of Mississippi, Oxford, Mississippi, USA L.M.
Cremaldi0000-0001-5550-7827

University of Nebraska-Lincoln, Lincoln, Nebraska, USA K.
Bloom0000-0002-4272-8900, D.R. Claes0000-0003-4198-8919, G.
Haza0009-0001-1326-3956, J. Hossain0000-0001-5144-7919, C.
Joo0000-0002-5661-4330, I. Kravchenko0000-0003-0068-0395, J.E.
Siado0000-0002-9757-470X, W. Tabb0000-0002-9542-4847, A.
Vagnerini0000-0001-8730-5031, A. Wightman0000-0001-6651-5320, F.
Yan0000-0002-4042-0785, D. Yu0000-0001-5921-5231

State University of New York at Buffalo, Buffalo, New York, USA H.
Bandyopadhyay0000-0001-9726-4915, L. Hay0000-0002-7086-7641, I.
Iashvili0000-0003-1948-5901, A. Kharchilava0000-0002-3913-0326, M.
Morris0000-0002-2830-6488, D. Nguyen0000-0002-5185-8504, S.
Rappoccio0000-0002-5449-2560, H. Rejeb Sfar, A. Williams0000-0003-4055-6532

Northeastern University, Boston, Massachusetts, USA G.
Alverson0000-0001-6651-1178, E. Barberis0000-0002-6417-5913, J. Dervan, Y.
Haddad0000-0003-4916-7752, Y. Han0000-0002-3510-6505, A.
Krishna0000-0002-4319-818X, J. Li0000-0001-5245-2074, M.
Lu0000-0002-6999-3931, G. Madigan0000-0001-8796-5865, R.
Mccarthy0000-0002-9391-2599, D.M. Morse0000-0003-3163-2169, V.
Nguyen0000-0003-1278-9208, T. Orimoto0000-0002-8388-3341, A.
Parker0000-0002-9421-3335, L. Skinnari0000-0002-2019-6755, B.
Wang0000-0003-0796-2475, D. Wood0000-0002-6477-801X

Northwestern University, Evanston, Illinois, USA S.
Bhattacharya0000-0002-0526-6161, J. Bueghly, Z. Chen0000-0003-4521-6086, S.
Dittmer0000-0002-5359-9614, K.A. Hahn0000-0001-7892-1676, Y.
Liu0000-0002-5588-1760, Y. Miao0000-0002-2023-2082, D.G.
Monk0000-0002-8377-1999, M.H. Schmitt0000-0003-0814-3578, A.
Taliercio0000-0002-5119-6280, M. Velasco

University of Notre Dame, Notre Dame, Indiana, USA G.
Agarwal0000-0002-2593-5297, R. Band0000-0003-4873-0523, R. Bucci, S.
Castells0000-0003-2618-3856, A. Das0000-0001-9115-9698, R.
Goldouzian0000-0002-0295-249X, M. Hildreth0000-0002-4454-3934, K.W.
Ho0000-0003-2229-7223, K. Hurtado Anampa0000-0002-9779-3566, T.
Ivanov0000-0003-0489-9191, C. Jessop0000-0002-6885-3611, K.
Lannon0000-0002-9706-0098, J. Lawrence0000-0001-6326-7210, N.
Loukas0000-0003-0049-6918, L. Lutton0000-0002-3212-4505, J. Mariano, N.
Marinelli, I. Mcalister, T. McCauley0000-0001-6589-8286, C.
Mcgrady0000-0002-8821-2045, C. Moore0000-0002-8140-4183, Y.
Musienko160009-0006-3545-1938, H. Nelson0000-0001-5592-0785, M.
Osherson0000-0002-9760-9976, A. Piccinelli0000-0003-0386-0527, R.
Ruchti0000-0002-3151-1386, A. Townsend0000-0002-3696-689X, Y. Wan, M.
Wayne0000-0001-8204-6157, H. Yockey, M. Zarucki0000-0003-1510-5772, L.
Zygala0000-0001-9665-7282

The Ohio State University, Columbus, Ohio, USA A. Basnet0000-0001-8460-0019,
B. Bylsma, M. Carrigan0000-0003-0538-5854, L.S. Durkin0000-0002-0477-1051, C.
Hill0000-0003-0059-0779, M. Joyce0000-0003-1112-5880, M. Nunez
Ornelas0000-0003-2663-7379, K. Wei, B.L. Winer0000-0001-9980-4698, B. R.
Yates0000-0001-7366-1318

Princeton University, Princeton, New Jersey, USA F.M.
Addesa0000-0003-0484-5804, H. Bouchamaoui0000-0002-9776-1935, P.
Das0000-0002-9770-1377, G. Dezoort0000-0002-5890-0445, P.
Elmer0000-0001-6830-3356, A. Frankenthal0000-0002-2583-5982, B.
Greenberg0000-0002-4922-1934, N. Haubrich0000-0002-7625-8169, G.
Kopp0000-0001-8160-0208, S. Kwan0000-0002-5308-7707, D.
Lange0000-0002-9086-5184, A. Loeliger0000-0002-5017-1487, D.
Marlow0000-0002-6395-1079, I. Ojalvo0000-0003-1455-6272, J.
Olsen0000-0002-9361-5762, A. Shevelev0000-0003-4600-0228, D.
Stickland0000-0003-4702-8820, C. Tully0000-0001-6771-2174

University of Puerto Rico, Mayaguez, Puerto Rico, USA S.
Malik0000-0002-6356-2655

Purdue University, West Lafayette, Indiana, USA A.S.
Bakshi0000-0002-2857-6883, V.E. Barnes0000-0001-6939-3445, S.
Chandra0009-0000-7412-4071, R. Chawla0000-0003-4802-6819, A.
Gu0000-0002-6230-1138, L. Gutay, M. Jones0000-0002-9951-4583, A.W.
Jung0000-0003-3068-3212, D. Kondratyev0000-0002-7874-2480, A.M. Koshy, M.
Liu0000-0001-9012-395X, G. Negro0000-0002-1418-2154, N.
Neumeister0000-0003-2356-1700, G. Paspalaki0000-0001-6815-1065, S.
Piperov0000-0002-9266-7819, V. Scheurer, J.F. Schulte0000-0003-4421-680X, M.
Stojanovic0000-0002-1542-0855, J. Thieman0000-0001-7684-6588, A. K.
Virdi0000-0002-0866-8932, F. Wang0000-0002-8313-0809, W.
Xie0000-0003-1430-9191

Purdue University Northwest, Hammond, Indiana, USA J.
Dolen0000-0003-1141-3823, N. Parashar0009-0009-1717-0413, A.
Pathak0000-0001-9861-2942

Rice University, Houston, Texas, USA D. Acosta0000-0001-5367-1738, T.
Carnahan0000-0001-7492-3201, K.M. Ecklund0000-0002-6976-4637, P.J. Fernández
Manteca0000-0003-2566-7496, S. Freed, P. Gardner, F.J.M.
Geurts0000-0003-2856-9090, W. Li0000-0003-4136-3409, O. Miguel
Colin0000-0001-6612-432X, B.P. Padley0000-0002-3572-5701, R. Redjimi, J.
Rotter0009-0009-4040-7407, E. Yigitbasi0000-0002-9595-2623, Y.
Zhang0000-0002-6812-761X

University of Rochester, Rochester, New York, USA A. Bodek0000-0003-0409-0341,
P. de Barbaro0000-0002-5508-1827, R. Demina0000-0002-7852-167X, J.L.
Dulemba0000-0002-9842-7015, A. Garcia-Bellido0000-0002-1407-1972, O.
Hindrichs0000-0001-7640-5264, A. Khukhunaishvili0000-0002-3834-1316, N.
Parmar, P. Parygin920000-0001-6743-3781, E. Popova920000-0001-7556-8969, R.
Taus0000-0002-5168-2932

The Rockefeller University, New York, New York, USA K.
Goulianos0000-0002-6230-9535

Rutgers, The State University of New Jersey, Piscataway, New Jersey, USA B.
Chiarito, J.P. Chou0000-0001-6315-905X, S.V. Clark0000-0001-6283-4316, D.
Gadkari0000-0002-6625-8085, Y. Gershtein0000-0002-4871-5449, E.
Halkiadakis0000-0002-3584-7856, M. Heindl0000-0002-2831-463X, C.
Houghton0000-0002-1494-258X, D. Jaroslawski0000-0003-2497-1242, O.
Karacheban290000-0002-2785-3762, I. Laflotte0000-0002-7366-8090, A.
Lath0000-0003-0228-9760, R. Montalvo, K. Nash, H. Routray0000-0002-9694-4625,
P. Saha0000-0002-7013-8094, S. Salur0000-0002-4995-9285, S. Schnetzer, S.
Somalwar0000-0002-8856-7401, R. Stone0000-0001-6229-695X, S.A.
Thayil0000-0002-1469-0335, S. Thomas, J. Vora0000-0001-9325-2175, H.
Wang0000-0002-3027-0752

University of Tennessee, Knoxville, Tennessee, USA H. Acharya, D.
Ally0000-0001-6304-5861, A.G. Delannoy0000-0003-1252-6213, S.
Fiorendi0000-0003-3273-9419, S. Higginbotham0000-0002-4436-5461, T.
Holmes0000-0002-3959-5174, A.R. Kanuganti0000-0002-0789-1200, N.
Karunarathna0000-0002-3412-0508, L. Lee0000-0002-5590-335X, E.
Nibigira0000-0001-5821-291X, S. Spanier0000-0002-7049-4646

Texas A&M University, College Station, Texas, USA D. Aebi0000-0001-7124-6911,
M. Ahmad0000-0001-9933-995X, O. Bouhali930000-0001-7139-7322, R.
Eusebi0000-0003-3322-6287, J. Gilmore0000-0001-9911-0143, T.
Huang0000-0002-0793-5664, T. Kamon940000-0001-5565-7868, H.
Kim0000-0003-4986-1728, S. Luo0000-0003-3122-4245, R.
Mueller0000-0002-6723-6689, D. Overton0009-0009-0648-8151, D.
Rathjens0000-0002-8420-1488, A. Safonov0000-0001-9497-5471

Texas Tech University, Lubbock, Texas, USA N. Akchurin0000-0002-6127-4350, J.
Damgov0000-0003-3863-2567, V. Hegde0000-0003-4952-2873, A.
Hussain0000-0001-6216-9002, Y. Kazhykarim, K. Lamichhane0000-0003-0152-7683,
S.W. Lee0000-0002-3388-8339, A. Mankel0000-0002-2124-6312, T.
Peltola0000-0002-4732-4008, I. Volobouev0000-0002-2087-6128

Vanderbilt University, Nashville, Tennessee, USA E. Appelt0000-0003-3389-4584,
Y. Chen0000-0003-2582-6469, S. Greene, A. Gurrola0000-0002-2793-4052, W.
Johns0000-0001-5291-8903, R. Kunnawalkam Elayavalli0000-0002-9202-1516, A.
Melo0000-0003-3473-8858, F. Romeo0000-0002-1297-6065, P.
Sheldon0000-0003-1550-5223, S. Tuo0000-0001-6142-0429, J.
Velkovska0000-0003-1423-5241, J. Viinikainen0000-0003-2530-4265

University of Virginia, Charlottesville, Virginia, USA B.
Cardwell0000-0001-5553-0891, B. Cox0000-0003-3752-4759, J.
Hakala0000-0001-9586-3316, R. Hirosky0000-0003-0304-6330, A.
Ledovskoy0000-0003-4861-0943, C. Neu0000-0003-3644-8627, C.E. Perez
Lara0000-0003-0199-8864

Wayne State University, Detroit, Michigan, USA P.E. Karchin0000-0003-1284-3470

University of Wisconsin - Madison, Madison, Wisconsin, USA A. Aravind, S.
Banerjee0000-0001-7880-922X, K. Black0000-0001-7320-5080, T.
Bose0000-0001-8026-5380, S. Dasu0000-0001-5993-9045, I. De
Bruyn0000-0003-1704-4360, P. Everaerts0000-0003-3848-324X, C. Galloni, H.
He0009-0008-3906-2037, M. Herndon0000-0003-3043-1090, A.
Herve0000-0002-1959-2363, C.K. Koraka0000-0002-4548-9992, A. Lanaro, R.
Loveless0000-0002-2562-4405, J. Madhusudanan Sreekala0000-0003-2590-763X, A.
Mallampalli0000-0002-3793-8516, A. Mohammadi0000-0001-8152-927X, S. Mondal, G.
Parida0000-0001-9665-4575, L. Pétré0009-0000-7979-5771, D. Pinna, A. Savin, V.
Shang0000-0002-1436-6092, V. Sharma0000-0003-1287-1471, W.H.
Smith0000-0003-3195-0909, D. Teague, H.F. Tsoi0000-0002-2550-2184, W.
Vetens0000-0003-1058-1163, A. Warden0000-0001-7463-7360

Authors affiliated with an institute or an international laboratory covered by
a cooperation agreement with CERN S. Afanasiev0009-0006-8766-226X, V.
Andreev0000-0002-5492-6920, Yu. Andreev0000-0002-7397-9665, T.
Aushev0000-0002-6347-7055, M. Azarkin0000-0002-7448-1447, I.
Azhgirey0000-0003-0528-341X, A. Babaev0000-0001-8876-3886, A.
Belyaev0000-0003-1692-1173, V. Blinov95, E. Boos0000-0002-0193-5073, V.
Borshch0000-0002-5479-1982, D. Budkouski0000-0002-2029-1007, V.
Bunichev0000-0003-4418-2072, M. Chadeeva950000-0003-1814-1218, V. Chekhovsky,
R. Chistov950000-0003-1439-8390, A. Dermenev0000-0001-5619-376X, T.
Dimova950000-0002-9560-0660, D. Druzhkin960000-0001-7520-3329, M.
Dubinin850000-0002-7766-7175, L. Dudko0000-0002-4462-3192, A.
Ershov0000-0001-5779-142X, G. Gavrilov0000-0001-9689-7999, V.
Gavrilov0000-0002-9617-2928, S. Gninenko0000-0001-6495-7619, V.
Golovtcov0000-0002-0595-0297, N. Golubev0000-0002-9504-7754, I.
Golutvin0009-0007-6508-0215, I. Gorbunov0000-0003-3777-6606, A.
Gribushin0000-0002-5252-4645, Y. Ivanov0000-0001-5163-7632, V.
Kachanov0000-0002-3062-010X, V. Karjavine0000-0002-5326-3854, A.
Karneyeu0000-0001-9983-1004, V. Kim950000-0001-7161-2133, M. Kirakosyan, D.
Kirpichnikov0000-0002-7177-077X, M. Kirsanov0000-0002-8879-6538, V.
Klyukhin0000-0002-8577-6531, O. Kodolova970000-0003-1342-4251, D.
Konstantinov0000-0001-6673-7273, V. Korenkov0000-0002-2342-7862, A.
Kozyrev950000-0003-0684-9235, N. Krasnikov0000-0002-8717-6492, A.
Lanev0000-0001-8244-7321, P. Levchenko980000-0003-4913-0538, N.
Lychkovskaya0000-0001-5084-9019, V. Makarenko0000-0002-8406-8605, A.
Malakhov0000-0001-8569-8409, V. Matveev950000-0002-2745-5908, V.
Murzin0000-0002-0554-4627, A. Nikitenko99,970000-0002-1933-5383, S.
Obraztsov0009-0001-1152-2758, V. Oreshkin0000-0003-4749-4995, V.
Palichik0009-0008-0356-1061, V. Perelygin0009-0005-5039-4874, S.
Petrushanko0000-0003-0210-9061, S. Polikarpov950000-0001-6839-928X, V.
Popov0000-0001-8049-2583, O. Radchenko950000-0001-7116-9469, R. Ryutin, M.
Savina0000-0002-9020-7384, V. Savrin0009-0000-3973-2485, V.
Shalaev0000-0002-2893-6922, S. Shmatov0000-0001-5354-8350, S.
Shulha0000-0002-4265-928X, Y. Skovpen950000-0002-3316-0604, S.
Slabospitskii0000-0001-8178-2494, V. Smirnov0000-0002-9049-9196, A.
Snigirev0000-0003-2952-6156, D. Sosnov0000-0002-7452-8380, V.
Sulimov0009-0009-8645-6685, E. Tcherniaev0000-0002-3685-0635, A.
Terkulov0000-0003-4985-3226, O. Teryaev0000-0001-7002-9093, I.
Tlisova0000-0003-1552-2015, A. Toropin0000-0002-2106-4041, L.
Uvarov0000-0002-7602-2527, A. Uzunian0000-0002-7007-9020, A.
Vorobyev${}^{\textrm{\textdagger}}$, N. Voytishin0000-0001-6590-6266, B.S.
Yuldashev100, A. Zarubin0000-0002-1964-6106, I. Zhizhin0000-0001-6171-9682, A.
Zhokin0000-0001-7178-5907

†: Deceased
1Also at Yerevan State University, Yerevan, Armenia
2Also at TU Wien, Vienna, Austria
3Also at Institute of Basic and Applied Sciences, Faculty of Engineering, Arab
Academy for Science, Technology and Maritime Transport, Alexandria, Egypt
4Also at Ghent University, Ghent, Belgium
5Also at Universidade Estadual de Campinas, Campinas, Brazil
6Also at Federal University of Rio Grande do Sul, Porto Alegre, Brazil
7Also at UFMS, Nova Andradina, Brazil
8Also at Nanjing Normal University, Nanjing, China
9Now at The University of Iowa, Iowa City, Iowa, USA
10Also at University of Chinese Academy of Sciences, Beijing, China
11Also at China Center of Advanced Science and Technology, Beijing, China
12Also at University of Chinese Academy of Sciences, Beijing, China
13Also at China Spallation Neutron Source, Guangdong, China
14Now at Henan Normal University, Xinxiang, China
15Also at Université Libre de Bruxelles, Bruxelles, Belgium
16Also at an institute or an international laboratory covered by a cooperation
agreement with CERN
17Now at British University in Egypt, Cairo, Egypt
18Now at Cairo University, Cairo, Egypt
19Also at Purdue University, West Lafayette, Indiana, USA
20Also at Université de Haute Alsace, Mulhouse, France
21Also at Department of Physics, Tsinghua University, Beijing, China
22Also at Tbilisi State University, Tbilisi, Georgia
23Also at The University of the State of Amazonas, Manaus, Brazil
24Also at Erzincan Binali Yildirim University, Erzincan, Turkey
25Also at University of Hamburg, Hamburg, Germany
26Also at RWTH Aachen University, III. Physikalisches Institut A, Aachen,
Germany
27Also at Isfahan University of Technology, Isfahan, Iran
28Also at Bergische University Wuppertal (BUW), Wuppertal, Germany
29Also at Brandenburg University of Technology, Cottbus, Germany
30Also at Forschungszentrum Jülich, Juelich, Germany
31Also at CERN, European Organization for Nuclear Research, Geneva,
Switzerland
32Also at Institute of Physics, University of Debrecen, Debrecen, Hungary
33Also at Institute of Nuclear Research ATOMKI, Debrecen, Hungary
34Now at Universitatea Babes-Bolyai - Facultatea de Fizica, Cluj-Napoca,
Romania
35Also at MTA-ELTE Lendület CMS Particle and Nuclear Physics Group, Eötvös
Loránd University, Budapest, Hungary
36Also at Physics Department, Faculty of Science, Assiut University, Assiut,
Egypt
37Also at HUN-REN Wigner Research Centre for Physics, Budapest, Hungary
38Also at Punjab Agricultural University, Ludhiana, India
39Also at University of Visva-Bharati, Santiniketan, India
40Also at Indian Institute of Science (IISc), Bangalore, India
41Also at Birla Institute of Technology, Mesra, Mesra, India
42Also at IIT Bhubaneswar, Bhubaneswar, India
43Also at Institute of Physics, Bhubaneswar, India
44Also at University of Hyderabad, Hyderabad, India
45Also at Deutsches Elektronen-Synchrotron, Hamburg, Germany
46Also at Department of Physics, Isfahan University of Technology, Isfahan,
Iran
47Also at Sharif University of Technology, Tehran, Iran
48Also at Department of Physics, University of Science and Technology of
Mazandaran, Behshahr, Iran
49Also at Helwan University, Cairo, Egypt
50Also at Italian National Agency for New Technologies, Energy and Sustainable
Economic Development, Bologna, Italy
51Also at Centro Siciliano di Fisica Nucleare e di Struttura Della Materia,
Catania, Italy
52Also at Università degli Studi Guglielmo Marconi, Roma, Italy
53Also at Scuola Superiore Meridionale, Università di Napoli ’Federico II’,
Napoli, Italy
54Also at Fermi National Accelerator Laboratory, Batavia, Illinois, USA
55Also at Ain Shams University, Cairo, Egypt
56Also at Consiglio Nazionale delle Ricerche - Istituto Officina dei
Materiali, Perugia, Italy
57Also at Department of Applied Physics, Faculty of Science and Technology,
Universiti Kebangsaan Malaysia, Bangi, Malaysia
58Also at Consejo Nacional de Ciencia y Tecnología, Mexico City, Mexico
59Also at Trincomalee Campus, Eastern University, Sri Lanka, Nilaveli, Sri
Lanka
60Also at Saegis Campus, Nugegoda, Sri Lanka
61Also at National and Kapodistrian University of Athens, Athens, Greece
62Also at Ecole Polytechnique Fédérale Lausanne, Lausanne, Switzerland
63Also at Universität Zürich, Zurich, Switzerland
64Also at Stefan Meyer Institute for Subatomic Physics, Vienna, Austria
65Also at Laboratoire d’Annecy-le-Vieux de Physique des Particules,
IN2P3-CNRS, Annecy-le-Vieux, France
66Also at Near East University, Research Center of Experimental Health
Science, Mersin, Turkey
67Also at Konya Technical University, Konya, Turkey
68Also at Izmir Bakircay University, Izmir, Turkey
69Also at Adiyaman University, Adiyaman, Turkey
70Also at Bozok Universitetesi Rektörlügü, Yozgat, Turkey
71Also at Marmara University, Istanbul, Turkey
72Also at Milli Savunma University, Istanbul, Turkey
73Also at Kafkas University, Kars, Turkey
74Now at stanbul Okan University, Istanbul, Turkey
75Also at Hacettepe University, Ankara, Turkey
76Also at Istanbul University - Cerrahpasa, Faculty of Engineering, Istanbul,
Turkey
77Also at Yildiz Technical University, Istanbul, Turkey
78Also at Vrije Universiteit Brussel, Brussel, Belgium
79Also at School of Physics and Astronomy, University of Southampton,
Southampton, United Kingdom
80Also at IPPP Durham University, Durham, United Kingdom
81Also at Monash University, Faculty of Science, Clayton, Australia
82Also at Università di Torino, Torino, Italy
83Also at Bethel University, St. Paul, Minnesota, USA
84Also at Karamanoğlu Mehmetbey University, Karaman, Turkey
85Also at California Institute of Technology, Pasadena, California, USA
86Also at United States Naval Academy, Annapolis, Maryland, USA
87Also at Bingol University, Bingol, Turkey
88Also at Georgian Technical University, Tbilisi, Georgia
89Also at Sinop University, Sinop, Turkey
90Also at Erciyes University, Kayseri, Turkey
91Also at Horia Hulubei National Institute of Physics and Nuclear Engineering
(IFIN-HH), Bucharest, Romania
92Now at an institute or an international laboratory covered by a cooperation
agreement with CERN
93Also at Texas A&M University at Qatar, Doha, Qatar
94Also at Kyungpook National University, Daegu, Korea
95Also at another institute or international laboratory covered by a
cooperation agreement with CERN
96Also at Universiteit Antwerpen, Antwerpen, Belgium
97Also at Yerevan Physics Institute, Yerevan, Armenia
98Also at Northeastern University, Boston, Massachusetts, USA
99Also at Imperial College, London, United Kingdom
100Also at Institute of Nuclear Physics of the Uzbekistan Academy of Sciences,
Tashkent, Uzbekistan


* [235] Shiyu Liang, Yixuan Li and R. Srikant  “Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks”  In _International Conference on Learning Representations (ICLR)_ , 2018
  * [236] Weixin Liang et al.  “Advances, Challenges and Opportunities in Creating Data for Trustworthy AI”  In _Nature Machine Intelligence_ 4.8, 2022, pp. 669–677
  * [237] Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li  “Energy-based Out-of-distribution Detection”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2020
  * [238] Ze Liu et al.  “Swin transformer: Hierarchical vision transformer using shifted windows”  In _IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2021
  * [239] Zhuang Liu et al.  “A convnet for the 2020s”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022
  * [240] Yifang Ma, Zhenyu Wang, Hong Yang and Lin Yang  “Artificial Intelligence Applications in the Development of Autonomous Vehicles: A Survey”  In _IEEE/CAA Journal of Automatica Sinica_ 7.2, 2020, pp. 315–329
  * [241] Aleksander Madry et al.  “Towards Deep Learning Models Resistant to Adversarial Attacks”  In _International Conference on Learning Representations (ICLR)_ , 2018
  * [242] Dhruv Mahajan et al.  “Exploring the limits of weakly supervised pretraining”  In _European Conference on Computer Vision (ECCV)_ , 2018
  * [243] Andrey Malinin et al.  “Shifts 2.0: Extending The Dataset of Real Distributional Shifts”  In _arXiv e-prints_ , 2022  DOI: 10.48550/arXiv.2206.15407
  * [244] Andrey Malinin et al.  “Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks”  In _Advances in Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks_ , 2021
  * [245] Horia Mania et al.  “Model Similarity Mitigates Test Set Overuse”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2019
  * [246] Horia Mania and Suvrit Sra  “Why Do Classifier Accuracies Show Linear Trends Under Distribution Shift?”  In _arXiv e-prints_ , 2020  DOI: 10.48550/arXiv.2012.15483
  * [247] Jonathan Masci, Ueli Meier, Dan Cireşan and Jürgen Schmidhuber  “Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction”  In _International Conference on Artificial Neural Networks (ICANN)_ , 2011
  * [248] Ninareh Mehrabi et al.  “A survey on bias and fairness in machine learning”  In _ACM Computing Surveys (CSUR)_ 54.6 ACM New York, NY, USA, 2021, pp. 1–35
  * [249] John P Miller et al.  “Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization”  In _International Conference on Machine Learning (ICML)_ , 2021
  * [250] Eric Mintun, Alexander Kirillov and Saining Xie  “On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2021
  * [251] Ishan Misra and Laurens van der Maaten  “Self-Supervised Learning of Pretext-Invariant Representations”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020
  * [252] Forrest E. Morgan et al.  “Military Applications of Artificial Intelligence: Ethical Concerns in an Uncertain World”  RAND Corporation, 2020
  * [253] Jishnu Mukhoti, Pontus Stenetorp and Yarin Gal  “On the Importance of Strong Baselines in Bayesian Deep Learning”  In _Bayesian Deep Learning Workshop at Neural Information Processing Systems (NeurIPS)_ , 2018
  * [254] Thao Nguyen et al.  “Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2022
  * [255] NIST  “AI Risk Management Framework: Second Draft”  In _National Institute of Standards and Technology (NIST)_ , 2022
  * [256] Mehdi Noroozi, Hamed Pirsiavash and Paolo Favaro  “Representation Learning by Learning to Count”  In _IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2017
  * [257] Curtis G Northcutt, Anish Athalye and Jonas Mueller  “Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks”  In _Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track_ , 2021
  * [258] Ziad Obermeyer, Brian Powers, Christine Vogeli and Sendhil Mullainathan  “Dissecting racial bias in an algorithm used to manage the health of populations”  In _Science_ 366.6464 American Association for the Advancement of Science, 2019, pp. 447–453
  * [259] OECD  “Recommendation of the Council on Artificial Intelligence”  In _Organisation for Economic Co-operation and Development (OECD)_ , 2019
  * [260] David Opitz and Richard Maclin  “Popular Ensemble Methods: An Empirical Study”  In _Journal of Artificial Intelligence Research_ 11, 1999, pp. 169–198
  * [261] Michael P. Perrone and Leon N. Cooper  “When Networks Disagree: Ensemble Methods for Hybrid Neural Networks”  In _How We Learn; How We Remember: Toward An Understanding Of Brain And Neural Systems: Selected Papers of Leon N. Cooper_ World Scientific, 1995, pp. 342–358
  * [262] Hieu Pham, Zihang Dai, Qizhe Xie and Quoc V Le  “Meta Pseudo Labels”  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 11557–11568
  * [263] Alec Radford et al.  “Learning Transferable Visual Models from Natural Language Supervision”  In _International Conference on Machine Learning (ICML)_ , 2021
  * [264] Alexander J Ratner et al.  “Data programming: Creating large training sets, quickly”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2016
  * [265] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt and Vaishaal Shankar  “Do ImageNet Classifiers Generalize to ImageNet?”  In _International Conference on Machine Learning (ICML)_ , 2019
  * [266] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto and Percy Liang  “Distributionally Robust Neural Networks”  In _International Conference on Learning Representations (ICLR)_ , 2020
  * [267] Shiori Sagawa et al.  “Extending the WILDS Benchmark for Unsupervised Adaptation”  In _International Conference on Learning Representations (ICLR)_ , 2022
  * [268] Kelley M. Sayler and Daniel S. Hoadley  “Artificial Intelligence and National Security”  In _Congressional Research Service_ , 2020
  * [269] Morgan Klaus Scheuerman, Alex Hanna and Emily Denton  “Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development”  In _Proceedings of the ACM on Human-Computer Interaction_ 5.CSCW2 ACM New York, NY, USA, 2021, pp. 1–37
  * [270] Ludwig Schmidt et al.  “Adversarially Robust Generalization Requires More Data”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2018
  * [271] Seonguk Seo, Paul Hongsuck Seo and Bohyung Han  “Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019
  * [272] Vaishaal Shankar et al.  “Do Image Classifiers Generalize Across Time?”  In _IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2021
  * [273] Apoorva Sharma, Navid Azizan and Marco Pavone  “Sketching curvature for efficient out-of-distribution detection for deep neural networks”  In _Conference on Uncertainty in Artificial Intelligence (UAI)_ , 2021
  * [274] Karen Simonyan and Andrew Zisserman  “Very Deep Convolutional Networks for Large-Scale Image Recognition”  In _arXiv e-prints_ , 2014, pp. arXiv:1409.1556  DOI: 10.48550/arXiv.1409.1556
  * [275] Mannat Singh et al.  “Revisiting weakly supervised pre-training of visual perception models”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022
  * [276] Kihyuk Sohn et al.  “FixMatch: Simplifying semi-supervised learning with consistency and confidence”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2020
  * [277] Robin Sommer and Vern Paxson  “Outside the Closed World: On Using Machine Learning for Network Intrusion Detection”  In _IEEE Symposium on Security and Privacy_ , 2010
  * [278] Banavar Sridhar  “Applications of Machine Learning Techniques to Aviation Operations: Promises and Challenges”  In _International Conference on Artificial Intelligence and Data Analytics for Air Transportation (AIDA-AT)_ , 2020, pp. 1–12
  * [279] Baochen Sun, Jiashi Feng and Kate Saenko  “Return of Frustratingly Easy Domain Adaptation”  In _AAAI Conference on Artificial Intelligence (AAAI)_ , 2016
  * [280] Baochen Sun and Kate Saenko  “Deep CORAL: Correlation Alignment for Deep Domain Adaptation”  In _European Conference on Computer Vision (ECCV)_ , 2016
  * [281] Mingxing Tan and Quoc Le  “EfficientNet: Rethinking model scaling for convolutional neural networks”  In _International Conference on Machine Learning (ICML)_ , 2019
  * [282] Rohan Taori et al.  “Measuring Robustness to Natural Distribution Shifts in Image Classification”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2020
  * [283] Sunil Thulasidasan et al.  “On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2019
  * [284] Hugo Touvron et al.  “Training Data-efficient Image Transformers & Distillation through Attention”  In _International Conference on Machine Learning (ICML)_ , 2021
  * [285] Dimitris Tsipras et al.  “Robustness May Be at Odds with Accuracy”  In _International Conference on Learning Representations (ICLR)_ , 2019
  * [286] Zhengzhong Tu et al.  “MaxViT: Multi-axis vision transformer”  In _European Conference on Computer Vision (ECCV)_ , 2022
  * [287] Vladimir Vapnik  “Principles of Risk Minimization for Learning Theory”  In _Advances in Neural Information Processing Systems (NeurIPS)_ 4, 1991
  * [288] William Wang et al.  “Safer Classification by Synthesis”  In _arXiv_ , 2017
  * [289] Yeming Wen, Dustin Tran and Jimmy Ba  “BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning”  In _arXiv preprint arXiv:2002.06715_ , 2020
  * [290] Mitchell Wortsman et al.  “Model Soups: Averaging Weights of Multiple Fine-Tuned Models Improves Accuracy Without Increasing Inference Time”  In _International Conference on Machine Learning (ICML)_ , 2022
  * [291] Mitchell Wortsman et al.  “Robust Fine-Tuning of Zero-Shot Models”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022
  * [292] Zhirong Wu, Yuanjun Xiong, Stella X Yu and Dahua Lin  “Unsupervised feature learning via non-parametric instance discrimination”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018
  * [293] Qizhe Xie, Minh-Thang Luong, Eduard Hovy and Quoc V Le  “Self-training with noisy student improves ImageNet classification”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020
  * [294] Zhenda Xie et al.  “SimMIM: A Simple Framework for Masked Image Modeling”  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022
  * [295] Han Xu et al.  “Adversarial Attacks and Defenses in Images, Graphs and Text: A Review”  In _International Journal of Automation and Computing_ 17.2, 2020
  * [296] Mingle Xu, Sook Yoon, Alvaro Fuentes and Dong Sun Park  “A Comprehensive Survey of Image Augmentation Techniques for Deep Learning”  In _Pattern Recognition_ 137, 2023, pp. 109347
  * [297] Ruijia Xu, Guanbin Li, Jihan Yang and Liang Lin  “Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation”  In _IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2019
  * [298] Larry Yaeger, Richard Lyon and Brandyn Webb  “Effective Training of a Neural Network Character Classifier for Word Recognition”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 1996
  * [299] Huaxiu Yao et al.  “Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time”  In _Shift Happens Workshop at International Conference on Machine Learning (ICML)_ , 2022
  * [300] David Yarowsky  “Unsupervised Word Sense Disambiguation Rivaling Supervised Methods”  In _Annual Meeting of the Association for Computational Linguistics_ , 1995
  * [301] Jiahui Yu et al.  “CoCa: Contrastive Captioners are Image-Text Foundation Models”  In _Transactions on Machine Learning Research_ , 2022
  * [302] Kun-Hsing Yu, Andrew L. Beam and Isaac S. Kohane  “Artificial intelligence in Healthcare”  In _Nature biomedical engineering_ 2.10 Nature Publishing Group UK London, 2018, pp. 719–731
  * [303] Daniel Ziegler et al.  “Adversarial Training for High-Stakes Reliability”  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2022
  * [304] Remco Zwetsloot, Helen Toner and Jeffrey Ding  “Beyond the AI Arms Race: America, China, and the Dangers of Zero-Sum Thinking”  In _Foreign Affairs_ , 2018

##  A Additional Experimental Setup Details

Here we describe various implementation details of the algorithmic approaches
we applied when training models for the experiments. All code is publically
available.444https://github.com/ancorso/holistic-reliability-evaluation We
conducted our experiments on NVIDIA A100 and V100 GPUs, with the V100s
dictating the maximum batch size due to their lower memory of $12$GB.

###  A.1 Evaluation Metric Details

Below we include additional details regarding the computation of the five
reliability metrics.

  * •

ID Performance. To compute ID performance we used $1024$ samples from the in-
distribution test set provided by the WILDS benchmark. For the Camelyon17
dataset, we scaled the ID performance metric to linearly map values from
($0.7$, $1.0$) to ($0.0$, $1.0$) since all models achieved high performance
($>0.7$).

  * •

DS Robustness. To compute the DS robustness, we evaluated each model on three
distribution shifts, the real-world “val” and “test” shifts provided by the
WILDS Benchmark and a synthetic corruption shift inspired by ImageNet-C [209].
For each shift we compute the DS performance and divide it by the ID
performance and average the results.

  * •

Adversarial Robustness. To compute adversarial robustness, we use the
AutoAttack package555https://github.com/fra31/auto-attack to adversarially
perturb $128$ images and take the ratio of the adversarial performance to ID
performance.

  * •

Calibration. We use the built-in calibration error function provided by
torchmetrics666https://torchmetrics.readthedocs.io/en/stable/classification/calibration_error.html
to compute expected calibration error for the ID predictions and the DS
predictions and average the resutls.

  * •

OOD Detection For each task we used data from the remaining tasks as well as
Gaussian noise and RxRx1 data for OOD detection. For example, the OOD
detection task for iWildCam consisted of distinguishing between iWildCam data
and mixed data from Camelyon17, FMoW, RxRx1 and Gaussian noise. We use the
implementation of OOD detection algorithms provided by the package pytorch-
ood777https://github.com/kkirchheim/pytorch-ood

  * •

HR Score. We used even weights ($0.2$) to compute the HR score for all
experiments.

###  A.2 Data Augmentation Experiments

For these experiments we used a similar training procedure of the WILDS
benchmark, while adding in different forms of data augmentation using the
RandAug888https://pytorch.org/vision/main/generated/torchvision.transforms.RandAugment.html
and
Augmix999https://pytorch.org/vision/main/generated/torchvision.transforms.AugMix.html
implementations built into torchvision. For iWildCam we used an image size of
($448$,$448$), a batch size of 24, and a Resnet-$50$ model. For FMoW we used
an image size of ($224$,$224$), a batch size of 72, and a Densenet-$121$
model. For Camelyon17 we used an image size of ($96$,$96$), a batch size of
168, and a Densenet-$121$ model. The learning rate was sampled from a log
uniform distribution between ${10}^{-4}$ and ${10}^{-2}$, label smoothing
values were sampled from the set ($0.1$, $0.01$, $0.0$), and the optimizer
options considered were ADAM, SGD, and ADAMW.

###  A.3 Adversarial Data Augmentation Experiments

For these experiments we used the same baseline models and input sizes as the
data augmentation experiments. For iWildCam the batch size was set to $12$,
for FMoW it was set to $64$ and for Camelyon17 it was set to $256$. We used
$10$ iterations of the adversarial attack for each training batch, causing the
training time to increase by a factor of about $10$ when PGD is used. The key
hyperparameters are the type of attack method, which we selected randomly
between Projected Gradient Descent and the Fast Gradient Sign Method, and the
strength of the attack, which we selected randomly from ($1/255$, $3/255$,
$8/255$).

###  A.4 Ensembling Experiments

An ensemble was generated from a specified number of trained models by
combining the logits of those models with a set of weights. The weights are
optimized to minimize the loss on the ID validation set. Note that when only
one model is present, this is the same process as temperature scaling (hence
the improvement in calibration for just 1 “ensemble” model). To find the
approximately best configuration of models to ensemble, we build $50$ random
ensembles (each with their own weights optimized on the validation set) and
then we pick the one with the lowest validation loss. This best model is then
evaluated across the other reliability metrics and included in our analysis.
We repeated this experiment at least three times for each size of ensemble for
each dataset.

###  A.5 Fine-Tuning Experiments

For the fine-tuning experiments we obtained the pre-trained models from a
variety of sources. The ViT, Swin Transformer, MaxViT, EfficientNet, and
ConvNeXt models (with both supervised pre-training and SWAG pre-training) came
from torchvision,101010https://pytorch.org/vision/stable/models.html the CLIP
models came from open_clip,111111https://github.com/mlfoundations/open_clip
and the MAE models came from the MAE
repo.121212https://github.com/facebookresearch/mae

For these experiments, we resized images to ($224$, $224$ and used a batch
size of $24$. We trained fro a maximum of $5$ epochs, though we used
validation accuracy to select the best model. When fine-tuning, we took the
approach of freezing most upstream layers and then unfreezing some number of
layers for for training. The number of layers we unfroze was a hyperparameter
that could take on values of ($1$, $2$, $4$, $8$). The learning rate was
sampled from a log uniform distribution between ${10}^{-4}$ and ${10}^{-2}$,
label smoothing values were sampled from the set ($0.1$, $0.01$, $0.0$), and
the optimizer options considered were ADAM, SGD, and ADAMW. To remove some
variation due to hyperparameter selection, we sampled a random configuration
of hyperaparameters and trained all fine-tuned models on that configuration.
We repeated with at least three different hyperparameter configurations for
each datset.

##  B Additional Results

This section provides the results figures for the FMoW and Camelyon17 datasets
that were not included in the main body of the paper. We provide some
additional commentary on the results, though we draw no major conclusions
outside of those summarized in section 5

###  B.1 Dataset Augmentation

Adversarial data augmentation drastically increases the adversarial robustness
of FMoW (see fig. 18) while having little effect on the Camelyon17 models (see
fig. 19), which already had higher levels of adversarial robustness. We
speculate that adversarial robustness is harder to achieve on models with more
classes as there are more decision boundaries that can be reached through
perturbation. AugMix and RandAug led to strong performance improvements across
most metrics for Camelyon17 with more modest improvements centered on
calibration for FMoW.

Figure 18: Effect of various data augmentation strategies for FMoW. Figure 19:
Effect of various data augmentation strategies for Camelyon17.

###  B.2 Model Ensembling

Ensembling shows improvements across all metrics for both Camelyon17 and FMoW
(see fig. 20 and fig. 21 respectively). The number of models where performance
saturates seems to be different for the different datasets. Camelyon17
benefits most from selecting the best model in he batch and actually has lower
performance with 5 models, while FMoW (and iWildCam to a lesser degree) has a
consistent increase on most metrics with more models. The relationship between
between task complexity and improvement from ensembling should be studied
further to understand this relationship.

Figure 20: Ensembles of models for FMoW. Figure 21: Ensembles of models for
Camelyon17.

###  B.3 Fine-tuning

As in the results section, we break down our analysis of fine-tuning into
various dimensions including pre-training algorithm, model type, and model
size.

#### Pre-Training Algorithm

Similar to the results form iWildCam each of the pre-training algorithms
seemed to lead to a consistent improvement across all metrics except
adversarial robustness for both FMoW and Camelyon17 (see fig. 22 and fig. 23
respectively). Interestingly, the adversarial robustness of the fine-tuned
Camelyon17 models dropped significantly compared to the baseline models. We
hypothesize that this is due to the large model sizes (and therefore large
number of degrees of freedom) of the pre-trained models, though more
investigation would be required to understand this phenomenon.

Figure 22: Effect of pre-training algorithm when fine-tuning on FMoW. Figure
23: Effect of pre-training algorithm when fine-tuning on Camelyon17.

#### Model Architecture

When comparing the results across model architectures, we find that the ViT
architecture has the highest HR scores for FMoW and Camelyon17, but only by a
slim margin (see fig. 24 and fig. 25 respectively)

Figure 24: Effect of model architecture when fine-tuning on FMoW. Figure 25:
Effect of model architecture when fine-tuning on Camelyon17.

#### Model Size

Similarly to iWildCam, the trend in model size is not particularly stong with
a slight improvement for larger models with FMoW (see fig. 26) but the
opposite relationship for Camelyon17 (see fig. 27).

Figure 26: Effect of transformer size when fine-tuning on FMoW Figure 27:
Effect of transformer size when fine-tuning on Camelyon17

###  B.4 Sub-Group Aware Training

The algorithms for sub-group aware training do not seem to have a strong
effect for any of the datasets they were trained on, leading to values within
the range of the baselines. This trend is consistent for FMoW and Camelyon17
(see fig. 28 and fig. 29 respectively).

Figure 28: Effect of including sub-group information for FMoW. Figure 29:
Effect of including sub-group information for Camelyon17.

###  B.5 Unlabeled Training Data

The effect of unlabeled training data is less consistent across datasets
compared to the other approaches investigated. While these approaches had a
modest positive effect on the iWildCam and FMoW datasets (see fig. 30), they
showed a larger improvement for Camelyon17 (see fig. 31). For FMoW, the
unlabeled data seemed to mostly improve the DS robustness of the models (true
for all algorithms), whereas for Camelyon17 it was primarily the calibration
the was improved consistently. SwAV was especially effective for Camelyon17
and might indicate that if a good contrastive representation is learned from
unlabeled data, it can drastically improve performance on the downstream task.

Figure 30: Effect of using unlabeled training data for FMoW. Figure 31: Effect
of using unlabeled training data for Camelyon17.

###  B.6 Distribution of Scores

The distribution of scores stratified by groups is shown for FMoW in fig. 32
and for Camelyon17 in fig. 33.

Figure 32: Distribution of scores for FMoW for various groups of models.
Figure 33: Distribution of scores for Camelyon17 for various groups of models.

###  B.7 Metric Correlations

While the group-mean-adjusted correlations showed that there were few
correlations between metrics, there was one set of correlations that seemed
consistent across datasets and that was the relationship between DS robustness
and calibration. Investigating further we found that this correlation was
strongest in the set of fine-tuned models. We show the metric correlations of
just the fine-tuned models in fig. 34. We see that in this group of models the
correlation is even stronger and is again consistent across datasets. We also
observe that there are more positive correlation across other pairs of metrics
in this group of models. This finding may suggest that some metrics become
correlated when fine-tuning from large pre-trained models. It may be
consistent with prior work that has uncovered a relationship between model
confidence and performance on DS data [200]. Additional research will need to
be done to understand this relationship better.

Figure 34: Correlations between reliability metrics for three datasets. Color
indicates Pearson correlation and value indicates $R^{2}$.


# Structural and Electronic properties of the 2D Superconductor CuS with
1$\frac{1}{3}$-valent Copper

I.I. Mazin  Code 6390, Naval Research Laboratory, Washington, DC 20375, USA

###### Abstract

We present first principle calculations of the structural and electronic
properties of the CuS covellite material. Symmetry-lowering structural
transition is well reproduced. However, the microscopic origin of the
transition is unclear. The calculations firmly establish that the so far
controversial Cu valency in this compound is 1.33. We also argue that recently
reported high-temperature superconductivity in CuS is unlikely to occur in the
stoichiometric defect-free material, since the determined Cu valency is too
close to 1 to ensure proximity to a Mott-Hubbard state and superexchange spin
fluctuations of considerable strength. On the other hand, one can imagine a
related system with more holes per Cu in the same structural motif ($e.g.$,
due to defects or O impurities) in which case combination of superexchange and
an enlarged compared to CuS Fermi surface may lead to unconventional
superconductivity, similar to HTSC cuprate, but, unlike them, of an $f$-wave
symmetry.

Copper sulfide in the so-called covellite structure (Fig. 1) has recently
attracted attention due to a new report about possible superconductivity at 40
KRaveau . This report has been met with understandable skepticism, because
previous researchessc1 ; sc2  reported reproducible superconductivity at
rather low temperatures, around 1-2 K. On the other hand, inspection of the
literature reveals that reported physical properties of covellite are
drastically different in different papers. For instance, one paper reported a
well defined Curie-Weiss magnetic susceptibility1st , while others observed a
nearly constant behavior consistent with the Pauli susceptibility in absence
of any local moments.

Figure 1: (color online) Crystal structure of covellite in the high-symmetry
phase. Large yellow spheres indicate sulfure, and blue spheres copper. The
dark spheres form the planar CuS layers (Cu1), and the light spheres form the
warped Cu2S2 bilayers (Cu2). Fat yellow sticks indicate strong covalent bonds
inside the S2-S2 dumbells.

Structural properties of the covellite are also intriguing. At room
temperature it consists of triangular layers of Cu and S, stacked as follows,
using standard hexagonal stacking notation: Cu1 and S1 form layers $A$ and
$B$, at the same height, so that Cu1 has coordination of three and no direct
overlap. Cu2 and S2 form layers $B$ and $C$, so that Cu2 is directly above S1
and bonds with it, too, albeit more weakly than to S2. Thus, compared to the
S2 layer, the Cu2 layer is closer to the Cu1+S1 one, and Cu2 appears to be
inside a tetrahedron, closer to its base. The next layer is again $C$, so that
two S2 atoms are right on top of each other and form a strongly covalent bond,
the shortest bond in this system, essentially making up an S2 molecule.

At $T=55$ K the system spotaneously undergoes a transition from a hexagonal
structure to a lower symmetry orthorhombic structure. To a good approximation,
the transition amounts to sliding the Cu2-S2 plane with respect to the Cu1-S1
plane by 0.2 Å, and the two neighboring Cu2-S2 planes by 0.1 Å with respect to
each other, in the same direction. The bond lengths change very little, one of
the three Cu2-S2 bonds shortens by 0.04 Å, and the S2-S2 bonds lengthens by
0.05 Å, and all other bonds remain essentially unchanged. Note that such
transitions are quite uncommon for metals, but rather characteristic of
insulating Jahn-Teller systems. Transport properties are hardly sensitive to
this transition, which is however clearly seen in the specific heat.

Thus there are three questions to be asked. First, what is the nature of the
low temperature symmetry-lowering? Second, why some experiments indicate pure
Pauli susceptibility, while others observe local moments (through Curie-Weiss
behavior)? Third, why one particular experiment sees indications of high
temperature superconductivity, while others do not? Of, course, there is
always a chance the “outliers” experiments are simply incorrect, but it is
always worth asking the question, whether some sample issues may

possibly account for such discrepancies.

In order to adress the first question, we have performed density functional
calculations (DFT) of both hexagonal (H) and orthorhombic (O) structure.
First, we optimized the crystal structure using the standard VASP program with
default settings (including gradient corrections), and starting from the
experimental structure as reported in Ref. 1st, .

After that, all calculations in the determined crystal structures were
performed using the standard all-electron LAPW code WIEN2k. We have also
verified that the calculated forces in the optimized structures are small
enough. As a technical note, to obtain full convergences in the energy
differences we had to go up to $RK_{\max}=9$.

Table 1: The calculated total energy (meV/cell) of the low-temperature orthorhombic and the high-temperature hexagonal structure, using either the experimental or the calculated optimized parameters. Structural parameters used, as well as selected bond length (Å) are also shown. Note that one unit cell includes 6 formula units. The cell volume is given in $\AA^{3}$ The last column corresponds to the orthorhombic structure with internal coordinates optimized, while keeping the experimental unit cell.  | H-exp | O-exp | H-calc | O-calc | O-c.o.
---|---|---|---|---|---
$a$ | 3.789 | 3.760 | 3.807 | 3.793 | 3.760
$b$ | 3.789 | 6.564 | 3.807 | 6.623 | 6.564
$c$ | 16.321 | 16.235 | 16.496 | 16.453 | 16.235
$z_{Cu2}$ | 0.1072 | 0.1070 | 0.1069 | 0.1077 | 0.1083
$z_{S2}$ | 0.0611 | 0.0627 | 0.0639 | 0.0646 | 0.0651
$y_{Cu1}$ | n/a | 0.6377 | n/a | 0.6227 | 0.6077
$y_{Cu2}$ | n/a | 0.3372 | n/a | 0.3410 | 0.3413
$y_{S1}$ | n/a | 0.3068 | n/a | 0.2917 | 0.2760
$y_{S2}$ | n/a | 0.0008 | n/a | 0.0064 | 0.0069
Cu1-S1 | 3$\times$2.19 | 2$\times 2.18$ | 3$\times 2.20$ | 2$\times 2.20$ | 2$\times 2.20$
|  | 2.17 |  | 2.19 | 2.19
Cu2-S1 | 2.33 | 2.33 | 2.36 | 2.33 | 2.34
Cu2-S2 | 3$\times 2.31$ | 2$\times 2.30$ | 3$\times 2.31$ | 2$\times 2.30$ | 2$\times 2.30$
|  | 2.28 |  | 2.33 | 2.33
S2-S2 | 1.99 | 2.04 | 2.11 | 2.13 | 2.12
Volume | 202.9 | 200.3 | 207.0 | 206.7 | 207.3
Energy | 0 | -85 | -258 | -265 | -189

The results are shown in Table 1. Even though there is some discrepancy
between the calculated and the experimental low-temperature structures (mostly
in terms of an overall overestimation of the equilibrium volume), the correct
symmetry lowering is well reproduced. In fact, given that only one paper has
reported internal positions for the orthorhombic structre, and the same paper
found a Curie-Weiss law, suggesting, as discussed below, crystallographic
defects in their sample, it is fairly possible that the calculations predict
the structure of an ideal material better than this one experiment has
measured.

A more important question now is, what the mechanism for this well-reproduce
symmetry lowering can be? Ionic symmetry-lowering mechanisms (such as Jahn-
Teller) are excluded in a wide band metal like CuS. Typically, a lower
symmetry is stabilized in a metal if it results in a reduced density of states
at the Fermi level (“quasinesting mechanism”). However, the density of states
at the Fermi level does not change at this transition (Fig. 2), and the states
below Fermi actually shift slightly upward. Thus, one-electron energy is not
the reason for the transition. A look at the calculated Fermi surfaces (Fig.
4) shows that while they become more 2D in the orthorhombic structure (the in-
plane plasma frequency remains the same, $\approx 4.0$ eV, while that out of
plane, 1.36 eV, drops by 12%), there is no shrinkage in their size.

Figure 2:  (color online) Calculated density of states in the high-temperature
(“hex”) and low temperature (“ortho”) structures, using in both cases
optimized parameters.

We cannot say definitively what causes the low-temperature symmetry lowering
in CuS, but we can say confidently that it is not van der Waals interaction as
conjectured in Ref. W,  (for one, it would not be reproduced in LGA/GGA
calculations with requested accuracy, and, also, as dicussed below, the
interplanar bonding is covalent, and not van der Waals), and not a typical
metallic mechanism driven by Fermi surface changes. One candidate is ionic
Coulomb interaction. Indeed the calculated Ewald energy is noticeably lower in
the orthorhombic structure. However the Ewald energy is only part of total
electrostatic energy, so from this fact alone one cannot derive definitive
conclusions.

Figure 3:  Calculated band dispersions in the hexagonal structure. The points
$\Gamma$, M and K are in the central plane ($k_{z}=0$) and A, L and H in the
basal plane ($k_{z}=\pi/c$). In the top panel, the width of the lines is
proportional to the amount of the S2-$p_{z}$ character in the corresponding
states.

Let us now discuss the electronic structure. Since the differences between the
two structures are very small, we shall limit our discussion by the high-
temperature hexagonal structure. The calculated band structure is shown in
Fig. 3. Note two sets of bands, one at -7 eV and the other at 1 eV, of strong
S2-$p_{z}$ character. These are bonding and antibonding bands of the S2-S2
dumbells. Historically, there has been a heated discussion of the Cu valency
in this compound, and what is an appropriate ionic model. Both
(Cu${}^{+1})_{3}$(S${}_{2}^{-2})($S${}^{-1})$ (Ref. 1st, ) and
(Cu${}^{+1})_{3}($S${}_{2}^{-1})($S${}^{-2})$ (Ref. W, ) have been discussed,
assuming monovalent copper. On the other hand, XPSXPS  and NQRIrek  data
indicated Cu valency larger than 1, but smaller than 1.5. From our
calculations it is immediately obvious that S1 is divalent, while S2 is
monovalent (the antibonding $p_{z}$ band of the S2-S2 dimer is 1 eV above the
Fermi level, while all S1-derived bands are below the Fermi level), so that Cu
has valency 1.33, and the appropriate ionic model is
(Cu${}^{+4/3})_{3}$(S${}_{2}^{-2})($S${}^{-2}).$

This means that the Cu $d-$band has 1/3 hole per Cu ion, 2.5 times fewer than
in the high-Tc cuprates (optimal doping corresponds to 0.8-0.85 holes). This
may be too far from half filling for strong correlation effects, but it is
nevertheless suggestive of possible spin fluctuations. We will return to this
point later.

Figure 4: (color online) Calculated Fermi surfaces in the hexagonal (top) and
orthorhombic (bottom) structure, viewed along the $c$-axis. Note reduced
$k_{z}$ dispersion in the bottom panel.

In order to understand the Cu $d$ bands near the Fermi level, let us consider
a simple tight binding model with two $d$ orbitals, with $m=\pm 2$
(corresponding to combinations of the $x^{2}-y^{2}$ and $xy$ cubic harmonics,
which belong to the same representation in the hexagonal group). Since these
orbitals are the ones spread most far in the plane, their hybridization with S
is the strongest and they form the highest antibonding states near the
$\Gamma$ point, crossing the Fermi level. Integrating out the S $p_{x,y}$
orbitals we arrive at the following model band structure:

$E_{k}=\frac{1}{2(\varepsilon_{d}-\varepsilon_{p})}\left[(3t_{pd\sigma}^{2}+4t_{pd\pi}^{2})\sum_{i}\cos(\mathbf{k\cdot
R}_{i})\pm(3t_{pd\sigma}^{2}-4t_{pd\pi}^{2})\sqrt{\sum_{i}\cos^{2}(\mathbf{k\cdot
R}_{i})-\sum_{i>j}\cos(\mathbf{k\cdot R}_{i})\cos(\mathbf{k\cdot
R}_{j})}\right],$ (1)

where $t$ are the Cu-S hopping amplitudes, and Ri are the three standard
triangular lattice vectors, $\sum_{i}\mathbf{R}_{i}=0.$ Note that these bands
are degenerate at $\Gamma,$ unless spin-orbit coupling is taken into account.
Near the top of the band the dispersion is isotropic, and away from it the
Fermi surface develops a characteristic hexagonal rosette shape (Fig. 4).

Let us now look at the calculated bands (Fig. 3). It is more instructive to
concentrate on the righ hand side of Fig. 3, where the $k_{z}$ dispersion does
not obscure the states degeneracy. We see, as predicted by the model, three
sets of nearly parabolic bands, each four times degenerate at the point
A=(0,0,$\pi/c).$ One of them is below the Fermi level and two above, forming
the eight FS sheets we see in Fig. 4. The middle bands are predominantly
formed by the Cu1 and the lower (fully occupied) and upper ones by the Cu2,
although there is substantial mixture of all three Cu orbitals. The average
occupation of Cu d orbitals, as described above, is 1/3 hole per Cu, too small
to form a magnetic state, even in LDA+U with $U\sim 8$ eV (as verified by
direct calculations). Formation of an ordered magnetic state is additionally
hindered by the fact that supexchange is this case is antiferromagnetic, and
frustrated, as it should be on a triangular lattice. One may think that
additional hole doping, achieved through Cu vacancies, broken S-S bonds or
interstitial oxygen (note that this structures includes large pores, one per
formula unit, in each Cu-S layer) should bring the $d-$bands closer to half-
occupancy and promote local magnetic moments. Note that in at least one
experimental paper a Curie-Weiss behavior was reported, corresponding to 0.28
$\mu_{B}/$Cu1st , and in another a weak, but inconsistent with the Pauli law,
temperature dependence was foundNMR , while other authors reported
temperature-independent susceptibility.

Figure 5: (color online) A model Fermi surface, calculated using Eq. 1,
overlapped with the wave vectors corresponding to superexchange on a
triangular lattice. The signs show a possible $f-$wave pairing state,
consistent with superexchnage-induced spin fluctuations.

One can speculate that the unexpected high-temperature superconductivity
observed by Raveau $et$ $al$Raveau  is a phenomenon of the same sort, namely
that this superconductivity forms in a portion of a sample, the same portion
where some previous researchers observed local magnetic moments. As discussed
above, it is highly unlikely that a stoichiometric, defectless CuS sample
would support either local moments or unconventional superconductivity.
However, it is of interest to consider a hypothetical situation that would
take place if such moments were present. Indeed in that case one can write
down the superexchange interaction between the nearest neighbors as
antiferromagnetic Heisenberg exchange, in which case in the reciprocal space
it will have the following functional form:

$J(\mathbf{q)=}J\sum_{i}\cos(\mathbf{k\cdot R}_{i}).$ (2)

In Fig. 5 we show an example of a Fermi surface generated for the model band
structure (Eq. 1), for the simplest case of $t_{pd\pi}=0$. The wave vectors
corresponding to the peaks of the superexchange interaction (2) are shown by
arrows. An interesting observation is that for this particular doping this
superexchange interaction (or, better to say, spin fluctuations generated by
this superexchange) would be pairing for a triplet $f-$state shown in the same
picturef . Indeed, the superexchange vectors always span the lobes of the
order parameter with the same sign. Since in a triplet case spin-fluctuations
generate an attractive interaction, it will be pairing for the geometry shown
in Fig. 5. Note that this is opposite to high-$T_{c}$ cuprates, where the
superexchange interaction spans parts with the opposite parts of the $d-$wave
order parameter, but in a singlet channel this interaction is repulsive, and
therefore pairing when the corresponding parts of the Fermi surface have
opposite signs of the order parameter.

While the model Fermi surface shown in Fig. 5 is roughly similar to that
calculated in the stoichiometric CuS, the system at this doping is too far
from the ordered magnetism to let us assume sizeable superexchange-like
magnetic fluctuations. As mentioned, our attempts to stabilize an
antiferromagnetic (more precisely, ferrimagnetic, since we only tried
collinear magnetic patterns) using a triple unit cell failed, even in LDA+U.
One may think of a hole doped system, where superexchange is operative and the
inner Fermi surfaces (albeit not the outer ones) have geometry similar to that
featured in Fig. 5.

Of course, it may not be possible to stabilize a system at sufficient hole
doping and retain the required crystallography. We prefer to think about the
model discussed in the perevious paragraphs as inspired by the CuS covellite,
bit not necessary applicable to actual materials derived from this one. The
reason we paid so much attention to it is that this is a simple generic model,
describing any triangular planar structure with transition metals and ligands
in the same plane, as in the covellite, in case where correlations are
sufficiently strong to bring about spin fluctuations controlled by
superexchange. It is quite exciting that, compared to the popular spin-
fluctuation scenario of superconductivity in cuprates, to which it is
conceptually so similar, this simple generic model results in a completely
different superconducting state, triplet $f$, as opposed to singlet $d$. This
finding may have implications far beyound this particular material and (yet
unconfirmed) superconductivity in it.

## References

  * (1) B. Raveau, T. Sarkar, Solid State Sciences 13, 1874 (2011)
  * (2) A.P. Gonçalves, E.B. Lopes, A. Casaca, M. Dias, M. Almeida, J. Cryst. Growth 310, 2742 (2008).
  * (3) Y. Takano, N. Uchiyama, S. Ogawa, N. Môri, Y. Kimishima, S. Arisawa, A. Ishii, T. Hatano, K. Togano, Physica C 341, 739 (2000).
  * (4) H. Fjellvag, F. Gronvold, S. Stolen, A.F. Andresen, R. Mueller-Kaefu, A. Simon, Z. für Kristallogr. 184, 111 (1988).
  * (5) W. Liang, M.H. Whangbo, Solid State Comm. 85, 405 (1993).
  * (6) C. I. Pearce, R. A. D. Pattrick, D. J. Vaughn, C. M. B. Henderson, and G. van der Laan, Geochim. Cosmochim. Acta 70, 4635 (2006).
  * (7) R. R. Gainov, A. V. Dooglav, I. N. Pen’kov, I. R. Mukhamedshin, N. N. Mozgova, I. A. Evlampiev, and I. A. Bryzgalov, Phys. Rev. B79, 075115 2009.
  * (8) Y. Itoh, A. Hayashi, H. Yamagata, M. Matsumura, K. Koga and Y. Ueda, J. Phys. Soc. Jpn. 65, 1953 (1996).
  * (9) To be specific, this states is $D_{6h}(\Gamma^{-}_{4})$, using the notations of Sigrist and UedaSU , with the basis functions ${\bf\hat{z}}(k_{y}(k_{y}^{2}-3k_{x}^{2})$ or $k_{z}[(k_{y}^{2}-k_{x}^{2}){\bf\hat{y}}-2k_{x}k_{y}{\bf\hat{x}}]$.
  * (10) M. Sigrost and K. Ueda, Rev. Mod. Phys. 63, 239 (1991).