# Effective & Ethical Mentorship in Physics and Astronomy through Grassroots
Organizations

Natasha Latouf NSF Graduate Research Fellow, 2415 Eisenhower Ave, Alexandria,
VA 22314  Department of Physics and Astronomy, George Mason University, 4400
University Drive MS 3F3, Fairfax, VA, 22030, USA  NASA Goddard Space Flight
Center, 8800 Greenbelt Road, Greenbelt, MD 20771, USA  Sellers Exoplanents
Environment Collaboration, 8800 Greenbelt Road, Greenbelt, MD 20771, USA  Emma
Schwartzman Department of Physics and Astronomy, George Mason University, 4400
University Drive MS 3F3, Fairfax, VA, 22030, USA  U.S. Naval Research
Laboratory, 4555 Overlook Ave SW, Washington, DC 20375, USA  Jeffrey McKaig
Department of Physics and Astronomy, George Mason University, 4400 University
Drive MS 3F3, Fairfax, VA, 22030, USA  Sara Doan Department of Physics and
Astronomy, George Mason University, 4400 University Drive MS 3F3, Fairfax, VA,
22030, USA  Joseph Weingartner  Department of Physics and Astronomy, George
Mason University, 4400 University Drive MS 3F3, Fairfax, VA, 22030, USA
Natasha Latouf<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Effective and ethical mentorship practices are crucial to improving
recruitment and retention especially for historically minoritized groups
(HMGs). Spectrum is a diversity, inclusion, equity, and accessibility (DEIA)
grassroots organization committed to empowering equitable excellence through
sustainable change. By improving transparency and DEIA within the fields of
physics and astronomy, we can empower the next generation of diverse
scientists and increase field retention. Starting within our home department
at George Mason University and moving outwards, we ensure our students leave
as advocates for DEIA and AJEDI (access, justice, equity, diversity, and
inclusion) through education and mentorship. Spectrum is providing
professionally trained peer mentors to aid students in all facets of their
academic and personal lives. Although the peer mentoring program existed since
the creation of Spectrum in Spring 2020, we have recently developed and
implemented a formal mentorship training for both student and faculty mentors
thus increasing the quality, trustworthiness, and confidence of our mentors.
Using the latest mentorship research available, this training is developed by
Spectrum for George Mason University, with the ability to implement the
training at any institution.

mentorship

##  1 Reducing Barriers in Physics and Astronomy through Mentorship

It is well established that historical and current systemic barriers have
resulted in disproportionately low rates of recruitment and retention for
historically minoritized groups (HMGs) in physics and astronomy. Women earn
only 21% and 33% of the bachelor’s degrees in physics and astronomy
respectively, percentages that have remained stagnant for nearly the last
decade (2007-2017) despite a 46% increase in the number of physics degrees
awarded in the same time period (Porter & Ivie, 2019). While there has been
some improvement in these numbers for women in the past few years, they still
lag behind the expected values given our population. Students from
historically minoritized racial groups, including Black, Latine, and
Indigenous students, have seen even less improvement, earning only 16% of
bachelor’s degrees, and 6% of PhDs in astronomy in 2014, with very little
improvement since (Jones, 2018), although astronomy is generally considered to
have higher parity rates than physics. It is prudent to note that
traditionally, the definition of HMGs includes only Black, Latine, and
Indigenous students, thus excluding a large number of other HMGs such as Arabs
and Arab Americans. If the definition is expanded to include intersectional
identities (i.e. identities that span several different axes across race,
ethnicity, gender, etc.), the statistics plummet, in some cases so low so as
to be unable to be quantified. For instance, although there were 59,894 PhDs
in physics awarded from 1972 to 2017, only 90 of those were awarded to Black
women (0.15% of the total amount) (Miller, 2019).

For each of these groups and identities, there are four traditionally
challenging barriers faced by HMGs: imposter phenomenon, stereotype threat,
othering, and microaggressions. One could argue there are additional
challenges, including false allies and the overburdening of early career
researchers (ECRs). These barriers keep the rates of recruitment and retention
low by continuing the cycle of oppression and exclusion in physics and
astronomy. Until these barriers are understood and addressed through
mentorship and organizational shifts, the rates of HMGs in physics and
astronomy will not improve to the levels expected given current population
statistics. What follows is an introduction to each barrier, along with the
strategies of mentorship that can aid in mitigating the effects of barriers.
These descriptions will mirror Markle et al. (2022); for further information,
please see Markle et al. (2022) and references therein.

###  1.1 Imposter Phenomenon

Imposter phenomenon is defined as the feeling that one is not truly as capable
or intelligent as others perceive them to be, resulting in the self-perception
that one is a fraud (Clance & Imes, 1978). Along with a deep-rooted conviction
of fraudulence, there is a constant fear of being discovered or exposed by
others that they believe “belong” in the field. Thus, it is difficult for
individuals experiencing imposter phenomenon to be assured that their success
is merited by their ability, hard work, and intelligence (Harvey & Katz, 1985;
Chakraverty, 2020; Feenstra et al., 2020; Markle et al., 2022). While imposter
phenomenon has historically been known as imposter syndrome, the use of the
term syndrome indicates that the fault lies within the individual experiencing
the phenomenon (i.e. it is an internal syndrome), and thus there has been a
shift to using the term imposter phenomenon instead, indicating a fault within
the system that aids in the oppression of individuals in the system. Imposter
phenomenon is more prevalent among students from HMGs, due to the lack of
diverse mentors and role models. The result is an increase in several mental
health concerns, such as depression, anxiety, low self-esteem, low self-
confidence, perfectionism, self-doubt, procrastination, and self-handicapping
(Thompson et al., 1998, 2000; Ferrari & Thompson, 2006; Fraenza, 2016;
Blondeau & Awad, 2018; Markle et al., 2022). The combination of the worsening
mental health and imposter phenomenon looming tends to result in “spirals,”
wherein a student consistently tries to prove to themselves that they do not
belong (e.g., by being too depressed to study well for an exam, and thus
performing poorly). The result is what seems to them to be proof that they do
not belong, which then furthers the imposter phenomenon and associated mental
health concerns, and the cycle continues.

Many studies indicate that having professional or personal relationships with
others in academia, as a peer, mentor, colleague, etc., can significantly
reduce the prevalence of imposter phenomenon, including among students from
HMGs (L. Baker et al., 2014; Feenstra et al., 2020; Markle et al., 2022).
Therefore, to attempt to reduce impostor phenomenon, individual
characteristics and the environment cannot be the only criterion taken into
account by a mentoring program; rather, the fit of the mentor and mentee is
crucial and there is significant evidence to suggest that the proper fit can
reduce imposter phenomenon (Sanford et al., 2015; Cohen & McConnell, 2019;
Barr-Walker et al., 2020; Chakraverty, 2020). This mentor-mentee fit needs to
account for three facets of identity perception: professional identity,
relational identity, and personal identity. These facets of identity indicate
the self-perception related to the roles of their academic career, roles
within family and friend relationships outside of a professional environment,
and the general sense of self and perception of characteristics, respectively
(L. Baker et al., 2014). By arranging mentor-mentee pairings based on their
self-perceptions in these three major areas, imposter phenomenon can be
reduced by building a sense of kinship and thus forging a deeper level of
trust in the relationship. Mentors can assuage worries about belonging with
their own experiences, and thus guide their mentees through the feelings of
imposter phenomenon. While this does not obliviate imposter phenomenon, it
aids students in realizing that others also struggle with belonging and they
are not alone.

###  1.2 Stereotype Threat

The performances of HMGs can be negatively impacted by identity stereotypes
when they are engaging in activities that are related to those stereotypes;
this phenomena is known as stereotype threat (Steele & Aronson, 1995; Thomas &
Erdei, 2018). Woodcock et al. (2012) used a large, diverse sample of HMGs in
STEM, and found that stereotype threat was associated with “scientific
disidentification”, i.e., HMGs find themselves belonging to groups they do not
wish to belong to. This leads to a decline in the affected student’s
persistence toward entering a STEM career. There is extensive literature that
demonstrates that the development of a physics identity is crucial to the
student persisting in STEM. Hazari et al. (2010) developed a model to better
understand two key aspects of a physics student’s identity:

  1. 1.

How the ability to conceptualize physics careers beyond academia impacts
female students’ physics identity and sense of belonging (especially when
considering students who take non-traditional paths in academia or career
trajectories).

  2. 2.

How physics leadership opportunities contributes to building physics identity
and sense of belonging in the field.

Mentors are crucial for providing the role model and support that many
students need, whether in the form of help conceptualizing physics careers
beyond academic or encouragement in the pursuit of leadership opportunities.
However, stereotype threat is also persistent for HMGs at the faculty level,
with similar effects to those noted at the student level (Casad & Bryant,
2016). Since these effects are so prevalent, less faculty positions are filled
by HMGs, and those in place are reluctant to pursue leadership roles that
would make them accessible to HMG students who would benefit from their
advocacy.

While many studies have sought prescriptions to reduce stereotype threat,
several findings are not replicable (see Markle et al. (2022) and references
therein). This is potentially due to the fact that these studies and
affirmation exercises are treating the symptoms of a systemic issue, while
overarching transformations in the social and academic systems in power are
what is needed to treat the disease itself. Culturally responsive mentoring is
defined as mentorship wherein the mentor acknowledges their privilege and uses
the cultural knowledge and frames of reference of HMGs to make learning more
relevant to and effective for them (Committee on Effective Mentoring in STEMM
et al., 2019). Culturally responsive mentoring has been shown to effectively
combat the effects of stereotype threat on HMGs. This is a long-term,
consistent, and institutionalized method to ensure that HMGs at all career
stages that partake receive the affirmation of identity and value they need,
thus further building and bolstering physics identity (Mondisa & McComb, 2015;
San Miguel & Kim, 2015; Committee on Effective Mentoring in STEMM et al.,
2019). This requires the mentor to be comfortable with being uncomfortable,
and assumes constant growth (Harris & Lee, 2019). There are recent studies
that support culturally responsive mentorship as an effective mentorship
method, based on student interviews and self-perception tests, thus providing
preliminary feedback that students benefit from interactions with mentors who
they can see themselves in, i.e. “looks like me” (Thomas & Erdei, 2018).
Looking specifically at undergraduate HMGs, those who received culturally
responsive mentoring felt greater confidence as researchers and were more
committed to continuing through a graduate degree (Haeger & Fresquez, 2016).
This builds on prior research wherein students named their role models and
mentors as their primary inspiration to pursue STEM degrees and persist in the
field through any challenges, thus further cementing the importance of
culturally responsive mentoring (Meador, 2018). Having a network of peers can
also be an effective strategy for mitigating stereotype threat as found in
Block et al. (2011), as well as other barriers.

###  1.3 Othering

Low levels of social connection and “belongingness” (i.e., the perception of
acceptance, connection, social support, feelings of mattering, respect, and
being valued) can result in a reduction of interest in STEM and low retention
in students from HMGs (Stachl & Baranger, 2020). On the other hand, having a
strong connection to one’s peers built on positive interactions can result in
higher rates of academic achievement, persistence in field, and retention in
STEM (Zaniewski & Reinholz, 2016; Ito & McPherson, 2018; Kim & Sinatra, 2018;
Robnett et al., 2019). There are several factors that lead students from HMGs
to feel isolated, especially at Predominantly White Institutions (PWIs), thus
leading to a social separation from their peers. These include a poor campus
climate, racist/sexist/xenophobic/other hateful interactions with White peers
or faculty members, and having a very strong connection to their racial
identity that is marginalized or bullied at their institution (Brown, 1990;
Locks et al., 2008; Thelamour et al., 2019). These feelings of isolation are a
powerful impediment to fostering the culturally inclusive environment at the
institutional level (that begins at the departmental level) necessary for both
the success of STEM students in HMGs (Fisher et al., 2019) and increasing
retention in students from HMGs in STEM (Johnson & Elliott, 2020).

Mentoring relationships can drastically improve a mentee’s sense of belonging
(Chan, 2008; Stolle-McAllister et al., 2011; Apriceno et al., 2020). This is
not specific to just students from HMGs; Apriceno et al. (2020) found that
students from all groups who have an engaged mentor early in their career
develop greater senses of belonging and self-confidence in academia and in
STEM. Engaged mentorship does not have to only come from a senior career
member in STEM; mentoring that is received from peers has a profound impact on
a student’s sense of belonging. Connections between students who are closer in
career stage (ideally with similar backgrounds or interests) can both
emphasize and validate their struggles within STEM and thus aid in the
development of their STEM/physics identities (Maton et al., 2000; Inzlicht &
Good, 2006; Zaniewski & Reinholz, 2016). Moreover, studies specifically within
physics have shown that students who participate as mentors in constellation
mentorship programs (programs that connect mentors and mentees from multiple
career stages) bolster their own physics identity, as well as aid in striking
a sustainable balance between their science and their commitment to the field,
leading to increased long-term satisfaction (Godwin et al., 2016; Pando,
2022). By combining the forces of student mentorship with positive senior
career researcher (SCR) mentors (i.e. emphasizing constellation mentorship
models), the chances of success of students from HMGs increases significantly.
Student mentors will naturally have more available time and a more similar
perspective, while the SCRs will be able to provide a better “big picture”
view of the field and help guide a mentee through career transition periods
with greater efficacy.

###  1.4 Microaggressions

The inability for a student to relate to a mentor that looks like them or can
empathize with them on multiple axes can intensify imposter phenomena and, in
many cases, can lead to the student being pushed out of the field. These
feelings are exacerbated by the constant microaggressions faced by HMGs on the
basis of race, gender, disability, sexual orientation, and other identity
markers (Tatum, 2000; Swim et al., 2001; Evans & Broido, 2002; Alleyne, 2004;
Sue, 2010; Young et al., 2015).

The term “microaggression” describes a covert, unconscious, subtly destructive
phenomenon related to individualized experiences with othering in society
(Young et al., 2015). Although used as an umbrella term, multiple different
forms of aggression can occur in the workplace environment that can be defined
as “microaggressions”. Sue (2010) researched microaggressions in everyday life
and developed a typology of microaggressions to better understand their
occurrence and impact. There are three distinct types of microaggressions:
microinsults, microinvalidation, and microassault. Microinsults convey
rudeness or insensitivity through behaviors, actions, or verbal remarks in an
attempt to diminish, exclude, or negate the experienced reality of HMGs.
Microinvalidations feature the use of actions to exclude, negate, or nullify
the experienced reality of HMGs (i.e. more insidiously excluding a person to
other them). Finally, microassaults are explicit put-downs to hurt the victim,
e.g. outright racism (Sue, 2010; Young et al., 2015). Microaggressions are
used to create and maintain a hostile environment for HMGs, leading to a
higher probability of burn out or field departure. Microaggressions cut
deepest when perpetuated by those who might be considered allies. Harris & Lee
(2019) noted that mentoring can combat the effects of microaggressions, but
all rely strongly on the connection between the mentor and mentee. To combat
microaggressions, the pair must:

  1. 1.

Openly discuss cultural responsibility and acknowledge different experiences
due to race or ethnicity.

  2. 2.

Openly discuss what the mentee needs specifically in relation to race- or
ethnicity- based challenges.

  3. 3.

Openly discuss the mentor’s plans for addressing social injustices and
supporting the mentee.

  4. 4.

Specifically the mentor must utilize positive micromessaging in order to
continuously affirm competency and foster growth (e.g. ”great question”,
”that’s a clever solution, but why don’t we try it this way”, etc.)

This open communication is crucial, as it ensures that the mentor/mentee pair
are in agreement on fundamental growth points and understand the goals that
they are each working towards. Especially when concerning topics relating to
minoritization, proper training on effective communication can make the
difference in a successful or harmful relationship.

###  1.5 False Allies and Overburdening ECRs

Although many people may call themselves an ally to the cause, and perpetuate
an image of a helpful advocate, in many cases, well-intentioned white male
physicists also aid in perpetuating inequity in the field both consciously and
unconsciously. Many have internalized sexism or racism, and justify their
inaction using three main themes found by Dancy & Hodari (2022). For further
information on each of the three themes listed below, as well as an in depth
discussion of the phenomena, please see Dancy & Hodari (2022):

  1. 1.

Physical Distancing: By persisting in the belief that racism and sexism do not
happen in their classrooms, departments, research groups, institutions, or the
field of physics, inaction is justified.

  2. 2.

The Problem is Too Large: By claiming that the problem is on too large a scale
for one person to have an impact, i.e. blaming the K-12 system, historical
legacy of sexism and racism, socioeconomic dynamics, or differential
expectations of parenting, inaction is justified.

  3. 3.

I am Helpless to Act: By refusing to see the issues, and thus using that as
justification for inaction, or claiming that action would create negative
consequences for the harrasser, they claim that they are not capable of acting
and thus inaction is justified.

By presenting themselves as powerless, yet still using the “ally” moniker,
many physicists in positions of power can perpetuate inequity and
microaggressions within their field and institution while maintaining a white-
savior complex. The burden of creating a more equitable field falls then on
the shoulders of historically minoritized and early career groups such as
undergraduate students, graduate students, and post-doctoral researchers. Many
organizations have formed to push the field forward, relying on early career
researchers (ECRs) volunteering to make a change. Groups such as Black in
Astro111https://www.blackinastro.com/ (Woods & Walker, 2022), and League of
Underrepresented Minoritized Astronomers
(LUMA)222https://www.lumamentoring.com/ are working to combat the state of the
field by providing a community for HMGs, although both are created and led by
ECRs. These organizations have resulted in an increase in acts of service,
done voluntarily and designed to improve the state of the field. However, they
have faced opposition at every turn, including while trying to implement
mentorship practices. One such organization was developed internally at George
Mason University. The development of these resource hubs and community groups
for students is crucial to improving recruitment and retention. However, the
most effective method of retention lies in ethical mentorship practices that
address the common barriers to STEM while specifically aiding HMGs (Committee
on Effective Mentoring in STEMM et al., 2019; Markle et al., 2022).

##  2 Applying Recommended Best Practices for Mentorship

Figure 1: Spectrum’s designed logo with tagline.

###  2.1 Spectrum: Empowering Equitable Excellence

In February of 2020, four physics and astronomy students from HMGs, including
author Natasha Latouf, were approached by a faculty member and asked to
develop the first Code of Professional Conduct for the George Mason University
(GMU) Department of Physics and Astronomy. Previous to this request, GMU
Department of Physics and Astronomy had had no such code, exposing students
and faculty from HMGs to suffer from the effects of sexism, racism,
retaliation, etc. The Code of Professional Conduct was drafted and adopted
into the GMU Department of Physics and Astronomy in May of 2020.
Spectrum333https://gmuspectrum.squarespace.com/ was officially founded shortly
thereafter.

Spectrum is a student-created and student-run access, justice, equity,
diversity, and inclusion (AJEDI) grassroots advocacy organization in the GMU
Department of Physics and Astronomy. The primary goal of this organization is
to drive sustainable change to improve transparency, diversity, equity, and
inclusion within the fields of physics and astronomy, starting within our home
department. To this end, a significant responsibility that Spectrum has taken
on is to ensure that our students not only leave our department as well-
rounded scientists, but also as strong ambassadors and advocates for AJEDI,
wherever their path leads them next. We believe in empowering equitable
excellence within physics and astronomy through education and mentorship.

The process of building Spectrum was arduous, having faced opposition within
the department and difficulties due to the COVID-19 pandemic, including
forming a community in a virtual environment. Spectrum was also unfunded for
the first two years of its existence, relying solely on the volunteering of
students. Despite these many barriers, the founders, leaders, and members of
Spectrum persevered, speaking out and conducting numerous events.

Over the last four years, Spectrum has undertaken several initiatives to
improve the status of the field. Spectrum co-founders worked with department
faculty and HR staff to develop and present a Bias and Inclusion training,
hosted as a part of the GMU departmental colloquia series. This training
garnered over 60 attendees, becoming the most well-attended seminar of the
semester to date. We also founded the Professional Development Seminar Series,
which focused on providing assistance in both traditional (i.e. resume
writing, science publication, research opportunities, etc.) and non-
traditional (mentorship, recognizing red flags in academia, imposter
phenomena, etc.) professional development. This included hosting invited guest
speakers with specialized expertise in topics of interest while also
developing and presenting talks ourselves to give a near-peer perspective on
strategies to be successful in a STEM career. This seminar series was weekly,
moving to bi-weekly, and all talks were recorded and uploaded to our YouTube
channel444https://www.youtube.com/channel/UCjkWQLgKgcW9YhtoFOibyOg to allow
watchers to avoid Zoom fatigue and learn on their personal schedule (Elbogen
et al., 2022). We also hosted several extended events and workshops, including
Pathways to Government, a Virtual Teaching and Learning Open Discussion,
Personal Statements and Presenting Workshop, Recognizing and Overcoming
Imposter Syndrome, etc.

To continue the learning experience for all members of the department,
Spectrum also undertook the task of building a website filled with resources
and educational opportunities. This includes a monthly themed webpage for DEIA
in honor of each diversity celebration month spotlighting at least three
scientists within each relevant month. The website provides an avenue of
education for students to learn from the past, and for HMG students to see
themselves represented. The website also features an extensive resources tab,
with focus on Skill Building, Career and Grad School Planning, Mental Health
and Time Management, DEIA in STEM and Society, Science Outreach, etc. to
provide students with a central location to access any manner of resources,
and break down traditionally gatekept knowledge barriers.

In an effort to build community with other STEM fields and more public
audiences, Spectrum has undertaken a number of social activities. Spectrum co-
hosted a Minorities in STEM mixer, collaborating with six other minority in
STEM groups across GMU’s College of Science in order to allow members of each
organization to mingle and make valuable connections across STEM fields.
Leaders also revitalized the Graduate Research Assistant Office, investing
personal funds and time to paint walls, purchase and build furniture, and
provide decor, in order to encourage a community among the graduate students
in the department. To increase education and interest external to the GMU
Department, Spectrum participated in the local Fairfax Fall Festival to shine
a spotlight on our mentoring and outreach initiatives to $ge$5,000 attendees.
Using this platform, we were able to inform prospective students about our
open source resources, increase engagement in science by providing an “Ask a
Scientist” section, and encouraged participation from younger children
interested in growing their scientific passion.

###  2.2 Spectrum Mentorship Program

Despite all of the other, very useful efforts of Spectrum, the most important
has been the development of a premier mentorship program within the physics
and astronomy department. Our mentoring program utilizes the theoretical
framework of a physics student’s identity, described above and by Hazari et
al. (2010), and was developed after the constellation mentorship model. After
an extensive intake process asking both personality and mentoring needs-based
questions, we match each mentee to two mentors based on factors that will
provide the most optimal personality match, such as most active time of day
and personal and professional interests. These matches are designed to best
reinforce and mold their student identities. By having two mentors per mentee,
we increase the diversity of perspectives each mentee is exposed to, and
reduce the time commitment of each individual mentor. Additionally, Dennehy &
Dasgupta (2017) demonstrated that when female peer mentors are introduced to
undergraduate female students in career transition periods (e.g. deciding on
graduate school, or industry, etc.), there was a marked increase in belonging
and self-efficacy, resulting in long-term higher retention. Similar results
were found for race and ethnicity (Trawick Monroe-White et al., 2021). As
such, we strive to match each mentee with at least one mentor of the same
gender and ethnic background, if available, and provide avenues to contact
other mentors of their same gender and ethnic background to further increase
the likelihood of retention. This aids in overcoming the barriers presented by
othering, microaggressions, and a lack of a sense of belonging. We have also
appointed faculty members to the Spectrum Advisory Board, who are available to
provide guidance upon request to Spectrum leadership, and as appointed,
trusted faculty mentors for students who are looking for the faculty
perspective. Our mentoring program currently services approximately
approximately 30% of the department (as of Spring 2024).

The primary source of training for mentors until Fall 2023 has been through
relatively informal education, through a compilation of resources discussing
best practices distributed through Spectrum’s website, with targeted
discussions on topics such as imposter phenomena, self-advocacy, stress
management, and effective communication. Our faculty mentoring committee’s
training has been equally informal. While some of our faculty mentors have had
the opportunity to take formal training in mentorship, others work primarily
off of their lived experience, best practices from the literature, and
Spectrum events, such as the Bias and Inclusion training departmental seminar.

As our mentorship program has grown, it became necessary to formalize and
standardize our training procedures to ensure that our mentors are well
equipped with the proper tools to best support their mentees, while also
knowing how to take care of themselves while advising students in potentially
sensitive situations. As described in Markle et al. (2022), mentor training is
a crucial component to an effective, holistic, non-dyadic mentorship model. A
holistic, non-dyadic mentorship model focuses on network-based mentorship as
an alternative to traditional mentorship (i.e., a dyadic model) wherein there
is only the connection between mentor and mentee. Holistic, non-dyadic
mentorship enables mentees to also be mentors and have a peer group at each
level, building a continuing web of aid and mentorship (e.g., a faculty member
(or members) mentors a grad student, who mentors an undergrad, while the grad
student and undergrad are connected to their peer communities). Studies have
found that students who had significant mentoring relationships with multiple
levels of expertise (faculty member, post doctoral fellow, graduate student)
reported better productivity, stronger scientific identity, and a higher
likelihood of pursuing higher education (Aikens et al., 2017; Joshi et al.,
2019). Similarly, Collings et al. (2014) found that undergrads who
participated in peer mentoring (i.e. being mentored by others in their peer
group) were less likely to leave their university. This cascading effect of
mentorship has been proven time and time again (Committee on Effective
Mentoring in STEMM et al., 2019; Markle et al., 2022), illustrating that a
holistic model is likely to be the most effective for recruitment and
retention.

Markle et al. (2022) and Committee on Effective Mentoring in STEMM et al.
(2019) both present figures (Figures 1 and 4-1d, respectively) describing the
set-up of a holistic non-dyadic mentorship model. In Markle et al. (2022), the
graduate students all look to one faculty member as a mentor, although it is
implied that there should be more than one faculty member in the model; there
is also no placement for ”mentor nodes”, which are rotating, short-term
positions that are filled by mentors for specific projects or academic
transition periods. In Committee on Effective Mentoring in STEMM et al.
(2019), they describe the differences between each mentorship model with panel
d presenting a network model; however, it is a combination of panels c and d
that best describe a constellation mentorship model, as panel d does not
feature a location for the multi-level mentorship or equivalent-level peers.
Extending this work, Figure 2 describes a robust holistic, non-dyadic
mentorship model, combining elements from both of the excellent figures
presented above. It includes the concept of “mentor nodes”, mentor slots which
are long-term relationships that emphasize both academic and personal growth,
peer slots to aid in reducing othering and increasing belonging, and mentee
slots to assist with developing leadership and physics identity. Each position
is described, and connected to the self in the central position. It is after
this model that we develop our training materials and mentorship model within
Spectrum. This mentorship model requires an effective strategic plan to
thrive, and that indicates the need for training.

Figure 2: We present a model of a holistic, non-dyadic mentorship model
(constellation mentorship) that includes all of the available mentorship
positions: mentor node, mentor, peer, and mentee, surrounding the central
self. Each position is described in detail, and one can imagine this same
model repeating per individual.

However, there is typically little to no training provided to mentors that
centers around research-backed best-practices in mentorship(Pfund et al.,
2006). It is generally accepted that a one-time short training is insufficient
for building durable skills, and that regularly occurring meetings or sessions
over time are more effective for imbuing critical points of mentorship
practices. However, inplementing these changes will require a greatly
reconfigured mentorship training. Given that the physics and astronomy
department at GMU has had no mentorship training in the past, the introduction
of smaller steps is more likely to engage busy faculty and students alike.

####  2.2.1 Mentorship Training

To ensure that all members are working from similar baselines of knowledge, we
were awarded an American Astronomical Society Education and Professional
Development Mini-Grant (AAS EPD) in Summer 2022 to develop a mentorship
training workshop, and the training was implemented in Fall 2023. The
workshops were designed based on lessons learned from the past years of
Spectrum mentoring, topics requested by our current and former mentors, and
the most recent research in effective mentorship practices (Appendix N,
National Academies of Sciences & Medicine, 2021). The training was split into
two parts, designed for faculty and students to take separately, so as to
encourage a safe space for growth without an intrinsic power dynamic. In
addition to all the topics covered in the SCR training, the training for the
ECRs includes topics to protect students, such as recognizing academic red
flags, entering into an effective mentorship relationship, and improving
communication skills. The workshops both emphasize the five core attributes of
effective mentorship developed by Pfund et al. (2016): research,
interpersonal, psycho-social/career, cultural responsibility, and sponsorship.
Each of these areas is presented and explained, along with the barriers
presented above and how each attribute can aid in mitigating the barriers. We
also encourage the mentors to think about their own behavior, and how they
could be contributing to either the barriers or the attributes, as well as how
to improve and grow.

As described by Flaxman et al. (1988), mentorship can be separated into two
categories: planned and natural. Planned mentorship is conducted through
structured programs, wherein mentors and mentees are carefully paired through
a formal process (such as Spectrum’s program). This is the most traditionally
used form of mentorship. In contrast, natural mentorship is conducted through
friendship, teaching, etc. and is neither formally assigned nor carefully
paired. These differences are crucial to the application of mentorship, and by
educating the next generation of mentors on these differences, we will
supplement understanding in mentorship expectations and increase the many
positive impacts of effective mentorship. While mentorship is sometimes viewed
as a position one can appoint themselves to (i.e. declaring oneself a mentor),
the title of “mentor” should be bestowed either by the mentee (natural) or by
programmatic appointment (planned). Additionally, there are distinct and
important differences between the roles of faculty members, advisors, and
mentors in student fostering. Faculty members are not required to mentor all
students, but can serve as mentors in multiple cases. Advisors are also not
required to mentor their students, however they are obligated to develop the
research and academic careers of their charges. Mentors are responsible for
the personal (e.g. integrity, responsibility, etc.) and scientific development
of their mentees, within any boundaries agreed upon in the mentor/mentee pair,
and should drive open communication and guidance. One of the most crucial
roles for mentors is to provide culturally-responsible mentorship. Mentors
should not be allies, but rather they should be advocates for students from
HMGs. Advocacy is a far more powerful role which utilizes the privilege of the
SCR to plead for or on behalf of students from HMGs, and create environments
wherein these students can thrive (Harris & Lee, 2019). Research has suggested
that race and ethnicity has a marked impact on mentoring relationships,
resulting in the relationships forming more slowly, if at all, in the case of
a mixed-race mentorship (Pruitt & Isaac, 1985). For an advocacy-based,
culturally responsible mentorship relationship to stand a higher chance of
success, the mentor must be educated or seek education on their students’
marginalized identities and experiences, be comfortable with facing discomfort
and their own prejudices in order to unlearn them, and recognize the benefits
and privileges they have experienced due to systemic oppression (Kendall,
2003). By doing this, the mentor will learn and practice advocacy and
mentorship from a responsible standpoint, while wielding their privilege as a
tool to better the field. To be clear, these mentors will still make mistakes;
advocacy and cultural responsibility does not require perfection, but rather
participants who will actively learn and from their mistakes (Kendall, 2003).

Our training broke down the roles and misconceptions for SCRs, and then built
up a healthy understanding of the role of mentor, a mastery of the boundaries
involved in each role, and a perception of their own comfort level using the
research described herein. Not only did this enable better ECR-SCR relations,
but the same awareness will help ECRs understand their own power in self-
appointing a mentor. The ECR training included all the same lessons, with the
addition of recognizing red flags in an mentoring or advising relationship as
well as additional information on open dialogues. Both training sessions
included a broad overview of peer mentorship: defining what a “good” mentor
is, setting proper expectations, understanding healthy communication in
mentorship, and providing resources that include accessible, baseline
definitions on a variety of AJEDI topics.

Although AJEDI is frequently lauded as an important aspect of field
development, one of the largest barriers in AJEDI education is misinformation,
compounded by a lack of unified and trusted resources. For many people at all
career levels, there is an ingrained misunderstanding of what AJEDI is, how
one can be an effective ally or advocate, or even where to begin unlearning
fundamental biases. Correcting this misunderstanding has always been crucial
for ethical progress in the field, but more recently has become important for
accessing funding. For example, all NASA and NSF-funded proposals now require
an Inclusion Plan that will soon be graded as part of the overall proposal.
This will force a significant portion of researchers in the field (many of
whom are AAS members) to upgrade their understanding of AJEDI topics. In the
process, they will inevitably find that the existing materials are
insufficient, and will resort to superficial language and vague descriptions.
Our training materials help support education in AJEDI topics such as anti-
racism, oppression systems, etc. that are crucial for culturally responsible
advocacy mentorship.

Additionally, the training delved into a discussion of the bystander
intervention, defined as an individual or group actively addressing a
situation they deem problematic that they are observing, and what doing so
could mean for an ECR or peer. We recommended seeking out a bystander
intervention training, which is frequently offered through a variety of
academic institutions. Our workshops also included role-play case studies in
which debriefs were used to teach mastering reacting, reporting structure,
etc. in difficult situations while in a controlled environment. These role-
plays were separated into group conflicts (e.g. student group conflicts),
faculty conflicts, crossing boundaries, and work-life balance. This was
replicated for the mentee and mentor training, however, the scenarios were
altered from student point of view to faculty point of view. These case
studies were particularly well received and experienced increased engagement.

Most importantly, the training provided extensive resources to both the
mentors and mentees - including the literature upon which the training was
based - so as to continue their education, further their training
opportunities, and a mentoring compact. A mentoring compact is an agreement
established between the mentor and mentee, including individual
responsibilities, possible consequences, and joint goals; it is explicitly a
compact and not a contract since there are no legal implications or methods to
uphold the agreement, but rather a good faith agreement to ensure that the
boundaries of the mentoring relationship are discussed. This mentoring compact
is meant to encourage open dialogue between mentors and mentees by providing
the boundaries and opened-ended questions that drive relationship progress
(e.g. setting goals, expectations, accountability, etc.). By providing the
compact, we remove the apprehension of broaching a “big picture” conversation,
and decrease the chance of missing an important topic. The mentoring compact
and case studies have been provided in Section 4. A Spectrum starter kit that
contains the entire mentoring training as well as all necessary items to start
a Spectrum sister organization are
available.555https://drive.google.com/drive/folders/1Bg9ZV9vmdGg6ht9xeuRsexRRu647KzZi?usp=sharing

##  3 Summary, Conclusions, & Future Works

There exists a wealth of literature on mentorship research regarding an
effective, ethical, culturally responsible, and advocacy-based mentorship
program. This paper presents a high-level summary that can be used to drive
development of such a mentorship program. Spectrum has developed and tended to
such a mentoring program, giving students a space to seek out peer mentorship
that is guided by research. This space is safe for students, encourages open
dialogue, and breaks down barriers that have prevented students from
succeeding in STEM: imposter phenomenon, stereotype threat, othering,
microaggressions, false allies, and overburdening ECRs.

The development of Spectrum was difficult, requiring immense effort and
perseverance. We believe that although the onus should not be on the students
to drive community improvement, they also should not struggle in the effort to
participate in improvement. As such, Spectrum has developed a starter kit of
information that is a living document, allowing students to start their own
sister organizations without the trial and error, and removing the burden of
development and research from the next generation of students. This includes a
research-backed mentorship training funded by AAS, a description of the
mentorship program with the tools needed to develop their own, all logos and
design aspects, a description of the professional development lunch talks, as
well as description of the leadership structure. The mentorship training
materials are also broken down into subsections for faculty/SCRs and student
mentors, to allow for a fine-tuned training without the added power dynamics.

In future works, we will lead the development of mentorship checklists,
designed to prompt critical thought on the environment of a research group. We
will design two checklists: one aimed at the mentee level, i.e. early career
researchers such as students and post-docs, and one aimed at the mentor level,
i.e. faculty, civil servants, PIs, etc. These checklists will work in tandem,
with similar questions on either side in order to compare results for a future
IRB study on the efficacy of the checklist.

The mentee checklist will aim to invoke thought about the feelings of the
student individually (i.e., do they feel safe, valued, heard by their mentor?)
and observation of their research environment (i.e. how does the group
interact? Are other students generally engaged? Do you feel that you are
treated differently than other students?). Ideally for the mentee, this will
draw a picture of whether their displeasure stems from their mentor, research,
or a combination of both. The mentee can then take next steps with confidence
on what will improve their happiness and engagement within the field.

The mentor checklist will aim to invoke thought about how the PI interacts
with students (i.e., do you feel that you communicate with each of your
students equitably? Do you provide requested guidance?) and how they have
structured the research group (i.e. are there multiple collaborators your
students can turn to? Are students encouraged to work together?). Ideally for
the mentor, this will draw a picture of whether they are mentoring well on
fundamental principles, if their group dynamic is established in an ethical
and effective manner, or if there are deficiencies in the mentorship scheme
present. It will also ideally prompt introspection on the inherent power
dynamics that exist in any mentor/mentee relationship, regardless of how the
mentor views the relationship.

These checklists will ideally be implemented into the developed mentorship
training described above as a measure to constantly check in throughout a
mentoring relationship. This will also work in combination with the mentorship
compacts that will be encouraged at the beginning of each mentoring
relationship.

By focusing on constellation mentorship models and giving students a safe
space to grow amongst peers, bolstering physics identity through peer and
faculty mentoring, and pointedly confronting and overcoming common barriers in
STEM, we can increase the rates of retention and eventually recruitment in
physics and astronomy.

The authors and Spectrum leaders would like to offer their sincerest thanks to
the other co-Founders of Spectrum: Dr. Jenna Cann, Carly Solis, and Kathryn
Fernandez. The authors and Spectrum leaders would like to thank the Faculty
Advisory Board of Spectrum, Dr. Paula Danquah-Brobby, Dr. Fernando Camelli,
Dr. Erhai Zhao, Dr. Patrick Vora, and Brooke Vaughn for their constant support
of Spectrum and their aid at each step. The authors thank Dr. Ferah Munshi for
providing funding. The authors also gratefully acknowledge Dr. Tom Rice and
the American Astronomical Society (AAS) for providing the education mini-grant
with which this work was made possible. N. L. gratefully acknowledges
financial support from an NSF GRFP.

##  4 Appendix

###  4.1 Mentoring Compact

We provide herein the mentoring compact that was developed as a part of the
mentorship training. This compact will aid in prompting open dialogue between
mentors and mentees, without putting the onus on one party or another to
discuss all facets of a successful relationship. The compact will be filled
out together, simultaneously, and ideally re-discussed with some consistency.

Figure 3: Page 1 of the Mentoring Compact Agreement, meant to be filled out by
mentors and mentees together. Figure 4: Page 2 of the Mentoring Compact
Agreement.

###  4.2 Faculty Case Studies

We provide herein the case studies that were developed for faculty
specifically to understand their roles in student and mentee lives. These
present difficult scenarios that require careful thought before answering,
while also covering a number of potential topics (bystander intervention,
white knighting, etc.). These case studies are provided, with two minutes to
consider, before entering into discussion and gently redirecting to the best
approach.

Figure 5: The first two provided case studies for faculty, with questions.
Figure 6: The third provided case studies for faculty, with questions. Figure
7: The fourth and fifth provided case studies for faculty, with questions.
Figure 8: The fifth and sixth provided case studies for faculty, with
questions.

## References

  * Aikens et al. (2017) Aikens, M. L., Robertson, M. M., Sadselia, S., et al. 2017, CBE—Life Sciences Education, 16, ar34, doi: 10.1187/cbe.16-07-0211
  * Alleyne (2004) Alleyne, A. 2004, Counselling & Psychotherapy Research, 4, 4, doi: 10.1080/14733140412331384008
  * Apriceno et al. (2020) Apriceno, M., Levy, S. R., & London, B. 2020, Journal of College Student Development, 61, 643, doi: 10.1353/csd.2020.0061
  * Barr-Walker et al. (2020) Barr-Walker, J., Werner, D. A., Kellermeyer, L., & Bass, M. B. 2020, Evidence Based Library and Information Practice, 15, 24
  * Block et al. (2011) Block, C. J., Koch, S. M., Liberman, B. E., Merriweather, T. J., & Roberson, L. 2011, The Counseling Psychologist, 39, 570, doi: 10.1177/0011000010382459
  * Blondeau & Awad (2018) Blondeau, L. A., & Awad, G. H. 2018, Journal of Career Development, 45, 253
  * Brown (1990) Brown, D. 1990, Virginia Law Review, 295
  * Casad & Bryant (2016) Casad, B. J., & Bryant, W. J. 2016, Frontiers in Psychology, 8
  * Chakraverty (2020) Chakraverty, D. 2020, International Journal of Doctoral Studies, 15, 433
  * Chan (2008) Chan, A. W. 2008, Mentoring & Tutoring: Partnership in Learning, 16, 263, doi: 10.1080/13611260802231633
  * Clance & Imes (1978) Clance, P. R., & Imes, S. A. 1978, Psychotherapy: Theory, research & practice, 15, 241
  * Cohen & McConnell (2019) Cohen, E. D., & McConnell, W. R. 2019, The Sociological Quarterly, 60, 457
  * Collings et al. (2014) Collings, R., Swanson, V., & Watkins, R. 2014, Higher Education, 68, 927, doi: 10.1007/s10734-014-9752-y
  * Committee on Effective Mentoring in STEMM et al. (2019) Committee on Effective Mentoring in STEMM, Board on Higher Education and Workforce, Policy and Global Affairs, & National Academies of Sciences, Engineering, and Medicine. 2019, The Science of Effective Mentorship in STEMM, ed. A. Byars-Winston & M. L. Dahlberg (Washington, D.C.: National Academies Press), doi: 10.17226/25568
  * Dancy & Hodari (2022) Dancy, M., & Hodari, A. 2022, arXiv e-prints, arXiv:2210.03522, doi: 10.48550/arXiv.2210.03522
  * Dennehy & Dasgupta (2017) Dennehy, T. C., & Dasgupta, N. 2017, Proceedings of the National Academy of Sciences, 114, 5964–5969, doi: 10.1073/pnas.1613117114
  * Elbogen et al. (2022) Elbogen, E. B., Lanier, M., Griffin, S. C., et al. 2022, Cyberpsychology, Behavior, and Social Networking, 25, 409
  * Evans & Broido (2002) Evans, N. J., & Broido, E. M. 2002, in Addressing Homophobia and Heterosexism on College Campuses, ed. E. P. Cramer (Binghamton, NY: Harrington Park Press), 29–42
  * Feenstra et al. (2020) Feenstra, S., Begeny, C. T., Ryan, M. K., et al. 2020, Frontiers in psychology, 11, 3206
  * Ferrari & Thompson (2006) Ferrari, J. R., & Thompson, T. 2006, Personality and Individual Differences, 40, 341
  * Fisher et al. (2019) Fisher, A. J., Mendoza-Denton, R., Patt, C., et al. 2019, PloS one, 14, e0209279
  * Flaxman et al. (1988) Flaxman, E., et al. 1988
  * Fraenza (2016) Fraenza, C. B. 2016, Online Learning, 20, 230
  * Godwin et al. (2016) Godwin, A., Potvin, G., Hazari, Z., & Lock, R. 2016, Journal of Engineering Education, 105, 312–340, doi: 10.1002/jee.20118
  * Haeger & Fresquez (2016) Haeger, H., & Fresquez, C. 2016, CBE—Life Sciences Education, 15, ar36
  * Harris & Lee (2019) Harris, T. M., & Lee, C. N. 2019, Communication Education, 68, 103, doi: 10.1080/03634523.2018.1536272
  * Harvey & Katz (1985) Harvey, J. C., & Katz, C. 1985, If I’m so successful, why do I feel like a fake?: The impostor phenomenon (St. Martin’s Press New York)
  * Hazari et al. (2010) Hazari, Z., Sonnert, G., Sadler, P. M., & Shanahan, M.-C. 2010, Journal of Research in Science Teaching, doi: 10.1002/tea.20363
  * Inzlicht & Good (2006) Inzlicht, M., & Good, C. 2006, in Stigma and group inequality (Psychology Press), 143–164
  * Ito & McPherson (2018) Ito, T. A., & McPherson, E. 2018, Frontiers in psychology, 9, 400169
  * Johnson & Elliott (2020) Johnson, A., & Elliott, S. 2020, Journal of microbiology & biology education, 21, 10
  * Jones (2018) Jones, N. 2018, 562, doi: 10.1038/D41586-018-06834-Y
  * Joshi et al. (2019) Joshi, M., Aikens, M. L., & Dolan, E. L. 2019, BioScience, 69, 389, doi: 10.1093/biosci/biz039
  * Kendall (2003) Kendall, F. E. 2003, How to Be an Ally if You Are a Person with Privilege, http://www.scn.org/friends/ally.html
  * Kim & Sinatra (2018) Kim, A. Y., & Sinatra, G. M. 2018, International journal of STEM education, 5, 1
  * L. Baker et al. (2014) L. Baker, V., J. Pifer, M., & A. Griffin, K. 2014, International Journal for Researcher Development, 5, 83, doi: 10.1108/IJRD-04-2014-0003
  * Locks et al. (2008) Locks, A. M., Hurtado, S., Bowman, N. A., & Oseguera, L. 2008, The Review of Higher Education, 31, 257, doi: 10.1353/rhe.2008.0011
  * Markle et al. (2022) Markle, R. S., Williams, T. M., Williams, K. S., et al. 2022, 7, 674669
  * Maton et al. (2000) Maton, K. I., Hrabowski III, F. A., & Schmitt, C. L. 2000, Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 37, 629
  * Meador (2018) Meador, A. 2018, School Science and Mathematics, 118, 61
  * Miller (2019) Miller, J. V. 2019, Physics Today, doi: 10.1063/PT.3.31335
  * Mondisa & McComb (2015) Mondisa, J.-L., & McComb, S. A. 2015, Mentoring & Tutoring: Partnership in Learning, 23, 149, doi: 10.1080/13611267.2015.1049018
  * National Academies of Sciences & Medicine (2021) National Academies of Sciences, E., & Medicine. 2021, Pathways to Discovery in Astronomy and Astrophysics for the 2020s, doi: 10.17226/26141
  * Pando (2022) Pando, J. 2022, in APS Meeting Abstracts, Vol. 2022, APS April Meeting Abstracts, B06.001
  * Pfund et al. (2016) Pfund, C., Byars-Winston, A., Branchaw, J., Hurtado, S., & Eagan, K. 2016, AIDS and Behavior, 20, 238, doi: 10.1007/s10461-016-1384-z
  * Pfund et al. (2006) Pfund, C., Maidl Pribbenow, C., Branchaw, J., Miller Lauffer, S., & Handelsman, J. 2006, Science, 311, 473, doi: 10.1126/science.1123806
  * Porter & Ivie (2019) Porter, A. M., & Ivie, R. 2019, Women in Physics and Astronomy: 2019.  https://www.aip.org/statistics/reports/women-physics-and-astronomy-2019
  * Pruitt & Isaac (1985) Pruitt, A. S., & Isaac, P. D. 1985, The Journal of Negro Education, 54, 526, doi: 10.2307/2294713
  * Robnett et al. (2019) Robnett, R. D., Nelson, P. A., Zurbriggen, E. L., Crosby, F. J., & Chemers, M. M. 2019, Emerging Adulthood, 7, 180
  * San Miguel & Kim (2015) San Miguel, A. M., & Kim, M. M. 2015, Journal of Career Development, 42, 133, doi: 10.1177/0894845314542248
  * Sanford et al. (2015) Sanford, A. A., Ross, E. M. R. M., Blake, S. J., & Cambiano, R. L. 2015, Advancing Women in Leadership Journal, 35, 31
  * Stachl & Baranger (2020) Stachl, C. N., & Baranger, A. M. 2020, PloS one, 15, e0233431
  * Steele & Aronson (1995) Steele, C. M., & Aronson, J. 1995, Journal of personality and social psychology, 69, 797
  * Stolle-McAllister et al. (2011) Stolle-McAllister, K., Sto. Domingo, M. R., & Carrillo, A. 2011, Journal of Science Education and Technology, 20, 5, doi: 10.1007/s10956-010-9228-5
  * Sue (2010) Sue, D. W. 2010, Microaggressions in Everyday Life: Race, Gender, and Sexual Orientation (Hoboken, NJ: Wiley)
  * Swim et al. (2001) Swim, J. K., Hyers, L. L., Cohen, L. L., & Ferguson, M. J. 2001, Journal of Social Issues, 57, 31, doi: 10.1111/0022-4537.00200
  * Tatum (2000) Tatum, B. 2000, in Readings for Diversity and Social Justice: An Anthology on Racism, Antisemitism, Sexism, Heterosexism, Ableism, and Classism, ed. M. Adams, W. J. Blumenfeld, R. Castaneda, H. Hackman, M. Peters, & X. Zuñiga (New York, NY: Routledge), 39–44
  * Thelamour et al. (2019) Thelamour, B., George Mwangi, C., & Ezeofor, I. 2019, Journal of Diversity in Higher Education, 12, 266
  * Thomas & Erdei (2018) Thomas, N., & Erdei, R. 2018, in 2018 CoNECD-The Collaborative Network for Engineering and Computing Diversity Conference
  * Thompson et al. (1998) Thompson, T., Davis, H., & Davidson, J. 1998, Personality and Individual differences, 25, 381
  * Thompson et al. (2000) Thompson, T., Foreman, P., & Martin, F. 2000, Personality and Individual differences, 29, 629
  * Trawick Monroe-White et al. (2021) Trawick Monroe-White, T., Joseph, M., Tolbert, N., Tola, J., & Haynes, J. 2021, doi: doi:10.1177/23733799211049238
  * Woodcock et al. (2012) Woodcock, A., Hernandez, P. R., Estrada, M., & Schultz, P. 2012, Journal of personality and social psychology, 103, 635
  * Woods & Walker (2022) Woods, P., & Walker, A. 2022, Nature Astronomy, 6, 622, doi: 10.1038/s41550-022-01704-0
  * Young et al. (2015) Young, K., Anderson, M., & Stewart, S. 2015, Journal of Diversity in Higher Education, 8, 61, doi: 10.1037/a0038464
  * Zaniewski & Reinholz (2016) Zaniewski, A. M., & Reinholz, D. 2016, International journal of STEM education, 3, 1


# Quality and Computation Time in Optimization Problems

Zhicheng He  Department of Computer Science and Engineering
Southern University of Science and Technology
Shenzhen, China
<EMAIL_ADDRESS>
###### Abstract

Optimization problems are crucial in artificial intelligence. Optimization
algorithms are generally used to adjust the performance of artificial
intelligence models to minimize the error of mapping inputs to outputs.
Current evaluation methods on optimization algorithms generally consider the
performance in terms of quality. However, not all optimization algorithms for
all test cases are evaluated equal from quality, the computation time should
be also considered for optimization tasks. In this paper, we investigate the
quality and computation time of optimization algorithms in optimization
problems, instead of the one-for-all evaluation of quality. We select the
well-known optimization algorithms (Bayesian optimization and evolutionary
algorithms) and evaluate them on the benchmark test functions in terms of
quality and computation time. The results show that BO is suitable to be
applied in the optimization tasks that are needed to obtain desired quality in
the limited function evaluations, and the EAs are suitable to search the
optimal of the tasks that are allowed to find the optimal solution with enough
function evaluations. This paper provides the recommendation to select
suitable optimization algorithms for optimization problems with different
numbers of function evaluations, which contributes to the efficiency that
obtains the desired quality with less computation time for optimization
problems.

###### Index Terms:

Optimization Algorithms, Evolutionary Algorithms, Bayesian Optimization,
Computation time.

##  I Introduction

To date, optimization problems are crucial in many applications, particularly
the artificial intelligence. An optimization problem is the task of finding
the desired solution from the feasible solution space. Optimization algorithms
have been widely to search the desired solutions for the optimization
problems. The current optimization algorithms are generally trial and error
methods. A trial and error method of problem solving where a candidate
solution is generated and tested to check if it is successful. If it is not
successful, then another candidate solution is generated to be alternative and
tested until a satisfactory solution is found.

Testing the performance of the candidate solution is a crucial step in an
optimization task. Current studies generally test the performance of the
candidate solution from the aspect of quality over function evaluations.
However, the computation time is also a crucial measurement, particularly the
applications in the real world. For the real optimization tasks, users
generally consider more about how long the optimization algorithms cost to
obtain the satisfactory solutions. Therefore, this paper studies both quality
and computation time in optimization problems. We select and test the well-
known optimization algorithms of Bayesian optimization and evolutionary
algorithms on the well-known test functions of Schwefel, Griewank, Rastrigin.
These algorithms are evaluated in terms of both quality and computation time.
The pros and cons of these algorithms are discussed from the aspect of quality
and computation time.

The results indicate that Bayesian optimization performs faster convergence
than other EAs in the early function evaluations. At the end of optimization,
Bayesian optimization and evolutionary algorithms perform the similar quality
with enough function evaluations. However, the results demonstrate that
Bayesian optimization performs substantially increasing computation time over
the number of function evaluations as that is well-known. To solve problems
faster, Bayesian optimization is suitable to the optimization problems that
can be solved in the limited function evaluations. Evolutionary algorithms
perform slower convergence but the much less computation time than BO, and
finally obtain the similar even higher quality at the end of optimization.
Therefore, evolutionary algorithms are suitable for the optimization problems
that are allowed to be tested with enough function evaluations.

In summary, this paper analysis the optimization algorithms in terms of
quality and computation time. This provides the recommendation to evaluate and
select the optimization algorithms for different optimization problems by
considering both quality and computation time instead of the one-for-all
evaluation of quality. Therefore, the suitable optimization algorithms can be
applied to the optimization problems, particularly the optimization tasks that
are sensitive to the computation time.

##  II Related Work

There are many optimization algorithms that can be used to solve optimization
problems. In general, the mainstream algorithms include Bayesian optimization
and evolutionary algorithms. These optimization algorithms are generally
evaluated in terms of quality over the number of function evaluations.
Typically, Bayesian optimization is a sequential design strategy that usually
employed to optimize expensive-to-evaluate functions [1]. Evolutionary
algorithms are a type of heuristic optimization algorithms that are inspired
by biological mechanisms of evolution, e.g., genetic algorithms, evolution
strategy, Covariance matrix adaptation evolution strategy (CMA-ES), particle
swarm optimization. In this section, we review the evaluations in the existing
works of optimization problems from the aspects of quality and computation
time of the mainstream optimization algorithms.

The current studies of Bayesian optimization generally evaluate the
performance of the optimization algorithms in terms of quality, i.e., the
quality over the number of function evaluations. Ru et al. [2] developed a new
optimization algorithm of the fast information-theoretic Bayesian optimisation
method to avoid the need for sampling the global minimiser, thus reducing
computation time. The performance of the optimization algorithms are evaluated
on three benchmark test functions over the number of function evaluations. Wu
et al. [3] presented a hyperparameter tuning algorithm for machine learning
models based on Bayesian optimization. Similarly, the algorithm was evaluated
on the well-known dataset MNIST over the number of function evaluations. The
recent works [4, 5] used Bayesian optimization to learn locomotion tasks in
modular robots, which the performance is evaluated in terms of quality as
well. These Bayesian optimization algorithms are generally evaluated over the
number of function evaluations for optimization problems. However, the
computation time of Bayesian optimization grows substantially with the number
of function evaluations. Therefore, the evaluation over the number of function
evaluations is unfeasible for large numbers of evaluations, which greatly
limits its applicability.

In the field of evolutionary algorithms, the methods in the current studies
are basically evaluated in terms of quality. Giráldez et al. [6] proposed a
novel data structure to reduce the time complexity of the evolutionary
algorithm by a fast evaluation of examples from the dataset. Deb et al. [7]
propose a computationally efficient evolutionary algorithm for real-parameter
optimization. The performance of these algorithms are evaluated over function
evaluations. Thiele et al. [8] proposed a new preference-based evolutionary
algorithm for multi-objective optimization. The quality in terms of multi-
objective are used to evaluate the performance of the algorithms. Young et al.
[9] proposed multi-node evolutionary neural networks as a method for
automating network selection. Unsurprisingly, the performance of the method is
evaluated with the quality over the number of function evaluations. The
current studies [10, 11, 12] applied evolutionary algorithms to learn the
controllers of robots. However, their performance are evaluated in terms of
quality. An recent study [13] applied the evolutionary algorithms to the
multiclass classification and evaluated the performance in terms of quality.
The performance of the optimization in classification is commonly evaluated on
classification accuracy. However, the running computation time on
classification model training, parameter-tuning, and testing is missing, which
is a very important concern for the real-world applications.

Interestingly, the recent study [14] evaluated the efficiency of optimization
algorithms with the quality over computation time. Finally, the algorithms are
evaluated from both quality and computation time. Computation time is an
important factor of the performance of the vectorization and line detection
algorithms, especially in industrial applications. Regarding the current
studies of computation time in machine learning, an early study [15] proposed
to evaluate optical flow algorithms for practical applications with a
2-dimensional scale that is a combination of quality and computation time.
Wenyin and Dori [16] proposed using the elapsed time of running algorithms to
evaluate the performance in terms of computation time. Zhang et al. [17]
provided an up-to-date comparison on the computation time of eleven state-of-
the-art classification algorithms. The computation time of optimization
algorithms is a crucial measurement, particularly the optimization tasks in
the real world.

In conclusion, the existing works of optimization problems are generally
evaluated in terms of quality over the number of function evaluations, but not
computation time. We suggest that computation time should be the reasonable
measurement to evaluate and select optimization algorithms. Therefore, in this
paper we investigate the quality and computation time of optimization
algorithms.

##  III Methodology

In this paper, we select and test the well-known optimization algorithms of
Bayesian optimization and evolutionary algorithms on the test functions. In
this section, we address the well-known optimization algorithms and the test
functions.

###  III-A Optimization Algorithms

####  III-A1 Bayesian Optimization

Init: $n$ initial solutions
$\overrightarrow{\mathcal{X}}_{1},\overrightarrow{\mathcal{X}}_{2},...,\overrightarrow{\mathcal{X}}_{i},...,\overrightarrow{\mathcal{X}}_{n}$;
$\overrightarrow{\mathcal{X}}_{i}$ is a $j$ dimensional vector
$\overrightarrow{\mathcal{X}}_{i}(x_{1},x_{2},...,x_{j})$.

Evaluate: to obtain fitness $f_{1},f_{2},...,f_{i},...,f_{n}$.

Create: initial GP: $\mu(\overrightarrow{\mathcal{W}}_{1:n})$,
$\sigma^{2}(\overrightarrow{\mathcal{W}}_{1:n})$.

1 for  _$k=n+1,n+2,...$_ do

2 select a new solution $\overrightarrow{\mathcal{X}}_{k}$ by optimizing
acquisition function $u(\overrightarrow{\mathcal{X}}_{k})$:
$\overrightarrow{\mathcal{X}}_{k}=\arg\max_{\overrightarrow{X}_{k}}u(\overrightarrow{\mathcal{X}}_{k}|\overrightarrow{X}_{1:k-1})$

3 evaluate the new solution $\overrightarrow{X}_{k}$ to obtain the objective
value $f_{k}$.

4 augment data
$(\overrightarrow{X}_{1:k},f_{1:k})=\\{\overrightarrow{X}_{1:(k-1)},(\overrightarrow{X}_{k},f_{k})\\}$.

5 update GP: $\mu_{k}(\overrightarrow{X}_{1:k})$,
$\sigma_{k}^{2}(\overrightarrow{X}_{1:k})$.

return data $(\overrightarrow{X}_{1:k},f_{1:k})$.

Algorithm 1 Bayesian optimization

Bayesian Optimization is a well-known data efficient optimization algorithm,
which builds a Gaussian process (GP) approximation of the objective function
$f$ of a task. It selects candidate solutions by the guiding of an acquisition
function that balances exploration and exploitation. In this paper, we use
_Gaussian Process Upper Confidence Bound_ (GP-UCB) that is a state-of-the-art
acquisition function [18]. In addition, the choice of the kernel function for
the GP is crucial, as it determines the shape of the functions that the GP can
fit [18]. We use Matérn function kernel that is currently a popular choice.
The pseudocode of Bayesian optimization is shown in algorithm 1. For more
details of Bayesian optimization, please refer to the literature [18].

####  III-A2 Evolutionary Algorithms

Evolutionary Algorithms are population-based, stochastic optimization
algorithms based on the Darwinian principles of evolution [19]. The
optimization process works by iteratively creating a new solution. The main
operators in the optimization process are fitness-based parent selection,
reproduction through mutation and/or crossover, and fitness-based survivor
selection. In this paper, we use the standard real-valued evolutionary
algorithm, evolutionary strategy. For more details of evolutionary algorithms,
please refer to the well-known book [18].

Init: $n$ initial solutions
$\overrightarrow{X}_{1},\overrightarrow{X}_{2},...,\overrightarrow{X}_{i},...,\overrightarrow{X}_{n}$
and the objective value $f_{1},f_{2},...,f_{i},...,f_{n}$.

1 for  _$k=1,2,3,...$_ do

2 if  _$k\neq 1$_ then

3 generate candidate sulotions $\overrightarrow{X}_{((k-1)*n+1):(k*n)}$ by
mutation and crossover.

4

5 evaluate solutions $\overrightarrow{X}_{((k-1)*n+1):(k*n)}$ to obtain the
objective values $f_{((k-1)*n+1):(k*n)}$.

6 select solutions from $X_{((k-1)*n+1):(k*n)}$ and their
$f_{((k-1)*n+1):(k*n)}$.

7 update solutions $\overrightarrow{X}_{1:(k*n)}$ and their objective values
$f_{1:(k*n)}$.

return data ($\overrightarrow{X}_{1:(k*n)},f_{1:(k*n)}$).

Algorithm 2 Typical Evolutionary Algorithms [20]. functions | definition | Global minimum | domain | scalable | local optima
---|---|---|---|---|---
Schwefel | $f(\overrightarrow{X})=418.9829d-\sum_{i=1}^{d}x_{i}\sin{\sqrt{|x_{i}|}}$ | $f(\overrightarrow{X}^{*})=0,\text{at~{}}\overrightarrow{X}^{*}=(420.97,...,420.97)$ | $x_{i}\in[-500,500]$ | $i=1,...,d$ | multiple
Griewank | $f(\overrightarrow{X})=\sum_{i=1}^{d}\frac{x_{i}^{2}}{4000}-\prod_{i=1}^{d}\cos{(\frac{x_{i}}{\sqrt{i}})}+1$ | $f(\overrightarrow{X}^{*})=0,\text{at~{}}\overrightarrow{X}^{*}=(0,...,0)$ | $x_{i}\in[-600,600]$ | $i=1,...,d$ | multiple
Rastrigin | $f(\overrightarrow{X})=10d+\sum_{i=1}^{d}[x_{i}^{2}-10cos(2\pi x_{i})]$ | $f(\overrightarrow{X}^{*})=0,\text{at~{}}\overrightarrow{X}^{*}=(0,...,0)$ | $x_{i}\in[-5.12,5.12]$ | $i=1,...,d$ | multiple
TABLE I: The objective functions and their characteristics.

###  III-B Test Functions

A optimization problem is about finding an optimal solution
$\overrightarrow{X}=(x_{1},...,x_{n})\in\mathcal{X}$ of an objective function
$f:\mathcal{X}\rightarrow\mathbb{R}$, that is:

$\overrightarrow{X}^{*}=\arg\max_{\overrightarrow{X}\in\mathcal{X}}f(\overrightarrow{X}),$
(1)

where the analytical form of the objective function $f(\overrightarrow{X})$ is
usually (but not necessarily) unknown, i.e., black-box optimization, in the
real optimization tasks [14, 21]. To clearly observe and analysis the quality
and computation time of the various dimensional optimization, we select many
benchmark test functions as the objective functions. The benchmark test
functions should meet the requirements of:

  * •

scalable, the dimension of a solution $\overrightarrow{X}=(x_{1},...x_{n})$
can be various.

  * •

given global minimum, which can be observed clearly.

  * •

multimodal and multiple local optima, which are sufficiently challenging so
that these algorithms cannot quickly converge to the optimal solution.

We select these test functions that are from the well-known Black-Box
Optimization Benchmark (BBOB) [22]. The source code of these objective
functions can be found in 111https://www.sfu.ca/~ssurjano/optimization.html.
In this paper, we finally selected the function Schwefel, Griewank, and
Rastrigin. Specifically, we set the dimension of the test functions as ten,
i.e., $n=10$ in the solution $\overrightarrow{X}=(x_{1},...x_{n})$. The
selected test functions and their characteristics are shown in Table I.

##  IV Results and Discussion

In this section, we present the quality and the computation time of the well-
known optimization algorithms on the test functions.

Schwefel Griewank Rastrigin

Figure 1: The optimization performance of the Bayesian optimization and
evolutionary strategy in terms of quality over ten function evaluations (one
generation in the EAs) averaged over ten runs on the function Schwefel,
Griewank, and Rastrigin. The solid lines represent the average objective value
in ten runs. The shadow intervals represent the confidence of the average
objective value.

###  IV-A Results in Quality

We run the optimization algorithms of BO, CMA-ES, EA (ES), and PSO on the ten
dimensions of test functions, Schwefel, Griewank, Rastrigin. In optimization
problems, quality is a mainstream metric to evaluate the performance of the
optimization algorithms. We run these optimization algorithms ten times on the
test functions and take the average of the objective value over ten function
evaluations (one generation in the EAs), as shown in Figure 1. The solid lines
represent the average objective values that convergence to the global minimum
of these test functions. The shadow intervals represent the confidence of the
average objective value over the ten runs. In the early stage, BO performs
better quality than other EAs during the given function evaluations, i.e.,
faster convergence. However, BO Although the EAs (CMA-ES, EA (ES), and PSO)
search the global minimum slower than BO, they convergence gradually during
the process and finally obtain the similar quality as BO. Therefore, we
conclude that BO is suitable to be applied in the optimization tasks that need
to be solved in the limited function evaluations. For the optimization tasks
that are allowed to be solved in a lot of function evaluations, both BO and
the EAs obtain the similar performance in terms of quality. Importantly, the
applications of optimization tasks in the real world have to consider the
computation time that we analysis in the next section.

###  IV-B Results in Computation Time

In the applications of optimization tasks in the real world, users actually
consider about how long the optimization algorithms obtain a desired solution.
Therefore, the computation time is one of the crucial measurements to evaluate
performance of optimization algorithms. It is well-known that the computation
time of BO grows substantially with the number of evaluations, limiting its
feasibility for longer optimization runs [14]. In this section, we analysis
the computation time of BO, CMA-ES, EA, and PSO on the test functions. We
collect the time stamp and thus calculate the computation time of each ten
function evaluations. The computation time of these optimization algorithms
are presented in Figure 2. The solid lines represent the average computation
time over ten runs, and the shadow intervals represent the confidence of the
average computation time. It is clear that BO performs substantially
increasing computation time over the number of function evaluations. In the
end of the optimization, BO costs significant more than the other algorithms.
The observed results show that BO finally cost about 280000 seconds (77 hours)
during the optimization because the substantially increasing computation time
over the number of function evaluations, which is also demonstrated in Figure
2. These EAs cost little of computation time during the optimization. For the
observed results, these EAs cost about from 20 to 200 seconds during the
optimization of 1500 function evaluations. The lines of the computation time
of the EAs (CMA-ES, EA, PSO) overlap each other. Therefore, the computation
time of these EAs are unable to be observed in Figure 2. To solve this issue,
we apply the logarithmic to analysis the computation time of these EAs. The
$log(Time(s))$ over ten function evaluations are shown in Figure 3. It is
clear the logarithmic of the computation time of these EAs are
distinguishable. The EA generally cost more computation time than CMA-ES and
PSO, specifically cost about 200 seconds during the optimization. CMA-ES and
PSO finish the optimization with the computation time of about 20 seconds.

Schwefel Griewank Rastrigin

Figure 2: The computation time of the optimization algorithms over ten
function evaluations (one generation in the EAs) averaged on the function
Schwefel, Griewank, and Rastrigin. The solid lines represent the average
computation time over ten runs. The shadow intervals represent the confidence
of the average computation time. Note that the EAs (CMA-ES, EA, PSO) perform
the similar computation time cost, which their lines overlap in the black
lines.

Schwefel Griewank Rastrigin

Figure 3: The details in the logarithmic of computation time of the
optimization algorithms over function evaluations averaged on the function
Schwefel, Griewank, and Rastrigin. The solid lines represent the averaged
logarithmic of computation time over ten runs. The shadow intervals represent
the confidence of the averaged logarithmic of computation time.

In conclusion, although the different optimization algorithms obtain the
optimal solutions towards the global minimum with the enough function
evaluation, they perform significant different computation time. BO performs
the significant increasing computation time, while the EAs convergence to the
optimal in the short computation time of 20 $\sim$ 200 seconds. For the
applications of optimization tasks in the real world, BO is suitable to be
applied in the tasks that are needed to obtained desired quality in a limited
function evaluations, and the EAs are suitable to search the optimal of the
tasks that are allowed to be run with enough function evaluations.

##  V Conclusion

In this paper, we proposed that the performance of optimization algorithms
should be evaluated from the aspects of quality and computation time. The
computation time is a crucial measurement for the optimization tasks in the
real world applications. We select and run the well-known optimization
algorithms of Bayesian optimization and evolutionary algorithms (including
CMA-ES, EA, and PSO) on the well-known test functions of Schwefel, Griewank,
and Rastrigin. The performance of these algorithms are analysed from the
aspects of quality and computation time. The results indicated that Bayesian
optimization is suitable for the optimization tasks that are needed to
obtained desired quality in a limited function evaluations, and the EAs are
suitable to search the optimal of the tasks that are allowed to be run with
enough function evaluations. This work provides the evaluation and the
selection of the optimization algorithms for different optimization problems.
In the future, we will combine the quality and computation time into a
measurement that can be used to evaluate the optimization algorithms
comprehensively. In addition, we will consider the results of quality and
computation time for the optimization in artificial intelligent applications
of robotics [11, 10], intelligent sensing [23, 24], virtual reality [25, 26],
autonomous driving [27] to improve the performance of these studies.

## References

  * [1] P. I. Frazier, “A tutorial on bayesian optimization,” _arXiv preprint arXiv:1807.02811_ , 2018.
  * [2] B. Ru, M. A. Osborne, M. McLeod, and D. Granziol, “Fast information-theoretic bayesian optimisation,” in _International Conference on Machine Learning_. PMLR, 2018, pp. 4384–4392.
  * [3] J. Wu, X.-Y. Chen, H. Zhang, L.-D. Xiong, H. Lei, and S.-H. Deng, “Hyperparameter optimization for machine learning models based on bayesian optimization,” _Journal of Electronic Science and Technology_ , vol. 17, no. 1, pp. 26–40, 2019.
  * [4] G. Lan, M. De Carlo, F. van Diggelen, J. M. Tomczak, D. M. Roijers, and A. E. Eiben, “Learning directed locomotion in modular robots with evolvable morphologies,” _Applied Soft Computing_ , vol. 111, p. 107688, 2021.
  * [5] G. Lan, M. van Hooft, M. De Carlo, J. M. Tomczak, and A. Eiben, “Learning locomotion skills in evolvable robots,” _Neurocomputing_ , vol. 452, pp. 294–306, 2021.
  * [6] R. Giráldez, J. S. Aguilar-Ruiz, and J. C. Riquelme, “Knowledge-based fast evaluation for evolutionary learning,” _IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)_ , vol. 35, no. 2, pp. 254–261, 2005.
  * [7] K. Deb, A. Anand, and D. Joshi, “A computationally efficient evolutionary algorithm for real-parameter optimization,” _Evolutionary computation_ , vol. 10, no. 4, pp. 371–395, 2002.
  * [8] L. Thiele, K. Miettinen, P. J. Korhonen, and J. Molina, “A preference-based evolutionary algorithm for multi-objective optimization,” _Evolutionary computation_ , vol. 17, no. 3, pp. 411–436, 2009.
  * [9] S. R. Young, D. C. Rose, T. P. Karnowski, S.-H. Lim, and R. M. Patton, “Optimizing deep learning hyper-parameters through an evolutionary algorithm,” in _Proceedings of the workshop on machine learning in high-performance computing environments_ , 2015, pp. 1–5.
  * [10] G. Lan, J. Chen, and A. Eiben, “Simulated and real-world evolution of predator robots,” in _2019 IEEE Symposium Series on Computational Intelligence (SSCI)_. IEEE, 2019, pp. 1974–1981.
  * [11] G. Lan, J. Chen, and A. E. Eiben, “Evolutionary predator-prey robot systems: From simulation to real world,” in _Proceedings of the Genetic and Evolutionary Computation Conference Companion_ , 2019, pp. 123–124.
  * [12] G. Lan, L. De Vries, and S. Wang, “Evolving efficient deep neural networks for real-time object recognition,” in _2019 IEEE Symposium Series on Computational Intelligence (SSCI)_. IEEE, 2019, pp. 2571–2578.
  * [13] Z. Gao and G. Lan, “A neat-based multiclass classification method with class binarization,” in _Proceedings of the Genetic and Evolutionary Computation Conference Companion_ , 2021, pp. 277–278.
  * [14] G. Lan, J. M. Tomczak, D. M. Roijers, and A. Eiben, “Time efficiency in optimization with a bayesian-evolutionary algorithm,” _Swarm and Evolutionary Computation_ , p. 100970, 2021.
  * [15] H. Liu, T.-H. Hong, M. Herman, T. Camus, and R. Chellappa, “Accuracy vs efficiency trade-offs in optical flow algorithms,” _Computer vision and image understanding_ , vol. 72, no. 3, pp. 271–286, 1998.
  * [16] L. Wenyin and D. Dori, “A protocol for performance evaluation of line detection algorithms,” _Machine Vision and Applications_ , vol. 9, no. 5, pp. 240–250, 1997.
  * [17] C. Zhang, C. Liu, X. Zhang, and G. Almpanidis, “An up-to-date comparison of state-of-the-art classification algorithms,” _Expert Systems with Applications_ , vol. 82, pp. 128–150, 2017.
  * [18] E. Brochu, V. M. Cora, and N. De Freitas, “A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning,” _arXiv preprint arXiv:1012.2599_ , 2010.
  * [19] A. E. Eiben and J. E. Smith, _Introduction to Evolutionary Computing_ , 2nd ed. Springer Publishing Company, 2015\.
  * [20] A. E. Eiben, J. E. Smith _et al._ , _Introduction to evolutionary computing_. Springer, 2003, vol. 53.
  * [21] J. Bossek _et al._ , “smoof: Single-and multi-objective optimization test functions.” _R J._ , vol. 9, no. 1, p. 103, 2017.
  * [22] N. Hansen, S. Finck, R. Ros, and A. Auger, “Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions,” Ph.D. dissertation, INRIA, 2009.
  * [23] G. Lan, X. Hu, and Q. Hao, “A bayesian approach for targets localization using binary sensor networks,” in _2015 8th International Symposium on Computational Intelligence and Design (ISCID)_ , vol. 2. IEEE, 2015, pp. 453–456.
  * [24] G. Lan, J. Liang, G. Liu, and Q. Hao, “Development of a smart floor for target localization with bayesian binary sensing,” in _2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA)_. IEEE, 2017, pp. 447–453.
  * [25] G. Lan, Z. Luo, and Q. Hao, “Development of a virtual reality teleconference system using distributed depth sensors,” in _2016 2nd IEEE International Conference on Computer and Communications (ICCC)_. IEEE, 2016, pp. 975–978.
  * [26] G. Lan, J. Sun, C. Li, Z. Ou, Z. Luo, J. Liang, and Q. Hao, “Development of uav based virtual reality systems,” in _2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)_. IEEE, 2016, pp. 481–486.
  * [27] H. Xu, G. Lan, S. Wu, and Q. Hao, “Online intelligent calibration of cameras and lidars for autonomous driving systems,” in _2019 IEEE Intelligent Transportation Systems Conference (ITSC)_. IEEE, 2019, pp. 3913–3920.


# Real-time calibration and alignment of the LHCb RICH detectors

Jibo HE (on behalf of the LHCb RICH collaboration)  University of Chinese
Academy of Sciences (UCAS), Beijing, China

###### Abstract

In 2015, the LHCb experiment established a new and unique software trigger
strategy with the purpose of increasing the purity of the signal events by
applying the same algorithms online and offline. To achieve this, real-time
calibration and alignment of all LHCb sub-systems is needed to provide
vertexing, tracking, and particle identification of the best possible quality.
The calibration of the refractive index of the RICH radiators, the calibration
of the Hybrid Photon Detector image, and the alignment of the RICH mirror
system, are reported in this contribution. The stability of the RICH
performance and the particle identification performance are also discussed.

##  I Introduction

The LHCb experiment Alves:2008zz  is one of the four large particle physics
experiments at the Large Hadron Collider (LHC), and is designed to search for
physics beyond the Standard Model by precision study of the beauty and charm
hadrons. The Ring Imaging CHerenkov (RICH) detectors of the LHCb experiment
employ the ${\rm C_{4}F_{10}}$ and ${\rm CF_{4}}$ radiators to provide
particle identification (PID) of charged particles in the momentum range of
2-100 GeV. This is essential for the LHCb core physics program. The centre-of-
mass energy of the LHC has been increased from 8 TeV to 13 TeV in 2015, and
the LHCb experiment will be upgraded LHCb-TDR-012 ; LHCb-TDR-014 , starting in
2019, and run at a 4-times higher luminosity than in LHC Runs I and II. A new
and unique software trigger strategy has been established at the LHCb
experiment with the purpose of increasing the purity of the signal events by
applying the same algorithms online and offline. This requires ultimate
quality of vertexing, tracking, and PID. Therefore, real-time calibration and
alignment of all LHCb sub-systems is required online. The calibration of the
refractive index of the RICH radiators, the calibration of the Hybrid Photon
Detector (HPD) image, and the alignment of the RICH mirror system are reported
here.

The LHCb trigger strategy in Run-II (2015-2018) has been changed with respect
to that in Run-I (2010-2013), as shown in Fig. 1. The online event
reconstruction in Run-I was simpler and faster than that used offline, and did
not have the latest detector calibration and alignment constants applied. In
the Run-II data-taking, the events selected by the first stage of the software
trigger are buffered on local disks, then an automatic detector calibration
and alignment is performed, and the resulting calibration and alignment
constants are used in the final stage of the software trigger. As the full
offline event reconstruction is run in the final stage of the software
trigger, its output can be used for physics analysis directly without further
offline processing.

Figure 1: Schematic diagram of the LHCb trigger data-flow in Run-I data taking
(left) compared to the data-flow in Run-II (right).

##  II RICH optical layout and reconstruction

The LHCb RICH system has two detectors, one upstream and one downstream of the
magnet, covering the low momentum range $\sim 2-60$ GeV$/c$ and the high
momentum range $\sim 15-100$ GeV$/c$ respectively Alves:2008zz . The side view
of the RICH detector upstream of the magnet is shown in Fig. 2. The Cherenkov
photons emitted by charged particle tracks in the radiator are reflected and
focused by the combination of spherical and secondary mirrors out of the LHCb
geometrical acceptance and then are detected by the Hybrid Photon Detectors
(HPD).

To reconstruct the photon candidate and the Cherenkov angle, one takes the
spatial position of the HPD pixel hits as the detection point, and the middle
point of the associated track in the radiator as the emission point, then
solves a quartic equation that fully describes the reflections of the photon,
given the RICH geometry CkvAngRec . The “global likelihood algorithm”
GlobalPID  is used to determine the PID, where information from all tracks,
all radiators and all pixel hits within a given event are considered
simultaneously, and the likelihood is maximized by comparing the observed
Cherenkov angles with those expected under different PID hypotheses.

To achieve the best PID performance, one needs to align the RICH mirrors,
detector planes and the tracking system, and calibrate the refractive index of
radiators and the HPD image with good precision. These factors are all time-
dependent, necessitating real-time calibration and alignment of the LHCb RICH
detectors, and the tracking system.

Figure 2: Side view of the LHCb RICH detector upstream of the magnet.

##  III Calibration and alignment

###  III.1 Calibration of the refractive index of the RICH radiators

The refractive index of the gas radiators depends on the ambient temperature
and pressure, and the exact composition of the gas mixture; so it can change
in time. These quantities are monitored by hardware to compute an expected
refractive index, but this does not have a precision that is high enough for
the physics analysis, therefore it needs to be further corrected. As shown in
Fig. 3, the distribution of the difference between the reconstructed and
expected Cherenkov angle is fitted to obtain the shift, which is then
converted to a scale factor of the expected refractive index according to
studies based on simulation.

About 50 Hz of events are sent to multiple online reconstruction tasks, which
run in parallel, and the resulting histograms are merged at the end of each
run. Then a dedicated task is used to fit the histograms merged run-by-run and
produce calibration constants to be used by the RICH reconstruction in the
final stage of the software trigger. The maximum run length is one hour.

Figure 3: Difference between the reconstructed and expected Cherenkov angle
before the calibration.

###  III.2 Calibration of the HPD images

Figure 4: Schematic drawing of the Hybrid Photon Detector (HPD).

The Hybrid Photon Detector is used to detect Cherenkov photons. As shown in
Fig. 4, the photoelectron produced at the photocathode is accelerated by a
high voltage of up to 20 kV onto a reverse-biased pixellated silicon detector,
with a de-magnification factor of about 5 LHCb-DP-2012-003 . The HPD anode
images are affected by the magnetic and electric fields, and have been
observed to move and change their size, possibly due to changes in these
residual fields when the high voltage is cycled each LHC fill. Such changes
could degrade the reconstruction of the Cherenkov angle and affect the PID
performance. Therefore the centre and radius of all the HPD images are
calibrated run-by-run. Figure 6 shows the calibration process. First, the
centre of the image is cleaned to eliminate ion feedback. Then a Sobel filter
is used to detect the edges of the image that are fitted to determine the
centre and the radius of the image, which are used by the RICH reconstruction
in the final stage of the software trigger. As only the raw HPD data needs to
be decoded, more than 500 Hz of events are processed run-by-run.

Figure 5: Calibration process of the HPD image. (a) HPD image for a typical
run. (b) The centre of the HPD image is cleaned to eliminate ion feedback. (c)
Edge of the HPD image detected by the Sobel filter. (d) Radius and centre of
the HPD image returned by the fit.

Figure 6: Difference between the measured and expected Cherenkov angle,
$\Delta\theta_{C}$ plotted as a function of the azimuthal angle $\phi$ and
fitted with $\theta_{x}\cos(\phi)+\theta_{y}\sin(\phi)$, for one side of the
RICH 2 detector LHCb-DP-2012-003 . The upper plot is prior to alignment, and
shows a dependency of the angle $\theta_{C}$ on the angle $\phi$. The bottom
plot is after the alignment correction, and $\Delta\theta_{C}$ is uniform in
$\phi$.

###  III.3 Alignment of the RICH mirror system

The Cherenkov photons emitted by the charged particles passing through the
RICH detectors are focused onto the photon-detector plane by the spherical and
secondary mirrors. In case of misalignment the centre of Cherenkov ring would
not correspond to the intersection point of the charged track, and this would
introduce a dependence of the difference between the measured and expected
Cherenkov angle on the azimuthal angle of the ring, as shown in Fig. 6. The
alignment constants for each mirror are determined by the fit of the Cherenkov
angle difference as a function of the azimuthal angle on the ring. The
correlation between the different mirror pairs is also taken into account. The
procedure is evaluated by an iterative procedure implemented in a dedicated
framework, which makes it possible to run the alignment in parallel using
about 1800 nodes of the software trigger farm. The alignment of the RICH
mirror system has been found to be stable enough to not affect the PID
performance, as shown in Fig. 7, and it runs routinely as a monitoring task.

Figure 7: Time dependence of the mirror alignment parameters for the RICH
detector downstream the magnet for the 2016 data sample, (upper) spherical
mirror, (bottom) secondary mirror. The shadow regions show the range of
alignment parameters for all the mirror pairs, the points show the average
value for all the mirror pairs.

###  III.4 Time alignment

In order to maximise the photon collection efficiency of the LHCb RICH
detectors, the HPD readout must be synchronised with the LHC bunch crossing to
within a few nanoseconds. The initial time alignment was performed in the
absence of beam using a pulsed laser, and has been improved further with
dedicated timing scan data taken during physics collisions. As shown in Fig.
8, all the HPDs have been time aligned to about 1 ns.

Figure 8: Distribution of the midpoints of timing scans in the upstream RICH
(upper) and downstream RICH (bottom) after time alignment with pp collisions.

##  IV Performance

###  IV.1 Stability of the RICH performance

The stability of the RICH performance is monitored by the Cherenkov angle
resolution obtained with the same RICH mirror alignment and with the dedicated
run-by-run calibrations. It has been found to be stable for the full 2015
data-taking, as shown in Fig. 9.

Figure 9: Time dependence of the Cherenkov angle resolution for the RICH
detector downstream the magnet for the 2015 data sample.

###  IV.2 Particle identification performance

Figure 10 shows the kaon efficiency (kaons identified as kaons) and pion
misidentification (pions misidentified as kaons), as a function of particle
momentum, obtained from imposing two different requirements on this
distribution in the Run-II data. One can see that the RICH detectors provide
excellent particle identification. Further details on the particle
identification performance in the Run-II data-taking of the LHCb experiment
can be found in Ref. Rich2016_Antonis .

Figure 10: Kaon identification efficiency and pion misidentification rate
measured on data as a function of track momentum. Two different
$\Delta\mbox{log}\mathcal{L}(K-\pi)$ requirements have been imposed on the
samples, resulting in the open and filled marker distributions, respectively.

##  V Summary

Novel real-time calibration and alignment of the LHCb RICH detectors have been
implemented in the Run-II data-taking of the LHCb experiment. This includes
the run-by-run calibration of the refractive index of the RICH radiators, and
that of the Hybrid Photon Detector image, and the alignment of the RICH mirror
system. The real-time calibration and alignment works well, and provides
excellent particle identification for the online trigger and offline analysis.

## References

  * (1) LHCb collaboration, A. A. Alves et al., The LHCb Detector at the LHC, JINST 3 (2008) S08005.
  * (2) LHCb collaboration, Framework TDR for the LHCb Upgrade: Technical Design Report, CERN-LHCC-2012-007.
  * (3) LHCb collaboration, LHCb PID Upgrade Technical Design Report, CERN-LHCC-2013-022.
  * (4) E. Nappi, T. Ypsilantis (Eds.), Experimental techniques of Cherenkov light imaging, in: Proceedings of the 1st Workshop on Ring Imaging Cherenkov Detectors, Bari, Italy, June 2–5, 1993.
  * (5) R. Forty, RICH pattern recognition for LHCb, Nucl.Instrum.Meth. A433 (1999) 257.
  * (6) M. Adinolfi et al., Performance of the LHCb RICH detector at the LHC, Eur. Phys. J. C73 (2013) 2431, arXiv:1211.6759.
  * (7) A. Papanestis, Performance of the LHCb RICH detectors during the LHC Run II, these proceedings.