These authors contributed equally to this work.

These authors contributed equally to this work.

[1]Timur O. Shegai

1]Department of Physics, Chalmers University of Technology, Göteborg, 412 96,
Sweden

2]Department of Microtechnology and Nanoscience, Chalmers University of
Technology,

Göteborg, 412 96, Sweden

3]Faculty of Physics, University of Warsaw, Pasteura 5, Warsaw, 02-093, Poland

# Defect-assisted reversible phase transition in mono- and few-layer ReS2

George Zograf  Andrew B. Yankovich  Betül Küçüköz  Abhay V. Agrawal  Alexander
Yu. Polyakov  Joachim Ciers  Fredrik Eriksson  Åsa Haglund  Paul Erhart
Tomasz J. Antosiewicz  Eva Olsson<EMAIL_ADDRESS>[  [  [

###### Abstract

Transition metal dichalcogenide (TMD) materials have attracted substantial
interest due to their remarkable excitonic, optical, electrical, and
mechanical properties, which are highly dependent on their crystal structure.
Controlling the crystal structure of these materials is essential for fine-
tuning their performance, e.g., linear and nonlinear optical, as well as
charge transport properties. While various phase-switching TMD materials, like
molybdenum telluride (MoTe2), are available, their transitions are often
irreversible. Here, we investigate the mechanism of a light-induced reversible
phase transition in mono- and bilayer flakes of rhenium disulfide (ReS2). Our
observations, based on scanning transmission electron microscopy, nonlinear
spectroscopy, and density functional theory calculations, reveal a transition
from the ground T′′ (double distorted T) to the metastable H′ (distorted H)
phase under femtosecond laser irradiation or influence of highly-energetic
electrons. We show that the formation of sulfur vacancies facilitates this
phenomenon. Our findings pave the way towards actively manipulating the
crystal structure of ReS2 and possibly its heterostructures.

###### keywords:

reversible phase transition, transition metal dichalcogenide, ReS2, second-
harmonic generation

## Introduction

Over the past two decades, following the seminal work on graphene [1], there
has been a substantial growth of interest in atomically thin two-dimensional
(2D) materials. Among these, transition metal dichalcogenides (TMDs) have
emerged as a particularly promising family due to their remarkable performance
in various nanophotonic [2, 3], nanoelectronic [4], and other nanotechnology
applications [5]. TMDs form a diverse material platform comprising transition
metal (e.g., Mo, W, Re, Nb, Ta, etc.) and chalcogenide elements (e.g., S, Se,
Te). The flexibility to choose these constituent elements during synthesis
enables the tunability of their optical [6] and electronic properties [7].
Furthermore, stacking atomically thin 2D materials in van der Waals
heterostructures [8] offers control over their properties enabling various
quantum phenomena, such as superconductivity in graphene-based structures [9,
10], Moiré excitons and correlated states in TMDs [11, 12, 13, 14, 15, 16],
topological excitons [17], advanced excitonic devices [18], and exciton
polaritons [19, 20, 21]. Following the fabrication of large TMD flakes,
achieved through methods like exfoliation [22] or epitaxial single crystal
growth [23, 24], further modification of material properties can be realized
through nanopatterning, e.g., using electron beam lithography (EBL) and/or
chemical etching. Nanopatterning of TMDs has enabled the production of high-
quality factor (high-Q) nanophotonic structures [25, 26, 27, 28], exploration
of higher order modes and anapole states [29, 30, 31, 32], and lasing from
indirect bandgap semiconductor microdisks [33]. Expanding control over
material properties beyond conventional fabrication methods will enable more
precise engineering of future devices and deeper insights into the fundamental
physics of TMD-related phenomena [34, 35].

The ability to actively and reversibly tune TMD properties post-fabrication is
of high interest within the 2D material community [36]. Realizing this control
to its full potential could expand the range of applications, including
neuromorphic computing [37, 38] as well as optically rewritable electronic
circuits and memories [39]. One approach for achieving active control over TMD
electronic properties involves photo-induced modifications of the
concentration of substitutional and interstitial point defects [40]. Another
approach leverages the substantial changes in properties associated with
material polymorphism or phase transitions. For instance, entire TMD flakes
can undergo phase transitions driven by electrostatic-doping [41] or
mechanical strain [42]. While these approaches show promise, they typically
lack temporal stability and/or precise spatial control at the nanometer scale.
Laser-induced phase patterning offers more precise spatial control but is
often irreversible due to ablation [43, 44], similar to thermal heating during
the annealing process in 2D materials [45]. Nevertheless, promising routes for
photo-induced phase switching exist [46, 47], such as the reversible
transition of TaS2 into a hidden stable quantum state, enabling drastic
changes in conductivity [48, 49]. Therefore, exploring new mechanisms for
phase transitions in TMDs and understanding their associated property changes
are crucial for achieving improved active control.

Rhenium disulfide (ReS2) is a promising TMD material for optical, electronic,
and gas-sensing applications [50, 51, 52, 53, 54, 55]. Its features include
being a direct bandgap semiconductor in a bulk van der Waals crystal [56] and
its high optical anisotropy, making it ideal for directionally selective
photonic applications [57, 58, 59, 6, 60]. Previous investigations have
focused on ReS2’s atomic structure and phases using density functional theory
(DFT) [56] and scanning transmission electron microscopy (STEM) [61].
Additionally, second-harmonic generation (SHG) studies have linked its
activity to stacking and thickness variations [62]. Notably, ReS2 monolayers
(ML) and bilayers (BL) have been demonstrated to exhibit a reversible phase
transition under near-infrared femtosecond (fs) laser irradiation,
characterized by a significant change in SHG activity and polarization-
resolved SHG [63]. It was hypothesized from the available data that ReS2
transitions from the 1T′ phase into the 1H phase [63]. Despite this promising
initial work, questions remain regarding the involved transition phases and
the mechanism enabling the transition.

Here, we investigate the phase transition in ReS2 using a combination of
electronic structure calculations, stability analysis, optical SHG microscopy,
STEM imaging and diffraction, and electronic transport measurements. Our study
results in several key observations: (i) femtosecond laser irradiation induces
a phase transition in ReS2 which converts back to the initial phase with time,
(ii) the initial and the “switched” phase are identified as T′′ and H′,
respectively, with the latter being metastable (as opposed to its unstable H
counterpart), and (iii) defects, particularly sulfur vacancies, play a crucial
role in facilitating the 1T′′-1H′ transition. Our investigation provides a
deeper understanding of the possible phases of ML and BL ReS2 and reveals the
importance of vacancy defect formation on the mechanisms of phase switching.

## Results & Discussion

Figure 1: Overview. (a) Schematic of the physical process underpinning the
laser-induced phase-transition. Femtosecond laser pulses in the near-IR range
induce a phase transition from the T′′ ground state of ReS2 flakes into the H′
transient state manifested by a structural change. (b) SHG polarization-
resolved intensity of 1T′′ and 1H′ monolayer flakes calculated using DFT. (c)
Structural changes in the lattice symmetry upon the phase transition between
1T′′ and 1H′ monolayer flakes of ReS2.

The main premise of our work is depicted in Figure 1. Under intense fs-laser
illumination or influence of high-energy electrons, 1T′′ ML and 2T′′ BL flakes
undergo a phase transition into what we hypothesize to be the 1H′ phase, a
detailed description of which is provided in the Electronic band structure
calculation section below. This phase transition is facilitated by generation
of a sufficient amount of sulfur vacancies that allow the transition to occur
by providing the necessary geometrical freedom. The H′ phase is metastable and
is protected by an activation energy barrier ($E_{a}$), illustrated
schematically in Figure 1a. The successful realization of this phase
transition is visually evident in a significantly amplified SHG signal that
strongly depends on the crystalline symmetry of the atomic structure, as shown
in Figure 1b for ML ReS2 using DFT calculations. The atomic rearrangement
during the phase transition from 1T′′ to 1H′ is illustrated in Figure 1c.

Our experimental investigation starts with the fabrication of ML and BL ReS2
flakes on two different substrates via mechanical exfoliation and dry transfer
from a host crystal. Using this approach, we deposited flakes onto: (i) a
silicon/silicon oxide (Si/SiO2) substrate for optical and field-effect
transistor (FET) transport studies, and (ii) a $\sim$ 20 nm thick silicon
nitride (SiN) membrane TEM grid for STEM and electron diffraction studies.
More details about the fabrication process are provided in the Methods
section.

The thickness of the fabricated flakes was identified in three steps: optical
imaging, SHG intensity, and atomic-force microscopy (AFM). Optical contrast in
reflection microscopy provides an accurate estimation of the thickness of the
flake [64, 65]. Owing to different symmetries upon stacking, the ReS2 flakes
demonstrate SHG activity depending on the number of layers [62].

### Evidencing the phase transition in mono- and bilayer ReS2 by electron
microscopy

Figure 2: ML and BL ReS2 STEM observations. (a) Experimental ADF STEM image of
undamaged ML ReS2. The lower left inset image is a template-matched average
image produced from the entire image in (a). The lower right inset image is a
simulated ADF STEM image of the 1T′′ phase. (b)(i) Experimental PACBED pattern
of undamaged ML ReS2. (b)(ii) Simulated PACBED pattern of the 1T′′ phase. (c)
Experimental ADF STEM images from different regions of electron-irradiated ML
ReS2. (c)(i) is from an undamaged 1T′′ region. (c)(ii-iv) are from different
damaged regions that exhibit a loss of the 1T′′ phase but maintain a dominant
distorted axis that rotates by 60 degrees around the out-of-plane c-axis to
produce three possible in-plane orientations. The green markers represent the
rearrangement of Re atoms upon phase transition. (d) Experimental ADF STEM
image of undamaged BL ReS2. The lower left inset image is a template-matched
average image produced from the entire image in (d). The lower right inset
image is a simulated ADF STEM image of the 2T′′-e phase. (e)(i) Experimental
PACBED pattern of undamaged BL ReS2. (e)(ii) Simulated PACBED pattern of the
2T′′e phase. (f) Experimental ADF STEM images extracted from an image series
showing the atomic structure evolution of BL ReS2 due to electron beam
irradiation. (f)(i) shows the initial undamaged structure that is consistent
with the 2T′′ phase. (f)(ii-iv) shows three distinct structures that are
observed in sequence during electron beam irradiation, which are followed by
complete destruction of the 2D crystalline lattice. The red and blue markers
in (a), (d), and (f) indicate the double distorted diamond structures of the
T” initial phase. The green markers in (c) and (f) indicate the single
distorted Re chains in MLs and the hexagonal symmetry in BLs, respectively,
upon phase transition.

To directly characterize the unperturbed starting phase of ML and BL ReS2, as
well as the evolution of their atomic structure during defect generation, we
conducted experimental and simulated atomic resolution annular dark-field
scanning transmission electron microscopy (ADF STEM) and position averaged
convergent beam electron diffraction (PACBED) investigations. ADF STEM imaging
enables one to directly visualize the positions of the Re atoms (see Methods
for more details about the selection of STEM imaging modes and resulting image
contrast) and, therefore, the atomic structure evolution. PACBED patterns
provide a wealth of structural information about crystalline samples [66, 67],
such as the phase, by recording momentum-resolved measurements of the forward
scattered electrons. PACBED enables the collection of this information from
extremely small areas of the sample (down to a single unit cell or even atom
[68]), which can be controlled by selecting the sample area where the STEM
probe is scanned. PACBED is particularly important to this work because it is
sensitive to the electron scattering from both the Re and S atoms, and thus
provides a S-sensitive phase characterization tool, in contrast to ADF STEM
imaging.

We identify the structure of unperturbed ML ReS2 using ADF STEM imaging and
PACBED while considering 5 possible ML ReS2 phases: 1H, 1T, 1T′, 1T′′, and
1H′. The experimental atomic resolution ADF STEM image of unperturbed ML ReS2
in Figure 2a and the associated template-averaged image in Figure 2(a,i)
reveals the positions of the Re atoms and the presence of a double distorted
structure that is indicated by the red diamonds. A simulated ADF STEM image of
the 1T′′ phase is shown in the lower right inset image. The similarity between
the experimental (Figure 2(a,i)) and simulated (Figure 2(a,ii)) ADF STEM
images, as well as the dissimilarity between the experimental ADF STEM image
and simulated images of the other possible ML ReS2 phases (see Figure S3),
provide evidence that unperturbed ML ReS2 has the 1T′′ phase. In addition, the
experimental PACBED pattern from ML ReS2 (Figure 2(b,i)) is in best agreement
with the simulated PACBED pattern from the 1T′′ phase (Figure 2(b,ii) and
Figure S3) providing further evidence that unperturbed ML ReS2 has the 1T′′
phase.

Electron beam irradiation is known to introduce damage to TMD materials,
primarily through a knock-on sputtering process that introduces chalcogenide
vacancies [69]. For the unperturbed phase measurements, special experimental
design and analysis were conducted to ensure there were negligible effects
from the electron beam irradiation on the phase identification (see Methods
and Supplementary Information (SI) for more details). However, we also
leveraged the chalcogenide vacancies production mechanism to design ADF STEM
imaging experiments that resolve the atomic structure evolution of ML and BL
ReS2 during increasing S-vacancy concentrations. This is achieved by acquiring
time-resolved ADF STEM image series with carefully selected electron beam
energy, electron dose, and STEM image scan parameters (see Methods and SI).

During time-resolved ADF STEM imaging experiments on ML ReS2 (see SI Movies 1
and 2 for example image series), electron irradiation damage occurs very
rapidly, partially because experiments are limited to using a 200 keV electron
beam due to sample constraints (see Methods and SI for more details). Similar
behavior was observed in ML MoTe2 during lower voltage TEM experiments [70].
The image series shown in SI Movies 1 and 2 reveal that ML ReS2 undergoes a
complete destruction of its 2D crystallinity after a few tens of image frames
(see SI for dose values). After a complete loss of the 2D crystalline
structure, it appears that Re nanoclusters remain, providing evidence that the
damage occurs through a sulfur sputtering and vacancy generation mechanism.
Interestingly, just prior to the loss of the 2D crystallinity, ML ReS2
undergoes a rearrangement of its atomic structure from a double distorted 1T′′
phase to what appears like nano-sized regions of a single distorted
nonstoichiometric phase that can have three possible in-plane orientations.
Figure 2(c,i-iv) shows extracted images from areas adjacent to and after SI
Movie 2. Figure 2(c,i) shows the starting 1T′′ phase, while Figure 2(c,ii-iv)
show the three possible orientations of the resulting single distorted
structure that are rotated by 60$\degree$ from each other. Figure S2 provides
details on the image locations of Figure 2(c,i-iv) and a Fast Fourier
transform analysis of these nano-sized regions that confirms a clear change in
structure and a 60$\degree$ rotation between possible single distorted phase
regions. Because the ADF STEM images are only sensitive to the Re atoms and
not the S atoms, we can not unambiguously identify the resulting single
distorted phase using this measurement. However, this investigation reveals ML
ReS2 transitions from the 1T′′ phase to a single distorted nonstoichiometric
phase that is consistent with 1H′ (Figure S3). Furthermore, the 1H′ phase is
consistent with our SHG data and DFT calculations, as we show below.

We now use ADF STEM imaging and PACBED to identify the structure of
unperturbed BL ReS2. For this analysis, we consider 15 possible BL ReS2
phases, including 2H, 2T, 2T′, and twelve different 2T′′ phases (named
2T′′c-n) that have previously been explored as possible stable phases [71].
The experimental atomic resolution ADF STEM image of unperturbed BL ReS2 in
Figure 2d and the associated template-averaged image in Figure 2(d,i) reveal
the presence of a stacked double distorted structure that is indicated by the
red and blue diamonds. A simulated ADF STEM image of the 2T′′e phase is shown
in Figure 2(d,ii). The similarity between the experimental, Figure 2(d,i), and
simulated, Figure 2(d,ii) ADF STEM images, as well as the dissimilarity
between the experimental ADF STEM image and simulated images of the other
possible BL ReS2 phases (see Figures S7-S8), provides evidence that
unperturbed BL ReS2 has the 2T′′e phase. In addition, the experimental PACBED
pattern from BL ReS2, Figure 2(e,i), is in best agreement with the simulated
PACBED pattern from the 2T′′e phase (Figure 2(e,ii), Figures S6 and S9)
providing further evidence that unperturbed BL ReS2 has the 2T′′e phase. After
considering all 12 possible phases, we can conclude that unperturbed BL ReS2
has the 2T′′ phase and most likely the 2T′′e variant.

Time-resolved ADF STEM imaging experiments also provide evidence for electron
beam-induced phase transitions in BL ReS2. SI Movies 7-8 (see Methods and SI
for data processing details and SI Movies 5-6 for unprocessed versions) are
two consecutive image series of the same area which show that BL ReS2 exhibits
obvious structural changes during electron beam irradiation before eventually
losing its 2D crystallinity. This is similar to what was observed for ML ReS2,
except BL ReS2 requires a substantially higher electron dose for a complete
loss of 2D crystallinity (see SI for dose values). Additionally, it undergoes
what appears like three distinct atomic structure transitions before it loses
its 2D crystallinity. Figure 2(f,i-iv) are extracted images from SI Movies 7-8
at four different times (see SI). Figure 2(f,i) shows the initial 2T′′ phase
characterized by the two stacked red and blue diamonds. After electron
irradiation and the generation of S-vacancies, the structure transitions
through 3 distinct phases shown in Figure 2(f,ii-iv). Figure 2(f,ii and iii)
show the atomic structure transitions through what appears like two different
single distorted phases. Following this, the final phase is characterized by
the loss of the double and single distortion axes of the Re atoms and the
presence of a hexagonal structure (see green hexagon in Figure 2(f,iv)). Based
on the atomic column contrast in Figure 2(f,iv), each column appears to
contain two Re atoms stacked in an A-A stacking configuration. Because the S
atoms are invisible in these ADF STEM images and they are critical to
identifying an H or T phase, it is not possible to unambiguously identify the
transitional phases from this data (the T phase is, however, inconsistent with
our SHG data, discussed below). Despite this, we can conclude the electron
irradiation and the subsequent generation of S-vacancies causes BL ReS2 to
evolve from a starting double distorted 2T′′ phase into nano-sized domains of
two different nonstoichiometric single distorted phases before losing its Re
distortion axes completely and transitioning into an A-A stacked hexagonal
nonstoichiometric structure with a subsequent loss of its 2D crystallinity,
leaving behind Re nanoclusters. Although we can not extract the S-vacancy
concentration throughout this evolution, we expect the S-vacancy concentration
to continuously increase during this process with the total electron dose,
highlighting the critical role of defects in enabling the T′′–H′ transition.

### Light-induced phase transition

Figure 3: Defect-induced phase transition in ReS2 flakes. (a) Time evolution
of SHG counts for ReS2 monolayer at 1040 nm fs-laser-induced point (static
position of the laser beam) patterning process at different incident powers.
$\tau_{\textrm{lag}}$ is the specific time of the SHG counts dramatic
increase. The dashed line corresponds to the third power slope. The inset
depicts the optical image of the flake, with the black square indicating the
area of interest. The scale bar is 5 $\mu$m. (b) The rate of the initiation of
the phase transition ($\tau^{-1}_{\textrm{lag}}$) as a function of incident
power of the fs-laser at 1040 nm pump wavelength. The color of the
experimental points corresponds to the same color as in the time evolution
curve in (a). The dashed line corresponds to the third-power slope. Insets
depict the SHG map before (upper) and after (lower) laser point patterning.
(c) and (d) are polarization-resolved SHG polar plots obtained from respective
areas of the panel (e) - red cross (e.ii - pristine) and red diamond (e.iii -
exposed to moderate UV light). The flake after the formation of fs-laser-
induced defects studied in (a,b) is subsequently exposed to UV light (e.iii).
Panel (e) also shows an optical image of the flake (e.i).

We now turn our attention to light-induced phase transition in ReS2. In these
experiments, we use a Si/SiO2 substrate with a 285-nm-thick SiO2 layer. We
start our analysis by studying the time-dependent formation of the SHG-active
1H′ phase. Figure 3a depicts the time evolution of the SHG intensity signal
from different areas of the ReS2 ML flake. The colors of the curves from blue
to red demonstrate the increase of the incident 1040 nm wavelength fs-laser
power. One can see similar features in every curve for different powers: (i)
some time delay, or, as we introduce, the lag time $\tau_{\textrm{lag}}$
before the SHG intensity begins to substantially increase, (ii) saturation of
the signal after reaching a certain level of counts. The analysis of the
inverted lag time $\tau^{-1}_{\textrm{lag}}$ as a function of incident power,
provides information about how soon the phase transition begins and how
nonlinear this initiation process is. Figure 3b depicts this information with
a clear third-order dependence, indicating that the initiation of the phase
transition of the ReS2 ML on a given substrate is a third-order nonlinear
optical process. This finding is of interest for patterning extended ReS2
areas, as shown in comparison in the insets of Figure 3b before exposure
(upper) and after exposure (lower). Such dependence of the lag time on the
laser power may arise from two key processes – multiple photon absorption of
the flake required to induce the transition into the 1H′ phase and the
formation of sulfur vacancies. The latter is in line with STEM and PACBED
observations, discussed above. Additionally, it has been reported that ReS2
MLs tend to host point defects more likely than, for example, MoS2 as the
covalent bond is rather soft in the 1T′′ double-distorted phase [72, 73].

Once the ML is heavily patterned by fs-laser (bottom inset Figure 3b) and,
therefore, hosts enough defects, one can induce a phase transition from 1T′′
to 1H′ using an ultraviolet (UV) source at significantly smaller power
compared to the fs-laser. Indeed, the polarization-resolved SHG plots in
Figure 3c,d before and after patterning manifest significant changes in the
structure upon UV exposure. The pronounced 3 pairs of 2-fold symmetric SHG
profiles with 60$\degree$ shift between each other in the patterned case
indicate the formation of a different phase. The three 2-fold symmetries are
in line with the STEM observations in Figure 2c and data provided in the SI,
which indicate that upon phase transition ML ReS2 forms distinct domains with
the $b$-axes in each domain oriented 60$\degree$ with respect to each other.
Individually, each domain yields an SHG spectrum with 2-fold symmetry, as
calculated in Figure 1b, but there is no guarantee that in a given SHG
measurement the surface area of the three mutually rotated 1H′ domains will be
equal. Hence, in general, the SHG signal is expected to be characterized by
three 2-fold symmetric profiles. Moreover, the intensity of the SHG from the
ML in Figure 3e confirms the phase transition. The AFM data (see Figure S10)
of the studied ReS2 flake confirms its monolayer nature.

We validate the defect-assisted nature of the light-induced phase transition
in ReS2 through various experimental techniques, including fs laser
patterning, SHG polarization-resolved spectroscopy, UV-induced phase
switching, and photoluminescence (PL) (the latter is discussed in SI, Figure
S11). Furthermore, using the Si/SiO2 substrate allows for conducting both
optical and FET experiments and therefore gaining a deeper understanding of
the electronic conductivity properties (see Figure S12). We note that the
final phase of the ReS2 can not be unambiguously identified solely from the
SHG data, however, we conclude that ReS2 switches into either H or H′ due to
SHG intensity and polarization-resolved data pointing to the broken inversion
symmetry of the lattice. Together with ADF STEM presented earlier and the
subsequent discussion on DFT, we propose the hypothesis of T′′–H′ phase
transitions, as this scenario aligns consistently with all the experimental
and theoretical methods used in our study as well as with observations from
previously published work [63]. Specifically, our observations indicate that a
certain level of defects is necessary to facilitate the T′′–H′ phase
transition. These defects are induced either by fs laser irradiation or
exposure to energetic electrons. The 1H′ phase is metastable due to the
presence of an activation energy barrier, and once defects are introduced, the
transition from the 1T′′ phase to the 1H′ phase can be initiated by various
external stimuli, such as UV light, fs laser pulses, or fast electrons. The
formation of defects is irreversible, as evidenced by the irreversibility of
the PL and FET measurements, which do not revert to their initial performance
(see SI for details). However, the phase change in terms of SHG activity can
be almost completely reversed at elevated temperatures, as discussed below.

### Electronic structure calculations

Figure 4: DFT analysis of monolayer ReS2 phases. Left: stable 1T′′
semiconducting phase. Middle: unstable 1H phase. Right: metastable 1H′
semimetal phase.

Our STEM and SHG data indicate a phase transition to a state with broken
inversion symmetry. A reasonable first assumption is that the transition
involves the standard undistorted H phase, similar to that of monolayers MoS2,
WS2, and others [63]. However, electronic structure calculations reveal that
this phase is unstable in monolayer ReS2 [52], see Figure 4. This suggests
that another phase, exhibiting broken inversion symmetry, likely underlies the
observed SHG and STEM behaviors. STEM data further reveal that the “switched”
phase is distorted (see Figure 2c), confirming its distinction from the 1H
phase. Hence, we propose that this phase corresponds to 1H′ (the distorted 1H
phase; space group $Amm2$, see Figure 1c). Our numerical simulations,
incorporating phononic and electronic band structure diagrams (see Figure 4),
indicate the stability of the 1H′ phase and its potential accessibility from
1T′′ via high-energy excitations utilizing photons (both multi- and single-
photon processes are possible) or electrons (see Figure S2). These
calculations confirm that the 1H phase is unstable and therefore unlikely to
be responsible for the observed behavior. Instead, ReS2 tends to switch into a
metastable 1H′ semimetal state, which aligns with our experimental
observations in ADF STEM and SHG. To our knowledge, this is the first
demonstration of the existence and stability of the 1H′ phase in ReS2.

Our calculations suggest the involvement of sulfur vacancies in facilitating
the T′′–H′ phase transition. Furthermore, they also suggest the reduced energy
required to form sulfur vacancies in the 1H′ phase compared to 1T′′. The role
of the sulfur defects consists in providing the necessary space for the
transition to occur. Importantly, both the direct T′′ to H′ and reverse H′ to
T′′ phase transitions require synchronized motion of large numbers of sulfur
atoms, which can only occur when there is sufficient geometrical freedom to
perform such motion. Additionally, molecular dynamics simulations demonstrate
that the back-transition from 1H′ to 1T′′ requires S-vacancies and represents
a first-order phase transition involving nucleation and growth of one
(switched) phase within the continuum of another (unswitched) phase (data not
shown), confirming the role of sulfur vacancies. These predictions align with
both light- and electron-induced 1T′′–1H′ phase transitions observed in SHG
and STEM measurements, as both require and indicate defect formation prior to
the transition (for instance, see UV-induced phase change data, Figure 3c-e).

The SHG pattern of the proposed 1H′ phase can be simulated by using the
$\chi^{(2)}$ polarizability tensor obtained from density functional theory
(DFT) and yields a polarization-dependent profile with two distinct lobes
(Figure 1b). Our experimental SHG data suggests the existence of three
possible orientation domains within the 1H′ phase (rotated with respect to
each other by 60$\degree$). Their stochastic combination accounts for a
6-fold-like symmetric pattern (Figure 3d), where SHG lobes vary in length,
reflecting the prevalence of specific orientations in the final orientation
domain composite (see STEM data in Figure 2c).

Although numerical electronic structure calculations offer substantial support
for the existence and relative stability of the 1H′ phase, alongside
information about sulfur vacancies and second-order optical nonlinearities,
they lack precision in estimating the activation energy barrier for the
reverse H′ to T′′ transition, which is crucial to characterize the stability
of the H′ phase. Thus, in the subsequent section, we shift our focus to
experimental observations of this back-switching process. The metastability of
the H′ phase implies its eventual inevitable return to the stable T′′ phase,
with the switching rate following a temperature-dependent Arrhenius-like
behavior, as we demonstrate below.

Figure 5: Fs-laser-induced patterning mechanism. (a) The average SHG intensity
over the flake upon several patterning-recovery cycles of a bilayer ReS2
flake. Blue circles correspond to the counts of SHG-active 2T′′ of a bilayer,
and the red crosses correspond to SHG counts of a 2H′ phase. The insets
demonstrate the SHG maps of a bilayer. All the patterning cycles were carried
out with the same conditions of the fs-laser. The dashed line links the
measurements in chronological order. (b) Recovery FOM as a function of time
for different temperature values that stimulated the recovery process. (c) FOM
rate as an exponentially fitted recovery FOM function vs. inverted
temperature. The linear fitting reflects the energy activation barrier,
$E_{a}$, between the phases. Colors and symbols correspond to the same colors
and symbols as in (b). Inset is the sketch of the energy diagram corresponding
to the T′′–H′ phase transition.

### Estimation of the activation energy barrier in bilayer ReS2

We use a pristine BL flake of ReS2 to study the activation energy barrier
height for the reverse transition from the H′ to the T′′ phase. Unlike an ML
(see Figure 3), the BL is highly SHG-active [62, 63], which is reflected in
the upper inset with a blue frame of Figure 5a. Upon fs-laser patterning of
the area with specific step size (50 nm), exposure time (200 ms), and power (8
mW), the flake significantly reduces the counts of the SHG signal, indicating
the phase transition from the T′′ into H′ phase (see Figure 5a, lower inset
with a red frame). The reduction of SHG signal in BL ReS2 with laser
patterning (as opposed to the enhancement of SHG in the ML case) is in line
with previous observations [63]. After some time, the sample recovers its
initial SHG intensity counts, signaling the reverse transition to the pristine
state. By applying the exposure conditions multiple times after each recovery,
we repeated the patterning process 7 times for the bilayer to prove the
reversibility of the phase transition, as shown in Figure 5a. The blue circles
depict the recovered (or pristine for the first blue circle) SHG-active T′′
bilayer states, whereas the red crosses mark the patterned SHG-inactive H′
ones. Notably, we observe a clear trend that the difference in SHG signal
between the blue and red points is reducing with the number of cycles. This
could be due to incomplete recovery or substantial defect formation. Another
important parameter is the time the BL ReS2 flake resides in the metastable H′
phase. This time is temperature-dependent and hence allows to perform an
Arrhenius-like study. The phase transition is reversible, therefore the
activation energy barrier between ground T′′ phase and H′ should not
significantly exceed the thermal energy $k_{B}T$. With that in mind, we
introduce the phase-transition recovery figure-of-merit (FOM), which relies on
the SHG intensity as a measure of the phase transition. If the
I${}_{\textrm{rec}}$ and I${}_{\textrm{patt}}$ are the SHG intensities of the
recovered and patterned bilayer flake respectively, then FOM($t$,T) =
$\frac{I_{\textrm{rec}}-I_{\textrm{T}}(t)}{I_{\textrm{rec}}-I_{\textrm{patt}}}$,
where T is the temperature of the flake, $t$ is the time after the patterning
took place and $I_{\textrm{T}}(t)$ – the intensity of the SHG of the flake at
a particular moment $t$ and at specific temperature T. For a range of
temperatures from 60$\degree$C to 80$\degree$C with a 5$\degree$C step size,
we performed a series of the measurements of the FOM($t$,T) as shown in Figure
5b, for the same sample from Figure 5a. FOM = 1 right at the moment of a
freshly patterned flake (2H′), whereas FOM = 0 means that the sample recovered
to its initial SHG intensity (2T′′). The sample recovery curves at different
temperatures can be fitted with a single exponential decay function, with a
characteristic time scale $\tau$. The rate of the reverse phase transition is,
therefore, $\tau^{-1}$. For every particular temperature, we plot the
logarithm of $\tau^{-1}$. Furthermore, we fit the $\textrm{-ln}(\tau)$ versus
1/T using a linear function and obtain $E_{a}=0.55\pm 0.03$ eV with 95$\%$
confidence (see Figure 5c). The outlier experimental point corresponding to 80
∘C is not taken into account due to the observation of irreversible phase-
switching behavior at higher temperatures, signaling the introduction of
another process. Based on these experimental findings, we deduce that the
stabilization energy barrier for the 2H′ phase is $\sim$ 0.55 eV. The phase
transition diagram is schematically depicted in Figure 5c. Initially, the H′
phase can not form due to the insufficient number of defects. However, upon
fs-laser-induced multiphoton absorption, combined with the simultaneous
creation of S-vacancies, a new excited state relaxation pathway opens up,
enabling the formation of the metastable H′ phase through the first-order
phase transition, which involves nucleation and growth.

## Conclusions

In this study, we present evidence of a reversible phase transition occurring
in mono-, bi-, and trilayer ReS2 (the latter is shown in Figure S11). The
transition occurs from the T′′ to the H′ phase at room temperature and ambient
conditions. Through a combination of several experimental and theoretical
methods, including electron microscopy, second-order nonlinear spectroscopy,
photoluminescence, electronic transport, and DFT calculations, we identify
that the transition is consistent with the metastable H′ (distorted H) phase.
Notably, our data suggest a crucial role of sulfur vacancies in enabling this
transition. Our analysis also confirms the ground state phase of the ReS2 ML
and BL flakes to be 1T′′ and 2T′′, respectively. Furthermore, we
experimentally determine the activation energy barrier for the reverse H′ to
T′′ transition in BL ReS2 to be approximately 0.55 eV. From a broader
perspective, these results expand the knowledge of light- and electron-induced
phase-switching materials and offer insights into the potential for optically
programmable properties of TMD-based devices.

## Methods

Sample fabrication. The ReS2 flakes were mechanically exfoliated from the
high-quality bulk crystal (HQ Graphene) using scotch-tape with a subsequent
transfer on polydimethylsiloxane stamps (PDMS) in a cleanroom (Nanofabrication
laboratory at Chalmers within MyFab). For the STEM experiments, ReS2 flakes
were transferred to 20 nm thick SiN membrane TEM grids (simpore.com) using an
all-dry transfer method [74]. For the laser patterning and all the optical
measurements, the flakes were transferred from the PDMS stamps to the glass
slides. For the electrical transport measurements, ReS2 flakes were
transferred to the 285-nm-thick SiO2/Si highly $n$-doped substrates (Graphene
Supermarket, USA) with pre-fabricated 20 nm Cr/200 nm Au back contact. To
produce source and drain top contacts for ReS2 flakes, $\sim$ 300 nm layer of
950 PMMA A4 resist (MicroChem, USA) was applied by spin-coating. PMMA was
patterned by EBL employing Raith EBPG 5200 (Germany) system operating at 100
kV accelerating voltage and 30 nA current. A dose of 1300 $\mu$C/cm2 was
applied, and 1:3 MBIK:IPA mixture was used as a developer. Then, 5 nm Cr/200
nm Au layer was evaporated by Lesker PVD 225 e-beam evaporation system (Kurt
J. Lesker Company, Germany). Overnight lift-off in acetone was used to
finalize the contact fabrication.

Optical measurements and light-induced phase switching. Second-harmonic
generation (SHG) measurements were carried out using a tunable (690–1040 nm)
Ti:sapphire femtosecond laser (MaiTai HP-Newport Spectra-Physics) with a
$\sim$ 100 fs pulse duration and 80 MHz repetition rate. The light was
polarized linearly with a broadband linear polarizer and focused through an
objective (Nikon, 40$\times$, NA = 0.95). The positioning of the sample was
controlled both in plane ($x-y$) and focus ($z$) by a piezoelectric stage. The
signal was collected with an avalanche photo-diode (APD, IDQ, ID100 Visible
Single-Photon Detector) for SHG mapping or a spectrometer (fiber-coupled Andor
500$i$, equipped with Newton 920 CCD camera) for SHG spectroscopy. The
polarization-resolved SHG measurements were performed on the same setup using
$\lambda/2$ plate to rotate the linear polarization of the excitation light.

PL measurements (Figure S11) were carried out using an inverted optical
microscope (Nikon Eclipse 2000E) equipped with a high-NA oil immersion
objective (Nikon, 60$\times$, NA = 1.49). The excitation 447 nm continuous-
wave laser (Becker & Hickl BDL-440-SMC Diode Laser) was coupled to an optical
fiber and focused on the bottom side of the sample through the objective. A
488 nm long-pass filter is placed in the detection path to suppress the laser
excitation. The setup allows for rotating the incident linear polarization and
introducing an analyzer in the signal collection channel. The maximum input
power is 10 mW.

The fs-laser induced phase switching experiments were performed using 1040 nm
pump wavelength of the Ti:sapphire femtosecond laser (MaiTai HP-Newport
Spectra-Physics) with a $\sim$ 100 fs pulse duration and 80 MHz repetition
rate in a similar manner as described elsewhere [63]. The laser beam was
focused on a specific area of the flake with an approximate spot size of 1
$\mu$m using an objective (Nikon, 40$\times$, NA = 0.95). The piezoelectric
stage allowed to position the sample in an $x-y$ plane with a 50 nm step every
200 ms to ensure a homogeneous phase switching over the flake’s area. The pump
powers used were between in the 3 – 7 mW range, as shown in Figure 3.

UV light phase switching experiments were performed using the fourth harmonic
of a Nd:YAG laser Teem Photonics SNU-20F-10C, emitting at 266 nm with 0.55 ns
pulse length and 20.5 kHz repetition rate. The laser beam was defocused to
keep the intensity low over the area of 3 $\times$ 5 mm2. Averaged over time
power was 2 mW, while the peak intensity reached $\sim$ 1.2 kW/cm2.

STEM measurements and calculations. STEM imaging and PACBED experiments were
carried out on a JEOL Mono NEO ARM 200F microscope. ADF STEM imaging was
conducted using an accelerating voltage of 200 keV, a probe convergence half-
angle of 26.7 mrad, an ADF detector collection angle range of 65-146 mrad, and
a probe current of 44 picoampere (pA). In order to improve the quality of the
ADF STEM images for the atomic structure analyses, a combination of non-rigid
registration of image series [75] and template matching was used depending on
the specific data set. Additional details about the acquisition parameters,
electron dose, and processing details of each STEM image are provided in the
SI. PACBED experiments [66] were conducted using an accelerating voltage of 80
keV, a probe convergence half-angle of 6.35 mrad, and a probe current of $<$3
pA. To resolve the ML and BL ReS2 PACBED patterns more clearly, careful
corrections of the detector dark signal and SiN support signal were manually
performed (more details provided in SI). ADF STEM and PACBED simulations were
calculated using the frozen phonon multislice algorithm implemented in the
GPU-accelerated software Prismatic 2.0 [76]. Additional details about the STEM
experiments and simulations are provided in the SI.

Electrical transport measurements. FET measurements were conducted using two
source channel -measurement units (B2901A, Keysight) connected through a
General Purpose Interface Bus (GPIB) cable with a computer. The typical FET
measurements were performed at room temperature. The back-gate voltage was
systematically adjusted within the range of -10 V to 10 V, while
simultaneously tuning the source-drain voltage between -1 V and 1 V.

Electronic structure calculations. The energetics of different phases, their
dynamic stability, defect formation energies, and electronic band structures
were obtained via DFT calculations that were carried out using the projector
augmented-wave method [77] as implemented in the Vienna ab-initio simulation
package [78, 79]. The exchange-correlation potential was represented using the
van-der-Waals density functional with consistent exchange method [80, 81]. All
calculations were carried out using a plane-wave energy cutoff of 520 eV and
for the calculation of the forces, a finer support grid was employed to
improve their numerical accuracy. The Brillouin zone was sampled using
$\Gamma$-centered grid with a linear $\vec{k}$-point spacing of about 0.25 Å-1
and Gaussian smearing with a width of 0.1 eV. This setup has been previously
employed and tested for other layered materials [82, 83].

For the ideal $1A$ and $Amm2$ structures the electronic band structures were
furthermore computed using the $G_{0}W_{0}$ method [84] on a $\Gamma$-centered
8x8x1 grid. These calculations employed 282 empty bands, thereby including
states up to at least 28 above the valence band edge ($1A$) or Fermi level
($Amm2$), and used 50 imaginary frequency and time grid points.

Phonon dispersions were computed via the phonopy package [85] in combination
with the hiphive package [86] using force constants from DFT calculations.

The $\chi^{(2)}$ polarizability tensor was calculated using the method and
tools outlined by Taghizadeh et al. [87]. Briefly, we calculated the nonlinear
optical response using the DFT open-source GPAW package [88, 89]. We used the
PBE exchange-correlation functional [90], the Kohn-Sham orbitals were expanded
using a plane wave basis set with an energy cutoff of 500 eV, we used a
$21\times 21\times 3$ Monkhorst-Pack grid for the $\mathbf{k}$-mesh and a grid
spacing of less than 0.2 Å. The number of empty bands included in the sum over
bands was set to three times the number of occupied bands. We used Fermi-Dirac
occupation number smearing with a factor of 20 meV and line-shape broadening
of 50 meV. Time-reversal symmetry was imposed in order to reduce the
$k$-integrals to half.

## Data Availability

Data generated during the study is available upon reasonable request from the
corresponding author. For Arxiv version the Supplementary file is not
provided.

## Conflicts of interest.

The authors declare no conflicts of interest.

## Acknowledgements.

We thank B. Munkhbat (DTU) for the ReS2 sample that has been studied in TEM.
T.O.S. acknowledges funding from the Swedish Research Council (VR, research
environment grant No. 2016-06059, VR project, grant No. 2017-04545 and grant
No. 2022-03347), Chalmers Excellence Initiative Nano, 2D-TECH VINNOVA
competence center (Ref. 2019-00068) and Olle Engkvist foundation (grant No.
211-0063). A.B.Y., E.O., P.E., and T.O.S. acknowledge funding from the Knut
and Alice Wallenberg Foundation (KAW, under grant No. 2019.0140). A.B.Y.
acknowledges funding from the Swedish Research Council (VR, under grant No.
2020-04986). T.J.A. acknowledges funding from the Polish National Science
Center, project 2019/34/E/ST3/00359. This work was performed in part at Myfab
Chalmers and the Chalmers Material Analysis Laboratory (CMAL). The DFT
computations were enabled by resources provided by the Swedish National
Infrastructure for Computing (SNIC) at NSC, C3SE, and PDC partially funded by
the Swedish Research Council through grant agreement No. 2018-05973 and by the
ICM-UW (Grant #G55-6).

## Authors contribution.

G.Z., A.B.Y, B.K., E.O., and T.O.S. conceived the project idea. G.Z. and B.K.
performed optical measurements. A.B.Y. performed and analyzed STEM and PACBED
data. A.V.A. performed FET measurements. A.Y.P. fabricated the samples for FET
experiments. J.C. and Å.H. assisted with UV experiments. F.E., P.E., T.J.A.
calculated dispersion with DFT. T.J.A. calculated the nonlinear response of
the flakes. G.Z., A.B.Y, and T.O.S. wrote the manuscript with contributions
from all co-authors.

## References

  *   * [1] Novoselov, K.S., Geim, A.K., Morozov, S.V., Jiang, D.-e., Zhang, Y., Dubonos, S.V., Grigorieva, I.V., Firsov, A.A.: Electric field effect in atomically thin carbon films. Science 306(5696), 666–669 (2004)
  * [2] Mak, K.F., Shan, J.: Photonics and optoelectronics of 2D semiconductor transition metal dichalcogenides. Nat. Photon. 10(4), 216–226 (2016)
  * [3] Trovatello, C., Marini, A., Xu, X., Lee, C., Liu, F., Curreli, N., Manzoni, C., Dal Conte, S., Yao, K., Ciattoni, A., et al.: Optical parametric amplification by monolayer transition metal dichalcogenides. Nat. Photon. 15(1), 6–10 (2021)
  * [4] Radisavljevic, B., Radenovic, A., Brivio, J., Giacometti, V., Kis, A.: Single-layer MoS2 transistors. Nat. Nanotechnol. 6(3), 147–150 (2011)
  * [5] Manzeli, S., Ovchinnikov, D., Pasquier, D., Yazyev, O.V., Kis, A.: 2D transition metal dichalcogenides. Nat. Rev. Mater. 2(8), 1–15 (2017)
  * [6] Munkhbat, B., Wróbel, P., Antosiewicz, T.J., Shegai, T.O.: Optical constants of several multilayer transition metal dichalcogenides measured by spectroscopic ellipsometry in the 300–1700 nm range: High index, anisotropy, and hyperbolicity. ACS Photonics 9(7), 2398–2407 (2022)
  * [7] Wang, Q.H., Kalantar-Zadeh, K., Kis, A., Coleman, J.N., Strano, M.S.: Electronics and optoelectronics of two-dimensional transition metal dichalcogenides. Nat. Nanotechnol. 7(11), 699–712 (2012)
  * [8] Geim, A.K., Grigorieva, I.V.: Van der Waals heterostructures. Nature 499(7459), 419–425 (2013)
  * [9] Balents, L., Dean, C.R., Efetov, D.K., Young, A.F.: Superconductivity and strong correlations in moiré flat bands. Nat. Phys. 16(7), 725–733 (2020)
  * [10] Cao, Y., Fatemi, V., Fang, S., Watanabe, K., Taniguchi, T., Kaxiras, E., Jarillo-Herrero, P.: Unconventional superconductivity in magic-angle graphene superlattices. Nature 556(7699), 43–50 (2018)
  * [11] Fang, Y., Pan, J., Zhang, D., Wang, D., Hirose, H.T., Terashima, T., Uji, S., Yuan, Y., Li, W., Tian, Z., et al.: Discovery of superconductivity in 2M WS2 with possible topological surface states. Adv. Mater. 31(30), 1901942 (2019)
  * [12] Jin, C., Regan, E.C., Yan, A., Iqbal Bakti Utama, M., Wang, D., Zhao, S., Qin, Y., Yang, S., Zheng, Z., Shi, S., et al.: Observation of moiré excitons in WSe2/WS2 heterostructure superlattices. Nature 567(7746), 76–80 (2019)
  * [13] Alexeev, E.M., Ruiz-Tijerina, D.A., Danovich, M., Hamer, M.J., Terry, D.J., Nayak, P.K., Ahn, S., Pak, S., Lee, J., Sohn, J.I., et al.: Resonantly hybridized excitons in moiré superlattices in van der Waals heterostructures. Nature 567(7746), 81–86 (2019)
  * [14] Brem, S., Linderälv, C., Erhart, P., Malic, E.: Tunable phases of Moiré excitons in van der Waals heterostructures. Nano Lett. 20, 8534 (2020)
  * [15] Tang, Y., Li, L., Li, T., Xu, Y., Liu, S., Barmak, K., Watanabe, K., Taniguchi, T., MacDonald, A.H., Shan, J., et al.: Simulation of hubbard model physics in WSe2/WS2 moiré superlattices. Nature 579(7799), 353–358 (2020)
  * [16] Huang, X., Wang, T., Miao, S., Wang, C., Li, Z., Lian, Z., Taniguchi, T., Watanabe, K., Okamoto, S., Xiao, D., et al.: Correlated insulating states at fractional fillings of the WS2/WS2 moiré lattice. Nat. Phys. 17(6), 715–719 (2021)
  * [17] Wu, F., Lovorn, T., MacDonald, A.H.: Topological exciton bands in moiré heterojunctions. Phys. Rev. Lett. 118(14), 147401 (2017)
  * [18] Ciarrocchi, A., Tagarelli, F., Avsar, A., Kis, A.: Excitonic devices with van der Waals heterostructures: valleytronics meets twistronics. Nat. Rev. Mater. 7(6), 449–464 (2022)
  * [19] Zhang, L., Wu, F., Hou, S., Zhang, Z., Chou, Y.-H., Watanabe, K., Taniguchi, T., Forrest, S.R., Deng, H.: Van der Waals heterostructure polaritons with moiré-induced nonlinearity. Nature 591(7848), 61–65 (2021)
  * [20] Munkhbat, B., Canales, A., Küçüköz, B., Baranov, D.G., Shegai, T.O.: Tunable self-assembled Casimir microcavities and polaritons. Nature 597(7875), 214–219 (2021)
  * [21] Dirnberger, F., Quan, J., Bushati, R., Diederich, G.M., Florian, M., Klein, J., Mosina, K., Sofer, Z., Xu, X., Kamra, A., et al.: Magneto-optics in a van der Waals magnet tuned by self-hybridized polaritons. Nature 620(7974), 533–537 (2023)
  * [22] Huang, Y., Pan, Y.-H., Yang, R., Bao, L.-H., Meng, L., Luo, H.-L., Cai, Y.-Q., Liu, G.-D., Zhao, W.-J., Zhou, Z., et al.: Universal mechanical exfoliation of large-area 2D crystals. Nat. Commun. 11(1), 2453 (2020)
  * [23] Dumcenco, D., Ovchinnikov, D., Marinov, K., Lazic, P., Gibertini, M., Marzari, N., Sanchez, O.L., Kung, Y.-C., Krasnozhon, D., Chen, M.-W., et al.: Large-area epitaxial monolayer MoS2. ACS Nano 9(4), 4611–4620 (2015)
  * [24] Li, T., Guo, W., Ma, L., Li, W., Yu, Z., Han, Z., Gao, S., Liu, L., Fan, D., Wang, Z., et al.: Epitaxial growth of wafer-scale molybdenum disulfide semiconductor single crystals on sapphire. Nat. Nanotechnol. 16(11), 1201–1207 (2021)
  * [25] Munkhbat, B., Yankovich, A.B., Baranov, D.G., Verre, R., Olsson, E., Shegai, T.O.: Transition metal dichalcogenide metamaterials with atomic precision. Nat. Commun. 11(1), 1–8 (2020)
  * [26] Munkhbat, B., Küçüköz, B., Baranov, D.G., Antosiewicz, T.J., Shegai, T.O.: Nanostructured transition metal dichalcogenide multilayers for advanced nanophotonics. Laser & Photonics Rev. 17(1), 2200057 (2023)
  * [27] Weber, T., Kühner, L., Sortino, L., Ben Mhenni, A., Wilson, N.P., Kühne, J., Finley, J.J., Maier, S.A., Tittl, A.: Intrinsic strong light-matter coupling with self-hybridized bound states in the continuum in van der Waals metasurfaces. Nat. Mater. 22, 970–976 (2023)
  * [28] Zotev, P.G., Wang, Y., Andres-Penares, D., Severs-Millard, T., Randerson, S., Hu, X., Sortino, L., Louca, C., Brotons-Gisbert, M., Huq, T., et al.: Van der waals materials for applications in nanophotonics. Laser & Photonics Rev. 17(8), 2200957 (2023)
  * [29] Verre, R., Baranov, D.G., Munkhbat, B., Cuadra, J., Käll, M., Shegai, T.: Transition metal dichalcogenide nanodisks as high-index dielectric Mie nanoresonators. Nat. Nanotechnol. 14(7), 679–683 (2019)
  * [30] Tselikov, G.I., Ermolaev, G.A., Popov, A.A., Tikhonowski, G.V., Panova, D.A., Taradin, A.S., Vyshnevyy, A.A., Syuy, A.V., Klimentov, S.M., Novikov, S.M., et al.: Transition metal dichalcogenide nanospheres for high-refractive-index nanophotonics and biomedical theranostics. Proc. Natl. Acad. Sci. 119(39), 2208830119 (2022)
  * [31] Maciel-Escudero, C., Yankovich, A.B., Munkhbat, B., Baranov, D.G., Hillenbrand, R., Olsson, E., Aizpurua, J., Shegai, T.O.: Probing optical anapoles with fast electron beams. Nat. Commun. 14, 8478 (2023)
  * [32] Zograf, G., Polyakov, A.Y., Bancerek, M., Antosiewicz, T., Kucukoz, B., Shegai, T.: Combining ultrahigh index with exceptional nonlinearity in resonant transition metal dichalcogenide nanodisks. arXiv preprint arXiv:2308.11504 (2023)
  * [33] Sung, J., Shin, D., Cho, H., Lee, S.W., Park, S., Kim, Y.D., Moon, J.S., Kim, J.-H., Gong, S.-H.: Room-temperature continuous-wave indirect-bandgap transition lasing in an ultra-thin WS2 disk. Nat. Photon. 16, 792–797 (2022)
  * [34] Ling, H., Li, R., Davoyan, A.R.: All van der waals integrated nanophotonics with bulk transition metal dichalcogenides. ACS Photonics 8(3), 721–730 (2021)
  * [35] Ling, H., Manna, A., Shen, J., Tung, H.-T., Sharp, D., Fröch, J., Dai, S., Majumdar, A., Davoyan, A.R.: Deeply subwavelength integrated excitonic van der waals nanophotonics. Optica 10(10), 1345–1352 (2023)
  * [36] Li, W., Qian, X., Li, J.: Phase transitions in 2D materials. Nat. Rev. Mater. 6(9), 829–846 (2021)
  * [37] Chialvo, D.R.: Emergent complex neural dynamics. Nat. Phys. 6(10), 744–750 (2010)
  * [38] Marković, D., Mizrahi, A., Querlioz, D., Grollier, J.: Physics for neuromorphic computing. Nat. Rev. Phys. 2(9), 499–510 (2020)
  * [39] Wuttig, M., Yamada, N.: Phase-change materials for rewriteable data storage. Nat. Mater. 6(11), 824–832 (2007)
  * [40] Seo, S.-Y., Moon, G., Okello, O.F., Park, M.Y., Han, C., Cha, S., Choi, H., Yeom, H.W., Choi, S.-Y., Park, J., et al.: Reconfigurable photo-induced doping of two-dimensional van der Waals semiconductors using different photon energies. Nat. Electron. 4(1), 38–44 (2021)
  * [41] Wang, Y., Xiao, J., Zhu, H., Li, Y., Alsaid, Y., Fong, K.Y., Zhou, Y., Wang, S., Shi, W., Wang, Y., et al.: Structural phase transition in monolayer MoTe2 driven by electrostatic doping. Nature 550(7677), 487–491 (2017)
  * [42] Voiry, D., Yamaguchi, H., Li, J., Silva, R., Alves, D.C., Fujita, T., Chen, M., Asefa, T., Shenoy, V.B., Eda, G., et al.: Enhanced catalytic activity in strained chemically exfoliated WS2 nanosheets for hydrogen evolution. Nat. Mater. 12(9), 850–855 (2013)
  * [43] Cho, S., Kim, S., Kim, J.H., Zhao, J., Seok, J., Keum, D.H., Baik, J., Choe, D.-H., Chang, K.J., Suenaga, K., et al.: Phase patterning for ohmic homojunction contact in MoTe2. Science 349(6248), 625–628 (2015)
  * [44] Guan, Y., Ding, Y., Fang, Y., Wang, G., Zhao, S., Wang, L., Huang, J., Chen, M., Hao, J., Xu, C., et al.: Femtosecond laser-driven phase engineering of WS2 for nano-periodic phase patterning and sub-ppm ammonia gas sensing. Small 19, 2303654 (2023)
  * [45] Yu, Y., Nam, G.-H., He, Q., Wu, X.-J., Zhang, K., Yang, Z., Chen, J., Ma, Q., Zhao, M., Liu, Z., et al.: High phase-purity 1T’-MoS2\- and 1T’-MoSe2-layered crystals. Nat. Chem. 10(6), 638–643 (2018)
  * [46] Nasu, K.: Photoinduced Phase Transitions. World Scientific, Singapore (2004)
  * [47] Yang, H., Kim, S.W., Chhowalla, M., Lee, Y.H.: Structural and quantum-state phase transitions in van der Waals layered materials. Nat. Phys. 13(10), 931–937 (2017)
  * [48] Stojchevska, L., Vaskivskyi, I., Mertelj, T., Kusar, P., Svetin, D., Brazovskii, S., Mihailovic, D.: Ultrafast switching to a stable hidden quantum state in an electronic crystal. Science 344(6180), 177–180 (2014)
  * [49] Vaskivskyi, I., Mraz, A., Venturini, R., Jecl, G., Vaskivskyi, Y., Mincigrucci, R., Foglia, L., De Angelis, D., Pelli-Cresi, J.-S., Paltanin, E., et al.: A high-efficiency programmable modulator for extreme ultraviolet light with nanometre feature size based on an electronic phase transition. Nat. Photon. 18, 458–463 (2024)
  * [50] Zhang, E., Jin, Y., Yuan, X., Wang, W., Zhang, C., Tang, L., Liu, S., Zhou, P., Hu, W., Xiu, F.: ReS2-based field-effect transistors and photodetectors. Adv. Funct. Mater. 25(26), 4076–4082 (2015)
  * [51] Pradhan, N.R., McCreary, A., Rhodes, D., Lu, Z., Feng, S., Manousakis, E., Smirnov, D., Namburu, R., Dubey, M., Hight Walker, A.R., et al.: Metal to insulator quantum-phase transition in few-layered ReS2. Nano Lett. 15(12), 8377–8384 (2015)
  * [52] Rahman, M., Davey, K., Qiao, S.-Z.: Advent of 2D rhenium disulfide (ReS2): fundamentals to applications. Adv. Funct. Mater. 27(10), 1606129 (2017)
  * [53] Xiang, D., Liu, T., Wang, J., Wang, P., Wang, L., Zheng, Y., Wang, Y., Gao, J., Ang, K.-W., Eda, G., et al.: Anomalous broadband spectrum photodetection in 2D rhenium disulfide transistor. Adv. Opt. Mater. 7(23), 1901115 (2019)
  * [54] Gogna, R., Zhang, L., Deng, H.: Self-hybridized, polarized polaritons in ReS2 crystals. ACS Photonics 7(12), 3328–3332 (2020)
  * [55] Xiong, X., Kang, J., Liu, S., Tong, A., Fu, T., Li, X., Huang, R., Wu, Y.: Nonvolatile logic and ternary content-addressable memory based on complementary black phosphorus and rhenium disulfide transistors. Adv. Mater. 34(48), 2106321 (2022)
  * [56] Tongay, S., Sahin, H., Ko, C., Luce, A., Fan, W., Liu, K., Zhou, J., Huang, Y.-S., Ho, C.-H., Yan, J., et al.: Monolayer behaviour in bulk ReS2 due to electronic and vibrational decoupling. Nat. Commun. 5, 3252 (2014)
  * [57] Lin, Y.-C., Komsa, H.-P., Yeh, C.-H., Bjorkman, T., Liang, Z.-Y., Ho, C.-H., Huang, Y.-S., Chiu, P.-W., Krasheninnikov, A.V., Suenaga, K.: Single-layer ReS2: two-dimensional semiconductor with tunable in-plane anisotropy. ACS Nano 9(11), 11249–11257 (2015)
  * [58] Chenet, D.A., Aslan, B., Huang, P.Y., Fan, C., Van Der Zande, A.M., Heinz, T.F., Hone, J.C.: In-plane anisotropy in mono-and few-layer ReS2 probed by raman spectroscopy and scanning transmission electron microscopy. Nano Lett. 15(9), 5667–5672 (2015)
  * [59] Liu, F., Zheng, S., He, X., Chaturvedi, A., He, J., Chow, W.L., Mion, T.R., Wang, X., Zhou, J., Fu, Q., et al.: Highly sensitive detection of polarized light using anisotropic 2D ReS2. Adv. Funct. Mater. 26(8), 1169–1177 (2016)
  * [60] Ermolaev, G.A., Voronin, K.V., Toksumakov, A.N., Grudinin, D.V., Fradkin, I.M., Mazitov, A., Slavich, A.S., Tatmyshevskiy, M.K., Yakubovsky, D.I., Solovey, V.R., et al.: Wandering principal optical axes in van der Waals triclinic materials. Nat. Commun. 15(1), 1552 (2024)
  * [61] Zhou, Y., Maity, N., Rai, A., Juneja, R., Meng, X., Roy, A., Zhang, Y., Xu, X., Lin, J.-F., Banerjee, S.K., et al.: Stacking-order-driven optical properties and carrier dynamics in ReS2. Adv. Mater. 32(22), 1908311 (2020)
  * [62] Song, Y., Hu, S., Lin, M.-L., Gan, X., Tan, P.-H., Zhao, J.: Extraordinary second harmonic generation in ReS2 atomic crystals. ACS Photonics 5(9), 3485–3491 (2018)
  * [63] Küçüköz, B., Munkhbat, B., Shegai, T.O.: Boosting second-harmonic generation in monolayer rhenium disulfide by reversible laser patterning. ACS Photonics 9(2), 518–526 (2022)
  * [64] Castellanos-Gomez, A., Agraït, N., Rubio-Bollinger, G.: Optical identification of atomically thin dichalcogenide crystals. Appl. Phys. Lett. 96(21) (2010)
  * [65] Anzai, Y., Yamamoto, M., Genchi, S., Watanabe, K., Taniguchi, T., Ichikawa, S., Fujiwara, Y., Tanaka, H.: Broad range thickness identification of hexagonal boron nitride by colors. Appl. Phys. Express 12(5), 055007 (2019)
  * [66] LeBeau, J.M., Findlay, S.D., Allen, L.J., Stemmer, S.: Position averaged convergent beam electron diffraction: Theory and applications. Ultramicroscopy 110(2), 118–125 (2010)
  * [67] Ophus, C.: Four-dimensional scanning transmission electron microscopy (4D-STEM): From scanning nanodiffraction to ptychography and beyond. Microscopy and Microanalysis 25(3), 563–582 (2019)
  * [68] Zhang, C., Feng, J., DaCosta, L.R., Voyles, P.M.: Atomic resolution convergent beam electron diffraction analysis using convolutional neural networks. Ultramicroscopy 210, 112921 (2020)
  * [69] Komsa, H.-P., Kotakoski, J., Kurasch, S., Lehtinen, O., Kaiser, U., Krasheninnikov, A.V.: Two-dimensional transition metal dichalcogenides under electron irradiation: defect production and doping. Phys. Rev. Lett. 109(3), 035503 (2012)
  * [70] Köster, J., Kretschmer, S., Storm, A., Rasper, F., Kinyanjui, M.K., Krasheninnikov, A.V., Kaiser, U.: Phase transformations in single-layer MoTe2 stimulated by electron irradiation and annealing. Nanotechnology 35, 145301 (2024)
  * [71] Qiao, X.-F., Wu, J.-B., Zhou, L., Qiao, J., Shi, W., Chen, T., Zhang, X., Zhang, J., Ji, W., Tan, P.-H.: Polytypism and unexpected strong interlayer coupling in two-dimensional layered ReS2. Nanoscale 8(15), 8324–8332 (2016)
  * [72] Huang, J., Gao, H., Xia, Y., Sun, Y., Xiong, J., Li, Y., Cong, S., Guo, J., Du, S., Zou, G.: Enhanced photoelectrochemical performance of defect-rich ReS2 nanosheets in visible-light assisted hydrogen generation. Nano Energy 46, 305–313 (2018)
  * [73] Horzum, S., Çakır, D., Suh, J., Tongay, S., Huang, Y.-S., Ho, C.-H., Wu, J., Sahin, H., Peeters, F.: Formation and stability of point defects in monolayer rhenium disulfide. Phys. Rev. B 89(15), 155433 (2014)
  * [74] Castellanos-Gomez, A., Buscema, M., Molenaar, R., Singh, V., Janssen, L., Van Der Zant, H.S., Steele, G.A.: Deterministic transfer of two-dimensional materials by all-dry viscoelastic stamping. 2D Materials 1(1), 011002 (2014)
  * [75] Yankovich, A.B., Berkels, B., Dahmen, W., Binev, P., Sanchez, S.I., Bradley, S.A., Li, A., Szlufarska, I., Voyles, P.M.: Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts. Nat. Commun. 5, 4155 (2014)
  * [76] DaCosta, L.R., Brown, H.G., Pelz, P.M., Rakowski, A., Barber, N., O’Donovan, P., McBean, P., Jones, L., Ciston, J., Scott, M., et al.: Prismatic 2.0–simulation software for scanning and high resolution transmission electron microscopy (STEM and HRTEM). Micron 151, 103141 (2021)
  * [77] Blöchl, P.E.: Projector augmented-wave method. Phys. Rev. B 50, 17953–17979 (1994)
  * [78] Kresse, G., Hafner, J.: Ab initio molecular dynamics for liquid metals. Phys. Rev. B 47, 558–561 (1993)
  * [79] Kresse, G., Furthmüller, J.: Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Comput. Mater. Sci. 6(1), 15–50 (1996)
  * [80] Dion, M., Rydberg, H., Schröder, E., Langreth, D.C., Lundqvist, B.I.: Van der Waals density functional for general geometries. Phys. Rev. Lett. 92, 246401 (2004)
  * [81] Berland, K., Hyldgaard, P.: Exchange functional that tests the robustness of the plasmon description of the van der Waals density functional. Phys. Rev. B 89, 035412 (2014)
  * [82] Lindroth, D.O., Erhart, P.: Thermal transport in van der Waals solids from first-principles calculations. Phys. Rev. B 94(11) (2016)
  * [83] Eriksson, F., Fransson, E., Linderälv, C., Fan, Z., Erhart, P.: Tuning the lattice thermal conductivity in van-der-Waals structures through rotational (dis)ordering. ACS Nano 17, 25565 (2023)
  * [84] Hedin, L., Lundqvist, S.: Effects of electron-electron and electron-phonon interactions on the one-electron states of solids. In: Frederick Seitz, D.T., Ehrenreich, H. (eds.) Solid State Phys. vol. 23, pp. 1–181 (1970)
  * [85] Togo, A., Tanaka, I.: First principles phonon calculations in materials science. Scr. Mater. 108, 1–5 (2015)
  * [86] Eriksson, F., Fransson, E., Erhart, P.: The hiphive package for the extraction of high-order force constants by machine learning. Adv. Theory Simul. 2, 1800184 (2019)
  * [87] Taghizadeh, A., Thygesen, K.S., Pedersen, T.G.: Two-dimensional materials with giant optical nonlinearities near the theoretical upper limit. ACS Nano 14(4), 7155–7167 (2021)
  * [88] Mortensen, J.J., Hansen, L.B., Jacobsen, K.W.: Real-space grid implementation of the projector augmented wave method. Phys. Rev. B 71, 035109 (2005)
  * [89] Enkovaara, J., Rostgaard, C., Mortensen, J.J., Chen, J., Dulak, M., Ferrighi, L., an C. Glinsvad, J.G., Haikola, V., Hansen, H.A., Kristoffersen, H.H., Kuisma, M., Larsen, A.H., Lehtovaara, L., Ljungberg, M., Lopez-Acevedo, O., Moses, P.G., Ojanen, J., Olsen, T., Petzold, V., Romero, N.A., Stausholm-Moller, J., Strange, M., Tritsaris, G.A., Vanin, M., Walter, M., Hammer, B., Hakkinen, H., Madsen, G.K.H., Mieminen, R.M., Norskov, J.K., Puska, M., Rantala, T.T., Schiotz, J., Thygesen, K.S., Jacobsen, K.W.: Electronic structure calculations with GPAW: a real-space implementation of the projector augmented-wave method. J. Phys.: Condens. Matter 22, 253202 (2010)
  * [90] Perdew, J.P., Burke, K., Ernzerhof, M.: Generalized gradient approximation made simple. Phys. Rev. Lett. 77, 3865–3868 (1996)


# Visual Feature Encoding for GNNs on Road Networks

Oliver Stromann1,2, Alireza Razavi2 and Michael Felsberg1  1Computer Vision
Laboratory, Linköping University, Sweden
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>2Autonomous Transport
Solution Research, Scania CV AB, Sweden
Email<EMAIL_ADDRESS>
###### Abstract

In this work, we present a novel approach to learning an encoding of visual
features into graph neural networks with the application on road network data.
We propose an architecture that combines state-of-the-art vision backbone
networks with graph neural networks. More specifically, we perform a road type
classification task on an Open Street Map road network through encoding of
satellite imagery using various ResNet architectures. Our architecture further
enables fine-tuning and a transfer-learning approach is evaluated by
pretraining on the NWPU-RESISC45 image classification dataset for remote
sensing and comparing them to purely ImageNet-pretrained ResNet models as
visual feature encoders. The results show not only that the visual feature
encoders are superior to low-level visual features, but also that the fine-
tuning of the visual feature encoder to a general remote sensing dataset such
as NWPU-RESISC45 can further improve the performance of a GNN on a machine
learning task like road type classification.

##  I Introduction

Connected and autonomous vehicles are an emerging trend in the transport
sector, and the road network is the core infrastructure on which these
vehicles operate. Knowledge about road networks is the key for making good
decisions for connected and autonomous vehicles; for immediate actions of
individual traffic users, as well as large-scale traffic management in the
transportation system as a whole. It is therefore of large importance to find
meaningful and efficient representations of road networks to enable or
simplify the knowledge generation [1].

However, the structural information encoded in spatial graphs like road
networks has two shortcomings: first, the incompleteness of the encoded
information and second the absence of information that cannot easily be
encoded in discretized variables. On the other hand, unstructured image data
contains complementary information with the potential to fill in those gaps
[1].

###  I-A Contributions

In [1] we have shown how spatial graphs can be enriched with hand-crafted low-
level visual features to improve performance of graph neural networks (GNNs)
on a machine learning task. In the present work, we propose to utilize state-
of-the-art pretrained and fine-tuned vision backbone networks as visual
feature encoders for GNNs to fully exploit the image data that can be
associated to a spatial graph.

We perform a road type classification task using a GNN over crowd-sourced road
network data from OpenStreetMap (OSM) [2] with visual feature encodings
produced by ResNets [3] using high-resolution satellite imagery from Maxar
Technologies [4].

The results do not only show that the visual feature encoders are superior to
hand-crafted low-level visual features, but also that a fine-tuning of the
visual feature encoder to a general remote sensing dataset such as NWPU-
RESISC45 [5] can further improve the performance of a GNN on a machine
learning task like road type classification.

##  II Related Work

###  II-A Geographical Data on Road Networks

####  II-A1 Road Network Graphs

Road network graphs are spatial graphs, which are nowadays made available by
mapping sources like Google Maps, Microsoft Bing Maps, or OSM (see [6] and
references therein). A road network graph is an attributed directed spatial
graph, with intersections as nodes and roads connecting these intersections as
edges [1]. Additional information about intersections and roads can be
associated to the node and edge attributes. Intersection attributes can be
information like the coordinates or the type of right-of-way; while road
attributes can be features information about the number of lanes, the speed
limit or the geometry of the road.

The crowd-sourced road network graphs from OSM [2] however, often have missing
attributes or attributes that are inconsistently set [7, 8]. For example,
information about the number of lanes is often limited to the case when it
deviates from the norm - which might vary over different geographical regions.
Moreover, the attributes might also be subject to changes over time. Lastly,
due to the data structure of road network graphs, usually only discretized
information is stored in the attributes, i.e. each attribute can hold only one
value, even if in reality the attribute varies over the length of the road.

####  II-A2 Remote Sensing Data

A suitable complement to the road network graphs is remote sensing data. This
is data collected from air or spaceborne sensors like radars, lidars or
cameras. Due to the bird’s-eye perspective, it has a complete ground coverage
without missing data. Nowadays, analysis-ready remote sensing data is
available that has undergone atmospheric, radiometric, and topographic error
correction [4]. It is usually composited of temporal stacks of imagery, such
that dynamic objects like clouds and vehicles are removed [9].

Especially, with the rise of free and open platforms like Google Earth Engine,
Open Data Cube or openEO, remote sensing data is becoming more accessible and
can be conveniently included as additional sources of information (see [10]
and references therein). However, in the context of road networks, aerial
imagery of very high spatial resolution below 1 m is favourable [11], which is
mostly still proprietary data.

###  II-B Machine Learning Concepts

####  II-B1 Graph Neural Networks (GNNs)

Recent years brought a surge of publications on GNNs. Many architectures have
been proposed to produce deep embeddings of nodes, edges, or whole graphs [12,
13]. With low-dimensional embeddings [14, 15], convolutions [16], skip-
connections [17], dropout- and normalization-layers [18], or attention
mechanisms [19, 20], many techniques from other deep learning domains such as
computer vision and natural language processing were adapted to GNNs [13].

####  II-B2 Vision Backbone Networks

Over the past decade, deep learning in computer vision has made giant leaps,
and many architectures have been proposed to improve performance on machine
learning tasks such as image classification. At the core, many of these
architectures roughly follow the structure of multiple consecutive
convolutional layers, making up a convolutional neural network (CNN) followed
by fully-connected (FC) layers.

The CNN which extracts image encodings of the image input, can be regarded as
vision backbone, while the FC layers which map the image encodings to class
labels, can be regarded as the classifier head.

Models of such architectures pretrained on the extensive ImageNet [21] dataset
can be adapted to novel datasets and tasks using transfer learning [22, 23].
This adaption can be done in two ways: by feature extraction or by fine-tuning
[24]. In feature extraction, the backbone is kept static, but the head is
further trained to the new task. In fine-tuning, the parameters of both the
backbone and head are updated through further training. In both cases, the
head can be replaced by another one suited to the novel task. This way, the
architecture can be changed to a different learning domain, such as
regression.

In our work, we use ResNet architectures [3] as backbones. ResNets use
residual connections between the convolutional layers, mitigating the
vanishing gradient problem and therefore enabling deeper stacks of
convolutional layers. ResNets are frequently used in transfer learning
approaches, achieving state-of-the-art results [22].

###  II-C Machine Learning on Geographical Data

####  II-C1 Machine Learning on Road Networks

Application such as location-based services, fleet planning, next-delivery
suggestions, or traffic flow optimization help to tackle congestions and
sustainability issues in the growing urban transportation systems [25]. Many
of these applications are enabled or can be enhanced with machine learning on
road networks. Typical machine learning tasks like classification, regression,
sequence prediction or clustering on road networks are enabling vehicle-
centric predictions such as next-turn, destination and time of arrival
predictions or network-centric predictions such as speed limit, travel time or
traffic flow predictions [1, 26, 27, 28, 29, 30].

The spatial graph representations of a road network makes it a prime candidate
for utilizing GNNs. Jepsen et al. [28] propose a relational fusion network
(RFN), which use different representations of a road network concurrently to
aggregate features. Wu et al. [26] developed a hierarchical road network
representation (HRNR) in which a three-level neural architecture is
constructed that encodes functional zones, structural regions and roads
respectively.

In previous work, we have shown how hand-crafted low-level visual features
from satellite imagery like pixel intensity histograms can increase
performance of a GNN on a classification task [1]. He et al. [30] proposed an
integration of visual features from satellite imagery through CNNs as node
features to a GNN. They use a generic CNN which is trained from scratch
together with the GNN in an end-to-end learning framework.

####  II-C2 Transfer Learning in Remote Sensing

Transfer learning is commonly used in remote sensing, solving challenges like
scene classification [23], object detection [31, 32], or semantic segmentation
[33]. However, when transferring from a CNN pretrained on ImageNet [21] to
different domains than natural camera image classification, a fine-tuning to
the respective dataset is required. This fine-tuning may gradually span from
readjusting weights in the final classification layers to readjusting even
weights up to the first convolutional layers - or a careful combination of
both for certain parts of the training [34, 33].

##  III Materials and Methods

###  III-A Material

####  III-A1 Road Network Graph

We use a graph representation $G{=}(V,E)$ of the road network, which encodes
roads as vertices $v\in V$ and connections between roads as edges $e\in E$.
This representation is commonly used in GNN applications to road networks [1,
29, 26, 27, 28, 35]. In a way, it resembles how humans intuitively
conceptualize road networks; seeing it as connections of roads rather than
connections of intersections [36]. We obtain the road network data from OSM
[2] through OSMnx [37].

####  III-A2 NWPU-RESISC45

The NWPU-RESISC45 dataset is a publicly available benchmark for REmote Sensing
Image Scene Classification (RESISC), created by Northwestern Polytechnical
University (NWPU) [5]. It contains 31,500 images, covering 45 scene classes
with 700 images in each class. The dataset consists of remote sensing imagery
of different spatial resolutions, and the classes represent objects of
different spatial scales. We use NWPU-RESISC45 to fine-tune ResNet-models
(pretrained on ImageNet) to remote sensing imagery.

Figure 1: Overview of the dataflow in our proposed architecture.

It should be noted, that the NWPU-RESISC45 dataset contains classes that are
not explicitly related to the target task of road type classification.
Instead, it contains general scenes from land use and land cover classes
(e.g., commercial area, farmland, forest, industrial area, mountain,
residential area), man-made object classes (e.g., airplane, airport, bridge,
church, palace, ship), as well as landscape nature object classes (e.g.,
beach, cloud, island, lake, river, sea ice) [5]. As such, the classes have a
wide variety of spatial and spectral patterns.

###  III-B Methods

####  III-B1 GNN Layers

We use graph convolutional layers (GCNs) as GNN layers. From the input graph
$G{=}(V,E)$, GCNs produce node representations of the $\mathbf{h}_{v}^{k}$ for
each node $v\in V$ at layer depth $k$. To do so, node representations of the
preceding GNN-layer from neighbouring nodes $\mathbf{h}_{n}^{k-1},\forall n\in
N(v)$ are aggregated using an aggregation function $\textsc{Agg}^{k}$ and
multiplied by a learnable weight matrix $\mathbf{W}^{k}$ before being passed
through a non-linear activation function $\sigma$.

$\mathbf{h}_{v}^{k}=\sigma\left(\mathbf{W}^{k}\cdot\textsc{Agg}^{k}\left(\\{\mathbf{h}_{n}^{(k-1)}\\}\right)\right).$
(1)

At layer depth $k{=}0$ the node representation consist of the node features,
such that $\mathbf{h}_{v}^{0}{=}\mathbf{x}_{v},\forall v\in V$. The node
representations after the final layer $\mathbf{h}_{v}^{K}$ is the latent node
representation - for notational convenience denoted as $\mathbf{z}_{v}$ \- and
is used in final classification or regression layers [38].

Depending on the architecture, the self-node representation might be
concatenated to the neighbouring node representations. This is either done
before the aggregation like in GCN [16], such that

$\mathbf{h}_{v}^{k}=\sigma\left(\mathbf{W}^{k}\cdot\textsc{Agg}^{k}\left(\\{\mathbf{h}_{v}^{(k-1)}\\}\oplus\\{\mathbf{h}_{n}^{(k-1)}\\}\right)\right),$
(2)

where $\oplus$ denotes concatenation, or after the aggregation like in
GraphSAGE [38], such that.

$\mathbf{h}_{v}^{k}=\sigma\left(\mathbf{W}^{k}\cdot\\{\mathbf{h}_{v}^{(k-1)}\\}\oplus\textsc{Agg}^{k}\left(\\{\mathbf{h}_{n}^{(k-1)}\\}\right)\right).$
(3)

For further details on the GNN layers, we refer the interested reader to Kipf
and Welling [16] for GCN, Hamilton et al. [38] for GraphSAGE and to the
PyTorch Geometric documentation of the implementation of these [39].

###  III-C Visual Feature Encodings for GNNs

Our proposed architecture consists of three parts:  i) a visual feature
encoder (VFE),  ii) a GNN producing latent node representations and  iii) a
classification head of FC layers mapping latent node representations to a road
type label.  Figure 1 illustrates the architecture.

The visual feature encoder or vision backbone takes image tiles as input and
produces image encodings. The image encodings, concatenated with non-image
node attributes, serve as input to the GNN. The GNN aggregates node features
of the local neighbourhood and produces latent node representations, which in
turn serve as input to a final FC layer and a soft-max activation for
classification. Figure 1 illustrates the architecture.

From the image tile $i_{v}$ of a node $v$, the visual feature encoder (VFE)
produces the image encodings

$\tilde{i}_{v}=f_{\mathrm{VFE}}(i_{v}).$ (4)

The image encodings $\tilde{i}_{v}$ are concatenated with the other node
attributes $\mathbf{x}_{v}$, to form the feature vector

$\tilde{\mathbf{x}}_{v}=\tilde{i}_{v}\oplus\mathbf{x}_{v},$ (5)

where $\oplus$ corresponds to a vector concatenation. $\boldsymbol{\rm X}$ are
the node attributes of all nodes excluding the image encodings,
$\tilde{\boldsymbol{\rm X}}$ are the node features of all nodes including the
image encodings, and $\boldsymbol{\rm A}$ is the adjacency matrix of the input
graph $G$. The GNN produces a latent representation $\mathbf{z}_{v}$ of the
self-node as described in Section III-B1, such that,

$\mathbf{z}_{v}=f_{\mathrm{GNN}}(\tilde{\boldsymbol{\rm X}},\boldsymbol{\rm
A}).$ (6)

Finally, the FC layer learns a mapping from the latent node representation
$\mathbf{z}_{v}$ to a road type label

$y_{v}=f_{\mathrm{FC}}(\mathbf{z}_{v}).$ (7)

Thus, the whole model is

$y_{v}=f_{\mathrm{FC}}(f_{\mathrm{GNN}}(\tilde{\boldsymbol{\rm
X}},\boldsymbol{\rm A})),$ (8)

where $f_{\mathrm{FC}}$ and $f_{\mathrm{GNN}}$ have learnable weights and
biases and are updated using stochastic gradient descent over a categorical
cross-entropy loss.

This study focusses on the visual feature encoder, and we evaluate different
architectures and pretraining strategies. More specifically, we compare hand-
crafted low-level visual features [1] against visual feature encodings
produced by ResNets pretrained on ImageNet. Moreover, we assess how a further
fine-tuning of these ResNets to the NWPU-RESISC45 dataset improves the
performance of GNN on road type classification. Conceptually, the motivation
for using the NWPU-RESISC45 to adapt towards the final task of road type
classification is illustrated in Figure 2.

All models are implemented in PyTorch Geometric [39] and the code will be
published.

Figure 2: Conceptual view of the class membership overlap of the different
datasets. ImageNet has little overlap with TrueOrtho for road type
classification. NWPU-RESISC45, though a general remote sensing dataset, has
larger overlap with TrueOrtho.

##  IV Experiments

To demonstrate how different visual feature encodings affect the performance
of a GNN on road networks, we run experiments on an exemplary machine learning
task, namely node-based multi-class classifications of road type labels [1].

###  IV-A Dataset

A road network from the city of Chengdu, China, is extracted from OSM [2].
Through random sampling, 1842 of the nodes in $E$ are selected for validation
and 1981 nodes are selected for testing. The remaining 18218 nodes are used
for training.

The highway label from OSM is used as the target class label. The label
describes the type of road and is an indicator for the importance of the road
within the network [40]. Figure 3 illustrates the class distributions for the
eight road type labels. The three classes living street, trunk and motorway
are underrepresented. This class imbalance makes the classification a
challenging problem.

Figure 3: Class distributions in percentage. Some classes are underrepresented
in the dataset.

The following attributes make up the feature vector $\mathbf{x}_{v}$ for each
node $v\in V$ and are identical to the attributes used in our previous work
[1]: Geographical attributes of length, bearing, centroid, and geometry, where
the latter is the road geometry resampled to a fixed number of equally-
distanced points and translated by the centroid to yield relative distances in
northing and easting (meters). Binary features depict one-way, bridge, and
tunnel.

These features are complemented with remote sensing data provided by Maxar
Technologies [4]. Specifically, we use high-resolution analysis-ready
orthorectified satellite imagery (TrueOrtho) with a spatial resolution of
$0.5\text{\,}\mathrm{m}$. TrueOrtho being analysis-ready means that it is a
temporal composite image in which clouds and vehicles have been removed and
all buildings and structures have been rectified. TrueOrtho contains
3-channels in RGB.

For each node $v\in V$, we extract a 120 x 120 pixel wide image tile $i_{v}$
from TrueOrtho rotated along the heading of the road.

###  IV-B ResNet Fine-tuning

For the visual feature encoders, we firstly use ResNet-18@ImageNet and
ResNet-50@ImageNet, which are the off-the-shelf ResNets pretrained on
ImageNet. To compare the impact of transfer learning to remote sensing, we
generate the ResNet-18@NWPU-RESISC45 and ResNet-50@NWPU-RESISC45 models by
fine-tuning ResNet-18@ImageNet and ResNet-50@ImageNet to the NWPU-RESISC45
dataset. This is done, by stripping away the FC classifier head of the
ResNets@ImageNet and replacing it with a similar FC with 45 output nodes. We
split the NWPU-RESISC45 dataset randomly such that for each class, 600 images
are used for training and 100 images are used for validation. We use
stochastic gradient descent with momentum and a categorical cross-entropy loss
to fine-tune the models. An exhaustive grid search over the learning rate
$\in[0.0001,0.001,0.01]$ and the momentum $\in[0.9,0.5]$ with a batch size of
512 and 100 epochs is performed to find the best models based on the
validation set.

###  IV-C Training

Next we train the GNNs using the various different visual feature encoders.
All models are trained in a semi-supervised, transductive setting, i.e. the
input graph is complete and features of training, validation, and test nodes
are reachable by the neighbourhood aggregation step of the GNN. However, only
training node labels are used to compute the categorical cross-entropy loss
and update the model parameters of the GNN and the FC.

The GNN consists of two layers (layer depth $K{=}2$) with a mean as aggregator
function $\textsc{Agg}^{k}$, ReLU activations and a dropout layer with a
dropout rate determined by a hyperparameter search. During training, local
neighbourhoods of the target nodes are sampled using the standard GraphSAGE
sampling strategy [38]. We evaluate GraphSAGE-layers and GCN-layers [38, 16].
Finally, a single FC-layer maps the GNN encodings to road type labels.

We perform an exhaustive grid search to find the best hyperparameters for each
model. Table I shows the search ranges. Hyperparameters are selected based on
the best validation set performance, but test set performances are reported.
The learnable model parameters are optimized using stochastic gradient descent
with a step-wise learning rate scheduler, multiplying the initial learning
rate by $\gamma$ every 25 epochs. All models were trained for a maximum of 100
epochs on an NVIDIA DGX-A100, with batch sizes varying from 32 to 64 depending
on the model size.

TABLE I: Hyperparameter search space. Hyperparameter | Search Space
---|---
learning rate | $[0.5,0.05]$
lr-scheduler $\gamma$ | $[0.2,0.5,0.8]$
weight decay | $[0.0004,0.0008]$
GNN dropout rate | $[0,0.15,0.3]$

##  V Results

Figure 4 shows the average micro-averaged F1-scores of the 5 top-performing
models in the hyperparameter search for the 8-class road type node
classification. Below is a description of the different models, in the order
of appearance:

  * •

GNN only does not use any visual feature encoder, and only node attributes
$\boldsymbol{\rm X}$ are available to the GNN.

  * •

Hist means that pixel intensity histograms from the RGB satellite imagery like
in [1] are appended to the node attributes to form the node features
$\boldsymbol{\rm\tilde{X}}$ for the GNN. 32 histogram values per channel (96
in total) are appended.

  * •

Hist+DSM has 32 additional histogram values compared to Hist (128 in total).
These stem from the digital surface model (DSM) as in [1].111In [1] the
training was done using the original GraphSAGE implementation in TensorFlow
[38], hence no result for GCN is reported.

  * •

ResNet-18 has visual feature encodings $\tilde{i}$ of size 512 produced from a
pretrained ResNet-18 appended to the node features.

  * •

Similarly, ResNet-50 has visual feature encodings $\tilde{i}$ of size 2048
produced from a pretrained ResNet-50 appended to the node features.

Figure 4: Micro-averaged F1-Scores for the different models, averaged over top
5 runs.

It is further indicated if the ResNets were pretrained on ImageNet (@ImageNet)
or if they were pretrained on ImageNet and then fine-tuned on NWPU-RESISC45
(@NWPU-RESISC45) as described in Section IV-B. The choice of GNN layers is
indicated by the orange cross for GraphSAGE and the blue diamond for GCN.

It can be seen that a GNN relying only on node attributes as features performs
worst, but that adding low-level visual features in the form of pixel
intensity histograms increase the classification performance drastically.
Visual feature encodings by ResNet-18@ImageNet increases the performance only
by a negligible margin compared to the low-level visual feature, and
ResNet-50@ImageNet performs just slightly better. However, a major leap in
performance can be observed from ResNets that were fine-tuned on remote
sensing data, especially with the larger ResNet-50@NWPU-RESISC45 which
achieves 0.75 or 0.72 micro-average F1-scores for GraphSAGE or GCN
architectures respectively.

##  VI Discussion and Conclusion

We have shown how visual feature encodings from state-of-the-art pretrained
vision backbones like ResNets can improve the performance of GNNs on spatial
graphs in a typical classification task on road networks. We also demonstrated
how these visual feature encodings are superior to hand-crafted low-level
visual features such as pixel intensity histograms, as previously presented in
[1]. Moreover, we have shown that pretrained ResNets fine-tuned to the general
remote sensing dataset of NWPU-RESISC45 can further boost the performance of
GNNs on road type classification.

The good performance of the ResNets@NWPU-RESISC45 suggests that the general
remote sensing dataset is closer to the target image tiles of different road
types extracted from TrueOrtho. Though, there is only a small class membership
overlap between NWPU-RESISC45 and TrueOrtho, it is still larger than between
ImageNet and TrueOrtho. Figure 2 illustrates how these datasets relate to
another. By fine-tuning the visual feature encoders from ImageNet to NWPU-
RESISC45, the encoders have moved closer to the target classifications in
TrueOrtho.

## Acknowledgment

We would like to gratefully acknowledge the support of Sweden’s innovation
agency, Vinnova, through project iQDeep (project number 2018-02700).

Map data copyrighted OpenStreetMap contributors and available from
https://www.openstreetmap.org.

We would like to gratefully thank Maxar Technologies for providing high-res
satellite imagery.

The computations were enabled by resources provided by the Swedish National
Infrastructure for Computing (SNIC), partially funded by the Swedish Research
Council through grant agreement no. 2018-05973.

We used Weights & Biases for experiment tracking and visualizations to develop
insights for this paper.

## References

  * [1] O. Stromann, A. Razavi, and M. Felsberg, “Learning to integrate vision data into road network data,” _arXiv preprint arXiv:2112.10624_ , 2021.
  * [2] OpenStreetMap, “Planet dump retrieved from https://planet.osm.org on 2021-06-10,” 2021 [Online].
  * [3] K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in _European conference on computer vision_. Springer, 2016, pp. 630–645.
  * [4] Maxar Technologies, “https://www.maxar.com/,” Aug. 2021. [Online]. Available: https://www.maxar.com/
  * [5] G. Cheng, J. Han, and X. Lu, “Remote sensing image scene classification: Benchmark and state of the art,” _Proceedings of the IEEE_ , vol. 105, no. 10, pp. 1865–1883, 2017.
  * [6] B. Veenendaal, M. A. Brovelli, and S. Li, “Review of web mapping: Eras, trends and directions,” _ISPRS International Journal of Geo-Information_ , vol. 6, no. 10, p. 317, 2017.
  * [7] S. Funke, R. Schirrmeister, and S. Storandt, “Automatic extrapolation of missing road network data in openstreetmap,” in _Proceedings of the 2nd International Conference on Mining Urban Data-Volume 1392_ , 2015, pp. 27–35.
  * [8] M. Haklay, “How good is volunteered geographical information? a comparative study of openstreetmap and ordnance survey datasets,” _Environment and planning B: Planning and design_ , vol. 37, no. 4, pp. 682–703, 2010.
  * [9] J. L. Dwyer, D. P. Roy, B. Sauer, C. B. Jenkerson, H. K. Zhang, and L. Lymburner, “Analysis ready data: enabling analysis of the landsat archive,” _Remote Sensing_ , vol. 10, no. 9, p. 1363, 2018.
  * [10] V. C. Gomes, G. R. Queiroz, and K. R. Ferreira, “An overview of platforms for big earth observation data management and analysis,” _Remote Sensing_ , vol. 12, no. 8, p. 1253, 2020.
  * [11] V. Mnih, “Machine learning for aerial image labeling,” Ph.D. dissertation, University of Toronto (Canada), 2013.
  * [12] W. L. Hamilton, “Graph representation learning,” _Synthesis Lectures on Artificial Intelligence and Machine Learning_ , vol. 14, no. 3, pp. 1–159, 2020\.
  * [13] J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, “Graph neural networks: A review of methods and applications,” _AI Open_ , vol. 1, pp. 57–81, 2020.
  * [14] A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in _Proceedings of the 22nd ACM SIGKDD_ , 2016, pp. 855–864.
  * [15] B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: Online learning of social representations,” in _Proceedings of the 20th ACM SIGKDD_ , 2014, pp. 701–710.
  * [16] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” _arXiv preprint arXiv:1609.02907_ , 2016.
  * [17] K. Xu, C. Li, Y. Tian, T. Sonobe, K.-i. Kawarabayashi, and S. Jegelka, “Representation learning on graphs with jumping knowledge networks,” in _International Conference on Machine Learning_. PMLR, 2018, pp. 5453–5462.
  * [18] J. Chen, J. Zhu, and L. Song, “Stochastic training of graph convolutional networks with variance reduction,” in _International Conference on Machine Learning_. PMLR, 2018, pp. 942–950.
  * [19] J. Zhang, X. Shi, J. Xie, H. Ma, I. King, and D.-Y. Yeung, “Gaan: Gated attention networks for learning on large and spatiotemporal graphs,” _arXiv e-prints_ , pp. arXiv–1803, 2018.
  * [20] P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” _stat_ , vol. 1050, p. 4, 2018.
  * [21] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein _et al._ , “Imagenet large scale visual recognition challenge,” _International journal of computer vision_ , vol. 115, no. 3, pp. 211–252, 2015.
  * [22] T. Hoeser and C. Kuenzer, “Object detection and image segmentation with deep learning on earth observation data: A review-part i: Evolution and recent trends,” _Remote Sensing_ , vol. 12, no. 10, p. 1667, 2020.
  * [23] R. Pires de Lima and K. Marfurt, “Convolutional neural network for remote-sensing scene classification: Transfer learning analysis,” _Remote Sensing_ , vol. 12, no. 1, p. 86, 2020.
  * [24] Z. Li and D. Hoiem, “Learning without forgetting,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 40, no. 12, pp. 2935–2947, 2017\.
  * [25] M. Veres and M. Moussa, “Deep learning for intelligent transportation systems: A survey of emerging trends,” _IEEE Transactions on Intelligent transportation systems_ , vol. 21, no. 8, pp. 3152–3168, 2019.
  * [26] N. Wu, X. W. Zhao, J. Wang, and D. Pan, “Learning effective road network representation with hierarchical graph neural networks,” in _Proceedings of the 26th ACM SIGKDD_ , 2020, pp. 6–14.
  * [27] J. Liu, G. P. Ong, and X. Chen, “Graphsage-based traffic speed forecasting for segment network with sparse data,” _IEEE Transactions on Intelligent Transportation Systems_ , 2020.
  * [28] T. S. Jepsen, C. S. Jensen, and T. D. Nielsen, “Relational fusion networks: Graph convolutional networks for road networks,” _IEEE Transactions on Intelligent Transportation Systems_ , 2020.
  * [29] Z. Gharaee, S. Kowshik, O. Stromann, and M. Felsberg, “Graph representation learning for road type classification,” _Pattern Recognition_ , p. 108174, 2021.
  * [30] S. He, F. Bastani, S. Jagwani, E. Park, S. Abbar, M. Alizadeh, H. Balakrishnan, S. Chawla, S. Madden, and M. A. Sadeghi, “Roadtagger: Robust road attribute inference with graph neural networks,” in _Proceedings of the AAAI_ , vol. 34, no. 07, 2020, pp. 10 965–10 972.
  * [31] Z. Chen, T. Zhang, and C. Ouyang, “End-to-end airplane detection using transfer learning in remote sensing images,” _Remote Sensing_ , vol. 10, no. 1, p. 139, 2018.
  * [32] C. Zhou, J. Zhang, J. Liu, C. Zhang, G. Shi, and J. Hu, “Bayesian transfer learning for object detection in optical remote sensing images,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 58, no. 11, pp. 7705–7719, 2020.
  * [33] B. Cui, X. Chen, and Y. Lu, “Semantic segmentation of remote sensing images using transfer learning and deep convolutional neural network with dense connection,” _IEEE Access_ , vol. 8, pp. 116 744–116 755, 2020.
  * [34] H.-C. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and R. M. Summers, “Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning,” _IEEE transactions on medical imaging_ , vol. 35, no. 5, pp. 1285–1298, 2016.
  * [35] Y. Zhang, T. Cheng, and Y. Ren, “A graph deep learning method for short-term traffic forecasting on large road networks,” _Computer-Aided Civil and Infrastructure Engineering_ , vol. 34, no. 10, pp. 877–896, 2019.
  * [36] M. Jilani, P. Corcoran, and M. Bertolotto, “Multi-granular street network representation towards quality assessment of openstreetmap data,” in _Proceedings of the Sixth ACM SIGSPATIAL International Workshop on Computational Transportation Science_ , ser. IWCTS ’13. New York, NY, USA: Association for Computing Machinery, 2013, p. 19–24. [Online]. Available: https://doi.org/10.1145/2533828.2533833
  * [37] G. Boeing, “Osmnx: New methods for acquiring, constructing, analyzing, and visualizing complex street networks,” _Computers, Environment and Urban Systems_ , vol. 65, pp. 126–139, 2017.
  * [38] W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in _Proceedings of the 31st NeurIPS_ , 2017, pp. 1025–1035.
  * [39] M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” in _ICLR Workshop on Representation Learning on Graphs and Manifolds_ , 2019.
  * [40] (2021, Aug.) Key:highway - OpenStreetMap Wiki. [Online]. Available: https://wiki.openstreetmap.org/wiki/Key:highway