# Exploring the impact of the inverse Faraday-effect on all-optical helicity-
dependent magnetization switching

M. Kohlmann  Institute of Physics, University of Greifswald, Greifswald,
Germany  L. Vollroth  Institute of Physics, University of Greifswald,
Greifswald, Germany  K. Jäckel  Institute of Physics, University of
Greifswald, Greifswald, Germany  K. Hovorakova  Faculty of Mathematics and
Physics,Department of Chemical Physics and Optics, Charles University, Prague,
Czech Republic  E. Schmoranzerova  Faculty of Mathematics and
Physics,Department of Chemical Physics and Optics, Charles University, Prague,
Czech Republic  K. Carva  Faculty of Mathematics and Physics, Department of
Condensed Matter Physics, Charles University, Prague, Czech Republic  D.
Hinzke  Department of Physics, University of Konstanz, Konstanz, Germany  U.
Nowak  Department of Physics, University of Konstanz, Konstanz, Germany  M.
Münzenberg  Institute of Physics, University of Greifswald, Greifswald,
Germany  J. Walowski  Institute of Physics, University of Greifswald,
Greifswald, Germany

###### Abstract

All-optical helicity-dependent magnetization switching (AO-HDS) is the
quickest deterministic technique for data storage by solely using ultrashort
laser pulses. Granular high data density magnetic storage media developed for
heat-assisted magnetic recording (HAMR) provide an ideal playground to
investigate the interplay of effects leading to magnetization switching. In
the latest perception, we identify two effects, the magnetic circular
dichroism (MCD) and the inverse Faraday effect (IFE), as the forces driving
the switching process. During photon absorption, which leads to a rapid
temperature rise and thus to magnetization quenching, the MCD ensures two
distinct electron temperatures due to helicity-dependent absorption. This
effect already holds a nonvanishing probability for magnetization switching.
At the same time, the IFE induces a magnetic moment within the material,
enhancing the switching probability. We present AO-HDS experiments using
ultrashort laser pulses ($\lesssim 200\,\mathrm{fs}$) in the near-infrared
range from $800\,\mathrm{nm}$ to $1500\,\mathrm{nm}$. The experiments
demonstrate a strong dependence of the switching efficiency on the absorbed
energy density, elevating the electron temperature in the vicinity of the
Curie point, allowing for the IFE to take full effect, inducing a magnetic
moment for deterministic switching in the quenched magnetization state. While
we do not observe an enhanced switching due to an increased MCD, a higher
induced magnetization usually improves the switching rate if the electron
temperature reaches the transition temperature vicinity. Therefore, we
conclude that the magnetic moment generated by the IFE is crucial for the
switching efficiency and the distinct deterministic character of the switching
process. Laser pulses with a higher absorption induce a higher magnetic moment
and switch magnetization at lower fluences.

FePt, inverse-Faraday effect, granular media, all-optical helicity dependent
switching, hard disk

Ultrafast magnetization dynamics remains a topic of ongoing research and has
been pushing into the attosecond range [1, 2], recently. At the femtosecond
timescale, however, the interest shifts to the search for applications. One
example is the deterministic magnetization control enabling the exploration
for applications like all-optical magnetization switching, which are
interesting for memory storage and magnetic sensors. Several switching
mechanisms in metals have been recently investigated in numerous publications,
splitting the field into all-optical helicity dependent switching (AO-HDS) [3,
4, 5, 6] and all-optical helicity independent switching (AO-HIS) [7, 8, 9,
10]. Utilizing these techniques opens possibilities for more specific
switching types such as toggle switching [11, 12] and switching by moving
domain walls in thin films [13, 14]. Additionally, investigations on more
complicated magnetic structures like dielectric crystals with a four-fold
magnetic anisotropy reveal the possibility of controlling magnetization on
ultra-fast time scales [15, 16]. More recently, even ways to control the
magnetic order through the phononic system are explored [17]. All these
experiments uncover the effects necessary for ultra-fast magnetization control
in various material systems and how their interplay affects the magnetic
ordering process. While in metals, this process requires heating the electron
system to the vicinity of the Curie Temperature, the switching in dielectrics
is conducted non-thermally.

AO-HDS is a promising approach for magnetic data recording in future
generations of magnetic storage devices because, in subsequent development
steps, it can be integrated into heat-assisted magnetic recording (HAMR)
technology. In contrast to HAMR, AO-HDS works purely optically, utilizing
femtosecond circularly polarized laser pulses. In the latest picture, for
ferromagnetic granular media, the MCD and the IFE are understood as the main
effects contributing to the magnetization switching in AO-HDS [5].

Multiple publications investigate the all-optical helicity-independent
switching (AO-HIS) [7, 8, 9, 10] showing that switching is possible utilizing
linear polarized femtosecond laser pulses. AO-HDS was first demonstrated on
ferrimagnets[3] and has been since applied to ferromagnetic materials
[Lambert.2014, 5, 18, 19, 20]. Recent theoretical calculations even show the
potential for switching magnetization in anti-ferromagnetic materials like
CrPt [21]. Recent developments in this research field have focused on thin
film and layered systems [22, 20, 14] and the reduction of pulses required to
drive the switching process [23]. Further, extensive research has been
performed on granular alloys like CoPt or FePt [24, 25, 5, 26, 27, 28]. These
materials are well suited for magnetic recording media production due to their
large coercive fields and saturation magnetization, making them magnetically
stable at room temperature. Besides this, the magnetic separation of single
grains allows magnetization switching of single grains with dimensions down to
$\sim 4\,$\mathrm{nm}$$, with possible bit area densities (AD) around
$4\,$\mathrm{T}\mathrm{b}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$
[29, 30].

Employment of AO-HDS in magnetic data storage devices for ultrafast recording
requires further elucidation of the involved processes. So far, the switching
experiments usually have been conducted with laser pulses at wavelengths
around $\lambda=800\,\mathrm{nm}$ or photon energies
$E=1.55\,$\mathrm{e}\mathrm{V}$$. The calculations published in [5, 31] point
to enhanced switching efficiency for smaller photon energies due to the
increasing difference in both the complex part of the refractive index and the
IFE coefficient for both helicities.

In this publication, we study the wavelength dependence for the switching
efficiency of granular FePt media in AO-HDS experiments. We present data for
$\lambda_{C}=800\,$\mathrm{nm}$$ and the range of
$\lambda_{C}=1200\,$\mathrm{nm}$-1500\,$\mathrm{nm}$$, gaining a deeper
insight into the switching efficiency dependence on photon energy. Our data
clearly shows a decreasing switching efficiency for smaller photon energies.
This outcome indicates less efficient heating of the electron system to the
vicinity of the Curie Temperature due to a decreasing overall absorption for
those wavelengths.

We conduct all switching experiments on FePt $L1_{0}$ granular media
previously examined in[5]. The individual grains with diameters around
$10\,$\mathrm{nm}$$ are embedded into a non-magnetic protective carbon matrix,
which separates the grains and their magnetization. They are grown by sputter
deposition on an MgO seed layer with a NiTa heat sink on glass substrates, see
schematic and electron microscope image in figure 1a) and figure 1b)
respectively. The grains have a strong perpendicular magnetic anisotropy with
coercive fields of $\mu_{0}H_{c}=4\,$\mathrm{T}$$ and saturation fields of
$\mu_{0}H_{s}=6\,$\mathrm{T}$$ [24]. The samples are demagnetized for
switching experiments in an ultra high vacuum chamber within a heating above
the Curie Temperature $T_{C}=730\,$\mathrm{K}$$ and cooling down to room
temperature cycle without an applied magnetic field. This procedure ensures an
initial state before AO-HDS with a $50\%-50\%$ up-down magnetic configuration
of the grains.

Figure 1: a) Schematic of the sample structure used in the experiments. The
magnetic grains, shown in white and grey, populate the MgO seed-layer
unordered. The arrows visualize the grains strong out-of-plane magnetization
anisotropy. b) SEM picture of the sample surface with an average grains size
$\sim 10\,$\mathrm{nm}$$. c) Experimental setup schematic. linearly polarized
Laser pulses from a Ti:Sapphire laser are circularly polarized by a quarter
wave plate and subsequently focused by a lens to a spot size of
$40\,$\mathrm{\SIUnitSymbolMicro m}$$, illuminating a sample mounted to a
transitional stage.

Figure 1c) displays the experimental setup schematics. We employ a Ti:Sapphire
Regenerative Amplifier RegA 9040 seeded by a Mira Ti:Sapphire mode-locked
oscillator (Coherent). The pulse repetition rate is set to
$250\,$\mathrm{kHz}$$, the central wavelength is
$\lambda_{C}=800\,$\mathrm{nm}$$ with a full width half maximum
$\mathrm{FWHM}\approx 35\,$\mathrm{nm}$$ spectral bandwidth, resulting in a
pulse duration $\tau_{P}=45\,$\mathrm{fs}$$ after compression. The pulses are
tuned by an optical parametric amplifier, Coherent OPA 9850, for experiments
in the wavelength range from $1200\,$\mathrm{nm}$-1500\,$\mathrm{nm}$$. After
conversion to higher wavelengths, the pulse length increases to
$90\,$\mathrm{fs}$-150\,$\mathrm{fs}$$ due to a higher bandwidth product. For
switching, a lens focusses the beam to a spot size with a diameter
$d=40\,$\mathrm{\SIUnitSymbolMicro m}$\pm 1\,$\mathrm{\SIUnitSymbolMicro m}$$
at the sample surface. The power deposited on the sample surface is set from
$0-60\,$\mathrm{mW}$$ generating fluences from
$3\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$-20\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$
and peak intensities from $\sim
5\,$\mathrm{G}\mathrm{W}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$-\sim
400\,$\mathrm{G}\mathrm{W}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$.
We set the helicity by a polarizer quarter-wave-plate sequence, using a
retarder for $\lambda=800\,$\mathrm{nm}$$ and an achromatic retarder
AQWP05M-1600 (Thorlabs Inc) for the
$\lambda=1200\,$\mathrm{nm}$-1500\,$\mathrm{nm}$$ range. The helicity of the
circular laser pulses is determined to $80\%\pm 10\%$.

The motorized $xyz$-stage moves the sample holder and enables transitions in
$xy$-direction to translate the laser beam on the sample surface and in
$z$-direction for spot size fine adjustment. The stage parameters allow for
$200\,$\mathrm{nm}$$ step sizes and speeds up to
$25\,$\mathrm{mm}\text{\,}{\mathrm{s}}^{-1}$$ in combination with the employed
repetition rate. Thus, we deposit $\sim 200$ pulses per FWHM beam waist at
every spot. This number of pulses ensures reaching the saturation state of
magnetization for each helicity [5]. Within the data analysis, we compare the
magnetization state after saturation for the employed wavelengths.

We image the switched magnetic areas with a magneto-optical Kerr microscope
with polar sensitivity. The microscope uses a Zyla 5.5 camera (Andor - Oxford
Instruments), a Zeiss ec epiplan x 50 with a numeric aperture $NA=0.75$
objective and a $\lambda=525\,$\mathrm{nm}$$ light source SOLIS-525C (Thorlabs
Inc) configured for Köhler-illumination. In addition to the Kerr rotation, the
light reflected from the sample surface contains Kerr ellipticity, which is
compensated for by an additional quarter-wave plate employed in the crossed
polarizer configuration. All evaluated images are averages over 100 captured
frames using background subtraction to eliminate illumination inhomogeneities
and obtain maximum magnetic contrast.

Figure 2: Kerr-microscopy image acquired as 16-bit gray scale values of a
demagnetized sample with lines switched by $\lambda_{C}=1300\,$\mathrm{nm}$$
pulses at fluence $F=8\,$\mathrm{mJ}\text{\,}{\mathrm{cm}}^{-2}$$. The red
arrow indicates the sample transition direction. The profiles are extracted by
integrating the area marked by the red rectangular box for signal-to-noise
ratio reduction.

For the switching efficiency discussion, we extract the profiles across the
written lines from the Kerr images to evaluate the Kerr-Contrast and switched
area size. Figure 2a) introduces a Kerr-microscope magnetic greyscale image of
two lines, after magnetization switching along the direction of the red arrow
using opposite helicities starting from a demagnetized state. The example
depicts the outcome employing the wavelength $\lambda_{C}=1300\,$\mathrm{nm}$$
and fluence set to $F\approx
8\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$.
Both circular laser beam spots are indicated in red ($\sigma^{+}$) and blue
($\sigma^{-}$). We obtain the intensity value profiles from the image,
averaging the magnetization contrast within the area edged in red over
multiple pixel rows to improve the signal-to-noise ratio (SNR) for data
evaluation. The two extracted representative profiles showing the cross-
section Kerr-Contrast are plotted in figure 2b). We define the areas switched
using right-hand helicity, $\sigma^{+}$ as the darker areas, decreasing the
Kerr signal, and the areas switched using left-hand helicity, $\sigma^{-}$ as
the brighter areas, increasing the Kerr signal. The resulting Kerr-Contrast is
proportional to the switched magnetization or the switching rate. The
nonuniform magnetization across the lines indicates a lower switching rate in
the center than on the edges. Due to the Gaussian beam profile, we deposit a
higher energy density in the beam center, generating nonuniform electron
temperatures and different magnetization dynamics across the beam. This non-
uniformity causes a reduced switching efficiency in the beam center. Besides
this, the profile width is $\sim 20\,$\mathrm{\SIUnitSymbolMicro m}$$, only
one-half of the beam spot diameter. Both extracted quantities strongly depend
on the laser fluence and applied wavelength, and therefore, we use them to
discuss the switching efficiency of the involved mechanisms and their
interplay.

Figure 3: Extracted line profiles switched using $\sigma^{-}$ helicity for all
wavelengths $\lambda_{C}=800\,$\mathrm{nm}$-1500\,$\mathrm{nm}$$ at the
deposited fluence of $F=9.3\pm
0.6\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$.
The grey-shaded Gaussian function in the background represents the spatial
laser pulse fluence distribution. The black dashed line marks the center of
the written lines. The blue and red dashed lines define the line widths for
switching with $\lambda_{C}=800\,$\mathrm{nm}$$ and
$\lambda_{C}=1450\,$\mathrm{nm}$$ pulses respectively.

First, we examine the differences and characteristics in the switching data
for all investigated wavelengths. Figure 3 shows the profiles after a
switching process using $\sigma^{-}$ helicity pulses for all applied
wavelengths with the incoming irradiation at the fluence $F=9.3\pm
0.6\,$\mathrm{mJ}\text{\,}{\mathrm{cm}}^{-2}$$. Usually, the Kerr images
exhibit slight deviations of less than $5\%$ in the overall signal from minor
brightness differences and illumination offsets. Therefore, we present those
profiles shifted, with Kerr-Contrast set to 0 representing the initial
demagnetized state and positive values describing the switched magnetization
for direct comparison. The profiles for both helicities show a high symmetry.
Accordingly, we limit the discussion to the $\sigma^{-}$ data. The grey-shaded
Gaussian function in the background represents the laser spot with a radius
$r=20\,$\mathrm{\SIUnitSymbolMicro m}$$ and a spacial fluence distribution
scaled on the right y-axis. The data obtained using
$\lambda_{C}=800\,$\mathrm{nm}$$ pulses for switching attracts attention with
a distinctly higher Kerr-Contrast over the whole profile and a slightly larger
switched magnetization area, corresponding to a radius $\approx
10\,$\mathrm{\SIUnitSymbolMicro m}$$, indicated by the blue dashed lines.
Compared to this, all other profiles exhibit a Kerr-Contrast reduced by $50\%$
and switched areas corresponding to radii $\lesssim
7.6\,$\mathrm{\SIUnitSymbolMicro m}$$, around $1/3$ of the laser spot radius,
indicated by the red dashed lines. Both quantities, the Kerr-Contrast and the
switched area, decrease by shifting the wavelength towards
$\lambda_{C}=1500\,$\mathrm{nm}$$. This behavior hints at an elevated
switching efficiency for $\lambda_{C}=800\,$\mathrm{nm}$$ pulses compared to
the other employed wavelengths.

Moreover, all profiles switched by wavelengths $\lambda_{C}\leq
1300\,$\mathrm{nm}$$ show a drop in the center of the switching area hinted by
the dashed black line in figure 3, indicating a reduced switching rate in this
area. One reason for the lower magnetization reversal is the Gaussian beam
shape, containing the largest fluence in the beam center. The laser pulse
deposits the highest energy in this place, generating the highest electron
temperature elevation. Through this enhanced heating, the electron temperature
dynamics in the beam center differs from that at the edges. While the
electrons at the edges reach temperatures above $T_{C}$ for a timeframe below
$\Delta\tau\leq 100\,$\mathrm{fs}$$ returning into the ferromagnetic state
immediately after the laser pulse ends, the electrons in the beam center reach
temperatures twice as high as $T_{C}$, remaining in this elevated state for up
to around $\Delta\tau\approx 1\,$\mathrm{ps}$$. Therefore, having a reduced
magnetization relaxation time, the electrons at the edges reverse their
magnetization towards the up-state, generating demagnetizing fields that
suppress the switching probability for the hot electrons in the beam center,
causing a lower switching rate. Our dataset is consistent with magnetization
dynamics and the extracted electron temperatures presented in [32]. Within our
data set, the switching rate reduction caused by the accelerated cooling
process at the edges differs for the employed wavelengths. However, it does
not show any significant trend in the wavelength dependence because the
switching procedure is more complex, containing other wavelength-dependent
effects discussed below.

Both wavelengths, $\lambda_{C}=1400\,$\mathrm{n}\mathrm{m}$$ and
$\lambda_{C}=1450\,$\mathrm{n}\mathrm{m}$$ achieve their maximum switching
capability indicated by the plateau and the peak in the profile plots,
respectively. These wavelengths do not reach the switching rate suppression in
the profile center due to lower overall electron temperatures. They merely
achieve electron temperatures sufficient for magnetization switching in the
center. Finally, no switching process is initiated for
$\lambda_{C}=1500\,$\mathrm{nm}$$ pulses at the applied fluence of
$F=9.3\,$\mathrm{mJ}\text{\,}{\mathrm{cm}}^{-2}$$. Lastly, the slight
asymmetries in the profiles originate from laser beam astigmatism, while the
peaks at Distance $\approx-13\,$\mathrm{\SIUnitSymbolMicro m}$$ showing in
some profile data originate from impurities on the sample surface.

For a systematic analysis, we extract the maximum Kerr-Contrast and the
switched areas from all profiles to discuss the relation between the amount of
deposited energy required for magnetization switching and to evaluate the
resulting efficiency.

Figure 4: a) Kerr-Contrast and b) switched area versus deposited laser fluence
for all employed wavelengths, lines are a guide to the eye. a) The Kerr-
Contrast shows a peak at an optimum fluence with a decreasing switching
efficiency for higher fluences. b) The switched area increases linearly with
the deposited laser fluence, reaching about one half of the spot size. The
gray lines (gray surface as the error) indicate a wavelength-dependent
threshold fluence increasing with the wavelength.

Figure 4 displays the maximum Kerr-Contrast a) and the switched area b),
plotted as a function of deposited irradiation fluence. The data are arranged
from $\lambda_{C}=1500\,$\mathrm{nm}$$ in the top panel to
$\lambda_{C}=800\,$\mathrm{nm}$$ in the bottom panel. Note that the scale
range for the lower panel is larger, allowing for constant scaling throughout
the panels. We determine the Kerr-Contrast error in figure 4a) as the standard
deviation of the Kerr microscope image noise, which is significantly smaller
than the signal, and calculate the switched area error using the radius
inaccuracies extracted from the line widths. Both data sets show a typical
laser fluence initiating the switching process within each spectrum, which
increases for longer wavelengths. We define this minimum fluence required for
a detectable switching rate as the threshold fluence, plotted in the
background by the grey lines across the panels as a guide to the eye, using
the fluence scale and the wavelength scale in the center. The light grey areas
represent the corresponding fluence error.

The Kerr-Contrast data shows an optimum fluence inducing the highest switching
efficiency, providing maximum Kerr-Contrast for each wavelength, which usually
occurs within
$1\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$ to
$3\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$
above the threshold fluence. When shifting the laser pulses to longer
wavelengths from $\lambda_{C}>800\,$\mathrm{nm}$$ to
$\lambda_{C}>1500\,$\mathrm{nm}$$, this Kerr-Contrast peak decreases by up to
$\sim 70\%$, and requires a larger fluence rise to occur. For all applied
wavelengths, increasing the fluence above the optimum point reduces the
maximum Kerr-Contrast, indicating a reduced switching efficiency.

This decreasing switching rate at higher fluences, which follow the optimum
switching fluence, is consistent with simulations using the multiscale
switching rate model published in [33]. Those simulations for an
antiferromagnet reveal that both processes, the differing heat entry through
the MCD and the IFE, contribute to magnetization switching and can trigger a
switching rate independently. However, the heat entry alone only induces very
low switching probabilities at fluences below the material melting point.
Therefore, the IFE, which scales with the laser fluence, significantly
increases the switching probability when heating achieves temperatures close
to the critical temperature. After that, higher temperatures induce more
disorder into the spin system, undermining the deterministic effect of the
IFE. The reduced magnetization switching rate we observe for some wavelengths
across the beam profile in figure 3 depends on the emerging demagnetizing
fields diminishing the switching procedure and the interplay of those two
effects. Therefore, we do not observe any simple wavelength-dependent trend
forming in the data.

The second parameter is the switched area size $A_{S}$, plotted in figure 4b)
as a function of fluence for each applied wavelength. This factor reveals the
switching efficiency regarding heat entering the magnetic material and its
distribution across the laser beam profile, while taking into account the
impact of the IFE. The data shows, in general, a linear switched area growth
with increasing laser fluence within the investigated range for a constant
laser spot size $A_{B}=1260\pm
260\,$\mathrm{{}^{2}}{\mathrm{\SIUnitSymbolMicro}\mathrm{m}}$$. The maximum
switched area $A_{S}$ remains below
$600\,$\mathrm{{}^{2}}{\mathrm{\SIUnitSymbolMicro}\mathrm{m}}$$, reaching
around half of $A_{B}$ for all employed wavelengths. The maximum switched area
decreases for the longer wavelengths, which is consistent with the Kerr-
Contrast behavior.

The second parameter, the switched area size $A_{S}$, plotted in figure 4b) as
a function of fluence for each applied wavelength, increases linearly with the
deposited fluence for all applied wavelengths for a constant laser spot size
$A_{B}=1260\pm 260\,$\mathrm{{}^{2}}{\mathrm{\SIUnitSymbolMicro}\mathrm{m}}$$.
Thata hints essentailly to a linearly increasing area, at which the critical
electron temperature enabling the switching process is reached. The maximum
switched area $A_{S}$ remains below
$600\,$\mathrm{{}^{2}}{\mathrm{\SIUnitSymbolMicro}\mathrm{m}}$$, reaching
around half of $A_{B}$ and decreasaes with increasing wavelength, which is
consistent with the Kerr-Contrast behavior.

The results presented in figure 4 suggest a superior switching efficiency for
$\lambda_{C}=800\,$\mathrm{nm}$$ laser pulses compared to the other employed
wavelengths. Within further data analysis, we elaborate on the effects acting
during the absorption of circularly polarized laser pulses and their balancing
to trigger magnetization reversal.

Figure 5:  a) Ab-initio calculated complex refractive index $n+ik$ with the
real and imaginery part, respectively for both helicities and the resulting
penetration depth $\Lambda_{\mathrm{opt}}$. b) Reflectivity $R$ and
transmission $T$ and the absorption $\alpha_{\mathrm{abs}}$ for the
investigated wavelength range.

Accordingly, we examine the optical properties derived from the helicity-
dependent refractive index for FePt with $\mathrm{L1_{0}}$ crystalline
structure calculated using ab initio methods. The data in figure 5 a) implies
a small change in reflectivity due to a rather constant real part $n$ of the
refractive index and a higher absorption for longer wavelengths due to the
increasing imaginary part $k$ of the refractive index. The penetration depth
$\Lambda_{\mathrm{opt}}=\frac{\lambda}{4\pi\cdot k}$ calculated from these
parameters increases from $13\,$\mathrm{nm}$$ for
$\lambda_{C}=800\,$\mathrm{nm}$$ to around $20\,$\mathrm{nm}$$ for the other
investigated wavelengths. Finally, using the derived reflectivity $R$ and
transmission $T$, for normal incidence, we calculate the absorption
$\alpha_{\mathrm{abs}}$ for $10\,$\mathrm{n}\mathrm{m}$$ sized grains. On the
one hand, this approach yields a larger overall absorption for
$\alpha_{\mathrm{abs}}(800\,$\mathrm{nm}$)=0.18$ dropping to
$\alpha_{\mathrm{abs}}(>1200\,$\mathrm{nm}$)<0.13$, shown in figure 5 b). On
the other hand, the absorption difference between the two helicities increases
for the longer wavelengths, resulting in a 5 to 10 times larger MCD. These
calculations confirm the tendency observed in the experimental data. The
absorption of shorter wavelength photons conducts higher electron temperatures
more efficiently, leading to a lower threshold fluence for switching. Electron
temperatures close to the curie temperature are crucial because, at those
temperatures, a quenched magnetization [32] facilitates switching by the low
induced magnetization of the inverse Faraday effect $\Delta M^{\mathrm{ind}}$.
Heating the electron system far above the Curie temperature is disadvantageous
because the growing thermal disorder diminishes the impact of $\Delta
M^{\mathrm{ind}}$ induced by the IFE. Further, the higher switching rates
reached by $\lambda_{C}=800\,$\mathrm{nm}$$ laser pulses hint at a negligible
impact of the MCD, which should deliver more distinct switching rates for the
longer wavelengths at higher fluences. Consequently, we review the extracted
Kerr-Contrast as a function of the absorbed fluence for all employed
wavelengths considering $\Delta M^{\mathrm{ind}}$ calculated from the
available IFE data.

Figure 6: a) Kerr-Contrast as a function of the absorbed laser fluence, with
the threshold fluence marked by the grey line. b) Magnetization $\Delta M$
induced by IFE for both helicities (blue and red lines) for the deposited
fluence of
$16\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$
(solid lines) and for the absorbed fluence of
$1.65\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$
(dashed lines) and the extracted Kerr-Contrast (black dots), for a deposited
fluence of
$16\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$.
c) Multi-scale modelling of the switching ratio for grains initially in the
magnetic ”down” state $N_{D}$ after illumination at the absorbed fluence for
three different IFE quantities.

The data rescaled in figure 6 a) confirms that taking into account absorption
from figure 5 b), the threshold fluence required for magnetization switching
remains at the same level over a long range up to
$\lambda_{C}=1400\,$\mathrm{n}\mathrm{m}$$ and doubles for the longer
wavelengths due to slightly increased pulse length. The grey line across the
panels marks the trend. A similar behavior is reported in [34] for GdFeCo in
the wavelength range
$\lambda_{C}=400\,$\mathrm{n}\mathrm{m}$-800\,$\mathrm{n}\mathrm{m}$$. A
comparison with temperatures extracted from the two-temperature model shows
that the threshold fluences generate electron temperatures around
$20\,$\mathrm{K}\mathrm{e}\mathrm{l}\mathrm{v}\mathrm{i}\mathrm{n}$$ below
$T_{C}$ and a magnetization quenched to below $20\%$ of the magnetization at
room temperature. The maximum Kerr contrast corresponding to the maximum
number of switched grains is achieved at fluences generating exactly $T_{C}$.
We see the decreasing trend in Kerr contrast at higher fluences for all
wavelengths. In the publication [35], the authors show that solely an MCD of
merely $0.5\%$ is enough to perform all-optical helicity-dependent switching.
However, in our data the wavelengths with a larger MCD do not achieve a higher
magnetization switching rate. We want to further elucidate the involved
effects to show, that the IFE increases the switching efficiency. For this, we
calculate the induced magnetization $\Delta M^{\mathrm{ind}}$ in units of
$\mathrm{\SIUnitSymbolMicro}_{B}$:

$\Delta M^{\mathrm{ind}}=K_{\mathrm{IFE}}\cdot\frac{I}{c},$ (1)

using $K_{\mathrm{IFE}}$ values from [5] in ${\mathrm{T}}^{-1}$, $I$ the
absorbed laser pulse intensity from experiments and $c$ the speed of light. At
threshold fluence, the calculated $\Delta M(\sigma^{-})$ remains below
$0.04\,$\mathrm{\SIUnitSymbolMicro}_{\mathrm{B}}$$ for all investigated
wavelengths. At those electron temperatures, the saturation magnetization is
quenched to below $10\%$ of the saturation magnetization at room temperature.
In figure 6 b), we compare $\Delta M^{\mathrm{ind}}$ for the plotted
wavelength spectrum. We consider two cases to compare the magnetic moment
induced by the IFE. First, for a deposited fluence of
$F=16\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$
(solid lines), we see that due to a higher overall absorption, $\Delta
M^{\mathrm{ind}}(\sigma^{-})=-0.21\,$\mu_{\mathrm{B}}$$ for
$\lambda_{C}=800\,$\mathrm{nm}$$, while the value is less than one third,
$\Delta M^{\mathrm{ind}}(\sigma^{-})<|-0.07|\,$\mu_{\mathrm{B}}$$ for the
other wavelengths. For $\sigma^{+}$ we observe a $\Delta M^{\mathrm{ind}}$
sign change for $\lambda_{C}>1200\,$\mathrm{nm}$$. Nevertheless, the small
value of $\Delta M^{\mathrm{ind}}(\sigma^{+})<0.01\,$\mu_{\mathrm{B}}$$ for
the longer wavelengths remains at the level achieved at threshold fluence for
$\sigma^{-}$ helicity, avoiding visible effects. Second, we compare the
calculated $\Delta M^{\mathrm{ind}}$ assuming the same absorbed fluence
$F_{\mathrm{abs}}=1.65\,$\mathrm{m}\mathrm{J}\mathrm{/}\mathrm{{}^{2}}{\mathrm{c}\mathrm{m}}$$
throughout the investigated wavelenght spectrum (dahsed lines). In this case,
the differences in $\Delta M^{\mathrm{ind}}$ reduce. However, the value for
$\Delta M^{\mathrm{ind}}(\sigma^{-})=-0.12\,$\mu_{\mathrm{B}}$$ is still twice
as large as for the remaining wavelengths. This shows, that due to the larger
$M^{\mathrm{ind}}$ for $\lambda_{C}=800\,$\mathrm{nm}$$, the switching
efficiency is always larger for this wavelength. The larger absorption enables
the maximum switching efficiency at lower deposited fluences. However,
increasing the fluence and thus $\Delta M^{\mathrm{ind}}$ does not result in a
higher switching efficiency because the electron temperature remains elevated
above $T_{C}$ on time scales in the picosecond range. During this time, spin
fluctuations destroy the deterministic switching character. Finally, in figure
6 b), the right scale shows the extracted Kerr-contrast values for the
investigated wavelengths at the discussed fluences. We see especially for
$\lambda_{C}=800\,$\mathrm{nm}$$ a higher Kerr-contrast for the lower fluence
(blue star) closer to the optimum than for the higher fluence, where we expect
a larger $\Delta M^{\mathrm{ind}}$. In the last step, we discuss the impact of
the inverse Faraday effect on the switching ratio, shown in figure 6 c). The
multiscale simulations use the same procedure as in [John.2018] show a
threshold fluence for switching, slightly higher than the experimental values.
The important statement shown here is that increasing the impact of the
inverse Faraday effect and leaving the remaining parameters stable leads to a
higher switching rate at lower fluences and decreases the switching rate at
higher fluences at higher temperatures above $T_{C}$.

In conclusion, our results show that the switching efficiency in all-optical
helicity-dependent switching strongly depends on the combined synergy of the
occurring effects. First, the amount of energy absorbed by the magnetic grains
needs to generate electron temperatures in close vicinity to $T_{C}$ to quench
the magnetization. Higher temperatures lead to a less efficient switching
process because thermal spin fluctuations annihilate the magnetization
direction induced by the photon helicity. In [36], the authors calculate the
corresponding temperature-dependent susceptibilities, $\chi_{\perp}$,
perpendicular and $\chi_{\parallel}$, parallel to the magnetization direction.
While $\chi_{\perp}$ vanishes at $T_{C}$, $\chi_{\parallel}$ shows a peak,
leading to a stronger attraction in this direction. Therefore, photons with
wavelengths inducing a higher magnetization $\Delta M^{\mathrm{ind}}$ generate
a higher magnetization switching probability at these temperatures. In our
data, this shows in a larger Kerr-contrast.

We are grateful to the German Science Foundation (DFG) for financial support
through the program: ’Fundamental aspects of all-optical single pulse
switching in nanometer-sized magnetic storage media’ Project number:
439225584. Further, we thank Tiffany Santos for the granular FePt hard disk
media.

## References

  * Siegrist  _et al._ [2019] F. Siegrist, J. A. Gessner, M. Ossiander, C. Denker, Y.-P. Chang, M. C. Schröder, A. Guggenmos, Y. Cui, J. Walowski, U. Martens, J. K. Dewhurst, U. Kleineberg, M. Münzenberg, S. Sharma, and M. Schultze, “Light-wave dynamic control of magnetism,” Nature 571, 240–244 (2019).
  * Neufeld  _et al._ [2023] O. Neufeld, N. Tancogne-Dejean, U. de Giovannini, H. Hübener, and A. Rubio, “Attosecond magnetization dynamics in non-magnetic materials driven by intense femtosecond lasers,” npj Computational Materials 9, 1–10 (2023).
  * Stanciu  _et al._ [2007] C. D. Stanciu, F. Hansteen, A. V. Kimel, A. Kirilyuk, A. Tsukamoto, A. Itoh, and T. Rasing, “All-optical magnetic recording with circularly polarized light,” Physical Review Letters 99, 047601 (2007).
  * Mangin  _et al._ [2014] S. Mangin, M. Gottwald, C.-H. Lambert, D. Steil, V. Uhlíř, L. Pang, M. Hehn, S. Alebrand, M. Cinchetti, G. Malinowski, Y. Fainman, M. Aeschlimann, and E. E. Fullerton, “Engineered materials for all-optical helicity-dependent magnetic switching,” Nature Materials 13, 286–292 (2014).
  * John  _et al._ [2017] R. John, M. Berritta, D. Hinzke, C. Müller, T. Santos, H. Ulrichs, P. Nieves, J. Walowski, R. Mondal, O. Chubykalo-Fesenko, J. McCord, P. M. Oppeneer, U. Nowak, and M. Münzenberg, “Magnetisation switching of fept nanoparticle recording medium by femtosecond laser pulses,” Scientific Reports 7, 4114 (2017).
  * Cheng  _et al._ [2020] F. Cheng, Z. Du, X. Wang, Z. Cai, L. Li, C. Wang, A. Benabbas, P. Champion, N. Sun, L. Pan, and Y. Liu, “All–optical helicity–dependent switching in hybrid metal–ferromagnet thin films,” Advanced Optical Materials 8, 2000379 (2020).
  * Radu  _et al._ [2011] I. Radu, K. Vahaplar, C. Stamm, T. Kachel, N. Pontius, H. A. Dürr, T. A. Ostler, J. Barker, R. F. L. Evans, R. W. Chantrell, A. Tsukamoto, A. Itoh, A. Kirilyuk, T. Rasing, and A. V. Kimel, “Transient ferromagnetic-like state mediating ultrafast reversal of antiferromagnetically coupled spins,” Nature 472, 205–208 (2011).
  * Le Guyader  _et al._ [2012] L. Le Guyader, S. El Moussaoui, M. Buzzi, R. V. Chopdekar, L. J. Heyderman, A. Tsukamoto, A. Itoh, A. Kirilyuk, T. Rasing, A. V. Kimel, and F. Nolting, “Demonstration of laser induced magnetization reversal in gdfeco nanostructures,” Applied Physics Letters 101, 022410 (2012).
  * Lalieu  _et al._ [2017] M. L. M. Lalieu, M. J. G. Peeters, S. R. R. Haenen, R. Lavrijsen, and B. Koopmans, “Deterministic all-optical switching of synthetic ferrimagnets using single femtosecond laser pulses,” Physical Review B 96 (2017), 10.1103/physrevb.96.220411.
  * Vergès  _et al._ [2023] M. Vergès, S. Perumbilavil, J. Hohlfeld, F. Freire-Fernández, Y. Le Guen, N. Kuznetsov, F. Montaigne, G. Malinowski, D. Lacour, M. Hehn, S. van Dijken, and S. Mangin, “Energy efficient single pulse switching of co/gd/ptn nanodisks using surface lattice resonances,” Advanced science (Weinheim, Baden-Wurttemberg, Germany) 10, e2204683 (2023).
  * Ignatyeva  _et al._ [2019] D. O. Ignatyeva, C. S. Davies, D. A. Sylgacheva, A. Tsukamoto, H. Yoshikawa, P. O. Kapralov, A. Kirilyuk, V. I. Belotelov, and A. V. Kimel, “Plasmonic layer-selective all-optical switching of magnetization with nanometer resolution,” Nature Communications 10, 4786 (2019).
  * Banerjee  _et al._ [2020] C. Banerjee, N. Teichert, K. E. Siewierska, Z. Gercsi, G. Y. P. Atcheson, P. Stamenov, K. Rode, J. M. D. Coey, and J. Besbas, “Single pulse all-optical toggle switching of magnetization without gadolinium in the ferrimagnet mn2ruxga,” Nature Communications 11, 4444 (2020).
  * Liao  _et al._ [2019] J.-W. Liao, P. Vallobra, L. O’Brien, U. Atxitia, V. Raposo, D. Petit, T. Vemulkar, G. Malinowski, M. Hehn, E. Martínez, S. Mangin, and R. P. Cowburn, “Controlling all-optical helicity-dependent switching in engineered rare-earth free synthetic ferrimagnets,” Advanced science (Weinheim, Baden-Wurttemberg, Germany) 6, 1901876 (2019).
  * Yoshikawa  _et al._ [2023] H. Yoshikawa, D. Tajima, Y. Kasatani, and A. Tsukamoto, “Different helicity dependency of all-optical magnetization switching in gdfeco films with optical interference layer,” Japanese Journal of Applied Physics 62, SB1009 (2023).
  * Stupakiewicz  _et al._ [2017] A. Stupakiewicz, K. Szerenos, D. Afanasiev, A. Kirilyuk, and A. V. Kimel, “Ultrafast nonthermal photo-magnetic recording in a transparent medium,” Nature 542, 71–74 (2017).
  * Stupakiewicz  _et al._ [2019] A. Stupakiewicz, K. Szerenos, M. D. Davydova, K. A. Zvezdin, A. K. Zvezdin, A. Kirilyuk, and A. V. Kimel, “Selection rules for all-optical magnetic recording in iron garnet,” Nature Communications 10, 612 (2019).
  * Stupakiewicz  _et al._ [2021] A. Stupakiewicz, C. S. Davies, K. Szerenos, D. Afanasiev, K. S. Rabinovich, A. V. Boris, A. Caviglia, A. V. Kimel, and A. Kirilyuk, “Ultrafast phononic switching of magnetization,” Nature Physics 17, 489–492 (2021).
  * Granitzka  _et al._ [2017] P. W. Granitzka, E. Jal, L. Le Guyader, M. Savoini, D. J. Higley, T. Liu, Z. Chen, T. Chase, H. Ohldag, G. L. Dakovski, W. F. Schlotter, S. Carron, M. C. Hoffman, A. X. Gray, P. Shafer, E. Arenholz, O. Hellwig, V. Mehta, Y. K. Takahashi, J. Wang, E. E. Fullerton, J. Stöhr, A. H. Reid, and H. A. Dürr, “Magnetic switching in granular fept layers promoted by near-field laser enhancement,” Nano letters 17, 2426–2432 (2017).
  * Yamamoto  _et al._ [2019] K. Yamamoto, Y. Kubota, M. Suzuki, Y. Hirata, K. Carva, M. Berritta, K. Takubo, Y. Uemura, R. Fukaya, K. Tanaka, W. Nishimura, T. Ohkochi, T. Katayama, T. Togashi, K. Tamasaku, M. Yabashi, Y. Tanaka, T. Seki, K. Takanashi, P. M. Oppeneer, and H. Wadati, “Ultrafast demagnetization of pt magnetic moment in l1 0 -fept probed by magnetic circular dichroism at a hard x-ray free electron laser,” New Journal of Physics 21, 123010 (2019).
  * Takahashi  _et al._ [2023] R. Takahashi, T. Ohkochi, D. Kan, Y. Shimakawa, and H. Wadati, “Optically induced magnetization switching in nico 2 o 4 thin films using ultrafast lasers,” ACS Applied Electronic Materials 5, 748–753 (2023).
  * Dannegger  _et al._ [2021a] T. Dannegger, M. Berritta, K. Carva, S. Selzer, U. Ritzmann, P. M. Oppeneer, and U. Nowak, “Ultrafast coherent all-optical switching of an antiferromagnet with the inverse faraday effect,” Physical Review B 104 (2021a), 10.1103/physrevb.104.l060413.
  * Liu  _et al._ [2023] Y. Liu, H. Cheng, P. Vallobra, H. Wang, S. Eimer, X. Zhang, G. Malinowski, M. Hehn, Y. Xu, S. Mangin, and W. Zhao, “Ultrafast single-pulse switching of tb-dominant cotb alloy,” Applied Physics Letters 122, 022401 (2023).
  * Yamada  _et al._ [2022] K. T. Yamada, A. V. Kimel, K. H. Prabhakara, S. Ruta, T. Li, F. Ando, S. Semin, T. Ono, A. Kirilyuk, and T. Rasing, “Efficient all-optical helicity dependent switching of spins in a pt/co/pt film by a dual-pulse excitation,” Frontiers in Nanotechnology 4, 6 (2022).
  * Weller  _et al._ [2013] D. Weller, O. Mosendz, G. Parker, S. Pisana, and T. S. Santos, “L1 0 feptx-y media for heat-assisted magnetic recording,” physica status solidi (a) 210, 1245–1260 (2013).
  * Takahashi  _et al._ [2016] Y. K. Takahashi, R. Medapalli, S. Kasai, J. Wang, K. Ishioka, S. H. Wee, O. Hellwig, K. Hono, and E. E. Fullerton, “Accumulative magnetic switching of ultrahigh-density recording media by circularly polarized light,” Physical Review Applied 6, 054004 (2016).
  * Suzuki and Takahashi [2023] I. Suzuki and Y. K. Takahashi, “Grain density control in fept granular films for heat-assisted magnetic recording media,” Japanese Journal of Applied Physics 62, SB0801 (2023).
  * Tsai  _et al._ [2023] J.-L. Tsai, C.-Y. Sun, J.-H. Lin, Y.-Y. Huang, and H.-T. Tsai, “Magnetic properties and microstructure of fept(bn, x, c) (x = ag, re) films,” Nanomaterials 13, 539 (2023).
  * Tham, Kushibiki, and Saito [2023] K. K. Tham, R. Kushibiki, and S. Saito, “Systematic study of nitride matrix of fept granular films for heat assisted magnetic recording media,” AIP Advances 13, 035034 (2023).
  * Weller  _et al._ [2014] D. Weller, G. Parker, O. Mosendz, E. Champion, B. Stipe, X. Wang, T. Klemmer, G. Ju, and A. Ajan, “A hamr media technology roadmap to an areal density of 4 tb/in$^2$,” IEEE Transactions on Magnetics 50, 1–8 (2014).
  * Suzuki  _et al._ [2023] I. Suzuki, T. Abe, H. Sepehri-Amin, K. Hono, and Y. K. Takahashi, “Microstructure evolution in fept-cr2o3 granular thin films,” Journal of Magnetism and Magnetic Materials 579, 170874 (2023).
  * Berritta  _et al._ [2016] M. Berritta, R. Mondal, K. Carva, and P. M. Oppeneer, “Ab initio theory of coherent laser-induced magnetization in metals,” Physical Review Letters 117, 137203 (2016).
  * Mendil  _et al._ [2014] J. Mendil, P. Nieves, O. Chubykalo-Fesenko, J. Walowski, T. Santos, S. Pisana, and M. Münzenberg, “Resolving the role of femtosecond heated electrons in ultrafast spin dynamics,” Scientific Reports 4, 3980 (2014).
  * Dannegger  _et al._ [2021b] T. Dannegger, M. Berritta, K. Carva, S. Selzer, U. Ritzmann, P. M. Oppeneer, and U. Nowak, “Ultrafast coherent all-optical switching of an antiferromagnet with the inverse faraday effect,” Physical Review B 104 (2021b), 10.1103/physrevb.104.l060413.
  * Steil  _et al._ [2011] D. Steil, S. Alebrand, A. Hassdenteufel, M. Cinchetti, and M. Aeschlimann, “All-optical magnetization recording by tailoring optical excitation parameters,” Physical Review B 84 (2011), 10.1103/PhysRevB.84.224408.
  * Gorchon, Yang, and Bokor [2016] J. Gorchon, Y. Yang, and J. Bokor, “Model for multishot all-thermal all-optical switching in ferromagnets,” Physical Review B 94 (2016), 10.1103/PhysRevB.94.020409.
  * Atxitia  _et al._ [2009] U. Atxitia, O. Chubykalo-Fesenko, R. W. Chantrell, U. Nowak, and A. Rebei, “Ultrafast spin dynamics: the effect of colored noise,” Physical Review Letters 102, 057203 (2009).


# C-Procgen: Empowering Procgen with Controllable Contexts

Zhenxiong Tan  Kaixin Wang  Xinchao Wang  National University of Sinagpore

(https://github.com/zxtan98/CProcgen)

###### Abstract

We present C-Procgen, an enhanced suite of environments on top of the Procgen
benchmark [6]. C-Procgen provides access to over 200 unique game contexts
across 16 games. It allows for detailed configuration of environments, ranging
from game mechanics to agent attributes. This makes the procedural generation
process, previously a black-box in Procgen, more transparent and adaptable for
various research needs. The upgrade enhances dynamic context management and
individualized assignments, while maintaining computational efficiency.
C-Procgen’s controllable contexts make it applicable in diverse reinforcement
learning research areas, such as learning dynamics analysis, curriculum
learning, and transfer learning. We believe that C-Procgen will fill a gap in
the current literature and offer a valuable toolkit for future works.

††footnotetext: * Equal contributions

##  1 Introduction

To make reinforcement learning (RL) algorithms more robust, adaptable, and
generalizable, the research community has built procedurally generated
environments [6, 8, 4] to facilitate evaluating algorithms. These
environments, characterized by their ability to introduce variations within a
single game using different contexts, require agents to avoid overfitting to
any particular set of contexts [17, 18, 15, 5]. Among various environments
available, Procgen [6] is one of the most comprehensive large-scale
benchmarks. With its 16 high-quality and diverse games, each complete with
unique landscapes and challenges, Procgen has been widely used in the study of
generalization and sample efficiency in reinforcement learning [9, 16, 1, 10,
7].

However, Procgen does not support precise control of the contexts. The
procedural generation of each level is a black-box process to users. For
example, in the game leaper (see Figure 1 for a screenshot), users are not
able to explicitly specify the number of lanes to create a curriculum. This
limits its full potential in domains like curriculum reinforcement learning
[12], transfer reinforcement learning [19], and meta reinforcement learning
[2], where precise manipulation of the environment’s contexts becomes
necessary. We will give a detailed discussion in Section 2. While some simple
environments with context control already exist, prior research has emphasized
the demand for a large-scale, high-quality, game-like environment [14, 13, 9].

To bridge this gap, we introduce C-Procgen to empower the original Procgen
with explicit context control. Specifically, C-Procgen encompasses all 16
games in Procgen, enhancing each game with a variety of adjustable context
parameters. These parameters span aspects such as game mechanics, agent
attributes, map complexity, and game-specific features. At the same time,
C-Procgen retains the high simulation speed of Procen, incurring only a
negligible overhead for switching context. We believe that C-Procgen offers a
more refined, flexible, and versatile environment for a wide range of research
avenues in reinforcement learning. To illustrate this, we highlight several
intriguing future prospects that can harness the strengths of C-Procgen
(Section 4).

##  2 What Hinders the Potential of Procgen?

Figure 1: Procgen does not offer precise control over the game contexts,
preventing us from generating the desired environments. Left: Different levels
in the same easy mode can differ greatly in difficulty. Right: Sometimes,
games in easy mode are even harder than those in hard mode.

In Procgen, the level generation process can only be controlled with the
difficulty mode parameter in a somewhat coarse manner. While the layout and
landscape of a given level are uniquely determined by the level ID, this
parameter is essentially a numerical identifier and lacks contextual
information about the specifics of the level. In the following, we will delve
into a detailed discussion on the limitations of Procgen.

Lack of Contextual Insight

In Procgen, each episode starts in a new level procedurally generated with
different contexts, unlike Atari games where the game’s context remains
constant. Thus, the agent’s behavior might vary considerably across different
contexts. However, Procgen does not offer a way to discern the contexts
without hacking the low-level source code, rendering the environment as a
black box. If we had access to the detailed context information for each
episode, we would be able to conduct a more comprehensive analysis of the
agent’s behavior. For example, we could investigate how the agent adapts to
various contextual factors, such as changes in the layout, obstacles, or
objectives within the procedurally generated levels. This level of insight
could enable us to uncover strategies and decision-making patterns that the
agent employs in response to different contexts. Additionally, having access
to detailed context information could facilitate the development of more
sophisticated and context-aware reinforcement learning algorithms, enhancing
the agent’s ability to generalize its skills across a wide range of scenarios.

Coarse-Grained Control Over Game Context

In addition to having no access to the underlying contexts, we are also
constrained by very limited control over the contextual parameters in Procgen.
The only available options are to select between two difficulty modes: easy or
hard. The way these contexts are chosen to construct a new game level has been
entirely hardcoded by the developers, which can significantly restrict
Procgen’s adaptability to various research requirements. At times, this lack
of fine-tuned control might lead to unexpected discrepancies. For instance,
levels may exhibit substantial variations in difficulty even when categorized
under the same easy mode, as seen in Figure 1 (left). Furthermore, there may
be instances where a level in the hard mode turns out to be much easier than
one in the easy mode, as shown in Figure 1 (right). If we were granted more
precise control over game contexts, we could define game distributions
tailored to our specific research needs. This would be particularly beneficial
in settings like meta reinforcement learning, where we can systematically
manipulate various context parameters and assess the agent’s adaptability.

Static Environment Parameters

When using Procgen’s parallel environments feature, each environment is
constrained to the same difficulty mode. Moreover, throughout the training
process, the environment parameters remain static and unmodifiable. Once the
environments are initialized, we are unable to make any changes to the context
configurations. This limitation can be particularly problematic in the context
of automatic curriculum learning [13], where the ability to dynamically adjust
environment contexts based on agent performance is crucial for efficient
training. For example, researchers might want to gradually increase the
difficulty of the environments by adjusting certain context parameters as
agents demonstrate improved performance, thereby promoting more efficient
learning. However, due to the static nature of Procgen’s environment
parameters, such adaptive curriculum adjustments are not attainable.

##  3 C-Procgen

Figure 2: An illustrative example comparing the original Procgen and our
C-Procgen.

We carefully refactor the source code of Procgen and expose many parameters
that determine the context, essentially graying the black-box generation
process. The resulting benchmark is C-Procgen, which augments the original
Procgen with Controllable Context parameters.

Direct and Rich Contextual Parameter Configuration

As illustrated in Fig 2, the original Procgen confines users to a simple level
ID, which offers only implicit control over the generation of game contexts.
In contrast, the enhanced version, C-Procgen, allows users to directly control
the context via explicit configuration of over 200 unique contextual
parameters across the suite of 16 Procgen games. The diversity of these
parameters encompasses:

  1. 1.

Game Mechanics: _e.g_., the maximum episode duration, the visible region size.

  2. 2.

Reward Structures: ones adjusting the intermediate reward values and the
rewards upon game end.

  3. 3.

Agent Attributes: _e.g_., the agent’s speed, health, and jumping capabilities.

  4. 4.

Map Complexity: ones allow users to set map dimensions, determine the number
of sections, or configure maze complexities.

  5. 5.

Game-Specific Features: for instance, one can specify the initial size of the
fish in bigfish, the variety of obstacles in cavefly, or the ball dimensions
dodgeball.

The flexibility and range of C-Procgen’s contexts allow researchers to craft
curricula within an expansive contextual space. Furthermore, it is
instrumental in some studies, such as edge-case testing and meta reinforcement
learning, by facilitating the design of specific game settings to suit
experimental needs.

Individualized Context Assignments in Vectorized Setups

In addition, C-Procgen also provides several engineering enhancements for
improved usability. Unlike the original Procgen, where the same configuration
is assigned to all environments during the initialization of vectorized
environments, C-Procgen provides the method (Listing 1) to assign distinct
contexts to each environment. Such flexibility is useful in exposing the
algorithm to richer game contexts during training.

⬇

# Define two different contexts

context_1 = {

"min_num_sections": 2,

"max_num_sections": 6,

}

context_2 = {

"air_control": 0.2,

"visibility": 6,

}

# Create C-Procgen environments with specific contexts

env = CProcgenEnv(

num_envs=2, env_name=’ninja’,

context_options=[context_1, context_2]

)

Listing 1: Defining and initializing game environments with specific contexts
using C-Procgen.

Dynamic Context Management and Analysis

Besides, one added convenience is the ability to modify the context of each
environment between two episodes without instantiating a new environment, as
shown in the ‘set_context_to’ method (Listing 2). This feature facilitates the
creation of a dynamic context distribution that could be of interest to the
curriculum reinforcement learning community.

⬇

# Define a new context

new_context = {

"min_num_sections": 1,

"max_num_sections": 1,

}

# Assign the new context to the first environment

env.set_context_to(0, new_context)

Listing 2: Assigning a new context to a running environment.

Moreover, for games where the context varies within a specified range and is
randomly generated for each episode, C-Procgen provides a method
env.get_context() to track the context for each episode.

High Efficiency

The enhancements brought by C-Procgen come with a minimal computation
overhead, maintaining simulation efficiency similar to the original Procgen.
Table 1 provides a speed comparison. For env.step(), the computation cost in
C-Procgen is only 10% higher than Procgen. The computational costs of the
env.get_context() and env.set_context_to() methods are approximately 1/5 and
2/3 of the original Procgen’s env.step(), respectively. Considering that these
two methods, in practical use, are generally executed only once per episode,
the additional computational overhead they would introduce can be considered
negligible.

Method | Time Cost (ms)
---|---
Procgen | C-Procgen | C/P
env.step() | 109,934 | 121,575 | 110.6%
env.get_context() | - | 23,992 | 21.8%
env.set_context_to() | - | 72,833 | 66.3%
Table 1: Computational cost comparison between C-Procgen and Procgen. Time
costs for key methods are listed for both versions. The time costs presented
in the table are measured under the condition of 64 environments running
concurrently. Each function is executed 1,000,000 times in every environment.
The “C/P” column shows the relative cost of C-Procgen methods as a percentage
of the baseline ‘env.step()’ from Procgen.

In summary, C-Procgen aims to provide more controllable context parameters and
flexibility in game environment setups. While offering explicit
configurability and individualized assignments, it retains computational
efficiency. C-Procgen holds the promise of catalyzing progress in curriculum
reinforcement learning research.

##  4 How to Use C-Procgen

The increased flexibility and fine-grained control offered by C-Procgen can
serve as a very useful tool for reinforcement learning research. Here are a
few illustrative examples and potential avenues for harnessing the
capabilities of C-Procgen:

  1. 1.

Contextual Insights into Learning Dynamic
With the inclusion of context information, we can now approach the analysis of
learning dynamics from the perspective of context. This provides a way to
explore how various metrics, such as score, loss, and policy entropy, evolve
as the agent learns across distinct contexts. Furthermore, analyzing episode
lengths within different contexts during the learning process is significant,
as it directly affects sample balance. Specifically, contexts with longer
episode lengths contribute more samples, paving the way for a deeper
understanding of learning dynamics in RL.

  2. 2.

Curriculum Learning
The capability of C-Procgen to dynamically modify contexts proves useful in
curriculum-based approaches [12, 13]. It serves as a foundation for
investigating the optimal selection and evolution of contexts that align with
an agent’s progressing abilities. Starting with basic configurations, the
agent can understand the game’s fundamental mechanics. As the agent advances,
the complexity rises, ensuring that contexts remain suitably challenging and
align with the agent’s skill level.

  3. 3.

Transfer Learning
C-Procgen allows for the training of agents in specific game contexts,
followed by evaluating their adaptability when exposed to drastically
different contexts. This method promotes the assessment of the adaptability of
various learning algorithms when faced with changes in game mechanics or
reward distribution.

  4. 4.

Context-aware Reinforcement Learning
C-Procgen facilitates context-aware reinforcement learning [11, 3], where
agents adjust their strategies based on the current context. This encourages
the development of adaptive agents that can generalize across different
scenarios, making their strategies more robust and versatile.

  5. 5.

Enhanced Diversity for Stronger Agents
With C-Procgen, it’s possible to introduce an array of contexts not present in
the original Procgen. This facilitates the creation of an environment set more
diverse than its precursor, promoting the training of more generalized and
resilient agents.

  6. 6.

Edge Case Analysis
The exhaustive parameter control provided by C-Procgen makes it viable to
construct edge cases or rare scenarios. This capability is invaluable for
thoroughly testing agents in non-standard conditions, evaluating their
resilience and adaptability. Furthermore, it allows for the generation of rare
yet crucial situations, potentially overlooked in the procedural generation of
Procgen.

  7. 7.

Environment Design Research
C-Procgen empowers researchers to accurately alter and examine specific game
mechanics, agent attributes, and map intricacies. This facilitates focused
studies on how different environmental elements influence agent learning, also
providing a platform for designing and experimenting with novel game dynamics,
ultimately enriching insights for refining RL environments.

The aforementioned applications of C-Procgen represent only a few examples of
its potential uses. We believe future investigations will uncover its full
potential.

##  5 Limitations, Discussion and Future Prospects

While C-Procgen brings improvements to procedural environment generation with
its diverse functionalities, it’s essential to recognize its challenges. In
this section, we’ll delve into its limitations, discuss inherent issues, and
touch upon potential paths forward.

Parameter Overload
While the introduction of over 200 unique contextual parameters allows for
fine-grained control, it may inadvertently lead to a parameter overload. This
increases the complexity and presents challenges in efficient environment
configuration and management. Keeping a balance between flexibility and
usability is something we need to consider.

Evaluation Challenges
Given that C-Procgen offers much more flexibility in terms of environment
configurations, comparing performance metrics across different studies could
be challenging. Variations in parameter settings can lead to inconsistencies
in benchmarks, making it difficult to compare algorithms in a standardized
manner.

Optimization Challenges
With a vast parameter space, optimization algorithms may encounter extended
computational times. Navigating this expansive parameter landscape could be a
double-edged sword, making it challenging to pin down the optimal
configurations promptly.

Lack of Evaluation Protocols
One of the clear limitations of our current work with C-Procgen is the absence
of specific evaluation protocols. Although the platform offers extensive
flexibility and configurability, it does not provide standardized benchmarks
or evaluation criteria.

Future Enhancements
The trajectory of C-Procgen seems promising. Anticipated future enhancements
might include mechanisms for automated optimal parameter detection or advanced
context analytics and visualization tools, assisting researchers in
understanding the effects of specific contexts on agent learning.

## References

  * Agarwal et al. [2021] Agarwal, R., Schwarzer, M., Castro, P. S., Courville, A. C., and Bellemare, M.  Deep reinforcement learning at the edge of the statistical precipice.  _Advances in neural information processing systems_ , 34:29304–29320, 2021.
  * Beck et al. [2023] Beck, J., Vuorio, R., Liu, E. Z., Xiong, Z., Zintgraf, L., Finn, C., and Whiteson, S.  A survey of meta-reinforcement learning.  _arXiv preprint arXiv:2301.08028_ , 2023.
  * Chen et al. [2021] Chen, B., Liu, Z., Zhu, J., Xu, M., Ding, W., Li, L., and Zhao, D.  Context-aware safe reinforcement learning for non-stationary environments.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pp. 10689–10695. IEEE, 2021.
  * Chevalier-Boisvert et al. [2023] Chevalier-Boisvert, M., Dai, B., Towers, M., de Lazcano, R., Willems, L., Lahlou, S., Pal, S., Castro, P. S., and Terry, J.  Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks.  _CoRR_ , abs/2306.13831, 2023.
  * Cobbe et al. [2019] Cobbe, K., Klimov, O., Hesse, C., Kim, T., and Schulman, J.  Quantifying generalization in reinforcement learning.  In _International Conference on Machine Learning_ , pp. 1282–1289. PMLR, 2019.
  * Cobbe et al. [2020a] Cobbe, K., Hesse, C., Hilton, J., and Schulman, J.  Leveraging procedural generation to benchmark reinforcement learning.  In _International conference on machine learning_ , pp. 2048–2056. PMLR, 2020a.
  * Cobbe et al. [2020b] Cobbe, K., Hilton, J., Klimov, O., and Schulman, J.  Phasic policy gradient.  _arXiv preprint arXiv: Arxiv-2009.04416_ , 2020b.
  * Justesen et al. [2018] Justesen, N., Torrado, R. R., Bontrager, P., Khalifa, A., Togelius, J., and Risi, S.  Illuminating generalization in deep reinforcement learning through procedural level generation.  _arXiv preprint arXiv: Arxiv-1806.10729_ , 2018.
  * Kirk et al. [2023] Kirk, R., Zhang, A., Grefenstette, E., and Rocktäschel, T.  A survey of zero-shot generalisation in deep reinforcement learning.  _J. Artif. Intell. Res._ , 76:201–264, 2023.  doi: 10.1613/jair.1.14174.  URL https://doi.org/10.1613/jair.1.14174.
  * Laskin et al. [2020] Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., and Srinivas, A.  Reinforcement learning with augmented data.  _Advances in neural information processing systems_ , 33:19884–19895, 2020.
  * Lee et al. [2020] Lee, K., Seo, Y., Lee, S., Lee, H., and Shin, J.  Context-aware dynamics model for generalization in model-based reinforcement learning.  In _International Conference on Machine Learning_ , pp. 5757–5766. PMLR, 2020.
  * Narvekar et al. [2020] Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., and Stone, P.  Curriculum learning for reinforcement learning domains: A framework and survey.  _The Journal of Machine Learning Research_ , 21(1):7382–7431, 2020.
  * Portelas et al. [2020] Portelas, R., Colas, C., Weng, L., Hofmann, K., and Oudeyer, P.  Automatic curriculum learning for deep RL: A short survey.  In Bessiere, C. (ed.), _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020_ , pp. 4819–4825. ijcai.org, 2020.  doi: 10.24963/ijcai.2020/671.  URL https://doi.org/10.24963/ijcai.2020/671.
  * Romac et al. [2021] Romac, C., Portelas, R., Hofmann, K., and Oudeyer, P.  Teachmyagent: a benchmark for automatic curriculum learning in deep RL.  In Meila, M. and Zhang, T. (eds.), _Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event_ , volume 139 of _Proceedings of Machine Learning Research_ , pp. 9052–9063. PMLR, 2021.  URL http://proceedings.mlr.press/v139/romac21a.html.
  * Song et al. [2019] Song, X., Jiang, Y., Tu, S., Du, Y., and Neyshabur, B.  Observational overfitting in reinforcement learning.  _arXiv preprint arXiv:1912.02975_ , 2019.
  * Wang et al. [2020] Wang, K., Kang, B., Shao, J., and Feng, J.  Improving generalization in reinforcement learning with mixture regularization.  _Advances in Neural Information Processing Systems_ , 33:7968–7978, 2020.
  * Zhang et al. [2018a] Zhang, A., Ballas, N., and Pineau, J.  A dissection of overfitting and generalization in continuous reinforcement learning.  _CoRR_ , abs/1806.07937, 2018a.  URL http://arxiv.org/abs/1806.07937.
  * Zhang et al. [2018b] Zhang, C., Vinyals, O., Munos, R., and Bengio, S.  A study on overfitting in deep reinforcement learning.  _arXiv preprint arXiv:1804.06893_ , 2018b.
  * Zhu et al. [2023] Zhu, Z., Lin, K., Jain, A. K., and Zhou, J.  Transfer learning in deep reinforcement learning: A survey.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2023.


# A Global Physician-Oriented Medical Information System

Axel Boldt  Department of Mathematics
Metropolitan State University
St. Paul, MN 55106, USA<EMAIL_ADDRESS>and  Michael Janich  B2 F5
Vantage Park
22 Conduit Rd
Mid Levels
Hong Kong<EMAIL_ADDRESS>
(Date: 8 October 2008)

###### Abstract.

We propose an Internet-based, free, world-wide, centralized medical
information system with two main target groups: practicing physicians and
medical researchers. After acquiring patients’ consent, physicians enter
medical histories, physiological data and symptoms or disorders into the
system; an integrated expert system can then assist in diagnosis and
statistical software provides a list of the most promising treatment options
and medications, tailored to the patient. Physicians later enter information
about the outcomes of the chosen treatments, data the system uses to optimize
future treatment recommendations. Medical researchers can analyze the
aggregate data to compare various drugs or treatments in defined patient
populations on a large scale.

###### Key words and phrases:

patient records; internet; diagnostic assistance; treatment recommendations;
open source software

##  1\. Introduction

The two main tasks performed by physicians are:

  * •

diagnosing a disorder, based on presented symptoms, the patient’s medical
history and features, and ordered tests; and

  * •

choosing the most appropriate treatment or medication for a given disorder and
a given patient.

In this article we propose a medical information system which aims to assist
physicians in both these tasks.

The ever increasing number of recognized diseases, combined with an explosion
in the number of marketed medications, poses formidable challenges to the
practicing physician. Many physicians rely mainly on four information sources:
their often outdated text books and lecture notes, a small selection of
medical journals, possibly biased informational material from pharmaceutical
companies, and their personal or anecdotal experiences. All of these are less
than ideal, and we maintain that they can and should be supplemented by more
rational decision aids based on modern data mining technology.

The core idea is simple: an expert system and treatment database that
physicians access over the Internet. After acquiring informed consent, they
enter a patient’s physiological data, symptoms and test results and the expert
system aids in diagnosis or recommends further tests. Once the disorder is
identified, the system recommends those treatments or medications with the
highest success probability for that particular patient. The physician chooses
a treatment and later records the outcome in the database. This outcome data
is used by the system to improve future treatment recommendations.

The system is designed to be used by physicians world wide, with special
regard for those working in developing countries. The use will be free of
charge and will only require an ordinary modem-speed connection to the
internet, something that is now available at reasonable cost in most
countries, often via mobile phones.

Since cost is often an important criterion when choosing a treatment or
medication, the system’s recommendations will be accompanied by cost estimates
specific to the physician’s location.

The two components of the system, diagnostic expert system and treatment
recommendation engine, use the same underlying patient database but are
logically independent and the system can go online as soon as one of the two
is fully functional. For example, physicians can eschew the expert system
altogether, rely on their own diagnostic skills and directly ask for treatment
recommendations for a particular patient’s disorder.

Apart from aiding physicians in their decision processes and thus improving
care and lowering cost on a global scale, the collected data will also provide
a rich resource for medical researchers. It will be possible to easily
determine the effectiveness of treatments and drugs in various patient
populations defined by combinations of characteristics such as age, sex,
ethnic group, pre-existing conditions, lifestyle, or any of a large number of
physiological measurements recorded by the system. In addition, the system
will be immediately useful by helping to compare new, expensive and heavily
marketed medications with older, more established alternatives. Evaluation of
alternative treatments like acupuncture or herbal remedies, heavily used all
over the world but rarely studied in a rigorous manner, will also become much
easier. Newly emerging epidemics will be detected in real time, much earlier
than is possible today. Lastly, it is likely that mining the database will
uncover rare but severe side effects of established medications that have so
far escaped detection.

##  2\. Detailed Description

###  2.1. Recruitment of physicians

The system will be open to all licensed practicing physicians world-wide.
Apart from the costs of a regular Internet connection, use of the system will
be free. Rather than attempting to contact all physicians directly, or to
advertise in relevant publications, it is hoped that the national medical
associations of the various countries can be recruited to promote the system
to their members and provide them with authorized access codes. This approach
has three advantages over a more direct marketing campaign:

  * •

it is significantly cheaper;

  * •

verification of physician’s credentials is done by the organizations best
suited for the task;

  * •

physicians extend a natural goodwill bonus to communications from their
respective medical association.

###  2.2. Physicians’ interaction with the system

Physicians are provided with access codes (passwords). They will normally
interact with the system by accessing the project’s website and logging in
with their name and password. The website will follow W3-standards and carry
few graphics in order to be easily accessable through slow modem connections
and simple devices, including mobile devices. It will be designed to make the
most common interactions simple and fast.

When accessing the system for the first time, new users are directed to a
tutorial about the system’s features and are required to agree to a set of
rules, mainly pertaining to patients’ informed consent and privacy (discussed
below in section 2.3).

In the standard use case, after authentication physicians are presented with a
list of their patients and past treatments and are invited to input outcome
data for these treatments in a quick and simple manner. They are then able to
view their patients’ records, create new records, use the expert system to
receive diagnostic assistance, or use the statistical database to get
treatment/medication recommendations. The top-rated treatments will be
presented along with estimates of their success probability and their cost.
The system further facilitates the physician’s decision process by providing
easy access to relevant background information, such as as reviews from the
Cochrane Library ([2]) and medical guidelines from the clearinghouse
responsible for the physician’s location ([1], [5]).

Signs, symptoms and diseases will be entered using the well-known ICD-10
classification system ([3]), and information about medications will be
accessable through the standard Anatomical Therapeutic Chemical Classification
System ([4]). It is to be expected that these systems will not be entirely
adequate for our purposes and will have to be modified and extended to a
certain degree.

In addition to the direct web-based interface, the system will also provide an
XML-based interface, allowing for the easy data exchange with physicians’
other software applications, for instance with their standard patient
management software or with their systems of interfacing with health insurers.

All patient information data transport will employ a layer of encryption to
avoid data being viewed or tampered with by unauthorized third parties.

###  2.3. Patient identifiers and privacy issues

The system will obey the world’s strictest privacy laws, like those commonly
found in countries of the European Union. This requires in particular that

  * •

data is collected with full consent and can only be used for the express
purposes given in the consent statement; and

  * •

patients retain the right to review their data and to have it removed at any
time.

Physicians are required to obtain informed consent before they may enter
patients’ data into the system. Physicians are not allowed to reject treatment
of patients who do not wish to participate in the system.

No personally identifiable information is ever entered into the database: no
names, no exact birthdates, no addresses etc. Instead, every patient is
identified by a PatientID, a simple number. The physician provides the patient
with their PatientID, so that the patient can authorize other physicians to
access their data if they so choose. Only physicians explicitly authorized by
a patient may access the records associated with that patient’s PatientID;
physicians are required to keep proof of this authorization on file.
Physicians will typically keep a record of all their patients’ PatientIDs on
their own computer.

It is possible and even to be expected that patients’ names together with
corresponding PatientIDs will occasionally fall into the wrong hands, for
instance if malware is installed on a physician’s computer, their office is
broken into or a patient’s private PatientID note is stolen. Since only
physicians associated with the PatientID may access the corresponding record,
the risk of data leaks is actually lower than in the current situation, where
all patient data would have been stored on the doctor’s compromised computer
rather than in the online database.

Physicians, unlike patients, do not remain anonymous and are recorded with
full name and address in the database. Further, all interactions of physicians
with the system are logged. This allows to trace and identify fraudulent use
by non-physicians.

During scientific analysis the database will only be queried in the aggregate
and no individual record will be accessed in its entirety. It is however
important to note that this does not completely protect against abuses; for
instance a query of the form ”what percentage of adult black males living in
Luxembourg and standing less than 1.6 m tall are HIV positive”, even though it
uses the database only in the aggregate, can still reveal very private
information about a small number of identifiable individuals. It is therefore
necessary that scientific analyses be reviewed ahead of time; see section 2.5
below for more details on this process.

###  2.4. Software

To maximize transparency and to encourage others to submit bug fixes and
feature enhancements, the project will make use of existing Open Source
software whenever possible and all software written by the project will be
published under an Open Source license ([7]).

Diagnostic expert systems have been developed before ([8]; [6] for a
comprehensive annotated list), and it is unrealistic to expect the project to
duplicate that work. Instead, existing expert systems will be evaluated and
the most appropriate one will be licensed and extended. Most modern systems of
this type employ Bayesian networks, and these would benefit tremendously from
the statistical knowledge stored in the patient database.

The treatment recommendation engine will have to be written from scratch,
using existing algorithms that have been developed for data mining
applications in marketing; given a database containing customer features and
outcomes of past marketing strategies, these algorithms can predict the most
promising marketing strategy taylored to a given customer. (See [9] for a
survey of data mining algorithms.) The algorithm will have to account for the
fact that information about patients is not necessarily complete; for example,
only some patient records will contain a recent blood glucose level.

As is usual in data mining applications, the comparatively low quality of data
collection procedures is compensated for by the large quantity of data. A
variety of different heuristics can be used to approach the recommendation
problem and a final algorithm can only be chosen after empirical evaluation.

As mentioned above, the system requires information about the prices of
treatments and medications in the various countries. It is possible and
desirable that the collection and maintenance of this data be carried out in
collaboration with the various national medical associations. The problem is
non-trivial, since drug prices can vary widely even within countries.

###  2.5. Scientific analysis of the collected aggregate data

For reasons outlined in section 2.3 above, it is necessary that all research
proposals be reviewed ahead of time. Credentialed medical researchers may
submit research proposals explaining the study’s rationale, accompanied by the
software that is to analyze the database. After review of proposal and
software, the software is run against the live database and the results are
returned to the researcher. Research proposal, analysis software and results
are made public to ensure that negative results are not suppressed. The entire
process is free of charge for the researcher.

Toy systems with the same database structure as the live system (but without
real-world data) are provided to researchers, so that their analysis software
can be tested and debugged ahead of time. The use of free software, as
described in section 2.4 above, will hopefully result in a rich ecosystem of
analysis software freely shared among researchers.

Every statistical analysis of the system s database needs to take into account
that the system does not and cannot guarantee that different PatientIDs always
correspond to different patients. Furthermore, conclusions based on analysis
of the database are of course valid only for the subpopulation of patients who
are cared for by participating physicians and who have given consent to
participating in the system; this may not be a representative sample. External
studies comparing this subpopulation to the general patient population will be
highly desirable.

As with all data mining applications, correlations discovered in the data will
have to be confirmed by traditional randomized trials.

###  2.6. Organizational structure and funding

The system will be developed, deployed and run by a non-profit organization,
to be set up in a jurisdiction that grants tax-free status to such
organizations and that has strong privacy protection laws. The organization
will be assisted by a board of external advisors.

To maintain neutrality and to avoid unduly influences on physicians decision
processes, the system’s website will not carry any advertisings. The project
will be financed completely by donations and grants. These may come from
individuals, companies (especially health insurers), charity foundations, or
national or international health organizations. It is possible that an
organization such as the NIH’s National Library of Medicine can be convinced
to host and run the system. An alternative model of funding, especially once
the system is established and accepted, would have the governments of rich
participating countries pay a (small) set amount per patient.

##  3\. Conclusion and Outlook

The proposed system, once fully implemented, will improve world-wide medical
care in a number of important ways:

  * •

practicing physicians, even in developing countries, will receive easy and
free access to a medical expert system that can assist in diagnosis;

  * •

physicians receive promising treatment options, tailored to the patient, based
on past collected outcome data;

  * •

inclusion of price data for treatments and medications allows physicians to
choose the most cost-effective option in any given situation;

  * •

with patients’ consent, medical histories stored in the database are easily
transferrable from one physician to another;

  * •

creative use of the collected aggregate data will allow medical researchers to
identify subpopulations of patients that respond particularly well to a
certain medication or treatment;

  * •

comparisons of new drugs with established generic medications for the same
condition become straightforward.

In the future, the system can be extended to incorporate patients’ genotype
data, thereby representing an important step towards the longstanding goal of
truly personalized medicine.

In addition to these quite concrete and immediate benefits, we would like to
express our hope that widespread adoption of the system will cause the
profession of physician to evolve: from a passive container of knowledge about
symptom-disease correlations and disease-treatment success probabilities to an
active partner of the patient who reassures, explains disorders and
treatments, inquires and provides advice about the patient s life
circumstances and in general maximizes the placebo effect in every way
possible. This, we believe, will ultimately turn out to be one of the main
benefits of the proposed system: the placebo effect, long considered a quirky
nuisance by western medicine, will return to its rightful place at the center
of the healer s work. Traditional systems of medicine will have much to teach
to physicians freed from the more mundane tasks of their profession.

## References

  * [1] G. Ollenschl ger _et al._ Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N). _Quality & safety in health care._ December 2004; 13(6):455-60.
  * [2] The Cochrane Collaboration. http://www.cochrane.org (accessed 27 September 2008).
  * [3] World Health Organization. _The International Statistical Classification of Diseases and Health Related Problems ICD-10, Second Edition_ , 2004
  * [4] WHO Collaborating Centre for Drug Statistics Methodology. _Anatomical Therapeutic Chemical Classification System._ 2008\.
http://www.whocc.no/atcddd/ (accessed 27 September 2008).

  * [5] Hatsek A, Young O, Shalom E, Shahar Y. DeGeL: a clinical-guidelines library and automated guideline-support tools. _Studies in health technology and informatics_ 2008;139:203-12.
  * [6] Judith Federhofer. _Medical Expert Systems. Doctor’s Silent Partners._
http://www.computer.privateweb.at/judith/index.html (accessed 31 December
2006, now only available through http://www.archive.org/).

  * [7] Andres M. St. Laurent. _Understanding Open Source and Free Software Licensing._ O’Reilly Media, 2004.
  * [8] E. Coiera. _The Guide to Health Informatics (2nd Edition)._ Arnold, London, October 2003.
  * [9] David Hand, Heikki Mannila, Padhraic Smyth. _Principles of Data Mining._ MIT Press, 2001