# Magnetic order of the hexagonal rare-earth manganite Dy0.5Y0.5MnO3

Joel S. Helton1,∗ Deepak K. Singh1,2 Harikrishnan S. Nair3 Suja Elizabeth4
1NIST Center for Neutron Research, National Institute of Standards and
Technology, Gaithersburg, MD 20899, USA  2Department of Materials Science and
Engineering, University of Maryland, College Park, MD 20742, USA  3Jülich
Centre for Neutron Science, Forschungszentrum Jülich, Outstation at FRM-II,
D-85747 Garching, Germany  4Department of Physics, Indian Institute of
Science, Bangalore 560012, India

###### Abstract

Hexagonal Dy0.5Y0.5MnO3, a multiferroic rare-earth manganite with
geometrically frustrated antiferromagnetism, has been investigated with
single-crystal neutron diffraction measurements. Below 3.4 K magnetic order is
observed on both the Mn (antiferromagnetic) and Dy (ferrimagnetic) sublattices
that is identical to that of undiluted hexagonal DyMnO3 at low temperature.
The Mn moments undergo a spin reorientation transition between 3.4 K and 10 K,
with antiferromagnetic order of the Mn sublattice persisting up to 70 K; the
antiferromagnetic order in this phase is distinct from that observed in
undiluted (h)DyMnO3, yielding a qualitatively new phase diagram not seen in
other hexagonal rare-earth manganites. A magnetic field applied parallel to
the crystallographic $c$ axis will drive a transition from the
antiferromagnetic phase into the low-temperature ferrimagnetic phase with
little hysteresis.

###### pacs:

75.25.-j, 75.85.+t, 75.50.Ee, 75.47.Lx

##  I INTRODUCTION

The crystalline structure of rare-earth manganites ($R$MnO3 with $R$ = Y, Sc,
or a lanthanide) is determined by the ionic radius of the $R^{3+}$ cation.
Materials with a large $R^{3+}$ ionic radius ($R$ = La through Tb) crystallize
in an orthorhombic perovskite structure, while materials with a smaller
$R^{3+}$ ionic radius ($R$ = Y, Sc, or Ho through Lu)Fröhlich et al. (1999);
A. Muñoz et al. (2000); Tomuta et al. (2001); Sugie et al. (2002); A. Muñoz et
al. (2001); Yen et al. (2005) crystallize in a hexagonal structure. DyMnO3
typically crystallizes in the orthorhombic structure,Prokhnenko et al. (2007)
but with proper growth conditions hexagonal (h)DyMnO3 can be stabilized.V. Yu.
Ivanov et al. (2006); Harikrishnan et al. (2009) The hexagonal rare-earth
manganites, (h)$R$MnO3, are paraelectric at very high temperatures but display
a structural transition ($T_{C}$ $\approx$ 1000 K) to a ferroelectric phase
with the noncentrosymmetric $P6_{3}cm$ space group.Van Aken et al. (2004) The
(h)$R$MnO3 materials feature slightly distorted triangular lattice planes of
Mn ions; the antiferromagnetic nearest-neighbor exchange interaction leads to
geometrically frustrated magnetism. Below a Néel temperature of $\approx$100 K
these materials are magnetically ordered, with easy-plane antiferromagnetic
order of the Mn sublattice coexisting with the ferroelectric order. Many
hexagonal rare-earth manganites display one or more spin reorientation
transitions of the Mn moments at lower temperaturesVajk et al. (2005); Th.
Lonkai et al. (2002) or under an applied field;Lorenz et al. (2005); Vajk et
al. (2006) the $R$ ions also form distorted triangular lattice planes and, for
magnetic ions, will order along the $c$ axis. Several of these materials have
attracted interest because of significant magnetoelectricLottermoser et al.
(2004); Ueland et al. (2010) or magnetoelasticLee et al. (2008) effects.
Despite the structural similarities between the various members of the
(h)$R$MnO3 family, the magnetically ordered structures often differ, with
structures that transform according to each of the four one-dimensional
irreducible representations of the point group observed in at least one
material. The dependence of the magnetic ordering of the Mn sublattice on the
rare-earth element has been attributed to the ionic radius of the $R^{3+}$
cationKozlenko et al. (2007) or a biquadratic 3$d$-4$f$ magnetic
coupling.Wehrenfennig et al. (2010)

Hexagonal (h)DyMnO3, with the largest rare-earth ionic radius of this series,
features an interesting magnetic phase diagram.Harikrishnan et al. (2009);
Nandi et al. (2008a); Wehrenfennig et al. (2010) In the low-temperature phase
(below $\approx$8 K) both the Mn and Dy sublattices are magnetically ordered
according to the $\Gamma_{2}$ irreducible representation. At a temperature of
8 K the Mn moments undergo a spin reorientation transition and are ordered in
the $\Gamma_{4}$ representation up to 68 K.Wehrenfennig et al. (2010) X-ray
resonant magnetic scattering measurements also find a weak Dy moment in this
temperature range, ordered according to the $\Gamma_{3}$ representation.Nandi
et al. (2008a) This incompatible order, with different irreducible
representations present on the two sublattices, calls into question long
standing assumptions about the rigidity of the 3$d$-4$f$ interaction in
(h)$R$MnO3 materials.Wehrenfennig et al. (2010) YMnO3 features magnetism only
on the Mn sublattice and is known to order in the $\Gamma_{3}$ irreducible
representation below $T_{N}$ $\approx$ 75 K.Fiebig et al. (2000) YMnO3 has
also been reported to feature large magnetoelastic effectsLee et al. (2008)
and coupling between electric and magnetic domains.Fiebig et al. (2002)
Compounds where magnetic rare-earth ions are partially replaced with
nonmagnetic Y3+ ions, such as Ho1-xYxMnO3Zhou et al. (2007); Vasic et al.
(2007) and Er1-xYxMnO3,Sekhar et al. (2005); Vajk et al. (2011) allow for
further examination of the role that the rare-earth ions play in determining
the magnetic structure and open up the possibility of novel phase diagrams not
observed in undoped compounds. We report single-crystal neutron diffraction
studies of hexagonal Dy0.5Y0.5MnO3 (DYMO) and find evidence for a spin
reorientation transition not seen in other hexagonal rare-earth manganites at
zero field.

##  II EXPERIMENT

Large, high-quality single-crystal samples of Dy0.5Y0.5MnO3 were prepared as
previously reported.Harikrishnan S. Nair et al. (2011) DYMO crystallizes in
the hexagonal $P6_{3}cm$ space group (#185) with lattice parameters (at 300 K)
of $a$ = $b$ = 6.161(1) Å and $c$ = 11.446(2) Å. As in other hexagonal rare-
earth manganites, Mn3+ ions ($S$ = 2) occupy the $6c$ positions at ($x$, 0, 0)
with $x$ $\approx$ $\frac{1}{3}$. A material with $x$ = $\frac{1}{3}$ would
feature perfect triangular lattice planes; in DYMO $x$ = 0.3379(4) yielding a
slightly distorted triangular lattice. The rare-earth $R^{3+}$ ions occupy two
crystallographically distinct sites, at the $2a$ and 4$b$ Wyckoff positions,
and also form distorted triangular lattice planes. In DYMO the rare-earth
sites are occupied with equal probability by nonmagnetic Y3+ ions and
${}^{6}H_{15/2}$ Dy3+ ions ($gJ$ = 10 $\mu_{B}$), as determined by x-ray
powder diffraction on crushed single crystals.Harikrishnan S. Nair et al.
(2011) Previously reported specific heat measurements found peaks at 3 K and
68 K; analogously with other hexagonal rare-earth manganites such as DyMnO3 it
was suggested that these peaks correspond to the onset of antiferromagnetic
order of the Mn lattice, $T_{N}^{\textrm{Mn}}$ $\approx$ 68 K, and
ferrimagnetic order of Dy moments on the rare-earth lattice,
$T_{N}^{\textrm{Dy}}$ $\approx$ 3 K.

Neutron diffraction experiments were carried out using the BT9 thermal triple-
axis spectrometer at the NIST Center for Neutron Research. The neutron initial
and final energies were selected using the (0 0 2) reflection of the pyrolytic
graphite (PG) monochromator and analyzer. We used 40′-47′-40′-open collimation
as well as PG filters to reduce contamination of the beam with higher-order
neutron wavelengths. The detailed temperature dependence of four magnetic
reflections (shown in Fig. 1) was measured using a large ($\approx$1 g) single
crystal mounted in the ($H$ 0 $L$) scattering plane at a fixed neutron energy
of 30.5 meV ($\lambda$ = 1.64 Å). Refinement of the ordered moments utilized
diffraction measurements taken at temperatures of 1.6 K, 25 K, and 120 K at a
fixed neutron energy of 30.5 meV. In order to minimize absorption of neutrons
by the sample, these measurements were taken with small single crystals: an
8-mg sample mounted in the ($H$ 0 $L$) scattering plane and a 6-mg sample
mounted in the ($H$ $K$ 0) scattering plane. Some magnetic reflections also
featured weak nuclear contributions which were removed by subtracting the 120
K intensity; the Debye-Waller factor has been ignored, which is justifiable,
given that the intensities of nuclear Bragg peaks with no magnetic intensity
[such as (0 0 $L$) where $L$ is even] remained constant within the statistical
uncertainties between 1.6 K and 120 K. The scattering intensity in absolute
units, and from this the value of the ordered moments, was determined by
normalizing the intensities of nuclear Bragg peaks measured at 120 K to the
calculated nuclear intensities. The intensities of the (1 0 0) and (3
$\bar{1}$ 0) reflections in a magnetic field were measured on the 6-mg sample
mounted in the ($H$ $K$ 0) scattering plane; the sample was placed inside a
helium flow dewar with a minimum temperature of 4 K inserted into a 7-T
vertical field superconducting magnet. These data were taken at a fixed
neutron energy of 14.7 meV ($\lambda$ = 2.36 Å). All neutron diffraction data
are reported in terms of the integrated intensity, integrated over a rocking
curve ($\theta$ scan) through the peak position.

The allowed magnetic structures in DYMO correspond to the six irreducible
representations of the $P6_{3}cm$ space group with propagation vector
$\vec{k}$ = 0; as in other hexagonal rare-earth manganites, only the four one-
dimensional irreducible representations (designated as $\Gamma_{1}$ through
$\Gamma_{4}$) are required to describe the observed structures.Sikora and
Syromyatnikov (1986); Munawar and Curnoe (2006) For each of these
representations the Mn sublattice displays antiferromagnetic order within the
$ab$ plane, with the three spins around any triangle oriented 120∘ apart. The
$R^{3+}$ sublattice is ordered along the crystallographic $c$ axis. The rare-
earth 4$b$ sites are antiferromagnetically ordered for the $\Gamma_{1}$,
$\Gamma_{3}$, and $\Gamma_{4}$ representations; the 2$a$ rare-earth sites are
antiferromagnetically ordered in the $\Gamma_{3}$ representation and
paramagnetic for the $\Gamma_{1}$ and $\Gamma_{4}$ representations. The 2$a$
and 4$b$ sites are each ferromagnetically ordered in the $\Gamma_{2}$
representation, with the coupling between the two sites yielding either
ferromagnetism or ferrimagnetism. In phases where only the Mn sublattice is
magnetically ordered a hexagonal rare-earth manganite with $x$ = $\frac{1}{3}$
(featuring undistorted triangular lattice planes) would have perfectly
identical neutron scattering structure factors in either the $\Gamma_{1}$ or
$\Gamma_{3}$ representations, as well as in the $\Gamma_{2}$ or $\Gamma_{4}$
representations. Unambiguous differentiation between these homometric magnetic
structures requires complementary measurement techniques such as optical
second harmonic spectroscopy.Fröhlich et al. (1999); Fiebig et al. (2003,
2000) The distortion of the triangular planes leads to some variation in the
neutron scattering structure factors; however, these results, from single-
crystal diffraction measurements on a strongly absorbing sample, will not be
able to distinguish between the $\Gamma_{1}$ and $\Gamma_{3}$ structures.

##  III RESULTS

###  III.1 Zero-field magnetic order

Figure 1 displays the temperature dependence of the integrated intensities of
the (1 0 0), (1 0 1), (2 0 0), and (2 0 1) Bragg reflections measured while
warming from 2.1 K to 130 K. The background and any high temperature nuclear
contributions to the intensities have been subtracted. At 2.1 K all four
reflections display magnetic intensity, with the (1 0 1) reflection the
strongest. Between 2.1 K and 3.4 K the measured intensities of the ($H$ 0 1)
reflections remain relatively constant while those of the ($H$ 0 0)
reflections steadily decrease with increasing temperature, with an intensity
at 3.4 K that is about 68% of the base temperature intensity. On increasing
the temperature beyond 3.4 K, the intensities of the ($H$ 0 0) peaks steadily
increase while the intensities of the ($H$ 0 1) peaks decrease so that the (1
0 0) reflection becomes the strongest measured; this rapid change in intensity
persists only to around 10 K. This will be shown to correspond to a spin
reorientation transition of the Mn moments and perhaps explains the 10 K
anomaly reportedHarikrishnan S. Nair et al. (2011) in the derivative of
1/$\chi$. Above 10 K the intensities of all peaks slowly decrease until
reaching zero at 70 K. The intensity can be fit to an order parameter form:
$I(T)\,\propto\,(|T-T_{N}|/T_{N})^{2\beta}$. The dotted red line in Fig. 1 is
a fit of the (1 0 0) intensity to this form with $\beta$ = 0.25 $\pm$ 0.05.

Figure 1: (Color online) Integrated magnetic intensities of the (1 0 0), (1 0 1), (2 0 0), and (2 0 1) Bragg reflections as a function of temperature. All data were measured while warming. The inset shows a closer view of the low temperature portion of the graph. Transitions are observed at $T_{N}^{\textrm{Mn}}$ = 70 K and $T_{N}^{\textrm{Dy}}$ = 3.4 K; these temperatures are designated with dashed vertical lines. The red dotted line reflects a fit of the (1 0 0) intensity to an order parameter form with $\beta$ = 0.25 $\pm$ 0.05. Error bars throughout this article are statistical in nature and represent one standard deviation. Table 1: Calculated magnetic scattering intensities for the reflections displayed in Fig. 1. The most intense reflection in each column has been normalized to 100. In the low-temperature phase below 3.4 K the data are consistent with $\Gamma_{2}$ order for both the Mn and Dy sublattice. In the antiferromagnetic phase between 10 K and 70 K the data are consistent with Mn sublattice order in either the $\Gamma_{1}$ or $\Gamma_{3}$ irreducible representation. | Mn order |  | Dy order
---|---|---|---
|  |  $\Gamma_{1}$ |  $\Gamma_{2}$ |  $\Gamma_{3}$ |  $\Gamma_{4}$ |  |  $\Gamma_{2}$
(1 0 0) |  | 100 | 0 | 100 | 0 |  | 100
(1 0 1) |  | 17 | 100 | 15 | 100 |  | 0
(2 0 0) |  | 32 | 0 | 27 | 0 |  | 38
(2 0 1) |  | 1 | 26 | 1 | 33 |  | 0

Table 1 displays the calculated magnetic intensities for the reflections shown
in Fig. 1 with the largest reflection in each column normalized to 100. The
reported intensities reflect the calculated magnetic cross section corrected
for the resolution function of a triple-axis neutron spectrometer. Calculated
intensities are shown for Mn moments ordered in each of the four one-
dimensional irreducible representations, as well as for Dy moments ordered in
the $\Gamma_{2}$ representation. Previous magnetization measurements on
DYMOHarikrishnan S. Nair et al. (2011) with $\vec{H}$ $||$ $\vec{c}$ revealed
a low-temperature state with a spontaneous magnetization of $\approx$0.5
$\mu_{B}$ per formula unit (measured at 2 K). Of the four irreducible
representations present in hexagonal rare-earth manganites only the
$\Gamma_{2}$ structure is consistent with either ferrimagnetism or
ferromagnetism; undoped hexagonal DyMnO3 is ferrimagnetic at low temperatures
and is known to order in the $\Gamma_{2}$ representation.Nandi et al. (2008a)
When ordering in the $\Gamma_{2}$ structure the magnetic cross sections of the
($H$ 0 1) reflections depend only on the ordered moment of the Mn ions while
those of the ($H$ 0 0) reflections depend only on the ordered moment of the Dy
ions. The Mn ordered moment is therefore found to be almost temperature
independent below 3.4 K, while the Dy ordered moment decreases continuously
with increasing temperature between 2.1 K and 3.4 K.

Assuming a ferrimagnetic structure with ordered moments on the 2$a$ and 4$b$
sites that are antiparallel but equal in magnitude, a refinement of magnetic
reflections measured at 1.6 K reveals ordered moments of 3.7 $\pm$ 0.4
$\mu_{B}$ for the Mn ions and 3.1 $\pm$ 0.3 $\mu_{B}$ for the Dy ions (1.6
$\pm$ 0.2 $\mu_{B}$ for each rare-earth site). This structure is displayed in
Fig. 2(a). (The crystal structures shown in Figure 2 were produced using
_VESTA_.Momma and Izumi (2008)) When magnetic domains are fully aligned this
ordered moment will lead to a net magnetization of 3.1 $\pm$ 0.3 $\mu_{B}$ per
unit cell (or 0.52 $\pm$ 0.05 $\mu_{B}$ per formula unit) along the
crystallographic $c$ axis, which is consistent with the reported bulk
spontaneous magnetization.Harikrishnan S. Nair et al. (2011) Allowing for
different ordered moments on the Dy $4b$ and $2a$ sites did not appreciably
improve the fit of the 1.6 K data, nor could allowing for different moments on
these sites produce a comparably good fit while yielding a net moment
consistent with magnetization measurements. A ferromagnetic ground state can
likewise be excluded. A ferromagnetic state with equal ordered moments on the
2$a$ and 4$b$ sites would give no intensity for the ($H$ 0 0) reflections;
when ordering in the $\Gamma_{2}$ structure the intensities of these
reflections depend on the difference in moment at the rare-earth sites:
$I(\vec{Q}_{H00})~{}\propto~{}|\vec{M}_{4b}-\vec{M}_{2a}|^{2}$, where the
ordered moments on the 2$a$ and 4$b$ sites are $\vec{M}_{2a}$ and
$\vec{M}_{4b}$. A ferromagnetic state with different ordered moments on the
2$a$ and 4$b$ sites is consistent with the neutron diffraction data only while
yielding a net moment inconsistent with magnetization measurements. The Mn
ordered moment at 1.6 K is close to the full moment value; however, the Dy
ordered moment is considerably reduced from the full 10 $\mu_{B}$ value of the
${}^{6}H_{15/2}$ Dy3+ ions. The low temperature magnetization in undoped
DyMnO3 would suggest a similarly reduced ordered moment for the Dy ions in the
ferrimagnetic phase.V. Yu. Ivanov et al. (2006)

Figure 2: (Color online) (a) Magnetic structure in the low-temperature
ferrimagnetic phase, present below $T_{N}^{\textrm{Dy}}$ = 3.4 K. The
structure belongs to the $P6_{3}c^{\prime}m^{\prime}$ magnetic space group
($\Gamma_{2}$ irreducible representation). Mn ions at $z$ = 0 are shown in
purple, Mn ions at $z$ = 1/2 are shown in orange, and the rare-earth ions are
shown in blue. The refined ordered moments are 3.7 $\pm$ 0.4 $\mu_{B}$ on the
Mn ions and 3.1 $\pm$ 0.3 $\mu_{B}$ on the Dy ions. (b) The two possible
magnetic structures for the antiferromagnetic phase: $P6_{3}cm$ magnetic space
group ($\Gamma_{1}$ irreducible representation) on the left and
$P6^{\prime}_{3}cm^{\prime}$ magnetic space group ($\Gamma_{3}$) on the right.
The Mn ordered moment is 3.5 $\pm$ 0.4 $\mu_{B}$. Any Dy ordered moment in
this temperature range is too small to be measured by neutron diffraction. The
structural unit cell is outlined in gray

The magnetic structure in the antiferromagnetic phase (10 K $<$ $T$ $<$ 70 K)
is consistent with an ordered moment on only the Mn sublattice, with order
according to either the $\Gamma_{1}$ or $\Gamma_{3}$ irreducible
representation. These magnetic structures are shown in Fig. 2(b). It should be
noted that neither of these possibilities are consistent with the structure of
the Mn moments in (h)DyMnO3, which order in the $\Gamma_{4}$ representation
above the low-temperature ferrimagnetic phase.Wehrenfennig et al. (2010) A
refinement of magnetic reflections measured at 25 K gives an ordered moment of
3.5 $\pm$ 0.4 $\mu_{B}$ for the Mn sublattice. The ordered moment at 25 K is
comparable in size to that determined at 1.6 K, suggesting a spin
reorientation transition where Mn spins rotate between 3.4 K and 10 K with
little change in magnitude. Element specific x-ray resonant magnetic
scattering measurements on hexagonal DyMnO3Nandi et al. (2008a) and
HoMnO3Nandi et al. (2008b) have reported a weak rare-earth moment at
comparable temperatures, induced by a splitting of the ground-state crystal
field doublet. While DYMO might similarly feature a weak Dy ordered moment in
this temperature range, neutron scattering measurements will be sensitive to
only the much larger ordered moment of the Mn sublattice and the refinement is
not improved by allowing for an ordered moment on the Dy sublattice at 25 K.

The magnetic structure of YMnO3 is known to be the $\Gamma_{3}$
representation.Fiebig et al. (2000) If the antiferromagnetic phase of DYMO
were likewise $\Gamma_{3}$, the spin reorientation transition into the low-
temperature $\Gamma_{2}$ phase would differ from the spin reorientation
transitions previously observed in hexagonal rare-earth manganites. A magnetic
structure in the $\Gamma_{1}$ irreducible representation can be transformed
into the $\Gamma_{2}$ representation through a 90∘ counterclockwise rotation
of all Mn moments. However, a transformation of a structure in the
$\Gamma_{3}$ representation as shown in Fig. 2(b) into the $\Gamma_{2}$
representation would require a 180∘ rotation of only the Mn moments in the $z$
= $\frac{1}{2}$ plane. In other hexagonal rare-earth manganites, it has been
suggestedFiebig et al. (2003) that spin reorientation transitions between the
four irreducible representations occur via in-phase or antiphase rotations
where the moments in adjacent Mn layers rotate with an equal (in-phase) or
opposite (antiphase) direction. For example, the Mn moments in HoMnO3 display
an in-phase reorientation transitionVajk et al. (2005) ($\Gamma_{4}$ to
$\Gamma_{3}$) at $T$ $\approx$ 40 K and an antiphase transition ($\Gamma_{3}$
to $\Gamma_{1}$) at $T$ $\approx$ 8 K, while the Mn moments in (h)DyMnO3
display an antiphase rotationWehrenfennig et al. (2010) ($\Gamma_{4}$ to
$\Gamma_{2}$) at $T$ $\approx$ 8 K. Further, while materials such as ScMnO3,A.
Muñoz et al. (2000) ErxY1-xMnO3,Sekhar et al. (2005) and YMnO3 under
pressureKozlenko et al. (2007) have been reported to feature broad temperature
ranges where the magnetic structure is not one of the four irreducible
representations, the observed structures can still be described as an in-phase
or antiphase rotation of all Mn moments away from one of the principal
structures.

###  III.2 Field dependence of the magnetic order

Below $T_{N}^{\textrm{Dy}}$ = 3.4 K DYMO is ferrimagnetic with a spontaneous
net magnetization. Above $T_{N}^{\textrm{Dy}}$, $M(H)$ curves display
symmetric magnetization steps at a critical field value that increases with
temperature; the size of these steps decreases with increasing temperature,
becoming immeasurably small around 40 K.Harikrishnan S. Nair et al. (2011)
This behavior is remarkably similar to that previously reported in
(h)DyMnO3,V. Yu. Ivanov et al. (2006) but the moment increase associated with
the magnetization steps in DYMO is about one half of the increase in
(h)DyMnO3. Figure 3 displays the intensity of the (1 0 0) magnetic Bragg
reflection as a function of field with $\vec{H}$ $||$ $\vec{c}$ at
temperatures between 4 K and 50 K, measured while increasing the field after
zero field cooling. For each of these temperatures, we find a sudden drop in
the (1 0 0) intensity at a critical field value that increases with
temperature. As was shown in Fig. 1, the intensity of the (1 0 0) reflection
will be much higher in the antiferromagnetic phase than in the low-temperature
phase, such that this behavior is easily associated with a field-induced
transition into the ferrimagnetic phase. The data definitively show a spin
reorientation of the Mn moments along with the field-induced ferrimagnetic
ordering of the Dy moments, as the intensity of this reflection would not drop
if the Mn moments remained ordered in the original structure. The intensity of
this reflection in the field-induced phase is weaker than in zero field at low
temperature [the dotted horizontal line represents the expected intensity of
the (1 0 0) reflection at 2 K and zero field]. This likely arises from a
considerably smaller Dy ordered moment, consistent with the smaller magnitude
of the magnetization steps at higher temperatures in the $M(H)$ curves.
Interestingly, neutron diffraction is sensitive enough that this transition is
still clearly measurable in the 50 K data, while it could not be measured
above 40 K in the magnetization data.Harikrishnan S. Nair et al. (2011)

Figure 3: (Color online) Integrated intensity of the (1 0 0) peak as a
function of field. The data were taken while increasing field. Lines are a
guide to the eye. The dotted horizonal line represents the expected intensity
at 2 K in zero field.

The field dependence of the intensities at the (1 0 0) and (3 $\bar{1}$ 0)
reflections at $T$ = 15 K is displayed in Fig. 4. In Fig. 4(a), both
reflections show the expected transition at $\mu_{0}H_{c}$ $\approx$ 1 T; data
taken in rising and falling field show only a slight hysteresis, consistent
with the small hysteresis of the magnetization steps in the bulk magnetization
data. In addition to DYMO, undoped (h)DyMnO3 displays little hysteresis while
magnetization steps in ErMnO3Fiebig et al. (2001) display considerable
hysteresis. The (3 $\bar{1}$ 0) reflection displays scattering in the low-
temperature $\Gamma_{2}$ phase and very little scattering in the
antiferromagnetic phase, such that this change in intensity is likewise
consistent with a field-induced transition between the two. A more detailed
view of the (1 0 0) intensity in the high-field region is shown in Fig. 4(b).
Interestingly, the intensity of this peak rises slowly with increasing field
up to about 4 T but then decreases with increasing field beyond 5 T. This
likely reflects the evolution of the system from ferrimagnetic to
ferromagnetic as the field is increased, as the fully polarized state
(ferromagnetic with an equal moment for the 4$a$ and 2$b$ sites) would display
no scattering at (1 0 0).

Figure 4: (Color online) (a) Integrated intensities of the (1 0 0) (left
scale) and (3 $\bar{1}$ 0) (right scale) peaks measured at $T$ = 15 K. Solid
symbols represent measurements taken with increasing field, and open symbols
represent those taken with a decreasing field. (b) Detail of the high field
region for the (1 0 0) intensity.

##  IV SUMMARY

Below $T_{N}^{\textrm{Dy}}$ = 3.4 K Dy0.5Y0.5MnO3 is magnetically ordered in
the $\Gamma_{2}$ irreducible representation with antiferromagnetic Mn order
and ferrimagnetic Dy order. Between 3.4 K and 10 K we observe a spin
reorientation transition of the Mn moments. Antiferromagnetic order of the Mn
sites persists up to $T_{N}^{\textrm{Mn}}$ = 70 K. The magnetic structure of
this phase is either $\Gamma_{1}$ or $\Gamma_{3}$, inconsistent with the
$\Gamma_{4}$ order observed in undiluted (h)DyMnO3.Wehrenfennig et al. (2010)
In the antiferromagnetic phase, a magnetic field applied parallel to the $c$
axis will drive a field-induced transition into the low-temperature
ferrimagnetic phase with little hysteresis. Further knowledge of the different
magnetic structures and reorientation transitions displayed by the Mn moments
in various (h)$R$MnO3 materials should lead to improved understanding of the
unusual 3$d$-4$f$ magnetic coupling present in these materials.Wehrenfennig et
al. (2010) While the magnetic structure of the antiferromagnetic phase cannot
be definitively determined from this experiment, it is clear that
Dy0.5Y0.5MnO3 displays a zero-field spin reorientation transition not
previously observed in any other hexagonal rare-earth manganite. If the
magnetic structure is $\Gamma_{3}$, then the $\Gamma_{3}$ to $\Gamma_{2}$ spin
reorientation transition would involve only half of the Mn ions in a manner
not observed in other (h)$R$MnO3 materials. The $H$-$T$ phase diagram of
HoMnO3 features a boundary between $\Gamma_{1}$ and $\Gamma_{2}$ phasesLorenz
et al. (2005); Munawar and Curnoe (2006) at $\mu_{0}H$ $\approx$ 2 T, yet a
zero field $\Gamma_{1}$ to $\Gamma_{2}$ spin reorientation transition has not
been reported. Further work will be necessary to distinguish between the
$\Gamma_{1}$ and $\Gamma_{3}$ structures in the antiferromagnetic phase and to
determine if the domain walls arising from this unique spin reorientation will
display magnetoelectric coupling as is observed in compounds such as
HoMnO3Ueland et al. (2010); Lottermoser and Fiebig (2004) or YMnO3.Fiebig et
al. (2002)

## ACKNOWLEDGEMENTS

We thank J.W. Lynn for guidance and helpful discussions. J.S.H. acknowledges
support from the NRC/NIST Postdoctoral Associateship Program. This work was
supported in part by the National Science Foundation under Agreement No.
DMR-0944772.

* email<EMAIL_ADDRESS>
## References

  * Fröhlich et al. (1999) D. Fröhlich, St. Leute, V. V. Pavlov, R. V. Pisarev, and K. Kohn, J. Appl. Phys. 85, 4762 (1999).
  * A. Muñoz et al. (2000) A. Muñoz, J. A. Alonso, M. J. Martínez-Lope, M. T. Casáis, J. L. Martínez, and M. T. Fernández-Díaz, Phys. Rev. B 62, 9498 (2000).
  * Tomuta et al. (2001) D. G. Tomuta, S. Ramakrishnan, G. J. Nieuwenhuys, and J. A. Mydosh, J. Phys.: Condens. Matter 13, 4543 (2001).
  * Sugie et al. (2002) H. Sugie, N. Iwata, and K. Kohn, J. Phys. Soc. Jpn. 71, 1558 (2002).
  * A. Muñoz et al. (2001) A. Muñoz, J. A. Alonso, M. J. Martínez-Lope, M. T. Casáis, J. L. Martínez, and M. T. Fernández-Díaz, Chem. Mater. 13, 1497 (2001).
  * Yen et al. (2005) F. Yen, C. R. dela Cruz, B. Lorenz, Y. Y. Sun, Y. Q. Wang, M. M. Gospodinov, and C. W. Chu, Phys. Rev. B 71, 180407(R) (2005).
  * Prokhnenko et al. (2007) O. Prokhnenko, R. Feyerherm, E. Dudzik, S. Landsgesell, N. Aliouane, L. C. Chapon, and D. N. Argyriou, Phys. Rev. Lett. 98, 057206 (2007).
  * V. Yu. Ivanov et al. (2006) V. Yu. Ivanov, A. A. Mukhin, A. S. Prokhorov, A. M. Balbashov, and L. D. Iskhakova, Phys. Solid State 48, 1726 (2006).
  * Harikrishnan et al. (2009) S. Harikrishnan, S. Rößler, C. M. Naveen Kumar, H. L. Bhat, U. K. Rößler, S. Wirth, F. Steglich, and Suja Elizabeth, J. Phys.: Condens. Matter 21, 096002 (2009).
  * Van Aken et al. (2004) B. B. Van Aken, T. T. M. Palstra, A. Filippetti, and N. A. Spaldin, Nature Mater. 3, 164 (2004).
  * Vajk et al. (2005) O. P. Vajk, M. Kenzelmann, J. W. Lynn, S. B. Kim, and S.-W. Cheong, Phys. Rev. Lett. 94, 087601 (2005).
  * Th. Lonkai et al. (2002) Th. Lonkai, D. Wohlwein, J. Ihringer, and W. Prandl, Appl. Phys. A 74, S843 (2002).
  * Lorenz et al. (2005) B. Lorenz, F. Yen, M. M. Gospodinov, and C. W. Chu, Phys. Rev. B 71, 014438 (2005).
  * Vajk et al. (2006) O. P. Vajk, M. Kenzelmann, J. W. Lynn, S. B. Kim, and S.-W. Cheong, J. Appl. Phys. 99, 08E301 (2006).
  * Lottermoser et al. (2004) T. Lottermoser, T. Lonkai, U. Amann, D. Hohlwein, J. Ihringer, and M. Fiebig, Nature 430, 541 (2004).
  * Ueland et al. (2010) B. G. Ueland, J. W. Lynn, M. Laver, Y. J. Choi, and S.-W. Cheong, Phys. Rev. Lett. 104, 147204 (2010).
  * Lee et al. (2008) S. Lee, A. Pirogov, M. Kang, K.-H. Jang, M. Yonemura, T. Kamiyama, S.-W. Cheong, F. Gozzo, N. Shin, H. Kimura, et al., Nature 451, 805 (2008).
  * Kozlenko et al. (2007) D. P. Kozlenko, S. Kichanov, S. Lee, J.-G. Park, and B. N. Savenko, J. Phys.: Condens. Matter 19, 156228 (2007).
  * Wehrenfennig et al. (2010) C. Wehrenfennig, D. Meier, Th. Lottermoser, Th. Lonkai, J.-U. Hoffmann, N. Aliouane, D. N. Argyriou, and M. Fiebig, Phys. Rev. B 82, 100414(R) (2010).
  * Nandi et al. (2008a) S. Nandi, A. Kreyssig, J. Q. Yan, M. D. Vannette, J. C. Lang, L. Tan, J. W. Kim, R. Prozorov, T. A. Lograsso, R. J. McQueeney, et al., Phys. Rev. B 78, 075118 (2008a).
  * Fiebig et al. (2000) M. Fiebig, D. Fröhlich, K. Kohn, St. Leute, Th. Lottermoser, V. V. Pavlov, and R. V. Pisarev, Phys. Rev. Lett. 84, 5620 (2000).
  * Fiebig et al. (2002) M. Fiebig, Th. Lottermoser, D. Fröhlich, A. V. Goltsev, and R. V. Pisarev, Nature 419, 818 (2002).
  * Zhou et al. (2007) H. D. Zhou, J. Lu, R. Vasic, B. W. Vogt, J. A. Janik, J. S. Brooks, and C. R. Wiebe, Phys. Rev. B 75, 132406 (2007).
  * Vasic et al. (2007) R. Vasic, H. D. Zhou, E. Jobiliong, C. R. Wiebe, and J. S. Brooks, Phys. Rev. B 75, 014436 (2007).
  * Sekhar et al. (2005) M. C. Sekhar, S. Lee, G. Choi, C. Lee, and J.-G. Park, Phys. Rev. B 72, 014402 (2005).
  * Vajk et al. (2011) O. P. Vajk, J. Gunasekera, Y. Wang, and T. Heitmann, J. Appl. Phys. 109, 07D910 (2011).
  * Harikrishnan S. Nair et al. (2011) Harikrishnan S. Nair, C. M. N. Kumar, H. L. Bhat, Suja Elizabeth, and Th. Brückel, Phys. Rev. B 83, 104424 (2011).
  * Sikora and Syromyatnikov (1986) W. Sikora and V. N. Syromyatnikov, J. Magn. Magn. Mat. 60, 199 (1986).
  * Munawar and Curnoe (2006) I. Munawar and S. H. Curnoe, J. Phys.: Condens. Matter 18, 9575 (2006).
  * Fiebig et al. (2003) M. Fiebig, Th. Lottermoser, and R. V. Pisarev, J. Appl. Phys. 93, 8194 (2003).
  * Momma and Izumi (2008) K. Momma and F. Izumi, J. Appl. Crystallog. 41, 653 (2008).
  * Nandi et al. (2008b) S. Nandi, A. Kreyssig, L. Tan, J. W. Kim, J. Q. Yan, J. C. Lang, D. Haskel, R. J. McQueeney, and A. I. Goldman, Phys. Rev. Lett. 100, 217201 (2008b).
  * Fiebig et al. (2001) M. Fiebig, C. Degenhardt, and R. V. Pisarev, Phys. Rev. Lett. 88, 027203 (2001).
  * Lottermoser and Fiebig (2004) T. Lottermoser and M. Fiebig, Phys. Rev. B 70, 220407(R) (2004).


# Classification of Human- and AI-Generated Texts
for English, French, German, and Spanish

Kristina Schaaff  Tim Schlippe  Lorenz Mindner
IU International University of Applied Sciences, Germany.
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper we analyze features to classify human- and AI-generated text for
English, French, German and Spanish and compare them across languages. We
investigate two scenarios: (1) The detection of text generated by AI from
scratch, and (2) the detection of text rephrased by AI. For training and
testing the classifiers in this multilingual setting, we created a new text
corpus covering 10 topics for each language. For the detection of AI-generated
text, the combination of all proposed features performs best, indicating that
our features are portable to other related languages: The F1-scores are close
with 99% for Spanish, 98% for English, 97% for German and 95% for French. For
the detection of AI-rephrased text, the systems with all features outperform
systems with other features in many cases, but using only document features
performs best for German (72%) and Spanish (86%) and only text vector features
leads to best results for English (78%).

##  1 Introduction

In recent years, chatbots have gained popularity and are now widely used in
everyday life Pelau et al. (2021). These systems are designed to simulate
human-like conversations and provide assistance, information, and emotional
support Dibitonto et al. (2018); Arteaga et al. (2019); Falala-Séchet et al.
(2019); Adiwardana et al. (2020). OpenAI’s ChatGPT has emerged as one of the
most commonly used tool for text generation Taecharungroj (2023). Within a
short span of only five days after its release, over one million users
registered Taecharungroj (2023). The application scenarios are manifold,
ranging from children seeking help with their homework to individuals seeking
medical advice or companionship.

As the use of chatbots like ChatGPT becomes more prevalent in our daily lives,
it is important to differentiate between human-generated and AI-generated
text. As AI algorithms improve, detecting AI-generated content accurately
becomes increasingly challenging, posing issues such as plagiarism, fake news
generation, and spamming. Thus, tools that can differentiate between human-
and AI-generated content are crucial.

In Mindner et al. (2023), we explored a large number of innovative features
such as text objectivity, list lookup features, and error-based features for
the detection of English (EN) text generated by ChatGPT. However, in the
current study, we extended this research to Spanish (ES), German (DE), and
French (FR). We selected these languages, as these are amongst the most
frequently used languages in the world Ethnologue (2023).

Consequently, our contributions are as follows:

  * •

We proved, that the features we investigated in Mindner et al. (2023) can be
successfully ported to other languages.

  * •

We extended our Human-AI-Generated Text
Corpus111https://github.com/LorenzM97/human-AI-generatedTextCorpus with FR, DE
and ES articles which cover 10 topics, providing a benchmark corpus for the
detection of AI-generated texts in EN, FR, DE and ES.

  * •

Our best systems significantly outperform the state-of-the-art system for the
detection of AI-generated text ZeroGPT.

##  2 Related Work

In the this section, we will describe the related work concerning ChatGPT and
the classification of human- and AI-generated texts.

###  2.1 ChatGPT

Since its release by OpenAI in late 2022, ChatGPT has revolutionized the field
of AI Mesko (2023) and several other generative AIs such as Google’s
Bard222https://bard.google.com or Llama333https://ai.meta.com/llama Touvron et
al. (2023) have been released. Those tools are capable of generating text in
response to user queries across a wide range of domains. Its successful
implementation has been demonstrated in areas like education Baidoo-Anu and
Owusu Ansah (2023), medicine Jeblick et al. (2022), and language translation
Jiao et al. (2023). ChatGPT is built on the Generative Pre-trained
Transformers (GPT) language model and undergoes fine-tuning using
reinforcement learning with human feedback. This approach allows ChatGPT to
grasp the meaning and intention behind user prompts, enabling it to provide
relevant and helpful responses. During the training process, a substantial
amount of text data is incorporated to ensure the safety and accuracy of the
generated text. While the quantity of training data has not been published, we
know that the previous GPT-3 model, which is substantially larger than other
language models such as BERT Kenton and Toutanova (2019), RoBERTa Liu et al.
(2019), and T5 Roberts et al. (2019), was trained with 175 billion parameters
and 499 billion crawled text tokens Brown et al. (2020). Through extensive
training on a diverse dataset, ChatGPT has acquired a sophisticated
understanding of human language, allowing it to generate text that closely
resembles that written by humans Mitrović et al. (2023).

| Human | AI-generated | AI-rephrased
---|---|---|---
Language | P | S | W | P | S | W | P | S | W
EN | 415 | 1.7k | 38.3k | 555 | 1.4k | 27.6k | 255 | 1.1k | 24.6k
FR | 415 | 1.2k | 31.0k | 524 | 1.3k | 26.5k | 157 | 0.8k | 18.7k
DE | 335 | 1.2k | 20.5k | 529 | 1.4k | 22.9k | 256 | 1.0k | 16.4k
ES | 450 | 1.4k | 38.0k | 514 | 1.2k | 26.8k | 190 | 0.8k | 18.9k
Table 1: AI-Generated/Rephrased Text
(P = #paragraphs, S = #sentences, W = #words).

###  2.2 Detecting Human- and AI-Generated Texts

Commercial tools and plagiarism apps, such as GPTZero Shrivastava (2023),
ZeroGPT444https://www.zerogpt.com, AI Content
Detector555https://copyleaks.com/ai-content-detector, and GPT-2 Output
Detector666https://openai-openai-detector–mqlck.hf.space Mitchell et al.
(2023), have been developed to identify AI-generated text. Furthermore,
researchers are working on developing new corpora for this task and finding
out which features and classifiers improve classification accuracy: For
example, Yu et al. (2023) present a corpus of human- and AI-generated
abstracts to investigate commercial and non-commercial systems—but only for
EN. Recent studies have explored various approaches to detect AI-generated
text, including XGBoost Shijaku and Canhasi (2023), decision trees Zaitsu and
Jin (2023), and transformer-based models Mitrović et al. (2023); Guo et al.
(2023): Mitrović et al. (2023) evaluated characteristics of AI-generated text
from EN customer reviews and built a transformer-based classifier that
achieved 79%. Zaitsu and Jin (2023) achieved 100% accuracy in the detection of
Japanese texts with decision trees combining stylometric features for Japanese
such as bigrams, comma position, and function word rates. Guo et al. (2023)
evaluated the characteristics of human-generated and AI-generated answers to
questions in EN and Chinese. They fine-tuned a RoBERTa model on their texts
and achieved 98.8% F1-score on the EN answers and 96.4% F1-score on the
Chinese answers. Shijaku and Canhasi (2023) addressed the detection of
generated essays written in EN and proposed an XGBoost model that achieved 98%
accuracy using features generated by TF-IDF and a set of hand-crafted
features. Soni and Wade (2023) analyzed human- and AI-generated text
summarization and achieved 90% accuracy using
DistilBERT777https://huggingface.co/docs/transformers/model_doc/distilbert
Sanh et al. (2019). Mindner et al. (2023) explored features to detect AI-
generated and -rephrased text for EN. They report an F1-score of 96% for AI-
generated text and 78% for AI-rephrased text on their text corpus which
contains different topics. These F1-scores were even achieved when the AI was
instructed to create the text in a way that a human would not recognize that
it was generated by an AI.

To the best of our knowledge, we are the first to explore a large set of
features and state-of-the-art classifiers across multiple languages with
XGBoost, Random Forrest and MLP. We compare our results with two popular
state-of-the-art tools that detect texts generated by AI: GPTZero and ZeroGPT.
GPTZero is used by over 1 million people Shrivastava (2023), but its results
are only reliable for EN texts. Consequently, we also used ZeroGPT for
comparison which is able to deal with other languages. As there is currently
no text corpus available, which contains human- and AI-generated texts in
multiple languages, we extended our Human-AI-Generated Text Corpus to cover
EN, FR, DE and ES.

##  3 Our Human-AI-Generated Text Corpus

As mentioned in the previous section, we extended our Human-AI-Generated Text
Corpus Mindner et al. (2023) to cover EN, FR, DE, and ES. In total, for each
language we used 100 human-generated, 100 AI-generated, and 100 AI-rephrased
articles for our multilingual analysis which contain the following 10 topics:
$biology$, $chemistry$, $geography$, $history$, $IT$, $music$, $politics$,
$religion$, $sports$, and $visualarts$.

The characteristics of our Human-AI-Generated Text Corpus for the respective
languages are summarized in Table 1: EN consistently has the highest counts
across all categories and types of text. On the other hand, the counts for FR,
DE, and ES vary substantially depending on whether the text was human-
generated, AI-generated, or AI-rephrased. This illustrates how languages
differ in the expression of information. The prompts which we used to receive
the AI-generated and AI-rephrased texts are listed in Table 2.

Lang. |  Prompt
---|---
Text Generation
EN |  Generate a text on the following topic: <topic>
FR |  Rédigez un texte sur le thème suivant: <topic>
DE |  Erstelle einen Text zum folgenden Thema: <topic>
ES |  Genera un texto sobre el siguiente tema: <topic>
Text Rephrasing
EN |  Rephrase the following text: <topic>
FR |  Reformulez le texte suivant: <topic>
DE |  Formuliere den folgenden Text um: <topic>
ES |  Reformule el siguiente texto: <topic>
Table 2: Prompts used for Generation and Rephrasing

##  4 Our Features for the Classification of Human- and AI-Generated Texts

As shown in Table 3, we analyzed 37 features for their suitability to
discriminate between human- and AI-generated text. More details of the
features are given in Mindner et al. (2023).

Category | Feature | Description
---|---|---
Perplexity | $PPL_{mean}$ | mean PPL
| $PPL_{max}$ | maximum PPL
Semantic | $sentiment_{polarity}$ | degree of positivity/negativity [-1,+1]
| $sentiment_{subjectivity}$ | degree of subjectivity [0,+1]
ListLookup | $stopWord_{count}$ | number of stop words
| $discourseMarker_{count}$ | number of discourse markers
| $titleRepetition_{count}$ | absolute repetitions of title
| $titleRepetition_{relative}$ | relative repetitions of title
| $personalPronoun_{count}$ | absolute number of personal pronouns
| $personalPronoun_{relative}$ | relative number of personal pronouns
Document | $wordsPerParagraph_{mean}$ | mean number of words per paragraph
| $wordsPerParagraph_{stdev}$ | stdev of $wordsPerParagraph$
| $sentencesPerParagraph_{mean}$ | mean number of sentences per paragraph
| $sentencesPerParagraph_{stdev}$ | stdev of $sentencesPerParagraph$
| $wordsPerSentence_{mean}$ | mean number of words per sentence
| $wordsPerSentence_{stdev}$ | stdev of $wordsPerSentence$
| $uniqWordsPerSentence_{mean}$ | mean number of unique words per sentence
| $uniqWordsPerSentence_{stdev}$ | stdev of $uniqWordsPerSentence$
| $words_{count}$ | number of running words
| $uniqWords_{count}$ | number of unique words
| $uniqWords_{relative}$ | relative number of unique words
| $paragraph_{count}$ | number of paragraphs
| $sentence_{count}$ | number of sentences
| $punctuation_{count}$ | number of punctuation marks
| $quotation_{count}$ | number of quotation marks
| $character_{count}$ | number of characters
| $uppercaseWords_{relative}$ | relative number of words in uppercase
| $POSPerSentence_{mean}$ | mean number of unique POS-tags/sentence
| $specialChar_{count}$ | number of special characters
ErrorBased | $grammarError_{count}$ | number of spelling/grammar errors
| $multiBlank_{count}$ | number of multiple blanks
Readability | $fleschReadingEase$ | Flesch Reading Ease score [0-100]
| $fleschKincaidGradeLevel$ | Readability as U.S. grade level [0-100]
AIFeedback | $AIFeedback$ | Ask AI if text was generated by AI
TextVector | TF-IDF | 500-dim TF-IDF vector of 1-/2-grams
| Sentence-BERT | mean Sentence-BERT vector
| Sentence-BERT-dist | mean distance of Sentence-BERT vectors
Table 3: Summary of our Features for the Classification of Generated Texts.

###  4.1 Perplexity-Based Features

Perplexity is a measure of how well a language model is able to predict a
sequence of words. The lower the perplexity, the better a language model will
perform to predict the next word in a sequence. As AI-generated texts are
usually based on statistical patterns and rules, they tend to be more
repetitive and therefore have a lower perplexity than human generated texts.
The perplexity-based features in our study are based on the findings by
Mindner et al. (2023); Gehrmann et al. (2019); Mitrović et al. (2023); Guo et
al. (2023).

For sentence tokenization, we use the Natural Language Toolkit
(NLTK)888https://www.nltk.org. Perplexity is calculated using evaluate
package999https://github.com/huggingface/evaluate and GPT-2 using the
respective models for EN101010https://huggingface.co/gpt2,
FR111111https://huggingface.co/dbddv01/gpt2-french-small,
DE121212https://huggingface.co/dbmdz/german-gpt2, and
ES131313https://huggingface.co/DeepESP/gpt2-spanish.

###  4.2 Semantic Features

In our study, semantic features refer to the properties of words or phrases
used to represent their meanings. Previous studies successfully used these
features for the differentiation between human- and AI-generated texts
Mitrović et al. (2023); Guo et al. (2023); Mindner et al. (2023).

Again, we use different Python packages for the respective languages:
TextBlob’s sentiment analysis for
English141414https://textblob.readthedocs.io/en/dev/quickstart.html, textblob-
fr151515https://github.com/sloria/textblob-fr for French, and textblob-
de161616https://textblob-de.readthedocs.io/en/latest/api_reference
.html#module-textblob_de.sentiments for German. Due to the absence of a
package that computes both, polarity and subjectivity, for ES texts were
translated these texts into EN using
Googletrans171717https://github.com/ssut/py-googletrans, despite potential
information loss, because of its high BLEU score and proficiency in ES-EN
translation.

###  4.3 List Lookup Features

With our ListLookup features, we analyze information about the word or
character class, e.g., whether it is a stop word or a special character. These
features have previously been used for this task by Mindner et al. (2023);
Shijaku and Canhasi (2023); Kumarage et al. (2023). For every language, we
used ChatGPT to generate a list of all discourse markers as well as the
personal pronouns. These lists were additionally evaluated by language
experts. To count stop words, we use NLTK for the respective languages.

###  4.4 Document Features

Our document features are related to the content and structure of a document
such as word frequencies, syntactic structures, and corpus statistics. These
features have been successfully used by Kumarage et al. (2023); Shijaku and
Canhasi (2023); Guo et al. (2023); Mitrović et al. (2023); Zaitsu and Jin
(2023); Mindner et al. (2023). To calculate sentence\- and word-related
features, the text is first divided into sentences and words using NLTK’s
sent_tokenize and word_tokenize functions. For the features related to Part-
of-speech (POS) in EN texts, we use the NLTK function pos_tag. As NLTK lacks
POS tags for the other three languages, we use spaCy NLP
library181818https://github.com/explosion/spaCy. For POS tags in DE texts, we
use de_core_news_sm191919https://spacy.io/models/de#de_core_news_sm, for FR
texts, we use fr_core_news_sm202020https://spacy.io/models/fr#fr_core_news_sm,
and for ES texts,
es_core_news_sm212121https://spacy.io/models/es#es_core_news_sm.

###  4.5 Error Based Features

This feature category introduced in Mindner et al. (2023) is based on errors
in the text such as grammar and spelling mistakes.

To count multiple blanks, we used regular expressions. Grammar and spelling
errors are detected using the open-source tool
LanguageTool222222https://github.com/jxmorris12/language_tool_python which
allows it to detect grammar errors in multiple languages. For the detection of
DE errors, the built-in class LanguageToolPublicAPI(de-DE) for querying the
tool’s public servers is used. For the other languages, the tool’s remote
server is applied using the function Language-Tool(language).

###  4.6 Readability Features

Readability features assess the readability level of texts as in Mindner et
al. (2023); Shijaku and Canhasi (2023); Flesch (1948); Kincaid et al. (1975).

To derive Flesch Reading Ease and Flesch-Kincaid Grade Level we use
Textstat232323https://github.com/textstat/textstat. This Python library
provides functions to calculate text statistics such as grade level,
complexity, and readability. Textstat supports calculating Flesch Reading
Ease, and Flesch-Kincaid Grade Level for EN, FR, DE, and ES texts. However, it
is important to note that these measures were originally developed for the
specific structure of words, sentences, and syllables of EN. Therefore, when
applying these measures to texts in FR, DE, and ES, the results may not be as
representative as those for EN.

###  4.7 AI Feedback Features

Our AI Feeback features reflect, how an AI categorizes the text Mindner et al.
(2023). For this purpose, we use ChatGPT with the prompts in Table 4.

Lang. |  Prompt
---|---
EN |  Was the following text generated by ChatGPT?
FR |  Le texte suivant a-t-il été généré par ChatGPT?
DE |  Wurde der folgende Text von ChatGPT generiert?
ES |  ¿El siguiente texto fue generado por ChatGPT?
Table 4: Prompts used for AI Feedback. Figure 1: Distribution of
$sentiment_{subjectivity}$

###  4.8 Text Vector Features

Our TextVector features analyze semantic content of a text, identifying
patterns and repetition Mindner et al. (2023); Shijaku and Canhasi (2023);
Solaiman et al. (2019); Reimers and Gurevych (2019).

For the features based on Sentence-BERT, we use the sentence-transformer model
distiluse- base-multilingual-cased-v2242424https://huggingface.co/sentence-
transformers/distiluse-base-multilingual-cased-v2, since it supports all the
languages used in this research. In addition to the four languages in our
experiments, it can be used for more than 50 languages, guaranteeing reliable
results for possible future research.

###  4.9 Summary of Our Analyzed Features

Our 8 feature categories contain 37 features. While the AI feedback category
consists of one feature, the perplexity, semantic, error-based, and
readability features each contain two features. The largest feature category
are document features, which contains 19 different features. Table 3
summarizes all the features that are part of our experiments.

##  5 Experimental Setup

In this section, we will describe our experiments with the different feature
categories and three classification approaches: The two more traditional
approaches XGBoost Shijaku and Canhasi (2023) and random forest (RF) Breiman
(2001) as well as a neural network-based approach with multilayer perceptrons
(MLP) Murtagh (1991). As in other studies like Guo et al. (2023); Kumarage et
al. (2023); Mitrović et al. (2023), we evaluated the classification
performance with accuracy (Acc) and F1-score (F1). First, we built text
generation detection systems which were trained, fine-tuned, and tested with
our human-generated and AI-generated texts. Second, we implemented text
rephrasing detection systems which were trained, fine-tuned, and tested with
our human-generated and AI-rephrased texts. To provide stable results, we used
a 5-fold cross-validation, randomly dividing our corpus into 80% training, 10%
validation, and 10% unseen test set. The numbers in all tables are the average
of the test set results. The best performances are highlighted in bold. As a
baseline, we choose two popular state-of-the-art tools which detect texts
generated by AI: GPTZero and ZeroGPT. GPTZero is used by over 1 million people
Shrivastava (2023). However, we found that GPTZero’s results were only
reliable for EN texts. Consequently, we used ZeroGPT as our baseline for FR,
DE and ES.

|  | Generated | Rephrased
---|---|---|---
|  | XGBoost | RF | MLP | XGBoost | RF | MLP
Category | Lang | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1
Perplexity | EN | 83.0 | 82.2 | 87.0 | 85.3 | 82.0 | 82.1 | 52.0 | 48.7 | 55.0 | 54.6 | 56.0 | 63.2
| FR | 62.0 | 60.3 | 69.0 | 66.8 | 68.0 | 69.0 | 50.0 | 50.2 | 53.0 | 44.2 | 56.0 | 58.8
| DE | 74.0 | 74.0 | 76.0 | 76.1 | 81.0 | 80.6 | 53.0 | 53.6 | 61.0 | 60.4 | 56.0 | 62.7
| ES | 82.0 | 82.3 | 83.0 | 82.4 | 82.0 | 83.6 | 56.0 | 55.4 | 63.0 | 63.7 | 62.0 | 67.3
Semantic | EN | 72.0 | 72.9 | 75.0 | 75.6 | 73.0 | 72.3 | 66.0 | 64.4 | 66.0 | 64.3 | 52.0 | 54.3
| FR | 61.0 | 55.8 | 67.0 | 65.6 | 63.0 | 59.4 | 55.0 | 48.2 | 57.0 | 50.0 | 51.0 | 52.9
| DE | 64.0 | 58.3 | 64.0 | 59.8 | 63.0 | 63.3 | 56.0 | 59.9 | 54.0 | 54.4 | 62.0 | 60.1
| ES | 72.0 | 69.9 | 75.0 | 73.8 | 76.0 | 75.7 | 58.0 | 56.1 | 58.0 | 52.4 | 53.0 | 56.3
ListLookup | EN | 72.0 | 72.1 | 79.0 | 78.5 | 71.0 | 67.8 | 72.0 | 73.9 | 67.0 | 67.5 | 69.0 | 70.3
| FR | 72.0 | 73.0 | 76.0 | 76.7 | 67.0 | 62.9 | 66.0 | 62.6 | 65.0 | 65.5 | 64.0 | 63.2
| DE | 74.0 | 75.8 | 79.0 | 77.8 | 72.0 | 74.1 | 57.0 | 59.1 | 58.0 | 59.2 | 50.0 | 52.0
| ES | 78.0 | 79.6 | 82.0 | 84.1 | 73.0 | 76.8 | 75.0 | 75.2 | 80.0 | 81.3 | 77.0 | 78.4
Document | EN | 91.0 | 91.6 | 92.0 | 92.6 | 87.0 | 86.0 | 70.0 | 69.6 | 71.0 | 70.8 | 78.0 | 76.1
| FR | 94.0 | 94.2 | 91.0 | 90.8 | 92.0 | 92.2 | 86.0 | 85.3 | 84.0 | 80.8 | 81.0 | 81.2
| DE | 87.0 | 87.2 | 90.0 | 89.6 | 88.0 | 88.0 | 72.0 | 71.9 | 67.0 | 66.7 | 71.0 | 71.3
| ES | 96.0 | 96.2 | 98.0 | 98.1 | 87.0 | 88.5 | 84.0 | 83.4 | 83.0 | 82.0 | 86.0 | 86.4
ErrorBased | EN | 55.0 | 61.7 | 55.0 | 61.7 | 56.0 | 63.9 | 62.0 | 68.0 | 62.0 | 68.0 | 62.0 | 68.0
| FR | 62.0 | 64.2 | 63.0 | 67.2 | 61.0 | 65.5 | 53.0 | 56.0 | 56.0 | 58.9 | 56.0 | 59.7
| DE | 67.0 | 67.1 | 67.0 | 67.1 | 67.0 | 69.8 | 62.0 | 61.9 | 62.0 | 63.5 | 56.0 | 50.7
| ES | 70.0 | 71.2 | 71.0 | 71.9 | 71.0 | 74.6 | 59.0 | 56.8 | 61.0 | 56.3 | 64.0 | 65.2
Readability | EN | 60.0 | 56.3 | 63.0 | 59.3 | 60.0 | 56.8 | 54.0 | 51.1 | 54.0 | 47.8 | 50.0 | 50.2
| FR | 61.0 | 64.7 | 62.0 | 66.0 | 65.0 | 67.4 | 59.0 | 58.3 | 60.0 | 60.6 | 52.0 | 31.6
| DE | 57.0 | 53.5 | 53.0 | 51.5 | 57.0 | 53.6 | 48.0 | 41.9 | 45.0 | 39.1 | 45.0 | 44.9
| ES | 74.0 | 73.7 | 74.0 | 72.1 | 69.0 | 66.6 | 54.0 | 49.1 | 61.0 | 50.7 | 56.0 | 52.5
AIFeedback | EN | 62.0 | 67.1 | 62.0 | 67.1 | 62.0 | 68.1 | 52.0 | 50.9 | 50.0 | 39.8 | 45.0 | 30.1
| FR | 52.0 | 24.2 | 52.0 | 24.2 | 48.0 | 37.2 | 42.0 | 33.6 | 42.0 | 33.6 | 55.0 | 53.4
| DE | 49.0 | 46.1 | 47.0 | 35.0 | 50.0 | 43.4 | 52.0 | 61.8 | 52.0 | 61.8 | 50.0 | 54.3
| ES | 52.0 | 7.3 | 52.0 | 7.3 | 52.0 | 20.6 | 50.0 | 0.0 | 52.0 | 7.3 | 49.0 | 25.7
TextVector | EN | 90.0 | 89.9 | 95.0 | 94.9 | 83.0 | 81.7 | 79.0 | 78.2 | 75.0 | 71.0 | 69.0 | 65.1
| FR | 94.0 | 94.1 | 93.0 | 93.0 | 85.0 | 85.4 | 77.0 | 77.3 | 75.0 | 75.2 | 68.0 | 64.2
| DE | 87.0 | 87.0 | 94.0 | 94.0 | 90.0 | 90.8 | 68.0 | 67.5 | 72.0 | 67.3 | 72.0 | 71.7
| ES | 84.0 | 84.5 | 91.0 | 89.5 | 81.0 | 76.6 | 76.0 | 74.0 | 76.0 | 73.6 | 68.0 | 64.4
All | EN | 90.0 | 90.9 | 98.0 | 98.0 | 87.0 | 87.8 | 77.0 | 77.6 | 71.0 | 69.8 | 72.0 | 71.9
| FR | 94.0 | 94.4 | 95.0 | 95.0 | 88.0 | 89.2 | 89.0 | 87.9 | 86.0 | 84.2 | 74.0 | 66.4
| DE | 94.0 | 93.8 | 97.0 | 96.9 | 87.0 | 86.6 | 70.0 | 71.6 | 71.0 | 68.3 | 70.0 | 71.6
| ES | 94.0 | 94.4 | 99.0 | 99.0 | 90.0 | 90.2 | 83.0 | 82.2 | 83.0 | 82.9 | 78.0 | 76.1
Table 5: Results for the Detection of EN FR, DE and ES AI-generated and AI-
Rephrased Texts.

##  6 Results

Table 5 lists Acc and F1 for detecting AI-generated and -rephrased texts in
EN, FR, DE, and ES. For each language classifiers trained on AI-generated
texts achieve better performances compared to classifiers trained on AI-
rephrased texts.

###  6.1 Results of Single Feature Categories

As shown in Figure 1 using the example of $sentiment_{subjectivity}$, the
distribution of feature values can differ depending on whether the text is
human-generated, AI-generated or AI-rephrased and depending on the language.
$sentiment_{subjectivity}$ denotes objectivity (low values) or subjectivity
(high values) of a text. Average $sentiment_{subjectivity}$ values tend to be
higher for AI-generated text than for human-generated and AI-rephrased text.
In general, DE texts are the most objective texts—be it human- or AI-
generated—while EN and ES are more subjective. Moreover, AI-generated texts
tend to be more subjective than AI-rephrased texts for our languages.

####  6.1.1 English

##### Text Generation Detection

The results for EN in Table 5 indicate that the system that combines all
features (All) in an RF performs best (Acc=98.0%, F1=98.0%). The 2nd-best
system is the MLP system that uses Document features (Acc=95.0%, F1=94.9%).
The RF system that uses TextVector features results in a similar performance
(Acc=95.0%, F1=94.9%). The worst-performing system is the XGBoost system that
uses the ErrorBased features (Acc=55.0%, F1=61.7%). Compared to GPTZero
($Acc_{GPTZero}$=76.0%, $F1_{GPTZero}$=78.9%), most of our systems perform
better. Our best system with all features (All) outperforms GPTZero by 28.9%
relative in Acc and 24.2% relative in F1. ZeroGPT reaches 78.0%
$Acc_{ZeroGPT}$ and 81.8% $F1_{ZeroGPT}$. Thus, our best system performs 25.6%
relatively better in Acc, and 19.8% relatively better in F1.

##### Text Rephrasing Detection

The performances for the EN text rephrasing detection systems are worse than
the text generation detection systems for all feature categories except
ErrorBased (Acc=62.0%, F1=68.0%). The best-performing system is the XGBoost
system that uses TextVector features (Acc=79.0%, F1=78.2%), followed by the
MLP system that uses Document features (Acc=78.0%, F1=76.1%). The worst-
performing system is the MLP system that uses the AIFeedback feature. All our
text rephrasing detection systems were able to outperform GPTZero
($Acc_{GPTZero}$=43.0% and $F1_{GPTZero}$=27.8%). Our the best-performing
TextVector feature system even outperforms GPTZero by 83.7% relative in Acc
and even 159.8% relative in F1. ZeroGPT reaches 49.0% $Acc_{ZeroGPT}$ and
43.9% $F1_{ZeroGPT}$. Thus, Document outperforms it by 61.2% relative in Acc
and 81.5% relative in F1.

####  6.1.2 French

##### Text Generation Detection

The results for FR in Table 5 demonstrate that the system that combines all
features (All) in an RF performs best (Acc=95.0%, F1=95.0%). The 2nd-best
system is the XGBoost system that uses Document features (Acc=86.0%,
F1=85.3%), followed by the XGBoost system that uses TextVector features
(Acc=77.0%, F1=77.3%). The worst-performing systems are those that use the
AIFeedback feature. Our best FR system with all features (All) outperforms
ZeroGPT ($Acc_{ZeroGPT}$=62.0, $F1_{ZeroGPT}$)=72.6%) by 53.2% relative in Acc
and 30.9% relative in F1.

##### Text Rephrasing Detection

The performances for the FR text rephrasing detection systems are worse than
the text generation detection systems for all feature categories except the
MLP system that uses the AIFeedback feature (Acc=55.0%, F1=53.4%). The best-
performing system is the system that that combines all features (All) in an
XGBoost (Acc=89.0%, F1=87.9%), followed by the XGBoost system that uses
Document features (Acc=86.0%, F1=85.3%) and the XGBoost system that uses
TextVector features (Acc=77.0%, F1=77.3%). The worst-performing systems are
again those that use the AIFeedback feature. Our best FR system with all
features (All) outperforms ZeroGPT ($Acc_{ZeroGPT}$=57.0,
$F1_{ZeroGPT}$)=67.4%) by 56.1% relative in Acc and 30.4% relative in F1.

####  6.1.3 German

##### Text Generation Detection

The results for DE in Table 5 indicate that the system that combines all
features (All) in an RF performs best (Acc=97.0%, F1=96.9%). The 2nd-best
system is the RF system that uses TextVector features (Acc=94.0%, F1=94.0%),
followed by the RF system that uses Document features (Acc=90.0%, F1=89.6%).
As for the previous languages, the worst-performing systems are those that use
the AIFeedback feature. Our best FR system with all features (All) outperforms
ZeroGPT ($Acc_{ZeroGPT}$=65.0, $F1_{ZeroGPT}$)=70.9%) by 49.2% relative in Acc
and 36.7% relative in F1.

##### Text Rephrasing Detection

The performances for the DE text rephrasing detection systems are worse than
the text generation detection systems for all feature categories except the
systems that use the AIFeedback features. The best-performing system is the
XGBoost system that that uses the Document features (Acc=72.0%, F1=71.9%),
followed by the MLP system that uses TextVector features (Acc=72.0%,
F1=71.7%). The worst-performing systems are those that use the Readability
feature. Our best DE system with the Document features outperforms ZeroGPT
($Acc_{ZeroGPT}$=48.0, $F1_{ZeroGPT}$=49.5%) by 45.5% relative in Acc and
45.3% relative in F1.

####  6.1.4 Spanish

##### Text Generation Detection

The results for ES in Table 5 show that the system that combines all features
(All) in an RF performs best (Acc=99.0%, F1=99.0%). The 2nd-best system is the
RF system that uses Document features (Acc=98.0%, F1=89.1%), followed by the
RF system that uses TextVector features (Acc=91.0%, F1=89.5%) and the RF
system that uses ListLookup features (Acc=82.0%, F1=84.1%). As for the
previous languages, the worst-performing systems are those that use the
AIFeedback feature. The F1 of 7.3% is so poor since the feature classifies the
text as AI-generated text in almost all cases. Our best ES system with all
features (All) outperforms ZeroGPT ($Acc_{ZeroGPT}$=60.0,
$F1_{ZeroGPT}$)=71.5%) by 65.0% relative in Acc and 38.5% relative in F1.

##### Text Rephrasing Detection

The performances for the ES text rephrasing detection systems are worse than
the text generation detection systems for all feature categories. The best-
performing system is the RF system that uses the Document features (Acc=86.0%,
F1=86.4%). The 2nd best system is the system that combines all features (All)
in an RF (Acc=83.0%, F1=82.9%), followed by the RF system that uses the
ListLookup features (Acc=80.0%, F1=81.3%). The worst-performing systems are
those that use the AIFeedback feature. The F1 of 0% and 7.3% are so poor since
the feature classifies the text as AI generated text in almost all cases. Our
best ES system with the Document features outperforms ZeroGPT
($Acc_{ZeroGPT}$=52.0, $F1_{ZeroGPT}$=63.7%) by 65.4% relative in Acc and
25.6% relative in F1.

####  6.1.5 Combination of All Features

As shown in Table 5, the best performances for the text generation detection
systems are achieved using a combination of all features (All). Looking at the
systems which use all features, the Acc for the AI-generated FR and DE texts
is similar with 97.0%, while the Acc for the AI-generated EN texts is 98.0%.
The best F1 for the AI-generated DE classifier is 96.9%. Thus, it is slightly
worse than the classifiers trained on our EN and FR texts which achieved 98.0%
and 97.1%, respectively. The best classifier trained on the AI-generated ES
texts achieved slightly better performances, with 99.0% Acc and 99.0% F1.
Comparing the performances of the systems trained on the AI-generated texts,
it can be summarized that the classifiers deliver comparable performances
across the languages.

The performances of the systems which use all features (All) vary more for the
AI-rephrased texts across the languages. While the best EN classifier reaches
79.0% Acc on the AI-rephrased texts, the best FR classifier achieves 89.0% Acc
on the AI-rephrased texts. The AI-rephrased detection system for DE only
achieves 72.0% Acc. Compared to the best DE text rephrasing detection system,
the FR system is 23.6% relatively better in Acc. The Acc for the ES text
rephrasing detection system is 1% worse than the FR system. For F1, comparable
conclusions can be drawn across the languages. Thus, our investigated features
do not deliver comparable performances for the detection of AI-rephrased texts
across the evaluated languages.

##  7 Conclusion and Future Work

In this paper, we investigated features to classify whether text is written by
a human, generated by AI from scratch or rephrased by AI. We conducted a
comparative analysis of the classification across the languages of EN, FR, DE,
and ES, assessing the performance of these features in their respective
linguistic contexts. To train and test classifiers which use the features, we
extended the Human-AI-Generated Text Corpus Mindner et al. (2023)—our new text
corpus, which covers 10 different topics for each of the four languages. For
AI-generated text, our classifier performed best when combining all features,
meaning that there are no substantial differences for features across
languages. Therefore, we conclude, that the same feature set could also be
used for other languages from the same language families. The accuracies are
close with 99% for ES, 98% for EN, 97% for DE and 95% for FR. In contrast to
that, for the detection of AI-rephrased text, the systems with all features
outperformed systems with other features in many cases. For DE (72%) and ES
(86%) we achieved the best results using only document features while for EN
the text vector features yielded the best results (79%).

Although our results indicate that the same feature set could be applied to
other languages within the same familie, future work could investigate the
applicability of these features across further language families. This would
help in understanding the robustness of our method across a more diverse set
of languages. Moreover, our corpus currently covers 10 different topics for
each language. Extending the corpus to include more topics, and possibly
considering different domains and genres, may help in generalizing the
findings and making the system more robust. Finally, experimenting with
different machine learning architectures such as transformer models could
potentially lead to further optimizations.

## Ethics Statement

The collected corpus is made freely available to the community. It is based on
Wikipedia and news texts. The research was conducted transparently, free from
bias and in compliance with applicable laws and regulations. The use of AI
models and data is intended to foster a deeper understanding of AI-generated
content, with the goal of promoting responsible use and technological
innovation.

## References

  * Adiwardana et al. (2020) Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V Le. 2020.  Towards a Human-Like Open-Domain Chatbot.  _ArXiv Preprint ArXiv:2001.09977_.
  * Arteaga et al. (2019) David Arteaga, Juan Arenas, Freddy Paz, Manuel Tupia, and Mariuxi Bruzza. 2019.  Design of Information System Architecture for the Recommendation of Tourist Sites in the City of Manta, Ecuador through a Chatbot.  In _2019 14th Iberian Conference on Information Systems and Technologies (CISTI)_ , pages 1–6. IEEE.
  * Baidoo-Anu and Owusu Ansah (2023) David Baidoo-Anu and Leticia Owusu Ansah. 2023.  Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning.  _Available at SSRN 4337484_.
  * Breiman (2001) Leo Breiman. 2001.  Random Forests.  _Machine Learning_ , 45(1):5–32.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language Models are Few-Shot Learners.  _CoRR_ , abs/2005.14165.
  * Dibitonto et al. (2018) Massimiliano Dibitonto, Katarzyna Leszczynska, Federica Tazzi, and Carlo M Medaglia. 2018.  Chatbot in a Campus Environment: Design of LiSA, a Virtual Assistant to Help Students in Their University Life.  In _Human-Computer Interaction. Interaction Technologies: 20th International Conference, HCI International 2018, Las Vegas, NV, USA, July 15–20, 2018, Proceedings, Part III 20_ , pages 103–116. Springer.
  * Ethnologue (2023) Ethnologue. 2023.  What are the top 200 most spoken languages?
  * Falala-Séchet et al. (2019) Clara Falala-Séchet, Lee Antoine, Igor Thiriez, and Catherine Bungener. 2019\.  OWLIE: A Chatbot that Provides Emotional Support for Coping With Psychological Difficulties.  In _Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents_ , pages 236–237.
  * Flesch (1948) Rudolf Franz Flesch. 1948.  A New Readability Yardstick.  _The Journal of applied psychology_ , 32 3:221–233.
  * Gehrmann et al. (2019) Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019.  GLTR: Statistical Detection and Visualization of Generated Text.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_ , pages 111–116, Florence, Italy. Association for Computational Linguistics.
  * Guo et al. (2023) Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023.  How close is chatgpt to human experts? comparison corpus, evaluation, and detection.
  * Jeblick et al. (2022) Katharina Jeblick, Balthasar Schachtner, Jakob Dexl, Andreas Mittermeier, Anna Theresa Stüber, Johanna Topalis, Tobias Weber, Philipp Wesp, Bastian Sabel, Jens Ricke, and Michael Ingrisch. 2022.  ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports.  _ArXiv E-Prints_.
  * Jiao et al. (2023) Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu. 2023.  Is ChatGPT a Good Translator? A Preliminary Study.  _ArXiv Preprint ArXiv:2301.08745_.
  * Kenton and Toutanova (2019) Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _Proceedings of NAACL-HLT_ , pages 4171–4186.
  * Kincaid et al. (1975) J. Peter Kincaid, Robert P. Fishburne, Richard L. Rogers, and Brad S. Chissom. 1975\.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel.
  * Kumarage et al. (2023) Tharindu Kumarage, Joshua Garland, Amrita Bhattacharjee, Kirill Trapeznikov, Scott Ruston, and Huan Liu. 2023.  Stylometric Detection of AI-Generated Text in Twitter Timelines.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _CoRR_ , abs/1907.11692.
  * Mesko (2023) Bertalan Mesko. 2023.  The chatgpt (generative artificial intelligence) revolution has made artificial intelligence approachable for medical professionals.  _Journal of medical Internet research_ , 25:e48392.
  * Mindner et al. (2023) Lorenz Mindner, Tim Schlippe, and Kristina Schaaff. 2023.  Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT.  _TBD_.
  * Mitchell et al. (2023) Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023.  DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature.  In _Proceedings of the 40th International Conference on Machine Learning (ICML 2023)_.
  * Mitrović et al. (2023) Sandra Mitrović, Davide Andreoletti, and Omran Ayoub. 2023.  ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-Generated Text.  _arXiv preprint arXiv:2301.13852_.
  * Murtagh (1991) Fionn Murtagh. 1991.  Multilayer Perceptrons for Classification and Regression.  _Neurocomputing_ , 2(5):183–197.
  * Pelau et al. (2021) Corina Pelau, Dan-Cristian Dabija, and Irina Ene. 2021.  What Makes an AI Device Human-Like? The Role of Interaction Quality, Empathy and Perceived Psychological Anthropomorphic Characteristics in the Acceptance of Artificial Intelligence in the Service Industry.  _Computers in Human Behavior_ , 122:106855.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Roberts et al. (2019) Adam Roberts, Colin Raffel, Katherine Lee, Michael Matena, Noam Shazeer, Peter J. Liu, Sharan Narang, Wei Li, and Yanqi Zhou. 2019.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.  Technical report, Google.
  * Sanh et al. (2019) Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019.  DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter.  In _Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS 2019)_.
  * Shijaku and Canhasi (2023) Rexhep Shijaku and Ercan Canhasi. 2023.  ChatGPT Generated Text Detection.
  * Shrivastava (2023) Rashi Shrivastava. 2023.  With Seed Funding Secured, AI Detection Tool GPTZero Launches New Browser Plugin.
  * Solaiman et al. (2019) Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang. 2019.  Release Strategies and the Social Impacts of Language Models.
  * Soni and Wade (2023) Mayank Soni and Vincent Wade. 2023.  Comparing Abstractive Summaries Generated by ChatGPT to Real Summaries Through Blinded Reviewers and Text Classification Algorithms.
  * Taecharungroj (2023) Viriya Taecharungroj. 2023.  “What Can ChatGPT Do?” Analyzing Early Reactions to the Innovative AI Chatbot on Twitter.  _Big Data and Cognitive Computing_ , 7(1):35.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023.  LLaMA: Open and Efficient Foundation Language Models.
  * Yu et al. (2023) Peipeng Yu, Jiahan Chen, Xuan Feng, and Zhihua Xia. 2023.  CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts.
  * Zaitsu and Jin (2023) Wataru Zaitsu and Mingzhe Jin. 2023.  Distinguishing ChatGPT(-3.5, -4)-generated and Human-Written Papers Through Japanese Stylometric Analysis.