# Strain-induced stacking transition in bilayer graphene

Nina C. Georgoulea1, Stephen R. Power1,2 and Nuala M. Caffrey3,4 1 School of
Physics, AMBER & CRANN Institute, Trinity College Dublin, Dublin 2, Ireland  2
School of Physical Sciences, Dublin City University, Dublin 9, Ireland  3
School of Physics, University College Dublin, Dublin 4, Ireland  4 Centre for
Quantum Engineering, Science, and Technology, University College Dublin,
Dublin 4, Ireland<EMAIL_ADDRESS>
###### Abstract

Strain, both naturally occurring and deliberately engineered, can have a
considerable effect on the structural and electronic properties of 2D and
layered materials. Uniaxial or biaxial heterostrain modifies the stacking
arrangement of bilayer graphene (BLG) which subsequently influences the
electronic structure of the bilayer. Here, we use Density Functional Theory
(DFT) calculations to investigate the interplay between an external applied
heterostrain and the resulting stacking in BLG. We determine how a strain
applied to one layer is transferred to a second, ‘free’ layer and at what
critical strain the ground-state AB-stacking is disrupted. To overcome
limitations introduced by periodic boundary conditions, we consider an
approximate system consisting of an infinite graphene sheet and an armchair
graphene nanoribbon (AGNR). We find that above a critical strain of $\sim
1\%$, it is energetically favourable for the free layer to be unstrained,
indicating a transition between uniform AB-stacking and non-uniform mixed
stacking. This is in agreement with a simple model estimate based on the
individual energy contributions of strain and stacking effects. Our findings
suggest that small levels of strain provide a platform to reversibly engineer
stacking order and Moiré features in bilayers, providing a viable alternative
to twistronics to engineer topological and exotic physical phenomena in such
systems.

##  1 Introduction

Two-dimensional (2D) materials, such as graphene, exhibit unique mechanical
and electronic properties [1, 2, 3, 4, 5]. 2D materials can further be
combined to create heterostructures which can have different properties to
their component layers due to interlayer interactions [6, 7]. For example, the
2D elastic moduli of bilayer heterostructures, such as graphene/MoS2 and
MoS2/WS2, are smaller than the sum of the moduli of the individual layers [8].
Interlayer interactions can be tuned, for example, by changing the relative
stacking of the layers. This allows a wide range of different behaviours to be
observed, even in structures with multiple layers of the same material.

Two monolayer graphene layers (MLGs) can be stacked to form bilayer graphene
(BLG) [9, 10]. In the ground-state AB-stacking, half of the carbon atoms in
each layer are directly above the centre of a hexagon on the other layer and
the other half are directly on top of another carbon atom [11, 12] (figure
1(a)). AA-stacking has also been observed in which every carbon atom of one
layer is directly above a carbon atom on the second layer (figure 1(b)). AB-
stacked BLG has parabolic electronic bands, whereas the bands of AA-stacked
BLG remain linear, as in MLG [13, 9]. As a result, AB- and AA-stacked BLG
behave very differently under the application of interlayer bias, which opens
a band gap proportional to the bias for AB-stacking [14], while the AA-system
remains semimetallic [15]. Between the AB- and AA- stacking limits lie a range
of different stacking possibilities. The electronic and topological properties
of these systems varies as a function of relative shift between the layers
[16, 17]. For example, the transport properties of BLG depend sensitively on
its stacking, with a large change in the transmission predicted when one layer
is shifted relative to the other [16]. Thus, manipulating the stacking of a
bilayer is a powerful tool to tune its electronic and transport behaviour.
Twisted BLG (TBLG) harnesses this to great effect [18, 19, 20, 21].

Both uniform and non-uniform strain can tune the electronic, transport and
optical properties of both MLGs and BLGs [22, 23, 24, 25, 26, 27, 28, 29, 30].
Strain can arise naturally when graphene is placed on a substrate due to
lattice mismatch between the two systems [31, 32]. Strain can also be
intentionally created and controlled in graphene by using a flexible substrate
[33, 34]. If BLG is placed on a flexible substrate, strain will be applied to
the layer which is in direct contact with the substrate [35]. It is reasonable
to expect that if the applied strain is sufficiently small, it will be
entirely transferred to the second graphene layer, i.e., both layers will
experience the same strain. However, for larger applied strains the second
layer can exhibit a different strain profile [36, 37, 35]. A heterostrain
modifies the stacking between layers in a manner similar to a relative twist
angle, creating a Moiré superlattice [38, 39, 40]. Theoretical works predict
that heterostrain can be used to open and tune an electronic energy gap in BLG
[38], or to induce a transition from a direct to an indirect band gap in the
presence of a bias field [41].

In this work, we investigate the interplay between uniaxial heterostrain and
stacking effects in BLG. The system consists of a bottom layer which is
uniaxially strained along the zigzag (ZZ) direction and a ‘free’ top layer as
shown in figure 1(c). For small amounts of strain applied to the bottom layer,
the free layer is expected to strain by the same amount, as the energetic cost
of straining is small compared to that of breaking uniform AB-stacking.
However, at a certain critical value of applied strain, the energetic cost of
maintaining strain in the free layer exceeds the cost of breaking AB-stacking.
At this point, strain is released in the free layer as the energy benefit in
doing so is greater than the energy penalty to be paid for a less-than-ideal
stacking configuration. At this critical strain, a transition between a
uniform AB-stacking order and a non-uniform stacking will occur, together with
the formation of Moiré superlattices, as in TBLG [42, 20]. Near this critical
point, the change in stacking order can have significant implications on
electronic properties [16, 17, 38].

Figure 1: (a) Structure of AB-stacked BLG, showing the high symmetry armchair
(AC) and ZZ directions. The filled and hollow symbols represent atoms from
each of the two sublattices. (b) Structure of AA-stacked BLG. (c) Structure of
the BLG system considered in this work, where uniaxial strain is applied only
to the bottom layer (orange) and we consider strained and unstrained
configurations of the free layer (blue).

##  2 Computational Details

Density functional theory calculations were performed using vasp-5.4.1 [43,
44, 45, 46, 47, 48]. The Perdew-Burke-Ernzerhof (PBE) [49] parametrization of
the generalized gradient approximation (GGA) was employed. Van der Waals (vdW)
interactions are included using the D2 semi-empirical method of Grimme [50].
The plane wave basis set was converged using an 950 eV energy cutoff. A
13$\times$21$\times$1 mesh was used to determine the total energies of MLG and
BLG. All structures were optimized until the residual forces were less than
0.01 eV/Å. A vacuum layer of at least 11.5 Å was included in the direction
normal to MLG or BLG to ensure no spurious interactions between repeating
slabs. The GGA calculated lattice constants of MLG and BLG were both found to
be 2.47 Å, in good agreement with the experimental value of 2.46 Å. The
interlayer distances in AB- and AA-stacked BLG were found to be 3.37 Å and
3.50 Å, respectively, in good agreement with previous studies [51, 52, 53].

To overcome restrictions caused by the use of periodic boundary conditions in
the graphene plane, the heterostrained BLG system was approximated using a
hydrogen-terminated armchair graphene nanoribbon (AGNR) adsorbed on top of a
MLG. The dangling C bonds at the AGNR edges were passivated with H atoms.
Heterostrain was then introduced by straining the MLG along the ZZ direction.
A 21$\times$5$\times$1 k-point mesh was used to converge the total energy of
the heterostrained structures. A distance of at least 12.5 Å was maintained
between periodic replicas of adsorbed AGNRs to ensure that they do not
interact. The optimal AGNR width was determined by comparing the stacking-
dependent AGNR binding energy ($E_{B}$) to the stacking-dependent binding
energy of BLG. In this case, the in-plane positions of the two central atoms
of the AGNR were held fixed at the chosen stacking and all other AGNR atoms
were allowed to relax both in-plane and out-of-plane. To determine the lowest
energy stacking configuration for different heterostrains, the carbon-hydrogen
bonds and the interlayer distance between the MLG and the AGNR were relaxed
while the in-plane positions of all other atoms were held fixed.

##  3 Results

To determine the critical strain for which a transition occurs from a
uniformly strained BLG with AB-stacking to a heterostrained system with
disrupted stacking, we compare the energetics of two limiting cases: when the
free layer either adopts the same strain as that applied to the bottom layer,
or it remains completely unstrained. Possible intermediate scenarios, where
the free layer adopts a non-zero strain different to that in the bottom layer
or displays a non-uniform strain distribution, are not considered in this work
due to their high computational cost. Similarly, in order to maintain
periodicity for the DFT calculations, we neglect the role of contraction in
the direction perpendicular to the applied strain. This is equivalent to
setting the Poisson ratio $\nu=0$. We discuss how these approximations can be
relaxed, and the expected consequences, in Section 4.

###  3.1 Simple Model

There are two principal energy costs, due to strain ($\Delta
E_{\mathrm{strain}}$) and stacking ($\Delta E_{\mathrm{stack}}$), which
determine the behaviour of the free layer when uniaxial strain is applied to
the bottom layer. To get a rough estimate of where the transition between a
strained and unstrained free layer occurs, we can compare the expected energy
costs of straining the free layer in isolation, and of breaking AB-stacking in
an unstrained BLG system.

Figure 2: (a) $\Delta E_{\mathrm{strain}}$ per atom as a function of applied
ZZ uniaxial strain in MLG. (b) $\Delta E_{\mathrm{stack}}$ per atom as a
function of the relative shift between layers in BLG for different initial
stackings (AA and AB-stackings). Shifts along both ZZ and AC directions are
shown.

We first consider $\Delta E_{\mathrm{strain}}=E-E_{\mathrm{unstrained}}$, the
energy cost associated with straining the free layer away from its relaxed
structure to match the strain applied to the bottom layer. The energy cost of
straining a graphene layer increases with the amount of strain considered, as
shown in figure 2(a) for MLG with strain applied along the ZZ direction.
$\Delta E_{\mathrm{strain}}$ displays almost identical behaviour in AA- and
AB-stacked BLG, and for strains along the AC direction in all three systems,
with a maximum variation of only 2.8 meV per atom. Since $\Delta
E_{\mathrm{strain}}$ is not significantly affected by the strain direction, or
the nature of the stacking in BLG systems, the curve in figure 2(a) should
also be an excellent approximation to the energetic cost of straining a single
layer in BLG in the absence of stacking effects.

If the amount of strain is different in the two layers, then the system is no
longer able to maintain energetically-favourable AB-stacking, and instead must
display a modulation of the stacking order with an associated energy cost
$\Delta E_{\mathrm{stack}}=E-E_{\mathrm{AB}}$. Although the modulation
wavelength depends on the strain mismatch, the energy cost per atom is roughly
constant, as a similar range of stackings will occur for any mismatch.
Therefore, $\Delta E_{\mathrm{stack}}$ should not depend sensitively on the
strain applied to the bottom layer, and we approximate it by considering
different stacking configurations in unstrained BLG. Figure 2(b) shows the
energetic cost of rigidly shifting one graphene layer over the other along ZZ
or AC directions, starting from either an initial AB- or AA-stacking. As the
layers are shifted, the in-plane positions of atoms are held fixed to maintain
the desired stacking configuration, but the interlayer distance is allowed to
relax. Any shift away from AB-stacking results in a positive $\Delta
E_{\mathrm{stack}}$, confirming that this is the preferred configuration.
$\Delta E_{\mathrm{stack}}$ is maximum for AA-stacking with a value of 5.84
meV/atom, in excellent agreement with Ref. [16]. The stacking modulation
arising from heterostrain contains a combination of various stackings, and the
corresponding $\Delta E_{\mathrm{stack}}$ can be approximated as an
appropriately weighted average of the values appearing in figure 2(b).

Comparing figure 2(a) and 2(b) allows us to understand the interplay between
strain and stacking in heterostrained BLG. For small strains applied to the
bottom layer, $\Delta E_{\mathrm{strain}}\ll\Delta E_{\mathrm{stack}}$, and it
is energetically favourable for the free layer adopt the same strain. However,
as the strain applied to the bottom layer is increased, the cost of uniformly
straining the free layer eventually balances the cost of breaking AB-stacking.
A transition occurs above this critical strain, releasing the strain in the
free layer and introducing a modulation of the stacking order. The maximum
possible value of critical strain is restricted by the finite range of $\Delta
E_{\mathrm{stack}}$: the maximum possible cost of breaking AB-stacking (i.e.,
$\Delta E_{\mathrm{stack}}^{AA}\sim 5.84$ meV/atom), corresponds to a strain
of 1.4%. In reality, the mix of different stackings that occur in a hetero-
strained BLG will give $0<\Delta E_{\mathrm{stack}}<\Delta
E_{\mathrm{stack}}^{AA}$. For an even distribution of stackings between AB and
AA, we can estimate $\Delta E_{\mathrm{stack}}\sim\frac{1}{2}\Delta
E_{\mathrm{stack}}^{AA}$, corresponding to a critical strain of 0.97%. Uneven
stacking distributions can occur if the considered strain excludes certain
stackings, or if a non-uniform strain distribution is allowed in the free
layer. However, even accounting for a significant reduction in $\Delta
E_{\mathrm{stack}}$ due to these effects does not dramatically change the
expected critical strain. For example, assuming $\Delta
E_{\mathrm{stack}}\sim\frac{1}{4}\Delta E_{\mathrm{stack}}^{AA}$ still gives a
critical strain of 0.66%. The results of this simple model strongly suggest
that the critical strain is near $1\%$, and that uniform AB-stacking will be
broken when larger strains are applied to the bottom layer.

###  3.2 AGNR on strained MLG

DFT calculations of heterostrained bilayers will now be used to test the
prediction of this simple model that the critical strain occurs near 1%.
However, periodic boundary conditions enforce a commensurability condition
when dealing with two infinite graphene sheets. Although neglecting
contraction in the transverse direction ($\nu=0$) simplifies matters
considerably, only certain values of strain would be achievable [38].
Furthermore, very large supercells would be required to investigate the
relevant strain range: $1\%$ strain requires 101 cells of the free layer and
100 cells of the strained layer. Getting sufficient resolution to determine
the critical strain quickly becomes computationally prohibitive.

Figure 3:  Left: Structure of BLG when the bottom (orange) layer is uniaxially
strained by 15% along the ZZ (vertical) direction. A modulation of the
stacking is clearly visible. Right: The systems considered in this work, where
AB and shifted regions of the BLG system are represented by finite-width
AGNRs.

To overcome this constraint, we instead investigate the interplay of strain
and stacking in a system where different regions of the free layer are
modelled by finite-width AGNRs. This is shown schematically on the left side
of figure 3, for an exaggerated strain of 15% along the ZZ (vertical)
direction applied to the bottom layer. A one-dimensional Moiré pattern is
evident in the full BLG with regions of AB-stacking (lower dashed box)
separated by other stacking types. Due to the ZZ strain direction and $\nu=0$
, no AA-stacking occurs and the furthest stacking from AB is that in the upper
dashed box, which we denote ‘Shifted’ and corresponds to rigidly shifting one
layer of unstrained BLG by half a graphene lattice constant in the ZZ
direction. We aim to determine the energetics of the complete system (left
hand side) by modelling different portions of it by a finite-width ribbon
adsorbed onto an infinite bottom layer (right hand side). The AGNR can be
rigidly shifted over the continuous bottom layer to approximate the different
stackings that occur in a heterostrained bilayer system. As the free layer is
no longer continuous, but now consists of a periodic array of AGNRs, we can
consider different strains in each layer using a constant-size supercell. We
consider hydrogen-passivated AGNRs to circumvent features including
unpassivated bonds or localised edge states. These may occur in certain AGNRs
but are not expected in extended bilayer systems [54, 55, 56].

The choice of AGNR width is determined by that which best approximates the
stacking-dependent binding energy of BLG at a reasonable computational cost.
To meaningfully compare the binding energy of BLG and the $n$-AGNR/MLG systems
we normalise $E_{B}$ by the number of carbon atoms in the top-layer, $N_{C}$:

$\beta=\frac{E_{B}}{N_{C}}$ (1)

Figure 4(a) shows how $\beta$ varies as the top layer is rigidly shifted along
the AC direction from AA to AB alignment with the bottom layer. The dashed
black curve shows $\beta$ for BLG while the other curves show $\beta$ for the
$n$-AGNR/MLG systems with $n=5,\dots,10$. While there is an offset between the
BLG binding energy and that of the AGNR/MLG, this offset is approximately
constant across all stackings between AA and AB. For widths in excess of $n=6$
the error is less than 10%. Figure 4(b) shows the binding energy difference
between AA and AB-stackings, $\beta_{AA}-\beta_{AB}$, as a function of width
$n$, compared to the corresponding quantity for BLG (dashed line). Agreement
between the BLG and AGNR/MLG systems improves in general with the increase of
$n$, but non-uniformly due to different behaviour of ribbons with widths
$n=3q,3q+1$ and $3q+2$, where $q=1,2,3,\dots$. Similar trends have been noted,
for example, for the band gap of AGNRs [57]. As we are interested in how
stacking changes the energetics, and not the absolute magnitude of the
$E_{B}$, the $6$-AGNR is deemed sufficiently wide for our purposes.

Figure 4: (a) $\beta$ (i.e., $E_{B}$ per overlap atom) as a function of
stacking for different width $n$-AGNR/MLG systems. (b) Energy difference
$\beta_{AA}-\beta_{AB}$ as a function of width $n$ for n-AGNR/MLG. The black
dashed lines show the corresponding infinite BLG results.

To estimate the critical strain in heterostrained BLG using the 6-AGNR/MLG
system, we consider the energy difference, $\Delta E_{X}$, between strained
and unstrained AGNR layers, for different stackings ($X=$ AB, shifted). We
emphasise that it is the _unstrained_ case which gives rise to broken
stacking, due to the strain applied to the bottom layer, whereas the
_strained_ case restores AB-stacking by matching the strain in the both
layers. The energy difference between strained and unstrained AGNR layers is
then given by:

$\Delta E=E_{\mathrm{AB,\,strain}}-E_{X,\mathrm{unstrain}}\,,$ (2)

where we note that the energy of the final state $E_{\mathrm{AB,\,strain}}$ is
the same in each case, as AB-stacking has been restored.
$E_{X,\mathrm{unstrain}}$ corresponds to the case that the MLG is strained and
the AGNR is unstrained, with the stacking, $X$, set by fixing the positions of
the central carbon atoms of the AGNR. The in-plane positions of the AGNR
carbon atoms are held fixed as determined by the strain and stacking, while
the hydrogen-carbon bonds and the interlayer distance are allowed to relax.

Figure 5: Energy difference $\Delta E$ between the strained and unstrained
free-layer geometries as a function of ZZ uniaxial strain applied to the
bottom layer. In the strained case, the layers are AB-stacked, whereas in the
unstrained case different stackings are possible, with different contributions
to the $\Delta E$. These calculations were performed using the $6$-AGNR/MLG
system. The averaged curve indicates a critical strain of $\sim 1.2\%$.

The full heterostrained system is considered as an average of the two stacking
extremes, AB and Shifted (more general cases will be discussed later):

$\Delta
E_{\mathrm{av}}=E_{\mathrm{AB,\,strain}}-\tfrac{1}{2}(E_{\mathrm{AB,\,unstrain}}+E_{\mathrm{shifted}\mathrm{,\,unstrain}}).$
(3)

Negative values of $\Delta E_{\mathrm{av}}$ imply that the free layer prefers
to be strained so that the bilayer system remains AB-stacked. Positive values
indicate that the free layer prefers to be unstrained, and the system adopts a
non-uniform stacking profile. Figure 5 shows $\Delta E$ and $\Delta
E_{\mathrm{av}}$, as a function of strain applied along the ZZ direction. The
curve for the AB-stacked AGNR/MLG system shows that it prefers to be
unstrained, i.e., that it is energetically favourable for the AGNR to break
perfect AB-stacking, instead of maintaining a strain of between 0.8% - 1.3%.
This is not surprising, as the stacking mismatch is not too significant in
this part of the modulated structure (c.f. figure 3). However, in the shifted
region, the stacking deviates furthest from AB when the free layer is
unstrained, and the associated $\Delta E$ curve shows that a strained, uniform
AB-stacking is preferred. The overall preference of the system is a
competition between these different regions. For a uniform distribution of
stackings in the modulated structure, the averaged case shown by the grey
curve in Figure 5 indicates that a transition occurs at a critical strain of
$\sim 1.2\%$. This is in very good agreement with the estimate given by the
simple model.

##  4 Discussion

Modulated stacking profiles in BLG systems lead to a wide range of new
properties, particularly when combined with an interlayer bias. Such a bias
will open a band gap in regions with particular stackings, leading to a
complex distribution of gapped and conducting regions which follow the
underlying stacking pattern. This has been widely studied in twisted bilayers
where, for example, networks of 1D topological channels have been found
between gapped AB- and BA- stacked regions of the Moiré pattern [58, 59, 60].
The localisation of electronic states in different regions of the lattice,
together with the resulting strong interaction effects, are connected with the
formation of correlated insulating states and unconventional superconductivity
in twisted systems [61].

Heterostrained, untwisted BLG, as considered here, could potentially host a
similar range of phenomena. The schematic structure in figure 3 shows the
formation of a 1D Moiré pattern with different stacking profiles, which would
also create a spatially-varying band gap landscape in the presence of an
interlayer bias. To maintain periodicity for our calculations, we neglected a
Poisson contraction perpendicular to the applied strain. Including such a
contraction would lead to a 2D modulation of the stacking pattern and a more
complete analogy with twisted systems. We note that setting $\nu\neq 0$ allows
a wider range of stackings, when the free layer is unstrained, than for the
simple 1D modulation considered in figures 3 and 5. In particular, more
energetically unfavourable stackings, such as AA, are now possible, which may
slightly increase the critical strain due to the increased energetic cost of
breaking uniform stacking.

Finally, the estimate of the critical strain in this work is based on the free
layer being either unstrained, or uniformly adopting the same strain as the
bottom layer. We have not considered, for example, an intermediate value of
strain in the free layer. We expect such cases to be less energetically
favourable than the unstrained free layer. This is because such cases will
have to pay both strain and stacking-related energy costs, and the latter is
expected to be largely strain independent, as discussed in Section 3.1.
Therefore the minimum energy heterostrained system should be that which
minimises $\Delta E_{\mathrm{strain}}$, namely that with an unstrained free
layer. Due to the periodicity constraints of our calculations, we also not
been able to explicitly consider the role of non-uniform strain in the free
layer. Such non-uniform strain profiles could arise due to lattice
relaxations, as occurs in TBLG [62, 63, 64], where it serves to reduce the
area of the AA-stacking region and maximize the area of the lower energy AB-
stacking region [65].

##  5 Conclusions

Our results strongly indicate the existence of a critical strain of $\sim 1\%$
which, when applied to one layer of BLG, can be used to tune the stacking
profile of the system. Below the critical strain, it is energetically
favourable to transfer the strain to the second layer in order to maintain a
uniform AB-stacking configuration. Above the critical value, the cost of
maintaining the strain in the second layer is too high and the system prefers
to release it and adopt a non-uniform stacking profile. This finding is
supported by DFT calculations which consider the energetic costs of strain and
stacking independently in infinite, periodic systems, and by further
calculations which consider both contributions simultaneously when one layer
is represented by finite AGNRs.

## Acknowledgements

The authors are grateful for the financial support from Advanced Materials and
Bioengineering Research centre (AMBER) (Grant No. caffreyn-AMBER 2-PF-Pillar-
phD4) and Science Foundation Ireland (SFI). N.M.C. was supported by a Science
Foundation Ireland Starting Investigator Research Grant (15/SIRG/3314). S.R.P.
wishes to acknowledge funding from the Irish Research Council under the
Laureate awards programme. Computational resources have been provided by
Trinity Centre for High Performance Computing (TCHPC) and by Irish Centre for
High-End Computing (ICHEC).

## References

  * [1] Deji Akinwande, Christopher J Brennan, J Scott Bunch, Philip Egberts, Jonathan R Felts, Huajian Gao, Rui Huang, Joon-Seok Kim, Teng Li, Yao Li, et al.  A review on mechanics and mechanical properties of 2d materials—graphene and beyond.  Extreme Mechanics Letters, 13:42–77, 2017.
  * [2] Jin Myung Kim, Chullhee Cho, Ezekiel Y Hsieh, and SungWoo Nam.  Heterogeneous deformation of two-dimensional materials for emerging functionalities.  Journal of materials research, 35(11):1369–1385, 2020.
  * [3] Kostya S Novoselov, D Jiang, F Schedin, TJ Booth, VV Khotkevich, SV Morozov, and Andre K Geim.  Two-dimensional atomic crystals.  Proceedings of the National Academy of Sciences, 102(30):10451–10453, 2005.
  * [4] Mikhail I Katsnelson.  Graphene: carbon in two dimensions.  Materials today, 10(1-2):20–27, 2007.
  * [5] Changgu Lee, Xiaoding Wei, Jeffrey W Kysar, and James Hone.  Measurement of the elastic properties and intrinsic strength of monolayer graphene.  science, 321(5887):385–388, 2008.
  * [6] Charalampos Androulidakis, Kaihao Zhang, Matthew Robertson, and Sameh Tawfick.  Tailoring the mechanical properties of 2d materials and heterostructures.  2D Materials, 5(3):032005, 2018.
  * [7] Hong Wang, Fucai Liu, Wei Fu, Zheyu Fang, Wu Zhou, and Zheng Liu.  Two-dimensional heterostructures: fabrication, characterization, and application.  Nanoscale, 6(21):12250–12272, 2014.
  * [8] Kai Liu, Qimin Yan, Michelle Chen, Wen Fan, Yinghui Sun, Joonki Suh, Deyi Fu, Sangwook Lee, Jian Zhou, Sefaattin Tongay, et al.  Elastic properties of chemical-vapor-deposited monolayer mos2, ws2, and their bilayer heterostructures.  Nano letters, 14(9):5097–5103, 2014.
  * [9] Kostya S Novoselov, Andre K Geim, Sergei V Morozov, De-eng Jiang, Yanshui Zhang, Sergey V Dubonos, Irina V Grigorieva, and Alexandr A Firsov.  Electric field effect in atomically thin carbon films.  science, 306(5696):666–669, 2004.
  * [10] Taisuke Ohta, Aaron Bostwick, Thomas Seyller, Karsten Horn, and Eli Rotenberg.  Controlling the electronic structure of bilayer graphene.  Science, 313(5789):951–954, 2006.
  * [11] Edward McCann and Mikito Koshino.  The electronic properties of bilayer graphene.  Reports on Progress in physics, 76(5):056503, 2013.
  * [12] Alexandr Vladimirovich Rozhkov, AO Sboychakov, AL Rakhmanov, and Franco Nori.  Electronic properties of graphene-based bilayer systems.  Physics Reports, 648:1–104, 2016.
  * [13] Jon-Hsu Ho, CL Lu, CC Hwang, CP Chang, and Min-Fa Lin.  Coulomb excitations in aa-and ab-stacked bilayer graphites.  Physical Review B, 74(8):085406, 2006.
  * [14] Eduardo V Castro, KS Novoselov, SV Morozov, NMR Peres, JMB Lopes Dos Santos, Johan Nilsson, F Guinea, AK Geim, and AH Castro Neto.  Biased bilayer graphene: semiconductor with a gap tunable by the electric field effect.  Physical review letters, 99(21):216802, 2007.
  * [15] EL Silva, MC Santos, JM Skelton, Tao Yang, T Santos, SC Parker, and A Walsh.  Electronic and phonon instabilities in bilayer graphene under applied external bias.  Materials Today: Proceedings, 20:373–382, 2020.
  * [16] Swastibrata Bhattacharyya and Abhishek K Singh.  Lifshitz transition and modulation of electronic and transport properties of bilayer graphene by sliding and applied normal compressive strain.  Carbon, 99:432–438, 2016.
  * [17] Young-Woo Son, Seon-Myeong Choi, Yoon Pyo Hong, Sungjong Woo, and Seung-Hoon Jhi.  Electronic topological transition in sliding bilayer graphene.  Physical Review B, 84(15):155410, 2011.
  * [18] JMB Lopes Dos Santos, NMR Peres, and AH Castro Neto.  Graphene bilayer with a twist: Electronic structure.  Physical review letters, 99(25):256802, 2007.
  * [19] S Shallcross, S Sharma, E Kandelaki, and OA Pankratov.  Electronic structure of turbostratic graphene.  Physical Review B, 81(16):165105, 2010.
  * [20] Shuyang Dai, Yang Xiang, and David J Srolovitz.  Twisted bilayer graphene: Moiré with a twist.  Nano letters, 16(9):5923–5927, 2016.
  * [21] Leo J McGilly, Alexander Kerelsky, Nathan R Finney, Konstantin Shapovalov, En-Min Shih, Augusto Ghiotto, Yihang Zeng, Samuel L Moore, Wenjing Wu, Yusong Bai, et al.  Visualization of moiré superlattices.  Nature Nanotechnology, 15(7):580–584, 2020.
  * [22] Zhen Hua Ni, Ting Yu, Yun Hao Lu, Ying Ying Wang, Yuan Ping Feng, and Ze Xiang Shen.  Uniaxial strain on graphene: Raman spectroscopy study and band-gap opening.  ACS nano, 2(11):2301–2305, 2008.
  * [23] Vitor M Pereira, AH Castro Neto, and NMR Peres.  Tight-binding approach to uniaxial strain in graphene.  Physical Review B, 80(4):045401, 2009.
  * [24] Zhonghua Ni, Hao Bu, Min Zou, Hong Yi, Kedong Bi, and Yunfei Chen.  Anisotropic mechanical properties of graphene sheets from molecular dynamics.  Physica B: Condensed Matter, 405(5):1301–1306, 2010.
  * [25] FMD Pellegrino, GGN Angilella, and R Pucci.  Strain effect on the optical conductivity of graphene.  Physical Review B, 81(3):035411, 2010.
  * [26] Marcin Mucha-Kruczyński, Igor L Aleiner, and Vladimir I Fal’ko.  Strained bilayer graphene: Band structure topology and landau level spectrum.  Physical Review B, 84(4):041404, 2011.
  * [27] Diana A Gradinar, Henning Schomerus, and Vladimir I Fal’Ko.  Conductance anomaly near the lifshitz transition in strained bilayer graphene.  Physical Review B, 85(16):165429, 2012.
  * [28] Fernando de Juan, Mauricio Sturla, and Maria AH Vozmediano.  Space dependent fermi velocity in strained graphene.  Physical review letters, 108(22):227205, 2012.
  * [29] Hang Zhang, Jhao-Wun Huang, Jairo Velasco Jr, Kevin Myhro, Matt Maldonado, David Dung Tran, Zeng Zhao, Fenglin Wang, Yongjin Lee, Gang Liu, et al.  Transport in suspended monolayer and bilayer graphene under strain: a new platform for material studies.  Carbon, 69:336–341, 2014.
  * [30] Mikkel Settnes, Stephen R. Power, Mads Brandbyge, and Antti-Pekka Jauho.  Graphene nanobubbles as valley filters and beam splitters.  Phys. Rev. Lett., 117:276801, Dec 2016.
  * [31] Zachary H Aitken and Rui Huang.  Effects of mismatch strain and substrate surface corrugation on morphology of supported monolayer graphene.  Journal of Applied Physics, 107(12):123531, 2010.
  * [32] Shengxue Yang, Yujia Chen, and Chengbao Jiang.  Strain engineering of two-dimensional materials: Methods, properties, and applications.  InfoMat, 3(4):397–420, 2021.
  * [33] Ting Yu, Zhenhua Ni, Chaoling Du, Yumeng You, Yingying Wang, and Zexiang Shen.  Raman mapping investigation of graphene on transparent flexible substrate: the strain effect.  The Journal of Physical Chemistry C, 112(33):12602–12605, 2008\.
  * [34] Mingyuan Huang, Hugen Yan, Changyao Chen, Daohua Song, Tony F Heinz, and James Hone.  Phonon softening and crystallographic orientation of strained graphene studied by raman spectroscopy.  Proceedings of the National Academy of Sciences, 106(18):7304–7308, 2009.
  * [35] Charalampos Androulidakis, Emmanuel N Koukaras, George Paterakis, George Trakakis, and Costas Galiotis.  Tunable macroscale structural superlubricity in two-layer graphene via strain engineering.  Nature communications, 11(1):1–11, 2020.
  * [36] Otakar Frank, Milan Bousa, Ibtsam Riaz, Rashid Jalil, Kostya S Novoselov, Georgia Tsoukleri, John Parthenios, Ladislav Kavan, Konstantinos Papagelis, and Costas Galiotis.  Phonon and structural changes in deformed bernal stacked bilayer graphene.  Nano Letters, 12(2):687–693, 2012.
  * [37] Kunqi Wang, Wengen Ouyang, Wei Cao, Ming Ma, and Quanshui Zheng.  Robust superlubricity by strain engineering.  Nanoscale, 11(5):2186–2193, 2019.
  * [38] Seon-Myeong Choi, Seung-Hoon Jhi, and Young-Woo Son.  Controlling energy gap of bilayer graphene by strain.  Nano letters, 10(9):3486–3489, 2010.
  * [39] Matthias Van der Donck, C De Beule, B Partoens, FM Peeters, and B Van Duppen.  Piezoelectricity in asymmetrically strained bilayer graphene.  2D Materials, 3(3):035015, 2016.
  * [40] Alireza Parhizkar and Victor Galitski.  Strained bilayer graphene, emergent energy scales, and moire gravity.  Physical Review Research, 4(2):L022027, 2022.
  * [41] JA Crosse.  Strain-dependent conductivity in biased bilayer graphene.  Physical Review B, 90(23):235403, 2014.
  * [42] Rafi Bistritzer and Allan H MacDonald.  Moiré bands in twisted double-layer graphene.  Proceedings of the National Academy of Sciences, 108(30):12233–12237, 2011.
  * [43] Georg Kresse and Jürgen Hafner.  Ab initio molecular dynamics for liquid metals.  Physical review B, 47(1):558, 1993.
  * [44] Georg Kresse and Jürgen Hafner.  Ab initio molecular-dynamics simulation of the liquid-metal–amorphous-semiconductor transition in germanium.  Physical Review B, 49(20):14251, 1994.
  * [45] Georg Kresse and Jürgen Furthmüller.  Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set.  Computational materials science, 6(1):15–50, 1996.
  * [46] Georg Kresse and Jürgen Furthmüller.  Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set.  Physical review B, 54(16):11169, 1996.
  * [47] Peter E Blöchl.  Projector augmented-wave method.  Physical review B, 50(24):17953, 1994.
  * [48] Georg Kresse and Daniel Joubert.  From ultrasoft pseudopotentials to the projector augmented-wave method.  Physical review b, 59(3):1758, 1999.
  * [49] John P Perdew, Kieron Burke, and Matthias Ernzerhof.  Generalized gradient approximation made simple.  Physical review letters, 77(18):3865, 1996.
  * [50] Stefan Grimme.  Semiempirical gga-type density functional constructed with a long-range dispersion correction.  Journal of computational chemistry, 27(15):1787–1799, 2006.
  * [51] Wang Tao, Guo Qing, Liu Yan, and Sheng Kuang.  A comparative investigation of an ab-and aa-stacked bilayer graphene sheet under an applied electric field: a density functional theory study.  Chinese Physics B, 21(6):067301, 2012.
  * [52] Jae-Kap Lee, Seung-Cheol Lee, Jae-Pyoung Ahn, Soo-Chul Kim, John IB Wilson, and Phillip John.  The growth of aa graphite on (111) diamond.  The Journal of chemical physics, 129(23):234709, 2008.
  * [53] JR Dahn, Rosamaria Fong, and MJ Spoon.  Suppression of staging in lithium-intercalated carbon by disorder in the host.  Physical Review B, 42(10):6424, 1990.
  * [54] Hassan Raza.  Edge and passivation effects in armchair graphene nanoribbons.  Physical Review B, 84(16):165425, 2011.
  * [55] Kyoko Nakada, Mitsutaka Fujita, Gene Dresselhaus, and Mildred S Dresselhaus.  Edge state in graphene ribbons: Nanometer size effect and edge shape dependence.  Physical Review B, 54(24):17954, 1996.
  * [56] Mitsutaka Fujita, Katsunori Wakabayashi, Kyoko Nakada, and Koichi Kusakabe.  Peculiar localized state at zigzag graphite edge.  Journal of the Physical Society of Japan, 65(7):1920–1923, 1996\.
  * [57] Katsunori Wakabayashi, Mitsutaka Fujita, Hiroshi Ajiki, and Manfred Sigrist.  Electronic and magnetic properties of nanographite ribbons.  Physical Review B, 59(12):8271, 1999.
  * [58] Pablo San-Jose and Elsa Prada.  Helical networks in twisted bilayer graphene under interlayer bias.  Physical Review B, 88(12):121408, 2013.
  * [59] Dmitry K. Efimkin and Allan H. MacDonald.  Helical network model for twisted bilayer graphene.  Phys. Rev. B, 98:035404, 2018.
  * [60] Shengqiang Huang, Kyounghwan Kim, Dmitry K. Efimkin, Timothy Lovorn, Takashi Taniguchi, Kenji Watanabe, Allan H. MacDonald, Emanuel Tutuc, and Brian J. LeRoy.  Topologically protected helical states in minimally twisted bilayer graphene.  Phys. Rev. Lett., 121:037702, 2018.
  * [61] Petr Stepanov, Ipsita Das, Xiaobo Lu, Ali Fahimniya, Kenji Watanabe, Takashi Taniguchi, Frank HL Koppens, Johannes Lischner, Leonid Levitov, and Dmitri K Efetov.  Untying the insulating and superconducting orders in magic-angle graphene.  Nature, 583(7816):375–378, 2020.
  * [62] Fernando Gargiulo and Oleg V Yazyev.  Structural and electronic transformation in low-angle twisted bilayer graphene.  2D Materials, 5(1):015019, 2017.
  * [63] Stephen Carr, Daniel Massatt, Steven B. Torrisi, Paul Cazeaux, Mitchell Luskin, and Efthimios Kaxiras.  Relaxation and domain formation in incommensurate two-dimensional heterostructures.  Phys. Rev. B, 98:224102, 2018.
  * [64] Nathanael P Kazmierczak, Madeline Van Winkle, Colin Ophus, Karen C Bustillo, Stephen Carr, Hamish G Brown, Jim Ciston, Takashi Taniguchi, Kenji Watanabe, and D Kwabena Bediako.  Strain fields in twisted bilayer graphene.  Nature materials, 20(7):956–963, 2021.
  * [65] Nguyen N. T. Nam and Mikito Koshino.  Lattice relaxation and energy band modulation in twisted bilayer graphene.  Phys. Rev. B, 96(12):075311, 2017.


# Synthetic Alone: Exploring the Dark Side of Synthetic Data for Grammatical
Error Correction

Chanjun Park  Seonmin Koo  Seolhwa Lee  Jaehyung Seo  Sugyeong Eo  Hyeonseok
Moon  Heuiseok Lim

###### Abstract

Data-centric AI approach aims to enhance the model performance without
modifying the model and has been shown to impact model performance positively.
While recent attention has been given to data-centric AI based on synthetic
data, due to its potential for performance improvement, data-centric AI has
long been exclusively validated using real-world data and publicly available
benchmark datasets. In respect of this, data-centric AI still highly depends
on real-world data, and the verification of models using synthetic data has
not yet been thoroughly carried out. Given the challenges above, we ask the
question: “Does data quality control (noise injection and balanced data), a
data-centric AI methodology acclaimed to have a positive impact, exhibit the
same positive impact in models trained solely with synthetic data?” To address
this question, we conducted comparative analyses between models trained on
synthetic and real-world data based on grammatical error correction (GEC)
task. Our experimental results reveal that the data quality control method has
a positive impact on models trained with real-world data, as previously
reported in existing studies, while a negative impact is observed in models
trained solely on synthetic data.

Machine Learning, ICML

##  1 Introduction

Data-centric AI research has been actively conducted in natural language
processing (NLP) to improve model performance without the need for significant
cost and model modification. Several data-centric AI methods have been
developed to achieve this goal, such as data management (Choi & Park, 2023),
data filtering (Koehn et al., 2020), noise injection (perturbation) (Sarp et
al., 2021; Partovyan et al., 2018), and data augmentation (Shorten &
Khoshgoftaar, 2019). Among these methods, the use of synthetic data
(Nikolenko, 2019) has gained increasing interest with the development of the
large language models (LLMs), such as GPT3 (Brown et al., 2020),
ChatGPT111https://chat.openai.com/, and LaMDA (Thoppilan et al., 2022), These
LLMs have demonstrated the potential for generating high-quality synthetic
data (Chen et al., 2023; Wang et al., 2021) and the possibility of replacing
the need for human-annotated data with synthetic data.

Correct sentence |  | 바비큐 그릴도 이용할 수 있나요?
---
(Can I also use a barbecue grill?)
Example of separation error |  | ㅂㅏㅂㅣ큐 그릴도 이용할 수 있나요?
---
(Can I also use a b ar b e cue grill?)
Example of vowel alteration error |  | 바비큐 그릴됴 이용할 수 있나요?
---
(Can I alsa use a barbecue grill?)
Example of pronunciation error |  | 바비큐 그릴도 이용할 수 잇나요?
---
(Canai also use a barbecue grill?)
Example of punctuation attachment errors |  | 바비큐 그릴도. 이용할 수 있나요?
---
(Can I also. use a barbecue grill?)
Example of loanword error |  | 비비큐 그릴도 이용할 수 있나요?
---
(Can I also use a bebecue grill?)
Example of neologism error |  | 바비큐 그릴도 O.l용할 수 있나요?
---
(Can 1 also use a barbecue grill?)

Table 1: Example of noise injection according to noise type.

However, we raise questions on the validity of prior research in data-centric
AI, which has been shown to impact model performance positively. Existing
data-centric AI studies have basically been conducted based on human-annotated
data or publicly open data. Still, validation of models using only synthetic
data has not been sufficiently conducted (Polyzotis & Zaharia, 2021; Mazumder
et al., 2022).

In the data-centric AI research, studies have focused on efficient methods of
generating synthetic data (Park et al., 2021a) and human-like data (Moon et
al., 2022). However, there has been limited validation of model performance
improvement through data quality control using fully synthetic data. This
paper analyzes whether a model trained only on synthetic data rather than
human-annotated data can still demonstrate a positive impact in a data-centric
approach.

To do this, we employ the grammatical error correction (GEC) task as it is one
of the closely related tasks to the real-world. We conduct experiments on the
GEC task using two models: (1) a BackTranscription (BTS) (Park et al.,
2021b)-based GEC model, which is a synthetic data generation method proposed
in recent speech recognition post-processing, and (2) a GEC model for learning
from real-world data (Park et al., 2020). To analyze the impact of data
quality control on performance, we apply methods such as noise injection
(Ivanovs et al., 2021) and balanced data (Park et al., 2022a; Chen et al.,
2021) to both models and compare their results.

Moreover, there have been studies on the effectiveness of synthetic data based
on self-supervised learning (Ng et al., 2020; Ruiter et al., 2021; Gan et al.,
2021). Also, through the scaling law that examines the performance of models
based on the size of the dataset, it has been demonstrated to be highly
effective to use data generated by models that increase the amount of data
geometrically concerning the size of the dataset (Jaiswal et al., 2020;
Raghunathan, 2021; Kaplan et al., 2020). In respect of this, we aim to revisit
the synthetic data research.

##  2 Design for Revisiting the impact of Synthetic Data

We raise the following question to revisit the impact of synthetic data—“Does
the data quality control manifest the same positive impact in models trained
only on synthetic data?”. To validate this question, we design the experiments
from two different perspectives. We investigate the following questions
through comparative analyses between the performance of the models trained on
synthetic and real-world data.

Figure 1: Experimental results of noise injection. (a) is the result of
inserting noise into real-world data. (b) is the result of inserting noise
into synthetic data. Note that the x-axis indicates the strength of noise
injection.

  * •

How does the strength of noise injection impact the model performance?

To address this question, we propose employing the noise injection method,
more specifically perturbation, which is representative of the data quality
control method, to assess the performance between synthetic and real-world
data.

For the perturbation of synthetic data BTS, six types of noise—i.e.
separation, vowel alteration, pronunciation, punctuation attachment, loanword,
and neologism errors—are applied to the source sentence. The separation error
refers to the case where the consonant and vowels of a character are
separated. The vowel alteration error is where the vowel of a character is
replaced with a different vowel. The pronunciation error indicates a case
where a character is altered by pronunciation. The punctuation attachment
error refers to a case where punctuation is attached in an unnecessary
position within a sentence. The loanword conversion error deals with cases
where part of a character is converted into English. The neologism error
refers to a case where the character is altered using grammar not included in
the existing grammatical system. See detailed examples in Table 1.

Regarding perturbation for real-world data Lang-8, the same correct sentence
pair is inserted into the source and target sentences. Due to the
characteristics of the GEC task, the source sentence already contains errors,
so it is considered noise to insert clean data into the source sentence, which
should have noise that is opposed to the characteristics of the data.

Subsequently, the performance comparison between the baseline model trained on
data without noise injection and the noised model trained on data with
perturbation is conducted to examine the impact of noise injection on the
model concerning synthetic and real-world data. Specifically, we conduct the
experiment according to the strength of noise injection ranging from 0.1 to
1.0. Noise is inserted based on the ratio of noise set at the word level for
each sentence. For example, if the noise ratio is 1.0, the noise will occur in
all words within each sentence.

  * •

How does the ratio of noised and cleaned text batches impact the model
performance?

The aim is to obtain answers to the question by comparing the performance of
synthetic data and real-world data by applying the balanced data method. The
balanced data method is a method of training by intentionally giving
appropriate ratios to noise and clean data. That is, when forming batches
during training, data with different features is composed based on the pre-set
ratios, and the training method is performed based on these ratios Park et al.
(2022a). The performance experiment of the balance between clean and noisy
data is conducted with five different ratios of synthetic and real-world
data—5:5, 4:6, 3:7, 2:8, and 1:9. The comparison of the performance between
the baseline model without any operations and the model trained with balanced
data method is then carried out to analyze the impact of the ratio of noise
and clean on the performance of the only synthetic and real-world data-based
models.

##  3 Experimental Settings

#### Real-world & Synthetic Data

We use the Lang-8 dataset222https://lang-8.com/ as our real-world data, a
fully human-annotated corpus. The data settings are consistent with those used
by Park et al. (2020).

We generate synthesized datasets fitting for the GEC task from the above
datasets (AI-HUB, TED) using BTS (Park et al., 2021b). BTS combines text-to-
speech (TTS) technology and speech-to-text (STT) technology to generate GEC
task synthesized data for speech recognition post-processor. Although BTS
cannot represent synthetic data, BTS is a simple and efficient methodology for
generating synthetic data. Thus, we use it for experiments. As raw data for
generating BTS-based synthetic data, we use AI-HUB (Park et al., 2022b), which
are representative Korean data platforms, and TED Korean
dataset333https://www.ted.com/talks?language=ko, the same as existing BTS
work. In addition, since the existing BTS research was also conducted in
Korean, this experiment also performs based on Korean for a fair evaluation.

Table 2 shows the specific data statistics used in the experiment. We use
92,000 sentences from AI-HUB data and 119,883 sentences from TED’s Korean
Transcript data to generate BTS-based synthetic data followed by Park et al.
(2020)’s method. These data are used as raw data for BTS, transformation into
speech using TTS, and outputting the converted result as text using STT.

Dataset | Train | Test | Type of data
---|---|---|---
Lang-8 | 1,075,513 | 631 | real-world
AI-HUB | 92,000 | 3,000 | synthetic(BTS based)
TED | 119,883 | 3,000 | synthetic(BTS based)
Table 2: Statistics on the number of sentences according to real-world and
synthetic data.

#### Implementation Details

We train the models using the vanilla Transformer (Vaswani et al., 2017) and
set the same for hyperparameters. Fairseq (Ott et al., 2019) is used for the
implementation. For subword tokenization, we utilize SentencePiece (Kudo &
Richardson, 2018) and set the vocabulary size to 50,000. We evaluate the
Lang-8-based real-world model and the BTS-based synthetic data model as GLEU
(Napoles et al., 2015) and BLEU (Papineni et al., 2002), respectively. These
are the same metrics as previous GEC and BTS papers.

Figure 2: Experimental results of balanced data. (a) is the balanced data
result between noise and clean data in real-world data. (b) is the balanced
data result between noise and clean data in synthetic data. Note that the
x-axis indicates the ratio as (clean:noise).

##  4 Experimental Results

###  4.1 Results of Question 1: Noise Injection

Figure 1 shows the results of applying noise injection (perturbation) methods.
Baseline for the real-world data (a) is 55.60. The performance tends to
improve when injecting noise. Mainly, the 0.4 noise ratio result obtains a
substantial gain of +2.44 to 58.04. Meanwhile, (b) indicates the experimental
results of synthetic data on AI-HUB and TED datasets.

We denote the probability of noise being injected into a token in a sentence
as the noise injection ratio. The baselines of AI-HUB and TED report 65.69 and
56.14, respectively, and models learned with the data subject to noise
injection show performance degradation in most cases. A marginal performance
gain is recorded when the noise ratio of the TED dataset is 0.2; rather, the
overall performance decreased except for this case. The results demonstrate
that data quality control through noise injection, which is known to impact
performance in many studies positively, has a negative impact when training a
model using only synthetic data.

These results starkly contrast with the experimental results using models
trained entirely of real-world data, highlighting our conclusion of the
negative impact of the noise injection on the synthetic data. Namely, we
recommend applying synthetic data after extensively verifying whether the
data-centric AI methods are effective in a synthetic data-only setting.

###  4.2 Results of Question 2: Balanced Data

The experimental results of applying balanced data methods are described in
Figure 2. As mentioned, we combine clean and noise data with a five ratio. The
noise injection ratio on the noisy data is 1.0, the highest noise for Section
4.1.

(a) is the experimental results for real-world data. Contrary to the synthetic
data, all cases perform better than the baseline. The performance on the ratio
of 1:9 is 58.84, showing a gain of 3.24 points from the baseline. As
previously demonstrated, we confirm that leveraging data quality control
techniques to real data positively impacts the model.

(b) is the result of the model training with synthetic data by intentionally
giving ratios to clean and noise. As a result, the model learned with
synthetic data performs worse than the baseline in all ratios. In particular,
as the ratio of noise data increases, performance tends to deteriorate.
Experimental results show a negative effect overall, and we analyze that data
quality control is less effective in model training in an environment
consisting only of synthetic data.

We conclude that data quality control positively affects real-world data-based
models but does not always guarantee a positive effect in an environment
consisting entirely of synthetic data. This implies that real-world and
synthetic data have distinctly contrasting characteristics and should be
treated differently. We argue that beyond effective synthetic data generation,
which is the focus of the data-centric AI, data quality control methods should
be inspected to ensure that models leveraging synthetic data produce
sufficient performance.

##  5 Conclusion and Future Work

In this work, we address the research question of “Is the data quality
control, a data-centric AI methodology known to have a positive effect, still
observed to have a positive impact when the models are trained only using
synthetic data?” by performing experiments and evaluating the results. Our
experiments reveal that while the conventional data-centric approach
positively impacted real-world data, models trained solely on synthetic data
showed a negative impact. This demonstrates that data-centric methodologies do
not necessarily guarantee positive effects, dependent on the characteristics
of the data. Based on the results, it was found that sufficient evaluation of
data-centric methods in synthetic data environments is needed. The
experimental results cannot be generalized since the experiment was limited to
the GEC task and not all data-centric AI methodologies were tested. However, a
clearly defined research question was addressed, and insightful results were
obtained through a structured comparison. In future work, we aim to further
analyze the characteristics of synthetic data environments by implementing
data-centric approaches other than noise injection and balanced data.

## Acknowledgements

This research was supported by the MSIT(Ministry of Science and ICT), Korea,
under the ITRC(Information Technology Research Center) support
program(IITP-2023-2018-0-01405) supervised by the IITP(Institute for
Information & Communications Technology Planning & Evaluation), and This work
was supported by the National Research Foundation of Korea(NRF) grant funded
by the Korea government(MSIT)(No. 2022R1A5A7026673).

## References

  * Brown et al. (2020) Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Chen et al. (2021) Chen, L., Wan, S., and Dou, L.  Improving diagnostic performance of high-voltage circuit breakers on imbalanced data using an oversampling method.  _IEEE Transactions on Power Delivery_ , 37(4):2704–2716, 2021.
  * Chen et al. (2023) Chen, M., Papangelis, A., Tao, C., Kim, S., Rosenbaum, A., Liu, Y., Yu, Z., and Hakkani-Tur, D.  Places: Prompting language models for social conversation synthesis.  _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_ , 2023.
  * Choi & Park (2023) Choi, E. and Park, C.  Dmops: Data management operation and recipes.  _arXiv preprint arXiv:2301.01228_ , 2023.
  * Gan et al. (2021) Gan, Z., Xu, H., and Zan, H.  Self-supervised curriculum learning for spelling error correction.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pp. 3487–3494, 2021.
  * Ivanovs et al. (2021) Ivanovs, M., Kadikis, R., and Ozols, K.  Perturbation-based methods for explaining deep neural networks: A survey.  _Pattern Recognition Letters_ , 150:228–234, 2021.
  * Jaiswal et al. (2020) Jaiswal, A., Babu, A. R., Zadeh, M. Z., Banerjee, D., and Makedon, F.  A survey on contrastive self-supervised learning.  _Technologies_ , 9(1):2, 2020.
  * Kaplan et al. (2020) Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D.  Scaling laws for neural language models.  _arXiv preprint arXiv:2001.08361_ , 2020.
  * Koehn et al. (2020) Koehn, P., Chaudhary, V., El-Kishky, A., Goyal, N., Chen, P.-J., and Guzmán, F.  Findings of the wmt 2020 shared task on parallel corpus filtering and alignment.  In _Proceedings of the Fifth Conference on Machine Translation_ , pp. 726–742, 2020.
  * Kudo & Richardson (2018) Kudo, T. and Richardson, J.  Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.  _arXiv preprint arXiv:1808.06226_ , 2018.
  * Mazumder et al. (2022) Mazumder, M., Banbury, C., Yao, X., Karlaš, B., Rojas, W. G., Diamos, S., Diamos, G., He, L., Kiela, D., Jurado, D., et al.  Dataperf: Benchmarks for data-centric ai development.  _arXiv preprint arXiv:2207.10062_ , 2022.
  * Moon et al. (2022) Moon, H., Park, C., Seo, J., Eo, S., and Lim, H.  An automatic post editing with efficient and simple data generation method.  _IEEE Access_ , 10:21032–21040, 2022.
  * Napoles et al. (2015) Napoles, C., Sakaguchi, K., Post, M., and Tetreault, J.  Ground truth for grammatical error correction metrics.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pp. 588–593, 2015.
  * Ng et al. (2020) Ng, N., Cho, K., and Ghassemi, M.  SSMBA: Self-supervised manifold based data augmentation for improving out-of-domain robustness.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pp. 1268–1283, Online, November 2020\. Association for Computational Linguistics.  doi: 10.18653/v1/2020.emnlp-main.97.  URL https://aclanthology.org/2020.emnlp-main.97.
  * Nikolenko (2019) Nikolenko, S. I.  Synthetic data for deep learning.  _arXiv preprint arXiv:1909.11512_ , 2019.
  * Ott et al. (2019) Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., and Auli, M.  fairseq: A fast, extensible toolkit for sequence modeling.  _arXiv preprint arXiv:1904.01038_ , 2019.
  * Papineni et al. (2002) Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_ , pp. 311–318, 2002.
  * Park et al. (2020) Park, C., Yang, Y., Lee, C., and Lim, H.  Comparison of the evaluation metrics for neural grammatical error correction with overcorrection.  _IEEE Access_ , 8:106264–106272, 2020.
  * Park et al. (2021a) Park, C., Lee, S., Moon, H., Eo, S., Seo, J., and Lim, H.  How should human translation coexist with nmt? efficient tool for building high quality parallel corpus.  _arXiv preprint arXiv:2111.00191_ , 2021a.
  * Park et al. (2021b) Park, C., Seo, J., Lee, S., Lee, C., Moon, H., Eo, S., and Lim, H.-S.  Bts: Back transcription for speech-to-text post-processor using text-to-speech-to-text.  In _Proceedings of the 8th Workshop on Asian Translation (WAT2021)_ , pp. 106–116, 2021b.
  * Park et al. (2022a) Park, C., Go, W.-Y., Eo, S., Moon, H., Lee, S., and Lim, H.  Mimicking infants’ bilingual language acquisition for domain specialized neural machine translation.  _IEEE Access_ , 10:38684–38693, 2022a.
  * Park et al. (2022b) Park, C., Shim, M., Eo, S., Lee, S., Seo, J., Moon, H., and Lim, H.  Empirical analysis of parallel corpora and in-depth analysis using liwc.  _Applied Sciences_ , 12(11):5545, 2022b.
  * Partovyan et al. (2018) Partovyan, A., Nourani, V., and ALAMI, M. T.  Noise injection–denoising techniques to improve artificial intelligence-based rainfall–runoff modeling.  _Water Resources Engineering_ , 11(36):81–94, 2018.
  * Polyzotis & Zaharia (2021) Polyzotis, N. and Zaharia, M.  What can data-centric ai learn from data and ml engineering?  _arXiv preprint arXiv:2112.06439_ , 2021.
  * Raghunathan (2021) Raghunathan, T. E.  Synthetic data.  _Annual review of statistics and its application_ , 8:129–140, 2021.
  * Ruiter et al. (2021) Ruiter, D., Klakow, D., van Genabith, J., and España-Bonet, C.  Integrating unsupervised data generation into self-supervised neural machine translation for low-resource languages.  In _Proceedings of Machine Translation Summit XVIII: Research Track_ , pp. 76–91, Virtual, August 2021. Association for Machine Translation in the Americas.  URL https://aclanthology.org/2021.mtsummit-research.7.
  * Sarp et al. (2021) Sarp, S., Kuzlu, M., Cali, U., Elma, O., and Guler, O.  Analysis of false data injection impact on ai based solar photovoltaic power generation forecasting.  _arXiv preprint arXiv:2110.09948_ , 2021.
  * Shorten & Khoshgoftaar (2019) Shorten, C. and Khoshgoftaar, T. M.  A survey on image data augmentation for deep learning.  _Journal of big data_ , 6(1):1–48, 2019.
  * Thoppilan et al. (2022) Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al.  Lamda: Language models for dialog applications.  _arXiv preprint arXiv:2201.08239_ , 2022.
  * Vaswani et al. (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Wang et al. (2021) Wang, Z., Yu, A. W., Firat, O., and Cao, Y.  Towards zero-label language learning.  _arXiv preprint arXiv:2109.09193_ , 2021.


Therefore by taking the $L^{q}({\mu})$-norm of both sides of (5)(with respect
to the variable $y$), and then using Minkowski’s inequality($q\geq 1$), (5)
and (5), we have

$\begin{split}&\Big{\|}\nu_{\vec{w}}(B(y,r))^{1/{\alpha}-m-1/q}\cdot\Big{[}\nu_{\vec{w}}\big{(}\big{\\{}x\in
B(y,r):\big{|}\big{[}{b},T_{\theta}\big{]}_{1}(\vec{f})(x)\big{|}>\lambda^{m}\big{\\}}\big{)}\Big{]}^{m}\Big{\|}_{L^{q}({\mu})}\\\
&\leq\Big{\|}J^{0,\dots,0}_{\ast}(y,r)\Big{\|}_{L^{q}({\mu})}+\sum_{(\beta_{1},\dots,\beta_{m})\in\mathfrak{L}}\Big{\|}J^{\beta_{1},\dots,\beta_{m}}_{\ast}(y,r)\Big{\|}_{L^{q}({\mu})}\\\
&\leq
C\bigg{\|}\prod_{k=1}^{m}\bigg{[}w_{k}(B(y,2r))^{1/{\alpha_{k}}-1/{q_{k}}}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{L\log
L(w_{k}),B(y,2r)}\bigg{]}\bigg{\|}_{L^{q}({\mu})}\\\ &+C\cdot
2^{m}\sum_{j=1}^{\infty}\bigg{\|}\prod_{k=1}^{m}\bigg{[}w_{k}\big{(}B(y,2^{j+1}r)\big{)}^{1/{\alpha_{k}}-1/{q_{k}}}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{L\log
L(w_{k}),B(y,2^{j+1}r)}\bigg{]}\bigg{\|}_{L^{q}({\mu})}\\\
&\times\big{(}j+1\big{)}\left(\frac{|B(y,r)|}{|B(y,2^{j+1}r)|}\right)^{\delta(1/{\alpha}-1/q)}.\end{split}$

Another application of Hölder’s inequality gives us that

$\begin{split}&\Big{\|}\nu_{\vec{w}}(B(y,r))^{1/{\alpha}-m-1/q}\cdot\Big{[}\nu_{\vec{w}}\big{(}\big{\\{}x\in
B(y,r):\big{|}\big{[}{b},T_{\theta}\big{]}_{1}(\vec{f})(x)\big{|}>\lambda^{m}\big{\\}}\big{)}\Big{]}^{m}\Big{\|}_{L^{q}(\mu)}\\\
&\leq
C\prod_{k=1}^{m}\bigg{\|}w_{k}(B(y,2r))^{1/{\alpha_{k}}-1/{q_{k}}}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{L\log
L(w_{k}),B(y,2r)}\bigg{\|}_{L^{q_{k}}(\mu)}\\\
&+C\sum_{j=1}^{\infty}\prod_{k=1}^{m}\bigg{\|}w_{k}\big{(}B(y,2^{j+1}r)\big{)}^{1/{\alpha_{k}}-1/{q_{k}}}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{L\log
L(w_{k}),B(y,2^{j+1}r)}\bigg{\|}_{L^{q_{k}}(\mu)}\\\
&\times\big{(}j+1\big{)}\left(\frac{|B(y,r)|}{|B(y,2^{j+1}r)|}\right)^{\delta(1/{\alpha}-1/q)}\\\
&\leq
C\prod_{k=1}^{m}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{(L\log
L,L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)}\\\
&+C\prod_{k=1}^{m}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{(L\log
L,L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)}\times\sum_{j=1}^{\infty}\big{(}j+1\big{)}\left(\frac{|B(y,r)|}{|B(y,2^{j+1}r)|}\right)^{\delta(1/{\alpha}-1/q)}\\\
&\leq
C\prod_{k=1}^{m}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{(L\log
L,L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)},\end{split}$

where the last inequality holds since $\delta(1/{\alpha}-1/q)>0$. This
completes the proof of Theorem 3.4. ∎

For the iterated commutator $\big{[}\Pi\vec{b},T_{\theta}\big{]}$, we can also
establish the following results in the same manner as in Theorems 3.3 and 3.4.
The proof then needs appropriate but minor modifications and we leave this to
the reader.

######  Theorem 5.2.

Let $m\geq 2$ and $\big{[}\Pi\vec{b},T_{\theta}\big{]}$ be the iterated
commutator of $\theta$-type Calderón–Zygmund operator $T_{\theta}$ with
$\theta$ satisfying the condition (1.1) and $\vec{b}\in\mathrm{BMO}^{m}$.
Assume that $1<p_{k}\leq\alpha_{k}<q_{k}<\infty$, $k=1,2,\ldots,m$ and
$p\in(1/m,\infty)$ with $1/p=\sum_{k=1}^{m}1/{p_{k}}$, $q\in[1,\infty)$ with
$1/q=\sum_{k=1}^{m}1/{q_{k}}$ and $1/{\alpha}=\sum_{k=1}^{m}1/{\alpha_{k}};$
$\vec{w}=(w_{1},\ldots,w_{m})\in A_{\vec{P}}$ with $w_{1},\ldots,w_{m}\in
A_{\infty}$ and $\mu\in\Delta_{2}$. Assume further that (3.1) holds. Then
there exists a constant $C>0$ such that for all
$\vec{f}=(f_{1},\ldots,f_{m})\in(L^{p_{1}},L^{q_{1}})^{\alpha_{1}}(w_{1};\mu)\times\cdots\times(L^{p_{m}},L^{q_{m}})^{\alpha_{m}}(w_{m};\mu)$,

$\big{\|}\big{[}\Pi\vec{b},T_{\theta}\big{]}(\vec{f})\big{\|}_{(L^{p},L^{q})^{\alpha}(\nu_{\vec{w}};\mu)}\leq
C\prod_{k=1}^{m}\big{\|}f_{k}\big{\|}_{(L^{p_{k}},L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)}$

with $\nu_{\vec{w}}=\prod_{k=1}^{m}w_{k}^{p/{p_{k}}}$.

######  Theorem 5.3.

Let $m\geq 2$ and $\big{[}\Pi\vec{b},T_{\theta}\big{]}$ be the iterated
commutator of $\theta$-type Calderón–Zygmund operator $T_{\theta}$ with
$\theta$ satisfying the condition (1.9) and $\vec{b}\in\mathrm{BMO}^{m}$.
Assume that $p_{k}=1$, $1\leq\alpha_{k}<q_{k}<\infty$, $k=1,2,\ldots,m$ and
$p=1/m$, $q\in[1,\infty)$ with $1/q=\sum_{k=1}^{m}1/{q_{k}}$ and
$1/{\alpha}=\sum_{k=1}^{m}1/{\alpha_{k}};$ $\vec{w}=(w_{1},\ldots,w_{m})\in
A_{(1,\dots,1)}$ with $w_{1},\ldots,w_{m}\in A_{\infty}$ and
$\mu\in\Delta_{2}$. Assume further that (3.1) holds. Then for any given
$\lambda>0$ and any ball $B(y,r)\subset\mathbb{R}^{n}$ with
$(y,r)\in\mathbb{R}^{n}\times(0,+\infty)$, there exists a constant $C>0$
independent of $\vec{f}=(f_{1},\ldots,f_{m})$, $B(y,r)$ and $\lambda$ such
that

$\begin{split}&\Big{\|}\nu_{\vec{w}}(B(y,r))^{1/{\alpha}-m-1/q}\cdot\Big{[}\nu_{\vec{w}}\Big{(}\Big{\\{}x\in
B(y,r):\big{|}\big{[}\Pi\vec{b},T_{\theta}\big{]}(\vec{f})(x)\big{|}>\lambda^{m}\Big{\\}}\Big{)}\Big{]}^{m}\Big{\|}_{L^{q}(\mu)}\\\
&\leq
C\cdot\prod_{k=1}^{m}\bigg{\|}\Phi^{(m)}\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{(L\log
L,L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)},\end{split}$

where $\nu_{\vec{w}}=\prod_{k=1}^{m}w_{k}^{1/{m}}$,
$\Phi(t)=t\cdot(1+\log^{+}t)$ and
$\Phi^{(m)}=\overbrace{\Phi\circ\cdots\circ\Phi}^{m}$. Here the norm
$\|\cdot\|_{L^{q}(\mu)}$ is taken with respect to the variable $y$, i.e.,

$\begin{split}&\Big{\|}\nu_{\vec{w}}(B(y,r))^{1/{\alpha}-m-1/q}\cdot\Big{[}\nu_{\vec{w}}\Big{(}\Big{\\{}x\in
B(y,r):\big{|}\big{[}\Pi\vec{b},T_{\theta}\big{]}(\vec{f})(x)\big{|}>\lambda^{m}\Big{\\}}\Big{)}\Big{]}^{m}\Big{\|}_{L^{q}(\mu)}\\\
=&\left\\{\int_{\mathbb{R}^{n}}\bigg{[}\nu_{\vec{w}}(B(y,r))^{1/{\alpha}-m-1/q}\cdot\Big{[}\nu_{\vec{w}}\Big{(}\Big{\\{}x\in
B(y,r):\big{|}\big{[}\Pi\vec{b},T_{\theta}\big{]}(\vec{f})(x)\big{|}>\lambda^{m}\Big{\\}}\Big{)}\Big{]}^{m}\bigg{]}^{q}\mu(y)\,dy\right\\}^{1/q}.\end{split}$

######  Remark 5.1.

In light of Theorem 5.3, we can say that the iterated commutator
$\big{[}\Pi\vec{b},T_{\theta}\big{]}$ is bounded from $(L\log
L,L^{q_{1}})^{\alpha_{1}}(w_{1};\mu)\times\cdots\times(L\log
L,L^{q_{m}})^{\alpha_{m}}(w_{m};\mu)$ into
$(WL^{p},L^{q})^{\alpha}(\nu_{\vec{w}};\mu)$ with $p=1/m$.

Finally, in view of the relation (4.18), we have the following results
concerning weighted estimates for the multilinear commutators
$\big{[}\Sigma\vec{b},T_{\theta}\big{]}$ and
$\big{[}\Pi\vec{b},T_{\theta}\big{]}$.

######  Corollary 5.1.

Let $m\geq 2$ and $\vec{b}\in\mathrm{BMO}^{m}$. Assume that
$1<p_{k}\leq\alpha_{k}<q_{k}<\infty$, $k=1,2,\ldots,m$ and $p\in(1/m,\infty)$
with $1/p=\sum_{k=1}^{m}1/{p_{k}}$, $q\in[1,\infty)$ with
$1/q=\sum_{k=1}^{m}1/{q_{k}}$ and $1/{\alpha}=\sum_{k=1}^{m}1/{\alpha_{k}};$
$\vec{w}=(w_{1},\ldots,w_{m})\in\prod_{k=1}^{m}A_{p_{k}}$ and
$\mu\in\Delta_{2}$. In addition, assume that (3.1) holds. Then both the
multilinear commutator $\big{[}\Sigma\vec{b},T_{\theta}\big{]}$ and the
iterated commutator $\big{[}\Pi\vec{b},T_{\theta}\big{]}$ satisfy

$\big{\|}\big{[}\Sigma\vec{b},T_{\theta}\big{]}(\vec{f})\big{\|}_{(L^{p},L^{q})^{\alpha}(\nu_{\vec{w}};\mu)}\leq
C\prod_{k=1}^{m}\big{\|}f_{k}\big{\|}_{(L^{p_{k}},L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)}$

and

$\big{\|}\big{[}\Pi\vec{b},T_{\theta}\big{]}(\vec{f})\big{\|}_{(L^{p},L^{q})^{\alpha}(\nu_{\vec{w}};\mu)}\leq
C\prod_{k=1}^{m}\big{\|}f_{k}\big{\|}_{(L^{p_{k}},L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)}$

with $\nu_{\vec{w}}=\prod_{k=1}^{m}w_{k}^{p/{p_{k}}}$, provided that $\theta$
satisfying the condition (1.1).

######  Corollary 5.2.

Let $m\geq 2$ and $\vec{b}\in\mathrm{BMO}^{m}$. Assume that $p_{k}=1$,
$1\leq\alpha_{k}<q_{k}<\infty$, $k=1,2,\ldots,m$ and $p=1/m$, $q\in[1,\infty)$
with $1/q=\sum_{k=1}^{m}1/{q_{k}}$ and
$1/{\alpha}=\sum_{k=1}^{m}1/{\alpha_{k}};$
$\vec{w}=(w_{1},\ldots,w_{m})\in\prod_{k=1}^{m}A_{1}$ and $\mu\in\Delta_{2}$.
In addition, assume that (3.1) holds. Then for any given $\lambda>0$ and for
any ball $B(y,r)$ with $y\in\mathbb{R}^{n}$ and $r>0$, there exists a constant
$C>0$ independent of $\vec{f}=(f_{1},\ldots,f_{m})$, $B(y,r)$ and $\lambda$
such that $(\nu_{\vec{w}}=\prod_{k=1}^{m}w_{k}^{1/{m}})$

$\begin{split}&\Big{\|}\nu_{\vec{w}}(B(y,r))^{1/{\alpha}-m-1/q}\cdot\Big{[}\nu_{\vec{w}}\Big{(}\Big{\\{}x\in
B(y,r):\big{|}\big{[}\Sigma\vec{b},T_{\theta}\big{]}(\vec{f})(x)\big{|}>\lambda^{m}\Big{\\}}\Big{)}\Big{]}^{m}\Big{\|}_{L^{q}(\mu)}\\\
&\leq
C\cdot\prod_{k=1}^{m}\bigg{\|}\Phi\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{(L\log
L,L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)},\end{split}$

provided that $\theta$ satisfies the condition (1.6), and

$\begin{split}&\Big{\|}\nu_{\vec{w}}(B(y,r))^{1/{\alpha}-m-1/q}\cdot\Big{[}\nu_{\vec{w}}\Big{(}\Big{\\{}x\in
B(y,r):\big{|}\big{[}\Pi\vec{b},T_{\theta}\big{]}(\vec{f})(x)\big{|}>\lambda^{m}\Big{\\}}\Big{)}\Big{]}^{m}\Big{\|}_{L^{q}(\mu)}\\\
&\leq
C\cdot\prod_{k=1}^{m}\bigg{\|}\Phi^{(m)}\bigg{(}\frac{|f_{k}|}{\lambda}\bigg{)}\bigg{\|}_{(L\log
L,L^{q_{k}})^{\alpha_{k}}(w_{k};\mu)},\end{split}$

provided that $\theta$ satisfies the condition (1.9).

### Acknowledgment

This work was supported by the Natural Science Foundation of China (Grant No.
XJEDU2020Y002 and 2022D01C407).

## References

  * [1] D. R. Adams, Morrey Spaces, Lecture notes in applied and numerical harmonic analysis, Birkhäuser/Springer, Cham, 2015.
  * [2] J. Alvarez, R. J. Bagby, D. S. Kurtz and C. Pérez, Weighted estimates for commutators of linear operators, Studia Math, 104(1993), 195–209.
  * [3] J. Bergh and J. Löfström, Interpolation Spaces. An Introduction, Springer–Verlag, 1976.
  * [4] Y. Ding, S. Z. Lu and K. Yabuta, On commutators of Marcinkiewicz integrals with rough kernel, J. Math. Anal. Appl, 275(2002), 60–68.
  * [5] J. Duoandikoetxea, Fourier Analysis, American Mathematical Society, Providence, Rhode Island, 2000.
  * [6] J. Feuto, I. Fofana and K. Koua, Weighted norm inequalities for a maximal operator in some subspace of amalgams, Canad. Math. Bull., 53(2010), 263–277.
  * [7] J. Feuto, Intrinsic square functions on functions spaces including weighted Morrey spaces, Bull. Korean Math. Soc., 50(2013), 1923–1936.
  * [8] J. Feuto, Norm inequalities in generalized Morrey spaces, J. Fourier Anal. Appl., 20(2014), 896–909.
  * [9] J. Feuto, Norm inequalities in some subspaces of Morrey space, Ann. Math. Blaise Pascal, 21(2014), 21–37.
  * [10] I. Fofana, Étude d’une classe d’espace de fonctions contenant les espaces de Lorentz(French), Afrika Mat., 1(1988), 29–50.
  * [11] J.J.F.Fournier and J. Stewart, Amalgams of $L^{p}$ and $l^{q}$, Bull. Amer. Math. Soc.,13(1985), 1–21.
  * [12] J. Garcia-Cuerva and J. L. Rubio de Francia, Weighted Norm Inequalities and Related Topics, North-Holland, Amsterdam, 1985.
  * [13] L. Grafakos, _Classical Fourier Analysis_ , Third Edition, Springer-Verlag, 2014
  * [14] L. Grafakos, _Modern Fourier Analysis_ , Third Edition, Springer-Verlag, 2014.
  * [15] L. Grafakos and R. H. Torres, Multilinear Calderón–Zygmund theory, Adv. Math., 165(2002), 124–164.
  * [16] F. Holland, Harmonic analysis on amalgams of $L^{p}$ and $l^{q}$, J. London Math. Soc., 10(1975), 295–305.
  * [17] F. John and L. Nirenberg, On functions of bounded mean oscillation,Comm. Pure Appl. Math, 14(1961), 415–426.
  * [18] N.Kikuchi, E.Nakai, N.Tomita, K.Yabuta and T.Yoneda, Calderón–Zygmund operators on amalgam spaces and in the discrete case, J. Math. Anal. Appl, 335(2007), 198–212.
  * [19] Y. Komori and S. Shirai, Weighted Morrey spaces and a singular integral operator, Math. Nachr, 282(2009), 219–231.
  * [20] A. K. Lerner, S. Ombrosi, C. Pérez, R. H. Torres and R. Trujillo-González, New maximal functions and multiple weights for the multilinear Calderón–Zygmund theory, Adv. Math., 220(2009), 1222–1264.
  * [21] Z. G. Liu and S. Z. Lu, Endpoint estimates for commutators of Calderón–Zygmund type operators, Kodai Math. J., 25(2002), 79–88.
  * [22] G. Z. Lu and P. Zhang, Multilinear Calderón–Zygmund operators with kernels of Dini’s type and applications, Nonlinear Anal., 107(2014), 92–117.
  * [23] D. Maldonado and V. Naibo, Weighted norm inequalities for paraproducts and bilinear pseudodifferential operators with mild regularity, J. Fourier Anal. Appl., 15(2009), 218–261.
  * [24] C. B. Morrey, On the solutions of quasi-linear elliptic partial differential equations, Trans. Amer. Math. Soc, 43(1938), 126–166.
  * [25] C. Pérez and R. H. Torres, Sharp maximal function estimates for multilinear singular integrals, Contemp. Math., 320(2003), 323–331.
  * [26] C. Pérez, G. Pradolini, R. H. Torres and R. Trujillo-González, _End-point estimates for iterated commutators of multilinear singular integrals_ , Bull. Lond. Math. Soc., 46(2014), 26–42.
  * [27] T. S. Quek and D. C. Yang, Calderón–Zygmund-type operators on weighted weak Hardy spaces over $\mathbb{R}^{n}$, Acta Math. Sinica (Engl. Ser), 16(2000), 141–160.
  * [28] M. M. Rao and Z. D. Ren, Theory of Orlicz Spaces, Marcel Dekker, New York, 1991.
  * [29] E. M. Stein and G. Weiss, Interpolation of operators with change of measures, Trans. Amer. Math. Soc, 87(1958), 159–172.
  * [30] E. M. Stein, Singular Integrals and Differentiability Properties of Functions, Princeton Univ. Press, Princeton, New Jersey, 1970.
  * [31] E. M. Stein, Harmonic Analysis: Real-Variable Methods, Orthogonality, and Oscillatory Integrals, Princeton Univ. Press, Princeton, New Jersey, 1993.
  * [32] H. Wang, Intrinsic square functions on the weighted Morrey spaces, J. Math. Anal. Appl, 396(2012), 302–314.
  * [33] H. Wang, Some estimates for $\theta$-type Calderón–Zygmund operators and linear commutators on certain weighted amalgam spaces, Anal. Theory Appl., 38(2022), 361–393.
  * [34] H. Wang, Estimates for fractional integral operators and linear commutators on certain weighted amalgam spaces, J. Funct. Spaces, 2020, Art. ID 2697104, 25 pp.
  * [35] H. Wang, Estimates for vector-valued intrinsic square functions and their commutators on certain weighted amalgam spaces, Adv. Oper. Theory, 7(2022), Paper No.42, 46 pp.
  * [36] P. W. Wang and Z. G. Liu, Weighted norm inequalities for multilinear Calderón–Zygmund operators in generalized Morrey spaces, J. Inequal. Appl., 2017, Paper No. 48, 10 pp.
  * [37] P. W. Wang and Z. G. Liu, Boundedness of multilinear integral operators and their commutators on generalized Morrey spaces, Adv. Oper. Theory, 2(2017), 269–286.
  * [38] X. M. Wei and S. P. Tao, The boundedness of Littlewood–Paley operators with rough kernels on weighted $(L^{q},L^{p})^{\alpha}(\mathbb{R}^{n})$ spaces, Anal. Theory Appl., 29(2013), 135–148.
  * [39] K. Yabuta, Generalizations of Calderón–Zygmund operators, Studia Math, 82(1985), 17–31.
  * [40] P. Zhang, Weighted endpoint estimates for commutators of Marcinkiewicz integrals, Acta Math. Sinica (Engl. Ser), 26(2010), 1709–1722.
  * [41] P. Zhang and H. Xu, Sharp weighted estimates for commutators of Calderón–Zygmund type operators, Acta Math. Sinica (Chin. Ser), 48(2005), 625–636.