# All-silicon quantum light source by embedding an atomic emissive center in a
nanophotonic cavity

Walid Redjem  Equal contribution  Department of Electrical Engineering and
Computer Sciences, University of California Berkeley, Berkeley, CA 94720, USA
Yertay Zhiyenbayev  Equal contribution  Department of Electrical Engineering
and Computer Sciences, University of California Berkeley, Berkeley, CA 94720,
USA  Wayesh Qarony  Equal contribution  Department of Electrical Engineering
and Computer Sciences, University of California Berkeley, Berkeley, CA 94720,
USA  Vsevolod Ivanov  Accelerator Technology and Applied Physics Division,
Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
Christos Papapanos  Department of Electrical Engineering and Computer
Sciences, University of California Berkeley, Berkeley, CA 94720, USA  Wei Liu
Accelerator Technology and Applied Physics Division, Lawrence Berkeley
National Laboratory, Berkeley, California 94720, USA  Kaushalya Jhuria
Accelerator Technology and Applied Physics Division, Lawrence Berkeley
National Laboratory, Berkeley, California 94720, USA  Zakaria Al Balushi
Department of Materials Science and Engineering, University of California
Berkeley, Berkeley, California 94720, USA  Materials Sciences Division,
Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA  Scott
Dhuey  Molecular Foundry, Lawrence Berkeley National Laboratory, Berkeley,
California 94720, USA  Adam Schwartzberg  Molecular Foundry, Lawrence Berkeley
National Laboratory, Berkeley, California 94720, USA  Liang Tan  Molecular
Foundry, Lawrence Berkeley National Laboratory, Berkeley, California 94720,
USA  Thomas Schenkel  Accelerator Technology and Applied Physics Division,
Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
Boubacar Kanté  Corresponding author<EMAIL_ADDRESS>Department of
Electrical Engineering and Computer Sciences, University of California
Berkeley, Berkeley, CA 94720, USA  Materials Sciences Division, Lawrence
Berkeley National Laboratory, Berkeley, California 94720, USA

###### Abstract

Silicon is the most scalable optoelectronic material, and it has
revolutionized our lives in many ways. The prospect of quantum optics in
silicon is an exciting avenue because it has the potential to address the
scaling and integration challenges, the most pressing questions facing quantum
science and technology. We report the first all-silicon quantum light source
based on a single atomic emissive center embedded in a silicon-based
nanophotonic cavity. We observe a more than 30-fold enhancement of
luminescence, a near unity atom-cavity coupling efficiency, and an 8-fold
acceleration of the emission from the quantum center. Our work opens avenues
for large-scale integrated all-silicon cavity quantum electrodynamics and
quantum photon interfaces with applications in quantum communication, sensing,
imaging, and computing.

††preprint: APS/123-QED

Quantum science and technologies promise to revolutionize our societies [1,
2]. In the search for the ideal quantum information processing platform,
“scaling” is perhaps the most challenging question due to the fundamental but
contradictory requirements for quantum systems to simultaneously be isolated
and controllable from the environment in large arrays of interacting qubits
[3, 4]. Among many quantum information platforms ranging from superconducting
qubits to trapped ions, quantum photons play a fundamental role because they
are necessary for future quantum networks to enable communication between
distant quantum nodes [5, 6, 7, 8, 9]. Quantum photons have been generated
from an extensive range of platforms, including quantum dots, color-centers in
diamonds such as NV, SiV, and SnV, or defects in two-dimensional materials
such as hBN [10, 11, 12, 13, 14]. The scaling challenge is currently being
addressed using hybrid material platforms and metamaterials in which quantum
light sources are optimized and integrated into more complex scalable systems,
following the example of heterogeneous integration in the classical domain
[15, 16, 17, 18]. However, the challenge for integrating quantum devices is
more significant than for classical systems because each interface allows
losses and decoherence that need to be minimized. It is thus fundamental to
minimize the number of interfaces by deeply integrating intrinsically scalable
platforms.

Silicon is currently the most scalable optoelectronic material. Despite the
lack of efficient classical light sources based on silicon, emissive centers
have been observed in silicon since the end of the 1980s [19]. It is only
during the last two years that single centers in silicon have been isolated
[20, 21, 22, 23]. Since then, emissive centers in silicon have been coupled to
waveguides, and more recently, an ensemble of centers has been integrated into
ring resonators [24, 25, 26, 27]. However, deterministic single photon sources
based on silicon emissive centers have remained elusive due to the lack of
controlled manufacturing approaches and the complexity of materials interfaces
after device fabrication. We report the first all-silicon quantum light source
based on an atomic emissive center in a silicon nanophotonic cavity. The
manufacturing of the centers in silicon-on-insulator substrates, with
controlled densities and dipole orientations, enables their overlap with
designed nanophotonic cavities. We demonstrate the successful alignment of a
quantum defect and nanophotonic cavity dipole moments and tune the
nanophotonic cavity to overlap its resonance with the zero-phonon line of the
silicon-based quantum defect. We achieve a more than 30-fold enhancement of
the luminescence intensity and an 8-fold acceleration of the single photon
emission rate. Our results open the door to large-scale integrated all-silicon
quantum optics devices and systems for applications in quantum communication,
sensing, imaging, and computing.

The proposed all-silicon atom-cavity system, presented in Fig. 1A, consists of
a single defect in silicon embedded in a photonic crystal (PhC) defect cavity.
The PhC cavity consists of three missing holes in a suspended triangular
lattice of holes [28]. The atomic defect, the G-center in silicon, is made of
two substitutional carbon atoms (black spheres) bound to the same silicon
self-interstitial (blue sphere). The manufacturing process starts with the
implantation of carbon (13C) with an energy of 36 keV in a commercial 230 nm
thick silicon-on-insulator (SOI) wafer. The implantation is followed by
electron beam lithography, dry etching, thermal annealing, and wet etching
(see Supplementary Materials). Secondary ion mass spectroscopy (SIMS)
measurements indicate that the implanted carbon and the atomic centers created
during the annealing process are located in the middle of the silicon layer
(see Supplementary Materials).

The dipole moment of the center is computed by density functional theory [29].
It is in the plane as indicated by the red arrow in the inset of Fig.1A. The
G-center is one of a broad diversity of recently observed emissive centers in
silicon and its electronic structure, presented in Fig. 1B, comprises a ground
singlet state, a dark excited triplet state, and an excited singlet state
[30]. The computed electromagnetic mode of the cavity for the transverse
electric polarization is superimposed on the sketch of the PhC, evidencing the
high confinement of the electromagnetic field in the region of missing holes
in the triangular lattice. This polarization matches the orientation of the
atomic defect dipole moment. The deterministic positioning of atomic-scale
defects in photonic cavities has been challenging for most platforms and has
not yet been achieved for silicon-emissive centers. It requires not only
overlap of the quantum defect with highly confined optical modes but also the
alignment of the dipole moments of the atom and the cavity. To overcome this
challenge in our platform, we first investigated the scalable manufacturing of
single emissive centers with controllable densities and inhomogeneous
broadening. We identified an annealing time window below which only ensembles
of centers are created and beyond which all single centers are destroyed (see
Supplementary Materials). We also find that shorter annealing time within that
window minimizes the inhomogeneous broadening of the zero-phonon line (ZPL) of
the quantum emitters, a critical requirement for overlapping the ZPL with a
designed nanophotonic resonance to enhance light-matter interaction (see
Supplementary Materials). The controlled density and inhomogeneous broadening
of quantum centers increase the probability of overlap with an array of
finite-size photonic crystal cavities. We subsequently investigated the
polarization response of created emissive centers, and a statistical analysis
presented in the supplementary materials indicates a preferential orientation
of the emitters in silicon. We then fabricate PhC cavities so that the dipole
moments of the cavities and centers align. Fig. 1C presents a scanning
electron microscope (SEM) image of a fabricated silicon-based atom-cavity
system. The inset presents the cavity with a mode volume of
$0.66(\lambda_{\text{cav}}/n)^{3}$. The successful embedding of a single
center in a cavity involved a controlled sequence of CMOS-compatible
fabrication steps.

Fig. 2A presents the photoluminescence (PL) raster scan of a device with
bright emission from a color center within the boundaries of the cavity. The
dashed white line indicates the boundary of the finite PhC, and the suspended
PhC is surrounded by the silicon-on-insulator (SOI) wafer. The
photoluminescence of the photonic device, presented in Fig. 2B, exhibits a
sharp peak at 1275 nm and a blue-shifted broader peak at 1272 nm,
corresponding to the ZPL of the color-center and the cavity resonance,
respectively. The cavity is further characterized in reflectivity using
resonant scattering measurements in Fig. 2C. The cavity is illuminated with a
linearly polarized white light source (white arrow) at 45 degrees with respect
to the cavity axis that is along the X-direction. The signal polarized
perpendicular to the excitation is collected (red arrow) to probe the cavity
mode, and resonance is observed at 1272 nm, in perfect agreement with the PL
measurement. The reflectivity is fitted with a Fano resonance line shape
giving an intrinsic quality factor ($Q$) of 3209. The experimental value is
comparable to the theoretical $Q$ of 6000 and the discrepancy is attributed to
fabrication imperfections. Fig. 2D presents the polarization diagram of the
cavity mode detuned from the ZPL in orange and the polarization diagram of a
quantum emitter alone in black. The polarizations agree well with a dipolar
model (solid lines) and have been successfully aligned.

In Fig. 3A, the spectrum of the quantum emitter over a broad range of energy
shows the zero-phonon line (ZPL) of the silicon emissive center and its phonon
sideband. Fig. 3B presents the spectrum of the quantum emitter using a high-
resolution grating. The ZPL is located at 972.43 meV and has a linewidth of
8.3 GHz. To demonstrate that the bright emission from the middle of the cavity
corresponds to a single emissive center, we performed quantum coherence
measurements of the emitter in the cavity. Autocorrelation measurements, shown
in Fig. 3C, are performed using a Hanbury-Brown and Twiss interferometer with
superconducting nanowire single-photon detectors (see Supplementary
Materials). The second-order correlation measurements of the emission from the
cavity under continuous excitation exhibit an antibunching, confirming the
successful spatial overlap of a single silicon emissive center with the
nanophotonic cavity with an antibunching at a zero delay $g^{(2)}(0)=0.30\pm
0.07$. Autocorrelation measurements under pulsed excitation at a repetition
rate of 10 MHz are presented in Fig. 3D and they demonstrate on-demand single-
photon generation from the all-silicon platform.

The enhancement of the single center in the cavity requires spatial and
spectral overlap. Spatial overlap was achieved in Fig. 2 and Fig. 3. To
achieve spectral overlap, the nanophotonic cavity is tuned using cycles of
argon gas injection. The injected gas condensates at the surface of the PhC
and modifies the effective index of cavity mode tuning the resonance
wavelength of the cavity that is shifted from 1269 nm to 1275 nm. In Fig. 4A,
as the cavity resonance is shifted towards the ZPL of the quantum center, the
photoluminescence is enhanced to reach a maximum at 1275 nm, where the
spectral overlap is achieved. In Fig. 4B, the ZPL intensity as a function of
the cavity detuning shows an enhancement larger than 30 achieved on resonance.
For cavity detuning varying from $\delta$ = 2.40 nm to $\delta$ = 0.00 nm, the
excited lifetime shortens from 53.6 ns to 6.7 ns. An 8-fold reduction in the
lifetime is experimentally observed when the overlap is achieved compared to
the off-resonance case. Light-matter interaction in cavities is usually
quantified using the Purcell factor ($F_{p}$) that measures the decay rate
enhancement of the atom from free space to the cavity
($\gamma_{\text{cav}}=F_{p}\gamma_{0}$). It can be estimated by
$F_{p}=(\tau_{\text{bulk}}/\tau_{\text{on}}-\tau_{\text{bulk}}/\tau_{\text{off}})/\eta$
where $\tau_{\text{bulk}}$ is the lifetime of a quantum emitter outside the
PhC (dark yellow dots in Fig. 4C), and $\tau_{\text{off}}$ is the lifetime for
a detuning of 2.4 nm. The lifetime measured off-resonance is slightly longer
than the one in the bulk because of the reduced density of state in the PhC
gap [31]. The percentage of photons emitted at the ZPL wavelength of the
emitter (Debye-Waller) is $\eta=15\%$, which was measured by comparing the
count rate with and without the ZPL bandpass filter. The experimental Purcell
factor of the defect cavity is $F_{p}\sim 29.0$. The coupling efficiency of
the center to the cavity mode ($\beta$ factor) can be estimated by
$1/\tau_{on}/[1/\tau_{on}+1/\tau_{off}]$, yielding a value of $\beta\sim
89\%$.

We thus reported the first all-silicon quantum light source based on an atomic
emissive center in a silicon-based nanophotonic cavity. The quantum light
source in silicon is one of a broad diversity of recently discovered centers
that we successfully embedded in a silicon-on-insulator photonic crystal
cavity. The quantum center is manufactured in silicon using a sequence of CMOS
compatible steps that control the density, inhomogeneous broadening, and
orientation of the dipolar moment of the emitters, enabling their efficient
overlap with designed nanophotonic cavities. The performance of the quantum
light source can be further improved by developing cavities with higher
quality factors as well as more deterministic positioning methods to further
improve the emitter-cavity spatial overlap. Our results will enable all-
silicon quantum optics interfaces with silicon-emissive centers for scalable
and integrated quantum optoelectronics.

## References

  * Nielsen  _et al._ [2002] M. A. Nielsen, I. Chuang, and L. K. Grover, Quantum Computation and Quantum Information, Am. J. Phys. 70 (2002).
  * MacFarlane  _et al._ [2003] A. G. J. MacFarlane, J. P. Dowling, and G. J. Milburn, Quantum technology: the second quantum revolution, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 361, 1655 (2003).
  * de Leon  _et al._ [2021] N. P. de Leon, K. M. Itoh, D. Kim, K. K. Mehta, T. E. Northup, H. Paik, B. S. Palmer, N. Samarth, S. Sangtawesin, and D. W. Steuerman, Materials challenges and opportunities for quantum computing hardware, Science 372, eabb2823 (2021).
  * Arute  _et al._ [2019] F. Arute, K. Arya, R. Babbush, D. Bacon, J. C. Bardin, R. Barends, R. Biswas, S. Boixo, F. G. S. L. Brandao, D. A. Buell, B. Burkett, Y. Chen, Z. Chen, B. Chiaro, R. Collins, W. Courtney, A. Dunsworth, E. Farhi, B. Foxen, A. Fowler, C. Gidney, M. Giustina, R. Graff, K. Guerin, S. Habegger, M. P. Harrigan, M. J. Hartmann, A. Ho, M. Hoffmann, T. Huang, T. S. Humble, S. V. Isakov, E. Jeffrey, Z. Jiang, D. Kafri, K. Kechedzhi, J. Kelly, P. V. Klimov, S. Knysh, A. Korotkov, F. Kostritsa, D. Landhuis, M. Lindmark, E. Lucero, D. Lyakh, S. Mandrà, J. R. McClean, M. McEwen, A. Megrant, X. Mi, K. Michielsen, M. Mohseni, J. Mutus, O. Naaman, M. Neeley, C. Neill, M. Y. Niu, E. Ostby, A. Petukhov, J. C. Platt, C. Quintana, E. G. Rieffel, P. Roushan, N. C. Rubin, D. Sank, K. J. Satzinger, V. Smelyanskiy, K. J. Sung, M. D. Trevithick, A. Vainsencher, B. Villalonga, T. White, Z. J. Yao, P. Yeh, A. Zalcman, H. Neven, and J. M. Martinis, Quantum supremacy using a programmable superconducting processor, Nature 574, 505 (2019).
  * Bouwmeester  _et al._ [1997] D. Bouwmeester, J.-W. Pan, K. Mattle, M. Eibl, H. Weinfurter, and A. Zeilinger, Experimental quantum teleportation, Nature 390, 575 (1997).
  * Clarke and Wilhelm [2008] J. Clarke and F. K. Wilhelm, Superconducting quantum bits, Nature 453, 1031 (2008).
  * Devoret and Schoelkopf [2013] M. H. Devoret and R. J. Schoelkopf, Superconducting Circuits for Quantum Information: An Outlook, Science 339, 1169 (2013).
  * Monroe and Kim [2013] C. Monroe and J. Kim, Scaling the Ion Trap Quantum Processor, Science 339, 1164 (2013).
  * Kimble [2008] H. J. Kimble, The quantum internet, Nature 453, 1023 (2008).
  * Santori  _et al._ [2002] C. Santori, D. Fattal, J. Vučković, G. S. Solomon, and Y. Yamamoto, Indistinguishable photons from a single-photon device, Nature 419, 594 (2002).
  * Gruber  _et al._ [1997] A. Gruber, A. Dräbenstedt, C. Tietz, L. Fleury, J. Wrachtrup, and C. v. Borczyskowski, Scanning Confocal Optical Microscopy and Magnetic Resonance on Single Defect Centers, Science 276, 2012 (1997).
  * Lee  _et al._ [2014] J. C. Lee, D. O. Bracher, S. Cui, K. Ohno, C. A. McLellan, X. Zhang, P. Andrich, B. Alemán, K. J. Russell, A. P. Magyar, I. Aharonovich, A. Bleszynski Jayich, D. Awschalom, and E. L. Hu, Deterministic coupling of delta-doped nitrogen vacancy centers to a nanobeam photonic crystal cavity, Applied Physics Letters 105, 261101 (2014).
  * Hayee  _et al._ [2020] F. Hayee, L. Yu, J. L. Zhang, C. J. Ciccarino, M. Nguyen, A. F. Marshall, I. Aharonovich, J. Vučković, P. Narang, T. F. Heinz, and J. A. Dionne, Revealing multiple classes of stable quantum emitters in hexagonal boron nitride with correlated optical and electron microscopy, Nature Materials 19, 534 (2020).
  * Xu  _et al._ [2021] X. Xu, Z. O. Martin, D. Sychev, A. S. Lagutchev, Y. P. Chen, T. Taniguchi, K. Watanabe, V. M. Shalaev, and A. Boltasseva, Creating Quantum Emitters in Hexagonal Boron Nitride Deterministically on Chip-Compatible Substrates, Nano Letters 21, 8182 (2021).
  * Elshaari  _et al._ [2020] A. W. Elshaari, W. Pernice, K. Srinivasan, O. Benson, and V. Zwiller, Hybrid integrated quantum photonic circuits, Nature Photonics 14, 285 (2020).
  * Wan  _et al._ [2020] N. H. Wan, T.-J. Lu, K. C. Chen, M. P. Walsh, M. E. Trusheim, L. De Santis, E. A. Bersin, I. B. Harris, S. L. Mouradian, I. R. Christen, E. S. Bielejec, and D. Englund, Large-scale integration of artificial atoms in hybrid photonic circuits, Nature 583, 226 (2020).
  * Carter  _et al._ [2013] S. G. Carter, T. M. Sweeney, M. Kim, C. S. Kim, D. Solenov, S. E. Economou, T. L. Reinecke, L. Yang, A. S. Bracker, and D. Gammon, Quantum control of a spin qubit coupled to a photonic crystal cavity, Nature Photonics 7, 329 (2013).
  * Santiago-Cruz  _et al._ [2022] T. Santiago-Cruz, S. D. Gennaro, O. Mitrofanov, S. Addamane, J. Reno, I. Brener, and M. V. Chekhova, Resonant metasurfaces for generating complex quantum states, Science 377, 991 (2022).
  * Davies [1989] G. Davies, The optical properties of luminescence centres in silicon, Physics Reports 176, 83 (1989).
  * Redjem  _et al._ [2020] W. Redjem, A. Durand, T. Herzig, A. Benali, S. Pezzagna, J. Meijer, A. Y. Kuznetsov, H. S. Nguyen, S. Cueff, J.-M. Gérard, I. Robert-Philip, B. Gil, D. Caliste, P. Pochet, M. Abbarchi, V. Jacques, A. Dréau, and G. Cassabois, Single artificial atoms in silicon emitting at telecom wavelengths, Nature Electronics 3, 738 (2020).
  * Hollenbach  _et al._ [2020] M. Hollenbach, Y. Berencén, U. Kentsch, M. Helm, and G. V. Astakhov, Engineering telecom single-photon emitters in silicon for scalable quantum photonics, Optics Express 28, 26111 (2020).
  * Durand  _et al._ [2021] A. Durand, Y. Baron, W. Redjem, T. Herzig, A. Benali, S. Pezzagna, J. Meijer, A. Kuznetsov, J.-M. Gérard, I. Robert-Philip, M. Abbarchi, V. Jacques, G. Cassabois, and A. Dréau, Broad Diversity of Near-Infrared Single-Photon Emitters in Silicon, Physical Review Letters 126, 083602 (2021).
  * Higginbottom  _et al._ [2022] D. B. Higginbottom, A. T. K. Kurkjian, C. Chartrand, M. Kazemi, N. A. Brunelle, E. R. MacQuarrie, J. R. Klein, N. R. Lee-Hone, J. Stacho, M. Ruether, C. Bowness, L. Bergeron, A. DeAbreu, S. R. Harrigan, J. Kanaganayagam, D. W. Marsden, T. S. Richards, L. A. Stott, S. Roorda, K. J. Morse, M. L. W. Thewalt, and S. Simmons, Optical observation of single spins in silicon, Nature 607, 266 (2022).
  * Prabhu  _et al._ [2022] M. Prabhu, C. Errando-Herranz, L. De Santis, I. Christen, C. Chen, and D. R. Englund, Individually Addressable Artificial Atoms in Silicon Photonics (2022), arXiv:2202.02342.
  * DeAbreu  _et al._ [2022] A. DeAbreu, C. Bowness, A. Alizadeh, C. Chartrand, N. A. Brunelle, E. R. MacQuarrie, N. R. Lee-Hone, M. Ruether, M. Kazemi, A. T. K. Kurkjian, S. Roorda, N. V. Abrosimov, H.-J. Pohl, M. L. W. Thewalt, D. B. Higginbottom, and S. Simmons, Waveguide-integrated silicon T centres (2022), arXiv:2209.14260.
  * Komza  _et al._ [2022] L. Komza, P. Samutpraphoot, M. Odeh, Y.-L. Tang, M. Mathew, J. Chang, H. Song, M.-K. Kim, Y. Xiong, G. Hautier, and A. Sipahigil, Indistinguishable photons from an artificial atom in silicon photonics (2022), arXiv:2211.09305.
  * Lefaucher  _et al._ [2022] B. Lefaucher, J.-B. Jager, V. Calvo, A. Durand, Y. Baron, F. Cache, V. Jacques, I. Robert-Philip, G. Cassabois, T. Herzig, J. Meijer, S. Pezzagna, M. Khoury, M. Abbarchi, A. Dréau, and J.-M. Gérard, Cavity-enhanced zero-phonon emission from an ensemble of G centers in a silicon-on-insulator microring (2022), arXiv:2210.05485.
  * Sakoda [2005] K. Sakoda,  _Optical Properties of Photonic Crystals_ , edited by W. T. Rhodes, T. Asakura, K.-H. Brenner, T. W. Hänsch, T. Kamiya, F. Krausz, B. Monemar, H. Venghaus, H. Weber, and H. Weinfurter, Springer Series in Optical Sciences, Vol. 80 (Springer, Berlin, Heidelberg, 2005).
  * Ivanov  _et al._ [2022] V. Ivanov, J. Simoni, Y. Lee, W. Liu, K. Jhuria, W. Redjem, Y. Zhiyenbayev, C. Papapanos, W. Qarony, B. Kanté, A. Persaud, T. Schenkel, and L. Z. Tan, Effect of localization on photoluminescence and zero-field splitting of silicon color centers, Physical Review B 106, 134107 (2022).
  * Udvarhelyi  _et al._ [2021] P. Udvarhelyi, B. Somogyi, G. Thiering, and A. Gali, Identification of a Telecom Wavelength Single Photon Emitter in Silicon, Physical Review Letters 127, 196402 (2021).
  * Yablonovitch [1987] E. Yablonovitch, Inhibited Spontaneous Emission in Solid-State Physics and Electronics, Physical Review Letters 58, 2059 (1987).

Figure 1: A single atomic emissive center embedded in a silicon-photonic
cavity. (A) Silicon quantum interface with an “atomic defect” located within
the “photonic defect” cavity of three missing holes in a triangular photonic
crystal (PhC). The atomic defect is the G-center in silicon made of two
substitutional carbon atoms (black spheres) bound to the same silicon self-
interstitial (blue sphere). The red arrow indicates the direction of the
dipole moment of the G-center. The G-center is one of a broad diversity of
recently observed emissive centers in silicon, the most scalable
optoelectronic material. The computed electromagnetic mode of the cavity is
superimposed on the sketch of the PhC, evidencing the high confinement of the
electromagnetic field in the region of missing holes in the triangular
lattice. The cavity is fabricated so that its dipole moment and the dipole
moment of the defect are colinear (see Supplementary Materials). The electric
field strength peaks at the center of the cavity and exponentially decays in
the bulk of the PhC. (B) Energy level diagram of the G-center in silicon
comprising a ground singlet state, a dark excited triplet state, and an
excited singlet state. The cavity can be tuned to be in resonance with the
radiative transition between the excited and ground singlet states to enhance
light-matter interaction. (C) Scanning electron microscope (SEM) image of a
fabricated silicon-based atom-cavity system suspended in the air. The
successful embedding of a single G-center in a photonic cavity involved a
controlled sequence of fabrication steps using a commercial 230 nm thick
silicon-on-insulator (SOI) wafer that is carbon implanted, followed by
electron beam lithography, dry etching, thermal annealing, and wet etching
(see Supplementary Materials). The fabrication steps, compatible with standard
CMOS processes, are optimized to increase the probability of single color
centers in cavities (see Supplementary Materials). Figure 2: Experimental
characterization of the silicon-based quantum emitter and cavity. (A)
Photoluminescence (PL) raster scan of a device with a single emitter in the
cavity. The boundary of the finite photonic crystal (PhC) is indicated by the
dashed white line and the suspended PhC is surrounded by the silicon-on-
insulator (SOI) wafer. The photoluminescence signal shows bright emission from
a color center within the boundaries of the cavity. (B) The photoluminescence
of the photonic device exhibits a sharp peak at 1275 nm and a blue-shifted
broader peak at 1272 nm, corresponding to the zero-phonon line of the color-
center and the cavity resonance, respectively. (C) Reflectivity of the
photonic crystal cavity obtained by resonant scattering measurements. The
cavity is illuminated with a linearly polarized white light source (white
arrow) at 45 degrees with respect to the cavity axis that is along the
X-direction. The polarized signal perpendicular to the excitation is collected
(red arrow) to probe the cavity mode. (D) The polarization diagram of the
cavity mode detuned from the ZPL is shown in orange. The polarization diagram
of a quantum emitter alone is shown in black. The polarizations agree well
with a dipolar model (solid lines) and are well aligned (see Supplementary
Materials). Figure 3: Quantum coherence measurements of the emitter in the
cavity. (A) Spectrum of the quantum emitter over a broad range of energy
showing the zero-phonon line (ZPL) of the silicon emissive center and its
phonon sideband. (B) Spectrum of the quantum emitter using a high-resolution
grating. The ZPL is located at 972.43 meV and has a linewidth of 8.3 GHz. (C)
Second-order autocorrelation measurements of the emission from the cavity
under continuous excitation. The antibunching at zero delay confirms the
successful spatial overlap of a single silicon emissive center with the
nanophotonic cavity with an antibunching at zero delay $g^{(2)}(0)=0.30\pm
0.07$. (D) Second-order autocorrelation measurements under pulsed excitation
at a repetition rate of 10 MHz demonstrating on-demand single-photon
generation from the all-silicon platform. Autocorrelation measurements are
performed using a Hanbury-Brown and Twiss interferometer with superconducting
nanowire single-photon detectors (see Supplementary Materials). Figure 4:
Spectral tuning of the nanocavity and enhanced atom-cavity interaction. (A)
Cavity tuning as a function of the argon gas injection cycle. The enhancement
of the single center in the cavity requires spatial and spectral overlap.
Spatial overlap was achieved in Fig. 2 and Fig. 3 and spectral overlap is
achieved here by tuning the nanophotonic cavity. Gas injection modifies the
effective index of cavity mode and tunes the resonance wavelength of the
cavity that is shifted from $\sim$1269 nm to $\sim$1275 nm. As the cavity
resonance is shifted towards the ZPL of the quantum center, the
photoluminescence is enhanced to reach a maximum at $\sim$1275 nm, where the
spectral overlap is achieved. (B) Zero-phonon line intensity as a function of
the cavity detuning. An enhancement larger than 30 is achieved on resonance.
(C) Excited lifetime for cavity detuning of $\delta$ = 2.40 nm, $\delta$ =
0.23 nm, and $\delta$ = 0.00 nm. The lifetime of emitted photons shortens from
53.6 ns to 6.7 ns when the detuning between the cavity and emitter is
decreased. An 8-fold reduction in the lifetime is experimentally observed when
the overlap is achieved compared to the off-resonance case. These results
constitute the first all-silicon quantum light source using a silicon emissive
center in a cavity, and the center can be further accelerated by designing
cavities with higher quality factors as well as more deterministic positioning
methods to further improve the emitter-cavity spatial overlap. The results
will enable all-silicon quantum optics interfaces with silicon-emissive
centers for scalable quantum optics.


# Identification of an [$\alpha$/Fe]–enhanced thick disk component in an edge-
on Milky Way Analogue

Nicholas Scott Sydney Institute for Astronomy, School of Physics, University
of Sydney, NSW 2006, Australia  ARC Centre of Excellence for All Sky
Astrophysics in 3 Dimensions (ASTRO 3D)  Jesse van de Sande Sydney Institute
for Astronomy, School of Physics, University of Sydney, NSW 2006, Australia
ARC Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)
Sanjib Sharma Sydney Institute for Astronomy, School of Physics, University of
Sydney, NSW 2006, Australia  ARC Centre of Excellence for All Sky Astrophysics
in 3 Dimensions (ASTRO 3D)  Joss Bland-Hawthorn Sydney Institute for
Astronomy, School of Physics, University of Sydney, NSW 2006, Australia  ARC
Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)  Ken
Freeman  Research School of Astronomy & Astrophysics, Australian National
University, ACT 2611, Australia  Ortwin Gerhard Max-Planck-Institut für
Extraterrestrische Physik, Giessenbachstrasse, D-85741 Garching, Germany
Michael R. Hayden  Sydney Institute for Astronomy, School of Physics,
University of Sydney, NSW 2006, Australia  ARC Centre of Excellence for All
Sky Astrophysics in 3 Dimensions (ASTRO 3D)  Richard McDermid  Research Centre
for Astronomy, Astrophysics, and Astrophotonics, Department of Physics and
Astronomy, Macquarie University, NSW 2109, Australia  ARC Centre of Excellence
for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)

###### Abstract

The Milky Way disk consists of two prominent components — a thick, alpha-rich,
low-metallicity component and a thin, metal-rich, low-alpha component.
External galaxies have been shown to contain thin and thick disk components,
but whether distinct components in the [$\alpha$/Fe]–[Z/H] plane exist in
other Milky Way-like galaxies is not yet known. We present VLT-MUSE
observations of UGC 10738, a nearby, edge-on Milky Way-like galaxy. We
demonstrate through stellar population synthesis model fitting that UGC 10738
contains alpha-rich and alpha-poor stellar populations with similar spatial
distributions to the same components in the Milky Way. We discuss how the
finding that external galaxies also contain chemically distinct disk
components may act as a significant constraint on the formation of the Milky
Way’s own thin and thick disk.

Galaxy: evolution — galaxies: evolution — galaxies: stellar content

††journal: ApJL††facilities: ESO(MUSE), Pan-STARRS††software: astropy (Astropy
Collaboration et al., 2013); MUSE Data Reduction Software v2.8.1 (Weilbacher
et al., 2016); ZAP (Soto et al., 2016); pPXF (Cappellari, 2012).

##  1 Introduction

The Milky Way is the galaxy for which we have the most detailed observational
data, with extremely precise astrometric and kinematic data from the Gaia
satellite (Gaia Collaboration et al., 2016, 2018), as well as accurate
chemical abundances for large numbers of stars from the APOGEE (Majewski et
al., 2017), GALAH (Buder et al., 2020) and LAMOST/LEGUE (Deng et al., 2012)
surveys. Striking features of the Milky Way are the chemically distinct thin
and thick disk (Gilmore & Reid, 1983; Haywood et al., 2013; Hayden et al.,
2015). Thick disk stars are enhanced in [$\alpha$/Fe] with respect to thin
disk stars of a similar metallicity, with the overall metallicity lower than
that of the thin disk. The two disks have distinct spatial distributions, with
the thick disk having a larger scale height of $\sim 900$ pc, compared to the
thin disk’s $\sim 300$ pc (Jurić et al., 2008; Bland-Hawthorn & Gerhard,
2016).

Figure 1: Pseudo-gri colour image of UGC 10738 created from the MUSE datacube.
The X/peanut-shaped bulge and prominent dust lane are clearly visible. The
white bar indicates a scale of 5 kpc at the distance of UGC 10738, $D=99$ Mpc.

While the Milky Way is the best object to study to understand ‘Galaxy’
evolution, it is challenging to make inferences about galaxy evolution in
general. From a sample of one, it is difficult to differentiate between
general features of galaxy evolution and aspects specific to the Milky Way’s
own unique formation history. Structural thick disks have been identified in
external galaxies from deep imaging surveys (Yoachim & Dalcanton, 2006;
Comerón et al., 2011, 2018; Martínez-Lombilla & Knapen, 2019) by applying
photometric structural decomposition techniques to images of edge-on galaxies,
and are thought to be ubiquitous in disk galaxies (Yoachim & Dalcanton, 2006;
Comerón et al., 2011). The evidence for chemically distinct thick disks in
external Milky Way-like galaxies is inconclusive (Yoachim & Dalcanton, 2008),
with convincing signatures identified primarily in early-type disk galaxies
(Comerón et al., 2016; Pinna et al., 2019a, b; Poci et al., 2019, 2021), with
the exception of a low-mass late-type galaxy in Comerón et al. (2015).

This work explores whether chemically distinct thick and thin disks are
present in nearby disk galaxies that are structurally similar to our own
Galaxy – Milky Way analogues (MWAs). Recent improvements in integral field
spectroscopy enable integrated light measurements that can sample physical
scales allowing direct comparison to Milky Way observations. We present Very
Large Telescope (VLT) – Multi Unit Spectroscopic Explorer (MUSE) observations
of UGC 10738, a nearby MWA. We derive spatially-resolved star formation and
chemical enrichment histories and compare these to similar measurements for
the Milky Way. We conclude by discussing the implications of these results for
the formation of Milky Way-like galaxies, and for the Milky Way itself.

##  2 Data and Observations

###  2.1 UGC 10738

We selected nearby ($D<100$ Mpc) galaxies from the Third Reference Catalogue
(RC3, de Vaucouleurs et al., 1991) with a morphological type similar to the
Milky Way (Sbc or SBbc, Hubble T-type $\sim 4$ de Vaucouleurs & Pence, 1978)
and a stellar mass $10.4<M_{\star}<11.2(M_{\odot})$. From this sample we
selected 9 galaxies in three inclination bins ($45^{\circ}<i<60^{\circ}$,
$60^{\circ}<i<75^{\circ}$ and $i>75^{\circ}$) and three colour bins based on
the ($B-I$) colour, or, when unavailable, multi-band images. UGC 10738 falls
in the most inclined and reddest bin.

UGC 10738 has a stellar mass of $8\times 10^{10}$ M⊙ and a $(g-r)$ colour of
0.95, highly comparable to the Milky Way (Bland-Hawthorn & Gerhard, 2016, and
references therein). We note that UGC 10738 is larger than the Milky Way by
$\sim 50$ per cent in effective radius, though a precise comparison is
unfeasible given the very different approaches to measuring size in
extragalactic objects and the Milky Way.

Magnitudes and effective radii were derived by fitting a Sérsic surface
brightness profile on Pan-STARRS $gri$ images (Waters et al., 2020) using the
code ProFound (Robotham et al., 2018) and ProFit (Robotham et al., 2017). M⋆
was estimated using the $g-i$ colour and $i$ magnitude following Taylor et al.
(2011) and Bryant et al. (2015) and assuming a Chabrier (2003) initial mass
function. A pseudo-gri image, derived from the reduced MUSE datacube is shown
in Figure 1.

Figure 2: Left panel: example spectral fit with the observed spectrum in
black, the stellar component of the fit in red and the gas component in pink
and orange. The residuals are in green, with masked regions in blue. Right
panels: mass fraction of stars with a given age, [Z/H] and [$\alpha$/Fe] that
best reproduce the observed spectrum.

###  2.2 Observations and data reduction

UGC 10738 was observed with MUSE on the VLT as part of observing program
0101.B-0706(A) (PI: van de Sande) on 16-04-2018 and 17-04-2018 in service
mode. MUSE provides a 1′ field-of-view with a spatial sampling of 0$\farcs$2
and spectral coverage from 480 to 930 nm at a resolving power R$\sim 2700$. At
our adopted distance, $D=99$ Mpc (Tully et al., 2013), the spatial sampling is
91 pc/spaxel.

Observations consisted of six 1240 second exposures in three locations, one
centred on the centre of the galaxy with the others offset by $\sim$1′ in
either direction along the major axis of the galaxy, combined with two 90∘
offset position angles. All observations were reduced using
pymusepipe111https://github.com/emsellem/pymusepipe version 2.9.9 MUSE Data
Reduction Software (Weilbacher et al., 2016, 2020) version 2.8.1, including
the standard steps of flat fielding, wavelength calibration, flux calibration
and telluric correction. Sky subtraction was performed by identifying regions
free from galaxy light in each exposure using ProFound (Robotham et al.,
2018), and subtracting the extracted sky spectra from the rest of the data
cube using the package ZAP (the Zurich Atmosphere Purge, Soto et al., 2016).
The 6 pointings were combined into a single, sky subtracted, flux calibrated
mosaic data cube, which was used for the following analysis.

##  3 Stellar Population Analysis

Our goal in this work is to compare the spatial variation of the [Z/H] and
[$\alpha$/Fe] distributions in UGC 10738 to those observed in the Milky Way.
We combine spectra into 36 binned regions, chosen to match those used in
Hayden et al. (2015). Additionally, this greatly increases the signal-to-noise
ratio of the binned spectra over the individual spaxels. We define an inner
region with $0<R<3$ kpc, and 8 further radial regions each 2 kpc in width,
evenly spaced from 3 to 19 kpc along the galaxy major axis. We define four
bins in height: $|z|<0.5$ kpc, $0.5<|z|<1$ kpc, $1<|z|<2$ kpc and $2<|z|<4$
kpc. These regions extend further than those used in Hayden et al. (2015), and
as these are projected quantities, the effective intrinsic radii they probe
will be relatively larger. Before binning, each spectrum is interpolated onto
a common velocity scale based on the velocity map of van de Sande (in prep.)
derived with the GIST pipeline (Galaxy IFU Spectroscopy Tool, Bittner et al.,
2019) using the penalized Pixel Fitting (pPXF) Python package (Cappellari &
Emsellem, 2004) and employing the Voronoi-binning method of Cappellari & Copin
(2003). Spaxels are co-added from all quadrants of the galaxy to form the
final binned spectra for the 18 spatial regions. Spectra covering the region
of Hayden et al. (2015) range in signal-to-noise ratio from $\sim 400$ to
$\sim 80$ per spectral pixel, sufficient to recover the distribution of
stellar age, [Z/H] and [$\alpha$/Fe] for each region (Liu, 2020).

To measure the star formation and chemical enrichment history we fit the
spectra from each region with the $\alpha$-variable SSP library of Vazdekis et
al. (2015), which is derived from the empirical MILES stellar spectral library
(Falcón-Barroso et al., 2011). The fit is performed over the overlapping
wavelength range, 465 to 741 nm. We select SSP templates that form a regular
grid with twelve bins in [Z/H] (-2.27, -1.79, -1.49, -1.26, -0.96, -0.66,
-0.35, -0.25, 0.06, 0.15, 0.26, 0.4) and fifty three logarithmically spaced
bins in age (0.03 to 14 Gyrs). Only two bins in [$\alpha$/Fe] (0.0, 0.4) are
used because empirically-based SSP spectral libraries with finer sampling in
this dimension were not available at time of publication.

Figure 3: Maps of the mass-weighted average [Z/H], [$\alpha$/Fe] and age (in
Gyrs) in UGC 10738. Values have been averaged over all four quadrants of the
galaxy. The dashed black line in the upper left of each panel indicates the
low S/N region.

To determine the combination of SSP template spectra that best reproduces each
observed spectrum we again use pPXF (Cappellari & Emsellem, 2004; Cappellari,
2012). In addition to the SSP template spectra described above, we include six
emission line templates, including three doublets for nine lines in total
(H$\beta$, H$\alpha$, [SII]$\lambda$6717,6731, [OI]$\lambda$6300,
[OIII]$\lambda$4959,5007 and [NII]$\lambda$6548,6583) to correct for gas
emission and a 10th order multiplicative polynomial to account for dust
extinction and residual flux calibration errors. pPXF determines the weights
of each SSP template required to reproduce the observed spectrum, which
represent the fraction of stars (by star-forming mass) with a given age, [Z/H]
and [$\alpha$/Fe] in the best fit.

Figure 4: Maps of the mass-weighted average [Z/H], [$\alpha$/Fe] and age (in
Gyrs) for the Milky Way, derived from the model presented in Sharma et al.
(2020), showing the same spatial extent as in Fig 3.

By default pPXF treats all the template spectra independently, which can lead
to unphysically spiky star formation and chemical enrichment histories. To
mitigate this we apply a second-order regularization to the template weights
in the age and [Z/H] dimensions, as described in Cappellari (2017) (no
regularization is imposed in the [$\alpha$/Fe] dimension). We perform an
initial fit without regularization to set a baseline goodness-of-fit
parameterised by $\chi^{2}_{0}$, then gradually increase the regularization
until the fit quality is reduced to the point that
$\chi^{2}_{\mathrm{final}}=\chi^{2}_{0}+\sqrt{2\times dof}$, where $dof$ is
number of spectral pixels in the fit. This results in the smoothest star
formation and chemical enrichment history that is consistent with the observed
spectrum.

An example full spectral fit for the region $5<|R|<7$ kpc and $0.5<|z|<1$ kpc
is shown in Figure 2, with the input (black) and best-fitting (red) spectra
shown in the left panel. The mass-weighted fraction of stars with a given age,
[Z/H] and [$\alpha$/Fe] is shown in the right panels. For each region we
measure the full star formation and chemical enrichment history (i.e. the
template weights), and also derive mass-weighted average age and [Z/H] and the
fraction of high and low [$\alpha$/Fe] stars.

UGC 10738 has a prominent dust lane through the centre of the galaxy. To
determine the impact of dust on our measured stellar population parameters we
perform separate population fits to binned spectra from each quadrant at a
given radial and height range. We find no significant difference between the
[$\alpha$/Fe] values measured above and below the midplane. We find a small, 1
Gyr offset between the stellar ages measured above and below the midplane at
the lowest scale heights, $|z|<0.5$ kpc, which is not statistically
significant given the scatter in ages measured between the different
quadrants. At larger scale heights we measure no difference in stellar age.
For [Z/H], we find a statistically significant offset for heights $|z|<1$ kpc,
in the sense that [Z/H] is 0.2 (0.15) dex lower in dust dominated quadrants at
heights $|z|<0.5$ kpc ($0.5<|z|<1$ kpc). In the dustier regions we probe
shallower lines-of-sight into the disk, so we are measuring properties of
stars at a larger average intrinsic radius and scale height where we expect
[Z/H] to be lower. In summary, we find that dust has a modest effect on the
measured stellar population properties at $|z|<1$ kpc, but negligible effect
at larger heights.

##  4 Results

Figure 5: Comparison of the spatially resolved chemistry of UGC 10738 and the
Milky Way. Mass fraction of stars as a function of [Z/H] and [$\alpha$/Fe]
(summed over age) in bins of projected radius and scale height. Regions are
chosen to match those of Hayden et al. (2015), though we note the scale radii
and height of the two galaxies are likely different. Contours enclose 50, 75,
90 and 99 per cent of Milky Way stars from Hayden et al. (2015). We find a
high degree of qualitative agreement between the two galaxies, with a low
metallicity, alpha-rich component at large scale heights evident in both.

In Figure 3 we show the average, mass-weighted age, [Z/H] and [$\alpha$/Fe] of
UGC 10738 in the extracted regions. From low to high $|z|$, [Z/H] generally
decreases and [$\alpha$/Fe] generally increases at all radii, with the
exception of the $|z|<0.5$ kpc region which is somewhat lower in [Z/H] than
the adjacent height bins, which may be due to dust extinction as described
above.

The mean age of stars increases with height above the plane but declines with
radius, such that the oldest stars typically lie in the centre of the galaxy
at large heights. At large projected radius and height we find a population
with old average ages and low [$\alpha$/Fe], perhaps indicative of a flaring
alpha-poor disk, though we note that uncertainties in this region are
increased to $\sim 0.15,0.2,0.1$ in log(age), [Z/H], [$\alpha$/Fe]
respectively (consistent with expectations from Liu, 2020).

Figure 4 shows the same quantities as figure 3 but for the Milky Way. The
Milky Way values were derived from the chemodynamical model of Sharma et al.
(2020) using a star-forming mass weighting equivalent to that used for the SSP
measurements presented here. Unlike figure 3, these maps show intrinsic
spatial regions, so, while the extent of the maps for the two galaxies are
comparable they are not exactly equivalent. Despite this we see good
qualitative agreement between the spatial variation of the average age,
metallicity and [$\alpha$/Fe]. The comparison further improves when noting the
relatively larger scale height and radius for UGC 10738 than for the Milky
Way. We also find evidence of disk flaring of the young, low-alpha stars in
UGC 10738 that is qualitatively similar to that seen in the Milky Way.

In Figure 5 we show the mass fraction of stars with a given [Z/H] and
[$\alpha/$Fe] in the same regions described above. Here we restrict the
regions shown to $3<R<15$ kpc and $|z|<2$ kpc, the same spatial extent
examined in the Milky Way by Hayden et al. (2015), again noting that Milky Way
regions are in intrinsic radius and height, whereas for UGC 10738 we have only
line-of-sight integrated quantities. The contours indicate the number density
of stars in the Milky Way from Hayden et al. (2015), transformed from [Fe/H]
to [Z/H] following Salaris & Cassisi (2005). We see good qualitative agreement
between the chemical enrichment pattern seen in the Milky Way and that shown
here for UGC 10738. At radii less than $\sim 11$ kpc and at $|z|<1$ kpc, we
find the chemical distribution of both galaxies’ stars are dominated by a
metal rich, alpha-poor component — the low-alpha thin disk. At the same radii,
but $|z|>1$ kpc a significant metal poor, high alpha component is present —
the alpha-rich thick disk.

At larger radii qualitative differences between the two galaxies arise,
particularly the presence of high-alpha stars in UGC 10738 that are not seen
in the Milky Way. This may indicate that UGC 10738’s alpha-rich disk is more
extended than that of the Milky Way. We also note that in all regions UGC
10738 exhibits contributions from a modest population of high-alpha, metal
rich stars that are not seen in the Milky Way. The most likely explanation for
the existence of these stars is the lack of intermediate [$\alpha$/Fe]
templates available for the full spectral synthesis fitting, which forces pPXF
to compensate for e.g. $[\alpha/\mathrm{Fe}]\sim 0.1$ stars found in the Milky
Way by adding in a small mass fraction of $[\alpha/\mathrm{Fe}]=0.4$ stars in
UGC 10738. This ambiguity is likely to be resolved as population templates
with intermediate [$\alpha$/Fe] become available.

##  5 Discussion

###  5.1 Extragalactic thick disks

It is clear that structural thin and thick disks are not uncommon in the
Universe (Yoachim & Dalcanton, 2006; Martínez-Lombilla & Knapen, 2019),
potentially being found in the majority of disk galaxies (Comerón et al.,
2018). This study demonstrates that chemically thick disks, i.e. galaxies that
contain high- and low-[$\alpha$/Fe] populations that have different spatial
distributions, are also found in other Milky Way-like galaxies. From this
object, and a small number of lenticular galaxies in the Fornax cluster (Pinna
et al., 2019a, b) which show similar enrichment patterns, we cannot yet say
whether chemically distinct thick disks are rare, common or ubiquitous,
however we do now know that this structure is not unique to the Milky Way.

The spatial distribution of [$\alpha$/Fe] rich and poor stars in UGC 10738 is
qualitatively similar to that seen in the Milky Way, as shown in Figure 5. At
present, empirically-based SSP models have limited resolution in the
[$\alpha$/Fe] axis, making a more quantitative comparison challenging, however
the ‘knee’ in the [$\alpha$/Fe] – [Fe/H] distribution in UGC 10738 may occur
at somewhat lower values of [Fe/H] than in the Milky Way. The metallicity of
the knee is indicative of the efficiency or rate of star formation during the
formation of the thick disk (Matteucci & Brocato, 1990; Pagel, 1997; Andrews
et al., 2017) — lower star formation efficiency, or lower star formation rates
result in less metal-enriched thick disk stars, indicating that thick disk
formation was less efficient in UGC 10738 than in the Milky Way.

###  5.2 Implications for the formation of the Milky Way

There are two leading explanations for the apparent chemical bimodality
observed in the Milky Way; a similarly bimodal formation history (Chiappini et
al., 1997; Haywood et al., 2016), or as a natural consequence of how the
$\alpha$ elements are produced (Schönrich & Binney, 2009; Sharma et al., 2020;
Vincenzo & Kobayashi, 2020). The bimodal formation history scenario consists
of an early starburst-like phase followed by rapid quenching and a more
gradual extended period of star formation. This initial starburst may be
triggered by an interaction (Grand et al., 2020), or may occur naturally as a
result of the gas accretion history from the cosmic web. If this assumed star
formation and merging history is a necessary prerequisite to form a bimodal
[$\alpha$/Fe] distribution, this would imply that all galaxies with bimodal
[$\alpha$/Fe] distributions should have experienced similar histories,
including a merger-triggered star burst phase $\gtrsim 7$ Gyrs ago.

The alternative scenario does not require a break or pause in star formation,
instead consisting of a smooth star formation history with the bimodality
arising from early, very high star formation rates producing a significant
number of high [$\alpha$/Fe], low [Z/H] stars before rapid enrichment of gas
from Type Ia supernovae causes a sharp transition to the low [$\alpha$/Fe]
sequence (Schönrich & Binney, 2009).

While the chemical enrichment patterns of the two scenarios are, by
construction, extremely similar, their relative likelihoods in a large
population of Milky Way-like galaxies should be very different. Galaxies that
experience a sharp pause in star formation at the required redshift,
especially one triggered by a substantial merger event, are likely to be
relatively rare; Evans et al. (2020) found only 5 per cent of Milky Way-mass
galaxies have similar assembly histories to the Milky Way (undergoing an
early, massive accretion event $\sim 10$ Gyrs ago) in the large-volume
cosmological hydrodynamical EAGLE simulation, with Mackereth et al. (2018)
finding chemical bimodalities to be rare in the same simulation (however Buck,
2020, find gas-rich mergers to be present in all of a small sample of high-
resolution Milky Way analogue simulations). In contrast, Hayden et al. (in
prep.) find chemical bimodalities to be ubiquitous in Milky Way-like galaxies
in a chemodynamical simulation with chemical enrichment chosen to reproduce
the ‘smooth’ scenario described above. These predictions strongly suggest that
the relative prevalence of chemical bimodalities in MWAs has the power to
distinguish between the likely origin of the Milky Way’s own chemical
bimodality.

This tension is further enhanced once early-type disk galaxies with
[$\alpha$/Fe]-enhanced thick disks are included in the population of present
day galaxies with similar formation histories (Pinna et al., 2019b; Poci et
al., 2019, 2021). That early-type disk galaxies are found to contain similar
abundance patterns as found in the Milky Way (i.e. increased mean
[$\alpha$/Fe] off the plane of the disk) is unsurprising. Such galaxies
represent one plausible end point for the Milky Way’s evolutionary history,
suggesting a shared and generic evolutionary pathway for disk galaxies. The
Milky Way is often identified as a so-called ‘green valley’ galaxy (Mutch et
al., 2011), a galaxy already undergoing a transition to the red sequence. When
fully quenched (and assuming no dramatic structural changes in the mean time)
the Milky Way will likely resemble a lenticular galaxy similar to FCC 170
(Pinna et al., 2019a).

If bimodal [$\alpha$/Fe] distributions and chemically distinct thick disks are
found to be generic features of Milky Way-like galaxies this may pose a
challenge for the merger-triggered starburst theory of the origin of the Milky
Way’s own thick disk. A ubiquitous feature is unlikely to be the result of a
highly stochastic process like a major merger, being more likely to arise from
secular processes or the integrated history of accretion and interaction with
the interstellar medium (Sharma et al., 2020; Vincenzo & Kobayashi, 2020). The
lack of chemical bimodalities in large-scale cosmological simulations such as
EAGLE (Mackereth et al., 2018) may indicate those simulations do not
sufficiently resolve the ISM to reproduce disk structures accurately (Lagos et
al., 2018), or include sufficiently sophisticated enrichment physics
(Kobayashi et al., 2020).

Characterising the relative frequency of chemically-distinct disk components
in Milky Way-like galaxies provides an important test of these two competing
theories of thick disk formation. This study demonstrates that, with
sufficiently high quality IFS data it is possible to meaningfully compare the
spatially-resolved chemical distributions of external galaxies to that of the
Milky Way. The next step is to expand the sample of Milky Way galaxies with
the necessary observations, providing the first measurement of the frequency
of chemically distinct components in a statistically significant sample of
Milky Way-like galaxies.

We thank the anonymous referee for their feedback on improving this article.
We would like to thank Eric Emsellem for assistance with the MUSE data
reduction. NS and JvdS acknowledge support of Australian Research Council
Discovery Early Career Research Awards (project numbers DE190100375 and
DE200100461, respectively) funded by the Australian Government. RMcD is the
recipient of an Australian Research Council Future Fellowship (project number
FT150100333).

## References

  * Andrews et al. (2017) Andrews, B. H., Weinberg, D. H., Schönrich, R., & Johnson, J. A. 2017, ApJ, 835, 224, doi: 10.3847/1538-4357/835/2/224
  * Astropy Collaboration et al. (2013) Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, A33, doi: 10.1051/0004-6361/201322068
  * Bittner et al. (2019) Bittner, A., Falcón-Barroso, J., Nedelchev, B., et al. 2019, A&A, 628, A117, doi: 10.1051/0004-6361/201935829
  * Bland-Hawthorn & Gerhard (2016) Bland-Hawthorn, J., & Gerhard, O. 2016, ARA&A, 54, 529, doi: 10.1146/annurev-astro-081915-023441
  * Bryant et al. (2015) Bryant, J. J., Owers, M. S., Robotham, A. S. G., et al. 2015, MNRAS, 447, 2857, doi: 10.1093/mnras/stu2635
  * Buck (2020) Buck, T. 2020, MNRAS, 491, 5435, doi: 10.1093/mnras/stz3289
  * Buder et al. (2020) Buder, S., Sharma, S., Kos, J., et al. 2020, arXiv e-prints, arXiv:2011.02505.  https://arxiv.org/abs/2011.02505
  * Cappellari (2012) Cappellari, M. 2012, pPXF: Penalized Pixel-Fitting stellar kinematics extraction.  http://ascl.net/1210.002
  * Cappellari (2017) —. 2017, MNRAS, 466, 798, doi: 10.1093/mnras/stw3020
  * Cappellari & Copin (2003) Cappellari, M., & Copin, Y. 2003, MNRAS, 342, 345, doi: 10.1046/j.1365-8711.2003.06541.x
  * Cappellari & Emsellem (2004) Cappellari, M., & Emsellem, E. 2004, PASP, 116, 138, doi: 10.1086/381875
  * Chabrier (2003) Chabrier, G. 2003, ApJ, 586, L133, doi: 10.1086/374879
  * Chiappini et al. (1997) Chiappini, C., Matteucci, F., & Gratton, R. 1997, ApJ, 477, 765, doi: 10.1086/303726
  * Comerón et al. (2015) Comerón, S., Salo, H., Janz, J., Laurikainen, E., & Yoachim, P. 2015, A&A, 584, A34, doi: 10.1051/0004-6361/201526815
  * Comerón et al. (2018) Comerón, S., Salo, H., & Knapen, J. H. 2018, A&A, 610, A5, doi: 10.1051/0004-6361/201731415
  * Comerón et al. (2016) Comerón, S., Salo, H., Peletier, R. F., & Mentz, J. 2016, A&A, 593, L6, doi: 10.1051/0004-6361/201629292
  * Comerón et al. (2011) Comerón, S., Elmegreen, B. G., Knapen, J. H., et al. 2011, ApJ, 741, 28, doi: 10.1088/0004-637X/741/1/28
  * de Vaucouleurs et al. (1991) de Vaucouleurs, G., de Vaucouleurs, A., Corwin, Herold G., J., et al. 1991, Third Reference Catalogue of Bright Galaxies
  * de Vaucouleurs & Pence (1978) de Vaucouleurs, G., & Pence, W. D. 1978, AJ, 83, 1163, doi: 10.1086/112305
  * Deng et al. (2012) Deng, L.-C., Newberg, H. J., Liu, C., et al. 2012, Research in Astronomy and Astrophysics, 12, 735, doi: 10.1088/1674-4527/12/7/003
  * Evans et al. (2020) Evans, T. A., Fattahi, A., Deason, A. J., & Frenk, C. S. 2020, MNRAS, 497, 4311, doi: 10.1093/mnras/staa2202
  * Falcón-Barroso et al. (2011) Falcón-Barroso, J., Sánchez-Blázquez, P., Vazdekis, A., et al. 2011, A&A, 532, A95, doi: 10.1051/0004-6361/201116842
  * Gaia Collaboration et al. (2016) Gaia Collaboration, Prusti, T., de Bruijne, J. H. J., et al. 2016, A&A, 595, A1, doi: 10.1051/0004-6361/201629272
  * Gaia Collaboration et al. (2018) Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2018, A&A, 616, A1, doi: 10.1051/0004-6361/201833051
  * Gilmore & Reid (1983) Gilmore, G., & Reid, N. 1983, MNRAS, 202, 1025, doi: 10.1093/mnras/202.4.1025
  * Grand et al. (2020) Grand, R. J. J., Kawata, D., Belokurov, V., et al. 2020, MNRAS, 497, 1603, doi: 10.1093/mnras/staa2057
  * Hayden et al. (2015) Hayden, M. R., Bovy, J., Holtzman, J. A., et al. 2015, ApJ, 808, 132, doi: 10.1088/0004-637X/808/2/132
  * Haywood et al. (2013) Haywood, M., Di Matteo, P., Lehnert, M. D., Katz, D., & Gómez, A. 2013, A&A, 560, A109, doi: 10.1051/0004-6361/201321397
  * Haywood et al. (2016) Haywood, M., Lehnert, M. D., Di Matteo, P., et al. 2016, A&A, 589, A66, doi: 10.1051/0004-6361/201527567
  * Jurić et al. (2008) Jurić, M., Ivezić, Ž., Brooks, A., et al. 2008, ApJ, 673, 864, doi: 10.1086/523619
  * Kobayashi et al. (2020) Kobayashi, C., Karakas, A. I., & Lugaro, M. 2020, ApJ, 900, 179, doi: 10.3847/1538-4357/abae65
  * Lagos et al. (2018) Lagos, C. d. P., Schaye, J., Bahé, Y., et al. 2018, MNRAS, 476, 4327, doi: 10.1093/mnras/sty489
  * Liu (2020) Liu, Y. 2020, MNRAS, 497, 3011, doi: 10.1093/mnras/staa2012
  * Mackereth et al. (2018) Mackereth, J. T., Crain, R. A., Schiavon, R. P., et al. 2018, MNRAS, 477, 5072, doi: 10.1093/mnras/sty972
  * Majewski et al. (2017) Majewski, S. R., Schiavon, R. P., Frinchaboy, P. M., et al. 2017, AJ, 154, 94, doi: 10.3847/1538-3881/aa784d
  * Martínez-Lombilla & Knapen (2019) Martínez-Lombilla, C., & Knapen, J. H. 2019, A&A, 629, A12, doi: 10.1051/0004-6361/201935464
  * Matteucci & Brocato (1990) Matteucci, F., & Brocato, E. 1990, ApJ, 365, 539, doi: 10.1086/169508
  * Mutch et al. (2011) Mutch, S. J., Croton, D. J., & Poole, G. B. 2011, ApJ, 736, 84, doi: 10.1088/0004-637X/736/2/84
  * Pagel (1997) Pagel, B. E. J. 1997, Nucleosynthesis and Chemical Evolution of Galaxies
  * Pinna et al. (2019a) Pinna, F., Falcón-Barroso, J., Martig, M., et al. 2019a, A&A, 623, A19, doi: 10.1051/0004-6361/201833193
  * Pinna et al. (2019b) —. 2019b, A&A, 625, A95, doi: 10.1051/0004-6361/201935154
  * Poci et al. (2019) Poci, A., McDermid, R. M., Zhu, L., & van de Ven, G. 2019, MNRAS, 487, 3776, doi: 10.1093/mnras/stz1154
  * Poci et al. (2021) Poci, A., McDermid, R. M., Lyubenova, M., et al. 2021, A&A, 647, A145, doi: 10.1051/0004-6361/202039644
  * Robotham et al. (2018) Robotham, A. S. G., Davies, L. J. M., Driver, S. P., et al. 2018, MNRAS, 476, 3137, doi: 10.1093/mnras/sty440
  * Robotham et al. (2017) Robotham, A. S. G., Taranu, D. S., Tobar, R., Moffett, A., & Driver, S. P. 2017, MNRAS, 466, 1513, doi: 10.1093/mnras/stw3039
  * Salaris & Cassisi (2005) Salaris, M., & Cassisi, S. 2005, Evolution of Stars and Stellar Populations
  * Schönrich & Binney (2009) Schönrich, R., & Binney, J. 2009, MNRAS, 396, 203, doi: 10.1111/j.1365-2966.2009.14750.x
  * Sharma et al. (2020) Sharma, S., Hayden, M. R., & Bland-Hawthorn, J. 2020, arXiv e-prints, arXiv:2005.03646.  https://arxiv.org/abs/2005.03646
  * Soto et al. (2016) Soto, K. T., Lilly, S. J., Bacon, R., Richard, J., & Conseil, S. 2016, MNRAS, 458, 3210, doi: 10.1093/mnras/stw474
  * Taylor et al. (2011) Taylor, E. N., Hopkins, A. M., Baldry, I. K., et al. 2011, MNRAS, 418, 1587, doi: 10.1111/j.1365-2966.2011.19536.x
  * Tully et al. (2013) Tully, R. B., Courtois, H. M., Dolphin, A. E., et al. 2013, AJ, 146, 86, doi: 10.1088/0004-6256/146/4/86
  * Vazdekis et al. (2015) Vazdekis, A., Coelho, P., Cassisi, S., et al. 2015, MNRAS, 449, 1177, doi: 10.1093/mnras/stv151
  * Vincenzo & Kobayashi (2020) Vincenzo, F., & Kobayashi, C. 2020, MNRAS, 496, 80, doi: 10.1093/mnras/staa1451
  * Waters et al. (2020) Waters, C. Z., Magnier, E. A., Price, P. A., et al. 2020, ApJS, 251, 4, doi: 10.3847/1538-4365/abb82b
  * Weilbacher et al. (2016) Weilbacher, P. M., Streicher, O., & Palsa, R. 2016, MUSE-DRP: MUSE Data Reduction Pipeline.  http://ascl.net/1610.004
  * Weilbacher et al. (2020) Weilbacher, P. M., Palsa, R., Streicher, O., et al. 2020, A&A, 641, A28, doi: 10.1051/0004-6361/202037855
  * Yoachim & Dalcanton (2006) Yoachim, P., & Dalcanton, J. J. 2006, AJ, 131, 226, doi: 10.1086/497970
  * Yoachim & Dalcanton (2008) —. 2008, ApJ, 683, 707, doi: 10.1086/590246


[orcid=0000-0001-5390-5872, ] [1] [orcid=0000-0002-6321-0131, ] [1]The work
was done when Krzysztof Olejniczak was an intern at Rossum. Copyright for this
paper by its authors. Use permitted under Creative Commons License Attribution
4.0 International (CC BY 4.0).

26th Computer Vision Winter Workshop, Robert Sablatnig and Florian Kleber
(eds.), Krems, Lower Austria, Austria, Feb. 15-17, 2023

# Text Detection Forgot About Document OCR

Krzysztof Olejniczak  University of Oxford, United Kingdom  Milan Šulc
Rossum.ai, Czech Republic

(2023)

###### Abstract

Detection and recognition of text from scans and other images, commonly
denoted as Optical Character Recognition (OCR), is a widely used form of
automated document processing with a number of methods available. Yet OCR
systems still do not achieve 100% accuracy, requiring human corrections in
applications where correct readout is essential. Advances in machine learning
enabled even more challenging scenarios of text detection and recognition "in-
the-wild" – such as detecting text on objects from photographs of complex
scenes. While the state-of-the-art methods for in-the-wild text recognition
are typically evaluated on complex scenes, their performance in the domain of
documents is typically not published, and a comprehensive comparison with
methods for document OCR is missing. This paper compares several methods
designed for in-the-wild text recognition and for document text recognition,
and provides their evaluation on the domain of structured documents. The
results suggest that state-of-the-art methods originally proposed for in-the-
wild text detection also achieve competitive results on document text
detection, outperforming available OCR methods. We argue that the application
of document OCR should not be omitted in evaluation of text detection and
recognition methods.

###### keywords:

Text Detection Text Recognition OCR Optical Character Recognition Text In The
Wild

Figure 1: Comparison of detection results performed on documents from FUNSD
dataset (cropped).

##  1 Introduction

Optical Character Recognition (OCR) is a classic problem in machine learning
and computer vision with standard methods [1, 2] and surveys [3, 4, 5, 6]
available. Recent advances in machine learning and its applications, such as
autonomous driving, scene understanding or large-scale image retrieval,
shifted the attention of Text Recognition research towards the more
challenging in-the-wild text scenarios, with arbitrarily shaped and oriented
instances of text appearing in complex scenes. Spotting text in-the-wild poses
challenges such as extreme aspect ratios, curved or otherwise irregular text,
complex backgrounds and clutter in the scenes. Recent methods [7, 8] achieve
impressive results on challenging text in-the-wild datasets like TotalText [9]
or CTW-1500 [10], with F1 reaching 90% and 87% respectively. Although
automated document processing remains one of the major applications of OCR, to
the best of our knowledge, the results of in-the-wild text detection models
were never comprehensively evaluated on the domain of documents and compared
with methods developed for document OCR. This paper reviews several recent
Text Detection methods developed for the in-the-wild scenario [11, 12, 13, 7,
14, 8], evaluates their performance (out of the box and fine-tuned) on
benchmark document datasets [15, 16, 17], and compares their scores against
popular Document OCR engines [18, 19, 2]. Additionally, we adopt publicly
available Text Recognition models [20, 21] and combine them with Text
Detectors to perform two-stage end-to-end text recognition for a complete
evaluation of text extraction.

##  2 Related Work

###  2.1 Document OCR

OCR engines designed for the "standard" application domain of documents range
from open-source projects such as TesseractOCR [2] and PP-OCR [1] to
commercial services, including AWS Textract [18] or Google Document AI [19].
Despite Document OCR being a classic problem with many practical applications,
studied for decades [22, 23], it still cannot be considered ’solved’ – even
the best engines struggle to achieve perfect accuracy. The methodology behind
the commercial cloud services is typically not disclosed. The most
popular111Based on the GitHub repository [24] statistics. open-source OCR
engine at the time of publication, Tesseract [2] (v4 and v5), uses a Long
Short-Term Memory (LSTM) neural network as the default recognition engine.

###  2.2 In-the-wild Text Detection

####  2.2.1 Regression-based Methods

Regression-based Methods follow the object classification approach, reduced to
a single-class problem. TextBoxes [25] and TextBoxes++ [26] locate text
instances with various lengths by using sets of anchors with different aspect
ratios. Various regression-based methods utilize an iterative refinement
strategy, iteratively enhancing the quality of detected boundaries. LOMO [27]
uses an Iterative Refinement Module, which in every step regresses coordinates
of each corner of the predicted boundary, with an attention mechanism. PCR
[28] proposes a top-down approach, starting with predictions of centres and
sizes of text instances, and iteratively improving the bounding boxes using
its Contour Localisation Mechanism. TextBPN++ [8] introduces an Iterative
Boundary Deformation Module, utilizing Transformer Blocks with multi-head
attention [29] encoder and a multi-layer perceptron decoder, to iteratively
adjust vertices of detected instances. Instead of considering vertices of the
bounding boxes, DCLNet [12] predicts quadrilateral boundaries by locating four
lines restricting the corresponding area, representing them in polar
coordinates system. To address the problem of arbitrary-shaped text detection
and accurately model the boundaries of irregular text regions, more
sophisticated bounding boxes representation ideas have been developed. ABCNet
[30] adapts cubic Bezier curves to parametrize curved text instances, gaining
the possibility of fitting non-polygon shapes. FCENet [31] proposes Fourier
Contour Embedding method, predicting the Fourier signature vectors
corresponding to the representation of the boundary in Fourier domain, and
uses them to generate the shape of the instance with Inverse Fourier
Transformation.

####  2.2.2 Segmentation-based Methods

Segmentation-based Methods aim to classify each pixel as either text or non-
text, and generate bounding boxes using post-processing on so obtained pixel
maps. The binary, deterministic nature of such pixel classification problem
may cause learning confusion on the borders of text instances. Numerous
methods address this issue by predicting text kernels (central regions of
instances) and appropriately gathering pixels around them. PSENet [32]
predicts kernels of different sizes and forms bounding boxes by iteratively
expanding their regions. PAN [14] generates pixel classification and kernel
maps, linking each classified text pixel to the nearest kernel.
CentripetalText [33] produces centripetal shift vectors that map pixels to
correct text centres. KPN [34] creates pixel embedding vectors, for each
instance locates the central pixel and retrieves the whole shapes by measuring
the similarities in embedding vectors with scalar product. Vast majority of
segmentation-based methods generate probability maps, representing how likely
pixels are to be contained in some text region, and using certain binarization
mechanism (e.g. by applying thresholding) convert them into binary pixel maps.
However, the thresholds are often determined empirically, and uncareful choice
of them may lead to drastic decrease in performance. To solve this problem,
DBNet [13] proposes a Differentiable Binarization Equation, making the step
between probability and classification maps end-to-end trainable and therefore
letting the network learn how to accurately binarise predictions. DBNet++ [7]
further improves on the baseline by extending the backbone network with an
Adaptive Scale Fusion attention module, enhancing the upscaling process and
obtaining deeper features. TextFuseNet [35] generates features on three
different levels: global-, word- and character-level, and fuses them to gain
relevant context and deeper insight into the image structure. Instead of
detecting words, CRAFT [11] locates text on character-level, predicting the
areas covered by single letters, and links characters of each instance with
respect to the generated affinity map.

##  3 Methods

###  3.1 Text Detection

To cover a wide range of text detectors, we selected methods from Section 2.2
with different approaches: for regression-based methods, we included TextBPN++
as a vertex-focused algorithm and DCLNet as an edge-focused approach. From
segmentation-based methods, we selected DBNet and DBNet++ as pure segmentation
and PAN as an approach linking text pixels to corresponding kernels. Finally,
CRAFT was chosen as a character-level method.

###  3.2 Text Recognition

The ultimate goal of text detection, especially in the case of document
processing, is to recognize the text within the detected instances. Therefore,
to evaluate the suitability of popular in-the-wild detectors for document OCR,
we perform end-to-end measurements with the following text recognition
engines: SAR [20], MASTER [36] and CRNN [21]. The open-source engines were
combined with the detection methods in a two-stage manner: the input image was
initially processed by a detector, which returned bounding boxes. Afterwards,
the corresponding cropped instances were passed to recognition models. As a
point of reference, we compare both the detection and end-to-end recognition
results of the selected methods with predictions of three common engines for
end-to-end document OCR: Tesseract [2], Google Document AI [19] and AWS
Textract [18].

###  3.3 Metric

To measure both detection and end-to-end performance, we used the CLEval [37]
metric. Contrary to metrics such as Intersection over Union (IoU) perceiving
text on word-level, CLEval measures precision and recall on character level.
As a consequence, it slightly reduces the punishment for splitting or merging
problematic instances (e.g dates), providing reliable and intuitive comparison
of the quality of detection and recognition. Additionally, the Recognition
Score evaluated by CLEval, approximately corresponding to the precision of
character recognition, informs about the quality of the recognition engine
specifically on the detected bounding boxes.

Table 1: Document datasets used in the experiments for text detection and recognition.  Dataset | Training images | Test images | Document Types | Language
---|---|---|---|---
FUNSD [15] | 149 | 50 | Distorted forms, surveys, reports | English
CORD [16] | 900 | 100 | Photos of Indonesian receipts | English
XFUND [17] | 1043 | 350 | Clean scanned forms | Multilingual

##  4 Experiments

###  4.1 Training Strategies

DBNet [13], DBNet++ [7] and PAN [14] were fine-tuned for 100 epochs (600
epochs in case of FUNSD) with batch size of 8 and initial learning rate set to
0.0001 and decreasing by a factor of 10 at the 60th and 80th epoch (200th and
400th for FUNSD). Baselines, pre-trained on SynthText [38] (DBNet, DBNet++) or
ImageNet [39] (PAN), were downloaded from the MMOCR 0.6.2 Model Zoo [40].
DCLNet [12] was fine-tuned from a pre-trained model [41] on each dataset for
150 epochs with batch size of 4, initial learning rate of 0.001, decaying to
0.0001. For each dataset, TextBPN++ [8] was fine-tuned from a pre-trained
model [42] for 50 epochs with batch size of 4, learning rate of 0.0001 and
data augmentations consisting of flipping, cropping and rotations. Given no
publicly-available training scripts for CRAFT, during the experiments, we used
the MLT model from the github repository [43] without fine-tuning. All
experiments were performed using Adam optimizer with momentum 0.9, on a single
GPU with 11 GB of VRAM (GeForce GTX-1080Ti).

Table 2: Comparison of the detection performance of the chosen methods on benchmark datasets, with respect to the CLEval metric. "P", "R" and "F1" represent the precision, recall and F1-score, respectively. Method | FUNSD | CORD | XFUND
---|---|---|---
P | R | F1 | P | R | F1 | P | R | F1
PAN [14] | 96.25 | 70.57 | 81.44 | 98.92 | 97.33 | 98.12 | 96.96 | 77.90 | 86.39
DBNet [13] | 96.02 | 96.11 | 96.07 | 97.94 | 99.17 | 98.55 | 97.04 | 95.58 | 96.30
DBNet++ [7] | 97.45 | 97.40 | 97.42 | 97.58 | 99.60 | 98.58 | 97.87 | 97.93 | 97.90
TextBPN++ [8] | 96.63 | 95.59 | 96.11 | 98.65 | 99.74 | 99.19 | 97.88 | 94.29 | 96.05
DCLNet [12] | 94.16 | 95.35 | 94.75 | 98.67 | 97.91 | 98.29 | 98.22 | 98.17 | 98.20
CRAFT [11] | 97.84 | 95.72 | 96.77 | 94.25 | 88.46 | 91.26 | 89.75 | 93.02 | 91.36
Tesseract [2] | 80.13 | 73.80 | 76.83 | 76.46 | 47.38 | 58.51 | 85.84 | 87.47 | 86.65
Document AI [19] | 95.56 | 89.77 | 92.57 | 92.90 | 93.71 | 93.30 | 89.49 | 90.68 | 90.08
AWS Textract [18] | 97.50 | 95.89 | 96.69 | 80.60 | 84.79 | 82.64 | 97.64 | 88.14 | 92.65

###  4.2 Detection Results

Results of the text detection methods selected in Section 3.1 on the datasets
from Table 1 are presented in Table 2. On FUNSD dataset, DBNet++ achieves both
the highest detection recall (97.40%) and F1-score (97.42%). The highest
precision rate, 97.84% was scored by CRAFT. PAN performed the weakest out of
all considered in-the-wild algorithms, scoring just 81.44% F1-score. Despite
having achieved better results on FUNSD, segmentation-based approaches were
outperformed by regression-based methods on CORD and XFUND. TextBPN++ proved
to be the best performing algorithm on CORD in terms of recall and F1-score,
scoring 99.74% and 99.19%, respectively. DCLNet, for which the best precision
rate on CORD (98.67%) was recorded, achieved superior results on XFUND,
outperforming the remaining methods with respect to all three measures:
precision - 98.22%, recall - 98.17% and F1-score \- 98.20%. Out of the
considered popular engines for end-to-end document OCR, AWS Textract presented
the best performance on the domain of scans of structured documents – FUNSD
and XFUND – scoring 96.69% and 92.65% F1-score, respectively. Google Document
AI generalized remarkably better to distorted photos of receipts from the CORD
dataset, achieving 93.30% F1-score, surpassing the scores of AWS Textract and
Tesseract. The results show that in-the-wild detectors fine-tuned on document
datasets can outperform popular OCR engines on the domain of structured
documents in terms of the CLEval detection metric. However, the results for
the predictions of pre-trained detectors may not be fully representative due
to differences in splitting rules. E.g. Document AI creates separate instances
for special symbols, e.g. brackets, leading to undesired splitting of words
like "name(s)" into several fragments, lowering precision and recall. On all
experimented datasets, all fine-tuned in-the-wild text detection models
reached high prediction scores, proving themselves capable of handling text in
structured documents. Qualitative analysis of detectors’ predictions revealed
that the major sources of error were incorrect splitting of long text
fragments (e.g e-mail addresses), merging instances in dense text regions and
missing short stand-alone text, such as single-digit numbers.

Table 3: Comparison of the recognition performance of the chosen text detection methods combined with MMOCR’s [44] SAR and MASTER default models, fine-tuned SAR, and docTR’s [45] CRNN default model, on FUNSD and CORD, with respect to the CLEval metric. "P", "R", "F1" and "S" represent the end-to-end precision, recall, F1-score and Recognition Score, respectively. Recognition | Detection | FUNSD | CORD
---|---|---|---
P | R | F1 | S | P | R | F1 | S
| PAN [14] | 76.14 | 74.17 | 75.14 | 79.79 | 82.04 | 84.27 | 83.14 | 84.76
DBNet [13] | 79.10 | 82.51 | 80.77 | 83.33 | 82.76 | 85.79 | 84.25 | 85.49
SAR [20] | CRAFT [11] | 83.75 | 85.16 | 84.45 | 85.92 | 79.62 | 76.93 | 78.25 | 86.37
(baseline) | TextBPN++ [8] | 84.90 | 87.87 | 86.36 | 88.86 | 83.56 | 87.00 | 85.25 | 86.58
| DBNet++ [7] | 80.04 | 83.53 | 81.75 | 82.85 | 82.95 | 86.66 | 84.76 | 85.89
| DCLNet [12] | 77.67 | 82.27 | 79.91 | 81.80 | 82.75 | 85.53 | 84.11 | 86.16
| PAN [14] | 86.37 | 76.61 | 81.20 | 90.23 | 87.73 | 88.95 | 88.34 | 90.59
DBNet [13] | 87.48 | 88.07 | 87.77 | 91.90 | 91.12 | 94.00 | 92.54 | 94.02
SAR [20] | CRAFT [11] | 88.14 | 86.48 | 87.30 | 90.39 | 84.98 | 79.19 | 81.99 | 91.53
(fine-tuned) | TextBPN++ [8] | 88.12 | 88.32 | 88.22 | 92.16 | 91.46 | 96.21 | 93.77 | 94.77
| DBNet++ [7] | 89.15 | 89.83 | 89.49 | 92.13 | 90.40 | 93.83 | 92.09 | 93.54
| DCLNet [12] | 86.10 | 87.30 | 86.70 | 90.46 | 87.69 | 90.02 | 88.84 | 91.58
MASTER [36] | PAN [14] | 77.50 | 74.58 | 76.01 | 81.10 | 90.25 | 92.12 | 91.17 | 93.16
DBNet [13] | 80.30 | 83.11 | 81.68 | 84.55 | 91.94 | 94.31 | 93.11 | 94.62
CRAFT [11] | 82.06 | 82.90 | 82.48 | 84.22 | 85.81 | 81.86 | 83.79 | 92.93
TextBPN++ [8] | 82.10 | 83.93 | 83.00 | 85.96 | 91.77 | 94.79 | 93.26 | 94.78
DBNet++ [7] | 81.33 | 83.99 | 82.64 | 84.13 | 91.39 | 94.63 | 92.98 | 94.48
DCLNet [12] | 79.55 | 82.85 | 81.17 | 83.31 | 90.01 | 92.28 | 91.13 | 93.71
CRNN [21] | PAN [14] | 90.31 | 87.14 | 88.70 | 94.00 | 95.70 | 96.52 | 96.10 | 98.65
DBNet [13] | 89.07 | 91.56 | 90.30 | 93.24 | 96.00 | 97.51 | 96.75 | 98.67
CRAFT [11] | 91.20 | 91.67 | 91.43 | 93.40 | 93.12 | 87.25 | 90.09 | 98.73
TextBPN++ [8] | 89.94 | 91.80 | 90.86 | 93.86 | 95.35 | 97.71 | 96.52 | 98.48
DBNet++ [7] | 90.97 | 93.52 | 92.23 | 93.71 | 95.43 | 97.85 | 96.62 | 98.51
DCLNet [12] | 89.84 | 92.95 | 91.37 | 93.16 | 95.04 | 96.34 | 95.69 | 98.52
Tesseract [2] | 73.84 | 73.84 | 69.09 | 88.48 | 73.96 | 44.33 | 55.43 | 93.55
Google Document AI [19] | 90.83 | 92.03 | 91.42 | 94.80 | 88.06 | 90.97 | 89.49 | 98.61
AWS Textract [18] | 93.61 | 95.46 | 94.53 | 95.78 | 84.53 | 82.13 | 83.32 | 96.63

###  4.3 Recognition Results

End-to-end text recognition results combining fine-tuned in-the-wild detectors
with SAR [20] and MASTER [36] models from MMOCR 0.6.2 Model Zoo [46], and CRNN
[21] from docTR [45] are listed in Table 3. The XFUND dataset was skipped for
this experiment since it contains Chinese and Japanese characters, for which
the recognition models were not trained. On FUNSD, the end-to-end measurement
outcomes followed the patterns from detection: equipped with CRNN as the
recognition engine, DBNet++ proved to be the best tuned model in terms of
CLEval end-to-end Recall (93.52%) and F1-score (92.23%), losing only to CRAFT
in terms of precision. Much higher F1-score (+2%) was measured for AWS
Textract, whose end-to-end results outperformed all of the considered
algorithms. It is important to note that the Recognition Score for AWS
Textract reached almost 96%, surpassing CRNN’s scores by c.a. 2%. This
suggests that the recognition engine used in AWS Textract, performing much
more accurately on FUNSD than the CRNN model, may have been a crucial reason
for the good results. When evaluated on CORD, models with Differentiable
Binarization scored the highest marks in all end-to-end measures: recall
(DBNet++), precision and F1-score (DBNet); significantly surpassing the
remaining methods. Interestingly, despite obtaining the best recall rate,
DBNet++ did not beat the simpler DBNet in terms of end-to-end F1-score. The
predictions of regression-based approaches, better than segmentation-based
ones when pure detection scores were measured, appeared to combine slightly
worse with CRNN. TextBPN++, however, remained competitive, achieving similar
results to DBNet and DBNet++. Recognition Scores of CRNN, regardless the
choice of in-the-wild detector, exceeded 93% on FUNSD and 98.5% on CORD, once
again demonstrating the suitability of applying these algorithms to document
text recognition. SAR model, not specifically trained on documents, presented
poorer performance: the highest measured F1-scores on FUNSD and CORD were
86.36% and 85.25%, respectively, both obtained by the combination with
TextBPN++. Fine-tuned SAR models achieved slightly higher F1-scores reaching
89.49% on FUNSD (equipped with DBNet++ as the detector) and 93.77% on CORD
(combined with TextBPN++ detections). Despite gaining a noticeable advantage
over the baseline, fine-tuned SAR models did not surpass the performance of
the pre-trained CRNN. Similarly to SAR, the pre-trained MASTER model [46]
worked the best in combination with TextBPN++, achieving F1 score of 83.00% on
FUNSD and 93.26% on CORD.

##  5 Conclusions

Text detection research has witnessed great progress in recent years, thanks
to advancements in deep machine learning. The recently introduced methods
widened the range of possible applications of text detectors, making them
viable for in-the-wild text spotting. This shifted the attention towards more
complex scenarios, including arbitrarily-shaped text or instances with non-
orthogonal orientations. With automated document processing remaining one of
the most relevant commercial OCR applications, we stress the importance of
determining whether the state-of-the-art methods for scene text spotting can
also improve document OCR. Our experiments prove that detectors designed for
in-the-wild text spotting can indeed be applied to documents with great
success. In particular, fine-tuning models such as DBNet++ or TextBPN++
yielded over 96% detection F1-score on FUNSD, over 98% detection F1-score on
CORD and over 96% detection F1-score on XFUND, with respect to the CLEval
metric, outperforming Google Document AI and AWS Textract. Moreover, combining
these detectors with a publicly-available CRNN recognition model in a two-
stage manner consistently achieves over 90% CLEval end-to-end F1-score, even
without explicit fine-tuning of CRNN. We hope the results will bring more
attention to evaluating future Text Detection methods not only in the text-in-
the-wild scenario, but also on the domain of documents.

## Acknowledgement

We acknowledge the help of Bohumír Zámečník, an expert on OCR systems, who
helped with the supervision of Krzysztof’s internship project.

## References

  * Du et al. [2020] Y. Du, C. Li, R. Guo, X. Yin, W. Liu, J. Zhou, Y. Bai, Z. Yu, Y. Yang, Q. Dang, H. Wang,  PP-OCR: A practical ultra lightweight OCR system,  CoRR abs/2009.09941 (2020). URL: https://arxiv.org/abs/2009.09941. arXiv:2009.09941.
  * Kay [2007] A. Kay,  Tesseract: An open-source optical character recognition engine,  Linux J. 2007 (2007) 2.
  * Hamad and Mehmet [2016] K. Hamad, K. Mehmet,  A detailed analysis of optical character recognition technology,  International Journal of Applied Mathematics Electronics and Computers (2016) 244–249.
  * Hegghammer [2021] T. Hegghammer, Ocr with tesseract, amazon textract, and google document ai: A benchmarking experiment, 2021\. URL: osf.io/preprints/socarxiv/6zfvs. doi:10.31235/osf.io/6zfvs.
  * Islam et al. [2017] N. Islam, Z. Islam, N. Noor,  A survey on optical character recognition system,  arXiv preprint arXiv:1710.05703 (2017).
  * Memon et al. [2020] J. Memon, M. Sami, R. A. Khan, M. Uddin,  Handwritten optical character recognition (ocr): A comprehensive systematic literature review (slr),  IEEE Access 8 (2020) 142642–142668.
  * Liao et al. [2022] M. Liao, Z. Zou, Z. Wan, C. Yao, X. Bai,  Real-time scene text detection with differentiable binarization and adaptive scale fusion,  IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).
  * Zhang et al. [2021] S. Zhang, X. Zhu, C. Yang, H. Wang, X. Yin,  Adaptive boundary proposal network for arbitrary shape text detection,  in: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, IEEE, 2021, pp. 1285–1294.
  * Ch’ng et al. [2020] C. K. Ch’ng, C. S. Chan, C. Liu,  Total-text: Towards orientation robustness in scene text detection,  International Journal on Document Analysis and Recognition (IJDAR) 23 (2020) 31–52. doi:10.1007/s10032-019-00334-z.
  * Liu et al. [2019] Y. Liu, L. Jin, S. Zhang, C. Luo, S. Zhang,  Curved scene text detection via transverse and longitudinal sequence connection,  Pattern Recognition 90 (2019) 337–345. URL: https://www.sciencedirect.com/science/article/pii/S0031320319300664. doi:https://doi.org/10.1016/j.patcog.2019.02.002.
  * Baek et al. [2019] Y. Baek, B. Lee, D. Han, S. Yun, H. Lee,  Character region awareness for text detection,  in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 9365–9374.
  * Bi and Hu [2021] Y. Bi, Z. Hu,  Disentangled contour learning for quadrilateral text detection,  in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 909–918.
  * Liao et al. [2020] M. Liao, Z. Wan, C. Yao, K. Chen, X. Bai,  Real-time scene text detection with differentiable binarization,  in: Proc. AAAI, 2020.
  * Wang et al. [2019] W. Wang, E. Xie, X. Song, Y. Zang, W. Wang, T. Lu, G. Yu, C. Shen,  Efficient and accurate arbitrary-shaped text detection with pixel aggregation network,  CoRR abs/1908.05900 (2019). URL: http://arxiv.org/abs/1908.05900. arXiv:1908.05900.
  * Guillaume Jaume [2019] J.-P. T. Guillaume Jaume, Hazim Kemal Ekenel,  Funsd: A dataset for form understanding in noisy scanned documents,  in: Accepted to ICDAR-OST, 2019.
  * Park et al. [2019] S. Park, S. Shin, B. Lee, J. Lee, J. Surh, M. Seo, H. Lee,  Cord: A consolidated receipt dataset for post-ocr parsing (2019).
  * Xu et al. [2022] Y. Xu, T. Lv, L. Cui, G. Wang, Y. Lu, D. Florencio, C. Zhang, F. Wei,  XFUND: A benchmark dataset for multilingual visually rich form understanding,  in: Findings of the Association for Computational Linguistics: ACL 2022, Association for Computational Linguistics, Dublin, Ireland, 2022, pp. 3214–3224. URL: https://aclanthology.org/2022.findings-acl.253. doi:10.18653/v1/2022.findings-acl.253.
  * Amazon [2022] Amazon, Amazon textract, https://aws.amazon.com/textract, 2022\. Accessed: 2022-09-25.
  * Google [2022] Google, Google cloud document ai, https://cloud.google.com/document-ai, 2022\. Accessed: 2022-09-25.
  * Li et al. [2018] H. Li, P. Wang, C. Shen, G. Zhang,  Show, attend and read: A simple and strong baseline for irregular text recognition,  CoRR abs/1811.00751 (2018). URL: http://arxiv.org/abs/1811.00751. arXiv:1811.00751.
  * Shi et al. [2015] B. Shi, X. Bai, C. Yao,  An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition,  CoRR abs/1507.05717 (2015). URL: http://arxiv.org/abs/1507.05717. arXiv:1507.05717.
  * Mori et al. [1999] S. Mori, H. Nishida, H. Yamada, Optical character recognition, John Wiley & Sons, Inc., 1999.
  * Schantz [1982] H. F. Schantz,  The history of ocr, optical character recognition,  Manchester Center, VT: Recognition Technologies Users Association (1982).
  * et al. [2022] S. W. et al., Tesseract open source ocr engine (main repository), https://github.com/tesseract-ocr/tesseract, 2022\. Accessed: 2022-10-14.
  * Liao et al. [2017] M. Liao, B. Shi, X. Bai, X. Wang, W. Liu,  Textboxes: A fast text detector with a single deep neural network,  in: AAAI, 2017.
  * Minghui Liao and Bai [2018] B. S. Minghui Liao, X. Bai,  TextBoxes++: A single-shot oriented scene text detector,  IEEE Transactions on Image Processing 27 (2018) 3676–3690. URL: https://doi.org/10.1109/TIP.2018.2825107. doi:10.1109/TIP.2018.2825107.
  * Zhang et al. [2019] C. Zhang, B. Liang, Z. Huang, M. En, J. Han, E. Ding, X. Ding,  Look more than once: An accurate detector for text of arbitrary shapes,  CoRR abs/1904.06535 (2019). URL: http://arxiv.org/abs/1904.06535. arXiv:1904.06535.
  * Dai et al. [2021] P. Dai, S. Zhang, H. Zhang, X. Cao,  Progressive contour regression for arbitrary-shape scene text detection,  in: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 7389–7398. doi:10.1109/CVPR46437.2021.00731.
  * Vaswani et al. [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin,  Attention is all you need,  CoRR abs/1706.03762 (2017). URL: http://arxiv.org/abs/1706.03762. arXiv:1706.03762.
  * Liu et al. [2020] Y. Liu, H. Chen, C. Shen, T. He, L. Jin, L. Wang,  Abcnet: Real-time scene text spotting with adaptive bezier-curve network,  CoRR abs/2002.10200 (2020). URL: https://arxiv.org/abs/2002.10200. arXiv:2002.10200.
  * Zhu et al. [2021] Y. Zhu, J. Chen, L. Liang, Z. Kuang, L. Jin, W. Zhang,  Fourier contour embedding for arbitrary-shaped text detection,  in: CVPR, 2021.
  * Wang et al. [2019] W. Wang, E. Xie, X. Li, W. Hou, T. Lu, G. Yu, S. Shao,  Shape robust text detection with progressive scale expansion network,  in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 9336–9345.
  * Sheng et al. [2021] T. Sheng, J. Chen, Z. Lian,  Centripetaltext: An efficient text instance representation for scene text detection,  in: Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
  * Zhang et al. [2022] S.-X. Zhang, X. Zhu, J.-B. Hou, C. Yang, X.-C. Yin, Kernel proposal network for arbitrary shape text detection, 2022\. URL: https://arxiv.org/abs/2203.06410. doi:10.48550/ARXIV.2203.06410.
  * Ye et al. [2020] J. Ye, Z. Chen, J. Liu, B. Du,  Textfusenet: Scene text detection with richer fused features,  in: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, International Joint Conferences on Artificial Intelligence Organization, 2020, pp. 516–522.
  * Lu et al. [2019] N. Lu, W. Yu, X. Qi, Y. Chen, P. Gong, R. Xiao,  MASTER: multi-aspect non-local network for scene text recognition,  CoRR abs/1910.02562 (2019). URL: http://arxiv.org/abs/1910.02562. arXiv:1910.02562.
  * Baek et al. [2020] Y. Baek, D. Nam, S. Park, J. Lee, S. Shin, J. Baek, C. Y. Lee, H. Lee,  Cleval: Character-level evaluation for text detection and recognition tasks,  CoRR abs/2006.06244 (2020). URL: https://arxiv.org/abs/2006.06244. arXiv:2006.06244.
  * Gupta et al. [2016] A. Gupta, A. Vedaldi, A. Zisserman,  Synthetic data for text localisation in natural images,  in: IEEE Conference on Computer Vision and Pattern Recognition, 2016.
  * Deng et al. [2009] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei,  Imagenet: A large-scale hierarchical image database,  in: 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255.
  * Kuang et al. [2022] Z. Kuang, H. Sun, Z. Li, X. Yue, T. H. Lin, J. Chen, H. Wei, Y. Zhu, T. Gao, W. Zhang, K. Chen, W. Zhang, D. Lin, Text detection models - mmocr 0.6.2 documentation, https://mmocr.readthedocs.io/en/latest/textdet_models.html, 2022\. Accessed: 2022-10-14.
  * Bi and Hu [2021] Y. Bi, Z. Hu, Pytorch implementation of dclnet "disentangled contour learning for quadrilateral text detection", https://github.com/SakuraRiven/DCLNet, 2021\. Accessed: 2022-10-13.
  * Zhang et al. [2022] S. Zhang, X. Zhu, C. Yang, H. Wang, X. Yin, Arbitrary shape text detection via boundary transformer, https://github.com/GXYM/TextBPN-Plus-Plus, 2022\. Accessed: 2022-09-29.
  * Baek et al. [2019] Y. Baek, B. Lee, D. Han, S. Yun, H. Lee, Official implementation of character region awareness for text detection (craft), https://github.com/clovaai/CRAFT-pytorch, 2019\. Accessed: 2022-10-13.
  * Kuang et al. [2021] Z. Kuang, H. Sun, Z. Li, X. Yue, T. H. Lin, J. Chen, H. Wei, Y. Zhu, T. Gao, W. Zhang, K. Chen, W. Zhang, D. Lin,  Mmocr: A comprehensive toolbox for text detection, recognition and understanding,  arXiv preprint arXiv:2108.06543 (2021).
  * Mindee [2021] Mindee, doctr: Document text recognition, https://github.com/mindee/doctr, 2021\.
  * Kuang et al. [2021] Z. Kuang, H. Sun, Z. Li, X. Yue, T. H. Lin, J. Chen, H. Wei, Y. Zhu, T. Gao, W. Zhang, K. Chen, W. Zhang, D. Lin, Text recognition models - mmocr 0.6.2 documentation, https://mmocr.readthedocs.io/en/latest/textrecog_models.html, 2021\. Accessed: 2022-10-14.