2021

These authors contributed equally to this work.

These authors contributed equally to this work.

1]Condensed Matter Physics Department, School of Physics and Astronomy,
Faculty of Exact Sciences, Tel Aviv University, Tel‐Aviv, 6997801, Israel

2]Center for Light‐Matter Interaction, Tel Aviv University, Tel‐Aviv, 6997801,
Israel

# Shaping Exciton Dynamics in 2D Semiconductors by Tailored Ultrafast Pulses

Omri Meron<EMAIL_ADDRESS>Uri Arieli  Eyal Bahar  Swarup Deb
Moshe Ben-Shalom  Haim Suchowski  [  [

###### Abstract

Excitonic resonances in two-dimensional semiconductors play a crucial role in
enhancing nonlinear light-matter interactions. However, the influence of
resonant dynamics on the time-dependent behavior of these nonlinear effects
has received little attention, as optical wave-mixing is commonly assumed to
be instantaneous. In this study, we investigate the impact of excited state
dynamics on the generation of second and third-order nonlinearities from
Monolayer $\text{WSe}_{\text{2}}$ in ambient conditions. By shaping a sub-10fs
driving pulse to complement the exciton resonant dynamics, we demonstrate a
significant enhancement of four-wave-mixing beyond what is achieved using a
transform-limited pulse. Conversely, a time-reversed pulse shape leads to
destructive quantum interference effects, suppressing nonlinear generation. We
apply this technique simultaneously to two successive exciton states,
showcasing the selective choice in the resonant state that produces
nonlinearity. The ability to control nonlinear carrier dynamics in two-
dimensional semiconductors holds promise for remote noninvasive optical
manipulation in optoelectronic devices.

###### keywords:

Ultrafast Physics, Coherent Control, pulse-shaping, Excitons, WSe2, TMD,
Nonlinear Optics

##  1 Introduction

Crystalline semiconductors are integral components of today’s electronic
devices and are thus of fundamental significance to science and technology
alike. A detailed understanding of their optical response, stemming from the
complex collective dynamics of their periodically arranged charge carriers, is
crucial for exploiting their interactions. Optical coherent control through
pulse-shaping is a well-developed method for approaching this challenge,
aiming to optimize the quantum interference between charge carrier excitations
by pre-designed broadband pulses Goswami2003 . Indeed, optical coherent
control applied to atoms, molecules Silberberg2009 , and quantum dots
Brash2017 , characterized by well-defined energy levels and coherence times
exceeding the picosecond timescale, has resulted in enhancement of the
transient population of an excited level Dudovich2002a , selective quantum
state control Dudovich2002 , creation and dissociation of chemical bonds
Chelkowski1990 ; Assion1998 ; Levin2015 , etc. In the case of crystalline 3D
semiconductors, however, the numerous excited states are separated by minute
energy levels, which makes room-temperature coherent control extremely
challenging.

Layered 2D semiconductors are positioned between these two extremes. The
ultimate out-of-plane confinement and the planner crystalline periodicity make
a perfect playground to explore electronic orders and unique optical
responses. Mono-layer transition metal dichalcogenides (TMDs), in particular,
exhibit a direct energy gap that hosts remarkably stable coulomb-bound
electron-hole pairs referred to as excitons. These quasiparticles, which
dominate the crystals’ optical properties, even at room temperature, exhibit a
Rydberg series of resonant excitonic transitions with decreasing oscillator
strengths Chernikov2014 ; Mueller2018 ; Brem2019 . The first and particularly
prominent 1s orbital state of the band-edge A exciton lays in the visible to
near-infrared optical range and is identified by its relatively high quantum
yield Mak2010 ; Amani2015 . This combination of unique semiconductor physical
properties and heterostructure stacking capabilities Geim2013 ; Ciarrocchi2022
, point TMDs as promising, versatile, and compact building blocks for future
optoelectronic devices Mueller2018 ; Chaves2020 .

Various experimental methods have successfully explored the ultrafast exciton
relaxation dynamics in TMDs Trovatello2020 ; Trovatello2022 ; Poellmann2015 ;
Steinleitner2017 . However, the question of how the coherent exciton evolution
affects the temporal dynamics of its’ nonlinear interactions, has been only
addressed under cryogenic conditions, where the low laying energy eigenstates
are substantially modified and the decoherence timescales are extended to
roughly $800fs$ Moody2015 ; Dey2016 ; Bauer2022 . In this work, we introduce a
coherent control approach to unlock the rapid photo-induced coherent dynamics
of excitonic resonances in $\text{WSe}_{\text{2}}$ at room temperature.
Utilizing a state-of-the-art pulse-shaper, we control the spectral phase of a
sub-10 fs laser pulse. The temporal manipulation of the multi-photon
interaction with the exciton resonance enables us to better understand whether
and how it contributes to the nonlinear wave-mixing response. Our findings
indicate that the 1s A-exciton resonance does not contribute to second
harmonic generation (SHG) and sum frequency generation (SFG) in ambient
conditions, however, it dominates the four-wave-mixing (FWM) response. In
particular, we demonstrate active control over the interfering spectral
pathways generating the FWM, tuning the nonlinearity between a near-zero
signal stemming from destructive interference, up to a 2.6-fold enhanced
constructive interference beyond the nonlinear generation by a transform-
limited pulse. The experimental results fit the dynamics calculated by a
driven damped anharmonic oscillator and rule out the seemingly equivalent
approximation of a weakly perturbed two-level system. Moreover, we
experimentally confirm the presence of the sequential $2s$ state and its
effect on the ultrafast dynamics. Such understanding allows us to further
demonstrate simultaneous control over the $1s$ and $2s$ excited states. This
method provides a unique selective control knob for the coherent exciton
response in TMDs and their heterostructures and marks an important step
towards future use in various on-chip ultrafast and nonlinear optical
applications, such as improved nonlinear sensing Malard2021 , compressed
broadband light sources Liu2017  and efficient frequency converters
Trovatello2021 .

##  2 Results

In our experiments, we use a spatial light modulator (SLM) based pulse shaping
apparatus, illustrated in Fig. 1. The SLM, a one-dimensional liquid crystal
pixelated array, is situated in the Fourier plane of a 4f system Weiner2000 ,
where the sub-$10fs$ ultra-broadband pulse is spatially dispersed into
independent co-aligned spectral components. The SLM allows us to temporally
shape the spectral phase of the pulse $\phi_{SLM}\left(\lambda\right)$, as
illustrated in Fig.1.a. In addition, the Fourier plane also functions as an
ultra-steep edge filter, used to sharply truncate the blue end of the
spectrum. Following the tight focus of a mirror objective, the shaped and
truncated pulse (1.2-1.77eV) interacts with a $\text{WSe}_{\text{2}}$ mono-
layer. The linear reflection measurements we have acquired using our ultra-
broadband pulse (displayed in Fig.1.b), show a resonant line shape around
$1.66eV$, confirming the photo-excitation of the $1s$ resonant state of the
A-exciton Gu2019 ; Mueller2018 . Apart from the linear interaction, the pulse
also interacts nonlinearly with the monolayer, generating various intrapulse
wave-mixing processes, such as SFG and FWM (see Fig1.c, more details in
supplementary note A). After attenuating the truncated fundamental pulse with
a complementary shortpass filter, we collect, via reflection, the anti-stokes
FWM signal from the illuminated sample (see Fig.1.b) Dudovich2002 ;
Suchowski2013 .

Figure 1: Coherent control of the A-exciton resonance in monolayer WSe2.
Experimental setup, an ultra-broadband sub-$10fs$ pulse ($650-1100nm$)
propagates through an SLM situated in the Fourier plane of a 4f system, also
used as a spectral edge filter. The shaped and truncated pulse (690-1100nm) is
then focused on a sample, generating a nonlinear signal which is collected via
reflection and redirected into a spectrometer. (a) Schematic illustration
describing how a negative arctangent spectral phase applied by the SLM,
affects the group delay of the pulse. (b) Linear reflection measurement
(colorful area), collected from a monolayer $\text{WSe}_{\text{2}}$, using the
ultra-broadband pulse ($695-1030nm$). An arctangent phase mask, applied by the
SLM (dashed orange), centered at the A-exciton $1s$ resonant frequency
$E_{X_{1s}}$ and linewidth $\gamma_{X_{1s}}$. The collected FWM signal (dark
blue, $600-685nm$) scaled by a factor of $\times 5\cdot 10^{-6}$ to match the
linear spectral amplitude. (c) Illustration of an intra-pulse FWM process in
which spectral counterparts interact nonlinearly with the 1s A-exciton. The 1s
state lays within the bandgap, separated by a binding energy of $E_{b}$ He2014
.

The noninstantaneous nature of resonant dynamics is characterized by a
temporally asymmetric photo-induced response Dudovich2001 . Provided that the
incoming pulse duration is shorter than the resonant decoherence time scale,
its temporally asymmetric nature can be probed by simply varying the pulse
group delay dispersion (GDD), i.e applying a parabolic spectral phase which
linearly varies the instantaneous frequency of the pulse (also referred to as
Chirping, see Fig.3.a.1) Chelkowski1990 ; Bahar2022 . Our attempt to probe the
temporal exciton dynamics of the 1s transition, in ambient conditions, using
SFG measurements resulted in a temporally symmetric curve. Meaning, the
nonlinear response we have collected is instantaneous and stems solely from
the inversion symmetry breaking along the zig-zag crystallographic axis of the
$\text{WSe}_{\text{2}}$ monolayer. This result does not agree with
observations that report quantum interference effects, stemming from the 1s
exciton state, while measuring single-photon resonant second-harmonic
generation (SHG) from monolayer $\text{WSe}_{\text{2}}$ in low temperatures
Lin2019 . Third-order nonlinearities, on the other hand, are independent of
inversion symmetry considerations, making them suitable for measuring a wider
range of ultrafast phenomena. In the following measurements, we examined the
anti-stokes FWM generated by a monolayer. Figure 2.a displays a GDD scan in
which we found a temporally asymmetric trend with respect to the transform-
limited pulse (TL), where the GDD equals zero. The FWM signal is maximal for a
GDD of $-16fs^{2}$ and reaches a 1.4-fold enhancement with respect to the
nonlinear yield by a TL pulse. There is no doubt that the signal originates
from a noninstantaneous resonant interaction, but the question remains, with
which resonant level do the photons interact?

The multi-leveled energy diagram characterizing excitons in
$\text{WSe}_{\text{2}}$ offers a variety of options for single and multi-
photon resonant excitations. We are interested to verify our ability to
selectively control the coherent dynamics of the $1s$ exciton state. Based on
reports stating the oscillator strength of the $1s$ resonant state is about
one order of magnitude larger than the following states in the exciton Rydberg
series Brem2019 , we assume the ultrafast coherent response of the exciton
resonance could be naively modeled as a single Lorentz oscillator Zhang2022 .
Third-order non-linearity must be accounted for by an anharmonic term, leaving
us with the anharmonic oscillator model (AHO)
$\ddot{x}(t)+2\gamma_{0}\dot{x}+\omega_{0}^{2}x=-\frac{eE(t)}{m}-\alpha
x^{3}$. Here $x\left(t\right)$ is the time-dependent electron-hole
displacement, $\gamma_{0}$ the exciton linewidth, $\omega_{0}$ the resonant
frequency, $e$ the electron charge, $m$ the exciton’s reduced mass, $E(t)$ the
driving ultrafast electric pulse and $\alpha$ the nonlinear correction to the
electron-hole binding potential. Assuming a small perturbation, we can write
the approximate solution as $x=x_{0}+\delta x$, where $x_{0}$ is the solution
to the Lorentz oscillator model, and $\delta x$ is a small perturbative
oscillation due to the nonlinear term $\alpha$. The Fourier-transform of the
solution for $x_{0}$ takes the following form:
$\tilde{x}_{0}(\omega)=\frac{e}{m}\mid\frac{\tilde{E}(\omega)}{D(\omega)}\mid
e^{i(\phi_{E}-\phi_{D})}$ , where
$D(\omega)=\omega^{2}-\omega_{0}^{2}-2i\omega\gamma_{0}$. After neglecting
second-order contributions, the solution to $\delta x$ can be calculated as a
3-fold auto-convolution (see supplementary note D for more information):

$\delta\tilde{x}(\omega)\approx-\alpha\frac{\tilde{x}_{0}(\omega)*\tilde{x}_{0}(\omega)*\tilde{x}_{0}(\omega)}{D(\omega)}.$
(1)

Unlike similar treatment for narrow-band sources where the convolution is
reduced to a sum over a few discrete mixing frequencies Boyd2003 , we are
interested in preserving all intra-pulse four-wave combinations within our
ultra-broadband source. From Eq. (1) it is clear that the nonlinear response,
$\lvert\delta\tilde{x}(\omega)\rvert$, is maximized if
$\tilde{x}_{0}(\omega)=\lvert\tilde{x}_{0}(\omega)\rvert$. This can be
achieved by setting the electric field phase to be:

$\phi_{E}(\omega)=\phi_{D}(\omega)=-tan^{-1}(\frac{2\gamma_{0}\omega}{\omega_{0}^{2}-\omega^{2}})\approx-
tan^{-1}\left(\frac{\omega-\omega_{0}}{\gamma_{0}}\right).$ (2)

The arctangent phase maximizes the instantaneous displacement of the
oscillator. As the oscillator is driven further away from its’ minimum energy,
the potential it experiences can no longer be considered quadratic and the
nonlinear motion should increase. In Fig.2.b we illustrate the relation
between the incoming pulse phase and its consequential influence on the AHO
displacement. Despite a small decrease in incoming pulse peak intensity caused
by the negative arctangent phase, the oscillator displacement actually
increases.

Figure 2: The noninstantaneous resonant dynamics of the 1s exciton state in
monolayer $\text{WSe}_{\text{2}}$. (a) The measured FWM spectrum generated by
a chirped pulse, with a varying GDD of $\beta$. (b) Left: An illustration of
an incident time-dependent electric field, for the TL pulse (orange), an
arctangent phased pulse whose linewidth $\Gamma=-\gamma_{X_{1s}}$ (yellow) and
linewidth $\Gamma=\gamma_{X_{1s}}$ (blue). The incident electric field $E(t)$
interactes with the exciton oscillator, modeled by the AHO model, generating a
polarization proportional to the oscillator displacement $x(t)$. Right: the
time-dependent oscillator displacement resulting from the incoming pulse
shape. A TL pulse generates a displacement with an arctangent phase (orange).
A negative arctangent phased pulse induces a TL displacement (yellow), while a
positive arctangent phased pulse induces a very weak displacement (blue). (c)
The measured FWM spectrum generated by an arctangent phased pulse centered
around the 1s exciton resonance, $\omega_{X_{1s}}$, while varying the spectral
linewidth $\Gamma$. the ploted linewidth is normalized by the exciton’s
linewidth $\Gamma/\gamma_{X_{1s}}$. Inset: three selected FWM spectra for the
three values of $\Gamma$ that correspond to the incoming shaped pulses from
(b).

To verify the AHO predictions we measured a set of arctangent phased pulses.
The phase function
$\phi_{E}(\omega)=tan^{-1}\left(\frac{\omega-\omega_{X_{1s}}}{\Gamma}\right)$
was inserted with a varying linewidth $\Gamma$ while holding the central
frequency $\omega_{{X_{1s}}}$ equal to the $1s$ resonance (obtained by linear
reflection as seen in Fig.1.b, and verified by photoluminescence,
supplementary Fig.C3). The FWM results displayed in Fig.2.c show a nearly
anti-symmetric intensity trend with respect to the sign of $\Gamma$, while the
maximal value is obtained for $\Gamma=-32\left[meV\right]$. This value, which
dictates the width of the arctangent curve, matches the negative value of the
linearly measured resonance linewidth $\gamma_{X_{1s}}$ from Fig.1.b, and
results with an enhancement factor of $2.3$ with respect to the extreme
measured values of $\Gamma$, where the phase approaches a TL pulse. As
$\Gamma$ increases to positive values, the FWM intensity falls dramatically
until it almost vanishes. A similar set of arctangent phased pulses was
measured for a varying resonant frequency $\Omega$ while holding a constant
linewidth of $-\gamma_{X_{1s}}$. A schematic description of the phase
functions and plot of the summed FWM intensity for all three measurement sets
is displayed in Fig.3. The line-shape obtained by the resonance frequency scan
$\Omega$ in Fig.3.c.2 reaches the maximal FWM for
$\Omega=\omega_{X_{1s}}=1.66eV$. These results indicate unequivocally, that
the arctangent phased pulses interfere with the inherent phase of the $1s$
exciton state, as they can be used to increase but also destroy the third-
order nonlinear polarization. Here we note again that SFG results for similar
arctangent phase scans display a nonresonant response (See supplementary
Fig.B2).

While a full theoretical treatment of the complex ultrafast exciton dynamics
of this multi-resonant system stays beyond the scope of this work, we compare
our experimental results with two approximate theoretical models to stress the
advantages of pulse-shaping as a felicitous method for model differentiation.
Apart from the classical AHO model, we solve the ground state to 1s state
transition, from a quantum mechanical point of view, as a two-level system
(TLS). Though the two models predict similar dynamics in the linear regime, a
perturbative derivation for the nonlinear FWM signal induced by continuously
varying pulse shapes introduces pronounced differences (see Supplementary note
D.2) Boyd2003 . In Fig.3 we plot our experimental FWM measurements against the
AHO and TLS predictions for the three different varying phase functions (see
Fig.3.a-c.1). The AHO model undoubtedly predicts the main features arising
from the measurements. In addition, Fig.3.b-c.2 show how the arctangent phase
parameters - width $\Gamma$ and central frequency $\Omega$ influence the FWM,
respectively. We find that the strongest enhancement is reached only when the
arctangent is centered around the 1s exciton resonance,
$\Omega=\omega_{X_{1s}}$, and as mentioned before, its’ width equals the
negative value of the exciton linewidth, $\Gamma=-\gamma_{X_{1s}}$. Such
behavior is observed in the AHO model but differs from the TLS.

Figure 3: FWM induced by different phases, experiment Vs. two theoretical
models: anharmonic oscillator (AHO) and two-level system (TLS). (a-c.1)
Analytical representation and plots of three phase functions applied to the
SLM. The varying parameter is marked in green. (a-c.2) Our experimental FWM
results plotted against theoretical predictions. (a) FWM intensity induced by
a chirped pulse, with a varying GDD of $\beta$. Inset: Theoretical prediction
of FWM induced by a TL pulse, in which The two models give a nearly
indiscernible result. (b-c) FWM intensity induced by an arctangent phased
pulse. (b) The arctangent phase is centered around the 1s exciton resonance
$\omega_{X{{}_{1}s}}$, while the linewidth $\Gamma$ is varied from negative to
positive values. (c) The arctangent phase is set with a constant linewidth
$-\gamma_{X{{}_{1}s}}$, and varying central frequency $\Omega$. In both cases,
(b) and (c), The FWM is maximized when $\Omega$ and $\Gamma$ reach the typical
values comlementary to the $X_{1s}$ inherent phase. All experimental results
are scaled by a factor of 1.2 to fit the Anharmonic oscillator amplitude.

We recognize a mismatch between the theoretical model of a single AHO
oscillator and the measurements shown in Fig.3.a-c.2. A small amplification,
observed along the $\Omega$ scan, for frequencies beyond our pulse bandwidth,
may indicate we are interacting with an additional resonant state. We
performed a phase scan using a superposition of two arctangent functions. one
centered around the $1s$ state while the second is varied in frequency and
linewidth. To our great surprise, These additional scans produced a stronger
enhancement for frequencies corresponding to the next resonant level in the
exciton Rydberg series, the 2s state. Even though there is no spectral overlap
between the pulse and the 2s resonant frequency or surrounding linewidth, the
nonlinear interaction is found to be very sensitive to its inherent phase.
Figure 4 displays two phase scans for frequency $\Omega$ and linewidth
$\Gamma$, and affirms that by the superposition of arctangent phases we can
control the FWM stemming from multiple resonant states. Furthermore, since the
tail of the inherent oscillator phase, crosses into our pulse bandwidth, we
find our method to be very sensitive to such detuned interaction. We reach a
total enhancement factor of $2.6$ with respect to the TL phase. The values
found to produce the optimal enhancement, resonant frequency
$\omega_{X_{2s}}=1.81[eV]$ and linewidth $\gamma_{X_{2s}}=35[meV]$,
qualitatively agree with the literature as the typical 2s state parameters for
ambient conditions Brem2019 .

Figure 4: FWM induced by a superposition of arctangent phases, indicating a
detuned interaction with the $X_{2s}$ state. (a) Linear reflection spectrum
(colorful area) collected from a monolayer $\text{WSe}_{\text{2}}$, using the
ultra-broadband laser. A superposition of arctangent phases applied by the SLM
(dashed orange) and its analytic continuation (dashed gray), centered at the
$X_{1s}$ and $X_{2s}$ resonant frequencies. The collected FWM signal (dark
blue). (b-c) Our experimental FWM results as a function of the superimposed
arctangent central frequency $\Omega$ (b) or linewidth $\Gamma$ (c), applied
by the SLM. In both cases, the FWM is maximized for values that match the $2s$
resonant state $\Omega=\omega_{X_{s2}}$ and $\Gamma=\gamma_{X_{s2}}$.

##  3 Methods

###  3.1 Sample preparation

$\text{WSe}_{\text{2}}$, obtained from HQ Graphene, was exfoliated onto
commercially available polydimethylsiloxane (PDMS) gel-pak. Single-layer
flakes of $\text{WSe}_{\text{2}}$ were identified using optical contrast.
Subsequently, suitable flakes were deterministically stamped from PDMS to
oxidized Si (Si/SiO2: 90nm) at room temperature and ambient conditions. To
rule out interference effects from the thin SiO2 cavity, we verified our
results with an additional sample consisting of a monolayer
$\text{WSe}_{\text{2}}$ on Sapphire.

###  3.2 Ultrabroad-band pulse shaping apparatus

A mode-locked Ti:sapphire femtosecond laser system (Venteon; $7fs$,
$650-1100nm$, $80MHz$) was used as the fundamental beam with linear
polarization. The beam propagates through a 4f Pulse shaping system in which a
spatial light modulator (Jenoptik; SLM-S640d) was placed in the Fourier plane,
where we also placed mechanical barriers serving as edge filters. After
propagating through a GVD mirror pair (Venteon; DCM11), The beam was focused
by a mirror objective (Pike; NA 0.78) to pump the sample ($40\mu W$). The
generated nonlinear signal was collected through the same objective via
reflection. After propagating through two short-pass filters (Semrock;
FF02-694/SP-25), and filtering out the fundamental beam, the emitted nonlinear
signals were focused onto a spectrometer (Princeton instruments;
Pixis:400BReXcelon). The zig-zag axis orientation of the
$\text{WSe}_{\text{2}}$ 2D crystal was found using a lambda-half polarizer
placed in front of the objective. The described apparatus was assembled aiming
for minimum dispersion, such that the focus plane will also serve as a
transform-limited phase plane. Fine dispersion tuning, to reach a TL pulse,
was accomplished using the SLM.

##  4 Conclusion

To conclude, by utilizing ultrafast pulse-shaping methods within the coherence
time of the exciton states, and varying the spectral phase of their driving
laser pulse, we confirmed the dominance of the exciton resonance in the third-
order nonlinear susceptibility $\chi^{(3)}$. We have gained full control over
the exciton coherent dynamics, steering the interfering quantum pathways from
nearly full destruction to an enhanced generation that goes much beyond the
maximal nonlinear yield by a TL pulse. In contrary, we did not find any
resonant contribution to the second-order nonlinear response. This result
indicates that the 1s exciton orbital does not break inversion symmetry, as
opposed to the comprising crystal. Furthermore, not only do our measurements
give good estimations to the resonant frequency and linewidth of the band-edge
A exciton 1s-orbital state, but also the 2s state which is much less
accessible in ambient conditions. The AHO model was shown to predict the main
features describing the ultrafast coherent exciton dynamics, as well as
enabling a reasonable estimation for the third-order nonlinear response from
arbitrarily shaped pulses. We have demonstrated differentiation capabilities
between two theoretical models with indistinguishable linear dynamics, by
solely using the spectral phase degree of freedom. We have performed an
initial demonstration of selective control between two resonant states by only
manipulating the spectral phase, exploiting what we believe is our ability to
control the transient exciton population within a selected excited state
Dudovich2002a . A claim we intend to further confirm experimentally.

Pulse shaping should be further utilized for studying TMDs using ab initio
theoretical modeling, taking into account the many-body effects which portray
ultrafast exciton dynamics Katsch2020a . We believe that by combining the
capabilities stated here, with the valley degree of freedom characterizing
TMDs and the unique heterostructure inter and intra-layer exciton dynamics,
may open the door to an exciting range of optoelectronic applications
Jones2013 ; Ciarrocchi2022 .

Supplementary information Supplementary Notes A-D, Figs. A-C and References
1–13.

Acknowledgments Contributions: U.A. and O.M. conceived and planned the
experiments. U.A. E.B. and O.M. assembled the experimental system. O.M.
carried out the experiments. U.A. planned and carried out the simulations.
S.D. contributed to sample preparation. O.M. took the lead in writing the
manuscript. H.S. and M.B.S provided critical feedback and helped shape the
research and manuscript. H.S. supervised the project.

## Declarations

## References

  *   * (1) Goswami, D.: Optical pulse shaping approaches to coherent control. Physics Reports 374(6), 385–481 (2003). https://doi.org/10.1016/S0370-1573(02)00480-5
  * (2) Silberberg, Y.: Quantum coherent control for nonlinear spectroscopy and microscopy. Annu. Rev. Phys. Chem. 60(1), 277–292 (2009). https://doi.org/10.1146/annurev.physchem.040808.090427
  * (3) Brash, A.J., Liu, F., Fox, A.M.: Ultrafast manipulation of excitons and spins in quantum dots. Quantum Dots for Quantum Information Technologies, 325–357 (2017)
  * (4) Dudovich, N., Oron, D., Silberberg, Y.: Coherent transient enhancement of optically induced resonant transitions. Physical Review Letters 88(12), 123004 (2002). https://doi.org/10.1103/physrevlett.88.123004
  * (5) Dudovich, N., Oron, D., Silberberg, Y.: Single-pulse coherently controlled nonlinear raman spectroscopy and microscopy. Nature 418(6897), 512–514 (2002). https://doi.org/10.1038/nature00933
  * (6) Chelkowski, S., Bandrauk, A.D., Corkum, P.B.: Efficient molecular dissociation by a chirped ultrashort infrared laser pulse. Physical Review Letters 65(19), 2355–2358 (1990). https://doi.org/10.1103/PhysRevLett.65.2355. Accessed 2022-09-12
  * (7) Assion, A., Baumert, T., Bergt, M., Brixner, T., Kiefer, B., Seyfried, V., Strehle, M., Gerber, G.: Control of Chemical Reactions by Feedback-Optimized Phase-Shaped Femtosecond Laser Pulses. Science 282(5390), 919–922 (1998). https://doi.org/10.1126/science.282.5390.919. Accessed 2023-01-01
  * (8) Levin, L., Skomorowski, W., Rybak, L., Kosloff, R., Koch, C.P., Amitay, Z.: Coherent Control of Bond Making. Physical Review Letters 114(23), 233003 (2015). https://doi.org/10.1103/PhysRevLett.114.233003. Accessed 2022-06-26
  * (9) Chernikov, A., Berkelbach, T.C., Hill, H.M., Rigosi, A., Li, Y., Aslan, O.B., Reichman, D.R., Hybertsen, M.S., Heinz, T.F.: Exciton binding energy and nonhydrogenic rydberg series in monolayer ${\mathrm{ws}}_{2}$. Phys. Rev. Lett. 113, 076802 (2014). https://doi.org/10.1103/PhysRevLett.113.076802
  * (10) Mueller, T., Malic, E.: Exciton physics and device application of two-dimensional transition metal dichalcogenide semiconductors. npj 2D Materials and Applications 2(1) (2018). https://doi.org/10.1038/s41699-018-0074-2
  * (11) Brem, S., Zipfel, J., Selig, M., Raja, A., Waldecker, L., Ziegler, J.D., Taniguchi, T., Watanabe, K., Chernikov, A., Malic, E.: Intrinsic lifetime of higher excitonic states in tungsten diselenide monolayers. Nanoscale 11, 12381–12387 (2019). https://doi.org/10.1039/C9NR04211C
  * (12) Mak, K.F., Lee, C., Hone, J., Shan, J., Heinz, T.F.: Atomically thin ${\mathrm{mos}}_{2}$: A new direct-gap semiconductor. Phys. Rev. Lett. 105, 136805 (2010). https://doi.org/10.1103/PhysRevLett.105.136805
  * (13) Amani, M., Lien, D.-H., Kiriya, D., Xiao, J., Azcatl, A., Noh, J., Madhvapathy, S.R., Addou, R., KC, S., Dubey, M., Cho, K., Wallace, R.M., Lee, S.-C., He, J.-H., Ager, J.W., Zhang, X., Yablonovitch, E., Javey, A.: Near-unity photoluminescence quantum yield in MoS2. Science 350(6264), 1065–1068 (2015). https://doi.org/10.1126/science.aad2114. Accessed 2022-09-19
  * (14) Geim, A.K., Grigorieva, I.V.: Van der Waals heterostructures. Nature 499(7459), 419–425 (2013). https://doi.org/10.1038/nature12385. Accessed 2022-09-19
  * (15) Ciarrocchi, A., Tagarelli, F., Avsar, A., Kis, A.: Excitonic devices with van der waals heterostructures: valleytronics meets twistronics. Nature Reviews Materials (2022). https://doi.org/10.1038/s41578-021-00408-7
  * (16) Chaves, A., Azadani, J.G., Alsalman, H., da Costa, D.R., Frisenda, R., Chaves, A.J., Song, S.H., Kim, Y.D., He, D., Zhou, J., Castellanos-Gomez, A., Peeters, F.M., Liu, Z., Hinkle, C.L., Oh, S.-H., Ye, P.D., Koester, S.J., Lee, Y.H., Avouris, P., Wang, X., Low, T.: Bandgap engineering of two-dimensional semiconductor materials. npj 2D Materials and Applications 4(1), 29 (2020). https://doi.org/10.1038/s41699-020-00162-4
  * (17) Trovatello, C., Katsch, F., Borys, N.J., Selig, M., Yao, K., Borrego-Varillas, R., Scotognella, F., Kriegel, I., Yan, A., Zettl, A., Schuck, P.J., Knorr, A., Cerullo, G., Conte, S.D.: The ultrafast onset of exciton formation in 2d semiconductors. Nature Communications 11(1), 5277 (2020). https://doi.org/10.1038/s41467-020-18835-5
  * (18) Trovatello, C., Katsch, F., Li, Q., Zhu, X., Knorr, A., Cerullo, G., Dal Conte, S.: Disentangling Many-Body Effects in the Coherent Optical Response of 2D Semiconductors. Nano Letters 22(13), 5322–5329 (2022). https://doi.org/10.1021/acs.nanolett.2c01309. Accessed 2022-09-18
  * (19) Poellmann, C., Steinleitner, P., Leierseder, U., Nagler, P., Plechinger, G., Porer, M., Bratschitsch, R., Schüller, C., Korn, T., Huber, R.: Resonant internal quantum transitions and femtosecond radiative decay of excitons in monolayer wse2. Nature Materials 14(9), 889–893 (2015). https://doi.org/10.1038/nmat4356
  * (20) Steinleitner, P., Merkl, P., Nagler, P., Mornhinweg, J., Schuller, C., Korn, T., Chernikov, A., Huber, R.: Direct observation of ultrafast exciton formation in a monolayer of WSe2. Nano Letters 17(3), 1455–1460 (2017). https://doi.org/10.1021/acs.nanolett.6b04422
  * (21) Moody, G., Dass, C.K., Hao, K., Chen, C.H., Li, L.J., Singh, A., Tran, K., Clark, G., Xu, X., Berghauser, G., Malic, E., Knorr, A., Li, X.: Intrinsic homogeneous linewidth and broadening mechanisms of excitons in monolayer transition metal dichalcogenides. Nature Communications 6(1) (2015). https://doi.org/10.1038/ncomms9315
  * (22) Dey, P., Paul, J., Wang, Z., Stevens, C.E., Liu, C., Romero, A.H., Shan, J., Hilton, D.J., Karaiskaj, D.: Optical coherence in atomic-monolayer transition-metal dichalcogenides limited by electron-phonon interactions. Phys. Rev. Lett. 116, 127402 (2016). https://doi.org/10.1103/PhysRevLett.116.127402
  * (23) Bauer, J.M., Chen, L., Wilhelm, P., Watanabe, K., Taniguchi, T., Bange, S., Lupton, J.M., Lin, K.-Q.: Excitonic resonances control the temporal dynamics of nonlinear optical wave mixing in monolayer semiconductors. Nature Photonics 16(11), 777–783 (2022). https://doi.org/%****␣sn-article.bbl␣Line␣500␣****10.1038/s41566-022-01080-1. Accessed 2022-12-18
  * (24) M. Malard, L., Lafeta, L., S. Cunha, R., Nadas, R., Gadelha, A., Gustavo Cançado, L., Jorio, A.: Studying 2D materials with advanced Raman spectroscopy: CARS, SRS and TERS. Physical Chemistry Chemical Physics 23(41), 23428–23444 (2021). https://doi.org/10.1039/D1CP03240B. Accessed 2022-11-23
  * (25) Liu, H., Li, Y., You, Y.S., Ghimire, S., Heinz, T.F., Reis, D.A.: High-harmonic generation from an atomically thin semiconductor. Nature Physics 13(3), 262–265 (2017). https://doi.org/10.1038/nphys3946. Accessed 2022-06-05
  * (26) Trovatello, C., Marini, A., Xu, X., Lee, C., Liu, F., Curreli, N., Manzoni, C., Dal Conte, S., Yao, K., Ciattoni, A., Hone, J., Zhu, X., Schuck, P.J., Cerullo, G.: Optical parametric amplification by monolayer transition metal dichalcogenides. Nature Photonics 15(1), 6–10 (2021). https://doi.org/10.1038/s41566-020-00728-0
  * (27) Weiner, A.M.: Femtosecond pulse shaping using spatial light modulators. Review of Scientific Instruments 71(5), 1929–1960 (2000). https://doi.org/10.1063/1.1150614. Accessed 2022-09-12
  * (28) Gu, H., Song, B., Fang, M., Hong, Y., Chen, X., Jiang, H., Ren, W., Liu, S.: Layer-dependent dielectric and optical properties of centimeter-scale 2d WSe2: evolution from a single layer to few layers. Nanoscale 11(47), 22762–22771 (2019). https://doi.org/10.1039/c9nr04270a
  * (29) Suchowski, H., O’Brien, K., Wong, Z.J., Salandrino, A., Yin, X., Zhang, X.: Phase Mismatch–Free Nonlinear Propagation in Optical Zero-Index Materials. Science 342(6163), 1223–1226 (2013). https://doi.org/10.1126/science.1244303. Accessed 2022-07-26
  * (30) He, K., Kumar, N., Zhao, L., Wang, Z., Mak, K.F., Zhao, H., Shan, J.: Tightly bound excitons in monolayer ${\mathrm{wse}}_{2}$. Phys. Rev. Lett. 113, 026803 (2014). https://doi.org/10.1103/PhysRevLett.113.026803
  * (31) Dudovich, N., Dayan, B., Gallagher Faeder, S.M., Silberberg, Y.: Transform-limited pulses are not optimal for resonant multiphoton transitions. Phys. Rev. Lett. 86, 47–50 (2001). https://doi.org/10.1103/PhysRevLett.86.47
  * (32) Bahar, E., Arieli, U., Stern, M.V., Suchowski, H.: Unlocking Coherent Control of Ultrafast Plasmonic Interaction. Laser & Photonics Reviews 16(7), 2100467 (2022). https://doi.org/10.1002/lpor.202100467. Accessed 2023-02-08
  * (33) Lin, K.-Q., Bange, S., Lupton, J.M.: Quantum interference in second-harmonic generation from monolayer WSe2. Nature Physics 15(3), 242–246 (2019). https://doi.org/10.1038/s41567-018-0384-5
  * (34) Zhang, S., Li, B., Chen, X., Ruta, F.L., Shao, Y., Sternbach, A.J., McLeod, A.S., Sun, Z., Xiong, L., Moore, S.L., Xu, X., Wu, W., Shabani, S., Zhou, L., Wang, Z., Mooshammer, F., Ray, E., Wilson, N., Schuck, P.J., Dean, C.R., Pasupathy, A.N., Lipson, M., Xu, X., Zhu, X., Millis, A.J., Liu, M., Hone, J.C., Basov, D.N.: Nano-spectroscopy of excitons in atomically thin transition metal dichalcogenides. Nature Communications 13(1), 542 (2022). https://doi.org/10.1038/s41467-022-28117-x. Accessed 2022-12-25
  * (35) Boyd, R.: Nonlinear Optics. Academic Press, San Diego, CA (2003)
  * (36) Katsch, F., Knorr, A.: Optical preparation and coherent control of ultrafast nonlinear quantum superpositions in exciton gases: A case study for atomically thin semiconductors. PRX 10(4), 041039 (2020). https://doi.org/10.1103/PhysRevX.10.041039
  * (37) Jones, A.M., Yu, H., Ghimire, N.J., Wu, S., Aivazian, G., Ross, J.S., Zhao, B., Yan, J., Mandrus, D.G., Xiao, D., Yao, W., Xu, X.: Optical generation of excitonic valley coherence in monolayer WSe2. Nature Nanotechnology 8(9), 634–638 (2013). https://doi.org/10.1038/nnano.2013.151. Accessed 2022-06-28


# RobustPointSet: A Dataset for Benchmarking Robustness of Point Cloud
Classifiers

Saeid Asgari Taghanaki & Jieliang Luo
Autodesk AI Lab
Ran Zhang, Ye Wang & Pradeep Kumar Jayaraman
Autodesk Research
Krishna Murthy Jatavallabhula
Mila, Université de Montréal
Equal contribution; corresponding author<EMAIL_ADDRESS>
###### Abstract

The 3D deep learning community has seen significant strides in pointcloud
processing over the last few years. However, the datasets on which deep models
have been trained have largely remained the same. Most datasets comprise
clean, clutter-free pointclouds canonicalized for pose. Models trained on
these datasets fail in uninterpretible and unintuitive ways when presented
with data that contains transformations “unseen” at train time. While data
augmentation enables models to be robust to “previously seen” input
transformations, 1) we show that this does not work for unseen transformations
during inference, and 2) data augmentation makes it difficult to analyze a
model’s inherent robustness to transformations. To this end, we create a
publicly available dataset for robustness analysis of point cloud
classification models (independent of data augmentation) to input
transformations, called RobustPointSet. Our experiments indicate that despite
all the progress in the point cloud classification, there is no single
architecture that consistently performs better—several fail drastically—when
evaluated on transformed test sets. We also find that robustness to unseen
transformations cannot be brought about merely by extensive data augmentation.
RobustPointSet can be accessed through
https://github.com/AutodeskAILab/RobustPointSet.

##  1 Introduction

The performance of deep neural networks is often measured by their predictive
behavior on a test set. However, evaluating a model on an independently and
identically distributed (i.i.d.) test set fails to capture its underlying
behavior even if the dataset is large enough Geirhos et al. (2020). Despite
the growing interest in robustness analysis of 2D deep models, less effort has
been made in studying the robustness of the models processing point clouds,
i.e., sparse permutation-invariant point sets representing 3D geometric data.

In practice, neural networks are often deployed to work with real-world
pointcloud data that are likely to be transformed in several ways. Point
clouds obtained from 3D scans could be corrupted by sensor noise, parts of the
object may be missing due to occlusions, the object could be in a different
translated or rotated coordinate system, etc. (Figure 1). Although data
augmentation might help a model to perform well facing a previously seen input
transformation, it does not improve the performance of a model on handling
different unseen transformations. Moreover, it hides the real performance of a
model processing transformed inputs e.g., rotation augmentation improves a
model’s performance on unseen rotated objects, however, this does not indicate
whether the model itself is rotation-invariant.

(a) Original

(b) Transformed

Figure 1: Pointcloud classification models trained on clean 3D datasets (a)
often fail on data with “unseen” transformations (b). We thoroughly analyze
several pointcloud processing architectures and observe that none of them
consistently outperform the others. To further research in the robustness of
pointcloud classifiers, we created RobustPointSet, a publicly available
dataset, and specified evaluation protocols to quantify the robustness of
models to “unseen” transformations during training.

In order to take a step towards critical analysis and study the robustness of
point cloud classification models independent of data augmentation, we created
a benchmark dataset called RobustPointSet which is a variant of ModelNet40
dataset Wu et al. (2015). In addition to the original training and test sets
of ModelNet40, our RobustPointSet consists of six different test sets each
with a plausible transformation.

In this paper, we make the following contributions:

  1. 1.

We create a benchmark dataset for augmentation-independent robustness analysis
of point cloud classification models.

  2. 2.

We empirically demonstrate that despite achieving high classification accuracy
on aligned and noise free objects, there is still a large space for improving
point cloud classification models to be able to operate well on challenging
test samples.

  3. 3.

We show that data-augmentation has a very small effect on unseen
transformations.

##  2 Related Work

Few robust methods exist for handling transformed point clouds during
inference. Recently, Yan et al. Yan et al. (2020) proposed an adaptive
sampling scheme to compute non-local features to make the model robust to
noise. While this method takes a critical step towards robustness analysis, it
is mainly tailored to handle noise. Xiao et al Xiao & Wachs (2020) presented a
descriptor that is insensitive to pose and sparsity. However, it is not clear
if their method generalizes to other unseen transformations during inference.

There are a limited number of datasets that can be used to train point cloud
classification methods. A common dataset is ModelNet40 Wu et al. (2015) which
contains 12,308 CAD models of 40 categories. ShapeNet Chang et al. (2015) is
another large-scale dataset of 3D CAD shapes with approximately 51,000 objects
in 55 categories. However, in these datasets objects are aligned, complete,
and free from any background noise which often is not the case in practice.
ScanObjectNN Uy et al. (2019) is an alternative relatively small dataset with
2,902 samples collected from scanned indoor scene data that are grouped into
15 classes. ScanObjectNN puts together objects from SceneNN Hua et al. (2016)
and ScanNet Dai et al. (2017) datasets to cover some of the real-world
challenges such as background noise and incomplete objects with the goal of
analyzing the general performance of point cloud methods in more challenging
scenarios. However, as a mix of multiple transformations might happen in both
training and test sets of ScanObjectNN, it is impossible to study models’
behaviors on particular transformations separately.

##  3 RobustPointSet

We present RobustPointSet, a dataset designed to evaluate and benchmark the
robustness of pointcloud classifiers to transformations “unseen” at train
time. RobustPointSet is based on ModelNet40 Wu et al. (2015), contains
$12,308$ CAD models of $40$ different categories. In addition to original
train and test sets of size $9,840$ and $2,468$, respectively found in
ModelNet40, RobustPointSet contains $6$ additional “transformed” versions of
the original training and test sets. These sets are used to train and evaluate
models under two different strategies detailed in subsections 4.1 and 4.2. As
shown in Figure 2, the transformations we applied in RobustPointSet include:

  * •

Noise: We add random noise sampled from a normal distribution to each point.

  * •

Missing part: We split each object to 8 axis-aligned chunks and randomly
delete one.

  * •

Occlusion: Given an object, we add a random wall and/or a random part from
another object to it. The random part is taken similar to ‘Missing part’ set,
while walls are points sampled from a plane added to the top, bottom, left,
right, front, or back of the object.

  * •

Sparse: We randomly set $\sim 93\%$ of the 2048 points of each object to zero
i.e., each object has only 128 valid points in this set.

  * •

Rotation: We apply a random rotation on an arbitrary axis to each object.

  * •

Translation: Each object is translated by a 3D vector $t$ sampled from the
standard distribution which is normalized as
$t=\alpha\times(\frac{t}{\|t\|})$, where $\alpha=0.1$.

For all the transformations above, we ensure both training and test sets
remain in the normalized range of $[-1,1]$ along each axis by clipping.

Figure 2: A few samples of the RobustPointSet dataset with different
transformations.

##  4 Baseline Experiments

We designed two strategies to train and evaluate two groups of general and
rotation-invariant models using our RobustPointSet dataset. For all the
methods, we used the default hyper-parameters as reported in their papers. To
assess the extent to which a model is robust to different input
transformations, we do not apply any data augmentation during training in any
of the two strategies explained in subsections 4.1 and 4.2. We encourage
researchers to follow the same setup.

We consider 10 different point cloud baseline models under two main groups:
PointNet Qi et al. (2017a), PointNet++ Qi et al. (2017b) (both MSG and SSG),
DGCNN Wang et al. (2019), DensePoint Liu et al. (2019a), PointCNN Li et al.
(2018), PointMask Taghanaki et al. (2020), PointConv Wu et al. (2019), and
Relation-Shape-CNN Liu et al. (2019b) as general approaches, while SPHnet
Poulenard et al. (2019) and PRIN You et al. (2018) as rotation invariant
methods.

###  4.1 Training-domain validation

In this setup, we trained each method with the original training set and
selected the best performing checkpoint on the original validation set for
each method. We then tested the models with selected checkpoints on the six
transformed unseen test sets. This experiment shows the vulnerability of the
models trained on original data to unseen input transformations. As reported
in Table 1, general classification methods mostly fail on the Noise and Sparse
test sets. Simple rigid transformations like translation and rotation are
sufficient to degrade test set classification accuracy by up to 10% and 80%,
respectively. Methods that leverage local information suffer greatly, with
reduction of up to 85% in accuracy when the notion of neighborhood is changed
in test time through the addition of noise or introduction of sparsity. Among
general approaches, PointNet outperforms others with average accuracy of
61.97%. Although rotation invariant methods such as SPHnet and PRIN show
robustness to rotations and translation, they fail on other transformations
with high error rates.

Table 1: Training-domain validation results on our RobustPointSet test sets. The Noise column for example shows the result of training on the Original train set and testing with the Noise test set. RotInv refers to rotation-invariant models. Type | Method | Original | Noise | Translation | MissingPart | Sparse | Rotation | Occlusion | Avg.
---|---|---|---|---|---|---|---|---|---
General | PointNet | 89.06 | 74.72 | 79.66 | 81.52 | 60.53 | 8.83 | 39.47 | 61.97
PointNet++ (M) | 91.27 | 5.73 | 91.31 | 53.69 | 6.65 | 13.02 | 64.18 | 46.55
PointNet++ (S) | 91.47 | 14.90 | 91.07 | 50.24 | 8.85 | 12.70 | 70.23 | 48.49
DGCNN | 92.52 | 57.56 | 91.99 | 85.40 | 9.34 | 13.43 | 78.72 | 61.28
PointMask | 88.53 | 73.14 | 78.20 | 81.48 | 58.23 | 8.02 | 39.18 | 60.97
DensePoint | 90.96 | 53.28 | 90.72 | 84.49 | 15.52 | 12.76 | 67.67 | 59.40
PointCNN | 87.66 | 45.55 | 82.85 | 77.60 | 4.01 | 11.50 | 59.50 | 52.67
PointConv | 91.15 | 20.71 | 90.99 | 84.09 | 8.65 | 12.38 | 45.83 | 50.54
| R-ShapeCNN | 91.77 | 48.06 | 91.29 | 85.98 | 23.18 | 11.51 | 75.61 | 61.06
RInv | SPHnet | 79.18 | 7.22 | 79.18 | 4.22 | 1.26 | 79.18 | 34.33 | 40.65
PRIN | 73.66 | 30.19 | 41.21 | 44.17 | 4.17 | 68.56 | 31.56 | 41.93

###  4.2 Leave-one-out validation

This experiment shows whether data augmentation helps in generalizing to an
unseen transformation. Here, we trained the models with six training sets and
tested on the test set of the held out set. For example, consider three sets
$\\{A_{\\{tr,ts\\}},B_{\\{tr,ts\\}},C_{\\{tr,ts\\}}\\}$ where $tr$ and $ts$
refer to training and testing sets, respectively. we first train a model using
$\\{A_{tr},B_{tr}\\}$ and test on $C_{ts}$. Next, we train with
$\\{A_{tr},C_{tr}\\}$ and test on $B_{ts}$. Finally, we train with
$\\{B_{tr},C_{tr}\\}$ and test on $A_{ts}$. This experiments shows whether
different transformations help to generalize to an unseen transformation.

Table 2: Leave-one-out validation strategy classification results on our RobustPointSet test sets. For example, the Noise column shows the result of training on {Original, Translation, Missing part, Sparse, Rotation, Occlusion} train sets and testing with the Noise test set. RotInv refers to rotation-invariant models. Type | Method | Original | Noise | Translation | MissingPart | Sparse | Rotation | Occlusion | Avg.
---|---|---|---|---|---|---|---|---|---
General | PointNet | 88.35 | 72.61 | 81.53 | 82.87 | 69.28 | 9.42 | 35.96 | 62.86
PointNet++ (M) | 91.55 | 50.92 | 91.43 | 77.16 | 16.19 | 12.26 | 70.39 | 58.56
PointNet++ (S) | 91.76 | 49.33 | 91.10 | 78.36 | 16.72 | 11.27 | 68.33 | 58.12
DGCNN | 92.38 | 66.95 | 91.17 | 85.40 | 6.49 | 14.03 | 68.79 | 60.74
PointMask | 88.03 | 73.95 | 80.80 | 82.83 | 63.64 | 8.97 | 36.69 | 62.13
DensePoint | 91.00 | 42.38 | 90.64 | 85.70 | 20.66 | 8.55 | 47.89 | 55.26
PointCNN | 88.91 | 73.10 | 87.46 | 82.06 | 7.18 | 13.95 | 52.66 | 57.90
PointConv | 91.07 | 66.19 | 91.51 | 84.01 | 19.63 | 11.62 | 44.07 | 58.30
| R-Shape-CNN | 90.52 | 36.95 | 91.33 | 85.82 | 24.59 | 8.23 | 60.09 | 56.79
RInv | SPHnet | 79.30 | 8.24 | 76.02 | 17.94 | 6.33 | 78.86 | 35.96 | 43.23
PRIN | 76.54 | 55.35 | 56.36 | 59.20 | 4.05 | 73.30 | 36.91 | 51.67

We observe from the results in Table 2 that minor improvements of upto 10% on
average are obtained on a few methods, while others remain relatively
unaffected. The improvement could be attributed to the use of different
transformations during training, which forces the model to choose a robust set
of features to make predictions. However, it is clear that none of the methods
are close to their accuracy on the “Original” test set.

##  5 Conclusion

We have presented RobustPointSet, a dataset for studying the robustness of
point cloud-based neural networks. We benchmarked 10 state-of-the-art models
and showed that none of the models, particularly the ones that exploit local
neighborhood information, do well on transformed test sets. Our results show
that while PointNet Qi et al. (2017a) and DGCNN Wang et al. (2019) perform
well on average, there is no single approach that consistently performs better
than the others across a range of data transformations. This strongly
highlights the need to consider robustness as a metric when evaluating models.
We hope our dataset will facilitate future research on robustness analysis for
point cloud classification models. While software frameworks for pointcloud
processing are increasingly being developed Jatavallabhula et al. (2019);
Chaton et al. (2020); Ravi et al. (2020), the datasets used for training
pointcloud classifiers have largely remaind the same. RobustPointSet addresses
this gap, and we experimentally show the need for developing models robust to
data transformations – particularly those unseen at train time. In future, we
are interested in exploring the design of robust pointcloud classifiers that
mitigate the effects introduced by bias in the datasets used to train them.

## References

  * Chang et al. (2015) Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al.  Shapenet: An information-rich 3d model repository.  _arXiv preprint arXiv:1512.03012_ , 2015.
  * Chaton et al. (2020) Thomas Chaton, Chaulet Nicolas, Sofiane Horache, and Loic Landrieu.  Torch-points3d: A modular multi-task frameworkfor reproducible deep learning on 3d point clouds.  In _2020 International Conference on 3D Vision (3DV)_ , 2020.
  * Dai et al. (2017) Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner.  Scannet: Richly-annotated 3d reconstructions of indoor scenes.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 5828–5839, 2017.
  * Geirhos et al. (2020) Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann.  Shortcut learning in deep neural networks.  _arXiv preprint arXiv:2004.07780_ , 2020.
  * Hua et al. (2016) Binh-Son Hua, Quang-Hieu Pham, Duc Thanh Nguyen, Minh-Khoi Tran, Lap-Fai Yu, and Sai-Kit Yeung.  Scenenn: A scene meshes dataset with annotations.  In _2016 Fourth International Conference on 3D Vision (3DV)_ , pp. 92–101. IEEE, 2016.
  * Jatavallabhula et al. (2019) Krishna Murthy Jatavallabhula, Edward Smith, Jean-Francois Lafleche, Clement Fuji Tsang, Artem Rozantsev, Wenzheng Chen, Tommy Xiang, Rev Lebaredian, and Sanja Fidler.  Kaolin: A pytorch library for accelerating 3d deep learning research.  _arXiv:1911.05063_ , 2019.
  * Li et al. (2018) Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen.  Pointcnn: Convolution on x-transformed points.  In _Advances in neural information processing systems_ , pp. 820–830, 2018.
  * Liu et al. (2019a) Yongcheng Liu, Bin Fan, Gaofeng Meng, Jiwen Lu, Shiming Xiang, and Chunhong Pan.  Densepoint: Learning densely contextual representation for efficient point cloud processing.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 5239–5248, 2019a.
  * Liu et al. (2019b) Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan.  Relation-shape convolutional neural network for point cloud analysis.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 8895–8904, 2019b.
  * Poulenard et al. (2019) Adrien Poulenard, Marie-Julie Rakotosaona, Yann Ponty, and Maks Ovsjanikov.  Effective rotation-invariant point cnn with spherical harmonics kernels.  In _2019 International Conference on 3D Vision (3DV)_ , pp. 47–56. IEEE, 2019.
  * Qi et al. (2017a) Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas.  Pointnet: Deep learning on point sets for 3d classification and segmentation.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 652–660, 2017a.
  * Qi et al. (2017b) Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas.  Pointnet++: Deep hierarchical feature learning on point sets in a metric space.  In _Advances in neural information processing systems_ , pp. 5099–5108, 2017b.
  * Ravi et al. (2020) Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari.  Accelerating 3d deep learning with pytorch3d.  _arXiv:2007.08501_ , 2020.
  * Taghanaki et al. (2020) Saeid Asgari Taghanaki, Kaveh Hassani, Pradeep Kumar Jayaraman, Amir Hosein Khasahmadi, and Tonya Custis.  Pointmask: Towards interpretable and bias-resilient point cloud processing.  _arXiv preprint arXiv:2007.04525_ , 2020.
  * Uy et al. (2019) Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung.  Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 1588–1597, 2019.
  * Wang et al. (2019) Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon.  Dynamic graph cnn for learning on point clouds.  _Acm Transactions On Graphics (tog)_ , 38(5):1–12, 2019.
  * Wu et al. (2019) Wenxuan Wu, Zhongang Qi, and Li Fuxin.  Pointconv: Deep convolutional networks on 3d point clouds.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 9621–9630, 2019.
  * Wu et al. (2015) Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao.  3d shapenets: A deep representation for volumetric shapes.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 1912–1920, 2015.
  * Xiao & Wachs (2020) Chenxi Xiao and J. Wachs.  Triangle-net: Towards robustness in point cloud classification.  _ArXiv_ , abs/2003.00856, 2020.
  * Yan et al. (2020) Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, and Shuguang Cui.  Pointasnl: Robust point clouds processing using nonlocal neural networks with adaptive sampling.  2020\.
  * You et al. (2018) Yang You, Yujing Lou, Qi Liu, Yu-Wing Tai, Weiming Wang, Lizhuang Ma, and Cewu Lu.  Prin: Pointwise rotation-invariant network.  _arXiv preprint arXiv:1811.09361_ , 2018.