# Diffusiophoresis and medium structure control macroscopic particle transport
in porous media

Mamta Jotkar<EMAIL_ADDRESS><EMAIL_ADDRESS>Universidad
Politécnica de Madrid, Spain,
Institute of Environmental Assessment and Water Research, Spanish National
Research Council, Barcelona, Spain  Pietro de Anna<EMAIL_ADDRESS>Institute of Earth Sciences, University of Lausanne, Switzerland  Marco Dentz
<EMAIL_ADDRESS>Institute of Environmental Assessment and Water
Research, Spanish National Research Council, Barcelona, Spain  Luis Cueto-
Felgueroso<EMAIL_ADDRESS>Universidad Politécnica de Madrid, Spain

###### Abstract

In this letter, we show that pore-scale diffusiophoresis of colloidal
particles along local salt gradients manifests in the macroscopic dispersion
of particles in a porous medium. Despite is transient character, this
microscopic phenomenon controls large-scale particle transport by altering
their partitioning between transmitting and dead-end pores. It determines the
distribution of residence and arrival times in the medium. Depending on the
diffusiophoretic mobility, particles can be mobilized from or trapped in dead-
end pores, which provides a means for the controlled manipulation of particles
in porous media.

Diffusiophoresis, anomalous dispersion, porous media

Diffusiophoresis (DP) Derjaguin_1947  is the motion of microscopic particles
driven by local gradients of solute concentration that has been demonstrated
both theoretically PrieveAnderson_1984 ; Anderson_1989_ARFM ; VelegolEtAl_2016
; AultEtAl_2018  and experimentally AbecassisEtAl_2008 ; PalacciEtAl_2010_PRL
; KarEtAl_2015 ; ShinEtAl_2016_PNAS ; BattatEtAl_2019 ; SinghEtAl_2020_PRL  to
be a powerful particle manipulation tool. The physical mechanisms that drive
this complex physicochemical phenomenon can be broken down to two components:
chemiphoresis that occurs due the osmotic pressure gradient along the surface
of a charged particle (at the scale of the particle) and electrophoresis
arising due to the difference in the diffusivities between the cation and the
anion in the electrolyte solution VelegolEtAl_2016 . The physics behind this
phenomenon is well-established PrieveAnderson_1984 ; Anderson_1989_ARFM .
Despite the studies in relatively simpler microfluidic setups that demonstrate
particle focusing ShiEtAl_2016_PRL , particle separation
ShinEtAl_2017_NatCommun ; ShinEtAl_2018 ; RasmussenEtAl_2020 ; JotkarCueto-
Felgueroso_2021 , particle banding StaffeldQuinn_1989 , particle trapping
SinghEtAl_2020_PRL , etc., with the help of DP, its effect on macroscopic
transport within intricate and spatially variable porous structures remains
unexplored.

Flow and transport of dissolved solutes and suspended particles through porous
and confined media are ubiquitous in natural and engineered systems Bear .
Most geological and biological porous media share the common feature of being
spatially variable or heterogeneous. The broad range of variability in pore
size is known to induce anomalous transport. Moreover, the diversity in shape
of the constituent grains induces a rich flow organization that plays an
important role in groundwater contamination and remediation KahlerEtAl_2019_gw
, enhanced hydrocarbon recovery EOR , transport through river sediments
Lei2022  and water filtration systems ShinEtAl_2017_NatCommun ; miele2019 .
The morphology of a porous medium is often characterized by the presence of
cavities or dead-end pores (DEP), which represent the part of the system that
cannot host net fluid transfer, resulting in stagnant flow Lever1985 ;
Nishiyama2017 ; Erktan2020 . These DEPs are connected via a network of
percolating channels or transmitting pores (TP). Such DEP-TP structures
characterize biological tissues Hapfelmeier2010 , soil Erktan2020  and filters
Phillip2011  and lead to anomalous transport of passive tracers
BordoloiEtAl_2022 . To date, the combined role of DP and the complexity of the
porous medium on particle transport remains elusive. Here, we use detailed
numerical pore-scale simulations and analytical modeling to elucidate how DP
couples with the medium structure to alter macroscopic particles transport.

We study a fluid-saturated porous system where a particle suspension gets
displaced by a continuously injected salt solution. The velocity experienced
by each transported particle results from advection in the flow field
$\mathbf{u}$, which is controlled by the medium structure and  imposed flow
rate Dentz2011 ; deAnnaPRF2017 ; Dentz2018 , and due to the diffusiophoretic
drift $\textbf{u}_{dp}$. In the thin Debye layer limit, for dilute solutions
with valence symmetric solutes (e.g., LiCl, NaCl), the diffusiophoretic drift
is proportional to the gradient of the logarithm of the solute concentration

$\textbf{u}_{dp}=\Gamma_{p}\nabla\ln s$ (1)

and the diffusiophoretic mobility $\Gamma_{p}$ is approximately a constant.
$\Gamma_{p}$ is determined by the size and surface charge of the particle
PrieveAnderson_1984 ; Anderson_1989_ARFM

$\displaystyle\Gamma_{p}=\frac{\varepsilon k_{B}T}{\nu
Ze}\left\\{D\zeta-\frac{2k_{B}T}{Ze}\ln\left[1-\tanh^{2}\left(\frac{Ze\zeta}{4k_{B}T}\right)\right]\right\\},$

where $\varepsilon$ is the dielectric permittivity of the medium, $\zeta$ is
the particle zeta potential, $\nu$ is the kinematic viscosity of the medium,
$k_{B}$ is the Boltzmann constant, $T$ is the absolute temperature, $Z$ is the
valence of the constituent ions of the solute, $e$ is the proton charge and
$D=(D_{+}-D_{-})/(D_{+}+D_{-})$ measures the difference in diffusivity $D_{+}$
of the cation and $D_{-}$ of the anion. The logarithmic dependence of the
particle velocity $\textbf{u}_{dp}$ on the solute concentration gradients
$\nabla s$ allows for rapid and efficient particle motion, even in low
concentration areas.

We consider a porous system characterized by DEPs of different size connected
to a network of TPs, similar to the one used in ref. BordoloiEtAl_2022 . The
computational domain is shown in figure 1.  The mean flow is driven from left
to right with the flow rate $U$. The medium is statistically homogeneous with
a mean pore-size $\lambda=30\mu\text{m}$ and porosity $\phi=0.39$. The dual
feature of this medium is characterized by TPs, and DEPs leading to stagnant
flow. While solutes typically diffuse and dissipate gradients within pores
over time scales shorter than the time $\tau_{L}=L/U$ needed to elute a pore-
volume, the junctures of TPs and DEPs serve as excellent candidates to retain
gradients of solute concentration, which in turn trigger DP.

Figure 1: Hyper-uniform porous structure characterized by DEPs and TPs.
Computational domain with white spaces indicating solid grains (bottom).

In the low Reynolds number limit, the fluid-solute-particle dynamics are
governed by the Stokes equation for fluid flow, and the advection-diffusion
equations for the solute and particle transport AultEtAl_2018

$0=\frac{1}{\rho}\left(-\nabla p+\mu{\nabla}^{2}\textbf{u}\right),$ (2a)
$\nabla\cdot\textbf{u}=0,$ (2b) $\frac{\partial s}{\partial
t}+\nabla\cdot(\textbf{u}s)=D_{s}{\nabla}^{2}s,$ (2c) $\frac{\partial
c}{\partial t}+\nabla\cdot(\textbf{u}+\textbf{u}_{dp})c=D_{p}{\nabla}^{2}c,$
(2d)

where u is the two-dimensional velocity field, $p$ is the pressure, $s$ is the
solute concentration and $c$ is the particle concentration, $\mu$ is the
dynamic viscosity of the fluid and $\rho$ its density. $D_{s}$ and $D_{p}$ are
the diffusion coefficient of solute and particles, respectively. Typically,
solutes diffuse much faster than particles such that $D_{s}\gg D_{p}$. Solute
and particle transport can be characterized by the respective Péclet numbers,
$Pe_{s}=U\lambda/D_{s}$ and $Pe_{p}=U\lambda/D_{p}$, which compare the
characteristic diffusion time scales $\tau_{D_{s}}=\lambda^{2}/D_{s}$ and
$\tau_{D_{p}}=\lambda^{2}/D_{p}$ to the advection time $\tau_{v}=\lambda/U$
over the  mean pore length. The diffusiophoretic particle velocity
$\textbf{u}_{dp}$ is given by Eq.(1). The medium is initially saturated with
particles and solute with initial concentrations $c_{i}$ and $s_{i}$,
respectively. At time $t>0$, a sharp front of solute at concentration
$s_{H}\gg s_{i}$ is injected such that the ratio $\chi=s_{i}/s_{H}\ll 1$. This
solute front induces a dynamic and heterogeneous solute concentration gradient
that drives DP. The governing equations (2) are solved numerically for the
pore geometry detailed above. The numerical setup is described in SM .

$t\sim 35\tau_{v}$ $t\sim 70\tau_{v}$ $t\sim 210\tau_{v}$ $t\sim 665\tau_{v}$

(a) Particle trapping $\Gamma_{p}<0$

(b) No DP $\Gamma_{p}=0$

(c) Particle extraction $\Gamma_{p}>0$

Figure 2: Temporal evolution of the dimensionless particle distributions
$c/c_{i}$ for (a) trapping, (b) no-DP, and (c) extraction cases. The
concentration scale is logarithmic between $10^{-5}$ (light) and $10$ (dark).
White spaces indicate solid grains.

Figure 2 shows the temporal evolution of the particle concentration field
without DP ($\Gamma_{p}=0$), and for diffusiophoretic trapping
($\Gamma_{p}<0$), and extraction ($\Gamma_{p}>0$). In the absence of DP,
majority of the particles get dispersed through the TPs leaving behind a small
fraction of particles that accumulate within the DEPs, where the flow is
stagnant, organized in convection rolls BordoloiEtAl_2022 . The only mechanism
through which these localized particles can escape into the TPs is diffusion,
the times scale of which is typically orders of magnitude  larger than
$\tau_{L}$.

The injection of a sharp front of solute at a higher concentration results in
local gradients of solute concentration that drive DP within the DEPs. For
$\Gamma_{p}<0$, that is, when particles migrate from high to low solute
concentrations, DP leads to the trapping of particles inside the DEPs, as
shown in figure 2(a).

When $\Gamma_{p}>0$, DP leads to particle mobilization out of the DEPs because
the particles move towards the higher solute concentration zones in the TPs.
This is seen in the particle distributions shown in Figure 2(c), where the
particles within the DEPs rapidly escape the DEPs and leave the domain from
the right outlet. Figure 3 shows the time evolution of the solute
concentration at a point within a DEP close to the bottom of the DEP. The
increase or decrease of concentration due to DP occurs on the time scale
$\tau_{D_{s}}$, which here is smaller than $\tau_{D_{p}}$ by a factor of
$10^{3}$. Thus, the action of DP on mass transfer between TPs and DEPs is
limited to relatively short initial times.

To understand how DP controls the macroscopic fate of the suspended particles,
we first estimate the fraction $\alpha$ of particles that are trapped in or
mobilized from the DEPs in the initial phase. To this end, we consider a
single DEP connected to a TP SM . For this geometry, we can derive the solute
concentration profile in the DEP, and thus obtain explicit expressions for the
diffusiophoretic drift $u_{dp}$. Combining the latter with conservation of
particle flux at the interface between TP and DEP, we obtain the following
expression for $\alpha$ as a function of
$\Gamma_{p}^{\ast}=\Gamma_{p}/U\lambda$

$\displaystyle\alpha=\alpha_{0}\left[1-\Gamma_{p}^{\ast}(1-\chi)Pe_{s}\right]+H(-\Gamma_{p}^{\ast})\left\\{\frac{{2\Gamma_{p}^{\ast}}^{2}(1-\chi)^{2}Pe_{s}Pe_{p}\ell_{0}^{\ast}}{\pi}\ln\left[\frac{2\Gamma_{p}^{\ast}(1-\chi)Pe_{p}\ell_{0}^{\ast}}{2\Gamma_{p}^{\ast}(1-\chi)Pe_{p}\ell_{0}^{\ast}-\pi}\right]\right\\},$
(3)

where $H(\cdot)$ is the Heaviside step function, and $\alpha_{0}$ the initial
fraction of particles in the DEPs in the absence of DP ($\Gamma_{p}=0$). The
dimensionless length $\ell_{0}^{\ast}$ denotes a characteristic diffusion
scale at the interface between TP and DEP. It is inversely proportional to the
particle Péclet number, $\ell_{0}^{\ast}\sim 1/Pe_{p}$. We set in the
following $\ell_{0}^{\ast}=\beta/Pe_{p}$ with $\beta$ a number of the order of
one. Expression (3) predicts that the particles are depleted from the DEPs for
$\Gamma_{p}^{\ast}\geq 1/(1-\chi)Pe_{s}$. Furthermore, for strongly negative
diffusiophoretic mobility, the rate at which particles are transferred to the
interface is eventually smaller than the diffusiophoretic drift. Thus,  by
taking the limit of $\Gamma_{p}^{\ast}\to-\infty$ in Eq. (3), we find that
$\alpha$ asymptotes toward

$\displaystyle\alpha_{\infty}=\alpha_{0}\left(1+\frac{\pi
Pe_{s}}{\beta}\right).$ (4)

The trapped particle fraction increases linearly with $Pe_{s}$ because the
solute gradients and thus the diffusiophoretic drift increase with increasing
$Pe_{s}$.

Figure 3: Temporal evolution of dimensionless particle concentration
($c/c_{i}$) at an arbitrary point (indicated by grey) within a DEP shown in
the inset for $\Gamma_{p}^{*}=-1$ (blue, trapping), $\Gamma_{p}^{*}=0$ (black,
no DP), and $\Gamma_{p}^{*}=0.4$ (red, extraction). The particle
concentrations shown in the inset are displayed in a logarithmic scale and
range from $10^{-6}$ (light) to $10$ (dark).

Figure 4: (Top panel) Arrival time distribution $F(t)$ at the outlet for (blue
circles) $\Gamma_{p}^{*}=-1$, (black triangles) $0$ and (orange circles)
$0.4$. The solid lines denote the travel-time model in Eq. (5). (Bottom panel)
Particle fraction $\alpha$ in the DEP from the (symbols) numerical data, and
(solid line) the analytical expression (3) for $\ell_{0}^{\ast}=0.65/Pe_{p}$.

Figure 4 shows the distribution of arrival times of particles at the outlet.
Similar to Ref. BordoloiEtAl_2022 , we observe two transport regimes. At times
of the order of $\tau_{L}$, particles at the outlet are produced by advection
and dispersion from the TPs. For times $t\gg\tau_{L}$, the arrival time
distribution deviates from the exponential decay predicted under the classical
dispersion framework and displays a power law tail. This is caused by the
particles that are initially trapped within the DEPs. For $\Gamma_{p}<0$,  an
increasing fraction of particles is trapped in the DEP, which manifests in a
stronger tail than without DP. For $\Gamma_{p}>0$, particles are extracted
from the DEP at initial times, and thus the tail is weaker than for
$\Gamma_{p}\leq 0$.

The arrival time distribution can be modeled as the superposition of the
residence time distributions in the TPs and DEPs BordoloiEtAl_2022

$F(t)=(1-\alpha)F_{0}(t)+\alpha\int_{0}^{\infty}\text{d}\tau\frac{g(t/\tau)}{\tau}f_{D}(\tau),$
(5)

respectively. Note that $\alpha$ is the fraction of trapped particles after
the short initial phase, which can be approximated by expression (3). We
assume that transport in the TP can be characterized by the mean flow velocity
and a hydrodynamic dispersion coefficient. Thus, $F_{0}(t)$ is given by the
superposition of inverse Gaussian distributions as,

$\displaystyle
F_{0}(t)=\frac{1}{L}\int\limits_{0}^{L}dx\frac{x\exp[-(x-\overline{u}t)^{2}/(4D_{h}t)]}{\sqrt{4\pi
D_{h}t^{3}}}.$ (6)

As we see in Figure 4, this approximation is able to capture the early arrival
times, but underestimates the arrival time distribution at intermediate times.
This can be traced back to the velocity variability between pores Dentz2018 .
The residence time distribution within the DEPs is expressed in terms of a
Gamma-distribution $g(t^{\prime})$ of dimensionless residence times
$\tau^{\prime}=\tau/\tau_{D_{p}}$ in a single DEP and the distribution $f_{D}$
of characteristic diffusion times $\tau_{D_{p}}$, given below
BordoloiEtAl_2022

$g(t^{\prime})=\frac{t^{\prime-2/3}\exp(-t^{\prime})}{\Gamma(1/3)},$ (7)

while $f_{D}$ is obtained from the distribution $f_{\Lambda}$ of aspect ratios
$\Lambda$ through the map $\Lambda\to\tau_{D_{p}}=(\lambda\Lambda)^{2}/D_{p}$.

Figure 4 shows the numerically estimated particle fractions versus the
dimensionless diffusiophoretic mobility, and the analytical expression (3) for
$\beta=0.65$. The analytical model describes the full dependence of $\alpha$
on $\Gamma_{p}$ for both extraction from and trapping in DEPs, and thus seems
to capture the controls of DP on the macro-scale dispersion of particles.

In conclusion, the microscopic interactions between DP and flow and transport
through porous media impact the macroscopic fate of particles. Depending on
the diffusiophoretic mobility $\Gamma_{p}$, DP may promote trapping within or
it may lead to particle mobilization from the DEPs. This can be exploited for
preferential particle deposition or removal. DP is a short-term phenomenon
that persists as long as the solute gradients are not dissipated by diffusion,
which is characterized by the time scale $\tau_{D_{s}}=\lambda^{2}/D_{s}$ and
is typically smaller than the characteristic advection time across the medium
i.e. $\tau_{D_{s}}\ll\tau_{L}$. However, despite its short timespan, DP has a
significant impact on macroscopic transport, by reorganizing the partitioning
of particles between the DEPs and TPs, which is quantified by the fraction
$\alpha$.

Our results suggest that DP provides an efficient way of controlling particle
transport through porous media in a reversible manner by changing the
direction of the solute gradient. Moreover, the existence of DEPs is quite
common in natural geological and biological porous media, and serve as
excellent candidates for retaining microscopic gradients of solute
concentration for relatively large times. This opens new avenues for
developing technological solutions to various problems of socio-economic
relevance such as groundwater remediation, enhanced oil recovery, water-
filtration systems, targeted drug delivery and microfluidics for biomedical
applications.

###### Acknowledgements.

This work has received funding from European Union’s Horizon 2020 research and
innovation program under the Marie Skłodowska-Curie G.A. No. 895569. MD and
LCF gratefully acknowledge funding from the Spanish Ministry of Science and
Innovation through the project HydroPore (PID2019-106887GB-C31-C33). P.d.A.
acknowledges the support of FET-Open project NARCISO (ID: 828890) and of Swiss
National Science Foundation (grant ID 200021 172587).

## References

  * (1) B. V. Derjaguin, G. P. Sidorenkov, E. A. Zubashchenkov, and E. V. Kiseleva.  Kinetic phenomena in boundary films of liquids.  Colloid J. USSR, 9:335–347, 1947.
  * (2) D. C. Prieve, J. L. Anderson, J. P. Ebel, and M. E. Lowell.  Motion of a particle generated by chemical gradients. part 2. Electrolytes.  J. Fluid Mech., 148:247–269, 1984.
  * (3) J. L. Anderson.  Colloid transport by interfacial forces.  Annu. Rev. Fluid Mech., 21(1):61–99, 1989.
  * (4) D. Velegol, A. Garg, R. Guha, A. Kar, and M. Kumar.  Origins of concentration gradients for diffusiophoresis.  Soft Matter, 12:4686–4703, 2016.
  * (5) J. T. Ault, S. Shin, and H. A. Stone.  Diffusiophoresis in narrow channel flows.  J. Fluid Mech., 854:420–448, 2018.
  * (6) B. Abécassis, C. Cottin-Bizonne, C. Ybert, A. Ajdari, and L. Bocquet.  Boosting migration of large particles by solute contrasts.  Nat. Mater., 7(10):785–789, 2008.
  * (7) J. Palacci, B. Abécassis, C. Cottin-Bizonne, C. Ybert, and L. Bocquet.  Colloidal motility and pattern formation under rectified diffusiophoresis.  Phys. Rev. Lett., 104:138302, 2010.
  * (8) A. Kar, T.-Y. Chiang, I. Ortiz Rivera, A. Sen, and D. Velegol.  Enhanced transport into and out of dead-end pores.  ACS Nano, 9(1):746–753, 2015.  PMID: 25559608.
  * (9) S. Shin, E. Um, B. Sabass, J. T. Ault, M. Rahimi, P. B. Warren, and H. A. Stone.  Size-dependent control of colloid transport via solute gradients in dead-end channels.  Proc. Natl. Acad. Sci., 113(2):257–261, 2016.
  * (10) S. Battat, J. T. Ault, S. Shin, S. Khodaparast, and H. A. Stone.  Particle entrainment in dead-end pores by diffusiophoresis.  Soft Matter, 15:3879–3885, 2019.
  * (11) N. Singh, Goran T. Vladisavljević, Fran çois Nadal, Cécile Cottin-Bizonne, Christophe Pirat, and Guido Bolognesi.  Reversible trapping of colloids in microgrooved channels via diffusiophoresis under steady-state solute gradients.  Phys. Rev. Lett., 125:248002, 2020.
  * (12) Nan Shi, Rodrigo Nery-Azevedo, Amr I. Abdel-Fattah, and Todd M. Squires.  Diffusiophoretic focusing of suspended colloids.  Phys. Rev. Lett., 117:258001, 2016.
  * (13) Shin S., Shardt O., Warren P. B., and Stone H. A.  Membraneless water filtration using $co_{2}$.  Nat. Commun., 8:15181, 2017.
  * (14) S. Shin, P. B. Warren, and H. A. Stone.  Cleaning by surfactant gradients: Particulate removal from porous materials and the significance of rinsing in laundry detergency.  Phys. Rev. Appl., 9:034012, 2018.
  * (15) M. K. Rasmussen, J. N. Pedersen, and R. Marie.  Size and surface charge characterization of nanoparticles with a salt gradient.  Nat. Commun., 11(1):2337, 2020.
  * (16) Jotkar M. and Cueto-Felgueroso L.  Particle separation through diverging nanochannels via diffusiophoresis and diffusioosmosis.  Phys. Rev. Applied, 16:064067, 2021.
  * (17) P. O. Staffeld and J. A. Quinn.  Diffusion-induced banding of colloid particles via diffusiophoresis: 1\. Electrolytes.  J. Colloid Interface Sci., 130(1):69–87, 1989.
  * (18) J. Bear.  Dynamics of fluids in porous media.  Courier Corporation, 1988.
  * (19) David M. Kahler and Zbigniew J. Kabala.  Acceleration of groundwater remediation by rapidly pulsed pumping: Laboratory column tests.  Journal of Environmental Engineering, 145(1):06018009, 2019.
  * (20) S. Thomas.  Oil and Gas Science and Technology - Rev. IFP, 63 (1):9–19, 2008\.
  * (21) Liang Lei, Taehyung Park, Karl Jarvis, Lingli Pan, Imgenur Tepecik, Yumeng Zhao, Zhuan Ge, Jeong-Hoon Choi, Xuerui Gai, Sergio Andres Galindo-Torres, Ray Boswell, Sheng Dai, and Yongkoo Seol.  Pore-scale observations of natural hydrate-bearing sediments via pressure core sub-coring and micro-ct scanning.  Scentifici Reports, 12(3471).
  * (22) F. Miele, P. de Anna, and M. Dentz.  Stochastic model for filtration by porous materials.  Phys. Rev. Fluids, 4:094101, 2019.
  * (23) Lever D.A., Bradbury M.H., and Hemingway S.J.  The effect of dead-end porosity on rock-matrix diffusion.  Journal of Hydrology, 80:45–76, 1985.
  * (24) Naoki Nishiyama and Tadashi Yokoyama.  Permeability of porous media: Role of the critical pore size.  Journal of Geophysical Research: Solid Earth, 122(9):6955–6971, 2017.
  * (25) Amandine Erktan, Dani Or, and Stefan Scheu.  The physical structure of soil: Determinant and consequence of trophic interactions.  Soil Biology and Biochemistry, 148:107876, 2020.
  * (26) Siegfried Hapfelmeier, Melissa A. E. Lawson, Emma Slack, Jorum K. Kirundi, Maaike Stoel, Mathias Heikenwalder, Julia Cahenzli, Yuliya Velykoredko, Maria L. Balmer, Kathrin Endt, Markus B. Geuking, Roy Curtiss, Kathy D. McCoy, and Andrew J. Macpherson.  Reversible microbial colonization of germ-free mice reveals the dynamics of iga immune responses.  Science, 328(5986):1705–1709, 2010.
  * (27) William A. Phillip, Rachel Mika Dorin, Jörg Werner, Eric M. V. Hoek, Ulrich Wiesner, and Menachem Elimelech.  Tuning structure and properties of graded triblock terpolymer-based mesoporous and hybrid films.  Nano Letters, 11(7):2892–2900, 2011.  PMID: 21648394.
  * (28) A. D. Bordoloi, D. Scheidweiler, M. Dentz, Bouabdellaoui M., Abbarchi M., and de Anna P.  Structure induced laminar vortices control anomalous dispersion in porous media.  Nat. Commun., 13:3820, 2022.
  * (29) Dentz Marco, Le Borgne Tanguy, Englert Andreas, and Bijeljic Branko.  Mixing, spreading and reaction in heterogeneous media: A brief review.  Journal of Contaminant Hydrology, 120-121:1–17, 2011.
  * (30) P. de Anna, B. Quaife, G. Biros, and R. Juanes.  Prediction of velocity distribution from pore structure in simple porous media.  Phys. Rev. Fluids, 2:124103, 2017.
  * (31) M. Dentz, M Icardi, and J. J. Hidalgo.  Mechanisms of dispersion in a porous medium.  J. of Fluid Mech., 841:851–882, 2018.
  * (32) See Supplmental Material at [URL will be inserted by publisher] for the setup of the detailed numerical simulations and the details on the analytical model.


# The Art of Prompting: Event Detection based on Type Specific Prompts

Sijia Wang♣, Mo Yu♠, Lifu Huang♣
♣Virginia Tech, ♠WeChat AI
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We compare various forms of prompts to represent event types and develop a
unified framework to incorporate the event type specific prompts for
supervised, few-shot, and zero-shot event detection. The experimental results
demonstrate that a well-defined and comprehensive event type prompt can
significantly improve the performance of event detection, especially when the
annotated data is scarce (few-shot event detection) or not available (zero-
shot event detection). By leveraging the semantics of event types, our unified
framework shows up to 24.3% F-score gain over the previous state-of-the-art
baselines.

##  1 Introduction

Event detection Grishman (1997); Chinchor and Marsh (1998); Ahn (2006) is the
task of identifying and typing event mentions from natural language text.
Supervised approaches, especially deep neural networks Chen et al. (2020); Du
and Cardie (2020); Lin et al. (2020); Liu et al. (2020); Li et al. (2020); Lyu
et al. (2021), have shown remarkable performance under a critical prerequisite
of a large amount of manual annotations. However, they cannot be effectively
generalized to new languages, domains or types, especially when the
annotations are not enough Huang et al. (2016); Huang and Ji (2020); Lai et
al. (2020b); Shen et al. (2021) or there is no annotations available Lyu et
al. (2021); Zhang et al. (2021b); Pasupat and Liang (2014).

Type Name |  Attack
---|---
Definition |  Violent or physical act causing harm
|  or damage
Seed Trigger |  Invaded, airstrikes, overthrew, ambushed
Type Structure |  Attack, Attacker, Instrument, Victim, Target, Place
APEX Prompt |  Attack, invaded airstrikes overthrew ambushed, an Attacker physically attacks a Target with Instrument at a Place
Table 1: Example of various forms prompt for the event type Conflict: Attack

Recent studies have shown that both the accuracy and generalizability of event
detection can be improved via leveraging the semantics of event types based on
various forms of prompts, such as event type specific queries Lyu et al.
(2021); Du and Cardie (2020); Liu et al. (2020), definitions Chen et al.
(2020), structures Lin et al. (2020); Wang et al. (2019), or a few prototype
event triggers Wang and Cohen (2009); Dalvi et al. (2012); Pasupat and Liang
(2014); Bronstein et al. (2015); Lai and Nguyen (2019); Zhang et al. (2021b);
Cong et al. (2021). Table 1 shows an example of each form of event type prompt
for detecting event mentions from the input sentence. These studies further
encourage us to take another step forward and think about the following three
questions: (1) does the choice of prompt matter when the training data is
abundant or scarce? (2) what’s the best form of prompt for event detection?
(3) how to best leverage the prompt to detect event mentions?

To answer the above research questions, we conduct extensive experiments with
various forms of prompts for each event type, including (a) event type name,
(b) prototype seed triggers, (c) definition, (d) event type structure based on
both event type name and its predefined argument roles, (e) free parameter
based continuous soft prompt, and (f) a more comprehensive event type
description (named APEX prompt) that covers all the information of prompts
(a)-(d), under the settings of supervised event detection, few-shot and zero-
shot event detection. We observe that (1) by considering the semantics of
event types with most forms of prompts, especially seed triggers and the
comprehensive event type descriptions, the performance of event detection
under all settings can be significantly improved; (2) Among all forms of event
representations, the comprehensive description based prompts show to be the
most effective, especially for few-shot and zero-shot event detection; (3)
Different forms of event type representations provide complementary
improvements, indicating that they capture distinct aspects and knowledge of
the event types.

In summary, our work makes the following contributions:

  * •

we investigate various forms of prompts to represent event types for both
supervised and weakly supervised event detection, and prove that a well-
defined and comprehensive event type prompt can dramatically improve the
performance of event detection and the transferability from old types to new
types.

  * •

we developed a unified framework to leverage the semantics of event types with
prompts for supervised, few-shot and zero-shot event detection, and
demonstrate state-of-the-art performance with up to 24.3% F-score improvement
over the strong baseline methods.

##  2 Related Work

#### Supervised Event Detection:

Most of the existing Event Detection studies follow a supervised learning
paradigm Ji and Grishman (2008); Liao and Grishman (2010); McClosky et al.
(2011); Li et al. (2013); Chen et al. (2015); Cao et al. (2015); Feng et al.
(2016); Yang and Mitchell (2016); Nguyen et al. (2016); Zhang et al. (2017);
Lin et al. (2020); Wang et al. (2021b), however, they cannot be directly
applied to detect new types of events. Recently studies have shown that, by
leveraging the semantics of event types based on type-specific questions Du
and Cardie (2020); Liu et al. (2020); Li et al. (2020); Lyu et al. (2021) or
seed event triggers Bronstein et al. (2015); Lai and Nguyen (2019); Wang et
al. (2021a), the event detection performance can be improved. However, it’s
still unknown that whether they are the best choices of representing the
semantics of event types.

#### Few-shot Event Detection:

Two primary learning strategies in few-shot classification tasks are Meta-
Learning Kang et al. (2019); Li et al. (2021); Xiao and Marlet (2020); Yan et
al. (2019); Chowdhury et al. (2021), and Metric Learning Sun et al. (2021);
Wang et al. (2020b); Zhang et al. (2021a); Agarwal et al. (2021). Several
studies have exploited metric learning to align the semantics of candidate
events with few examples of the novel event types for few-shot event detection
Lai et al. (2020a); Deng et al. (2020); Lai et al. (2020b); Cong et al.
(2021); Chen et al. (2021); Shen et al. (2021). However, due to the limited
annotated data and the diverse semantics of event mentions, it’s hard to
design a metric distance to accurately capture the semantic similarity between
the seed mentions and new ones.

#### Zero-shot Event Detection:

The core idea of zero-shot learning is to learn a mapping function between
seen classes and their corresponding samples, and then apply it to ground new
samples to unseen classes. Huang et al. (2018) first exploited zero-shot event
extraction by leveraging Abstract Meaning Representation Banarescu et al.
(2013) to represent event mentions and types into a shared semantic space.
Recent studies Zhang et al. (2021b); Lyu et al. (2021) further demonstrate
that without using any training data, by leveraging large external corpus with
abundant anchor triggers, zero-shot event detection can also be achieved with
decent performance. However, such approaches cannot properly identify event
mentions, i.e., distinguishing event mentions from none-event tokens.

#### Prompt Learning

Prompt learning aims to learn a task-specific prompt while keeping most of the
parameters of the model freezed Li and Liang (2021); Hambardzumyan et al.
(2021); Brown et al. (2020). It has shown competitive performance in a wide
variety of applications in natural language processing Raffel et al. (2020);
Brown et al. (2020); Shin et al. (2020); Jiang et al. (2020); Lester et al.
(2021); Schick and Schütze (2021b). Previous work either use a manual Petroni
et al. (2019); Brown et al. (2020); Schick and Schütze (2021a) or automated
approach Jiang et al. (2020); Yuan et al. (2021); Li and Liang (2021) to
create prompts. In this work, we compare various forms of template based and
free-parameter based prompts for event detection task under both supervised
and weakly supervised setting.

Figure 1: Overview of the unified framework for event detection based on event
type specific prompts.

##  3 Problem Formulation

In this work, we aim to compare various forms of prompts to represent the
event types under different settings, including supervised event detection,
few-shot event detection and zero-shot event detection. Here, we first provide
a definition for each setting of the event detection task and then describe
the various forms of event type prompts.

###  3.1 Settings of Event Detection

#### Supervised Event Detection

We follow the conventional supervised event detection setting where both the
training, validation and evaluation data sets cover the same set of event
types. The goal is to learn a model $f$ on the training data set and evaluate
its capability on correctly identifying and classifying event mentions for the
target event types.

#### Few-shot Event Detection

There are two separate training data sets for few-shot event detection: (1) A
large-scale base training data set
$\mathcal{D}_{base}=\\{(\mathbf{x}_{i},\mathbf{y}_{i})\\}_{i=1}^{M}$ that
covers the old event types (named base types) with abundant annotations and
$M$ denotes the number of base event types; (2) a smaller training data set
$\mathcal{D}_{novel}=\\{(\mathbf{x}_{j},\mathbf{y}_{j})\\}_{j=1}^{N\times K}$
that covers $N$ novel event types, with $K$ examples each. Note that the base
and novel event types are disjoint except the Other class. The model $f$ will
be first optimized on $\mathcal{D}_{base}$, and then further fine-tuned on
$D_{novel}$. The validation data set contains the mentions of both base and
novel event types, while the evaluation data set only includes mentions of
novel event types. The goal is to evaluate the generalizability and
transferability of the model from base event types to new event types with few
annotations.

#### Zero-shot Event Detection

The only difference between zero-shot and few-shot event detection lies in the
training data sets. In zero-shot event detection, there is only a large-scale
base training data set
$\mathcal{D}_{base}=\\{(\mathbf{x}_{i},\mathbf{y}_{i})\\}_{i=1}^{M}$ with
sufficient annotations for the base event types. The model $f$ will be only
optimized on base event types and evaluated on the novel types, which is to
measure the transferability of the model under a more challenging setting.

###  3.2 Event Type Prompts

We compare the following five forms of prompts to represent the event types:

#### Event Type Name

The most straightforward and intuitive representation of an event type is the
type name, which usually consists of one to three tokens. As the most basic
and discriminative representations of event types, we include them in all the
following text-based event type prompts.

#### Definition

The type name sometimes cannot accurately represent the semantics of an event
type due to the ambiguity of the type name as well as the variety of the event
mentions. For example, execute can either refer to putting a legal punishment
into action or performing a skillful action or movement. The definitions
instead formally describe the meaning of the event types. Taking the event
type Attack from ACE as an example, its definition is violent or physical act
causing harm or damage

#### Prototype Seed Triggers

Seed trigger based representation consists of the type name and a list of
prototype triggers. Given an event type $t$ and its annotated triggers,
following Wang et al. (2021a), we select the top-$K$111In our experiments, we
set $K=4$. ranked words as the prototype triggers based on the probability
$f_{t}/f_{o}$ of each word, where $f_{o}$ is the frequency of the word from
the whole training dataset and $f_{t}$ is the frequency of the word being
tagged as an event trigger of type $t$. Thus, for the event type Attack, we
represent it as attack invaded airstrikes overthrew ambushed.

#### Event Type Structure

Each event is associated with several arguments, indicating the core
participants. Our preliminary experiment shows that for certain event types,
the arguments can help determine the existence of its corresponding events.
For example, given a sentence, if no person presents in the context, there
should be no Meet events. Given that, we define an event type structure, which
consists of the event type name and argument roles, to represent the event
type, e.g., attack attacker victim target instrument place for Attack.

#### Continuous Soft Prompt

Inspired by the recent success of prompt tuning methods in various NLP
applications, we also adopt a continuous soft prompt, i.e., a free vector of
parameter, to represent each event type. More details regarding the learning
of soft prompts are described in Section 4.

#### APEX Prompt

We assume a better representation of an event type should cover the important
information of all the above prompts. Thus we define a more comprehensive
description (named APEX prompt) for each event type by concatenating its event
type name, seed triggers, and definition which covers all the argument roles.
For example, The APEX prompt for Attack event type is attack, invaded
airstrikes overthrew ambushed, an attacker physically attacks a target with an
instrument at a place.

In our experiments, the event type names and event type structures are
automatically extracted from the target event ontology, such as ACE Linguistic
Data Consortium (2005), ERE Song et al. (2015) and MAVEN Wang et al. (2020a).
The prototype seed triggers for each event type are automatically selected
from its annotated data. The definitions and APEX prompts are based on the
official annotation guides for each target event ontology Linguistic Data
Consortium (2005); Song et al. (2015); Wang et al. (2020a) and the available
definitions in FrameNet Baker et al. (1998) with manual editing.

##  4 A Unified Framework for Event Detection

Figure 1 shows the overview of our unified framework, which leverages event
type specific prompts to detect events under supervised, few-shot and zero-
shot settings. Next, we will describe the details of this framework.

#### Context Encoding

Given an input sentence $W=\\{w_{1},w_{2},\dots,w_{N}\\}$, we take each event
type prompt $T=\\{\tau_{1}^{t},\tau_{2}^{t},\dots,\tau_{K}^{t}\\}$ as a query
to extract the corresponding event triggers. Specifically, we first
concatenate them into a sequence as follows:

$\displaystyle\text{[CLS]}\;\tau_{1}^{t}\;...\;\tau_{K}^{t}\;\text{[SEP]}\;w_{1}\;...\;w_{N}\;\text{[SEP]}$

where [SEP] is a separator from the BERT encoder Devlin et al. (2019). We use
a pre-trained BERT encoder to encode the whole sequence and get contextual
representations for the input sentence
$\boldsymbol{W}=\\{\boldsymbol{w}_{0},\boldsymbol{w}_{2},...,\boldsymbol{w}_{N}\\}$
as well as the event type prompt
$\boldsymbol{T}=\\{\boldsymbol{\tau}_{0}^{t},\boldsymbol{\tau}_{1}^{t},...,\boldsymbol{\tau}_{K}^{t}\\}$.222We
use bold symbols to denote vectors.

#### Event Type Aware Contextual Representation

Given a prompt of each event type, we aim to extract corresponding event
triggers from the input sentence automatically. To achieve this goal, we need
to capture the semantic correlation of each input token to the event type.
Thus we apply attention mechanism to learn a weight distribution over the
sequence of contextual representations of the event type query for each token:

$\displaystyle\boldsymbol{A}_{i}^{T}=\sum_{j=1}^{|T|}\alpha_{ij}\cdot\boldsymbol{T}_{j},\,\,\textrm{where}\,\,\alpha_{ij}=\cos(\boldsymbol{w}_{i},\
\boldsymbol{T}_{j}),$

where $\boldsymbol{T}_{j}$ is the contextual representation of the $j$-th
token in the sequence
$T=\\{t,\tau_{1}^{t},\tau_{2}^{t},\dots,\tau_{K}^{t}\\}$. $\cos(\cdot)$ is the
cosine similarity function between two vectors. $\boldsymbol{A}_{i}^{T}$
denotes the event type $t$ aware contextual representation of token $w_{i}$.

#### Event Detection

With the aforementioned event type prompt attention, each token $w_{i}$ from
the input sentence will obtain a enriched contextual representations
$\boldsymbol{A}_{i}^{T}$. We concatenate them with the original contextual
representation $\boldsymbol{w}_{i}$ from the encoder, and classify it into a
binary label, indicating it as a candidate trigger of event type $t$ or not:

$\displaystyle\boldsymbol{\tilde{y}}_{i}^{t}=\boldsymbol{U}_{o}([\boldsymbol{w}_{i};\
\boldsymbol{A}_{i}^{T};\boldsymbol{P}_{i}])\;,$

where $[;]$ denotes concatenation operation, $\boldsymbol{U}_{o}$ is a
learnable parameter matrix for event trigger detection, and
$\boldsymbol{P}_{i}$ is the one-hot part-of-speech (POS) encoding of word
$w_{i}$.

For continuous soft prompt based event detection, we follow Li and Liang
(2021) where a prefix index $q$ is prepended to the input sequence
$W^{\prime}=[q;\;W]$. The prefix embedding is learned by
$\boldsymbol{q}=\text{MLP}_{\theta}(\boldsymbol{Q}_{\theta}[q])$, where
$\boldsymbol{Q}_{\theta}\in\mathbb{R}^{|\mathcal{Q}|\times k}$ denotes the
embedding lookup table for the vocabulary of prefix indices. Both
$\text{MLP}_{\theta}$ and $\boldsymbol{Q}_{\theta}$ are trainable parameters.
After obtaining the prefix embedding $\boldsymbol{q}$, we concatenate it with
the initialized token embeddings of the input sentence and feed them to BERT
encoder. For each token $w_{i}$, we obtain its contextual representation
$\boldsymbol{w}_{i}$, concatenate it with its POS tag encoding
$\boldsymbol{P}_{i}$, and then classify the token into a binary label.

Dataset | ACE05-E+ | ERE-EN | MAVEN | Notes
---|---|---|---|---
# Types | Base | 18 | 25 | 120 |  -
Novel | 10 | 10 | 45 |  -
# Mentions | Base | 3572 | 5449 | 93675 |  -
Novel | 1724 | 3183 | 3201 |  -
Train | Few-shot | 3216 | 3886 | 88085 |  Include mentions of base types and a small set of mentions for novel types
Zero-shot | 3116 | 3786 | 87635 |  Include mentions of base types
Validation | 900 | 2797 | 3883 |  Mentions of base and novel types
( 51%/49% ) | ( 53%/47% ) | ( 71%/23% ) |  Indicate the base/novel mention ratio
Evaluation | 1195 | 2012 | 1652 |  Include mentions of novel types
Table 2: Data statistics for ACE2005, ERE and MAVEN datasets under the few-
shot and zero-shot event detection settings.

#### Learning Strategy

The learning strategy varies for supervised learning, few-shot learning and
zero-shot learning. For supervised learning, we optimize the following
objective for event trigger detection

$\displaystyle\mathcal{L}=-\frac{1}{|\mathcal{T}||\mathcal{N}|}\sum_{t\in\mathcal{T}}\sum_{i=1}^{|\mathcal{N}|}\boldsymbol{y}_{i}^{t}\cdot\log\boldsymbol{\tilde{y}}_{i}^{t}\;,$

where $\mathcal{T}$ is the set of target event types and $\mathcal{N}$ is the
set of tokens from the training dataset. $\boldsymbol{y}_{i}^{t}$ denotes the
groundtruth label vector.

For few-shot event detection, we optimize the model on both base training data
set and the smaller training data set for novel event types:

$\displaystyle\mathcal{L}=$
$\displaystyle-\frac{1}{|\mathcal{T}^{B}||\mathcal{N}^{B}|}\sum_{t\in\mathcal{T}^{B}}\sum_{i=1}^{|\mathcal{N}^{B}|}\boldsymbol{y}_{i}^{t}\cdot\log\boldsymbol{\tilde{y}}_{i}^{t}$
$\displaystyle-\alpha\frac{1}{|\mathcal{T}^{N}||\mathcal{N}^{N}|}\sum_{t\in\mathcal{T}^{N}}\sum_{i=1}^{|\mathcal{N}^{N}|}\boldsymbol{y}_{i}^{t}\cdot\log\boldsymbol{\tilde{y}}_{i}^{t}$

where $\mathcal{T}^{B}$ and $\mathcal{N}^{B}$ denote the set of base event
types and tokens from the base training data set, respectively.
$\mathcal{T}^{N}$ is the set of novel event types. $\mathcal{N}^{N}$ is the
set of tokens from the training data set for novel event types. $\alpha$ is a
hyper-parameter to balance the two objectives.

For zero-shot event detection, as we only have the base training data set, we
minimize the following objective:

$\displaystyle\mathcal{L}=-\frac{1}{|\mathcal{T}^{B}||\mathcal{N}^{B}|}\sum_{t\in\mathcal{T}^{B}}\sum_{i=1}^{|\mathcal{N}^{B}|}\boldsymbol{y}_{i}^{t}\cdot\log\boldsymbol{\tilde{y}}_{i}^{t}\;.$

Method | Supervised ED | Few-shot ED | Zero-shot ED
---|---|---|---
State of the art | 73.3 | 35.2∗ | $49.1^{*}$
Nguyen et al. (2021) | Lai et al. (2020b) | Zhang et al. (2021b)
(a) Event Type name | 72.2 | 52.7 | 49.8
(b) Definition | 73.1 | 46.7 | 45.5
(c) Seed Triggers | 73.7 | 53.8 | 52.4
(d) Event Type Structure | 72.8 | 50.4 | 48.0
(e) Continuous Soft Prompt | 68.1 | 48.2 | -
Majority Voting of (a)-(e) | 73.9 | 52.1 | 48.7
(f) APEX Prompt | 74.9 | 57.4 | 55.3
Table 3: Results of event detection (ED) on ACE05 (F1-score, %) ∗ indicates evaluation on our data set split.  Method | Supervised ED | Few-shot ED | Zero-shot ED
---|---|---|---
State of the art | 59.4 | 33.0∗ | $41.2^{*}$
Lu et al. (2021) | Lai et al. (2020b) | Zhang et al. (2021b)
(a) Event Type Name | 58.2 | 44.8 | 40.5
(b) Definition | 57.9 | 44.2 | 40.4
(c) Seed Triggers | 60.4 | 50.4 | 49.8
(d) Event Type Structure | 59.1 | 48.5 | 48.7
(e) Continuous Soft Prompt | 55.6 | 41.7 | -
Majority Voting of (a)-(e) | 60.2 | 47.9 | 48.3
(f) APEX Prompt | 63.4 | 52.6 | 49.9
Table 4: Results of event detection (ED) on ERE (F1-score, %). ∗ indicates evaluation on our data set split. Method | Supervised | Few-shot | Zero-shot
---|---|---|---
State of the art | 68.5 | 57.0 | 40.2*
Wang et al. (2021b) | Chen et al. (2021) | Zhang et al. (2021b)
(a) Event type name | 68.8 | 63.4 | 58.8
(b) Definition | 67.1 | 56.9 | 52.9
(c) Seed Triggers | 68.7 | 65.1 | 62.2
(e) Continuous Soft Prompt | 64.5 | 38.6 | -
Majority Voting of (a)-(e) | 68.4 | 63.4 | 58.6
(f) APEX Prompt | 68.8 | 68.4 | 64.5
Table 5: Results of event detection (ED) on MAVEN (F1-score, %). ∗ indicates
evaluation on our data set split.

##  5 Experiment Setup

###  5.1 Datasets

We perform experiments on three public benchmark datasets, include ACE05-E+
(Automatic Content Extraction)333https://catalog.ldc.upenn.edu/LDC2006T06, ERE
(Entity Relation Event) Song et al. (2015)444Following Lin et al. (2020), we
merge LDC2015E29, LDC2015E68, and LDC2015E78 as the ERE dataset., and
MAVENWang et al. (2020a). On each dataset, we conduct experiments under three
settings: supervised event detection, few-shot and zero-shot event detection.

For supervised event detection, we use the same data split as the previous
studies Li et al. (2013); Wadden et al. (2019); Lin et al. (2020); Du and
Cardie (2020); Lin et al. (2020); Nguyen et al. (2021); Wang et al. (2020a) on
all the three benchmark datasets.

For few-shot and zero-shot event detection on MAVEN, we follow the previous
study Chen et al. (2021) and choose 120 event types with the most frequent
mentions as the base event types and the rest 45 event types as novel ones.
For few-shot and zero-shot event detection on ACE and ERE, previous studies
Lai et al. (2020b, a); Chen et al. (2021) follow different data splits and
settings, making it hard for fair comparison. Considering the research goals
of few-shot and zero-shot event detection, we define the following conditions
to split the ACE and ERE datasets:

  * •

The base event types and novel event types should be disjoint except Other.

  * •

Each base or novel event type should contain at least 15 instances.

  * •

The training set should contain sufficient annotated event mentions.

To meet the above conditions, for ACE, we define the event types of 5 main
event categories: Business, Contact, Conflict, Justice and Movement as the
base event types, and types of the remaining 3 main categories: Life,
Personnel and Transaction as the novel event types. In total, there are 18
qualified base types and 10 qualified novel types (the others do not satisfy
the second condition). For ERE, we use the exact same 10 novel event types as
ACE, and the rest 25 types as base event types.

After defining the base and novel event types, we further create the training,
validation and evaluation splits for all three datasets. For few-shot event
detection, we use the sentences with only base event type mentions as the base
training data set, and randomly select 10 sentences with novel event type
mentions as the additional smaller training data set. We use the sentences
with both base and novel event type mentions as the development set, and use
the remaining sentences with only novel event type mentions as the evaluation
dataset. For zero-shot event detection, we use the same development and
evaluation set as few-shot event detection, and remove the instances with
novel event mentions from the training set. For both zero-shot and few-shot
event detection, we randomly split the sentences without any event annotations
proportionally to the number of sentences with event mentions in each set.
Table 2 shows the detailed data statistics for all the three datasets under
the few-shot and zero-shot event extraction settings.

###  5.2 Hyperparameters and Evaluation

For a fair comparison with the previous baseline approaches, we use the same
pre-trained bert-large-uncased model for fine-tuning and optimizing our model
with BertAdam. For supervised event detection, we optimize the parameters with
grid search: training epoch 3, learning rate $\in[3e\text{-}6,1e\text{-}4]$,
training batch size $\in\\{8,12,16,24,32\\}$, dropout rate
$\in\\{0.4,0.5,0.6\\}$. The running time is up to 3 hours on one Quadro RTX
8000. For evaluation, we use the same criteria as previous studies Li et al.
(2013); Chen et al. (2015); Nguyen et al. (2016); Lin et al. (2020): an event
mention is correct if its span and event type matches a reference event
mention.

Figure 2: F-score distribution of all novel types based on various event type
prompts under the few-shot event detection setting on ACE (Best view in color)

##  6 Results and Discussion

#### Overall Results

The experimental results for supervised, few-shot and zero-shot event
detection on ACE05, ERE and MAVEN are shown in Table 3-5, from which we see
that (1) the APEX prompt achieves the best performance among all the forms of
prompts under all the settings of the three benchmark datasets. Comparing with
the previous state of the art, the APEX prompt shows up to 4% F-score gain for
supervised event detection (on ERE), 22.2% F-score gain for few-shot event
detection (on ACE), and 24.3% F-score gain for zero-shot event detection (on
MAVEN); (2) All the forms of prompts provide significant improvement for few-
shot and zero-shot event detection, demonstrating the benefit of leveraging
the semantics of event types via various forms of prompts for event detection,
especially when the annotations are limited or not available. (3) Continuous
soft prompt does not provide comparable performance as other forms of event
type representations, which proves the necessity of leveraging event type
specific prior knowledge to the representations; (4) The majority voting does
not show improvement over individual prompts, due to the fact that each
individual prompt captures a particular aspect of the event type semantics.

#### Supervised Event Detection

By carefully investigating the event mentions that are correctly detected by
the APEX prompt while missed by other prompts, we find that the APEX prompt is
more effective in detecting two types of event mentions: homonyms (multiple-
meaning words) and intricate words. General homonyms are usually hard to be
detected as event mentions as they usually have dozens of meanings in
different contexts. For example, consider the following two examples: (i)
Airlines are getting [Transport:Movement] flyers to destinations on time more
often . (ii) If the board cannot vote to give [Transaction:Transfer-Money’]
themselves present money. Here, “get”, and “give” are not detected based on
the event type name or seed triggers but correctly identified by the
definition and APEX prompts. In general, the definition and APEX prompts make
10% and 7% fewer false predictions than seed triggers on general homonyms. For
intricate words, their semantics usually cannot be captured with an individual
prompt. In the following two examples: (i) It is reasonable, however, to
reimburse board members for legitimate expenses (ii) $\cdot\cdot\cdot$ ever
having discussed being compensated by the board in the future
$\cdot\cdot\cdot$, “reimburse” and “compensated” indicate sophisticated
meaning of Transaction:Transfer-Money, which may not be captured by prompts,
such as seed triggers. With the event definition and the argument roles in the
APEX prompt, the highly correlated contexts, such as “board members” and
“legitimate expenses”, can help the model correctly detect reimburse as an
event mention of Transaction:Transfer-Money.

#### Few-shot Event Detection

Figure 2 shows the F-score distribution of all novel types based on various
forms of event type prompts, from which we observe that: (1) The event type
name, seed triggers, and APEX prompt generally perform better than definition
and structure, as they carry more straightforward semantics of event types.
(2) Event type name based prompts show lower performance on Personnel:End-
Position, Personnel:Start-Position and Transaction:Transfer-Money than other
event types, as the semantics of these event type names are less indicative
than other event types. (3) Seed triggers based prompts perform worse than
event type name and APEX prompts on two event types, Life:injure and Life:die,
probably because the prototype seed triggers are not properly selected. (4)
The structure based prompt outperforms the other prompts on Life:Injure as
Life:Injure events require the existence of a person or victim. (5) APEX
prompt shows consistently (almost) best performance on all the event types,
due to the fact that it combines all the information of other prompts. (6) We
also observe that the performance of Life:Be-Born, Life:Die, Life:Marry, and
Personnel:Elect based on various forms of prompts are consistently better than
the other types as the intrinsic semantics of those types the corresponding
event triggers are concentrated.

#### Zero-shot Event Detection

The proposed prompt-based method is more affordable to be generalized
comparing with the prior state-of-the-art approach Zhang et al. (2021b). The
average length of created APEX prompts is less than 20 tokens, thus manually
creating them won’t take much human effort. On the contrary, Zhang et al.
(2021b) requires a large collection of anchor sentences to perform zero-shot
event detection, e.g., 4,556,237 anchor sentences for ACE and ERE. This
process is time consuming and expensive.

#### Remaining Challenges

We have demonstrated that a proper description can provide much better
performance for both supervised and weakly supervised event detection.
However, the event types from most existing ontologies are not properly
defined. For example, in ACE annotation guideline Linguistic Data Consortium
(2005), transfer-money is defined as “giving, receiving, borrowing, or lending
money when it is not in the context of purchasing something”, however, it’s
hard for the model to accurately interpret it, especially the constraints “not
in the context of purchasing something”. In addition, many event types from
MAVEN, e.g., Achieve, Award, and Incident, are not associated with any
definitions. A potential future research direction is to leverage mining-based
approaches or state-of-the-art generators to automatically generate a
comprehensive event type description based on various sources, such as
annotation guidelines, example annotations, and external knowledge bases.

##  7 Conclusion

We investigate a variety of prompts to represent the semantics of event types,
and leverage them with a unified framework for supervised, few-shot and zero-
shot event detection. Experimental results demonstrate that, a well-defined
and comprehensive description of event types can significantly improve the
performance of event detection, especially when the annotations are limited
(few-shot event detection) or even not available (zero-shot event detection),
with up to 24.3% F-score gain over the prior state of the art. In the future,
we will explore mining-based or generation-based approaches to automatically
generate a comprehensive description of each event type from available
resources and external knowledge base.

## References

  * Agarwal et al. (2021) Ashutosh Agarwal, Anay Majee, Anbumani Subramanian, and Chetan Arora. 2021.  Attention guided cosine margin for overcoming class-imbalance in few-shot road object detection.
  * Ahn (2006) David Ahn. 2006.  The stages of event extraction.  In _Proceedings of the Workshop on Annotating and Reasoning about Time and Events_ , pages 1–8.
  * Baker et al. (1998) Collin F Baker, Charles J Fillmore, and John B Lowe. 1998.  The berkeley framenet project.  In _36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1_ , pages 86–90.
  * Banarescu et al. (2013) Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013\.  Abstract Meaning Representation for sembanking.  In _Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse_ , pages 178–186, Sofia, Bulgaria. Association for Computational Linguistics.
  * Bronstein et al. (2015) Ofer Bronstein, Ido Dagan, Qi Li, Heng Ji, and Anette Frank. 2015.  Seed-based event trigger labeling: How far can event descriptions get us?  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 372–376.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Cao et al. (2015) Kai Cao, Xiang Li, Miao Fan, and Ralph Grishman. 2015.  Improving event detection with active learning.  In _Proceedings of the International Conference Recent Advances in Natural Language Processing_ , pages 72–77, Hissar, Bulgaria. INCOMA Ltd. Shoumen, BULGARIA.
  * Chen et al. (2021) Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2021.  Honey or poison? solving the trigger curse in few-shot event detection via causal intervention.
  * Chen et al. (2015) Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015.  Event extraction via dynamic multi-pooling convolutional neural networks.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 167–176.
  * Chen et al. (2020) Yunmo Chen, Tongfei Chen, Seth Ebner, Aaron Steven White, and Benjamin Van Durme. 2020.  Reading the manual: Event extraction as definition comprehension.  In _Proceedings of the Fourth Workshop on Structured Prediction for NLP_ , pages 74–83, Online. Association for Computational Linguistics.
  * Chinchor and Marsh (1998) Nancy Chinchor and Elaine Marsh. 1998.  Muc-7 information extraction task definition.  In _Proceeding of the seventh message understanding conference (MUC-7), Appendices_ , pages 359–367.
  * Chowdhury et al. (2021) Arkabandhu Chowdhury, Mingchao Jiang, and Chris Jermaine. 2021.  Few-shot image classification: Just use a library of pre-trained feature extractors and a simple classifier.  abs/2101.00562.
  * Cong et al. (2021) Xin Cong, Shiyao Cui, Bowen Yu, Tingwen Liu, Yubin Wang, and Bin Wang. 2021.  Few-shot event detection with prototypical amortized conditional random field.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP_.
  * Dalvi et al. (2012) Bhavana Dalvi, William W. Cohen, and Jamie Callan. 2012.  Websets: extracting sets of entities from the web using unsupervised information extraction.  _ArXiv_ , abs/1307.0261.
  * Deng et al. (2020) Shumin Deng, Ningyu Zhang, Jiaojian Kang, Yichi Zhang, Wei Zhang, and Huajun Chen. 2020.  Meta-learning with dynamic-memory-based prototypical network for few-shot event detection.  _Proceedings of the 13th International Conference on Web Search and Data Mining_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Du and Cardie (2020) Xinya Du and Claire Cardie. 2020.  Event extraction by answering (almost) natural questions.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 671–683, Online. Association for Computational Linguistics.
  * Feng et al. (2016) Xiaocheng Feng, Lifu Huang, Duyu Tang, Heng Ji, Bing Qin, and Ting Liu. 2016.  A language-independent neural network for event detection.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 66–71, Berlin, Germany. Association for Computational Linguistics.
  * Grishman (1997) Ralph Grishman. 1997.  Information extraction: Techniques and challenges.  In _International summer school on information extraction_ , pages 10–27. Springer.
  * Hambardzumyan et al. (2021) Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021.  WARP: Word-level Adversarial ReProgramming.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4921–4933, Online. Association for Computational Linguistics.
  * Huang et al. (2016) Lifu Huang, Taylor Cassidy, Xiaocheng Feng, Heng Ji, Clare Voss, Jiawei Han, and Avirup Sil. 2016.  Liberal event extraction and event schema induction.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 258–268.
  * Huang and Ji (2020) Lifu Huang and Heng Ji. 2020.  Semi-supervised new event type induction and event detection.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 718–724.
  * Huang et al. (2018) Lifu Huang, Heng Ji, Kyunghyun Cho, Ido Dagan, Sebastian Riedel, and Clare Voss. 2018.  Zero-shot transfer learning for event extraction.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2160–2170, Melbourne, Australia. Association for Computational Linguistics.
  * Ji and Grishman (2008) Heng Ji and Ralph Grishman. 2008.  Refining event extraction through cross-document inference.  In _Proceedings of ACL-08: Hlt_ , pages 254–262.
  * Jiang et al. (2020) Zhengbao Jiang, Frank F. Xu, J. Araki, and Graham Neubig. 2020.  How can we know what language models know?  _Transactions of the Association for Computational Linguistics_ , 8:423–438.
  * Kang et al. (2019) Bingyi Kang, Zhuang Liu, Xin Wang, Fisher Yu, Jiashi Feng, and Trevor Darrell. 2019\.  Few-shot object detection via feature reweighting.  In _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 8419–8428.
  * Lai et al. (2020a) Viet Dac Lai, Franck Dernoncourt, and Thien Huu Nguyen. 2020a.  Exploiting the matching information in the support set for few shot event classification.  _Pacific-Asia Conference on Knowledge Discovery and Data Mining_ , page 233–245.
  * Lai and Nguyen (2019) Viet Dac Lai and Thien Huu Nguyen. 2019.  Extending event detection to new types with learning from keywords.  _arXiv preprint arXiv:1910.11368_.
  * Lai et al. (2020b) Viet Dac Lai, Thien Huu Nguyen, and Franck Dernoncourt. 2020b.  Extensively matching for few-shot learning event detection.  In _Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events_ , pages 38–45, Online. Association for Computational Linguistics.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.  The power of scale for parameter-efficient prompt tuning.  In _EMNLP_.
  * Li et al. (2021) Bohao Li, Boyu Yang, Chang Liu, Feng Liu, Rongrong Ji, and Qixiang Ye. 2021.  Beyond max-margin: Class margin equilibrium for few-shot object detection.  _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 7359–7368.
  * Li et al. (2020) Fayuan Li, Weihua Peng, Yuguang Chen, Quan Wang, Lu Pan, Yajuan Lyu, and Yong Zhu. 2020.  Event extraction as multi-turn question answering.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 829–838, Online. Association for Computational Linguistics.
  * Li et al. (2013) Qi Li, Heng Ji, and Liang Huang. 2013.  Joint event extraction via structured prediction with global features.  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 73–82, Sofia, Bulgaria. Association for Computational Linguistics.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , abs/2101.00190.
  * Liao and Grishman (2010) Shasha Liao and Ralph Grishman. 2010.  Using document level cross-event inference to improve event extraction.  In _Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics_ , pages 789–797.
  * Lin et al. (2020) Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020.  A joint neural model for information extraction with global features.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7999–8009, Online. Association for Computational Linguistics.
  * Linguistic Data Consortium (2005) Linguistic Data Consortium. 2005.  English annotation guidelines for events.  https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/english-events-guidelines-v5.4.3.pdf.
  * Liu et al. (2020) Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and Xiaojiang Liu. 2020.  Event extraction as machine reading comprehension.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1641–1651, Online. Association for Computational Linguistics.
  * Lu et al. (2021) Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, and Shaoyi Chen. 2021.  Text2Event: Controllable sequence-to-structure generation for end-to-end event extraction.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 2795–2806, Online. Association for Computational Linguistics.
  * Lyu et al. (2021) Qing Lyu, Hongming Zhang, Elior Sulem, and Dan Roth. 2021.  Zero-shot Event Extraction via Transfer Learning: Challenges and Insights.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 322–332, Online. Association for Computational Linguistics.
  * McClosky et al. (2011) David McClosky, Mihai Surdeanu, and Christopher D Manning. 2011.  Event extraction as dependency parsing.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 1626–1635.
  * Nguyen et al. (2021) Minh Van Nguyen, Viet Dac Lai, and Thien Huu Nguyen. 2021.  Cross-task instance representation interactions and label dependencies for joint information extraction with graph convolutional networks.
  * Nguyen et al. (2016) Thien Huu Nguyen, Kyunghyun Cho, and Ralph Grishman. 2016.  Joint event extraction via recurrent neural networks.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 300–309, San Diego, California. Association for Computational Linguistics.
  * Pasupat and Liang (2014) Panupong Pasupat and Percy Liang. 2014.  Zero-shot entity extraction from web pages.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 391–401.
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019.  Language models as knowledge bases? In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2463–2473, Hong Kong, China. Association for Computational Linguistics.
  * Raffel et al. (2020) Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _JMLR_.
  * Schick and Schütze (2021a) Timo Schick and Hinrich Schütze. 2021a.  Few-shot text generation with pattern-exploiting training.
  * Schick and Schütze (2021b) Timo Schick and Hinrich Schütze. 2021b.  It’s not just size that matters: Small language models are also few-shot learners.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics_ , pages 2339–2352.
  * Shen et al. (2021) Shirong Shen, Tongtong Wu, Guilin Qi, Yuan-Fang Li, Gholamreza Haffari, and Sheng Bi. 2021.  Adaptive knowledge-enhanced bayesian meta-learning for few-shot event detection.  In _Findings of the Association for Computational Linguistics_ , page 2417–2429. Association for Computational Linguistics (ACL).  Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing 2021, ACL-IJCNLP 2021 ; Conference date: 01-08-2021 Through 06-08-2021.
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020.  AutoPrompt: Eliciting knowledge from language models with automatically generated prompts.  In _Empirical Methods in Natural Language Processing (EMNLP)_.
  * Song et al. (2015) Zhiyi Song, Ann Bies, Stephanie Strassel, Tom Riese, Justin Mott, Joe Ellis, Jonathan Wright, Seth Kulick, Neville Ryant, and Xiaoyi Ma. 2015.  From light to rich ere: annotation of entities, relations, and events.  In _Proceedings of the the 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation_ , pages 89–98.
  * Sun et al. (2021) Bo Sun, Banghuai Li, Shengcai Cai, Ye Yuan, and Chi Zhang. 2021.  Fsce: Few-shot object detection via contrastive proposal encoding.  _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 7348–7358.
  * Wadden et al. (2019) David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019.  Entity, relation, and event extraction with contextualized span representations.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5784–5789, Hong Kong, China. Association for Computational Linguistics.
  * Wang and Cohen (2009) Richard C Wang and William Cohen. 2009.  Character-level analysis of semi-structured documents for set expansion.  In _Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing_ , pages 1503–1512.
  * Wang et al. (2021a) Sijia Wang, Mo Yu, Shiyu Chang, Lichao Sun, and Lifu Huang. 2021a.  Query and extract: Refining event extraction as type-oriented binary decoding.  _arXiv preprint arXiv:2110.07476_.
  * Wang et al. (2019) Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019.  Heterogeneous graph attention network.  In _The World Wide Web Conference_ , WWW ’19, page 2022–2032, New York, NY, USA. Association for Computing Machinery.
  * Wang et al. (2020a) Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, and Jie Zhou. 2020a.  MAVEN: A massive general domain event detection dataset.  In _Proceedings of EMNLP 2020_.
  * Wang et al. (2020b) Xin Wang, Thomas E. Huang, Trevor Darrell, Joseph E Gonzalez, and Fisher Yu. 2020b.  Frustratingly simple few-shot object detection.
  * Wang et al. (2021b) Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, and Jie Zhou. 2021b.  CLEVE: Contrastive Pre-training for Event Extraction.  In _Proceedings of ACL-IJCNLP_ , pages 6283–6297, Online. Association for Computational Linguistics.
  * Xiao and Marlet (2020) Yang Xiao and Renaud Marlet. 2020.  Few-shot object detection and viewpoint estimation for objects in the wild.  In _ECCV_.
  * Yan et al. (2019) Xiaopeng Yan, Ziliang Chen, Anni Xu, Xiaoxi Wang, Xiaodan Liang, and Liang Lin. 2019\.  Meta r-cnn: Towards general solver for instance-level low-shot learning.  _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 9576–9585.
  * Yang and Mitchell (2016) Bishan Yang and Tom M. Mitchell. 2016.  Joint extraction of events and entities within a document context.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 289–299, San Diego, California. Association for Computational Linguistics.
  * Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021.  BARTScore: Evaluating generated text as text generation.  In _Advances in Neural Information Processing Systems_.
  * Zhang et al. (2021a) Gongjie Zhang, Kaiwen Cui, Rongliang Wu, Shijian Lu, and Yonghong Tian. 2021a.  Pnpdet: Efficient few-shot detection without forgetting via plug-and-play sub-networks.  _2021 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , pages 3822–3831.
  * Zhang et al. (2021b) Hongming Zhang, Haoyu Wang, and Dan Roth. 2021b.  Zero-shot Label-aware Event Trigger and Argument Classification.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 1331–1340, Online. Association for Computational Linguistics.
  * Zhang et al. (2017) Tongtao Zhang, Spencer Whitehead, Hanwang Zhang, Hongzhi Li, Joseph Ellis, Lifu Huang, Wei Liu, Heng Ji, and Shih-Fu Chang. 2017.  Improving event extraction via multimodal integration.  In _Proceedings of the 25th ACM international conference on Multimedia_ , pages 270–278.

##  Appendix A APEX prompt examples for ACE

Event Rep Type |  Comprehensive Prompt
---|---
Business:Declare-Bankruptcy |  Declare Bankruptcy [SEP] bankruptcy bankruptcies bankrupting [SEP] Organization request legal protection from debt collection at a Place
Business:End-Org |  End Organization [SEP] dissolving disbanded [SEP] an Organization goes out of business at a Place
Business:Merge-Org |  Merge Organization [SEP] merging merger [SEP] two or more Organizations come together to form a new organization at a Place
Business:Start-Org |  Start Organization [SEP] founded [SEP] an Agent create a new Organization at a Place
Conflict:Attack |  Attack [SEP] invaded airstrikes overthrew ambushed [SEP] An Attacker physically attacks a Target with Instrument at a Place
Conflict:Demonstrate |  Demonstrate [SEP] demonstrations protest strikes riots [SEP] Entities come together in a Place to protest or demand official action
Contact:Meet |  Meet [SEP] reunited retreats [SEP] two or more Entities come together at same Place and interact in person
Contact:Phone-Write |  Phone Write [SEP] emailed letter [SEP] phone or written communication between two or more Entities
Justice:Acquit |  Acquit [SEP] acquitted [SEP] a trial of Defendant ends but Adjudicator fails to produce a conviction at a Place
Justice:Appeal |  Appeal [SEP] appeal [SEP] the decision for Defendant of a court is taken to a higher court for Adjudicator review with Prosecutor
Justice:Arrest-Jail |  Arrest Jail [SEP] arrested locked [SEP] the Agent takes custody of a Person at a Place
Justice:Charge-Indict |  Charge Indict [SEP] indictment [SEP] a Defendant is accused of a crime by a Prosecutor for Adjudicator
Justice:Convict |  Convict [SEP] pled guilty convicting [SEP] an Defendant found guilty of a crime by Adjudicator at a Place
Justice:Execute |  Execute [SEP] death [SEP] the life of a Person is taken by an Agent at a Place
Justice:Extradite |  Extradite [SEP] extradition [SEP] a Person is sent by an Agent from Origin to Destination
Justice:Fine |  Fine [SEP] payouts financial punishment [SEP] a Adjudicator issues a financial punishment Money to an Entity at a Place
Justice:Pardon |  Pardon [SEP] pardoned lift sentence [SEP] an Adjudicator lifts a sentence of Defendant at a Place
Justice:Release-Parole |  Release Parole [SEP] parole [SEP] an Entity ends its custody of a Person at a Place
Justice:Sentence |  Sentence [SEP] sentenced punishment [SEP] the punishment for the defendant is issued by a state actor
Justice:Sue |  Sue [SEP] lawsuits [SEP] Plaintiff initiate a court proceeding to determine the liability of a Defendant judge by Adjudicator at a Place
Justice:Trial-Hearing |  Trial Hearing [SEP] trial hearings [SEP] a court proceeding initiated to determine the guilty or innocence of a Person with Prosecutor and Adjudicator at a Place
Life:Be-Born |  Be Born [SEP] childbirth [SEP] a Person is born at a Place
Life:Die |  Die [SEP] deceased extermination [SEP] life of a Victim ends by an Agent with Instrument at a Place
Table 6: APEX templates for ACE event types Event Rep Type |  Comprehensive Prompt
---|---
Life:Divorce |  Divorce [SEP] people divorce [SEP] two Person are officially divorced at a place
Life:Injure |  Injure [SEP] hospitalised paralyzed dismember [SEP] a Victim experiences physical harm from Agent with Instrument at a Place
Life:Marry |  Marry [SEP] married marriage marry [SEP] two Person are married at a Place
Movement:Transport |  Transport [SEP] arrival travels penetrated expelled [SEP] an Agent moves an Artifact from Origin to Destination with Vehicle at Price
Personnel:Elect |  Elect [SEP] reelected elected election [SEP] a candidate Person wins an election by voting Entity at a Place
Personnel:End-Position |  End Position [SEP] resigning retired resigned [SEP] a Person stops working for an Entity or change office at a Place
Personnel:Nominate |  Nominate [SEP] nominate [SEP] a Person is nominated for a new position by another Agent at a Place
Personnel:Start-Position |  Start Position [SEP] hiring rehired recruited [SEP] a Person begins working for an Entity or change office at a Place
Transaction:Transfer-Money |  Transfer Money [SEP] donations reimbursing deductions [SEP] transfer Money from the Giver to the Beneficiary or Recipient at a Place
Transaction:Transfer-Ownership |  Transfer Ownership [SEP] purchased buy sell loan [SEP] buying selling loaning borrowing giving receiving of Artifacts from Seller to Buyer or Beneficiary at a Place at Price
Table 7: APEX templates for ACE event types (continued)