# Using JWST transits and occultations to determine $\sim 1\%$ stellar radii
and temperatures of low-mass stars

Alexandra S. Mahajan Center for Astrophysics | Harvard & Smithsonian, 60 Garden St, Cambridge, MA 02138, USA  Jason D. Eastman Center for Astrophysics | Harvard & Smithsonian, 60 Garden St, Cambridge, MA 02138, USA  James Kirk Department of Physics, Imperial College London, Prince Consort Road, London, SW7 2AZ, UK Alexandra Mahajan<EMAIL_ADDRESS>
###### Abstract

Using JWST observations of a primary transit and two secondary eclipses for GJ
1214b, we determine an eccentricity that is more precise than a decade of
HARPS data, which enables us to measure the stellar density to $2.62$%.
Coupled with a prior on the stellar mass from a dynamically calibrated $K_{\rm
S}$-$M_{*}$ relation, we determine $R_{*}$ to $1.13$% – 3 times more precise
than any other published analysis of this system. Then, using the bolometric
flux from an SED model, we determine $T_{\rm eff}$ to $1.39$% – 40% more
precise than systematic floors from spectroscopy. Within the global model,
these also improve the planetary radius and insolation. This is a proof of
concept for a new method to determine accurate $R_{*}$ and $T_{\rm eff}$ to a
precision currently achieved for only a small number of low-mass stars. By
applying our method to all high signal-to-noise planetary transits and
occultations, we can expand the sample of precisely measured stars without
assuming tidal circularization and calibrate new relations to improve our
understanding of all low-mass stars.

stars: low-mass, stars: eclipses, stars: occultations, techniques:
photometric, instrumentation: analysis, instrumentation: spectroscopic

##  1 Introduction

The masses, radii, and temperatures of planets depend directly on the same
quantities as those of their host stars. Often, the limiting factor for
determining precise planetary parameters is our knowledge of their host stars.
Therefore, the higher the precision on the stellar parameters we can achieve,
the better we will be able to understand climates, habitability, formation,
and evolution of individual planets, as well as test and refine theoretical
models for stellar atmospheres and evolution. This is the case for all planet
hosting stars, but is particularly important for low-mass host stars because
they are the least well understood and they are critical targets in the search
for habitable earth-like planets.

The limitations of our understanding of stars is highlighted by Tayar et al.
(2022), who showed that for spectroscopically measured temperatures,
theoretical Spectral Energy Distributions (SEDs), and evolutionary model
constraints of stars, typical systematic errors are around 4.2% in the stellar
radius $R_{*}$, 2.0% in stellar temperature $T_{\rm eff}$, and 2.4% in
bolometric flux $F_{\rm bol}$. It is widely accepted that the discrepancy
between theory and observation gets larger for low-mass stars, where
theoretically predicted stellar radii are only reliable to $\sim 10\%$ (e.g.
Stauffer et al. 2007; Bell et al. 2012; Torres 2013; Thompson et al. 2014).

While empirical relations improve these constraints, Casagrande et al. (2014)
noted significant discrepancies between measurements of $R_{*}$ from presumed
gold-standard Long Baseline Optical Interferometry (LBOI) using CHARA, and
their own determination using the Infrared Flux Method (IRFM). Despite the
LBOI method’s claimed precision of $\sim 1\%$ and the IRFM’s precision of
$\sim 3\%$ (limited by the $\sim 2\%$ systematic errors in the spectroscopic
determination of $T_{\rm eff}$) the largest discrepancies were seen in the
stars with the smallest angular size (regardless of physical size). This
implies the gold-standard LBOI measurements were to blame. Tayar et al. (2022)
confirmed this behavior by highlighting discrepancies in different groups’
LBOI measurements of the same systems using CHARA. More modern LBOI
measurements (using additional calibration stars) may be more reliable, but
still achieve a $\sim$1% precision and only 29 stars are big, bright, and
close enough to measure (Mann et al. 2015). Another option for $\sim 1\%$
stellar radii is finding detached, double lined eclipsing binaries, but these
are also relatively rare, with only a handful $M_{*}<0.7\,M_{\odot}$ known
Torres et al. (2010).

Similar to IRFM, Mann et al. (2015) determined a semi-empirical relationship
between the absolute $K_{\rm S}$ band magnitude and the masses and radii of
low-mass stars, where the masses and radii of the calibration sample were
determined through LBOI measurements and spectroscopically-determined
temperatures and metallicities, with a surprisingly tight scatter: 1.8% in
mass and 2.7% in radius.

However, the discrepancy between the $M_{*}$ derived from the semi-empirical
Mann et al. (2015) relations and the $M_{*}$ later determined from the
empirical $K_{\rm S}$-$M_{*}$ Mann et al. (2019) relation calibrated
dynamically, exceeds the scatter of the relations in some cases (e.g., GJ
1214) by a factor of 5.

This implies strong systematic errors in the spectroscopic temperatures and
metallicities used to calibrate the semi-empirical models. That, and the fact
that they also used the controversial LBOI radii to calibrate their relation,
suggests the 2.7% scatter about the Mann et al. (2015) $K_{\rm S}$-$R_{*}$
relation may not be a reliable estimate of its underlying uncertainty. That
is, for most low-mass stars, the community optimistically achieves a radius
precision of $\sim 3\%$, limited by the $\sim 2\%$ systematic uncertainty in
the spectroscopically determined $T_{\rm eff}$, and a mass precision of 2.3%,
limited by the scatter in the empirical $K_{\rm S}$-$M_{*}$ relation (Mann et
al. 2019).

Meanwhile, Seager & Mallén-Ornelas (2003) showed that Kepler’s law can be re-
arranged in terms of the transit light curve’s observables to directly measure
the stellar density,

$\,\rho_{*}=\frac{3\pi}{GP^{2}}\left(\frac{a}{R_{*}}\right)^{3},$ (1)

where $G$ is the gravitational constant, $P$ is the planetary period
constrained by the frequency of transits and $a/R_{*}$, the planet’s semi-
major axis $a$ divided by $R_{*}$, is constrained by the duration of the
transit when the orbit is circular. Unfortunately, eccentricity $e$ and the
argument of periastron $\omega_{*}$111Here we we mean the argument of
periastron of the stellar orbit in a left-handed coordinate system, as is the
century-old and commonly misunderstood convention. We have explicitly
confirmed this definition when referencing equations stated ambiguously from
other work. change the planet’s velocity during transit, making the transit
duration more closely related to

$\frac{a}{R_{*}}\frac{\sqrt{1-e^{2}}}{1+e\sin{\omega_{*}}}.$ (2)

That is, $\omega_{*}$, $e$, and $\,\rho_{*}$ all change the observed transit
duration (one observable and three unknowns). Typically, $e$ and $\omega_{*}$
are constrained from radial velocities, but to a precision that dominates the
uncertainty in computing $\,\rho_{*}$ from the light curve. However, with the
duration of the primary transit, the duration of the secondary eclipse, and
the phase offset between the two (three observables), we can precisely
determine all three unknowns (Kopal 1946; Charbonneau et al. 2005; Ford et al.
2008; Huber et al. 2017; Alonso 2018; Eastman et al. 2023) – leading to an
extremely precise determination of $e$ (and $\omega_{*}$ when $e\neq 0$).

In particular, Alonso (2018) summarized the constraint on $e$ and $\omega_{*}$
from the transit and occultation as approximately

$e\cos{\omega_{*}}=\frac{\pi}{P}\frac{\left(T_{\rm S}-T_{\rm
C}-P/2\right)}{1+\csc^{2}i},$ (3)

and

$e\sin{\omega_{*}}=\frac{\beta T_{\rm FWHM,S}-T_{\rm FWHM}}{\beta T_{\rm
FWHM,S}+T_{\rm FWHM}},$ (4)

where $i$ is the inclination of the orbit, $T_{\rm S}$ is secondary eclipse
time, $T_{\rm C}$ is the primary transit time, and $\beta$ is the $\delta$
from Alonso (2018), but renamed to remove confusion with the transit depth,
equal to

$\beta=\sqrt{\frac{1-\left(\frac{R_{*}}{\,R_{\rm
P}+R_{*}}\right)^{2}\cos^{2}i}{1-\left(\frac{\,R_{\rm P}}{\,R_{\rm
P}+R_{*}}\right)^{2}\cos^{2}i}}.$ (5)

Note that the epochs of $T_{\rm S}$ and $T_{\rm C}$ must minimize the
numerator in equation 3, and should be corrected for the observed delay due to
the light travel time between the primary and secondary eclipses.

While these equations are approximate, detailed models like the projected
Keplerian orbits used by EXOFASTv2 determine accurate values for $e$ and
$\omega_{*}$ numerically. Then, we can determine $a/R_{*}$ from equation 2,
and finally $\,\rho_{*}$ from equation 1. Eastman et al. (2023) then showed,
from the definition of $\,\rho_{*}\equiv 3M_{*}/4\pi R_{*}^{3}$ how its error
propagates to other quantities of interest. In particular, they determined the
fractional error in $M_{*}$ is reduced by a factor of 3 as it propagates
through $\,\rho_{*}$ to $R_{*}$.

$\left(\frac{\sigma_{R_{*}}}{R_{*}}\right)^{2}=\left(\frac{\sigma_{\,\rho_{*}}}{3\,\rho_{*}}\right)^{2}+\left(\frac{\sigma_{M_{*}}}{3M_{*}}\right)^{2}$
(6)

So by determining the mass to 2.3% (the scatter in the dynamically calibrated
$K_{\rm S}$-$M_{*}$ relation) and the stellar density to 2.7% (a typical
constraint achieved with JWST observations of transits and occultations), we
can derive the stellar radius to 1% just from the definition of $\,\rho_{*}$
and using standard error propagation in equation 6.

That is, relying on well-understood physics without any reliance on stellar
atmospheric or evolutionary models or a spectroscopic measurement of $T_{\rm
eff}$, our precision is $\sim 3\times$ better than IRFM and comparable to the
controversial precision claimed by gold standard LBOI observations, or the
rare detached double-lined eclipsing binaries. Not only do we precisely
determine the properties of specific systems and improve the precision of
fundamental planet parameters, but it can also serve as an independent check
on LBOI observations and expand the sample of precise stellar radii with which
to calibrate future relations similar to Mann et al. (2015).

Finally, because we no longer rely on the spectroscopic $T_{\rm eff}$ to
determine $R_{*}$, Eastman et al. (2023) also showed that we can leverage this
empirically derived radius, a precise distance from Gaia, and the definition
of the bolometric flux $F_{\rm bol}$

$F_{\rm bol}\equiv\,\sigma_{SB}T_{\rm
eff}^{4}\left(\frac{R_{*}}{d}\right)^{2},$ (7)

to determine a precise temperature,

$\left(\frac{\sigma_{T_{\rm eff}}}{T_{\rm
eff}}\right)^{2}=\left(\frac{\sigma_{d}}{2d}\right)^{2}+\left(\frac{\sigma_{F_{\rm
bol}}}{4F_{\rm
bol}}\right)^{2}+\left(\frac{\sigma_{R_{*}}}{2R_{*}}\right)^{2},$ (8)

where $\,\sigma_{SB}$ is the Stefan-Boltzmann constant and $d$ is the distance
to the star.

While this temperature loosely depends on stellar model atmospheres or
bolometric corrections to determine $F_{\rm bol}$, Tayar et al. (2022) showed
that a realistic systematic uncertainty in $F_{\rm bol}$ is $\sim 2.4\%$.
Given that they focused on solar-type stars and M dwarfs are more poorly
known, we assume the floor for our low-mass stars is 4%. Equation 8 shows that
the fractional uncertainty in $F_{\rm bol}$ is reduced by a factor of 4 as it
propagates to $T_{\rm eff}$. Thus, even with a 4.0% systematic floor in
$F_{\rm bol}$ and 1% uncertainty in $R_{*}$, we can determine a 1.25%
uncertainty in $T_{\rm eff}$ – almost twice as precise as the systematic
floors from spectroscopy.

While all these ideas have been done individually before (e.g., Charbonneau et
al. 2005; Huber et al. 2017; Mann et al. 2017; Eastman et al. 2023), combining
them in this way is new and allows many more targets to be measured with a
precision that makes them suitable calibration stars. We present a proof of
concept for this novel method by analyzing a JWST full phase light curve of
the M-dwarf GJ 1214 and its transiting super-Earth GJ 1214 b to determine the
stellar radius and temperature to a mere $\sim$1%.

We discuss the various data used in our analysis, including a JWST full phase
light curve, in §2. Our methods for our models as created by EXOFASTv2 are
discussed in §3. Our results are presented in §4. A discussion of the
importance of our research as a whole as well as future applications is given
in §6.

##  2 Observations

###  2.1 JWST

Kempton et al. (2023) observed a full white light phase curve of GJ 1214 with
JWST, which included two secondary eclipses and a primary transit observed
with JWST’s Mid-Infrared Instrument Low Resolution Spectrometer (MIRI LRS) in
the Slitless prism mode with 6.68 seconds of integration per frame. GJ 1214
was observed continuously from just before the secondary eclipse on 2022-07-20
until just after the next secondary eclipse on 2022-07-22. The JWST data
presented in this article were obtained from the Mikulski Archive for Space
Telescopes (MAST) at the Space Telescope Science Institute. The specific
observations analyzed can be accessed via https://doi.org/10.17909/19mj-5h97
(catalog DOI: 10.17909/19mj-5h97).

While Kempton’s white light phase curve of their observations is available
online222https://zenodo.org/record/7703086#.ZAZk1dLMJhE, we discovered that
its time stamps did not match the desired INT_MID_BJD_TDB keyword from the
data in the archive, resulting in a $\sim$2 minute offset in the observed
transit time. Kempton et al. (2023) were unaware of the INT_MID_BJD_TDB
keyword, and computed their own $\rm{BJD_{TDB}}$ using the Barycentric time
correction BARTDELT keyword. However, due to a known issue in that
keyword333https://jwst-docs.stsci.edu/jwst-calibration-pipeline-caveats/known-
issues-with-jwst-data-products, their times were incorrect. They noticed the
problem late in their analysis and applied a correction to their modeled
transit time, but did not update the publicly available light curves (Kempton,
et. al., priv. comm).

After noticing that an $\sim$8s ($\sim 7\sigma$) difference remained, we
ultimately determined the remaining discrepancy to be a long-standing bug in
EXOFASTv2. The EXOFASTv2 model corrects the observed times to the target’s
barycentric frame ($\rm{TJD_{TDB}}$) to do its internal modeling (e.g., to
account for the light travel time between the primary and secondary eclipse
times to constrain the eccentricity), but failed to correct back to the
“observed” solar system barycenter frame ($\rm{BJD_{TDB}}$), and erroneously
reported the times to be in $\rm{BJD_{TDB}}$. EXOFASTv2 now correctly
identifies which times are $\rm{TJD_{TDB}}$ and which are $\rm{BJD_{TDB}}$,
but all timestamps of all prior models quoted using EXOFASTv2 are actually
$\rm{TJD_{TDB}}$, which would make transit times appear systematically early
(by the light travel time) relative to stated $\rm{BJD_{TDB}}$. The results in
Table LABEL:tab:gj1214 correctly report the transit times.

Prior to receiving the corrected light curve from Kempton et al. (2023), we
extracted our own white light phase curve using the Tiberius pipeline (Kirk et
al. 2017, 2021). We began by processing the uncalibrated images (uncal.fits)
files through stage 1 of the JWST Science Calibration Pipeline (version 1.8.2
https://doi.org/10.5281/zenodo.7314521 (catalog DOI: 10.5281/zenodo.7314521)).
We followed the default steps for processing MIRI time series observations,
apart from the jump step that has been demonstrated to lead to increased noise
in time series data (e.g., Rustamkulov et al. 2023). Instead, we performed our
own outlier detection by comparing all pixels in the time-series to a median-
combined integration. Any pixels in the time-series that deviated by
$>5\sigma$ from the median pixel value were replaced with the median value. We
then proceeded to run our spectral extraction on the outlier cleaned
integrations. We began by tracing the centers of the stellar spectra with a
quartic polynomial and performed standard aperture photometry with an aperture
full width of 8 pixels, equal to the choice of Kempton et al. (2023), after
removing the background at every row on the detector. The background was
measured with a linear polynomial fit to two 6-pixel-wide regions offset by 15
pixels from either side of the extraction aperture. Like Kempton et al.
(2023), we also saw an exponential ramp at the beginning of our light curve.
Following the approach of Kempton et al. (2023), we clipped the first 550
integrations (63 minutes) and used a running median with a sliding box of 216
data points to remove outliers. In our study, we decided to remove $>3\sigma$
outliers by this method. The RMS of the Tiberius-extracted light curve after
removing our best-fit transit model, was 302 ppm – very close the 306 ppm RMS
we achieved for the Kempton-extracted light curve.

###  2.2 Other Transit Photometry

There is a wealth of ground based observations on GJ 1214, beginning with its
discovery (Charbonneau et al. 2009). Despite minor systematics, we opted to
include their Fred Lawrence Whipple Observatory (FLWO) data and their MEarth
data as the 14-year baseline led to an improvement in the precision of the
period by a factor of 10,000, relative to the JWST light curve alone. However,
their overall contribution to the total SNR of the combined light curve was
dwarfed by JWST, thus the minor systematics have a negligible effect on the
determination of $\,\rho_{*}$.

Very Large Telescope (VLT) data from Berta et al. (2011) was nominally very
precise, but showed strong systematic errors by eye. We modeled the VLT data
including quadratic detrending as done in Berta et al. (2011). However, their
inclusion did not measurably impact either the values or precision of any
parameter. Because of that, and out of a concern for the significant increase
in run time for our models, we did not include them in our final analysis.

We also tried including Spitzer light curves from Cloutier et al. (2021), who
re-extracted observations by Fraine et al. (2013) and Gillon et al. (2017).
The Spitzer data contained large systematics that needed to be removed with
heroic effort. Gillon et al. (2014) detrended each light curve with an
optimized set of parameters specific to each day that data was collected
whereas Cloutier et al. (2021) used a more sophisticated approach which relied
on Gaussian processes for detrending. We found that the addition of the
detrended data from Cloutier (Priv. Comm, 2023) did not significantly improve
the precision except for the period, but our global model showed a systematic
shift of $\sim$0.7 sigma in $\,\rho_{*}$, $R_{*}$, and $T_{\rm eff}$. Out of a
concern that the systematic removal process may have been imperfect, we chose
to discard the Spitzer data in our global analysis.

Finally, we checked for TESS data, but according to TESS-point, GJ 1214 was
not observed in any sector to date, nor will it be included in any planned
sector through sector 83 ending on 2024-10-01.

###  2.3 Stellar Photometry

The broadband photometry used for the SED (Spectral Energy Distribution)
models is summarized in Table 1 and shown in Figure 1. The uncertainties were
rounded based on standard procedures where we assume a 0.02-0.03 magnitude
systematic floor in determining the absolute magnitude. We used data from Gaia
DR3 (Data Release 3), 2MASS (Two Micron All Sky Survey), and WISE (Wide Field
Infrared Survey Explorer). Note that GJ 1214 was not automatically cross
matched in TICv8.2 Stassun & Torres (2018), so WISE and GaiaDR3 matching was
done by determining the star with the smallest Ks-WISE1 color within 30”, as
is now automatically codified in the EXOFASTv2 utility MKTICSED. The
corresponding IDs are included in Table 1.

Figure 1: The upper panel shows the SED model for GJ 1214 of brightness versus
wavelength in microns. The black line represent the model’s atmosphere and the
red horizontal bars represent wavelength span of the observations in the star.
The blue points represent the model and the average over the span of the width
of the red horizontal bars. The red vertical bars represent the uncertainty in
the measurement; the bottom panel represents the model’s residuals of observed
minus calculated divided by the uncertainty.  Table 1: Literature and Measured
Properties for GJ 1214

Other identifiers |  |  |
---|---|---|---
GJ 1214 |
LHS 3275, TIC 467929202 |
2MASS J17151894+0457496, Gaia DR2 4393265392167891712 |
Gaia DR3 4393265392168829056, ALLWISE J171519.33+045742.2 |
Parameter | Description | Value | Source
$\alpha_{J2000}$ | Right Ascension (RA) | 17:15:18.934 | 1
$\delta_{J2000}$ | Declination (Dec) | 04:57:50.067 | 1
$l$ | Galactic Longitude |  26.16210287∘ | 1
$b$ | Galactic Latitude |  23.60692189∘ | 1
${\rm G}$ |  Gaia $G$ mag. |  12.997$\pm 0.020$ | 1
GBP |  Gaia BP mag. |  14.912$\pm 0.020$ | 1
GRP |  Gaia RP mag. |  11.683$\pm 0.020$ | 1
J | 2MASS J mag. |  9.75 $\pm$ 0.024 | 2
H | 2MASS H mag. |  9.094 $\pm$ 0.024 | 2
KS |  2MASS ${\rm K_{S}}$ mag. |  8.782 $\pm$ 0.020 | 2
WISE1 |  WISE1 mag. |  8.598 $\pm$ 0.030 | 3
WISE2 |  WISE2 mag. |  8.442 $\pm$ 0.030 | 3
WISE3 |  WISE3 mag. |  8.254 $\pm$ 0.030 | 3
WISE4 |  WISE4 mag. |  8.084 $\pm$ 0.236 | 3
$\mu_{\alpha}$ |  Gaia DR2 proper motion |  580.202 $\pm$ 0.059 | 1
|  in RA (mas yr-1) |  |
$\mu_{\delta}$ |  Gaia DR2 proper motion |  -749.713 $\pm$ 0.047 | 1
|  in DEC (mas yr-1) |  |
$\pi^{\parallel}$ |  Gaia Parallax (mas) |  68.299 $\pm$ 0.065 | 1
$RV$ | Systemic radial | $20.91\pm 0.66$ | 1
|  velocity ( km s-1) |  |
$d$ | Distance (pc) |  14.642$\pm 0.014$ | 1

NOTES: The uncertainties of the photometry have a systematic error floor
applied.
$\ddagger$ RA and Dec are in epoch J2000. The coordinates come from Vizier
where the Gaia RA and Dec have been precessed and corrected to J2000 from
epoch J2015.5.
$\parallel$ Values from Gaia DR3 are raw because they could not be corrected
for the systematic offset as prescribed by Lindegren et al. (2021). There was
no value for nu_eff_used_in_astrometry in DR3 for this star.

References are: 1Gaia Collaboration (2020),2Cutri et al. (2003), 3Cutri et al.
(2012)

###  2.4 Radial Velocities

We used the same 165 spectra from the High Accuracy Radial velocity Planet
Searcher (HARPS; Mayor et al. 2003) spectra from Cloutier et al. (2021), which
span from 2009-07-24 to 2019-09-02 and includes a re-analysis of the discovery
Radial Velocities from Charbonneau et al. (2009), and their own observations.
In 2015-06, the HARPS fiber was upgraded. We modeled the pre-upgraded and
post-upgraded data with separate zero points and jitter terms.

##  3 Methods

We performed a global model of the GJ 1214 system using EXOFASTv2 and the data
described in §2. We used the default options as outlined in Eastman et al.
(2019), except the following.

We opted not to use the default MIST evolutionary model and instead used a
prior on the $k_{S}$ apparent magnitude and the recently integrated Mann et
al. (2019) $K_{\rm S}$-$M_{*}$ relation, as the dynamical relation is much
more reliable for low-mass stars. We fit a spline (Vanderburg & Johnson 2014)
and changed the default knot spacing from 0.75 days to 0.4 days to remove the
phase curve variability of the JWST extracted light curve. The value of the
knot spacing was an appropriate compromise between being much smaller than the
period ($\sim 0.25\times$) in order to take out the phase variations, but much
larger than the transit duration ($\sim 10\times$) so as to not bias the
transit shape.

We fit a secondary eclipse to the JWST data due the fact that it was
noticeably observed in the light curve and provided a strong constraint on the
eccentricity (see equations 3 to 5). We disabled the constraint from the limb
darkening tables (Claret 2017) because the light curve was capable of
constraining this on its own, there is no constraint given by Claret (2017)
for the JWST MIRI LRS Slitless prism, and theoretically predicted limb
darkening coefficients can be especially poor for low-mass stars (Patel &
Espinoza 2022).

We detrended FLWO and MEarth light curves with airmass simultaneously with the
fit, following the procedure in the discovery paper (Charbonneau et al. 2009).

We used a parallax prior from Gaia DR3 (Gaia Collaboration 2020). Note that we
did not correct for the systematic offset as prescribed by Lindegren et al.
(2021) because there was no value for the required effective wavenumber of the
source used in the astrometric solution nu_eff_used_in_astrometry for GJ 1214
in DR3, though we do not expect this to be a significant source of error
(Stassun & Torres 2021). Further, El-Badry et al. (2021) showed that the Gaia
DR3 uncertainties can be underestimated by as much as 30% for bright
($G\lesssim 18$) stars, which we accounted for by inflating the error in the
parallax by 30%. However, because the uncertainty is so small ($0.12$%), the
impact on the rest of the fit is negligible. Our inflated parallax differs by
0.5$\sigma$ from the EDR3 (Early Data Release 3) value used by Cloutier et al.
(2021). We also used a metalicity prior from Cloutier et al. (2021) and an
upper limit on the extinction from Schlafly & Finkbeiner (2011) and Schlegel
et al. (1998). These priors are summarized in Table 2.

Measuring bolometric fluxes is difficult. This is especially true for low-mass
stars which have poorly understood atmospheres. Our method of computing
bolometric flux relies on the SED model, which requires that we understand the
filter profiles and zero-points for each of the modeled bandpasses. We chose
to use the MIST bolometric correction tables, which use the detailed filter
profiles, including the latest updated profiles for Gaia DR3.

However, remaining systematics in filter profiles, zero-points, and
atmospheric models must be accounted for. In order to do this, we used three
independent procedures. First we inflated the per point measurement
uncertainites to a floor of between 0.02 to 0.03 magnitudes to ensure the
proper relative weighting of each magnitude. Next, EXOFASTv2 fits a term
called $\sigma_{\rm SED}$ (as seen in Table LABEL:tab:gj1214) which scales all
SED photometry errors together so that the uncertainties in the data are well
matched to the model ($\chi^{2}_{R}\sim 1$).

Finally, EXOFASTv2 includes a systematic error floor in the SED model to
prevent over constraining $F_{\rm bol}$ and $T_{\rm eff}$ from the SED. It is
implemented by fitting $F_{\rm Bol,SED}$ and $T_{eff,SED}$ – constrained only
by the SED – and the same quantities again, $F_{Bol}$ and $T_{\rm eff}$, but
used everywhere else in the fit. They are then linked to each other with
penalties that account for our presumed systematic error floors,
$\sigma_{T_{\rm eff,sys}}$, and $\sigma_{F_{\rm Bol,sys}}$,

$\ln{\mathscr{L}}+=-0.5\left(\frac{T_{\rm eff,SED}-T_{\rm eff}}{\sigma_{T_{\rm
eff,sys}}}\right)^{2},$ (9)

and

$\ln{\mathscr{L}}+=-0.5\left(\frac{F_{\rm Bol,SED}-F_{\rm Bol}}{\sigma_{F_{\rm
Bol,sys}}}\right)^{2}.$ (10)

By default, EXOFASTv2 assumes $\sigma_{T_{\rm eff,sys}}=2\%$, and
$\sigma_{F_{\rm Bol,sys}}$=2.4%, as recommended by Tayar et al. (2022).
However, we further inflated $\sigma_{F_{\rm Bol,sys}}$ to 4% to account for
larger expected systematics in M-dwarfs. This gives us our final precision in
$F_{\rm bol}$ of $5.24$%.

Note that since the density of the star coupled with the mass from Mann et al.
(2019) is the primary constraint on the radius of the star, the impact of the
SED model on $R_{*}$ is negligible. This means that systematic errors imparted
by the SED will only impact our measurement of $T_{\rm eff}$. We also note
that the final uncertainty in $T_{\rm eff}$ can be better than $\sigma_{T_{\rm
eff,sys}}$ when it is determined from $\sigma_{d}$, $\sigma_{F_{\rm bol}}$,
and $\sigma_{R_{*}}$ through equation 8, as it is here and is typically the
case any time $\sigma_{R_{*}}$ is small.

Table 2: Priors in the EXOFASTv2 Global Fits for GJ 1214

Param. | Description | Prior | Source
---|---|---|---
$\varpi$ | Parallax [mas] |  $\mathcal{N}$(68.299, 0.0845) | 1
$\left[\mathrm{Fe/H}\right]$ | Metallicity [dex] |  $\mathcal{N}$(0.29 0.12) | 2
$k_{s}$ | Apparent $K_{s}$ [mag] |  $\mathcal{N}$(8.782, 0.020) | 3
$A_{V}$ | Reddening [mag] |  $\mathcal{U}$[0, 0.4668507] | 4,5

††footnotetext: Here, $\mathcal{N}$(a,b) is a Gaussian distribution centered
at a with a standard deviation of b, and $\mathcal{U}$[c, d] is a uniform
distribution bounded inclusively between c and d.
References are:1Gaia Collaboration (2020),2Cloutier et al. (2021), 3Cutri et
al. (2003), 4Schlafly & Finkbeiner (2011), 5Schlegel et al. (1998)

##  4 Results

All fitted and derived parameters from our analysis of GJ 1214 are summarized
in Table LABEL:tab:gj1214. Figures 2 and 3 show the primary transit and
secondary eclipse from JWST, respectively. Figure 4 shows all light curves
used for our analysis (i.e., JWST, MEarth, and FLWO). Notably, our analysis
dramatically improves the precision for $R_{*}$ ($3.27$$\times$), $T_{\rm
eff}$ ($2.33$$\times$), and $\,\rho_{*}$ ($4.89$$\times$) compared to the next
best result by Cloutier et al. (2021), as shown in the “Improvement” column in
Table LABEL:tab:gj1214.

The secondary transit timing also constrains the eccentricity of the orbit
$3.32$ times better, despite Cloutier et al. (2021) using over a decade of
HARPS data. We note that the eccentricity and its uncertainty from our global
model ($e=$$0.0062^{+0.0079}_{-0.0044}$) is in excellent agreement with values
numerically propagated the using equations 3 \- 5, and our MCMC chain values
for inputs ($e=0.0058_{-0.0040}^{+0.0073}$).

Finally, the improved stellar parameters propagate to improved accuracy and
precision in the planetary parameters, including the planet radius $\,R_{\rm
P}$ ($1.59$$\times$) and the incident flux on the planet, $\langle\rm
F\rangle$ ($2.80$$\times$), which is critical for understanding exoplanets and
their climates.

We note that the tidal circularization timescale reported by EXOFASTv2 by
default uses $Q=10^{6}$ computed from Adams & Laughlin (2006, eq. 3), but a
more appropriate value for a rocky planet like GJ 1214b is $Q=100$. We have
manually corrected the tidal circularization timescale to $0.0218\pm 0.0014$
Myr in Table LABEL:tab:gj1214, assuming $Q=100$. Given that, we expect the
system to be tidally circularized, consistent with our eccentricity of
$0.0062^{+0.0079}_{-0.0044}$.

All of our values are consistent with Cloutier et al. (2021) within $3\sigma$,
except the depth ($-3.76$$\sigma$), which is expected due to differences in
observed wavelength and stellar variability. We note that the discrepancy is
negligible when propagated to the planetary radius. A few other values differ
by more than $1\sigma$. $T_{\rm eff}$ is $-1.37$$\sigma$ lower than their
spectroscopically determined value. $T_{\rm eff}$-derived quantities ($T_{\rm
eq}$, $\langle\rm F\rangle$) are similarly discrepant. This is a relatively
small difference given that the two methods are nearly completely independent
determinations of the stellar temperature, though we expect ours is more
direct and therefore more accurate. Our inclination and period also differ by
$2.13$$\sigma$ and $1.53$$\sigma$, respectively.

We note that the limb darkening for our MIRI LRS white light curve ($u_{1}=$
$0.062^{+0.029}_{-0.028}$, $u_{2}=$ $0.056\pm 0.053$) is better constrained
here than from theoretical predictions (Patel & Espinoza 2022). Because of the
strong covariance between limb darkening and $a/R_{*}$ (and ultimately
$\,\rho_{*}$), it is important that we have accurate values with uncertainties
accounted for within the global model.

Table 3: Median values and 68% confidence interval for GJ 1214.

Parameter | Description | This Work | Cloutier et al. (2021) | Improvement1
---|---|---|---|---
Stellar Parameters: |  |  |
$M_{*}$ | Mass ($\,M_{\odot}$) | $0.1820^{+0.0042}_{-0.0041}$ | $0.178\pm 0.010$ | $2.41$
$R_{*}$ | Radius ($\,R_{\odot}$) | $0.2162^{+0.0025}_{-0.0024}$ | $0.215\pm 0.008$ | $3.27$
$R_{*,SED}$ | Radius2 ($\,R_{\odot}$) | $0.2207^{+0.0030}_{-0.0033}$ | – | –
$L_{*}$ | Luminosity ($\,L_{\odot}$) | $0.00390^{+0.00021}_{-0.00020}$ | $0.0046^{+0.0007}_{-0.0006}$ | $3.17$
$F_{Bol}$ | Bolometric Flux (cgs) |  $5.82^{+0.31}_{-0.30}$$\times 10^{-10}$ | – | –
$\rho_{*}$ | Density (cgs) | $25.41^{+0.62}_{-0.71}$ | $25.4^{+3.5}_{-3.0}$ | $4.89$
$\log{g}$ | Surface gravity (cgs) | $5.0286^{+0.0078}_{-0.0088}$ | $5.026\pm 0.040$ | $4.82$
$T_{\rm eff}$ | Effective Temperature (K) | $3101\pm 43$ | $3250\pm 100$ | $2.33$
$T_{\rm eff,SED}$ | Effective Temperature2 (K) | $3077^{+39}_{-35}$ | – | –
$[{\rm Fe/H}]$ | Metallicity (dex) | $0.24\pm 0.11$ | $0.29\pm 0.12$ | $1.09$
$K_{s}$ | Absolute Ks (mag) | $7.953\pm 0.020$ | $7.956\pm 0.020$ | $1.00$
$k_{s}$ | Apparent Ks (mag) | $8.781\pm 0.020$ | – | –
$A_{V}$ | V-band extinction (mag) | $0.22^{+0.15}_{-0.14}$ | – | –
$\sigma_{SED}$ | SED photometry error scaling | $2.03^{+0.73}_{-0.47}$ | – | –
$\varpi$ | Parallax (mas)3 | $68.298^{+0.085}_{-0.084}$ | $68.348\pm 0.056$ | $0.66$
$d$ | Distance (pc) | $14.642\pm 0.018$ | $14.631\pm 0.012$ | $0.67$
Planetary Parameters: |  |  |
$P$ | Period (days) | $1.580404531^{+0.000000018}_{-0.000000017}$ | $1.58040433\pm 0.00000013$ | $7.43$
$R_{P}$ | Radius ($\,R_{\rm\earth}$) | $2.733^{+0.033}_{-0.031}$ | $2.733^{+0.050}_{-0.052}$ | $1.59$
$M_{P}$ | Mass ($\,M_{\earth}$) | $8.41^{+0.36}_{-0.35}$ | $8.17\pm 0.43$ | $1.21$
$T_{C}$ | Time of conjunction4 ($\rm{BJD_{TDB}}$) | $2459639.7812619^{+0.0000090}_{-0.0000089}$ | $2459639.78092\pm 0.00033$ | $36.87$
$T_{T}$ | Time of min proj sep5 ($\rm{BJD_{TDB}}$) | $2459636.6204528^{+0.0000090}_{-0.0000089}$ | – | –
$a$ | Semi-major axis (AU) | $0.01505\pm 0.00011$ | $0.01490\pm 0.00026$ | $2.36$
$i$ | Inclination (Degrees) | $88.980^{+0.094}_{-0.085}$ | $88.7\pm 0.1$ | $1.12$
$e$ | Eccentricity | $0.0062^{+0.0079}_{-0.0044}$ | – | –
$e$ | Eccentricity 95% Upper Limit | $<0.019$ | $<0.063$ | $3.32$
$\omega_{*}$ | Argument of Periastron (Degrees) | $77.0^{+7.5}_{-61}$ | – | –
$T_{\rm eq}$ | Equilibrium temperature6 (K) | $567.0\pm 7.6$ | $596\pm 19$ | $2.50$
$\tau_{\rm circ}$ | Tidal circ timescale (Myr) | $0.0218\pm 0.0014$ | $<1$ | –
$K$ | RV semi-amplitude (m/s) | $14.38^{+0.57}_{-0.56}$ | $14.36\pm 0.53$ | $0.94$
$R_{P}/R_{*}$ | Radius of planet in stellar radii | $0.11589\pm 0.00016$ | $0.11677\pm 0.00017$ | $1.06$
$a/R_{*}$ | Semi-major axis in stellar radii | $14.97^{+0.12}_{-0.14}$ | $14.85\pm 0.16$ | $1.23$
$\delta$ |  $\left(R_{P}/R_{*}\right)^{2}$ 3 | $0.013430^{+0.000037}_{-0.000038}$ | $0.013635\pm 0.000040$ | $1.07$
$\delta_{\rm I}$ | Transit depth in I (fraction) | $0.0173^{+0.0012}_{-0.0011}$ | – | –
$\delta_{\rm MIRI}$ | Transit depth in MIRI LRS (fraction) | $0.01382\pm 0.00021$ | – | –
$\delta_{\rm V}$ | Transit depth in V (fraction) | $0.0161^{+0.0021}_{-0.0016}$ | – | –
$\tau$ | Ingress/egress transit duration (days) | $0.004023^{+0.000050}_{-0.000051}$ | – | –
$T_{14}$ | Total transit duration (days) | $0.036236^{+0.000046}_{-0.000045}$ | – | –
$T_{FWHM}$ | FWHM transit duration (days) | $0.032212^{+0.000043}_{-0.000042}$ | – | –
$b$ | Transit impact parameter | $0.264^{+0.020}_{-0.023}$ | $0.325\pm 0.025$ | $1.16$
$b_{S}$ | Eclipse impact parameter | $0.268^{+0.021}_{-0.023}$ | – | –
$\tau_{S}$ | Ingress/egress eclipse duration (days) | $0.004078^{+0.000087}_{-0.000072}$ | – | –
$T_{S,14}$ | Total eclipse duration (days) | $0.03664^{+0.00055}_{-0.00038}$ | – | –
$T_{S,FWHM}$ | FWHM eclipse duration (days) | $0.03257^{+0.00048}_{-0.00033}$ | – | –
$\delta_{S,2.5\mu m}$ | BB eclipse depth at 2.5$\mu$m (ppm) | $2.83^{+0.32}_{-0.29}$ | – | –
$\delta_{S,5.0\mu m}$ | BB eclipse depth at 5.0$\mu$m (ppm) | $129.1^{+6.4}_{-6.3}$ | – | –
$\delta_{S,7.5\mu m}$ | BB eclipse depth at 7.5$\mu$m (ppm) | $403\pm 12$ | – | –
$\rho_{P}$ | Density (cgs) | $2.26\pm 0.11$ | $2.20^{+0.17}_{-0.16}$ | $1.50$
$logg_{P}$ | Surface gravity (cgs) 3 | $3.043\pm 0.019$ | $3.026\pm 0.028$ | $1.47$
$\Theta$ | Safronov Number | $0.01793^{+0.00074}_{-0.00073}$ | – | –
$\langle\rm F\rangle$ | Incident Flux (109 erg s-1 cm-2) 3 | $0.0234^{+0.0013}_{-0.0012}$ | $0.0286\pm 0.0035$ | $2.80$
$T_{P}$ | Time of Periastron ($\rm{TJD_{TDB}}$) | $2459639.725^{+0.033}_{-0.27}$ | – | –
$T_{S}$ | Time of eclipse ($\rm{BJD_{TDB}}$) | $2459638.99226\pm 0.00020$ | – | –
$T_{E}$ | Time of sec min proj sep ($\rm{BJD_{TDB}}$) | $2459785.96988\pm 0.00020$ | – | –
$T_{A}$ | Time of Ascending Node ($\rm{TJD_{TDB}}$) | $2459640.9702^{+0.0040}_{-0.0028}$ | – | –
$T_{D}$ | Time of Descending Node ($\rm{TJD_{TDB}}$) | $2459640.1740^{+0.0028}_{-0.0040}$ | – | –
$V_{c}/V_{e}$ | Scaled velocity 7 | $0.9940^{+0.0056}_{-0.0080}$ | – | –
$e\cos{\omega_{*}}$ |  | $0.00137\pm 0.00020$ | – | –
$e\sin{\omega_{*}}$ |  | $0.0060^{+0.0080}_{-0.0056}$ | – | –
$M_{P}\sin i$ | Minimum mass ($\,M_{\rm J}$) 3 | $8.41^{+0.36}_{-0.35}$ | $8.17\pm 0.43$ | $1.21$
$M_{P}/M_{*}$ | Mass ratio 3 | $0.0001389^{+0.0000056}_{-0.0000055}$ | $0.000138\pm 0.000011$ | $1.98$
$d/R_{*}$ | Separation at mid transit | $14.89^{+0.19}_{-0.25}$ | – | –
$P_{T}$ | A priori non-grazing transit prob | $0.05939^{+0.0010}_{-0.00073}$ | – | –
$P_{T,G}$ | A priori transit prob | $0.07496^{+0.0013}_{-0.00093}$ | – | –
$P_{S}$ | A priori non-grazing eclipse prob | $0.05869^{+0.00031}_{-0.00032}$ | – | –
$P_{S,G}$ | A priori eclipse prob | $0.07407^{+0.00041}_{-0.00042}$ | – | –
Wavelength Parameters: |  |  |
$u_{1,I}$ | Linear limb-darkening coeff | $0.48^{+0.10}_{-0.11}$ | – | –
$u_{2,I}$ | Quadratic limb-darkening coeff | $0.06\pm 0.16$ | – | –
$u_{1,V}$ | Linear limb-darkening coeff | $0.35^{+0.20}_{-0.19}$ | – | –
$u_{2,V}$ | Quadratic limb-darkening coeff | $0.33^{+0.29}_{-0.30}$ | – | –
$u_{1,MIRI}$ | Linear limb-darkening coeff | $0.062^{+0.029}_{-0.028}$ | – | –
$u_{2,MIRI}$ | Quadratic limb-darkening coeff | $0.056\pm 0.053$ | – | –
$\delta_{S}$ | Measured eclipse depth (ppm) | $398\pm 13$ | – | –
HARPS Parameters: |  |  |
$\gamma_{\rm rel}$ | Pre-upgrade Relative RV Offset (m/s) | $-3.19^{+0.55}_{-0.56}$ | $-3.09^{+1.65}_{-1.66}$ | $2.98$
$\sigma_{J}$ | Pre-upgrade RV Jitter (m/s) 3 | $4.14^{+0.54}_{-0.48}$ | $3.52\pm 0.56$ | $1.10$
$\sigma_{J}^{2}$ | Pre-upgrade RV Jitter Variance 3 | $17.1^{+4.8}_{-3.8}$ | $12.4\pm 4.1$ | $0.95$
$\gamma_{\rm rel}$ | Post-upgrade Relative RV Offset (m/s) | $-0.12^{+0.55}_{-0.58}$ | $-0.61^{+1.76}_{-1.90}$ | $3.24$
$\sigma_{J}$ | Post-upgrade RV Jitter (m/s) 3 | $3.29^{+0.63}_{-0.60}$ | $2.25\pm 0.71$ | $1.15$
$\sigma_{J}^{2}$ | Post-upgrade RV Jitter Variance 3 | $10.8^{+4.6}_{-3.6}$ | $5.1\pm 3.9$ | $0.95$

††footnotetext: See Table 3 in Eastman et al. (2019) for a detailed
description of all parameters

11footnotetext: The improvement is calculated as the average error bar from
Cloutier et al. (2021) divided by the average error bar from our analysis.
Values less than one are less precise, largely due to more pessimistic priors.

22footnotetext: This value ignores the systematic error and is for reference
only

33footnotetext: Cloutier et al. (2021) values were not explicitly stated in
their paper but we derived them from their other parameters assuming
uncorrelated error bars with a Gaussian width of their average error bar

44footnotetext: Time of conjunction is commonly reported as the ”transit time”

55footnotetext: Time of minimum projected separation is a more correct
”transit time”

66footnotetext: Assumes no albedo and perfect redistribution

77footnotetext: See Eastman (2024) for more information regarding this
quantity

Figure 2:  The upper panel shows the primary JWST transit of GJ 1214b. The
black points represent each respective data point, the red line represent the
best-fit model using EXOFASTv2, and the lower panel representing the model’s
residuals of observed minus calculated.  Figure 3:  Same as Figure 2, but for
the JWST secondary eclipse. Note the transit has been normalized so that the
stellar flux (i.e., during the secondary eclipse) is 1.  Figure 4:  All
primary transit light curves used in our global model, plotted with an
arbitrary offset. The black points represent each respective data point and
the red lines represent the best-fit model using EXOFASTv2. Labels above each
transit state the telescope, epoch, and filter used for the observation.

##  5 Systematics

The residuals in Figure 2 show clear signs of correlated noise that could be
due to star spots or other systematics. To assess their impact on the transit
duration, we shifted the residuals from our final fit by -30, -20, -10, +10,
+20, and +30 minutes, added back our best-fit model, and refit it, following
exactly the same procedure as our original fit. Among these six fits, the
largest discrepancy in $R_{*}$, $T_{\rm eff}$, or $\,\rho_{*}$ was $\sim
1\sigma$. If our fits were completely independent measurements of the same
quantities, such a discrepancy is statistically likely, suggesting that the
correlated noise has no statistically significant impact on our inferred
values. We also ran a fit that simultaneously detrended against x position, y
position, and time, but the impact on the final results was negligible
($<0.1\sigma$).

Moran et al. (2023) noted $\sim 6\sigma$ discrepancies in $a/R_{*}$ for JWST
observations of a different star, GJ 486. Their comparison only changed the
extraction pipeline, suggesting that light curve extraction systematics may be
significant. At Moran et al. (2023)’s 0.2% quoted precision in $a/R_{*}$, the
uncertainty they would get in GJ 486’s $R_{*}$ and $T_{\rm eff}$ would be
dominated by the 2.3% uncertainty in $M_{*}$, but we would still be concerned
that such large systematics could bias the determination of $\,\rho_{*}$,
$R_{*}$ and $T_{\rm eff}$.

To ensure that light curve extraction systematics were not important for our
analysis of GJ 1214, we repeated the analysis described in §3, only swapping
our Tiberius-extracted light curve with the Kempton-extracted (timestamp-
corrected) light curve and found that the differences in $a/R_{*}$,
$\,\rho_{*}$, $R_{*}$, $T_{\rm eff}$, $e$, $\,R_{\rm P}$, and $\langle\rm
F\rangle$ were all below $0.4\sigma$. The only parameters that differed by
more than $1\sigma$ were $\,R_{\rm P}$/$R_{*}$ ($2.4\sigma$), the MIRI limb
darkening coefficients ($1.2\sigma$), and the secondary eclipse time
($1.2\sigma$). We also note that Moran et al. (2023)’s 0.2% $a/R_{*}$
uncertainty implies a $\,\rho_{*}$ uncertainty that is 10$\times$ smaller than
expected from equation 11.

Although we show in §6 that systematics in ground based data are not relevant
for our analysis of GJ 1214, in general, it could be a significant factor and
care must be taken to ensure that other sources of systematics (due to, e.g.,
airmass, atmospheric variability) are properly accounted for.

While negligible for GJ 1214, there are other caveats that one should be aware
of when apply this method more generally. For very rapidly rotating stars, the
oblateness means the inferred radius is dependent on the star-planet
orientation. This is negligible in the case of GJ 1214 but may be relevant for
rapidly rotating stars.

Other potential sources of systematic error in measuring $\,\rho_{*}$ are
detailed in Kipping (2014) and Eastman et al. (2023), but generally, one
should be careful when applying this method to stars with light from another
source that is unaccounted for, a non-neglibible planet mass, grazing
transits, or non-Keplerian motion.

##  6 Discussion

The ability to accurately measure the radii of M stars to such a degree is
uncommon, having only been achieved for a small number of other low-mass
targets. As GJ 1214 is an archetype for super-Earths around M-dwarfs, precise
constraints are invaluable and contribute to a deeper understanding of
habitability, planet formation, and stellar evolution for all other M-dwarf
planetary systems.

Our method applies to any high signal-to-noise (SNR) transit with a low-mass
host where the eccentricity can be measured very precisely. We identified 6
additional planets in the JWST archive with primary and secondary eclipses
that should allow us to precisely measure this eccentricity as summarized in
Table LABEL:tab:observations.

The expected precision in $\,\rho_{*}$ can be computed with the equation from
Stevens et al. (2018),

$\frac{\sigma_{\,\rho_{*}}}{\,\rho_{*}}\approx\left(\frac{27}{2Q^{2}\theta}\right)^{1/2},$
(11)

where $Q\equiv\sqrt{\Gamma T_{\rm FWHM}}\delta/\sigma$,
$\theta\equiv\tau/T_{\rm FWHM}$, $\sigma$ is the point measurement error at
the data sampling rate $\Gamma$, $\delta$ is the transit depth, $\tau$ is the
ingress/egress time, and $T_{\rm FWHM}$ is the duration of the transit from
mid ingress to mid egress. This calculation ignores the contribution from limb
darkening and eccentricity and should be treated as a rough guide, but for
reference, our predicted 1.5% precision of $\,\rho_{*}$ for GJ 1214 is
reasonably close to the $2.62$% precision achieved above.

While we included ground based transits from 2012 and RVs for our fit of GJ
1214, it was not necessary for this method of determining a precise $R_{*}$
and $T_{\rm eff}$. We only did that to provide a new, definitive reference for
GJ 1214 that incorporates the best knowledge of the planetary mass and period
as possible. We also ran a fit for GJ 1214 only using JWST data and found that
the eccentricity, stellar radius, temperature, and density were constrained to
a similar precision. The only change in precision was in the period and planet
mass, which is expected. While the targets in Table LABEL:tab:observations
also typically have a long baseline of ground based observations and public
RVs, we expect their contribution to $\,\rho_{*}$ and $e$ to be similarly
negligible.

We used the JWST ETC (Exposure Time Calculator) to calculate the $\sigma$ as
used in the equation for $Q$. The expected precision using equation 11 is
shown in Table LABEL:tab:observations as $\sigma_{\,\rho_{*}}/\,\rho_{*}$. The
differences in expected precision for different light curves of the same
object is entirely due to the expected difference in the SNR and ignores time
sampling, covariance with limb darkening, and eccentricity uncertainty.

Table 4: JWST archival observations publicly available by Cycle 3 for $0.1<M_{*}/\,M_{\odot}<0.7$ with at least one primary transit and secondary eclipse showing the predicted fractional uncertainty in $\,\rho_{*}$ using equation 11 and the JWST Exposure time calculator. Note that the dashes in the table correspond to eclipses, which are used to determine the eccentricity and not $\,\rho_{*}$. Target name | Event | $\sigma_{\,\rho_{*}}/\,\rho_{*}$ | Instrument
---|---|---|---
GJ 1132 | Eclipse | – | MIRI.LRS
GJ 1132 | Transit x 2 | 1.68% | NIRSPEC.BOTS+G395H
GJ 1214 | Transit x 2 | 0.76% | NIRSPEC.BOTS+G395H
GJ 1214 | PhaseC | 1.53% | MIRI.LRS
GJ 486 | Eclipse x 2 | – | MIRI.LRS
GJ 486 | Transit x 2 | 3.25% | NIRSPEC.BOTS+G395H
LTT 1445A | Transit | 1.94% | NIRSPEC.BOTS+G395H
LTT 1445A | Eclipse x 3 | – | MIRI.LRS
NGTS-10 | PhaseC | 0.34% | NIRSPEC.BOTS+PRISM
WASP-43 | PhaseC | 0.18% | NIRSPEC.BOTS+G395H
WASP-43 | PhaseC | 1.28% | MIRI.LRS
WASP-80 | Transit | 0.44% | MIRI.LRS
WASP-80 | Eclipse | – | MIRI.LRS
WASP-80 | Transit | 0.10% | NIRCAM.GRISMR+F322W2
WASP-80 | Transit | 0.16% | NIRCAM.GRISMR+F444W
WASP-80 | Eclipse | – | NIRCAM.GRISMR+F322W2
WASP-80 | Eclipse | – | NIRCAM.GRISMR+F444W
WASP-80 | Transit | 0.05% | NIRISS.SOSS

Our method is immediately applicable to six additional stars currently
available in the JWST archive, increasing the sample of precise stellar radii
with which to calibrate future relations similar to Mann et al. (2015) and is
a valuable proof of concept to add many more. The number of possible low-mass
targets that can be analyzed to achieve high precision measurements similar to
GJ 1214 using our method will continue to grow as JWST observations are taken
for atmospheric science goals. In fact, in November 2023, the preliminary JWST
Exoplanet working group recommended that JWST observe 15-20 rocky planets
around M-dwarfs, all of which should be good targets for this method.

While we have focused on low-mass host stars, the same basic technique applies
to any star where we can measure the eccentricity using transits and
occultations (e.g., from JWST), the density from the transit, the radius using
a stellar mass, and a temperature from the bolometric flux. The only
difference to account for is the stellar mass – the Mann et al. (2019)
relation used in our analysis does not apply for stars with a mass larger than
0.7 $\,M_{\odot}$. Instead, we could use a mass constraint from e.g.,
evolutionary models.

We thank the anonymous referee for their insightful comments that improved the
quality of our manuscript. We are grateful for discussions about systematic
errors with Sarah Moran, Jacob Bean, Eliza Kempton, and Michael Zheng. Work by
A.S.M. and J.D.E. was funded by NASA ADAP 80NSSC19K1014. J.K. acknowledges
financial support from Imperial College London through an Imperial College
Research Fellowship grant. This work is based in part on observations made
with the NASA/ESA/CSA JWST. The data were obtained from the Mikulski Archive
for Space Telescopes at the Space Telescope Science Institute, which is
operated by the Association of Universities for Research in Astronomy, Inc.,
under NASA contract NAS 5-03127 for JWST. These observations are associated
with program #1803. This research has made use of the Exoplanet Follow-up
Observation Program website, which is operated by the California Institute of
Technology, under contract with the National Aeronautics and Space
Administration under the Exoplanet Exploration Program. This research has made
use of the NASA Exoplanet Archive, which is operated by the California
Institute of Technology, under contract with the National Aeronautics and
Space Administration under the Exoplanet Exploration Program. This work has
made use of data from the European Space Agency (ESA) mission Gaia
(https://www.cosmos.esa.int/gaia), processed by the Gaia Data Processing and
Analysis Consortium (DPAC,
https://www.cosmos.esa.int/web/gaia/dpac/consortium). Funding for the DPAC has
been provided by national institutions, in particular the institutions
participating in the Gaia Multilateral Agreement. This publication makes use
of data products from the Two Micron All Sky Survey, which is a joint project
of the University of Massachusetts and the Infrared Processing and Analysis
Center/California Institute of Technology, funded by the National Aeronautics
and Space Administration and the National Science Foundation. This publication
makes use of data products from the Wide-field Infrared Survey Explorer, which
is a joint project of the University of California, Los Angeles, and the Jet
Propulsion Laboratory/California Institute of Technology, funded by the
National Aeronautics and Space Administration.

## References

  * Adams & Laughlin (2006) Adams, F. C., & Laughlin, G. 2006, ApJ, 649, 1004
  * Alonso (2018) Alonso, R. 2018, in Handbook of Exoplanets, ed. H. J. Deeg & J. A. Belmonte, 40
  * Bell et al. (2012) Bell, C. P. M., Naylor, T., Mayne, N. J., Jeffries, R. D., & Littlefair, S. P. 2012, MNRAS, 424, 3178
  * Berta et al. (2011) Berta, Z. K., Charbonneau, D., Bean, J., et al. 2011, ApJ, 736, 12
  * Casagrande et al. (2014) Casagrande, L., Portinari, L., Glass, I. S., et al. 2014, MNRAS, 439, 2060
  * Charbonneau et al. (2005) Charbonneau, D., Allen, L. E., Megeath, S. T., et al. 2005, ApJ, 626, 523
  * Charbonneau et al. (2009) Charbonneau, D., Berta, Z. K., Irwin, J., et al. 2009, Nature, 462, 891
  * Claret (2017) Claret, A. 2017, A&A, 600, A30
  * Cloutier et al. (2021) Cloutier, R., Charbonneau, D., Deming, D., Bonfils, X., & Astudillo-Defru, N. 2021, AJ, 162, 174
  * Cutri et al. (2012) Cutri, R. M., Wright, E. L., T., C., & at al. 2012, VizieR Online Data Catalog, 2311, 0
  * Cutri et al. (2003) Cutri, R. M., Skrutskie, M. F., van Dyk, S., et al. 2003, 2MASS All Sky Catalog of point sources.
  * Eastman (2024) Eastman, J. D. 2024, PASP, 136, 014502
  * Eastman et al. (2023) Eastman, J. D., Diamond-Lowe, H., & Tayar, J. 2023, AJ, 166, 132
  * Eastman et al. (2019) Eastman, J. D., Rodriguez, J. E., Agol, E., et al. 2019, arXiv e-prints, arXiv:1907.09480
  * El-Badry et al. (2021) El-Badry, K., Rix, H.-W., & Heintz, T. M. 2021, MNRAS, 506, 2269
  * Ford et al. (2008) Ford, E. B., Quinn, S. N., & Veras, D. 2008, ApJ, 678, 1407
  * Fraine et al. (2013) Fraine, J. D., Deming, D., Gillon, M., et al. 2013, ApJ, 765, 127
  * Gaia Collaboration (2020) Gaia Collaboration. 2020, VizieR Online Data Catalog, I/350
  * Gillon et al. (2014) Gillon, M., Demory, B. O., Madhusudhan, N., et al. 2014, A&A, 563, A21
  * Gillon et al. (2017) Gillon, M., Demory, B.-O., Van Grootel, V., et al. 2017, Nature Astronomy, 1, 0056
  * Huber et al. (2017) Huber, K. F., Czesla, S., & Schmitt, J. H. M. M. 2017, A&A, 597, A113
  * Kempton et al. (2023) Kempton, E. M. R., Zhang, M., Bean, J. L., et al. 2023, Nature, 620, 67
  * Kipping (2014) Kipping, D. M. 2014, MNRAS, 440, 2164
  * Kirk et al. (2017) Kirk, J., Wheatley, P. J., Louden, T., et al. 2017, MNRAS, 468, 3907
  * Kirk et al. (2021) Kirk, J., Rackham, B. V., MacDonald, R. J., et al. 2021, AJ, 162, 34
  * Kopal (1946) Kopal, Z. 1946, in Harvard Observatory Monographs, Vol. 6, 1
  * Lindegren et al. (2021) Lindegren, L., Bastian, U., Biermann, M., et al. 2021, A&A, 649, A4
  * Mann et al. (2015) Mann, A. W., Feiden, G. A., Gaidos, E., Boyajian, T., & von Braun, K. 2015, ApJ, 804, 64
  * Mann et al. (2017) Mann, A. W., Dupuy, T., Muirhead, P. S., et al. 2017, AJ, 153, 267
  * Mann et al. (2019) Mann, A. W., Dupuy, T., Kraus, A. L., et al. 2019, ApJ, 871, 63
  * Mayor et al. (2003) Mayor, M., Pepe, F., Queloz, D., et al. 2003, The Messenger, 114, 20
  * Moran et al. (2023) Moran, S. E., Stevenson, K. B., Sing, D. K., et al. 2023, ApJ, 948, L11
  * Patel & Espinoza (2022) Patel, J. A., & Espinoza, N. 2022, AJ, 163, 228
  * Rustamkulov et al. (2023) Rustamkulov, Z., Sing, D. K., Mukherjee, S., et al. 2023, Nature, 614, 659
  * Schlafly & Finkbeiner (2011) Schlafly, E. F., & Finkbeiner, D. P. 2011, ApJ, 737, 103
  * Schlegel et al. (1998) Schlegel, D. J., Finkbeiner, D. P., & Davis, M. 1998, ApJ, 500, 525
  * Seager & Mallén-Ornelas (2003) Seager, S., & Mallén-Ornelas, G. 2003, ApJ, 585, 1038
  * Stassun & Torres (2018) Stassun, K. G., & Torres, G. 2018, ApJ, 862, 61
  * Stassun & Torres (2021) —. 2021, ApJ, 907, L33
  * Stauffer et al. (2007) Stauffer, J. R., Hartmann, L. W., Fazio, G. G., et al. 2007, ApJS, 172, 663
  * Stevens et al. (2018) Stevens, D. J., Gaudi, B. S., & Stassun, K. G. 2018, The Astrophysical Journal, 862, 53
  * Tayar et al. (2022) Tayar, J., Claytor, Z. R., Huber, D., & van Saders, J. 2022, ApJ, 927, 31
  * Thompson et al. (2014) Thompson, B., Frinchaboy, P., Kinemuchi, K., Sarajedini, A., & Cohen, R. 2014, AJ, 148, 85
  * Torres (2013) Torres, G. 2013, Astronomische Nachrichten, 334, 4
  * Torres et al. (2010) Torres, G., Andersen, J., & Giménez, A. 2010, A&A Rev., 18, 67
  * Vanderburg & Johnson (2014) Vanderburg, A., & Johnson, J. A. 2014, PASP, 126, 948


# Discrete-continuous Action Space Policy Gradient-based Attention for Image-
Text Matching

Shiyang Yan1 Corresponding Author Li Yu1 Yuan Xie211footnotemark: 1
1Nanjing University of Information Science and Technology  Nanjing  China
2East China Normal University  Shanghai  China
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Image-text matching is an important multi-modal task with massive
applications. It tries to match the image and the text with similar semantic
information. Existing approaches do not explicitly transform the different
modalities into a common space. Meanwhile, the attention mechanism which is
widely used in image-text matching models does not have supervision. We
propose a novel attention scheme which projects the image and text embedding
into a common space and optimises the attention weights directly towards the
evaluation metrics. The proposed attention scheme can be considered as a kind
of supervised attention and requiring no additional annotations. It is trained
via a novel Discrete-continuous action space policy gradient algorithm, which
is more effective in modelling complex action space than previous continuous
action space policy gradient. We evaluate the proposed methods on two widely-
used benchmark datasets: Flickr30k and MS-COCO, outperforming the previous
approaches by a large margin.

##  1 Introduction

Computer Vision and Natural Language Processing are two important areas of
modern artificial intelligence, which can be processed jointly in cross-modal
tasks. A large amount of research has been conducted to bridge the vision and
language modalities [32, 35, 5, 19, 18]. Image-text matching or retrieval is
one of the critical topics in this area, which has a huge application scope in
many real-world scenarios. The image-text matching requires a machine learning
model to extract the high-level semantic representations and measure the
similarities across modalities accordingly.

Figure 1: Our Motivations: The attention weights are utilised as a projection
from each modality to a common space. Existing continuous PG assumes a simple
Normal distribution. Instead, we considered the mean values as discrete
actions first and then use multiple Normal distributions to form a compound
distribution, which is more realistic. Best Viewed in Colour.

Existing methods use deep learning models to extract the image and language
features, and apply various metric learning techniques to automatically find
the semantic similarities between the samples from the two modalities [5, 19,
18]. Metric learning is powerful in visual semantic embedding as it tries to
measure and manipulate the similarities between samples regardless of the
domain differences. However, it is not designed to perform an explicit
transformation from one modality to another, often leading to sub-optimal
performance. Though there are approaches apply Instance Loss [37], i.e., a
classification over image and text categories, to form a multi-task learning
approach with metric learning loss for image-text matching, the performance
gain is limited as the Instance Loss optimises the embedding in the category
domain, which does not perform explicit transformation either. An image often
contains many fine-grained objects. A flat vector representation from a
vanilla deep CNN model such as ResNet [9] is not powerful enough to discover
these objects and their relationships. Hence, advanced methods employ image
features from a pre-trained object detector [28] and apply the visual
attention mechanism [35] on these features to discriminate the important
features over irrelevant ones [1]. Attention mechanism plays a significant
role in varying computer vision tasks. It is considered as hidden neurons in
these models, but often leads to incorrect selection of image features for
lacking a direct supervision [23].

In this paper, to make an explicit transformation and provide supervision to
the attention mechanism in image-text matching, we propose a policy gradient
(PG) [30] optimised attention adjustment over the visual and text features in
image-text matching. The attention weights in our approach can be considered
as a transformation from a specific modality to a common space, as the
attention weights perform a vector transformation in the last image and text
vectors used for matching, instead of selecting important features in the
previous layers of the deep learning models [35]. The attention weights are
trained by the PG method with the batch-wise ranking metrics and the instance-
wise Average Precision (AP) as the reward function. These attention weights
are directly optimised via PG algorithm to achieve optimal ranking results and
higher AP metrics. It can be considered as a kind of supervised attention
mechanism, and this supervision does not need any additional annotations. This
PG-based attention mechanism is straightforward and is optimised towards the
evaluation metrics. It is also more accurate than the conventional soft
attention which is only a regular neuron.

To be more specific, as shown in Figure 1, we consider the attention weights
generation as an action selecting process in PG, whose space can be flexibly
pre-defined. The action space in conventional PG is discrete, which is not
suitable for the feature adjustment like in the attention mechanism. One
solution is applying a continuous action space PG algorithm [20], which
consider the action space as a Gaussian distribution and sample action values
from this distribution. Restricting the action distribution to Normal is not
optimal, and such a hypothesis lacks theoretical and practical support. In
reality, the distribution of the action space might be very complex, which
cannot be described via a simple Normal distribution. Hence, we consider the
action is continuous and sampled from multiple Normal distributions with a
different mean ($\mu$) and standard deviation ($\sigma$) values. We first
treat the $\mu$ as discrete actions, sampled from a pre-defined action space
while $\sigma$ is obtained from a neural model as it is continuous. We want to
use this $\mu$ and $\sigma$ to form a Normal distribution and sample
continuous action from this distribution, which is applied as the attention
weights to adjust the feature representations for both the visual embedding
and the text embedding. Usually, in conventional PG, we do not need the $\mu$
to be trainable as we only back-propagate the gradient to the log-
probabilities. In contrast, in this case, the subsequently obtained Normal
distributions need the $\mu$ being able to be back-propagated to make the
Normal distribution learn-able. As there involves sampling in obtaining the
$\mu$, it is not trainable in this current form. To make this $\mu$
differentiable, instead of directly using greedy sampling or $\epsilon$-greedy
sampling. We use a Gumbel-softmax to relax the discreteness [12] and make the
sampled $\mu$ trainable together with the Normal distribution. We call this
method “Discrete-continuous PG” as it involves both discrete and continuous
action space, making them benefit from each other. In fact, by using the
discrete and continuous action space, the action space used to sample the
attention weights is a compound distribution, which can model a high complex
distribution. We evaluate our algorithm and model on image-text matching task,
achieving state-of-the-art performance on two widely-used benchmark datasets.
To summarise, our contributions are threefold: (1) We propose a novel
attention supervision scheme for image-text matching task based on policy
gradient. (2) We propose a new Discrete-continuous policy gradient algorithm
by leveraging both the discrete and continuous action space. (3) The achieved
state-of-the-art results validate the effectiveness of the attention
supervisions scheme and the novel policy gradient algorithm.

##  2 Related Works

###  2.1 Image-text Matching

Frome et al. [6] propose a feature embedding method via CNNs and Skip-Gram for
cross-modal matching. They also utilise a ranking loss to measure the distance
between similar pairs. Faghri et al. [5] focus on the hard negative mining in
the Triplet loss, with improved results. Zheng et al. [37] utilise an Instance
Loss over a large number of categories. They find that the Instance Loss is
helpful in image-text matching. Gu et al. [8] improve the cross-modal problem
by looking into the generative models. Li et al. [19] propose a visual
semantic reasoning framework by leveraging graph neural networks and image
captioning loss. The visual semantic reasoning model can reason on the
semantic relationship of the image features, with good performance.

###  2.2 Attention Mechanism

The visual attention mechanism [35] has been widely applied in many types of
computer vision applications. Notably, bottom-up attention model [1] is the
current mainstream for image captioning, visual question answering and image-
text matching. However, there is not much research on supervised attention.
Gan et al. [7] propose a supervised attention scheme on visual question
answering using attention annotations. Kamigaito et al. [13] also use
attention annotations for supervised attention in natural language processing
task. Instead, we propose a supervised attention mechanism based on
reinforcement learning, which can make the attention module directly optimise
towards a specific goal such as AP. Also, the proposed attention module does
not need any additional annotations.

###  2.3 Continuous Action Space Policy Gradient

The continuous control problem has long been investigated. For example,
Lillicrap et al. [20] propose the deep deterministic policy gradient by
considering a continuous action space. Previous research has exploited the
relationship between discrete and continuous action space. For instance,
Dulacc-Arnold et al. [3] leverage the continuity in the underlying continuous
action space for generalisation on discrete actions. Pazis et al. [27] convert
the continuous control problem into discrete ones, by using a binary discrete
action space. Tang et al. [31] show that discretizing action space for
continuous control is a simple yet powerful technique for on-policy
optimisation. We also consider the combination of discrete and continuous
action space for on-policy optimisation. We prove that a compound distribution
is superior to a strict assumption of one Normal distribution.

##  3 The Proposed Method

Figure 2: A schematic diagram of the proposed method: The image and text are
forwarded into the model. The extracted image features are first processed via
a GCN model to reason on the semantic relationships. The region features are
then inputted to the proposed Discrete-continuous PG algorithm to generate
attention maps, which are applied to adjust and fuse the region features
subsequently. Similarly, the text embedding is also adjusted via the attention
maps generated by the Discrete-continuous PG algorithm. The final image and
text embedding are then connected with Metric Learning losses, the Discrete PG
Loss and the Continuous PG loss for training. Best Viewed in Colour.

Our goal is to adjust the generated visual and text features to facilitate the
image-text matching. We first apply Graph Convolutional Neural Networks [34]
on the bottom-up attention [1] features of the images, which is similar to the
Visual Semantic Reasoning Networks (VSRN) [19]. Once the visual features are
obtained, we then use our Discrete-continuous action space PG to generate the
attention weights, which are used to adjust the visual features. Similarly,
the text features are also adjusted via the proposed Discrete-continuous PG-
based attention mechanism. The obtained image and text embedding are trained
via multi-task loss, including the Triplet Loss, Instance Loss and Text
Decoding Loss. A schematic diagram of the proposed method is shown in Figure
2.

###  3.1 Image and Text Features Extraction

#### GCN for image region features reasoning.

We apply a GCN model similar to VSRN approach [19]. Specifically, the semantic
relationship between image region features is measured via pairwise affinity.

$Relation(F_{i},F_{j})=E_{i}(F_{i})^{T}E_{i}(F_{j}),$ (1)

where $F_{i}$ and $F_{j}$ are two bottom-up image region features obtained
from Faster R-CNN detectors. $E_{i}$ and $E_{j}$ are embedding functions,
which are usually matrices multiplications, which are can be learnt via
backpropagation.

Then a fully-connected relationship graph $G_{r}=(V,E)$ is constructed. $V$ is
the set of detected image region features and $E$ is the set of edges where
each of the edges is described by the affinity matrices
$Relation(F_{i},F_{j})$, which is presented in Equation 1. We apply the GCN to
perform reasoning on this fully-connected graph. The output of the GCN
reasoning is denoted as $Image=\\{I^{1},...,I^{t},...,I^{T}\\}$.

#### Text Embedding.

Given one-hot text representations, represented as $w$, a linear word
embedding layer is constructed to obtain the word representations, represented
as $W_{e}=\\{w_{e}^{1},...,w_{e}^{i},...,w_{e}^{N}\\}$, where
$w_{e}^{i}=word\text{-}embedding(w^{i})$.

###  3.2 The Proposed Discrete-continuous Action Space PG

PG is usually with discrete action space for two reasons: many control
problems are modeled in discrete action space which leads to high performance
as it can model complex action distribution. However, when meeting with
continuous action space control problem, we have to develop corresponding PG
algorithms. However, as discussed previously, continuous action space PG
normally assumes the actions follow a Normal distribution, which is too
strict. We propose an approach to essentially sample the continuous action
from a compound distribution, which can better model the real distribution.

#### Discrete Action Sampling.

As shown in Figure 2, we first model the attention weights generation process
as a finite Markov Decision Process (MDP) and sample a discrete action by
using Multinomial Sampling. We define $n$ action categories, i.e.,
$A=\\{a_{1},a_{2},...,a_{n}\\}$, The state space contains the input region
features and the attention weights generated so far, which are
$s_{t}=\\{I^{0},Att^{0}...,I^{t-1},Att^{t-1}\\}$. The policy is parametrised
via a GRU model to explore the environment and sample the action. More
formally:

$\begin{split}&h^{t}=GRU_{mdp}(I^{t},h^{t-1}),\ \ t=1,...,T,\\\
&F_{I}^{t}=h^{t},\\\ &a^{t}=F_{I}^{t}*W_{\mu}^{t},\\\
&a_{g}^{t}=Gumbel\text{-}softmax(a^{t}),\\\ &a_{s}^{t}\sim
Multinomial(a_{g}^{t}),\\\ &logprob_{a}^{t}=\log a_{g}^{t}(a_{s}^{t}),\\\
\end{split}$ (2)

where $I^{t}$ is the $t_{th}$ image feature after the GCN reasoning.
$GRU_{mdp}$ is the Gated Recurrent Unit (GRU) used to model the attention
weights generation problem as MDP. $W_{\mu}^{t}\in\mathcal{R}_{s\times n}$ are
the weights need to be learnt. $s$ is the size of the feature vector.
$a_{g}^{t}$ is the probability of each actions after the
$Gumbel\text{-}softmax$ activation.

$\begin{split}&\mu^{t}=Logistic\left(\frac{a_{s}^{t}}{n}\right),\\\
&{std}^{t}=F_{I}^{t}*W_{std}^{t},\\\ \end{split}$ (3)

where $W_{std}\in\mathcal{R}_{s\times 1}$ are the weights need to be learnt.

#### Continuous Action Sampling.

The sampled $\mu$ and $\sigma$ form a Normal Distribution, described as
follows:

$\begin{split}&Sample\sim\mathcal{N}(\mu^{t},\sigma^{t}),\\\
&Att^{t}=Sigmoid(Sample),\end{split}$ (4)

where $Att^{t}$ are the attention weights sampled from this particular Normal
Distribution. The log probabilities of this Normal Distribution is expressed
as:

$\begin{split}&\log{\left(f\left(Att^{t};\,\mu^{t},{\sigma^{t}}^{2}\right)\right)}=\\\
&-\frac{n}{2}\log{\left(2\pi\right)}-\frac{n}{2}\log{\left({(\sigma^{t})}^{2}\right)}\\\
&-\frac{1}{2{(std^{t})}^{2}}\sum{{\left(Att-\mu^{t}\right)}^{2}}.\end{split}$
(5)

#### Discrete PG Optimisation.

To be simple and efficient, we formulate the PG as an on-line learning method,
specifically, the REINFORCE algorithm [33]. The PG for discrete action space
is then to maximise the long-term reward with the following expression:

$\begin{split}&\nabla_{\theta}J(\theta)=\\\
&\mathbb{E}_{\tau\sim\pi_{\theta}(\tau)}\left[\left(\sum_{t=0}^{T}\nabla_{\theta}\log{\pi_{\theta}}(a_{t}\mid
s_{t})\right)\left(\sum_{t=0}^{T}r(s_{t},a_{t})\right)\right],\end{split}$ (6)

we use the one sample Monte-Carlo to approximate the accumulative reward,
i.e., $\sum_{t=0}^{T}r(s_{t},a_{t})=\sum_{t=0}^{T}\mathcal{R}$, where
$\mathcal{R}$ is the reward and will be defined later. Also,
$\log{\pi_{\theta}}(a_{t}\mid s_{t})=logprob_{a}^{t}$, which is obtained from
Equation 2. Hence, Equation 6 can lead to a PG loss function as follows,

$\begin{split}&Loss_{PG_{D}}=-\sum_{b=1}^{B}\left[\left(\sum_{t=0}^{T}\nabla_{\theta}logprob_{a}^{t}\right)\left(\sum_{t=0}^{T}\mathcal{R}\right)\right],\end{split}$
(7)

where $B$ is the size of each mini-batch. Note that the negative notation on
the right-hand side means that we want to minimise the loss so as to maximise
the $\mathcal{R}$.

#### Continuous PG Optimisation.

Equation 5 provides a straightforward definition of the log probabilities of a
Normal Distribution. Similarly, the PG loss for the continuous action space is
presented as follows:

$\begin{split}&Loss_{PG_{C}}=\\\
&-\sum_{b=1}^{B}\left[\left(\sum_{t=0}^{T}\log{\left(f\left(Att^{t};\,\mu^{t},{\sigma^{t}}^{2}\right)\right)}\right)\left(\sum_{t=0}^{T}\mathcal{R}\right)\right].\end{split}$
(8)

#### Reward Function Formulation.

The reward signal is of vital significance as it guides the attention
generation process, which is the initial goal of PG method. The reward signal
is obtained from an on-line evaluation of the image and text embedding using
R@K and Average Precision (AP). We consider a batch of samples as the gallery,
and each sample as a query to compute the instance-wise AP. Specifically, we
treat each of the samples as one category, and calculate the R@1 and AP of it
on-line in a batch of samples. The reward signal can thus be expressed as a
linear combination of the R@1 and the AP results:

$\mathcal{R}=R\text{@}1+AP,$ (9)

we then use this reward to guide the proposed PG algorithm to generate
attention weights to automatically adjust the image and text features to
formulate a more effective embedding for the image-text matching task. To
further reduce the variance and make the PG training more stable, we
additionally apply a PG baseline, which is an average of the rewards from all
the other instances within a batch, expressed as:

$b_{k}=\frac{1}{K-1}\sum_{j\neq k}\mathcal{R}_{j},$ (10)

where $K$ is the batch size, $b_{k}$ is the baseline for $k_{th}$ instance and
$R_{j}$ is the reward of $j_{th}$ instance. We apply a coefficient $\beta=0.5$
over the baseline, which is empirically better.

###  3.3 Feature Fusion

The image embedding can be adjusted by using the generated attention weights.
Recall the image region features as $Image=\\{I^{1},...,I^{t},...,I^{T}\\}$,
and the generated attention weights are
$ATT=\\{Att^{1},...,Att^{t},...,Att^{T}\\}$, we use element-wise
multiplication to adjust the image region features with the attention weights.

$\begin{split}&I_{A}=Image*(\lambda*ATT^{I}),\\\
&h_{g}^{t}=GRU_{gr}^{I}(I_{A}^{t},h_{g}^{t-1}),\ t=1,...,T,\\\
&I_{E}=h_{g}^{T}+[\sum_{t=1}^{T}I_{A}]/T,\end{split}$ (11)

where $I_{A}$ stands for adjusted image region features. $GRU_{gr}^{I}$ is
used to perform global reasoning of the adjusted image features. The fused
features involve a summation of the outputs of the $GRU_{gr}$ and the adjusted
image region features. $I_{E}$ is the image embedding.

Similarly, we apply the same approach to the text embedding generation. Note
that we directly apply the proposed Discrete-continuous PG on the word
embedding $W_{e}$.

Then the feature adjustment and fusion of text embedding generation can be
presented as follows:

$\begin{split}&T_{A}=W_{e}*(\lambda*ATT^{T}),\\\
&h_{g}^{i}=GRU_{gr}^{T}(T_{A}^{i},h_{g}^{i-1}),\ i=1,...,N,\\\
&T_{E}=h_{g}^{N}+[\sum_{i=1}^{N}T_{A}]/N,\end{split}$ (12)

where $T_{A}$ is the adjusted text features and $ATT^{T}$ are the attention
weights generated for text embedding. $T_{E}$ is the text embedding.

###  3.4 Loss Functions

To fulfill the image-text matching task, we apply cross-modal Triplet Loss,
Instance Loss, Text Decoding Loss, and together with the proposed PG loss, to
train the model. The final loss objective function of the model is described
as follows:

$\begin{split}\mathbb{L}=&Loss_{triplet}+Loss_{xe}+loss_{td}^{I}+loss_{td}^{T}\\\
&+Loss_{PG_{c}}^{I}+Loss_{PG_{d}}^{I}+Loss_{PG_{c}}^{T}+Loss_{PG_{d}}^{T},\end{split}$
(13)

where $Loss_{triplet}$ is the hinge-based Triplet ranking loss [5, 14, 18].
The $Loss_{xe}$ is the cross-entropy classification loss which treats each
instance as one class categories [37]. The $Loss_{td}^{I}$ and $Loss_{td}^{T}$
are the Image-to-Text Decoding Loss and Text-to-Text Decoding Loss,
respectively. They decode the image or text embedding into sentences. Note the
weights of the Text Decoding Module are shared between image and text
branches.

The Triplet loss is expressed as follows:

$\begin{split}Loss_{metric}=&[\alpha-S(I,T)+S(I,\hat{T})]_{+}+\\\
&[\alpha-S(I,T)+S(\hat{I},T)]_{+},\end{split}$ (14)

where $\alpha$ is the margin hyper-parameter.$[x]_{+}=max(x,0)$. $S(\cdot)$ is
the similarity function. $\hat{I}$ and $\hat{T}$ are the hardest negatives for
a positive pair $(I,T)$.

For the Text Decoding Loss, We apply the convolutional image captioning model
[2] as the decoder of the image and text decoding module. We use the same loss
functions as in [2], which has a parallel training capability for text
decoding and is much efficient than the RNN-based one.

Networks | Methods | Caption Retrieval | Image Retrieval
---|---|---|---
R@1 | R@5 | R@10 | R@1 | R@5 | R@10
AlexNet | DSVA [14] | 22.2 | 48.2 | 61.4 | 15.2 | 37.7 | 50.5
HMlstm [25] | 38.1 | - | 76.5 | 27.7 | - | 68.8
VGG | FV [16] | 35.0 | 62.0 | 73.8 | 25.0 | 52.7 | 66.0
VQA [22] | 33.9 | 62.5 | 73.8 | 25.0 | 52.7 | 66.0
SMlstm [10] | 42.5 | 71.9 | 81.5 | 30.2 | 60.4 | 72.3
2wayN [4] | 49.8 | 67.5 | - | 36.0 | 55.6 | -
ResNet | RRF [24] | 47.6 | 77.4 | 87.1 | 35.4 | 68.3 | 79.9
VSE [5] | 52.9 | 79.1 | 87.2 | 39.6 | 69.6 | 79.5
SCO [11] | 55.5 | 82.0 | 89.3 | 41.1 | 70.5 | 80.1
Faster R-CNN | SCAN [18] | 67.4 | 90.3 | 95.8 | 48.6 | 77.7 | 85.2
VSRN [19] | 71.3 | 90.6 | 96.0 | 54.7 | 81.8 | 88.2
| Ours | 82.8 | 95.9 | 97.9 | 62.2 | 89.3 | 93.8

Table 1: Comparison of the image-text matching on Flickr30k Dataset.

Networks | Methods | Caption Retrieval | Image Retrieval
---|---|---|---
R@1 | R@5 | R@10 | R@1 | R@5 | R@10
AlexNet | DSVA [14] | 38.4 | 69.9 | 80.5 | 27.4 | 60.2 | 74.8
HMlstm [25] | 43.9 | - | 87.8 | 36.1 | - | 86.7
VGG | FV [16] | 39.4 | 67.9 | 80.9 | 25.1 | 59.8 | 76.6
VQA [22] | 50.5 | 80.1 | 89.7 | 37.0 | 70.9 | 82.9
SMlstm [10] | 53.2 | 83.1 | 91.5 | 40.7 | 75.8 | 87.4
2wayN [4] | 55.8 | 75.2 | - | 39.7 | 63.3 | -
ResNet | RRF [24] | 56.4 | 85.3 | 91.5 | 43.9 | 78.1 | 88.6
VSE [5] | 64.6 | 89.1 | 95.7 | 52.0 | 83.1 | 92.0
GXN [8] | 68.5 | - | 97.9 | 56.6 | - | 94.5
| SCO [11] | 69.9 | 92.9 | 97.5 | 56.7 | 87.5 | 94.3
Faster R-CNN | SCAN [18] | 72.7 | 94.8 | 98.4 | 58.8 | 88.4 | 94.8
VSRN [19] | 76.2 | 94.8 | 98.2 | 62.8 | 89.7 | 95.1
Ours | 84.0 | 95.8 | 97.8 | 63.9 | 88.9 | 95.6

Table 2: Comparison of the image-text matching on MS-COCO Dataset of 1K test
set.

Networks | Methods | Caption Retrieval | Image Retrieval
---|---|---|---
R@1 | R@5 | R@10 | R@1 | R@5 | R@10
AlexNet | DSVA [14] | 11.8 | 32.5 | 45.4 | 8.9 | 24.9 | 36.3
VGG | FV [16] | 17.3 | 39.0 | 50.2 | 10.8 | 28.3 | 40.1
VQA [22] | 23.5 | 50.7 | 63.6 | 16.7 | 40.5 | 53.8
OEM [10] | 23.3 | - | 84.7 | 31.7 | - | 74.6
ResNet | VSE [5] | 41.3 | 69.2 | 81.2 | 30.3 | 59.1 | 72.4
GXN [8] | 42.0 | - | 84.7 | 31.7 | - | 74.6
SCO [11] | 42.8 | 72.3 | 83.0 | 33.1 | 62.9 | 75.5
Faster R-CNN | SCAN [18] | 50.4 | 82.2 | 90.0 | 38.6 | 69.3 | 80.4
VSRN [19] | 53.0 | 81.1 | 89.4 | 40.5 | 70.6 | 81.1
| Ours | 68.7 | 88.7 | 93.0 | 46.2 | 77.8 | 85.5

Table 3: Comparison of the image-text matching on MS-COCO Dataset of 5K test
set.

Methods | Caption Retrieval | Image Retrieval
---|---|---
R@1 | R@5 | R@10 | R@1 | R@5 | R@10
Triplet Loss | 68.2 | 88.8 | 93.6 | 52.6 | 75.5 | 86.0
Triplet + Instance | 69.3 | 89.5 | 93.5 | 52.1 | 77.8 | 87.8
Triplet + Instance + Text (Baseline) | 70.9 | 89.0 | 93.5 | 52.2 | 78.1 | 87.2
Baseline + Discrete PG | 78.0 | 93.4 | 94.6 | 56.0 | 80.6 | 89.1
Continuous PG | 76.8 | 90.2 | 91.6 | 54.3 | 78.4 | 87.4
Multi-head Continuous PG | 78.1 | 91.6 | 92.0 | 56.2 | 79.9 | 89.2
Baseline + Our PG (Our scheme) | 81.0 | 94.5 | 97.1 | 60.6 | 86.5 | 92.4
Our Scheme + Reward R@1 | 79.3 | 94.5 | 96.0 | 57.8 | 82.3 | 90.2
Our Scheme + Reward AP | 80.4 | 95.2 | 96.8 | 60.8 | 84.9 | 92.7
Our Scheme + Reward R@1+AP | 81.0 | 94.5 | 97.1 | 60.6 | 86.5 | 92.4
Our scheme + ($\lambda=10$) | 80.2 | 95.3 | 96.9 | 58.4 | 82.2 | 90.6
Our scheme + ($\lambda=20$) | 81.0 | 94.5 | 97.1 | 60.6 | 86.5 | 92.4
Our scheme + ($\lambda=30$) | 79.2 | 94.7 | 96.9 | 57.1 | 83.0 | 90.3
Our scheme (+PG baseline) | 80.3 | 94.6 | 97.5 | 60.8 | 86.9 | 92.4
Our scheme (+multi-head) | 81.4 | 94.9 | 97.7 | 61.2 | 87.5 | 92.6
Our scheme (+GloVe Embedding) | 82.8 | 95.9 | 97.9 | 62.2 | 89.3 | 93.8

Table 4: Ablation Studies on Fickr30k Dataset. Figure 3: Visualisation of the
caption retrieval results and attention mechanism. We select top 3 retrieval
results where a ✓means the retrieval is correct whilst the $\times$ indicates
a wrong result. Best Viewed in Colour. Figure 4: Visualisation of the image
retrieval results and attention mechanism. Best Viewed in Colour.

Figure 5: The Instance Loss, Triplet (Retrieval) Loss, Text Decoding Loss and
Reward curves are shown in the figures.

##  4 Experiments

To evaluate the effectiveness of the proposed Discrete-continuous PG
algorithm, we follow previous research and perform two kinds of experiments
which include sentence retrieval using image and image retrieval using a
sentence.

###  4.1 Datasets and Protocols

We evaluate the performance of our method on the Flickr30K [36] and Microsoft-
COCO datasets [21]. Flickr30K contains 31,783 images. Each image corresponds
to 5 human-annotated text descriptions. We use the standard training,
validation and testing split [14], which consist of 28,000 images, 1, 000
images and 1, 000 images, respectively. We follow the splits of [5, 8, 14, 18]
for MS-COCO dataset, which includes 113, 287 images for training, 5, 000
images for validation and 5, 000 images for testing. Each image has five
captions. We use the same evaluation protocol as the previous research [5, 8,
18, 19], which is the recall performance at K (R@K) defined as the proportion
of queries for which the correct item is retrieved in the nearest K samples to
each query.

###  4.2 Implementation Details

We build our model based on PyTorch [26]. We use the pre-trained bottom-up
attention image features provided by [19]. The word embedding size is 300 and
the dimension of the image and text embedding is 2048. The hidden size of the
GRU modules used in our model is 2048. We pre-define 100 discrete action
categories which are $\\{0,1,2,...,a_{i},...100\\}$ , where $a_{i}$
corresponds an action of enlarging the features with a value of
$a_{i}/\lambda$, where $\lambda$ is a hyper-parameter. Note the choice of the
number of action categories is mostly empirical. We choose 100 as it is close
to the maximum number of regions of an image, and also close to the maximum
number of words of a sentence, which is powerful enough to describe the
difference between each item of the image regions and the sentence. The
detailed explanation is presented in Equation 11 and 12. For training, we
apply Adam optimiser [15] to train the model with 30 epochs with a mini-batch
size of 128. We start the training with a learning rate of 4e-4 for 15 epochs
and lower the learning rate to 4e-5 for another 15 epochs. We apply the early
stopping tricks to select the model which performs the best in the validation
set. For the cross-modal Triplet ranking loss, the margin is set 0.2 for all
the experiments. For the classification loss, there are 29, 783 categories for
Flickr30K dataset and 113, 287 categories for MS-COCO dataset. We perform all
the experiments on a server equipped with an Nvidia Geforce 2080-TI GPU card,
and a Windows 10 operating system.

###  4.3 Comparison with the State-of-the-art

#### Results on Flickr30k.

We show the results on the Flickr30k dataset and comparison with the current
state-of-the-art methods in Table 1. We also indicate the backbone networks
that used for each of the state-of-the-art methods, such as AlexNet [17], VGG
[29], ResNet [9], Faster R-CNN [28]. The proposed method outperforms other
approaches by a large margin. SCAN [18] and VSRN [19] are two approaches that
close to ours. Our method is different from them mainly on the proposed PG-
based supervised feature attention mechanism as both VSRN and our method use
the same cross-modal Triplet loss and the Text Decoding Loss. Hence, the main
performance gain is from the proposed Discrete-continuous PG algorithm, which
is effective in improving the existing baseline model that is similar to the
VSRN model [19]. Specifically, we achieve 82.8% R@1 in captioning retrieval
using the image, and 62.2% R@1 image retrieval using the caption.

#### Results on MS-COCO.

We present the experimental results on the 1K and 5K MS-COCO dataset and
comparison with the state-of-the-art models in Table 2 and Table 3,
respectively. For the 1K testing protocol, the results are obtained by
averaging over 5 folds of 1K test images. When comparing with the current best
method SCAN [18] and VSRN [19], we follow the same strategy to combine results
from two trained proposed models by averaging their predicted similarity
scores. As shown in Table 2, our proposed model achieves 84.0% R@1 on caption
retrieval using an image, and 63.9 % R@1 on image retrieval using the caption,
respectively. The results outperform the VSRN and SCAN by a large margin. For
the 5K testing protocol, we evaluate the proposed model by using the whole 5K
testing samples. From Table 3, it is obvious that our method achieves the new
state-of-the-art, with 68.7% R@1 and on 46.2% R@1 on caption retrieval using
image and image retrieval using the caption, respectively.

###  4.4 Ablation Studies

#### Baseline.

We perform ablation studies on each component of the proposed model. which are
shown in Table 4. We first evaluate the model with only Triplet Loss, with
relatively poor results. Adding an Instance Loss to the model brings an
limited increase in the ranking results. Similarly, the Text Decoding Loss
also improves the performance of the model, which proves that it is helpful to
narrow the domain gap between different modalities. Our baseline model include
all of the three Loss functions.

#### The Impact of the Discrete-continuous PG Method.

Based on the baseline model, to validate the superiority of the proposed
Discrete-continuous action space policy gradient algorithm, we first compare
it with the conventional discrete action space policy gradient scheme. To
realise the Discrete PG scheme, we remove the continuous action space sampling
and utilise the discrete action directly as the attention weights. The
proposed method yields better results than the Discrete PG scheme. Second, we
solely apply a single Gaussian-based continuous action space PG scheme. The
results of our scheme is also better than the single Gaussian PG as we form a
complex distribution which better describe the real distribution of the action
space, the results are shown in Table 4.

#### The Impact of Different Reward Function.

We then perform ablation studies on the reward function, the results show that
using the batch-wise R@1 combined the instance-wise AP as the reward has the
best performance. Note that AP alone is better than R@1 reward, as the AP
evaluation is more comprehensive and instance-wise reward is more accurate
than the batch-wise one. To further reduce the variance and make the PG
training more stable, we additionally apply a PG baseline. The impact of the
PG baseline is evaluated subsequently, which yields a slightly better
performance as the PG baseline can stabilise the training and reduce the
variance of this on-line PG method.

#### The Impact of the Different Values of $\lambda$.

We evaluate the proposed method which largely improves the performance in our
ablation studies, with more than 5% increase on the R@1 metric of both the
image and caption retrieval. The value of $\lambda$ controls the scale of the
attention weights, which is with significant importance. The ablation studies
show that a suitable value of $\lambda$ (20) is critical in maintaining good
performance, though our method with different $\lambda$ is all with superior
results.

#### The Impact of Applying a Multi-head Mechanism.

Multi-head Mechanism is widely applied in well-known models like Transformer,
often with extra improvement. We validate the positive effect of this multi-
head mechanism on the proposed PG algorithm. Specifically, we apply a multi-
head mechanism on the latent discrete $\mu$ and $\sigma$ values with a head
number of 2. Increasing the head number would increase the computing burden,
which is less practical. The empirical results reveal that the multi-head
mechanism can improve the performance, by essentially reflecting different
aspects of the sampled latent distribution.

#### The Impact of Utilising a Pre-trained GloVe Word Embedding.

In the vanilla VSRN baseline, the word embedding module is trainable. We
investigate the impact of a pre-trained GloVe Word Embedding module as shown
in the table. Applying a pre-trained GloVe embedding can improve the matching
performance slightly as it embeds some prior information.

###  4.5 Visualisations

We visualise the retrieval results and attention maps of both the image and
text in Figure 3 and Figure 4. It is clear from the figures that the attention
maps can capture the expected image regions, and the language attention maps
can reflect the important semantics. Some incorrect examples are also provided
in the figures, which have similar semantic contents or have similar visual
layouts. Visualisation on the training loss curves and the reward function
curve are presented in Figure 5. The Triplet loss, Instance Loss and Text
Decoding Loss all decrease as the training is performed. The reward value
increases which validates the proposed Discrete-continuous PG method.

##  5 Conclusions

In this paper, we propose a novel policy gradient-based attention mechanism to
transform the image and text embedding to a common space and optimise them
towards higher AP. To model complex action space in the attention weights
sampling, we propose a Discrete-continuous action space policy gradient
algorithm, with a compound action space distribution. Comprehensive
experiments on two widely-used benchmark datasets validate the effectiveness
of the proposed method, leading to state-of-the-art performance.

##  6 Acknowledgements

This work is supported by the National Natural Science Foundation of China
(61772524, 62002172), the Natural Science Foundation of Shanghai (20ZR1417700)
and CAAI-Huawei MindSpore Open Fund.

## References

  * [1] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang.  Bottom-up and top-down attention for image captioning and visual question answering.  In CVPR, 2018.
  * [2] Jyoti Aneja, Aditya Deshpande, and Alexander Schwing.  Convolutional image captioning.  In CVPR, 2018.
  * [3] Gabriel Dulac-Arnold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, and Ben Coppin.  Deep reinforcement learning in large discrete action spaces.  arXiv preprint arXiv:1512.07679, 2015.
  * [4] Aviv Eisenschtat and Lior Wolf.  Linking image and text with 2-way nets.  In CVPR, 2017.
  * [5] Fartash Faghri, David J Fleet, Jamie Ryan Kiros, and Sanja Fidler.  VSE++: Improving visual-semantic embeddings with hard negatives.  In BMVC, 2018.
  * [6] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc’Aurelio Ranzato, and Tomas Mikolov.  Devise: A deep visual-semantic embedding model.  In NIPS, 2013.
  * [7] Chuang Gan, Yandong Li, Haoxiang Li, Chen Sun, and Boqing Gong.  Vqs: Linking segmentations to questions and answers for supervised attention in vqa and question-focused semantic segmentation.  In ICCV, 2017.
  * [8] Jiuxiang Gu, Jianfei Cai, Shafiq R Joty, Li Niu, and Gang Wang.  Look, imagine and match: Improving textual-visual cross-modal retrieval with generative models.  In CVPR, 2018.
  * [9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  2016\.
  * [10] Yan Huang, Wei Wang, and Liang Wang.  Instance-aware image and sentence matching with selective multimodal lstm.  In CVPR, 2017.
  * [11] Yan Huang, Qi Wu, Chunfeng Song, and Liang Wang.  Learning semantic concepts and order for image and sentence matching.  In CVPR, 2018.
  * [12] Eric Jang, Shixiang Gu, and Ben Poole.  Categorical reparameterization with gumbel-softmax.  In ICLR, 2017.
  * [13] Hidetaka Kamigaito, Katsuhiko Hayashi, Tsutomu Hirao, Hiroya Takamura, Manabu Okumura, and Masaaki Nagata.  Supervised attention for sequence-to-sequence constituency parsing.  In IJCNLP, 2017.
  * [14] Andrej Karpathy and Li Fei-Fei.  Deep visual-semantic alignments for generating image descriptions.  In CVPR, 2015.
  * [15] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In ICLR, 2015.
  * [16] Benjamin Klein, Guy Lev, Gil Sadeh, and Lior Wolf.  Associating neural word embeddings with deep image representations using fisher vectors.  In CVPR, 2015.
  * [17] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Communications of the ACM, 60(6):84–90, 2017.
  * [18] Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He.  Stacked cross attention for image-text matching.  In ECCV, 2018.
  * [19] Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, and Yun Fu.  Visual semantic reasoning for image-text matching.  In ICCV, 2019.
  * [20] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra.  Continuous control with deep reinforcement learning.  In ICLR, 2016.
  * [21] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In ECCV, 2014.
  * [22] Xiao Lin and Devi Parikh.  Leveraging visual question answering for image-caption ranking.  In ECCV, 2016.
  * [23] Lemao Liu, Masao Utiyama, Andrew Finch, and Eiichiro Sumita.  Neural machine translation with supervised attention.  In COLING, 2016.
  * [24] Yu Liu, Yanming Guo, Erwin M Bakker, and Michael S Lew.  Learning a recurrent residual fusion network for multimodal matching.  In ICCV, 2017.
  * [25] Zhenxing Niu, Mo Zhou, Le Wang, Xinbo Gao, and Gang Hua.  Hierarchical multimodal lstm for dense visual-semantic embedding.  In ICCV, 2017.
  * [26] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in pytorch.  2017\.
  * [27] Jason Pazis and Michail G Lagoudakis.  Binary action search for learning continuous-action control policies.  In ICML, 2009.
  * [28] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In NIPS, 2015.
  * [29] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [30] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour.  Policy gradient methods for reinforcement learning with function approximation.  In NIPS, 2000.
  * [31] Yunhao Tang and Shipra Agrawal.  Discretizing continuous action space for on-policy optimization.  In AAAI, 2020.
  * [32] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan.  Show and tell: A neural image caption generator.  In CVPR, 2015.
  * [33] Ronald J Williams.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  Machine learning, 8(3-4):229–256, 1992.
  * [34] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip.  A comprehensive survey on graph neural networks.  IEEE Transactions on Neural Networks and Learning Systems, 2020\.
  * [35] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio.  Show, attend and tell: Neural image caption generation with visual attention.  In ICML, 2015.
  * [36] Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  Transactions of the Association for Computational Linguistics, 2:67–78, 2014.
  * [37] Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, and Yi-Dong Shen.  Dual-path convolutional image-text embeddings with instance loss.  ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 16(2):1–23, 2020.