11institutetext:  Department of Physics, Chungbuk National University,
Cheongju 28644, Republic of Korea
11email<EMAIL_ADDRESS>22institutetext: Department of
Astronomy, The Ohio State University, 140 W. 18th Ave., Columbus, OH 43210,
USA 33institutetext: Max Planck Institute for Astronomy, Königstuhl 17,
D-69117 Heidelberg, Germany 44institutetext: Department of Earth and Space
Science, Graduate School of Science, Osaka University, Toyonaka, Osaka
560-0043, Japan 55institutetext: Korea Astronomy and Space Science Institute,
Daejon 34055,Republic of Korea 66institutetext: University of Canterbury,
Department of Physics and Astronomy, Private Bag 4800, Christchurch 8020, New
Zealand 77institutetext: National Astronomical Observatories, Chinese Academy
of Sciences, Beijing 100101, China 88institutetext: Department of Astronomy,
Tsinghua University, Beijing 100084, China 99institutetext: Department of
Particle Physics and Astrophysics, WeizmannInstitute of Science, Rehovot
76100, Israel 1010institutetext: Center for Astrophysics—Harvard & Smithsonian
60 Garden St., Cambridge, MA 02138, USA 1111institutetext: School of Space
Research, Kyung Hee University, Yongin, Kyeonggi 17104, Republic of Korea
1212institutetext: Institute for Space-Earth Environmental Research, Nagoya
University, Nagoya 464-8601, Japan 1313institutetext: Code 667, NASA Goddard
Space Flight Center, Greenbelt, MD 20771, USA 1414institutetext: Department of
Astronomy, University of Maryland, College Park, MD 20742, USA
1515institutetext: Institute of Natural and Mathematical Sciences, Massey
University, Auckland 0745, New Zealand 1616institutetext: Department of Earth
and Planetary Science, Graduate School of Science, The University of Tokyo,
7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan 1717institutetext: Instituto de
Astrofísica de Canarias, Vía Láctea s/n, E-38205 La Laguna, Tenerife, Spain
1818institutetext: Department of Astronomy, Graduate School of Science, The
University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
1919institutetext: National Astronomical Observatory of Japan, 2-21-1 Osawa,
Mitaka, Tokyo 181-8588, Japan 2020institutetext: Department of Physics,
University of Auckland, Private Bag 92019, Auckland, New Zealand
2121institutetext: Institute of Space and Astronautical Science, Japan
Aerospace Exploration Agency, 3-1-1 Yoshinodai, Chuo, Sagamihara, Kanagawa,
252-5210, Japan 2222institutetext: University of Canterbury Mt. John
Observatory, P.O. Box 56, Lake Tekapo 8770, New Zealand

# KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretations

Cheongho Han KMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy
between triple-lens and higher-order binary-lens interpretations Andrew Gould
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretationsKMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Yuki Hirao KMT-2021-BLG-0322: Severe
degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Chung-Uk Lee
(Leading authors)
Michael D. Albrow KMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy
between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy
between triple-lens and higher-order binary-lens interpretations Sun-Ju Chung
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Kyu-Ha Hwang
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Youn Kil Jung
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Doeon Kim
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Shude Mao
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretationsKMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Yoon-Hyun Ryu KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations In-Gu Shin KMT-2021-BLG-0322: Severe
degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Yossi Shvartzvald  KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Jennifer C. Yee KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Weicheng Zang KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Sang-Mok Cha KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy
between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Dong-Jin Kim KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Hyoun-Woo Kim KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Seung-Lee Kim KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Dong-Joo Lee KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Yongseok Lee KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Byeong-Gon Park KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Richard W. Pogge
(The KMTNet Collaboration) KMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretationsKMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretations
Fumio Abe KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-
order binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Richard Barry
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations David P. Bennett
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretationsKMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Aparna Bhattacharya
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretationsKMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Ian Bond KMT-2021-BLG-0322: Severe
degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Martin Donachie KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Hirosane Fujii KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Akihiko Fukui KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy
between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Yoshitaka Itow KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Rintaro Kirikawa KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Iona Kondo KMT-2021-BLG-0322: Severe
degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Naoki Koshimoto KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy
between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Man Cheung Alex Li KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Yutaka Matsubara KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Yasushi Muraki KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Shota Miyazaki KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Clément Ranc KMT-2021-BLG-0322:
Severe degeneracy between triple-lens and higher-order binary-lens
interpretationsKMT-2021-BLG-0322: Severe degeneracy between triple-lens and
higher-order binary-lens interpretations Nicholas J. Rattenbury
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Yuki Satoh
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Hikaru Shoji
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Takahiro Sumi
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Daisuke Suzuki
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Yuzuru Tanaka
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Paul J. Tristram
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Tsubasa Yamawaki
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations Atsunori Yonehara
(The MOA Collaboration)
KMT-2021-BLG-0322: Severe degeneracy between triple-lens and higher-order
binary-lens interpretationsKMT-2021-BLG-0322: Severe degeneracy between
triple-lens and higher-order binary-lens interpretations

(Received ; accepted)

###### Abstract

Aims.  We investigate the microlensing event KMT-2021-BLG-0322, for which the
light curve exhibits three distinctive sets of caustic-crossing features. It
is found that the overall features of the light curve are approximately
described by a binary-lens (2L1S) model, but the model leaves substantial
residuals. We test various interpretations with the aim of explaining the
residuals.

Methods.  We find that the residuals can be explained either by considering a
nonrectilinear lens-source motion caused by the microlens-parallax and lens-
orbital effects or by adding a low-mass companion to the binary lens (3L1S
model). The degeneracy between the higher-order 2L1S model and the 3L1S model
is very severe, making it difficult to single out a correct solution based on
the photometric data. This degeneracy was known before for two previous events
(MACHO-97-BLG-41 and OGLE-2013-BLG-0723), which led to the false detections of
planets in binary systems, and thus the identification of the degeneracy for
KMT-2021-BLG-0322 illustrates that the degeneracy can be not only common but
also very severe, emphasizing the need to check both interpretations of
deviations from 2L1S models.

Results.  From the Bayesian analysis conducted with the measured lensing
observables of the event timescale, angular Einstein radius, and microlens
parallax, it was estimated that the binary lens components have masses
$(M_{1},M_{2})=(0.62^{+0.25}_{-0.26}\leavevmode\nobreak\
M_{\odot},0.07^{+0.03}_{-0.03}\leavevmode\nobreak\ M_{\odot})$, for both 2L1S
and 3L1S solutions, and the mass of the tertiary lens component according to
the 3L1S solution is $M_{3}=6.40^{+2.64}_{-2.78}\leavevmode\nobreak\ M_{\rm
J}$.

###### Key Words.:

gravitational microlensing – planets and satellites: detection

##  1 Introduction

Searching for planets in binary or multiple systems is of scientific
importance for two major reasons. First, planets in multiple systems act as
test beds for investigating various mechanisms of planet formation and
evolution because the gravity of the stellar companion may influence the
formation and the subsequent dynamical evolution of planets. Second,
systematic searches for planets in multiple systems are important in the
estimation of the global planet frequency because binaries are very common
among field stars. An up-to-date overview of exoplanet statistics and
theoretical implications is available in Zhu & Dong (2021).

Gravitational microlensing is an important tool for detecting planets in
binary systems, especially those belonging to binaries composed of faint
stars, which are the most common population of stars in the Galaxy.
Microlensing detections of planets in such systems are possible because of the
lensing characteristic that does not depend on the luminosity of a lensing
object. There exist six confirmed microlensing planets in binary systems,
including OGLE-2008-BLG-092L (Poleski et al., 2014), OGLE-2007-BLG-349L
(Bennett et al., 2016), OGLE-2013-BLG-0341L (Gould et al., 2014),
OGLE-2016-BLG-0613L (Han et al., 2017), OGLE-2018-BLG-1700L (Han et al.,
2020c), and KMT-2020-BLG-0414L (Zang et al., 2021), and for all of these
systems, the planet hosts are less massive, and thus fainter, than the Sun.
Besides these systems, the lens of the event OGLE-2019-BLG-0304 is likely to
be a planetary system in a binary, but this interpretation is not conclusive
due to the possibility of an alternative interpretation (Han et al., 2021).

Planets in binaries manifest themselves via signals of various types in
lensing light curves. The first type is an independent short single-lensing
(1L1S) light curve that is well separated from the binary-lensing (2L1S) light
curve produced by the host binary stars. The planetary signals in the lensing
events OGLE-2008-BLG-092 and OGLE-2013-BLG-0341 were detected through this
channel. The signal of the second type is generated by the source passage
through the mingled caustic region, in which the two sets of lensing caustics
induced by the planet and binary companion overlap. Here the caustic refers to
the positions on the source plane at which the lensing magnification of a
point source would be infinite. In this case, the planetary signal superposes
with the signal of the binary companion (Lee et al., 2008). The planetary
signals in the lensing events OGLE-2016-BLG-0613 and OGLE-2018-BLG-1700 were
found through this channel. The signal of the third type is a small distortion
from a 2L1S lensing light curve caused by the presence of a tertiary lens
component, either a companion binary star or a planet. Such a signal was
detected in the case of the lensing event OGLE-2007-BLG-349, for which the
overall lensing light curve was approximated as that of a 2L1S event produced
by a star-planet pair, but an additional binary companion to the host was
needed to precisely describe the observed light curve. In addition, although
the major planetary signal of OGLE-2013-BLG-0341 was detected through the
independent channel, the presence of the planet was additionally confirmed by
the deviation of the 2L1S model in the region of the binary-induced anomaly.

Light curves produced by triple-lens (3L1S) systems with planets can be
degenerate with those of 2L1S events deformed by higher-order effects. This
degeneracy was known for two previous events MACHO-97-BLG-41 and
OGLE-2013-BLG-0723. For both events, the light curves were originally
interpreted as 3L1S events by Bennett et al. (1999) and Udalski et al. (2015),
respectively, and it was subsequently shown that the signals of the third body
were spurious and the residuals from the 2L1S models could be explained with
the consideration of the lens orbital motion by Albrow et al. (2000) and Jung
et al. (2013) for MACHO-97-BLG-41 and by Han et al. (2016) for
OGLE-2013-BLG-0723.

In this work, we report the analysis of the lensing event
KMT-2021-BLG-0322/MOA-2021-BLG-091. The light curve of the event exhibits
three distinctive sets of caustic features, for all of which the rising and
falling sides of the caustic crossings were resolved by lensing surveys.
Although the overall features of the lensing light curve are approximately
described by a 2L1S model, the model leaves residuals from the model. We
investigate the origin of the residuals by testing various interpretations of
the lensing system.

For the presentation of the analysis, we organize the paper as follows. In
Sect. 2, we describe observations conducted to acquire the data used in the
analysis. The anomalous features in the observed lensing light curve are
depicted in the same section. In Sect. 3, we conduct modeling of the observed
lensing light curve under two interpretations of the lens system and present
the results of the analysis. In Sect. 4, we describe the detailed procedure of
measuring the lensing observables that can constrain the physical lens
parameters. In Sect. 5, we estimate the physical lens parameters. In Sect. 6,
we summarize the result and conclude.

Figure 1:  Light curve of the lensing event KMT-2021-BLG-0322. The colors of
the data points are set to match those of the telescopes used for the data
acquisition.

##  2 Observations and data

The source star of the lensing event KMT-2021-BLG-0322/MOA-2021-BLG-091 is
located toward the Galactic bulge field at the equatorial coordinates $({\rm
RA},{\rm decl.})=(18:03:38.80,-29:36:15.80)$, which correspond to the galactic
coordinates $(l,b)=(1^{\circ}\hskip-2.0pt.409,-3^{\circ}\hskip-2.0pt.731)$.
The source brightness had remained constant before the lensing magnification
with a baseline magnitude of $I_{\rm base}=18.49$ according to the KMTNet
scale.

The increase in the source flux induced by lensing was first found by the
AlertFinder algorithm (Kim et al., 2018) of the Korea Microlensing Telescope
Network (KMTNet: Kim et al., 2016) survey on 2021-04-09 (${\rm
HJD}^{\prime}\equiv{\rm HJD}-2450000\sim 9313.5$), when the source became
brighter than the baseline by $\Delta I\sim 0.34$ mag. The Microlensing
Observations in Astrophysics (MOA: Bond et al., 2001) group independently
found the event 11 days after the KMTNet discovery and dubbed the event as
MOA-2021-BLG-091. Hereafter, we designate the event as KMT-2021-BLG-0322
following the convention that the event ID of the first discovery survey is
used as a representative designation.

The KMTNet observations of the event were conducted utilizing the three 1.6 m
telescopes located in the three continents of the Southern Hemisphere: the
Siding Spring Observatory in Australia (KMTA), the Cerro Tololo Interamerican
Observatory in Chile (KMTC), and the South African Astronomical Observatory in
South Africa (KMTS). Each KMTNet telescope is equipped with a camera yielding
4 deg2 field of view. The MOA survey used the 1.8 m telescope of the Mt. John
Observatory in New Zealand, and the camera mounted on the telescope yields a
2.2 deg2 field of view. The principal observations of the KMTNet and MOA
surveys were done in the $I$ band and the customized MOA-Red band,
respectively, and for both surveys, a fraction of images were acquired in the
$V$ band to measure the color of the source star. In Sect. 4, we present the
procedure of the source color measurement.

In Figure 1, we present the lensing light curve of the event constructed using
the combined data from the KMTNet and MOA surveys. The light curve is
characterized by five strong peaks that occurred at ${\rm HJD}^{\prime}\sim
9323.1$, 9321.9, 9329.7, 9332.4, and 9334.0. Figure 2 shows the zoom-in view
of the peaks. The U-shape trough region between the first and second peaks
suggests that these peaks are a pair of spikes produced when the source
entered and exited a caustic. A similar pattern between the fourth and fifth
peaks suggests the same origin as those of the first and second peaks. The
third peak does not exhibit a U-shape trough, and this suggests that the peak
was produced by the source passage over the tip (cusp) of a caustic. We note
that all the caustic spikes were resolved by the combination of the KMTNet
data sets, which were acquired with a 1.0 hr cadence for KMTC data set and
with a 0.75 hr cadence for KMTA+KMTS data sets. Although the event was not in
the KMTNet prime fields, which are covered with a 0.25 hr cadence, the cadence
of the KMTNet observations was adequate in part because of relatively long
durations of the caustic crossings. In the analysis, we did not use the MOA
data first because these data did not resolve any of the caustic spikes, and
second because the precision of the data was low compared to the KMTNet data
sets, especially at lower magnifications.

Figure 2:  Zoom-in view of the anomaly region of the lensing light curve. The
inset in the top panel shows the enlarged view around the first peak. The
solid, dotted, and dashed curves drawn over the data points are the model
curves of the 3L1S, higher-order 2L1S, and standard 2L1S solutions,
respectively. The three lower panels show the residuals from the individual
models. The curves of the 3L1S and higher-order 2L1S models are difficult to
be distinguished with the line width.

Data reduction was carried out using the KMTNet photometry pipeline (Albrow,
2017) developed based on the difference imaging algorithm (Tomaney & Crotts,
1996; Alard & Lupton, 1998). Following the standard procedure described in Yee
et al. (2012), we rescaled the error bars of the data estimated from the
pipeline, $\sigma_{0}$, by

$\sigma=k(\sigma_{\rm min}^{2}+\sigma_{0}^{2})^{1/2},$ (1)

where $\sigma_{\rm min}$ is a factor added in the quadrature to take into
account the scatter of data, and the other factor $k$ is used to make
$\chi^{2}$ per degree of freedom for each data set unity. In Table 1, we list
the values of $k$ and $\sigma_{\rm min}$ along with the numbers of data
points, $N_{\rm data}$, for the individual data sets.

For the event, the light curve was analyzed in real time with the progress of
the event. Y. Hirao of the MOA group first released a lensing model when the
data covered the first two peaks, and this model interpreted the event as a
2L1S events produced by a binary lens with a projected separation (normalized
to the angular Einstein radius $\theta_{\rm E}$) and mass ratio of
$(s,q)\sim(1.1,0.1)$. C. Han of the KMTNet group conducted modeling after the
third peak was covered, and found a model that is similar to that of Y. Hirao.
From the additional modeling conducted with updated data after the final peak
was covered, C. Han realized that a 2L1S model under the assumption of a
rectilinear relative lens-source motion (standard 2L1S model) could not
precisely describe all the caustic features, although the model described the
overall pattern of the light curve. In the following section, we depict in
detail the inadequacy of the standard 2L1S model in describing the observed
data and investigate the origin of this deviation.

##  3 Interpretations of lensing light curve

###  3.1 2L1S interpretation

The caustic features in the lensing light curve suggest that the event was
produced by a lens composed of multiple masses. We, therefore, start with a
standard 2L1S model for the interpretation of the observed lensing light
curve. The modeling is done by searching for the set of the lensing parameters
that best describe the observed light curve. For a 1L1S event, the lensing
light curve is described by three parameters of $(t_{0},u_{0},t_{\rm E})$,
which are the time of the closest lens-source approach, the separation at that
time, and the event timescale, respectively. Describing a caustic-crossing
2L1S light curve requires four additional parameters of $(s,q,\alpha,\rho)$,
which denote the separation and mass ratio between the binary lens components,
$M_{1}$ and $M_{2}$, the angle between the source trajectory and the line
connecting the binary lens components, and the ratio of the angular source
radius $\theta_{*}$ to the angular Einstein radius, that is,
$\rho=\theta_{*}/\theta_{\rm E}$ (normalized source radius), respectively. The
normalized source radius is included in modeling to account for finite-source
effects during the caustic crossings of the source.

Table 1: Error bar normalization factors Data set | $k$ |  $\sigma_{\rm min}$ (mag) | $N_{\rm data}$
---|---|---|---
KMTA | 1.135 | 0.02 | 121
KMTC | 1.000 | 0.02 | 345
KMTS | 1.072 | 0.02 | 178

The 2L1S modeling is conducted in two steps. In the first step, we divide the
lensing parameters into two groups, and the parameters $s$ and $q$ are
searched for via a grid approach, while the other parameters are found via a
downhill approach. For the downhill approach, we use the Markov Chain Monte
Carlo (MCMC) algorithm. For the MCMC parameters $(t_{0},u_{0},t_{\rm E})$, the
initial values are given based on the peak time, magnification, and duration
of the event. For the source trajectory angle $\alpha$, modeling is done with
multiple starting values that are evenly divided in the 0–$2\pi$ range. From
the modeling in this step, we obtain a $\Delta\chi^{2}$ map on the $s$–$q$
parameter plane, and identify local solutions. In the second step, we refine
the local solutions identified in the first-round modeling by allowing all
lensing parameters to vary. For the computations of finite-source
magnifications, we use the map-making method described in Dong et al. (2006).

In Figure 2, we present the 2L1S model curve (dashed curve plotted over the
data points). The lensing parameters of the model (standard 2L1S model) are
listed in Table 2. The binary lensing parameters $(s,q)\sim(1.1,0.14)$ are
similar to those of the Hirao model obtained from the modeling conducted in
the early stage of the event. The lens system configuration is shown in Figure
3, in which the caustic and the source trajectory are drawn in gray. The
coordinates of the configuration are scaled to $\theta_{\rm E}$ corresponding
to the total mass of the binary lens, and the origin of the coordinates is set
at the photocenter defined by Di Stefano & Mao (1996) and An & Han (2002).
According to the best-fit 2L1S model, the caustic forms a single closed curve
with six cusps (resonant caustic), and the source passed the left side of the
caustic, passing through the caustic three times, with the individual passages
producing the caustic-crossing features in the lensing light curve. Due to the
resonant nature of the caustic, the solution is unique without any degeneracy.

Table 2: Best-fit lensing parameters of 2L1S solution

Parameter | Standard | Higher-order
---|---|---
$\chi^{2}$ | 3049.1 | 632.3
$t_{0}$ (HJD′) |  9328.618 $\pm$ 0.001 |  9328.546 $\pm$ 0.009
$u_{0}$ |  -0.069 $\pm$ 0.001 |  -0.086 $\pm$ 0.002
$t_{\rm E}$ (days) |  21.19 $\pm$ 0.26 |  25.27 $\pm$ 0.61
$s$ |  1.135 $\pm$ 0.002 |  1.148 $\pm$ 0.005
$q$ |  0.143 $\pm$ 0.002 |  0.115 $\pm$ 0.003
$\alpha$ (rad) |  4.787 $\pm$ 0.001 |  4.782 $\pm$ 0.008
$\rho$ (10-3) |  3.306 $\pm$ 0.060 |  2.683 $\pm$ 0.073
$\pi_{{\rm E},N}$ | – |  -2.31 $\pm$ 2.05
$\pi_{{\rm E},E}$ | – |  -0.28 $\pm$ 0.18
$ds/dt$ (yr-1) | – |  0.93 $\pm$ 0.04
$d\alpha/dt$ (yr-1) | – |  3.57 $\pm$ 2.12

111 ${\rm HJD}^{\prime}={\rm HJD}-2450000$.

It is found that the standard 2L1S model cannot precisely explain the observed
light curve. This can be seen in the bottom panels of Figure 2, in which we
present the residuals from the 2L1S model. Although the model depicts the
overall features of the light curve, there is an offset in the time of the
first peak between the model and observed data (see the zoom-in view around
the fist caustic crossing shown in the inset inserted in the upper panel of
Figuree 2). Besides the region of this peak, there exist noticeable residuals
throughout the region of the caustic features. This indicates that the
standard 2L1S interpretation is not adequate and a more sophisticated model is
needed for the precise description of the data.

We checked the possibility that the major deviation from the 2L1S model, that
is, the offset between the model and data in the region of the first peak, can
be explained by higher-order effects. We considered two higher-order effects
that cause the relative lens-source motion to be nonrectilinear: microlens-
parallax and lens-orbital effects. The former effect is caused by the
nonlinear motion of an observer caused by the Earth’s orbital motion around
the Sun (Gould, 1992), and the latter effect is caused by the orbital motion
of the binary lens (Dominik, 1998; Ioka et al., 1999). In order to check these
higher-order effects, we conduct additional modeling by adding two extra pairs
of parameters $(\pi_{{\rm E},N},\pi_{{\rm E},E})$ and $(ds/dt,d\alpha/dt)$:
higher-order 2L1S model. The first pair of the higher-order lensing parameters
represent the north and east components of the microlens-parallax vector
$\mbox{\boldmath$\pi$}_{\rm E}$,

$\mbox{\boldmath$\pi$}_{\rm E}\equiv{\pi_{\rm rel}\over\theta_{\rm
E}}\,{\mbox{\boldmath$\mu$}_{\rm rel}\over\mu_{\rm rel}},$ (2)

respectively, and the second pair represent the annual change rates of the
binary separation and source trajectory angle, respectively. Here
$\mbox{\boldmath$\mu$}_{\rm rel}$ represents the relative lens-source proper
motion, $\pi_{\rm rel}={\rm AU}(D_{\rm S}^{-1}-D_{\rm L}^{-1})$ is the
relative lens-source parallax, and $D_{\rm L}$ and $D_{\rm S}$ denote the
distances to the lens and source, respectively. In this modeling, we restrict
the lensing parameters to satisfy the dynamical condition of $({\rm
KE/PE})_{\perp}\leq{\rm KE/PE}\leq 1.0$, where $({\rm KE/PE})$ and $({\rm
KE/PE})_{\perp}$ represent the intrinsic and projected kinetic-to-potential
energy ratio, respectively (Dong et al., 2009).

Figure 3:  Lens system configuration according to the 2L1S model. The concave
curve represents the caustic, the line with an arrow is the source trajectory,
and the filled dots marked by $M_{1}$ and $M_{2}$ indicate the positions of
the lens components. The caustic of the standard model are drawn in gray. For
the higher-order model, in which the lens position and caustic vary in time
because of the lens orbital motion, we mark the lens and caustic at two epochs
of ${\rm HJD}^{\prime}=9323$ (marked in blue color) and 9334 (in red color).
The source trajectories of the standard and higher-order models are drawn in
gray and black. The inset shows the zoom-in view of the region around $M_{1}$.
The coordinates are centered at the photocenter and lengths are scaled to the
angular Einstein radius corresponding to the total mass of the lens. The
dashed unit circle centered at the origin represents the Einstein ring.

We find that the residuals from the standard 2L1S model are greatly reduced
with the consideration of the higher-order effects. This is shown in Figure 2,
in which we present the model curve (dotted curve) and residuals from the
model. It is found that the higher-order model explains not only the major
deviation in the offset of the first peak but also improves the fit throughout
the anomaly region. The $\chi^{2}$ difference between the standard and higher-
order models is $\Delta\chi^{2}=2416.8$. The lensing parameters of the higher-
order 2L1S model are listed in Table 2. The lens system configuration of the
higher-order 2L1S model is presented in Figure 3, which shows that the caustic
shape varies in time due to the lens-orbital effect and that the source
trajectory becomes nonrectilinear due to the microlens-parallax effect. Figure
4 shows the $\Delta\chi^{2}$ distribution of points in the MCMC chain on the
planes of higher-order parameters. It shows that these parameters are strongly
correlated because the microlens-parallax and lens-orbital effects induce
similar deviations in lensing light curves (Batista et al., 2011). As a
result, the uncertainties of the higher-order lensing parameters are
considerable, for example, $\pi_{{\rm E},N}=-2.31\pm 2.05$ and
$d\alpha/dt=(3.57\pm 2.12)$ yr-1, despite the obvious effects on the lensing
light curve. Also shown in Figure 4 is the distribution of the projected
kinetic-to-potential energy ratio. The mean and standard deviation of the
ratio distribution are $({\rm KE/PE})_{\perp}=0.46\pm 0.25$, which is well
within the range of $0.2\lesssim({\rm KE/PE})_{\perp}\lesssim 0.5$ for
moderate eccentricity binaries that are observed at usual viewing angles.

Figure 4:  $\Delta\chi^{2}$ distribution of points in the MCMC chain of the 2L1S solution on the planes of higher-order parameters: $\pi_{{\rm E},N}$, $\pi_{{\rm E},E}$, $ds/dt$, and $d\alpha/dt$. Points with different colors indicate those with $\Delta\chi^{2}\leq 1\sigma$ (red), $\leq 2\sigma$ (yellow), $\leq 3\sigma$ (green), $\leq 4\sigma$ (cyan), and $\leq 5\sigma$ (blue). The upper right panel shows the distribution of the projected kinetic-to-potential energy ratio.  Table 3: Best-fit lensing parameters of 3L1S solutions Parameter | Wide | Close
---|---|---
| standard | parallax | standard | parallax
$\chi^{2}$ | 642.1 | 630.5 | 665.2 | 642.3
$t_{0}$ (HJD′) |  9328.686 $\pm$ 0.003 |  9328.682 $\pm$ 0.005 |  9328.661 $\pm$ 0.002 |  9328.674 $\pm$ 0.006
$u_{0}$ |  -0.053 $\pm$ 0.001 |  -0.050 $\pm$ 0.002 |  -0.057 $\pm$ 0.001 |  -0.053 $\pm$ 0.002
$t_{\rm E}$ (days) |  25.04 $\pm$ 0.50 |  26.37 $\pm$ 0.62 |  24.99 $\pm$ 0.53 |  25.88 $\pm$ 0.54
$s_{2}$ |  1.124 $\pm$ 0.003 |  1.135 $\pm$ 0.006 |  1.129 $\pm$ 0.003 |  1.139 $\pm$ 0.006
$q_{2}$ |  0.109 $\pm$ 0.003 |  0.104 $\pm$ 0.003 |  0.115 $\pm$ 0.002 |  0.109 $\pm$ 0.004
$\alpha$ (rad) |  4.762 $\pm$ 0.001 |  4.765 $\pm$ 0.002 |  4.768 $\pm$ 0.001 |  4.766 $\pm$ 0.003
$s_{3}$ |  1.231 $\pm$ 0.029 |  1.239 $\pm$ 0.040 |  0.788 $\pm$ 0.009 |  0.711 $\pm$ 0.011
$q_{3}$ ($10^{-3}$) |  9.88 $\pm$ 0.90 |  9.75 $\pm$ 1.32 |  3.20 $\pm$ 0.20 |  5.97 $\pm$ 0.64
$\psi$ (rad) |  0.321 $\pm$ 0.017 |  0.295 $\pm$ 0.022 |  0.519 $\pm$ 0.003 |  0.431 $\pm$ 0.009
$\rho$ (10-3) |  2.76 $\pm$ 0.07 |  2.66 $\pm$ 0.08 |  2.76 $\pm$ 0.07 |  2.68 $\pm$ 0.08
$\pi_{{\rm E},N}$ | – |  -0.06 $\pm$ 0.91 | – |  -0.14 $\pm$ 1.20
$\pi_{{\rm E},E}$ | – |  -0.30 $\pm$ 0.091 | – |  -0.43 $\pm$ 0.093

###  3.2 3L1S interpretation

We also test a 3L1S modeling for the interpretation of the residuals from the
standard 2L1S model. In addition to the 2L1S lensing parameters, a 3L1S
modeling requires extra parameters in order to describe the third lens
component, $M_{3}$. These parameters are $(s_{3},q_{3},\psi)$, which denote
the projected separation and mass ratio between $M_{1}$ and $M_{3}$, and the
orientation angle of $M_{3}$ as measured from the $M_{1}$–$M_{2}$ axis with a
center at the position of $M_{1}$. Hereafter, we denote the separation and
mass ratio between $M_{1}$ and $M_{2}$ as $(s_{2},q_{2})$ to distinguish them
from those of the third body. We check a 3L1S model because it is known that a
third body of a lens can induce a small distortion of the caustic,
particularly in the neighborhood of a cusp, and this may explain the
deviations from the standard 2L1S model. For example, Gould et al. (2014)
showed that the three parameters $(s_{3},q_{3},\psi)$ of the planetary
companion in the 3L1S event OGLE-2013-BLG-0341 could be fully recovered even
when the data that were affected by the planetary caustic were eliminated from
the fit, due to the distortion induced by the planet on a cusp associated with
the binary-lens caustic. In particular, they argued that planets could be
discovered from such cusp distortions even when the source did not pass near
or over the planetary caustic.

Figure 5:  $\Delta\chi^{2}$ map on the $\log s_{3}$–$\log q_{3}$ plane
obtained from the grid search. Color coding is set to represent points with
$\Delta\chi^{2}\leq 1n\sigma$ (red), $\leq 2n\sigma$ (yellow), $\leq 3n\sigma$
(green), $\leq 4n\sigma$ (cyan), $\leq 5n\sigma$ (blue), $\leq 6n\sigma$
(purple), where $n=3$.

Similar to the 2L1S modeling, the 3L1S modeling is carried out in two steps.
In the first step, we conduct grid searches for the triple-lens parameters,
that is, $(s_{3},q_{3},\psi)$, by fixing the 2L1S parameters as those of the
best-fit standard 2L1S model. We fix the 2L1S parameters because the 2L1S
model describes the overall pattern of the light curve and the variation in
the lensing light curve by the third body would be minor (Bozza, 1999; Han et
al., 2001). This step yields $\Delta\chi^{2}$ maps on the
$s_{3}$–$q_{3}$-$\psi$ planes, and we identify local solutions from the maps.
In the second step, the individual local solutions found from the first step
are refined by allowing all parameters to vary.

Figure 6:  Lens system configurations according to the wide (upper panel) and
close (lower panel) 3L1S solutions. In each panel, the nested concave curve
represents the caustic, the line with an arrow is the source trajectory, and
the three filled red dots marked by $M_{1}$, $M_{2}$, and $M_{3}$ indicate the
positions of the lens components. The caustic curve and the source trajectory
according to the static 2L1S model are drawn in gray to show the caustic
variation by the third body $M_{3}$. Other notations are same as those in Fig.
3.

Figure 5 shows the $\Delta\chi^{2}$ map on the $\log s_{3}$–$\log q_{3}$ plane
obtained from the grid search. The map shows two locals at $(\log s_{3},\log
q_{3})\sim(0.1,-2.0)$ (wide solution) and $\sim(-0.1,-2.0)$ (close solution).
The lensing parameters of the two solutions obtained by refining these locals
are listed in Table 3 (standard model) along with the $\chi^{2}$ values of the
fits for the individual solutions. The $M_{1}$–$M_{3}$ separations of the two
solutions are approximately in the relation of $s_{3,{\rm w}}\times s_{3,{\rm
c}}\sim 1.0$, indicating that the degeneracy between the two solutions is
caused by the close-wide degeneracy (Griest & Safizadeh, 1998; Dominik, 1999).
Here $s_{3,{\rm w}}$ and $s_{3,{\rm c}}$ denote the $M_{1}$–$M_{3}$
separations of the solutions with $s_{3}>1.0$ and $s_{3}<1.0$, respectively.
It is found that the wide solution provides a better fit than the close
solution with $\Delta\chi^{2}=23.1$, which is significant enough to resolve
the degeneracy between the solutions. According to the wide 3L1S solution, the
mass ratio between $M_{1}$ and $M_{3}$, $q_{3}=9.9\times 10^{-3}$, is very
small, indicating that the third body is a planet-mass object. The projected
separations of $M_{2}$ and $M_{3}$ from $M_{1}$ for the (wide) 3L1S solution,
$s_{2}\sim 1.12$ and $s_{3}\sim 1.23$, are similar to each other. According to
Equation (1) of Holman & Wiegert (1999), the maximum ratio between
$M_{3}$–$M_{1}$ and $M_{2}$–$M_{1}$ separations for the dynamical stability of
the planet is $\sim 0.42$ assuming a circular planet orbit. The ratio
$s_{3}/s_{2}\sim 1.1$ is substantially greater than this critical ratio. Then,
$M_{2}$ should lie at a large separation along the line of sight in front of
or at the back of $M_{1}$ in order for the planetary system to be dynamically
stable.

The model curves of the wide 3L1S model (solid curve) and the residuals from
the model are shown in Figure 2. It is found that the 3L1S model well
describes all the anomaly features, significantly improving the fit, by
$\Delta\chi^{2}=2407$, with respect to the standard 2L1S model. The fit
improvement occurs not only around the region of the first peak, for which the
2L1S model exhibited a time offset between the model and data, but also
throughout the whole anomaly region.

The lens system configuration of the 3L1S solutions is shown in Figure 6:
upper panel for the wide solution and lower panel for the close solution. The
positions of the individual lens components are represented by red filled
dots, marked by $M_{1}$, $M_{2}$, and $M_{3}$. From the comparison of the
caustic with the 2L1S caustic, drawn in gray, it is found that the tertiary
lens component has two effects on the caustic. First, $M_{3}$ induces a new
set of caustic at around the source position of $(x_{\rm s},y_{\rm
s})\sim(0.35,-0.1)$ for the wide solution and $\sim(-0.3,-0.3)$ for the close
solution, and this makes the caustic nested and self-intersecting. Second,
$M_{3}$ additionally causes a slight distortion of the caustic from that of
the 2L1S solution. The source did not pass the region near the additional
caustic structures induced by $M_{3}$. Nevertheless, the light curve deviates
from the 2L1S form due to the distortion of the caustic, and this explains the
residuals from the standard 2L1S model. If the 3L1S interpretation is correct,
KMT-2021-BLG-0322 is the third case in which a planet belonging to a binary is
detected through the signal from the caustic distortion induced by a tertiary
lens component after the cases of OGLE-2007-BLG-349 and OGLE-2013-BLG-0341.

Figure 7:  Cumulative distribution of $\chi^{2}$ difference between higher-
order 2L1S and 3L1S models. The light curve in the upper panel is inserted to
show the region of $\chi^{2}$ difference.

###  3.3 2L1S versus 3L1S interpretations

The analyses conducted in the previous subsections show that the deviations
from a standard 2L1S model in the lensing light curve of KMT-2021-BLG-0322 can
be explained almost equally well by considering the higher-order effects or by
introducing a low-mass tertiary lens component. The degeneracy between the two
models can be seen by comparing the residuals from the two models presented in
Figure 2. In Figure 7, we present the cumulative distribution of
$\Delta\chi^{2}=\chi^{2}_{\rm 2L1S}-\chi^{2}_{\rm 3L1S}$ to better show the
subtle difference between the fits of the two models. The distribution shows
that $\Delta\chi^{2}\lesssim 2$ throughout the major anomaly region,
indicating that the degeneracy between the two models is very severe.

The degeneracy between a higher-order 2L1S model and a triple-lens model for
KMT-2021-BLG-0322 is very similar to the degeneracies identified in the two
previous lensing events MACHO-97-BLG-41 and OGLE-2013-BLG-0723, for which the
residuals from the standard 2L1S models were initially interpreted as signals
of a planetary-mass companion to a binary lens but later explained by the
higher-order effects of the 2L1S models. For these previous events, the
degeneracies could be resolved because the 2L1S models yielded better fits
than the corresponding 3L1S models. For KMT-2021-BLG-0322, on the other hand,
the degeneracy between the two interpretations is so severe that it cannot be
resolved based on only the photometry data. Therefore, KMT-2021-BLG-0322,
together with the two previous events, demonstrates that the degeneracy can be
not only common but also very severe.

The degeneracy also calls into question the idea from Gould et al. (2014) that
it would be possible to detect planets from the distortions that they induce
on the cusps. This may be possible in some cases, but at least in this case,
if the planet was the real cause of the light curve discrepancy, it could not
be distinguished from a nonrectilinear motion. This emphasizes the need to
check both interpretations when such deviations are detected in lensing light
curves.

##  4 Lensing observables

Because it is difficult to single out a model for the observed data, we
estimate two sets of the physical lens parameters based on the observables of
the two possible interpretations of the event. The mass $M$ and distance
$D_{\rm L}$ to the lens can be constrained by measuring lensing observables.
The event timescale $t_{\rm E}$ is the first such an observable, and it is
related to $M$ and $D_{\rm L}$ by

$t_{\rm E}={\theta_{\rm E}\over\mu};\qquad\theta_{\rm E}=(\kappa M\pi_{\rm
rel})^{1/2},$ (3)

where $\kappa=4G/(c^{2}{\rm AU})$. The other two observables are the microlens
parallax $\pi_{\rm E}$ and angular Einstein radius $\theta_{\rm E}$, with
which $M$ and $D_{\rm L}$ are uniquely determined by

$M={\theta_{\rm E}\over\kappa\pi_{\rm E}};\qquad D_{\rm L}={{\rm
AU}\over\pi_{\rm E}\theta_{\rm E}+\pi_{\rm S}},$ (4)

where $\pi_{\rm S}={\rm AU}/D_{\rm S}$ is the parallax to the source (Gould,
2000).

Figure 8:  $\Delta\chi^{2}$ distribution of points in the MCMC chain for the
higher-order 2L1S (left panel) and 3L1S wide (right panel) solutions plotted
on the $\pi_{{\rm E},E}$–$\pi_{{\rm E},N}$ plane. Color coding is same as that
of Fig. 4.

The microlens parallax is measured from the slight deviation in the lensing
light curve caused by the orbital motion of Earth. The microlens-parallax
parameters of the 2L1S model are listed in Table 2. For the measurements of
the parallax parameters corresponding to the 3L1S model, we conduct an
additional modeling by considering the microlens-parallax effect, and the
lensing parameters of the model including $(\pi_{{\rm E},N},\pi_{{\rm E},E})$
are listed in Table 3. It is found that the consideration of the microlens-
parallax effect improves the 3L1S fit by $\Delta\chi^{2}=11.6$ with respect to
the standard model. The left and right panels of Figure 8 show the
$\Delta\chi^{2}$ plots of MCMC points on the $\pi_{{\rm E},E}$–$\pi_{{\rm
E},N}$ plane for the 2L1S and 3L1S solutions, respectively. For both models,
it is found that the uncertainty of the north component of the microlens-
parallax vector is large, while the east component is relatively well
constrained.

The angular Einstein radius is measured by analyzing the deviations of the
lensing light curve from a point-source form caused by finite-source effects
during the caustic crossings. This analysis yields the normalized source
radius $\rho$, and the angular Einstein radius is determined by $\theta_{\rm
E}=\theta_{*}/\rho$, where the angular source radius $\theta_{*}$ is deduced
from the color and brightness of the source. We estimate the extinction and
reddening-corrected (dereddened) source color and brightness, $(V-I,I)_{0}$,
from the instrumental values, $(V-I,I)$, using the Yoo et al. (2004) method,
in which the centroid of red giant clump (RGC) in the color-magnitude diagram
(CMD) is used as a reference for this conversion. The RGC centroid can be used
as a reference because its dereddened color and brightness, $(V-I,I)_{\rm
RGC,0}$, are known, and the source and RGC stars, both of which are located in
the bulge, experience similar reddening and extinction.

Figure 9:  Positions of the source with respect to the centroid of red giant
clump (RGC) in the instrumental color-magnitude diagram of stars located
around the source.

Figure 9 shows the source position (blue dot) with respect to the RGC centroid
(red dot) in the CMD of stars around the source. The source color and
magnitude are estimated from the regression of the KMTC pyDIA data to the
model, and align them to the OGLE-III system to show calibrated color and
magnitude values. The instrumental colors and magnitudes of the source and RGC
centroid are $(V-I,I)=(2.19\pm 0.02,19.64\pm 0.01)$ and $(V-I,I)_{\rm
RGC}=(2.04,15.55)$, respectively. From the measured offsets in the color and
magnitude between the source and RGC centroid, $\Delta(V-I,I)$, together with
the known dereddened values of the RGC centroid $(V-I,I)_{\rm
RGC,0}=(1.06,14.38)$ (Bensby et al., 2013; Nataf et al., 2013), the dereddened
source color and brightness are estimated as

$\displaystyle{(V-I,I)_{0}}$ | $\displaystyle{{}=(V-I,I)_{{\rm RGC},0}+\Delta(V-I,I)}$
---|---
| $\displaystyle{{}=(1.22\pm 0.02,18.47\pm 0.01).}$
(5)

We note that the source color and brightness are estimated using the higher-
order 2L1S and the 3L1S model result in consistent values. The measured $V-I$
color is then converted into $V-K$ color using the Bessell & Brett (1988)
color relation, and we then estimate $\theta_{*}$ from the
$(V-K)$–$\theta_{*}$ relation of Kervella et al. (2004). This procedure yields
the angular source radius of

$\theta_{*}=1.15\pm 0.08\leavevmode\nobreak\ \mu{\rm as}.$ (6)

With the measured angular source radius, the angular Einstein radius and the
relative lens-source proper motion are determined as

$\theta_{\rm E}={\theta_{*}\over\rho}=0.43\pm 0.03\leavevmode\nobreak\ {\rm
mas},$ (7)

and

$\mu={\theta_{\rm E}\over t_{\rm E}}=6.18\pm 0.48\leavevmode\nobreak\ {\rm
mas\leavevmode\nobreak\ yr}^{-1},$ (8)

respectively.

Figure 10:  Bayesian posteriors of the primary lens mass ($M_{1}$,upper panel)
and distance (lower panel) to the lens. In each panel, the solid and dotted
curves represent the distributions obtained using the observables of the 2L1S
and 3L1S solutions, respectively.

##  5 Physical lens parameters

In this section, we estimate the physical parameters of the mass and distance
to the lens by conducting a Bayesian analysis using the measured observables
of $(t_{\rm E},\theta,\pi_{\rm E})$ together with the prior models of the lens
mass function and the physical and dynamical distributions of Galactic
objects. Because the degeneracy between the 2L1S and 3L1S solutions cannot be
broken, we conduct two sets analysis based on the parameters and observables
of the individual solutions.

We carry out the Bayesian analysis in two steps. In the first step, we
generate a large number ($6\times 10^{6}$) of artificial lensing events by
conducting a Monte Carlo simulation, in which the mass of the lens, the
distance to the lens and source, and the transverse lens-source velocity,
$v_{\perp}$, are deduced from the Galactic model. In the Galactic model used
in the simulation, we adopt the Han & Gould (2003), Han & Gould (1995), and
Zhang et al. (2020) models for the physical, dynamical distributions, and mass
function, respectively. For the individual simulated events, we compute the
lensing observables using the relations $t_{\rm E}=D_{\rm L}\theta_{\rm
E}/v_{\perp}$, $\theta_{\rm E}=(\kappa M\pi_{\rm rel})^{1/2}$, and $\pi_{\rm
E}=\pi_{\rm rel}\theta_{\rm E}$. In the second step, we construct the
posterior distributions of the $M$ and $D_{\rm L}$ for the artificial events
with observables that are consistent with the measured values.

Figure 10 shows the posterior distributions of the primary lens mass ($M_{1}$)
and distance obtained from the Bayesian analysis. The distributions of the
2L1S (solid curve) and 3L1S (dotted curve) solutions are similar to each other
due to the similarity of the observables between the two solutions. The
estimated masses of the binary lens components are

$M_{1}=0.62^{+0.25}_{-0.26}\leavevmode\nobreak\ M_{\odot};\qquad
M_{2}=0.07^{+0.03}_{-0.03}\leavevmode\nobreak\ M_{\odot},$ (9)

for both 2L1S and 3L1S solutions, indicating that the binary is composed of a
K-type dwarf and a low-mass object at the star/brown-dwarf boundary. The mass
of the tertiary lens component according to the 3L1S solution is

$M_{3}=6.40^{+2.64}_{-2.78}\leavevmode\nobreak\ M_{\rm J},$ (10)

which is in the planetary mass regime. If the 3L1S interpretation could be
shown to be correct, then the planet would be the seventh microlensing planet
in a binary. The system is located at a distance from Earth of

$D_{\rm L}=6.6^{+0.9}_{-1.1}\leavevmode\nobreak\ {\rm kpc}.$ (11)

For the estimation of $M$ and $D_{\rm L}$, we take the median as
representative values and the errors are estimated as the 16% and 84% of the
distributions.

##  6 Conclusion

We investigated the microlensing event KMT-2021-BLG-0322, for which the
lensing light curve exhibited three distinctive sets of caustic-crossing
features. It was found that the overall feature of the light curve was
approximately described by a binary-lens model, but the model left substantial
residuals. We tested various interpretations with the aim of explaining the
residuals. From this investigation, it was found that the residuals could be
explained either by considering a nonrectilinear relative lens-source motion
caused by the microlens-parallax and lens-orbital effects or by introducing a
low-mass companion to the binary lens. The degeneracy between the higher-order
2L1S model and the triple-lens model was very severe, making it difficult to
single out a correct solution based on only the photometric data. This
degeneracy was known before for two previous events (MACHO-97-BLG-41 and
OGLE-2013-BLG-0723), which led to the false detections of planets in binary
systems. Therefore, KMT-2021-BLG-0322, together with the two previous events,
demonstrates that the degeneracy can be not only common but also very severe,
emphasizing the need to check both models in the interpretations of deviations
from 2L1S models. From the Bayesian analysis conducted with the measured
lensing observables, it was estimated that the binary lens components have
masses $(M_{1},M_{2})=(0.62^{+0.25}_{-0.26}\leavevmode\nobreak\
M_{\odot},0.07^{+0.03}_{-0.03}\leavevmode\nobreak\ M_{\odot})$, for both 2L1S
and 3L1S solutions, and the mass of the tertiary lens component according to
the 3L1S solution is $M_{3}=6.40^{+2.64}_{-2.78}\leavevmode\nobreak\ M_{\rm
J}$.

###### Acknowledgements.

Work by C.H. was supported by the grants of National Research Foundation of
Korea (2020R1A4A2002885 and 2019R1A2C2085965). This research has made use of
the KMTNet system operated by the Korea Astronomy and Space Science Institute
(KASI) and the data were obtained at three host sites of CTIO in Chile, SAAO
in South Africa, and SSO in Australia. Work by I.K. was supported by JSPS
KAKENHI Grant Number 20J20633. Work by D.P.B., A.B., and C.R. was supported by
NASA through grant NASA-80NSSC18K027. T.S. acknowledges the financial support
from the JSPS, JSPS23103002, JSPS24253004, and JSPS2624702. Work by N.K. is
supported by JSPS KAKENHI Grant Number JP18J00897. The MOA project is
supported by J SPS KAK-ENHI Grant Number JSPS24253004, JSPS26247023,
JSPS23340064, JSPS15H00781, JP16H06287, 17H02871, and 19KK0082.

## References

  * Albrow (2017) Albrow, M. 2017, MichaelDAlbrow/pyDIA: Initial Release on Github,Versionv1.0.0, Zenodo, doi:10.5281/zenodo.268049
  * Albrow et al. (2000) Albrow, M. D., Beaulieu, J.-P., Caldwell, J. A. R., et al. 2000, ApJ, 534, 894
  * Batista et al. (2011) Batista, V., Gould, A., Dieters, S., et al. 2011, A&A, 529, 102
  * Bennett et al. (1999) Bennett, D. P., Rhie, S. H., Becker, A. C., et al. 1999, Nature, 402, 57
  * Bennett et al. (2016) Bennett, D. P., Rhie, S. H., Udalski, A., et al. 2016, AJ, 152, 125
  * Bozza (1999) Bozza, V. 1999, A&A, 348, 311
  * Jung et al. (2013) Jung, Y. K., Han, C., Gould, A., et al. 2013, ApJ, 768, L7
  * Alard & Lupton (1998) Alard, C., & Lupton, R. H. 1998, ApJ, 503, 325
  * An & Han (2002) An, J. H., & Han, C. 2002, ApJ, 573, 351
  * Bensby et al. (2013) Bensby, T., Yee, J. C., Feltzing, S., et al. 2013, A&A, 549, A147
  * Bessell & Brett (1988) Bessell, M. S., & Brett, J. M. 1988, PASP, 100, 1134
  * Bond et al. (2001) Bond, I. A., Abe, F., Dodd, R. J., et al. 2001, MNRAS, 327, 868
  * Di Stefano & Mao (1996) Di Stefano, R., & Mao, S. 1996, ApJ, 457, 93
  * Dominik (1998) Dominik, M. 1998, A&A, 329, 361
  * Dominik (1999) Dominik, M. 1999, A&A, 349, 108
  * Dong et al. (2006) Dong, S., DePoy, D. L., Gaudi, B. S., et al. 2006, ApJ, 642, 842
  * Dong et al. (2009) Dong, S., Gould, A., Udalski, A., et al. 2009, ApJ, 695, 970
  * Gonzalez et al. (2012) Gonzalez, O. A., Rejkuba, M., Zoccali, M., Valenti, E., Minniti, D., Schultheis, M., Tobar, R., Chen, B. 2012, A&A, 543, A13
  * Gould (1992) Gould, A. 1992, ApJ, 392, 442
  * Gould (2000) Gould, A. 2000, ApJ, 542, 785
  * Gould et al. (2014) Gould, A., Udalski, A., Shin, I.-G., et al. 2014, Science, 345, 46
  * Griest & Safizadeh (1998) Griest, K., & Safizadeh, N. 1998, ApJ, 500, 37
  * Han et al. (2016) Han, C., Bennett, D. P., Udalski, A., et al. 2016, ApJ, 825, 8
  * Han et al. (2001) Han, C., Chang, H.-Y., An, J. H., & Chang, K. 2001, MNRAS, 328, 986
  * Han & Gould (1995) Han, C., & Gould, A. 1995, ApJ, 447, 53
  * Han & Gould (2003) Han, C., & Gould, A. 2003, ApJ, 592, 172
  * Han et al. (2020c) Han, C., Lee, C.-U., Udalski, A., et al. 2020, AJ, 159, 48
  * Han et al. (2017) Han, C., Udalski, A., Gould, A., et al. 2017, AJ, 154, 223
  * Han et al. (2021) Han, C., Udalski, A., Lee, C.-U., et al. 2021, AJ, submitted
  * Holman & Wiegert (1999) Holman, M. J., & Wiegert, P. A. 1999, AJ, 117, 621
  * Ioka et al. (1999) Ioka, K., Nishi, R., & Kan-Ya, Y. 1999, Prog. Theo. Phys., 102, 98
  * Kervella et al. (2004) Kervella, P., Thévenin, F., Di Folco, E., & Ségransan, D. 2004, A&A, 426, 29
  * Kim et al. (2018) Kim, D.-J., Kim, H.-W., Hwang, K.-H., et al. 2018, AJ, 155, 76
  * Kim et al. (2016) Kim, S.-L., Lee, C.-U., Park, B.-G., et al. 2016, JKAS, 49, 37
  * Lee et al. (2008) Lee, D.-W., Lee, C.-U., Park, B.-G., et al. 2008, ApJ, 672, 623
  * Nataf et al. (2013) Nataf, D. M., Gould, A., Fouqué, P., et al. 2013, ApJ, 769, 88
  * Poleski et al. (2014) Poleski, R., Skowron, J., Udalski, A., et al. 2014, ApJ, 795, 42
  * Szymański et al. (2011) Szymański, M. K., Udalski, A., Soszyński, I., Kubiak, M., Pietrzyński, G., Poleski, R., Wyrzykowski, Ł., & Ulaczyk, K. 2011, Acta Astron., 61, 83
  * Tomaney & Crotts (1996) Tomaney, A. B., & Crotts, A. P. S. 1996, AJ, 112, 2872
  * Udalski et al. (2015) Udalski, A., Jung, Y. K., Han, C., et al. 2015, ApJ, 812, 47
  * Yee et al. (2012) Yee, J. C., Shvartzvald, Y., Gal-Yam, A., et al. 2012, ApJ, 755, 102
  * Yoo et al. (2004) Yoo, J., DePoy, D. L., Gal-Yam, A., et al. 2004, ApJ, 603, 139
  * Zang et al. (2021) Zang, W., Han, C., Kondo, I., et al. 2021, Res. in Astron. and Astrophys., in press (arXiv:2103.01896)
  * Zhang et al. (2020) Zhang, X., Zang, W., Udalski, A., et al. 2020, AJ, 159, 116
  * Zhu & Dong (2021) Zhu, W., & Dong, S. 2021, ARA&A, 59, 293


# Asymmetric Norms to Approximate the Minimum Action Distance

Lorenzo Steccanella, Anders Jonsson
Dept. Information and Communication Technologies,
Universitat Pompeu Fabra, Barcelona, Spain
{lorenzo.steccanella<EMAIL_ADDRESS>

###### Abstract

This paper presents a state representation for reward-free Markov decision
processes. The idea is to learn, in a self-supervised manner, an embedding
space where distances between pairs of embedded states correspond to the
minimum number of actions needed to transition between them. Unlike previous
methods, our approach incorporates an asymmetric norm parametrization,
enabling accurate approximations of minimum action distances in environments
with inherent asymmetry. We show how this representation can be leveraged to
learn goal-conditioned policies, providing a notion of similarity between
states and goals and a useful heuristic distance to guide planning.

To validate our approach, we conduct empirical experiments on both symmetric
and asymmetric environments. Our results show that our asymmetric norm
parametrization performs comparably to symmetric norms in symmetric
environments and surpasses symmetric norms in asymmetric environments.

##  1 Minimum Action Distance

In this section, we describe the notion of Minimum Action Distance, and we
derive useful ways of computing this measure on finite MDPs and continuous or
large MDPs.

We start by introducing some notation.

###### Definition 1

The Minimum Action Distance (MAD)
$d_{MAD}:\mathcal{S}^{2}\rightarrow\mathbb{R}^{+}$ is defined as the minimum
number of decision steps to transition between any pair of states
$(s_{i},s_{j})\in\mathcal{S}^{2}$.

We can observe that the MAD is an asymmetric distance function [Mennucci,
2013] and must satisfy the following properties:

  * •

$d_{MAD}\geq 0$ and $\forall s\in\mathcal{S}$, $d_{MAD}(s,s)=0$.

  * •

$d_{MAD}(s_{i},s_{j})=d_{MAD}(s_{j},s_{i})=0$ implies $s_{i}=s_{j}$.

  * •

$d_{MAD}(s_{i},s_{j})\leq d_{MAD}(s_{i},s_{k})+d_{MAD}(s_{k},s_{j})$
$\forall(s_{i},s_{j},s_{k})\in\mathcal{S}^{3}$.

###  1.1 Learning Minimum Action Distance from Adjacency Matrix

In discrete and finite MDPs we can compute the state-transition graph
$G=(V,E)$ of an MDP. In this section, we will revise how to learn the minimum
action distance from the graph adjacency matrix.

A state-transition graph $G=(V,E)$ of an MDP
$\mathcal{M}=\langle\mathcal{S},\mathcal{A},\mathcal{P},r\rangle$ is a graph
with nodes representing the states in the MDP and the edges representing state
adjacency in the MDP. More precisely, $V=\mathcal{S}$,
$e\left(s_{i},s_{j}\right)\in E\,$ iff $\,\exists
a\>\mathcal{P}\left(s_{i},a,s_{j}\right)>0$. An adjacency matrix represents a
graph with a square matrix of size $|\mathcal{S}|\times|\mathcal{S}|$ with
$(i,j)$-value being 1 if $e\left(s_{i},s_{j}\right)\in E$ and 0 otherwise.

$A_{ij}^{G}=\left\\{\begin{array}[]{ll}0&s_{i}=s_{j}\quad or\quad
e(s_{i},s_{j})\notin E\\\ 1&e(s_{i},s_{j})\in E\end{array}\quad
i,j=1,\ldots,|\mathcal{S}|.\right.$ (1)

Having access to the adjacency matrix $A^{G}$ we can simply compute the
minimum action distance by using the Floyd-Warshall algorithm [Floyd, 1962,
Roy, 1959, Warshall, 1962].

The Floyd-Warshall algorithm compares all possible paths through the graph
between each pair of vertices. It is able to do this with
$\Theta\left(|V|^{3}\right)$ comparisons in a graph, even though there may be
up to $\Omega\left(|V|^{2}\right)$ edges in the graph, and every combination
of edges is tested. It does so by incrementally improving an estimate on the
shortest path between two vertices until the estimate is optimal.

This dynamic programming procedure relies on having access to the edge
weights, which in the case of MAD reduces to having access to the adjacency
matrix $A^{G}$ of $a_{i,j}=1$ when $s_{i},s_{j}$ are connected by an edge.

Thanks to this we can define the shortest path on the $A^{G}$ by just first
computing the base cases:

$d(s_{i},s_{j})=\left\\{\begin{array}[]{lll}0&s_{i}=s_{j}\\\ 1&a_{i,j}=1\\\
\infty&a_{i,j}=0\quad and\quad s_{i}\neq s_{j}\end{array}\quad
i,j=1,\ldots,|\mathcal{S}|,\right.$ (2)

and subsequently computing the recursive case leveraging the triangle
inequality property:

$d(s_{i},s_{j})=\min(d(s_{i},s_{j}),d(s_{i},s_{k})+d(s_{k},s_{j}))\quad\forall(s_{i},s_{j},s_{k})\in\mathcal{S}^{3}.$
(3)

Note that in case we do not have access to the adjacency matrix $A^{G}$ this
can be retrieved by interacting with the environment by visiting all the one-
step transitions $(s,s^{\prime})$.

###  1.2 Symmetric embeddings

The Minimum Action Distance between states is a priori unknown and is not
directly observable in continuous and/or noisy state spaces where we cannot
simply enumerate the states and construct the adjacency matrix of the MDP.
Instead, we will approximate it using the distances between states observed on
trajectories. We introduce the notion of Trajectory Distance (TD) as follows:

###### Definition 2

(Trajectory Distance) Given any trajectory
$\tau=\\{s_{0},...,s_{n}\\}\sim\mathcal{M}$ collected in an MDP $\mathcal{M}$
and given any pair of states along the trajectory $(s_{i},s_{j})\in\tau$ such
that $0\leq i\leq j\leq n$, we define $d_{TD}(s_{i},s_{j}\mid\tau)$ as

$d_{TD}(s_{i},s_{j}\mid\tau)=(j-i),$ (4)

i.e. the number of decision steps required to reach $s_{j}$ from $s_{i}$ on
trajectory $\tau$.

We can observe that given any state trajectory $\tau=\\{s_{0},...,s_{n}\\}$,
choosing any pair of states $(s_{i},s_{j})\in\tau$ with $0\leq i\leq j\leq n$,
their distance along the trajectory represents an upper bound of the MAD.

$d_{MAD}(s_{i},s_{j})\leq d_{TD}(s_{i},s_{j}\mid\tau).$ (5)

Given a dataset of trajectories $\mathcal{D}$ collected by any unknown
behavior policy, we can retrieve the MAD $d_{MAD}$ by solving the following
constrained optimization problem:

$\displaystyle\begin{split}\min_{\theta}\quad&\sum_{\tau\in\mathcal{D}}\sum_{(s_{i},s_{j})\in\tau}(d_{\theta}(s_{i},s_{j})-d_{TD}(s_{i},s_{j}\mid\tau))^{2},\\\
\textrm{s.t.}\quad&d_{\theta}(s_{i},s_{j})\leq
d_{TD}(s_{i},s_{j}\mid\tau)\quad\forall(s_{i},s_{j})\in\mathcal{S}_{\mathcal{D}}^{2}\end{split}$
(6)

As a first step we can leverage the triangle inequality to simplify the
constrain in 6 and reduce the dependency on the quality of the trajectories in
the dataset $\mathcal{D}$.

$\displaystyle\begin{split}\min_{\theta}\quad&\sum_{\tau\in\mathcal{D}}\sum_{(s_{i},s_{j})\in\tau}(d_{\theta}(s_{i},s_{j})-d_{TD}(s_{i},s_{j}\mid\tau))^{2},\\\
\textrm{s.t.}\quad&d_{\theta}(s,s^{\prime})\leq
d_{TD}(s,s^{\prime}\mid\tau)\quad\forall\tau\in\mathcal{D},\forall(s,s^{\prime})\in\tau,\\\
&d_{\theta}(s_{i},s_{j})\leq
d_{\theta}(s_{i},s_{k})+d_{\theta}(s_{k},s_{j}),\quad\forall(s_{i},s_{j},s_{k})\in\mathcal{S}_{\mathcal{D}}^{3}\end{split}$
(7)

where $(s,s^{\prime})\in\tau$ refers to a one-step transition (i.e.
$d_{TD}(s,s^{\prime}|\tau)=1$) in the trajectory $\tau\in\mathcal{D}$ while
$(s_{i},s_{j},s_{k})\in\mathcal{S}_{\mathcal{D}}^{3}$ indicates all the
combinations of $3$ states contained in the trajectory dataset
$\mathcal{S}_{\mathcal{D}}$.

Note that the first constraint in 7 imposes an upper bound on one-step
transitions, i.e. it says that two states $(s,s^{\prime})$ at distance one
along a trajectory $d_{TD}(s,s^{\prime}|\tau)=1$ are either the same state
$s=s^{\prime}$ or they must satisfy $d_{MAD}=1$. This allows us to approximate
the MAD without having to identify whether two states along a trajectory are
the same state or not.

Note that the second constraint in 7 corresponds to the triangle inequality,
which thus is a property that holds for the MAD. Moreover, this second
constraint implies that we have to calculate it for all the combinations of
$3$ states contained in $\mathcal{S}_{\mathcal{D}}$ which can become
intractable for large state spaces.

To address this issue we proposed an alternative formulation based on
embedding the MAD in a parametric embedding space
$\phi_{\theta}:\mathcal{S}\rightarrow\mathbb{R}^{dim}$ where a chosen distance
metric that respects the triangle inequality (e.g. any norm $||\cdot||_{p}$)
can be used to enforce the triangle inequality constraint.

The goal is to learn a parametric state embedding
$\phi_{\theta}:\mathcal{S}\rightarrow\mathbb{R}^{dim}$ such that the distance
$d$ between any pair of embedded states approximates the Minimum Action
Distance.

Steccanella and Jonsson [2022] used this formulation to favour symmetric
embeddings, where they use norms as distance functions, e.g. the L1 norm
$d(z,y)=||z-y||_{1}$. If we use symmetric embeddings we will have that for any
pair of states $(s_{i},s_{j})\in\mathcal{S}$,

$d(\phi_{\theta}(s_{i}),\phi_{\theta}(s_{j}))\approx\min(d_{MAD}(s_{i},s_{j}),d_{MAD}(s_{j},s_{i})).$
(8)

The problem of learning this embedding can then be formulated as a constrained
optimization problem:

$\displaystyle\min_{\theta}$
$\displaystyle\sum_{\tau\in\mathcal{D}}\sum_{(s_{i},s_{j})\in\tau}(\left\|\phi_{\theta}(s_{i})-\phi_{\theta}(s_{j})\right\|_{p}-d_{TD}(s_{i},s_{j}\mid\tau))^{2},$
(9) s.t.
$\displaystyle\left\|\phi_{\theta}(s)-\phi_{\theta}(s^{\prime})\right\|_{p}\leq
d_{TD}(s,s^{\prime}\mid\tau)\;\;\;\forall\tau\in\mathcal{D},\forall(s,s^{\prime})\in\tau.$

Intuitively, the objective is to make the embedded distance between pairs of
states as close as possible to the observed trajectory distance while
respecting the upper bound constraints. Without constraints, the objective is
minimized when the embedding matches the expected Trajectory Distance
$\mathbb{E}\left[d_{TD}\right]$ between all pairs of states observed on
trajectories in the dataset $\mathcal{D}$. In contrast, constraining the
solution to match the minimum TD with the upper-bound constraints
$\left\|\phi_{\theta}(s)-\phi_{\theta}(s^{\prime})\right\|_{p}\leq
d_{TD}(s,s^{\prime}\mid\tau)$ allows us to approximate the MAD. The precision
of this approximation depends on the quality of the given trajectories.

To make the constrained optimization problem tractable, we can relax the hard
constraints in (9) and convert them into a penalty term to retrieve a simple
unconstrained formulation. Moreover, we rely on sampling
$(s_{i},s_{j},d_{TD}(s_{i},s_{j}\mid\tau))$ and
$(s,s^{\prime},d_{TD}(s,s^{\prime}\mid\tau))$ from the dataset of trajectories
$\mathcal{D}$ making this formulation amenable for gradient descent and to fit
within the optimization scheme of neural networks.

$\displaystyle\leavevmode\resizebox{422.77661pt}{}{$\mathcal{L}=\mathbb{E}_{(s_{i},s_{j},d_{TD}(s_{i},s_{j}\mid\tau))\sim\mathcal{D}}\left[(\left\|\phi_{\theta}(s_{i})-\phi_{\theta}(s_{j})\right\|_{p}-d_{TD}(s_{i},s_{j}\mid\tau))^{2}\right]+C,$}$
(10)

where $C$ is our penalty term defined as

$\displaystyle\leavevmode\resizebox{422.77661pt}{}{$C=\mathbb{E}_{(s,s^{\prime},d_{TD}(s,s^{\prime}\mid\tau))\sim\mathcal{D}}\left[\max\left(0,\left\|\phi_{\theta}(s)-\phi_{\theta}(s^{\prime})\right\|_{p}-d_{TD}(s,s^{\prime}\mid\tau)\right)^{2}\right]$}.$
(11)

The penalty term $C$ introduces a quadratic penalization of the objective for
violating the upper-bound constraints
$\left\|\phi_{\theta}(s)-\phi_{\theta}(s^{\prime})\right\|_{p}<=d_{TD}(s,s^{\prime}\mid\tau)$.

###  1.3 Asymmetric embeddings

In the previous section, we have seen how it is possible to define the MAD
embedding problem with the use of norms $||\cdot||_{p}$. While the formulation
is useful to understand how it is possible to remove the triangle inequality
constraint in 7 the Minimum Action Distance is naturally asymmetric and we
would like embedding that preserves this asymmetry.

A norm is a function $\|\cdot\|:\mathcal{X}\rightarrow\mathbb{R}$ satisfying,
$\forall x,y\in\mathcal{X},\alpha\in\mathbb{R}^{+}$:

  * •

$\mathbf{N1}$ (Pos. def.). $\|x\|>0$, unless $x=0$.

  * •

$\mathbf{N2}$ (Pos. homo.). $\alpha\|x\|=\|\alpha x\|$, for $\alpha\geq 0$.

  * •

$\mathbf{N3}$ (Subadditivity). $\|x+y\|\leq\|x\|+\|y\|$.

  * •

$\mathbf{N4}$ (Symmetry). $\|x\|=\|-x\|$.

An asymmetric semi-norm satisfies $\mathbf{N2}$, $\mathbf{N3}$ but not
necessarily $\mathbf{N1}$, $\mathbf{N4}$.

A convex function $f:\mathcal{X}\rightarrow\mathbb{R}$ is a function
satisfying $\mathbf{C1}:\forall x,y\in\mathcal{X},\alpha\in[0,1]:f(\alpha
x+(1-\alpha)y)\leq$ $\alpha f(x)+(1-\alpha)f(y)$. The commonly used ReLU
activation, $\operatorname{relu}(x)=\max(0,x)$, is convex.

Is easy to observe that any $\mathbf{N2}$ and any $\mathbf{N3}$ function is
convex and thus that any asymmetric semi-norm is convex.

Motivated by this relationship between convex functions and norms,Pitis et al.
[2020] introduced Wide Norms, a parametric distance that models symmetric and
asymmetric norms.

A Wide Norm is any combination of symmetric/asymmetric semi-norms. They are
based on the Mahalanobis norm of $x\in\mathbb{R}^{dim}$, parametrized by
$W\in\mathbb{R}^{m\times n}$, defined as $||x||_{W}=||Wx||_{2}$.

Asymmetric Wide Norms are defined as:

$||x|_{WN}=||Wrelu(x::-x)||_{2}\text{ where }W_{i}\in\mathbb{R}^{m_{i}\times
n}\text{ with }m_{i}\leq n\text{. }$

We can then use the parametrized Wide Norm distance to constrain the
triangular inequality on the embedding space:

$\displaystyle\leavevmode\resizebox{422.77661pt}{}{$\mathcal{L}=\mathbb{E}_{(s_{i},s_{j},d_{TD}(s_{i},s_{j}\mid\tau))\sim\mathcal{D}}\left[(||\phi_{\theta}(s_{i})-\phi_{\theta}(s_{j})|_{WN}-d_{TD}(s_{i},s_{j}\mid\tau))^{2}\right]+C,$}$
(12)

where $C$ is our penalty term defined as

$\displaystyle\leavevmode\resizebox{422.77661pt}{}{$C=\mathbb{E}_{(s,s^{\prime},d_{TD}(s,s^{\prime}\mid\tau))\sim\mathcal{D}}\left[\max\left(0,||\phi_{\theta}(s)-\phi_{\theta}(s^{\prime})|_{WN}-d_{TD}(s,s^{\prime}\mid\tau)\right)^{2}\right]$}.$
(13)

##  2 Learning the Transition Model

In the previous section, we showed how to learn a state representation that
encodes a distance metric between states. This distance allows us to measure
how far we are from the goal state, i.e.
$d(\phi_{\theta}(s_{t}),\phi_{\theta}(s_{goal}))$. However, on its own, the
distance metric does not directly give us a policy for reaching the desired
goal state.

In this section we propose a method to learn a transition model of actions,
that combined with our state representation allows us to plan directly in the
embedded space and derive policies to reach any given goal state. Given a
dataset of trajectories $\mathcal{T}$ and a state embedding
$\phi_{\theta}(s)$, we seek a parametric transition model
$\rho_{\zeta}(\phi_{\theta}(s),a)$ such that for any triple
$(s,a,s^{\prime})\in\mathcal{T}$,
$\rho_{\zeta}(\phi_{\theta}(s),a)\approx\phi_{\theta}(s^{\prime})$.

We propose to learn this model simply by minimizing the squared error as:

$\displaystyle\min_{\zeta}$
$\displaystyle\sum_{t}^{\mathcal{D}}\sum_{s,a,s^{\prime}}^{t}\left[(\rho_{\zeta}(\phi_{\theta}(s),a)-\phi_{\theta}(s^{\prime}))^{2}\right].$
(14)

Once we learned the model, given a goal $g$ we can select actions that
minimize the $d_{MAD}$ distance to the goal in the latent space
$\phi_{\theta}$.

$\arg\min_{a\in A}||\rho_{\zeta}(\phi_{\theta}(s),a)-\phi_{\theta}(g)|_{WN}$

##  3 Empirical Evaluation

In this section we report a comparison of symmetric norms and WideNorms
performance on symmetric and asymmetric environments.

We refer to $L1$ as the agent that learns the $d_{MAD}$ using an L1 norm and
to $WideNorm$ agent the agent that uses WideNorms to approximate the
$d_{MAD}$.

In Figure 1 we can observe that $L1$ is unable to approximate the MAD in
asymmetric environments ( Asymmetric PointMass Environment) converging to
$\min(d_{MAD}(s_{i},s_{j}),d_{MAD}(s_{j},s_{i}))$ while $WideNorm$ approximate
the asymmetric $d_{MAD}$ correctly.

In Figure 2 we report the performance of the planning on the embedded space
using the learned transition model. We can observe that in symmetric
environments ( Planning Symmetric PointMass and Planning reach ) both $L1$ and
$Widenorm$ have similar performance. But when we move to asymmetric
environments ( Planning Asymmetric PointMass ) we can observe that $L1$ agent
is unable to learn the true $d_{MAD}$ and as a consequence the planning
performance degrades. In contrary $WideNorm$ is able to learn the true
$d_{MAD}$ and outperforms $L1$ agent in planning.

The empirical evaluation demonstrates that symmetric norms such as the L1 norm
$||\cdot||_{1}$ used in this experiment are unable to approximate the MAD in
asymmetric environments converging to
$\min(d_{MAD}(s_{i},s_{j}),d_{MAD}(s_{j},s_{i}))$ while WideNorms approximate
the asymmetric $d_{MAD}$ correctly.

We release the software to reproduce the results at the following repository:
https://github.com/lorenzosteccanella/SRL under the branch "NIPS-GCRL-
Workshop" and some notebooks that ease the understanding of the code under the
branch "main".

Figure 1: MSE respect to the true $d_{MAD}$. Figure 2: Comparison of WideNorms
against P-norms on symmetric and asymmetric environments.

## References

  * Mennucci [2013] Andrea CG Mennucci.  On asymmetric distances.  _Analysis and Geometry in Metric Spaces_ , 1(2013):200–231, 2013.
  * Floyd [1962] Robert W. Floyd.  Algorithm 97: Shortest path.  _Commun. ACM_ , 5(6):345–, June 1962.  ISSN 0001-0782.  doi: 10.1145/367766.368168.  URL http://doi.acm.org/10.1145/367766.368168.
  * Roy [1959] Bernard Roy.  Transitivité et connexité.  In _Extrait des comptes rendus des séances de l’Académie des Sciences_ , pages 216–218. Gauthier-Villars, July 1959.  http://gallica.bnf.fr/ark:/12148/bpt6k3201c/f222.image.langFR.
  * Warshall [1962] Stephen Warshall.  A theorem on boolean matrices.  _J. ACM_ , 9(1):11–12, January 1962.  ISSN 0004-5411.  doi: 10.1145/321105.321107.  URL http://doi.acm.org/10.1145/321105.321107.
  * Steccanella and Jonsson [2022] Lorenzo Steccanella and Anders Jonsson.  State representation learning for goal-conditioned reinforcement learning.  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ , pages 84–99. Springer, 2022.
  * Pitis et al. [2020] Silviu Pitis, Harris Chan, Kiarash Jamali, and Jimmy Ba.  An inductive bias for distances: Neural nets that respect the triangle inequality.  _arXiv preprint arXiv:2002.05825_ , 2020.