Self-dual interval orders and row-Fishburn matrices

Sherry H. F. Yan, Yuexiao Xu
Department of Mathematics, Zhejiang Normal University, Jinhua 321004, P.R.
China
<EMAIL_ADDRESS>

Abstract. Recently, Jelínek derived that the number of self-dual interval
orders of reduced size $n$ is twice the number of row-Fishburn matrices of
size $n$ by using generating functions. In this paper, we present a bijective
proof of this relation by establishing a bijection between two variations of
upper-triangular matrices of nonnegative integers. Using the bijection, we
provide a combinatorial proof of the refined relations between self-dual
Fishburn matrices and row-Fishburn matrices in answer to a problem proposed by
Jelínek.

Key words: self-dual interval order, self-dual Fishburn matrix, row-Fishburn
matrix.

AMS Mathematical Subject Classifications: 05A05, 05C30.

##  1 Introduction

A poset is said to be an interval order ( also known as $(2+2)$-free poset) if
it does not contain an induced subposet that is isomorphic to $2+2$, the union
of two disjoint $2$-element chains. Let $P$ be a poset with a strict order
relation $\prec$. A strict down-set of an element $x\in P$ is the set $D(x)$
of all the elements of P that are smaller than $y$, i.e., $D(y)=\\{y\in
P:y\prec x\\}$. Similarly, the strict up-set of $x$, denoted by $U(x)$, is the
set $\\{y\in P:y\succ x\\}$. A poset $P$ is $(2+2)$-free if and only if its
sets of strict down-sets, $D(P)=\\{D(x):x\in P\\}$ can be written as

$D(P)=\\{D_{1},D_{2},\ldots,D_{m}\\}$

where $\emptyset=D_{1}\subset D_{2}\subset\ldots\subset D_{m}$, see [1, 2]. In
such context, we say that $x\in P$ has level $i$ if $D(x)=D_{i}$. An element
$x$ is said to be a minimal element if $x$ has level $1$. Following Fishburn
[7], we call the number $m$ of distinct strict down-sets the magnitude of P.
It turns out that $m$ is also equal to the number of distinct strict up-sets,
and we can order the strict up-sets of $P$ into a decreasing chain

$D(P)=\\{U_{1},U_{2},\ldots,U_{m}\\}$

where $U_{1}\supset U_{2}\supset\ldots\supset U_{m}=\emptyset$, see [7, 8]. We
say that $x$ has up-level $i$ if $U(x)=U_{i}$. An element $x$ is said to be a
maximal element if $x$ has up-level $m$.

The dual of a poset $P$ is the poset $\overline{P}$ with the same elements as
$P$ and an order relation $\overline{\prec}$ defined by $x\overline{\prec}y$
$\Longleftrightarrow$ $y\prec x$. A poset is self-dual if it is isomorphic to
its dual.

Fishburn [7, 9] did pioneering work on interval orders; for instance, he
showed the basic theorem that a poset is an interval order if and only if it
is $(2+2)$-free and established a bijection between interval orders and a
certain kind of integer matrices, called Fishburn matrices. Recently,
Bousquet-Mélou et al. [2] constructed bijections between interval orders and
ascent sequences, between ascent sequences and permutations avoiding a certain
pattern, between interval orders and regular linearized chord diagrams by
Stoimenow [12]. Several other papers have focused on bijections between
interval orders and other objects. For instance, Dukes and Parviainen [4] have
described a direct bijection between Fishburn matrices and ascent sequences,
while the papers of Claesson et al. [3] and Dukes et al. [6] extend the
bijection between interval orders and Fishburn matrices to more general
combinatorial structures.

A Fishburn matrix of size $n$ is an upper-triangular matrix with nonnegative
integers which sum to $n$ and each row and each column contains a nonzero
entry. Throughout this paper that each matrix has its rows numbered from top
to bottom, and columns numbered left-to-right, starting with row and column
number one. We let $M_{i,j}$ denote the entry of M in row $i$ and column $j$.
The size of a matrix $M$ is the sum of all its entries. Moreover, the
dimension of an upper triangular matrix is defined to the number of rows.

The dual matrix of $M$, denoted by $\overline{M}$, is obtained from $M$ by
transposition along the diagonal running from bottom-left to top-right. More
precisely, for $1\leq i,j\leq m$, we have $\overline{M}_{i,j}=M_{m+1-j,m+1-i}$
where $m$ is the dimension of $M$. If a matrix M is equal to $\overline{M}$,
we call it self-dual.

Fishburn [7, 9] showed that an interval order $P$ of magnitude m corresponds
to an $m\times m$ Fishburn matrix $M$ with $M_{i,j}$ being equal to the number
of elements of $P$ that have level $i$ and up-level $j$. Jelínek [10] showed
that the Fishburn’s bijection turns out to be a bijection between self-dual
interval orders of size $n$ and self-dual Fishburn matrices of size $n$.

Following the terminologies given in [10], we distinguish three types of cells
in a Fishburn matrix $M$ of dimension $k$ : a cell $(i,j)$ is a diagonal cell
if $i+j=k+1$, i.e., $(i,j)$ belongs to the north-east diagonal of the matrix.
If $i+j<k+1$ (i.e., $(i,j)$ is above and to the left of the diagonal) then
$(i,j)$ is a North-West cell, or NW-cell, while if $i+j>k+1$, then $(i,j)$ is
an SE-cell. Clearly, NW-cells and diagonal cells together determine a self-
dual Fishburn matrix. The reduced size of a self-dual fishburn matrix $M$ is
the sum of all diagonal cells and NW-cells. The reduced size of a self-dual
interval order $P$ is the reduced size of its corresponding self-dual Fishburn
matrix under Fishburn’s bijection.

A row-Fishburn matrix of size $n$ is defined to be an upper-triangular matrix
with nonnegative integers which sum to $n$ and each row contains a nonzero
entry. In a matrix $A$, the sum of a column (resp. row) is defined to the sum
of all the entries in this column (resp. row). A column or a row is said to be
zero if it contains no nonzero entries. The set of self-dual Fishburn matrices
of reduced size $n$ is denoted by $\mathcal{M}(n)$. Denote by
$\mathcal{M}(n,k,p)$ be the set of self-dual Fishburn matrices of reduced size
$n$ whose first row has sum $k$ and diagonal cells have sum $p$. Let
$\mathcal{RM}(n)$ be the set of row-Fishburn matrices of size $n$. The set of
row-Fishburn matrices in $\mathcal{RM}(n)$ whose last column has sum $k$ is
denote by $\mathcal{RM}(n,k)$. Denote by $\mathcal{RM}(n,k,p)$ be the set of
row-Fishburn matrices in $\mathcal{RM}(n,k)$ whose first row has sum $p$.
Moreover, the set of self-dual interval orders of reduced size $n$ is denoted
by $\mathcal{I}(n)$.

Based on the bijection between interval orders and Fishburn matrices, Jelínek
[10] presented a new method to derive formulas for the generating functions of
interval orders, counted with respect to their size, magnitude, and number of
minimal and maximal elements, which generalize previous results on refined
enumeration of interval orders obtained by Bousquet-Mélou et al. [2], Kitaev
and Remmel [11], and Dukes et al. [5]. Applying the new method, Jelínek [10]
obtained formulas for the generating functions of self-dual interval orders
with respect to analogous statistics. From the obtained generating functions,
relations between self-dual Fishburn matrices and row-Fishburn matrices were
derived, that is,

$|\mathcal{M}(n,k,0)|=|\mathcal{RM}(n,k)|,$ (1.1)

and for $p\geq 1$

$|\mathcal{M}(n,k,p)|=|\mathcal{RM}(n,k,p)|.$ (1.2)

Combining the bijection between self-dual interval orders and self-dual
Fishburn martices, formulas (1.1) and (1.2), Jelínek derived that for $n\geq
1$,

$|\mathcal{I}(n)|=|\mathcal{M}(n)|=2|\mathcal{RM}(n)|,$ (1.3)

and asked for bijective proofs of (1.1) and (1.2). The main objective of this
paper is to present bijective proofs of these formulas by establishing a one-
to-one correspondence between two variations of upper-triangular matrices of
nonnegative integers.

Let $\mathcal{M}(n,k)$ be the set of self-dual Fishburn matrices of reduced
size $n$ whose first row has sum $k$. Denote by $\mathcal{EM}(n,k)$ (resp.
$\mathcal{OM}(n,k)$ ) be the set of self-dual Fishburn matrices in
$\mathcal{M}(n,k)$ whose dimension are even (resp. odd). Using the bijection
between two variations of upper-triangular matrices of nonnegative integers,
we derive that

$|\mathcal{EM}(n,k)|=|\mathcal{OM}(n,k)|=|\mathcal{RM}(n,k)|.$ (1.4)

##  2 The bijective proofs

Recall that a self-dual Fishburn matrix is determined by its NW-cells and
diagonal cells. Given a self-dual Fishburn matrix $M$, the reduced matrix of
$M$, denoted by $R(M)$, is a matrix obtained from $M$ by filling all the SE-
cells with zeros. An upper-triangular matrix is said to a super  triangular
matrix if all its SE-cells are zero.

###### Lemma 2.1

Let $M^{\prime}$ be a super triangular matrix of dimension $m$. Then
$M^{\prime}$ is a reduced matrix of a self-dual Fishburn matrix if and only if
it satisfies the following two conditions:

  * (i)

for $1\leq i\leq\lceil{m\over 2}\rceil$, each column $i$ contains a nonzero
entry;

  * (ii)

for $1\leq i\leq\lceil{m\over 2}\rceil$, either row $i$ or column $m+1-i$
contains a nonzero entry.

Proof. Let $M$ be a self-dual Fishburn matrix with $R(M)=M^{\prime}$. Clearly,
$M^{\prime}$ is a super triangular matrix. Since the first $\lceil{m\over
2}\rceil$ columns of $M^{\prime}$ are the same as those in $M$, the condition
$(i)$ follows immediately. It remains to show that $M^{\prime}$ satisfies
condition $(ii)$. Since $M$ is self-dual Fishburn matrix, for all $1\leq i\leq
m$, row $i$ must contains a nonzero entry, that is,

$\sum_{j=1}^{m}M_{i,j}=\sum_{j=1}^{m-i}M_{i,j}+\sum_{j=m+1-i}^{m}M_{i,j}=\sum_{j=1}^{m-i}M_{i,j}+\sum_{j=1}^{i}M_{j,m+1-i}>0.$

Hence, for $1\leq i\leq\lceil{m\over 2}\rceil$, either row $i$ or column
$m+1-i$ of $R(M)$ contains a nonzero entry. Therefore, the condition $(ii)$
holds for $R(M)$.

Conversely, given a super triangular matrix $M^{\prime}$ satisfying conditions
$(i)$ and $(ii)$, We can recover a self-dual matrix $M$ from $M^{\prime}$ by
filling the SE-cell $(m+1-j,m+1-i)$ with $M^{\prime}_{i,j}$. If $1\leq
i\leq\lceil{m\over 2}\rceil$, the sum of row $i$ of $M$ is given by

$\sum_{j=i}^{m}M_{i,j}=\sum_{j=i}^{m-i}M_{i,j}+\sum_{j=m+1-i}^{m}M_{i,j}=\sum_{j=1}^{m-i}M_{i,j}+\sum_{j=1}^{i}M_{j,m+1-i}=\sum_{j=1}^{m-i}M^{\prime}_{i,j}+\sum_{j=1}^{i}M^{\prime}_{j,m+1-i}.$

By the condition $(ii)$, we have
$\sum_{j=i}^{m}M_{i,j}=\sum_{j=1}^{m-i}M^{\prime}_{i,j}+\sum_{j=1}^{i}M^{\prime}_{j,m+1-i}>0$,
which implies that row $i$ contains a nonzero entry. If $\lceil{m\over
2}\rceil+1\leq i\leq m$, the sum of row $i$ of $M$ is given by

$\sum_{j=i}^{m}M_{i,j}=\sum_{j=i}^{m}M^{\prime}_{m+1-j,m+1-i}=\sum_{j=1}^{m+1-i}M^{\prime}_{j,m+1-i},$

which implies that the sum of row $i$ of $M$ is the same as that of column
$m+1-i$ of $M^{\prime}$. By condition $(i)$, row $i$ contains a nonzero entry.
Hence $M$ is a self-dual Fishburn matrix with $R(M)=M^{\prime}$. This
completes the proof.

Denote by $\mathcal{SM}_{k}(n)$ the set of all super triangular matrices of
size $n$ and dimension $2k+1$ having the following two properties:

  1. $(a)$

for $1\leq i\leq k$, each column $i$ contains a nonzero entry;

  2. $(b)$

for $1\leq i\leq k$, either row $k+1-i$ or column $k+1+i$ contains a nonzero
entry.

Let $\mathcal{SM}(n)=\bigcup_{k\geq 0}\mathcal{SM}_{k}(n)$.

Now we proceed to present a map $\alpha$ from $\mathcal{M}(n)$ to
$\mathcal{SM}(n)$. Given a nonempty self-dual matrix $M$ of dimension $m$, let
$\alpha(M)$ be the matrix obtained from $M$ by the following procedure.

  * •

If $m=2k+1$ for some integer $k\geq 0$, then let $\alpha(M)$ be the matrix
obtained from the reduced matrix $R(M)$ of $M$ by interchanging the cell
$(i,k+1)$ and the diagonal cell $(i,m+1-i)$ for $1\leq i\leq k$.

  * •

If $m=2k$ for some integer $k\geq 1$, then let $A$ be the matrix obtained from
$R(M)$ by adding one zero row and one zero column immediately after column $k$
and row $k$. Define $\alpha(M)$ to be the matrix obtained from $A$ by
interchanging the cell $(i,k+1)$ and the diagonal cell $(i,m+1-i)$ of the
resulting matrix $A$.

Obviously, $\alpha(M)$ is a super triangular matrix of dimension $2k+1$ and
size $n$. It easy to check that the map $\alpha$ preserves the first $k$
columns and the total sum of row $i$ and column $m+1-i$ of the reduced matrix
$R(M)$. By Lemma 2.1, the matrix $\alpha(M)$ has properties $(a)$ and $(b)$.
Hence $\alpha(M)$ is a super triangular matrix in $\mathcal{SM}(n)$.

Conversely, given a super triangular matrix $M^{\prime}$ in $\mathcal{SM}(n)$
of dimension $2k+1$, we can recover a matrix $M\in\mathcal{M}(n)$ with
$\alpha(M)=M^{\prime}$. First we interchange the cell $(i,k+1)$ with the
diagonal cell $(i,m-i)$ for $1\leq i\leq k$. Then we obtain a matrix $A$ by
deleting column $k+1$ and row $k+1$ if they are zero. It is easy to check that
properties $(a)$ and $(b)$ ensure that the obtained matrix $A$ is the reduced
matrix of a self-dual Fishburn matrix. Let $M$ be a self dual Fishburn matrix
with $R(M)=A$. Hence $\alpha$ is a bijection between $\mathcal{M}(n)$ and
$\mathcal{SM}(n)$.

Let $M$ be a super triangular matrix of dimension $2k+1$, then column $k+1$ is
called a center column. From the construction of the bijection $\alpha$, we
see that the map $\alpha$ transforms the sum of the diagonal cells of a self-
dual matrix to the sum of the center column of a super triangular matrix.
Hence, we have the following result.

###### Theorem 2.2

The map $\alpha$ is a bijection between $\mathcal{M}(n)$ and
$\mathcal{SM}(n)$. Moreover, the bijection $\alpha$ preserves the sum of the
first row, and transforms the sum of the diagonal cells of a self-dual matrix
to the sum of the center column of a super triangular matrix.

###### Example 2.3

Consider a matrix $A\in\mathcal{M}(5)$,

$A=\begin{bmatrix}1&0&1&0&0\\\ 0&1&1&1&0\\\ 0&0&0&1&1\\\ 0&0&0&1&0\\\
0&0&0&0&1\\\ \end{bmatrix}.$

The reduced matrix of $A$ is given by

$R(A)=\begin{bmatrix}1&0&1&0&0\\\ 0&1&1&1&0\\\ 0&0&0&0&0\\\ 0&0&0&0&0\\\
0&0&0&0&0\\\ \end{bmatrix},$

and we have

$\alpha(A)=\begin{bmatrix}1&0&0&0&1\\\ 0&1&1&1&0\\\ 0&0&0&0&0\\\ 0&0&0&0&0\\\
0&0&0&0&0\\\ \end{bmatrix}.$

Let $\mathcal{B}(n)$ the set of upper-triangular matrices of size $n$ in which
each row contains a nonzero entry except for the first row. Given a nonempty
matrix $A\in\mathcal{RM}(n)$, we can get two distinct matrices in
$\mathcal{B}(n)$ from $A$ by either doing nothing or adding a zero row and a
zero column before the first row and the first column. Thus for $n\geq 1$ we
have the following relation

$|\mathcal{B}(n)|=2|\mathcal{RM}(n)|.$ (2.1)

Now we proceed to construct a bijection between the set $\mathcal{SM}(n)$ and
the set $\mathcal{B}(n)$. Before constructing the bijection, we need some
definitions. In a matrix $A$ with $m$ rows, the operation of adding column $i$
to column $j$ is defined by increasing $A_{k,j}$ by $A_{k,i}$ for each
$k=1,2,\ldots,m$.

Let $\mathcal{B}(n,k,p)$ be the set of matrices in $\mathcal{B}(n)$ whose
whose first row has sum $p$ and last column has sum $k$. Similarly, let
$\mathcal{SM}(n,k,p)$ be the set of matrices in $\mathcal{SM}(n)$ whose first
row has sum $k$ and center column has sum $p$.

###### Theorem 2.4

There is a bijection $\beta$ between $\mathcal{SM}(n)$ and $\mathcal{B}(n)$.
Moreover, the map $\beta$ is essentially a bijection between
$\mathcal{SM}(n,k,p)$ and $\mathcal{B}(n,k,p)$.

Proof. Given a nonempty triangular matrix $A\in\mathcal{SM}(n)$ of dimension
$2k+1$, we recursively construct a sequence of super triangular matrices
$A^{(0)},A^{(1)},\ldots,A^{(l)}$. Let $A^{(0)}=A$ and assume that we have
obtained the matrix $A^{(j)}$. Let $A^{(j)}$ be a super triangular matrix of
dimension $2r+1$ for some integer $r$. For $1\leq i\leq r$, if each column
$r+1+i$ is zero, then let $A^{(l)}=A^{(j)}$. Otherwise, we proceed to generate
the matrix $A^{(j+1)}$ by the following insertion algorithm.

  * •

Find the largest value $i$ such that column $r+1+i$ contains a nonzero entry.
Then fill the entries of column $r+1+i$ with zeros.

  * •

Insert one column immediately after column $r+1-i$, one zero row immediately
after row $r+1-i$, one zero column immediately before column $r+1+i$ and one
zero row immediately before row $r+1+i$. Let the entry in row $j$ of the new
inserted column after column $r+1-i$ be filled with the entry in row $j$ of
column $r+1+i$ of $A^{(j)}$ for $1\leq j\leq 2r+1$.

Suppose that $A^{(l)}$ is of dimension $2q+1$. Then the last $q$ rows and $q$
columns of $A^{(l)}$ are zero rows and columns. Let $B$ be an upper-triangular
matrix obtained from $A^{(l)}$ by deleting the last $q$ columns and $q$ rows.
From the above insertion procedure to generate $A^{(j+1)}$ form $A^{(j)}$ , we
see that the inserted column after column $r+1-i$ contains a nonzero entry.
This ensures that each matrix $A^{(j)}$ has property $(a)$ with $0\leq j\leq
l$. Hence each column $i$ of $A^{(l)}$ contains a nonzero entry with $1\leq
i\leq q$. Hence, $B$ is an upper-triangular matrix in which each column
contains a nonzero entry except for the last column. Moreover, the insertion
algorithm preserves the sum of each nonzero row of $A$, which implies that $B$
is of size $n$. Let $\beta(A)$ be the dual matrix of $B$. Hence we have
$\beta(A)\in\mathcal{B}(n)$.

Conversely, we can construct a matrix $A=\beta^{\prime}(A^{\prime})$ in
$\mathcal{SM}(n)$ from a matrix $A^{\prime}$ of dimension $k+1$ in
$\mathcal{B}(n)$. Let $B$ be the dual matrix of $A^{\prime}$. Define $M$ to be
a matrix of dimension $2k+1$ obtained from $B$ by adding $k$ consecutive zero
rows and $k$ consecutive zero columns immediately after column $k+1$ and row
$k+1$. Clearly, the obtained matrix is a super triangular matrix having
property $(a)$. If for all $1\leq i\leq k$, either row $k+1-i$ or column
$k+1+i$ contains a nonzero entry, then we do nothing for $M$ and let $A=M$.
Otherwise, we can construct a new super triangular matrix $A$ by the following
removal algorithm.

  * •

Find the least value $i$ such that neither row $k+1-i$ nor column $k+1+i$
contains a nonzero entry. Then we obtain a super triangular matrix by adding
column $k+1-i$ to column $k+2+i$ and removing columns $k+1+i$, $k+1-i$ and
rows $k+1-i$, $k+1+i$.

  * •

Repeat the above procedure for the resulting matrix until the obtained matrix
has property $(b)$.

Obviously, the obtained matrix $A$ is a super triangular matrix having
properties $(a)$ and $(b)$. Since the algorithm preserves the sums of entries
in each non-zero row of $B$, the matrix $A$ is of size $n$ and the sum of the
first row of $A$ is the same as that of $B$. The property $(b)$ ensures that
the inserted columns in the insertion algorithm are the removed columns in the
removal algorithm. Thus the map $\beta^{\prime}$ is the inverse of the map
$\beta$. From the construction of the removal algorithm, the sum of the center
column of $A$ is equal to the sum of the last column of $B$ as well as the the
sum of the first row of $A^{\prime}$. This completes the proof.

###### Example 2.5

Consider a matrix $A\in\mathcal{SM}(6)$,

$A=\begin{bmatrix}1&0&0&1&1\\\ 0&1&1&1&0\\\ 0&0&0&0&0\\\ 0&0&0&0&0\\\
0&0&0&0&0\\\ \end{bmatrix}.$

Let $A^{(0)}=A$. By applying the insertion algorithm, we get

$A^{(1)}=\begin{bmatrix}1&{\textbf{1}}&0&0&1&\textbf{0}&0\\\
\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}\\\
0&\textbf{0}&1&1&1&\textbf{0}&0\\\ 0&\textbf{0}&0&0&0&\textbf{0}&0\\\
0&\textbf{0}&0&0&0&\textbf{0}&0\\\
\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}\\\
0&\textbf{0}&0&0&0&\textbf{0}&0\\\ \end{bmatrix},$

$A^{(2)}=\begin{bmatrix}1&1&0&\textbf{1}&0&\textbf{0}&0&0&0\\\
0&0&0&\textbf{0}&0&\textbf{0}&0&0&0\\\ 0&0&1&\textbf{1}&1&\textbf{0}&0&0&0\\\
\bf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}\\\
0&0&0&\textbf{0}&0&\textbf{0}&0&0&0\\\ \textbf{0}&\textbf{0}&\textbf{0
}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}&\textbf{0}\\\
0&0&0&\textbf{0}&0&\textbf{0}&0&0&0\\\ 0&0&0&\textbf{0}&0&\textbf{0}&0&0&0\\\
0&0&0&\textbf{0}&0&\textbf{0 }&0&0&0\\\ \end{bmatrix},$

where the inserted rows and columns are illustrated in bold at each step of
the insertion algorithm. Removing the last $4$ zero rows and $4$ zero columns,
we get

$B=\begin{bmatrix}1&1&0&1&0\\\ 0&0&0&0&0\\\ 0&0&1&1&1\\\ 0&0&0&0&0\\\
0&0&0&0&0\\\ \end{bmatrix}.$

Finally, we obtain

$A^{\prime}=\beta(A)=\begin{bmatrix}0&0&1&0&0\\\ 0&0&1&0&1\\\ 0&0&1&0&0\\\
0&0&0&0&1\\\ 0&0&0&0&1\\\ \end{bmatrix}$

Conversely, given $A^{\prime}\in\mathcal{B}(6)$, by applying removal
algorithm, we can get $A\in\mathcal{SM}(6)$, where the removed rows and
columns are illustrated in bold at each step of the removal algorithm.

Combining the bijection between self-dual interval orders and self-dual
Fishburn matrices and Theorems 2.2 and 2.4, we get a bijective proof of
$(\ref{eq3})$.

From Theorems 2.2 and 2.4, we have

$|\mathcal{M}(n,k,p)|=|\mathcal{SM}(n,k,p)|=|\mathcal{B}(n,k,p)|.$

Given a matrix $M\in\mathcal{B}(n,k,0)$, we can get a matrix
$A\in\mathcal{RM}(n,k)$ by deleting the first row and the first column.
Conversely, given a matrix $A^{\prime}\in\mathcal{RM}(n,k)$, we can obtain a
matrix $M^{\prime}\in\mathcal{B}(n,k,0)$ by inserting a zero row and a zero
column before the first row and the first column. This yields that

$|\mathcal{M}(n,k,0)|=|\mathcal{B}(n,k,0)|=|\mathcal{RM}(n,k)|.$ (2.2)

If $p>0$, then $\mathcal{B}(n,k,p)$ is the same as $\mathcal{RM}(n,k,p)$.
Hence, if $p>0$ then we have

$|\mathcal{M}(n,k,p)|=|\mathcal{B}(n,k,p)|=|\mathcal{RM}(n,k,p)|.$ (2.3)

Therefore, we get combinatorial proofs of (1.1) and (1.2), in answer to the
problem posed by Jelínek [10].

Now we proceed to prove (1.4). Given a matrix $A\in\mathcal{EM}(n,k)$ of
dimension $2m$ for some integers $m\geq 1$, let $R(A)$ be its reduced matrix.
We obtain a super triangular matrix $A^{\prime}$ from $A$ by inserting a zero
column and a zero row immediately after column $m$ and row $m$. By Lemma 2.1,
we have $A^{\prime}\in\mathcal{SM}(n,k,0)$.

Conversely, given a matrix $A^{\prime}\in\mathcal{SM}(n,k,0)$ of dimension
$2m+1$ for some integer $m\geq 1$, we can recover a self-dual matrix
$A\in\mathcal{EM}(n,k)$ as follows. First, we get a super triangular matrix
$B$ from $A^{\prime}$ by deleting column $m+1$ and row $m+1$. Let $A$ be a
matrix with $B=R(A)$. Obviously, we have the matrix $A\in\mathcal{EM}(n,k)$.
Hence, we get $|\mathcal{EM}(n,k)|=|\mathcal{SM}(n,k,0)|.$ By (2.2), we deduce
that

$|\mathcal{EM}(n,k)|=|\mathcal{SM}(n,k,0)|=|\mathcal{RM}(n,k)|.$ (2.4)

From (2.2) and (2.3), we have

$\begin{array}[]{lll}|\mathcal{M}(n,k)|&=&|\mathcal{M}(n,k,0)|+\sum_{p\geq
1}|\mathcal{M}(n,k,p)|\\\ &=&|\mathcal{RM}(n,k)|+\sum_{p\geq
1}|\mathcal{RM}(n,k,p)|\\\ &=&2|\mathcal{RM}(n,k)|.\end{array}$

Meanwhile, we have
$|\mathcal{M}(n,k)|=|\mathcal{EM}(n,k)|+|\mathcal{OM}(n,k)|$. Hence, (1.4)
follows from (2.4).

Acknowledgments. This work was supported by the National Natural Science
Foundation of China (10901141).

## References

  * [1] K.P. Bogart, An obvious proof of Fishburn’s interval order theorem, Discrete Math. 118 (1993), 239–242.
  * [2] M. Bousquet-Mélou, A. Claesson, M. Dukes, S. Kitaev, $(2+2)$-free posets, ascent sequences and pattern avoiding permutations, J. Combin. Theory Ser. A 117 (2010), 884–909.
  * [3] A. Claesson, M. Dukes, and M. Kubitzke, Partition and composition matrices, J. Combin. Theory, Ser. A, 118 (2011), 1624–1637.
  * [4] M. Dukes, R. Parviainen, Ascent sequences and upper triangular matrices containing non-negative integers, Electronic J. combin.  17  (2010), R53.
  * [5] M. Dukes, S. Kitaev, J. Remmel, and E. Steingrímsson, Enumerating (2+2)-free posets by indistinguishable elements, arXiv:1006.2696, 2010\.
  * [6] M. Dukes, V. Jelínek, and M. Kubitzke, Composition matrices, (2+2)-free posets and their specializations, Electronic J. Combin., 18 (2011), P44.
  * [7] P. C. Fishburn, Interval lengths for interval orders: A minimization problem, Discrete Mathematics, 47 (1983), 63–82.
  * [8] P. C. Fishburn, Interval graphs and interval orders, Discrete Mathematics, 55 (1985), 135–149.
  * [9] P. C. Fishburn, Interval orders and interval graphs: A study of partially ordered sets, John Wiley & Sons, 1985.
  * [10] V. Jelínek, Counting self-dual interval orders, arXiv:1106.2261, 2011.
  * [11] S. Kitaev, J. Remmel, Enumerating $(2+2)$-free posets by the number of minimal elements and other statistics, Disctere Appl. Math., 159  (2011), 2098–2108.
  * [12] A. Stoimenow, Enumumeration of chord diagrams and an upper bound for Vassiliev invariants, J. Knot Theory Ramifications 7 (1998), 93–114.
  * [13] D. Zagier, Vassiliev invariants and a stange identity related to the Dedeking eta-function, Topology 40 (2001), 945–960.


11institutetext: Instituto de FÃsica de Cantabria (CSIC-UC), Avda. de los
Castros s/n, 39005 Santander, Spain

# An updated and improved thermal SZ $y$-map from Planck PR4 data

Jyothis Chandran 11<EMAIL_ADDRESS>Mathieu Remazeilles 11 R. B.
Barreiro 11

###### Abstract

In 2015, the Planck Collaboration released an all-sky map of the thermal
Sunyaev-Zeldovich (SZ) effect, obtained by implementing the Needlet Internal
Linear Combination (NILC) method on the Planck PR2 data. The quality of the
Planck data has significantly improved since then. The Planck PR4 data release
offers upgraded full-sky maps in the LFI and HFI frequency bands with improved
systematics and sensitivity. We present a new all-sky thermal SZ Compton
$y$-parameter map derived from the Planck PR4 data using NILC and highlight
improvements, particularly in noise reduction and handling residual foreground
contamination. The PR4 NILC Compton $y$-parameter map has been made publicly
available to support further analyses.

##  1 Introduction

Cosmic Microwave Background (CMB) photons, on their journey from the last-
scattering surface to us, experience inverse Compton scattering when they
interact with high-energy electrons within hot ionized gas regions along the
way. This process occurs not only within galaxy clusters but also in the
unbound gas regions between clusters. The CMB photons, on average, gain energy
from these high-energy electrons, resulting in a shift toward higher
frequencies in their energy distribution. This is visible as a spectral
distortion of the CMB blackbody spectrum and is called the thermal Sunyaev-
Zeldovich (SZ) effect Zeldovich:1969 . It is possible to produce a Compton
$y$-parameter map (or $y$-map), tracing the amplitude of the thermal SZ effect
across the sky, by using weighted linear combinations of multifrequency sky
observations. Such a map can serve as a direct tracer of the hot baryons
within and between galaxy clusters and an indirect tracer of the dark matter
distribution. Either alone or in combination with primary CMB or large-scale
structure observations, the thermal SZ $y$-map can provide valuable
constraints on cosmological parameters such as the amplitude of dark matter
fluctuations, $\sigma_{8}$, and the matter density, $\Omega_{\rm m}$.

The Planck satellite mission provided full-sky coverage across multiple
frequency channels. Notably, the nine Planck frequencies ranging from 30 to
857 GHz are distributed to effectively capture the spectral response of the
thermal SZ effect (see Fig. 1). Therefore, using the Planck data release 2
(PR2, PR2:Plk2015I ), two $y$-maps were released in 2015 by
Remazeilles:Plk2015XXII , generated using two different implementations, NILC
and MILCA, of the internal linear combination (ILC) method to combine the
frequency maps with optimal weighting. Subsequently, the quality of the Planck
data has improved, culminating in the Planck data release 4 (PR4)
NPIPE:Plk2020LVII . In this proceedings paper, we present a new improved
$y$-map derived from Planck PR4 data using the NILC (Needlet Internal Linear
Combination, NILC2009 ) method, as detailed in NILC_PR4_ymap .

##  2 Reconstruction of the Compton $y$-map

The Planck observations consist of full-sky maps across nine frequency bands,
each of them containing a combination of the thermal SZ signal, the CMB,
various astrophysical foreground emissions, and instrumental noise. Extracting
and mapping the faint thermal SZ signal with high fidelity is a real component
separation challenge.

Figure 1: The frequency response, or spectral energy distribution (SED), of
the thermal SZ effect in intensity units and the PR4 bandpass integrated
coefficients corresponding to the Planck frequency channels. The frequency
channels of the Planck instruments trace the frequency response of the thermal
SZ across its negative peak ($\sim$ 140 GHz), null ($\sim$ 220 GHz), and
positive peak ($\sim$ 350GHz).

###  2.1 Planck Release 4 (PR4) data

In this work, we use the nine full-sky frequency maps from the Planck PR4
release NPIPE:Plk2020LVII . This data set has lower instrumental noise levels
compared to earlier releases, primarily due to $\sim$8% additional data from
reporting manoeuvre. Enhanced destriping using a short baseline has
effectively reduced the striping artefacts observed in the scanning direction
of the Planck satellite in earlier data releases. Unlike previous releases,
the Low-Frequency Instrument (LFI) and High-Frequency Instrument (HFI) PR4
data were processed coherently within a single pipeline (NPIPE). Calibration
and HFI frequency bandpasses also exhibit differences in this release. It is
worth mentioning that the Solar dipole and a frequency-dependent kinematic
quadrupole component were retained in the PR4 frequency maps, requiring
subtraction before component separation. In addition to the full-mission PR4
maps, we use the PR4 half-ring (HR) data splits to generate two supplementary
$y$-maps (HR1 and HR2 $y$-maps) for noise characterization.

###  2.2 NILC implementation

In this study, we implement the NILC method NILC2009 , following the procedure
described in Remazeilles:Plk2015XXII , with some differences as outlined
below. As an ILC method, NILC takes advantage of the known spectral response
of the tSZ signal to perform a constrained weighted linear combination of the
frequency maps that preserves the signal while minimizing the variance of the
foregrounds. Furthermore, being a _wavelet-based_ ILC, NILC enables localized
filtering in both pixel and harmonic domains simultaneously, with the NILC
weights adjusting dynamically based on changing foreground contamination
conditions across the sky and different angular scales.

We perform NILC across 10 multipole ranges, using Gaussian-shaped _needlet_
(spherical wavelet) windows. A small mask covering $\sim$2% of the PR4 maps is
applied before NILC processing to prevent ringing effects due to the brightest
pixels in the Galactic centre. The non-Gaussian beams of the PR4 data are used
to deconvolve the effect of the instrument beam across frequencies, a
departure from the PR2 $y$-map reconstruction, which employed Gaussian beam
approximations. The choice of Planck frequency channels in NILC is selective
and depends on the specific range of angular scales. This strategy prevents
the low-resolution LFI channels from contributing to the small scales in the
$y$-map and mitigates the contamination from the cosmic infrared background
(CIB) by excluding the $857$ GHz channel at those scales. For more on
methodology, see section 3.2 of NILC_PR4_ymap .

##  3 Results

###  3.1 Map inspection

Figure 2: The NILC $y$-map from Planck PR4 data in orthographic projection
(10’ resolution). The left side corresponds to the northern hemisphere, and
the right side corresponds to the southern hemisphere in Galactic coordinates.
The black line outlines the boundary of the Galactic mask used to mitigate
residual Galactic foreground contamination when computing power spectra and
other statistics.

Figure 2 shows the new PR4 $y$-map produced with NILC. The thermal SZ signal
from galaxy clusters is visible as red spots, with examples like Coma and
Virgo visible in the central region of the northern hemisphere. Along the
edges, residual Galactic foreground contamination is noticeable. In comparison
to the PR2 $y$-map, the new PR4 $y$-map exhibits lower granularity due to
reduced noise and extragalactic foreground contamination. Further details
about this improvement are discussed in the subsequent sections. Moreover, the
PR4 $y$-map lacks the patchy blue artefacts near the bottom of the southern
hemisphere observed in the PR2 version.

Figure 3: The difference between the PR4 and PR2 $y$-maps (_left_) in a large
sky area ($\sim 60^{\circ}\times 60^{\circ}$). Residual stripes from the PR2
$y$-map along the Planck scanning direction are observable. Low Galactic
latitude galaxy cluster CIZA J2302.7+7137 in the PR4 (_middle_) and PR2
(_right_) NILC $y$-maps.

The PR2 $y$-maps were known to have significant residual large-scale
striations along the scanning direction due to $1/f$ noise (see section 4.1 of
Remazeilles:Plk2015XXII ). Even though the striations are large-scale
features, they can change the brightness of compact SZ sources along their
direction. Thanks to an improved destriping strategy implemented in the PR4
frequency maps by NPIPE, the resulting PR4 $y$-map exhibits reduced levels of
striping in comparison to the PR2 $y$-maps. This is illustrated in Figure 3,
where we show the difference between the PR4 and PR2 NILC $y$-maps (_left
panel_), highlighting the contrast in striping levels between the two
$y$-maps. Figure 3 also highlights a galaxy cluster (CIZA J2302.7+7137 CIZA )
positioned at low Galactic latitude near the Galactic plane in both the PR4
(_middle panel_) and PR2 (_right panel_) NILC $y$-maps. The cluster exhibits
improved resolution and morphology in the new map.

Figure 4: A $3^{\circ}\times 3^{\circ}$ region near the Galactic plane
featuring a dust filament in the PR4 $y$-map (_left_), the PR2 $y$-map
(_middle_), and the IRIS (reprocessed IRAS) 100-micron map (_right_). The PR4
$y$-map exhibits significantly reduced residual Galactic dust contamination in
comparison to the PR2 $y$-map.

Galactic thermal dust emission acts as a major foreground contaminant at large
angular scales and low Galactic latitudes. As evident from Figure 4, the PR2
$y$-map (_middle panel_) exhibits significant residual contamination from a
thermal dust filament identified in the IRIS 100-micron map (_right panel_),
which is notably reduced in the PR4 $y$-map (_left panel_). The RMS (root mean
square) of the PR4 $y$-map within the Galactic region outlined by the black
line in Figure 2 is also 5% lower than that of the PR2 $y$-map. This
corroborates the fact that the new map has lower contamination from Galactic
foregrounds.

###  3.2 One-point statistics (1-PDF)

The thermal SZ effect has a characteristic non-Gaussian distribution, with a
positively skewed tail sz_skewness . Figure 5 shows the one-point probability
density function (1-PDF) of the PR2 $y$-map (red) and the PR4 $y$-map (black),
obtained by computing the histogram of the maps over $\sim 56\%$ of the sky
after masking the Galactic region (outlined in black in Figure 2) and the
point-sources detected at frequencies below $217$ GHz. Both $y$-maps display
consistent positive tails, indicating a coherent reconstruction of the signal.

Figure 5: The 1-PDF of the PR2 (red) and PR4 (black) NILC $y$-maps. The
positive tail characteristic of thermal SZ emission is consistent between the
two $y$-maps, while the width (variance) of the distribution originating from
contaminants is reduced for the PR4 $y$-map.

However, the width of the two distributions differs due to lower noise and
foreground contamination in the PR4 $y$-map, resulting in a $17\%$ lower
variance in the 1-PDF compared to the PR2 $y$-map. To estimate the noise
contribution to the 1-PDF of the PR2 and PR4 $y$-maps, we also computed the
1-PDF of the half-difference of $y$-maps reconstructed from half-ring data
splits (HR1 and HR2 $y$-maps, as described in section 2.1). The analysis
revealed a $7\%$ reduction in the variance of the noise 1-PDF for the PR4
$y$-map compared to the PR2 $y$-map.

###  3.3 Two-point statistics (angular power spectrum)

The angular power spectrum of the thermal SZ $y$-map is as a robust
cosmological probe sz_power_spectrum . Benefiting from the extensive sky
coverage provided by the Planck $y$-map, we can glean information across a
broad range of angular scales, covering emissions from diffuse hot gas to
compact galaxy clusters. Figure 6 shows the power spectra of the PR4/PR2
$y$-maps (_left_ , black/red dashed) over $56\%$ of the sky in the multipole
range $\ell=2$ to $2048$, after correcting for the mask, beam, and pixel
windows. The PR4 $y$-map exhibits lower power than the PR2 $y$-map at small
angular scales primarily due to reduced noise levels. To correct for the noise
bias in the power spectra, we also compute the cross-power spectrum between
the HR1 and HR2 $y$-maps (solid lines), as their noise is largely
uncorrelated.

After correcting for the noise bias, the PR4 $y$-map (solid black) shows
consistent reconstruction at intermediate multipoles
($\ell\leavevmode\nobreak\ \sim$ 30-300) and lower power at high multipoles
compared to the PR2 $y$-map (solid red). This difference at high multipoles is
due to lower residual contamination from extragalactic foregrounds, in
particular from CIB, as demonstrated hereafter.

Figure 6: _Left_ : The angular power spectra of the PR4 NILC $y$-map (black)
and PR2 NILC $y$-map (red), including their auto-power spectra (dashed) and
cross-power spectra of HR1 and HR2 $y$-maps (solid) for noise bias correction.
_Right_ : The cross power spectra of the PR4 NILC (blue), PR2 NILC (red) and
PR2 MILCA (yellow) $y$-maps with the Planck GNILC $857$ GHz CIB template.

To assess residual contamination from CIB, cross-power spectra between the
various $y$-maps and the Planck GNILC CIB template at $857$ GHz
GNILC_CIB:Plk2015XLVIII were computed over $50\%$ of the sky. The $857$ GHz
CIB template was chosen because this frequency channel was not used for
reconstructing the $y$-maps at multipoles above $\ell=300$, preventing
spurious correlations between data products due to shared frequency channels.
Results in the multipole range $\ell=600$ to $2000$ are shown in Figure 6
(_right_). The PR4 NILC $y$-map (solid blue) exhibits significantly lower CIB
contamination compared to the PR2 NILC (solid red) and PR2 MILCA (solid
yellow) $y$-maps. On average, the PR4 $y$-map demonstrates a $34\%$ decrease
in residual CIB contamination across the multipole range $\ell=600$-$2048$. As
the noise variance is much higher than the CIB variance, even a slight
reduction in noise in the PR4 data set enables NILC to effectively remove a
substantial portion of CIB contamination at small angular scales, resulting in
the observed improvements for the PR4 $y$-map.

##  4 Conclusions

We have presented a new all-sky Compton $y$-parameter map using the Planck PR4
data, which shows significant improvements over the Planck PR2 $y$-maps.
Through various checks in the pixel and harmonic domains at the map, 1-PDF and
power spectra levels, we showed that the PR4 $y$-map has reduced contamination
from instrumental noise ($\sim$ 7%), CIB ($\sim$ 34%), and Galactic dust
compared to the PR2 $y$-maps.

The Planck PR4 NILC $y$-map, the half-ring (HR1 and HR2) $y$-maps, and the
associated masks are available for public use at
https://doi.org/10.5281/zenodo.7940376 and in the Planck Legacy Archive
(http://pla.esac.esa.int/pla).

## Acknowledgements

We thank Marcos LÃpez-Caniego for providing point-source masks for PR4 data.
JC acknowledges financial support from the ConcepciÃn Arenal Programme of the
Universidad de Cantabria. MR acknowledges financial support from the CSIC
programme âAyuda a la IncorporaciÃn de CientÃficos Titularesâ provided under
the project 202250I159. Some of the results in this paper have been derived
using the healpy healpy and HEALPix HEALPix packages.

## References

  * (1) Zeldovich Ya. B., Sunyaev R. A., Astrophys. Space Sci., 4, 301 (1969)
  * (2) Planck Collaboration I 2016, A&A, 594, A1 (2016)
  * (3) Planck Collaboration XXII 2016, A&A, 594, A22 (2016)
  * (4) Planck Collaboration Int. LVII 2020, A&A, 643, A42 (2020)
  * (5) Delabrouille J., et al., A&A, 493, 835-857 (2009)
  * (6) Chandran J., Remazeilles M., Barreiro R. B., MNRAS, doi: 10.1093/mnras/stad3156 (2023)
  * (7) Kocevski, et al., APJ, 662, 224-235 (2007)
  * (8) RubiÃo-MartÃn J. A., Sunyaev R. A., MNRAS, 344, 1155-1174 (2003)
  * (9) Komatsu E., Seljak U., MNRAS, 336, 1256-1270 (2002)
  * (10) Planck Collaboration Int. XLVIII 2016, A&A, 596, A109 (2016)
  * (11) Zonca A., et al., Journal of Open Source Software, 4, 1298 (2019)
  * (12) GÃrski, K. M., et al., APJ, 622, 759-771 (2005)


itemizestditemize enumeratestdenumerate

# Cross-Modal Contrastive Learning of Representations for Navigation using
Lightweight, Low-Cost Millimeter Wave Radar for Adverse Environmental
Conditions

Jui-Te Huang, Chen-Lung Lu, Po-Kai Chang, Ching-I Huang, Chao-Chun Hsu,
Zu Lin Ewe, Po-Jui Huang, and Hsueh-Cheng Wang∗ All authors are affiliated
with National Chiao Tung University, Taiwan. Corresponding author email:
<EMAIL_ADDRESS>
###### Abstract

Deep reinforcement learning (RL), where the agent learns from mistakes, has
been successfully applied to a variety of tasks. With the aim of learning
collision-free policies for unmanned vehicles, deep RL has been used for
training with various types of data, such as colored images, depth images, and
LiDAR point clouds, without the use of classic map–localize–plan approaches.
However, existing methods are limited by their reliance on cameras and LiDAR
devices, which have degraded sensing under adverse environmental conditions
(e.g., smoky environments). In response, we propose the use of single-chip
millimeter-wave (mmWave) radar, which is lightweight and inexpensive, for
learning-based autonomous navigation. However, because mmWave radar signals
are often noisy and sparse, we propose a cross-modal contrastive learning for
representation (CM-CLR) method that maximizes the agreement between mmWave
radar data and LiDAR data in the training stage. We evaluated our method in
real-world robot compared with 1) a method with two separate networks using
cross-modal generative reconstruction and an RL policy and 2) a baseline RL
policy without cross-modal representation. Our proposed end-to-end deep RL
policy with contrastive learning successfully navigated the robot through
smoke-filled maze environments and achieved better performance compared with
generative reconstruction methods, in which noisy artifact walls or obstacles
were produced. All pretrained models and hardware settings are open access for
reproducing this study and can be obtained at https://arg-
nctu.github.io/projects/deeprl-mmWave.html.

##  I Introduction

Navigation and collision avoidance are fundamental capabilities for mobile
robots, and state estimation and path planning, which are integral to such
capabilities, have undergone much development. Deep reinforcement learning
(RL) is another method for realizing these capabilities, where robots learn to
solve problems through trial and error. Extensive efforts have been made to
formulate training methods for deep RL models using various types of data,
such as those from RGB cameras, depth images [1, 2], and laser range finder
(LiDAR) point clouds [3, 4]. However, optical sensors are often useless in
adverse weather and environmental conditions: LiDAR inputs are degraded by
rain, fog, and dust, and cameras are sensitive to variable lighting in dark
and sunny weather. In search and rescue missions, for example, adverse
environmental conditions are commonly encountered and result in degraded
sensing. In the DARPA Subterranean Tunnel Circuit, there are challenging
environments in unknown and long coal mine tunnels with smoke filled adverse
conditions that impair commonly used camera and LiDAR sensors.

Figure 1: Cross-modal contrastive learning of representations is proposed to
generate representations of range data from mmWave radars, which can operate
even in smoke-filled environments. The proposed method is compared with a
generative reconstruction model.

Millimeter wave (mmWave) radar is an alternative sensor that has promise for
overcoming these challenges that can operate despite variable weather and
lighting conditions [5, 6]. Recently, low-cost, low-power, and lightweight
single-chip mmWave radar has emerged as an innovative sensor modality. Methods
are proposed to provide state estimation [7] through neural networks or
reconstruct dense grid maps [8] with conditional GAN. Although the results
show promise, navigation using maps reconstructed from mmWave data faces
problems from sensor noise and multipath reflections, which may be mistakenly
rendered as obstacles on the reconstructed map. Another thought is to have the
RL agent learn the noise model of mmWave radars and correspond accordingly.
However, directly training an end-to-end deep RL policy network in a
simulation using mmWave data requires some channel models [9] that are not yet
available in commonly used simulation frameworks in the robotics literature.
Therefore, a large amount of mmWave training data from the real world for
offline RL training is needed which makes the method inefficient to almost
infeasible.

To this extent, we explored contrastive learning to learn cross-modal
representations between mmWave and LiDAR. Cross-modal representations were
analyzed in [10], where the authors proposed a variational autoencoder
(VAE)–based method, which was to be applied in real-world aerial navigation,
for learning robust visuomotor policies from simulated data alone. In our
study, we leveraged the contrastive learning framework by regarding mmWave
radar data as an augmentation of LiDAR data. Specifically, an encoder was
trained to learn a sufficient number of patterns in mmWave data for the RL
agent, which then navigated an unmanned ground vehicle (UGV) using only noisy
raw radar inputs with no reconstruction.

We used lightweight and low-cost single-chip mmWave sensors, which can operate
under challenging foggy or smoke-filled environments. To enable navigation
despite noisy and low-resolution mmWave radar inputs, we proposed approaches
for learning how to control UGVs and evaluated these approaches in a smoke-
filled environment. We summarize our contributions as follows:

  1. 1.

We show how cross-modal contrastive learning of representations (CM-CLR) can
be used to maximize the agreement between mmWave and LiDAR inputs: We
introduced contrastive loss into our network and formulated a training method
for achieving consistency between mmWave and LiDAR latent representations
during forward propagation. We collected a dataset comprising both mmWave and
LiDAR inputs in various indoor environments, such as corridors, open areas,
and parking lots. All LiDAR data were only used for training.

  2. 2.

We demonstrate that deep RL with cross-modal representations can be used for
navigation: The representation obtained from contrastive learning was
integrated into an end-to-end deep RL network $\pi_{con}$ (Fig. 1). We
demonstrated that inputs from only mmWave radar signals enabled navigation,
unlike LiDAR and camera signals, in visually challenging smoke-filled maze
environments.

  3. 3.

We evaluated our approaches in real-world smoke-filled environments: We
comprehensively evaluated a) the proposed end-to-end contrastive learning
method and b) deep generative models that generate LiDAR-like range data as
inputs for a deep RL–based control policy $\pi_{L}$ network. We used a
conditional generative adversarial network (cGAN) and VAEs, where raw mmWave
inputs are used to reconstruct dense LiDAR-like range data. We showed that the
navigation achieved by our proposed method can better avoid obstacles and
respond to situations in which the vehicle is trapped.

##  II Related Work

###  II-A Contrastive Learning

Contrastive learning is a type of learning framework where the model learns
patterns in a dataset, which are organized in terms of similar versus
dissimilar pairs, while subject to similarity constraints. In [11], a simple
contrastive learning framework (SimCLR) was proposed for learning visual
representations, with the application of simple data augmentation for
contrastive prediction tasks, and the authors incorporated cosine similarity
temperature-scaled cross-entropy loss into their method. The performance of
the proposed framework matched that of supervised learning methods. That study
further analyzed the effects of different data augmentation operators and
concluded that a combination of cropping and color distortions maximizes
learning efficiency. Another study [12] proposed the CURL framework for
learning with high-dimensional inputs (i.e., images), in which contrastive
learning, for representation learning, is combined with RL, for control policy
learning. The framework outperformed state-of-the-art image-based RL in
several DeepMind control suite tasks and Atari game tasks and even nearly
matched the sample efficiency of using state-based features for RL. In this
work, we considered mmWave data to be an augmentation of LiDAR data applied a
contrastive learning framework to learn cross-modal representations through
maximization of the agreement between encoded mmWave radar data and LiDAR
data.

###  II-B Deep Reinforcement Learning for Navigation

Recent studies have trained deep networks either through a pre-generated
occupancy map for motion commands or a mapless end-to-end approach that
relates range inputs to actions. Kahn et al. [13] demonstrated affordance
prediction for future paths with a network trained through self-supervised
learning, which allowed for navigation in paths that were previously regarded
as untraversable in the classic SLAM and planning approach. Niroui et al. [14]
presented a map-based frontier prediction network that was trained using the
RL framework in urban search and rescue applications. Both of these works
combined learning modules with classic control modules, in which deep networks
are not responsible for predicting motion commands. On the other hand, [15]
proposed a method for multi-robot navigation with a high-level voroni-based
planar and a low-level deep RL-based controller, where the RL agent is only
responsible for the controller. Mapless end-to-end approaches (e.g., in[3, 4,
16]) use range data as inputs to predict suitable motion commands through a
trained network. Pfeiffer et al. [3] developed a network that learns through
feature extraction in a convolutional neural network (CNN) layer and through
decision making in fully connected (FC) layers using data-driven behavioral
cloning. In another study [16], an agent network was trained through RL
algorithms [17, 18] in virtual environments. Another study [4] pretrained the
agent network using behavioral cloning prior to training with the RL
framework. Finally, in two studies[4, 16], further downsampling was performed
or fewer range data points were used to avoid overfitting or obtain better
sim-to-real performance.

However, all these studies used LiDAR devices or cameras as input sensors,
which are vulnerable in obscurant-filled environments. There has been earlier
work using an array of sonars [19, 20], which used the propagation of acoustic
energy and could penetrate smoke to sense obstacles from the environment [21].
However, low angular resolution and short range limited the applications.
Thus, we used mmWave radar instead of LiDAR for range sensing because mmWave
signals can penetrate smoke and fog. Our network architecture is similar to
that proposed in [3], which included feature extraction CNN layers and
decision-making FC layers.

##  III Deep Reinforcement Learning with Cross-Modal Contrastive Learning

###  III-A Pre-processing

We projected all range data to a two-dimensional (2D) plane with a 240∘ front-
facing field of view and 1∘ resolution. Our range data were obtained from
LiDAR as $x_{l}\in\mathbb{R}^{241}$ and mmWave as $x_{m}\in\mathbb{R}^{241}$
at each time step. Each of the four mmWave radars were running at 15 Hz, and
synchronized into 10 Hz. We then applied an outlier removal filter by
clustering the point clouds before projection. To do so, we first accumulated
5 consecutive frames to increase features. Then each point would be kept if
there exist 3 neighbor points within 1.2 meter radius. There may exist missing
value at a certain angles after applying the filtering, and we applied a
padding of a maximum range value of 5 m to ensure sufficient input dimensions.

(a) Sensor configuration 1.

(b) Sensor configuration 2.

Figure 2: Two configurations of mmWave radar modules placements mounted on a
Clearpath Husky. The field of views of both configurations were constrained to
240∘. In (a) the Velodyne 3D LiDAR was only for data collection. All
experiments and data collection in the paper were conducted using
configuration 1. (b) Configuration 2 was designed for the potential use of
smaller robots, and the discussions are described in Sec.V-E

###  III-B Contrastive Learning

Figure 3: Query encoder for mmWave radar range data $q_{mm}$ is used to
generate a representation $q$ that is similar to the representation $k$
generated by the key encoder $q_{L}$, which are trained from LiDAR range data
in simulation. The weights of $q_{L}$ are set unchanged during contrastive
learning.

To overcome the problem of noisy and sparse mmWave radar signals in
navigation, we proposed contrastive learning, the agreement between mmWave and
LiDAR inputs is maximized through the tuning of a trainable encoder with
contrastive loss.

Intuitively, learning a control policy can be done directly from mmWave-
obtained range data, without the use of LiDAR data. However, there exist two
issues of noise and lack of simulator access. Training using real-world data,
which lack false cases (e.g., running into walls), results in agents that have
a high frequency of collisions with obstacles. In contrastive learning, the
agent learns a good representation of some high-dimensional inputs through
different forms of data augmentation and contrastive loss. We regarded mmWave
range data as a noisy and randomly dropped augmentation of LiDAR range data,
and we trained an encoder using mmWave data to generate a latent
representation that is identical to that from the LiDAR ground truth. Because
the agent in the simulation has learnt a good navigation strategy with such a
representation, our new agent should, in theory, be capable of navigating
highly similar representations that are generated from the trained encoder and
mmWave input.

###  III-C Learning Using LiDAR Data for Navigation in a Virtual Subterranean
Environment

First, we aimed to train the agent to navigate in long corridors, tunnels, or
caves in subterranean environments using RL algorithms. The reward settings
were expected to encourage the agent to move forward and avoid collisions at
the same time to obtain high rewards. The agent should also receive some
rewards to keep moving and exploring. High angular velocity was expected to
decrease the rewards so that the agent will not make sharp turns unless there
was no opening sensed at front. The model was used as a low-level collision
avoidance controller instead of a high-level planar.

To do so, we used a variety of virtual subterranean environments [22] in
Gazebo simulation software, comprising an artificial tunnel with obstacles and
cliffs, a subway station, and a natural cave. We used a cave environment
because it has the most diverse sizes and shapes of tunnels and ramps. We
followed the OpenAI Gym coding style to integrate our gym environment with
Gazebo, analyze data on the agent’s observations, calculate rewards, assign
actions, and reset the agent. The settings and justifications for them are as
follows:

  * •

Observations: The three-dimensional simulated LiDAR inputs constituted an
observation space. As stated in Section III, we sampled the range data for
$-120^{\circ}$ to $120^{\circ}$, with a $1^{\circ}$ resolution from the point
cloud of a height reaching the UGV’s height, resulting in a total of 241 range
values. The agent is also given the information of motion direction, the
relative 2D position of the agent is defined with respect to the previous time
point by $x_{p}\in\mathbb{R}^{2}$. The total observation $O$ of our RL agent
is the combination of $x_{l}$ and $x_{p}$, which has the dimension of
$O\in\mathbb{R}^{243}$.

  * •

Actions: A continuous space of linear and angular velocities was used to
represent actions. The linear velocity space was normalized to $[0,1]$, and
the angular velocity space was normalized to $[-1,1]$. Angular velocity was
represented as a continuous rather than discrete space to allow for smooth
motion (which is desirable), and linear velocity was represented to prevent
the agent from moving backward.

  * •

Reward: To encourage the agent to navigate indefinitely, we designed the
rewards according to three preference. The first, $r\\_action$, incentivizes
the agent to move straight and perform as few turns as possible. The second,
$r\\_move$, incentivizes the agent to keep moving. Finally, the third,
$r\\_collision$, disincentivizes the agent from colliding into obstacles,
which results in the end of an episode. Specifically, these three rewards and
their combination were designed as follows.

$\displaystyle r\\_action=2^{(1-|\omega|)\times 5}\text{, where }\omega\text{
is the angular velocity}$ $\displaystyle
r\\_move=\begin{cases}10\quad{if}\quad\text{the distance}\\\
\quad\quad\quad\quad\text{between frames}>0.04m\\\ -10\quad{else}\end{cases}$
$\displaystyle r\\_collision=\begin{cases}-50\quad{if}\quad\text{collision}\\\
0\quad{else}\end{cases}$ $\displaystyle r=\lambda\times
r\\_action+r\\_move+r\\_collision$

The reward value that was sent to the agent was converted using a log scale:
$r\\_agent=sign(r)\times log(1+|r|)$.

The deep deterministic policy gradient (DDPG) [17] is an off-policy RL
algorithm that trains a deterministic policy to be the agent’s control policy.
The DDPG uses an actor-critic method to learn a deterministic policy
$\pi^{\theta}$ with parameter $\theta$ and Q-value estimator $Q^{\phi}$ with
parameter $\phi$. In contrast to the DDPG, where the network is updated using
transitions $t=(o_{t},a,r,o_{t+1})$ sampled from the experience replay buffer,
the recurrent deterministic policy gradient (RDPG) algorithm [23] uses the
recurrent network and trains the network with an entire trajectory history
$h_{t}=(o_{1},a_{1},o_{2},a_{2},\dots,a_{t-1},o_{t})$ and back propagation
through time. The critic network updates $\phi$ to minimize the Bellman error,
and the actor network’s objective is to maximize $Q^{\phi}(h_{t},a_{t})$.

We used DDPG and RDPG algorithms to train collision avoidance policies with
data collected in the Gazebo virtual subterranean environment. Our neural
network architecture comprised minimum pooling for downsampling the inputs,
two convolution layers for feature extraction, two FC layers, one recurrent
layer with an LSTM cell, and one FC layer to generate action outputs. The
network architecture was similar to those of [3] and [24], which used three
consecutive frames, whereas our network includes LSTM cells. Studies have
reported the importance of including a recurrent network when learning
sequential actions. Training process took 55 hours for 4 million steps on an
Intel i7-8700 CPU with Nvidia RTX2070 GPU. We denoted our control policy
trained in the simulation using LiDAR data as $\pi_{L}$.

###  III-D Navigation With Policy From Contrastive Learning

To calculate the contrastive loss, we separated the original RL network into a
feature extraction network, which comprised the convolution layers, and a
decision-making network, which comprised the FC layers and the recurrent
layer. The goal of the contrastive learning was to maximize the similarity of
the encoded representations between the mmWave input and the matching ground-
truth LiDAR input; this was done to allow the mmWave representation to be used
by the decision-making network, which was trained with RL in the simulation.
Thus, the key encoder $q_{L}(k|x_{l})$ and the query encoder $q_{mm}(q|x_{m})$
of contrastive learning were designed as the architecture of the feature
extraction layers of the RL policy network. During contrastive learning, real-
world LiDAR range data were used as the input for the key encoder. The mmWave
range data, which can be construed as an augmentation of ground truth LiDAR
data, were taken as the input of the query encoder. In CURL[12], the bilinear
inner product with a momentum key encoder was used for training with
contrastive loss. However, we did not want to change the key encoder, which
was already trained with RL during simulation. We used a fixed key encoder
with euclidean distances to measure the similarity between $k$ and $q$.

$\mathcal{L}_{contrastive}=\mathbb{E}_{x_{l},x_{m}}[(q_{mm}(q|x_{m})-q_{L}(k|x_{l}))^{2}]$

We trained the network with the collected dataset to converge (20 epochs), and
stopped the training to prevent overfitting. The performance of the trained
mmWave control policy $\pi_{con}$ was evaluated in a real-world maze.

##  IV Generative Reconstruction Baselines

We considered generative reconstructions of mmWave range data into LiDAR-like
data to serve as a key baseline because prior work [8] showed promising
results. We implemented two state-of-the-art supervised deep generative
models, which were later combined with control policies for navigation tasks.

###  IV-A Conditional Generative Adversarial Network for Range Data

A cGAN comprises a generator $G$ and a discriminator $D$. Given an mmWave
range data point $x_{m}$ and a random vector $z$ as the input, the generator
attempts to generate a much clearer and denser range data point $y$ that is
similar to the ground truth LiDAR range data point $x_{l}$. By contrast, the
discriminator attempts to distinguish the real LiDAR range data point $x_{l}$
generated from $y$. The objective of a cGAN can be expressed as follows.

$\mathcal{L}_{cGAN}(G,D)=\mathbb{E}_{x_{m},y}[logD(x_{m},y)]+\\\
\mathbb{E}_{x_{m},z}[log(1-D(x_{m},G(x_{m},z)))]$

To make the generated range data even more similar to the LiDAR ground truth,
we adapted L1 loss in the objective function following the pix2pix approach of
a previous work[25].

$\mathcal{L}_{L1}(G)=\mathbb{E}_{x_{m},y,z}[|y-G(x_{m},z)|]$

The final objective function for the generator is as follows.

$\mathcal{L}_{total}=\mathcal{L}_{cGAN}+\lambda_{1}\mathcal{L}_{L1}$

We designed the generator to have an encoder–decoder structure. The encoder
has the same convolution feature extraction architecture as that of our deep
RL networks. For the decoder model, we implemented three deconvolution layers
and an FC layer to adjust the output dimension. For our discriminator, we used
the Markovian discriminator technique[26], also called patchGAN in
pix2pix[25]. The patched discriminator has four layers of one-dimensional
convolution for feature extraction, and the output is a $1\times 14$ patch. We
also tested a nonpatched discriminator that outputs a single value to
discriminate real objects from counterfeit ones. We found that the nonpatched
discriminator had slightly better L1 performance. We trained the model using
our collected dataset. The data collection process is described in Section
V-A.

###  IV-B Variational Autoencoder

In addition to a cGAN, we used a similar network architecture to train a VAE
[27], which comprised a probabilistic encoder $q(z|m)$ and a generative model
$p(l|z)$, where $z$ is the latent variable, sampled from the output of the
encoder. The objective function of the VAE minimizes the variational lower
bound of the log likelihood. In [28], the objective function was interpreted
as accounting for probabilities across data modalities. The objective function
in our case is as follows.

$\mathbb{E}_{z\sim
q(z|x_{m})}\left[logp(x_{l}|z)-D_{KL}(q(z|x_{m})||p(z))\right]$

##  V Evaluation

###  V-A Data Collection

Range data from four 60-GHz TI 6843 mmWave radar modules and a Velodyne VLP-16
LiDAR and motion commands were recorded at a frame rate of 5 Hz as training
data using a Clearpath Husky. A total of 110 381 frames (approximately 6 robot
hours) were recorded. The experimental environments included a corridor, a
parking area, and outdoor open areas. The indoor corridors had widths from 2
to 4 m; the parking area had a wide passage with a maximum width of 6 m, and
the outdoor open area had a maximum width of 10 m. All data in this study,
which comprised the aforementioned types of data as well as images and wheel
odometry, are open source and in ROS bag and hdf5 file formats.

Figure 4: Raw mmWave range data and ground truth LiDAR range data collected in
different environments with different reconstruction methods.

(a) A maze environment built by cardboard.

(b) A general indoor environment.

Figure 5: Navigation experiments and visualized trajectories of various
control policies. (a) A maze environment built by cardboard. The robot was
trapped due to the smoke (LiDAR + $\pi_{L}$) or phantom walls
(mmWave-$cGAN+\pi_{L}$). (b) A general indoor environment, where
mmWave-$cGAN+\pi_{L}$ has some cases of collisions and instances of being
trapped caused by reconstruction errors.

###  V-B Reconstruction Results

In addition to the previously mentioned reconstruction methods, the polynomial
curve fitting method in the polar coordinate was used as a baseline. Among the
1st-order to 15th-order curves, the 8th-order curve had the lowest L1 distance
value. To evaluate the performance of different reconstruction methods, we
compared the L1 distance between the reconstructed mmWave data and the ground
truth LiDAR data. The results are displayed in Table I. The qualitative
results of different reconstruction methods are illustrated in Fig. 4. Data
collected in scenes that were from a corridor or parking lot and were not a
maze used in the training steps were all considered in the evaluation.

TABLE I: L1 loss of various reconstruction methods in different environments. Method | Corridor | Parking Lot | Maze
---|---|---|---
raw mmWave | 1.99 | 0.39 | 2.53
curve fitting | 0.75 | 1.52 | 1.84
VAE | 0.23 | 0.27 | 1.65
cGAN | 0.09 | 0.11 | 1.53

As indicated in Table I, the raw mmWave data had a lower L1 distance in the
parking lot than in the other environments. This is because the parking lot
had wider passages, which caused most range data points to exceed the
stipulated maximum distance of 5 m. Thus, most mmWave data and LiDAR ground
truth data points were forced to be at 5 m, which resulted in a low L1
distance, regardless of whether the value was absent or higher. Among all the
reconstruction methods, the cGAN had the lowest L1 distance, which implies
that the reconstruction data of the cGAN were most similar to the ground truth
LiDAR data. Both the cGAN and VAE could generate geometries that were similar
to the ground truth, but the cGAN outperformed the VAE by generating
straighter, smoother, and more consistent walls and fewer “phantom walls.” We
found that both methods had difficulty reconstructing the exact shapes of cars
in the parking lot scene using sparse mmWave data. By contrast, reconstructing
the geometry of corridors was easy, as indicated by the lower L1 loss in the
corridor scene. We noted that the average performance decreased in our test
scene in the maze of the underground basement. This might be because the
mmWave signals could penetrate the cardboard, resulting in inaccurate
reconstructions.

###  V-C Navigation in a Smoke-Filled Maze

####  V-C1 Experiment Environment

The testing environment was a cardboard maze in a basement (23 m $\times$ 28
m) with widths ranging from 2.25 m to 6 m, and the walls were 1.78 m high. One
lap was approximately 100 m long and contained 20 turns, which were mostly
$90^{\circ}$ turns, except for some sharp turns. We used four chairs and one
mannequin as obstacles. The robot navigated in the maze, similar to that in
the tunnel without intersections. The LeGO-LOAM method[29] was applied with
the LiDAR data to obtain robot trajectories; the trajectories of some methods
are plotted in Fig. 5 for qualitative evaluation.

####  V-C2 Metrics

Robot navigation failures were quantified according to the number of
collisions and the number of instances of being trapped, which were a result
of noisiness or sparseness in the mmWave radar data. Specifically, the robot
collided into an obstacle when the obstacle was undetected and became trapped
when the range estimates were inaccurate, which could be due, for example, to
phantom walls. We also recorded odometry results from whee and LiDAR, but the
distance traveled did not significantly differ between the methods evaluated.

####  V-C3 Ground truth

We used navigation with LiDAR data, using both the RL control policy $\pi_{L}$
and classic A* planning $\pi_{cl}$ in a smoke-free environment, as the ground
truth. The goal points of classic planning were set to follow the walls, and a
pure pursuit algorithm and PID controller were used. Both methods resulted in
the robot safely traversing the maze without colliding or being trapped.

####  V-C4 Smoke trials

A smoke machine emitted dense smoke at three locations (Fig. 5) for 15s before
the UGV arrived. To adhere to safety regulations, we did not fill the entire
basement with smoke. We conducted a few smoke trials for $\pi_{L}$,
$cGAN+\pi_{L}$, and $\pi_{con}$ to observe how the smoke affected the control
policies. The smoke prevented the LiDAR signals from penetrating the passage,
which caused the UGV to be trapped. We found that navigation methods using
mmWave radar were unaffected by smoke.

TABLE II: Navigation performance in maze with respect to number of collisions and instances of being trapped.  Inputs | Methods |  | Avg.
---
Coll.
| Avg.
---
Trapped
| Env.
---
Note
LiDAR | $\pi_{L}$ | 0 | 0 | No Smoke, GT
| $\pi_{cl}$ | 0 | 0 | No Smoke, GT
| $\pi_{L}$ | 0 | 3 | Trapped in Smoke
mmWave | $\pi_{cl}$ | - | - | Unable to navigate
| $cGAN+\pi_{cl}$ | 1.60 | 1.60 | Collision and trapped
| $\pi_{L}$ | 1.20 | 0.60 | Baseline - w/o CM
| $cGAN+\pi_{L}$ | 0 | 1.17 | Baseline - generative
| $\pi_{con}$ | 0 | 0 | Ours - contrastive

####  V-C5 Generative reconstruction baselines

For each method, the test was run for five laps, and we recorded the number of
collisions and instances of being trapped. We evaluated the classic $\pi_{cl}$
and RL $\pi_{L}$ approaches, comparing them with $\pi_{cl}$ and $\pi_{L}$
enhanced by a reconstruction method (i.e., cGAN) using direct raw mmWave
inputs. We found that the local map constructed using raw mmWave inputs lacked
occupancy, resulting in a failure to navigate from the use of $\pi_{cl}$.
Conversely, through the use of the cGAN to reconstruct the range data, the
robot could navigate using the classic planning method, albeit with
considerable collisions and instances of being trapped due to reconstruction
errors. The control policy $\pi_{L}$ was more robust to noisy raw mmWave
inputs without any tuning of the neural network, and $\pi_{L}$ managed to
navigate through the maze even without reconstruction. When $cGAN+\pi_{L}$ was
used, the robot collided less frequently but got trapped more frequently.

####  V-C6 Cross-modal contrastive learning of representations

$\pi_{con}$ performed the best among all of the evaluated methods, where the
deep RL control policy was used after training with contrastive loss.
$\pi_{con}$, as an end-to-end network, allowed the robot to safely traverse
the entire maze in all five trials without colliding or being trapped.

###  V-D Navigation in a General Indoor Corridor

To further evaluate the navigation performance in a general indoor
environment, we conducted the second experiment in a corridor located at
another building which was not used for training data collection. The
environment has two major corridors of 30 and 50 meters, respectively, two
$90^{\circ}$ turns and two narrow gates. For safety considerations, we did not
emit smoke in this environment. The details of this environment are shown in
Fig. 5.

Figure 6: We used truncated LiDAR baseline to emulate sonar ring inputs (green
line). The emulated sonar ring has $15^{\circ}$ resolution and 3 meters of
maximum range. In the experiments we found that low angular resolution made it
difficult to find an opening to navigate through narrow passage (left); short
range is also a disadvantage of making turns (right).

####  V-D1 Truncated LiDAR (Emulated Sonar Ring) baseline

In addition to previously mentioned baselines for navigation, we introduce a
truncated LiDAR baseline to emulate a sonar ring to be compared against. In
the review [21] a sonar ring was in the resolution of $15^{\circ}$ resolution
and a maximum range of 3 meters. Following such settings, we carried out an
experiment with decimated and truncated LiDAR measurements of 24 beams up to 3
meters. For the size of the Husky robot (67cm in width) low angular resolution
makes it difficult for the robot to find an opening in a relatively narrow
passage (1.45 meters in width in our experiment). Short range measurement is
also a disadvantage while making turns, shown in Fig. 6. The quantitative
results in Table III showed that the robot was trapped at both of the narrow
passages.

####  V-D2 Generative reconstruction baselines

With the help of the reconstruction model, both $\pi_{cl}$ and $\pi_{L}$ are
able to navigate through the corridor with some collisions and instances of
being trapped, but they happened more frequently than in the maze environment.
We observed that the reconstruction model sometime generates corridor geometry
with errors in angle compared to the ground truth when the vehicle is
$10^{\circ}\sim 45^{\circ}$ relative to the wall. The errors in angle caused
minor reconstruction loss but misled the vehicle to finally collide into the
wall. This implies our dataset should be more diverse with more data of the
robot not parallel to the walls to improve the generative reconstruction
performance.

####  V-D3 Cross-modal contrastive learning

$\pi_{con}$ also performed the best in this environment. Only one instance of
being trapped happened among all five trials when robot navigate to the narrow
gate in the environment.

TABLE III: Navigation performance at indoor corridor. Inputs | Methods |  | Avg.
---
Coll.
| Avg.
---
Trapped
| Env.
---
Note
LiDAR | $\pi_{L}$ | 0 | 0 | No Smoke, GT
| $\pi_{cl}$ | 0 | 0 | No Smoke, GT
Truncated | $\pi_{cl}$ | 0 | 2 | Emulate Sonar
mmWave | $\pi_{cl}$ | - | - | Unable to navigate
| $cGAN+\pi_{cl}$ | 3.2 | 1 | Collision and trapped
| $\pi_{L}$ | 7 | 0 | Baseline - w/o CM
| $cGAN+\pi_{L}$ | 0.6 | 0.6 | Baseline - generative
| $\pi_{con}$ | 0 | 0.2 | Ours - contrastive

###  V-E General Discussions

We found that the results of Table II and III are consistent that the proposed
contrastive learning obtained better performance than baseline methods. The
results may indicate that learning the representation may be more robust
across different environments. We conjecture that for $cGAN+\pi_{L}$ to
achieve the same performance as $\pi_{con}$, it is better for the
reconstructed mmWave data to be similar to detailed LiDAR data. Without such
similarity, the feature extraction network in $\pi_{L}$ may generate
misleading representations for the decision-making network. The results of
generative baseline in different environments were found to decrease the
number of trapped cases, and a slightly increased number of collisions in
dense wall environments. The results may indicate that the generative
reconstruction in different environments may be sensitive to the training
dataset, which is consistent with the results of Table I. Future work could
collect more training data in diverse environments to further verify the
influences of training data to learn reconstruction.

We had tested two different mmWave sensor configurations on the same robot
(Husky), shown in Fig. 2. Configuration 1 was used to collect training data
and conduct the experiments, and configuration 2 was designed for potential
use of smaller robots. We found that the different mmWave sensor
configurations did not affect navigation performance on the same Husky robot.
However, We suggest that additional transfer learning efforts considering
robot size, speed, and reward settings may be needed to apply the method to a
different robot.

An alternative to contrastive learning is hallucination, which has been used
in other cross-modal tasks [30, 31]. [30] training a modality hallucination
architecture to learn a multimodal convolutional network for object detection.
The hallucination was trained to take an RGB input image and mimic the depth
mid-level activations. Cross-modal studies on high-bandwidth inputs like
images and low-bandwidth inputs (LiDAR/mmWave) could be further studied.

##  VI Conclusions

Learning how to navigate is difficult under adverse environmental conditions,
where sensing data are degraded. As a remedy, we leveraged mmWave radars and
proposed a method that uses cross-modal contrastive learning of
representations with both simulated and real-world data. Our method allows for
automated navigation in obscurant-filled environments using data from only
single-chip mmWave radars. In a comprehensive evaluation, we incorporated
state-of-the-art generative reconstruction with a control policy for
navigation tasks. Our method performed best in the smoke-filled environment
against its counterparts, with results that were comparable to those from
groundtruth LiDAR approaches in a smoke-free environment. Future studies can
apply our method to learning across modes and platforms.

## Acknowledgments

The research was supported by Taiwan’s Ministry of Science and Technology
(grants 109-2321-B-009-006, 109-2224-E-007-004, and 109-2221-E-009-074). This
work was funded in part by Qualcomm through the Taiwan University Research
Collaboration Project.

## References

  * [1] M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik, D. Parikh, and D. Batra, “Habitat: A Platform for Embodied AI Research,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2019.
  * [2] J. Zhang, J. T. Springenberg, J. Boedecker, and W. Burgard, “Deep reinforcement learning with successor features for navigation across similar environments,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2017, pp. 2371–2378.
  * [3] M. Pfeiffer, M. Schaeuble, J. Nieto, R. Siegwart, and C. Cadena, “From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots,” in _2017 ieee international conference on robotics and automation (icra)_. IEEE, 2017, pp. 1527–1533.
  * [4] M. Pfeiffer, S. Shukla, M. Turchetta, C. Cadena, A. Krause, R. Siegwart, and J. Nieto, “Reinforced imitation: Sample efficient deep reinforcement learning for mapless navigation by leveraging prior demonstrations,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, pp. 4423–4430, 2018\.
  * [5] S. H. Cen and P. Newman, “Radar-only ego-motion estimation in difficult settings via graph matching,” _arXiv preprint arXiv:1904.11476_ , 2019.
  * [6] ——, “Precise ego-motion estimation with millimeter-wave radar under diverse and challenging conditions,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2018, pp. 1–8.
  * [7] Y. Almalioglu, M. Turan, C. X. Lu, N. Trigoni, and A. Markham, “Milli-rio: Ego-motion estimation with low-cost millimetre-wave radar,” _IEEE Sensors Journal_ , 2020.
  * [8] C. X. Lu, S. Rosa, P. Zhao, B. Wang, C. Chen, J. A. Stankovic, N. Trigoni, and A. Markham, “See through smoke: Robust indoor mapping with low-cost mmwave radar,” in _ACM International Conference on Mobile Systems, Applications, and Services (MobiSys)_ , 2020.
  * [9] I. A. Hemadeh, K. Satyanarayana, M. El-Hajjar, and L. Hanzo, “Millimeter-wave communications: Physical channel models, design considerations, antenna constructions, and link-budget,” _IEEE Communications Surveys & Tutorials_, vol. 20, no. 2, pp. 870–913, 2017.
  * [10] R. Bonatti, R. Madaan, V. Vineet, S. Scherer, and A. Kapoor, “Learning controls using cross-modal representations: Bridging simulation and reality for drone racing,” _arXiv preprint arXiv:1909.06993_ , 2019.
  * [11] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” _arXiv preprint arXiv:2002.05709_ , 2020.
  * [12] M. Laskin, A. Srinivas, and P. Abbeel, “Curl: Contrastive unsupervised representations for reinforcement learning,” _Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria, PMLR 119_ , 2020, arXiv:2003.06417.
  * [13] G. Kahn, P. Abbeel, and S. Levine, “Badgr: An autonomous self-supervised learning-based navigation system,” 2020.
  * [14] F. Niroui, K. Zhang, Z. Kashino, and G. Nejat, “Deep reinforcement learning robot for search and rescue applications: Exploration in unknown cluttered environments,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 610–617, 2019.
  * [15] J. Hu, H. Niu, J. Carrasco, B. Lennox, and F. Arvin, “Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning,” _IEEE Transactions on Vehicular Technology_ , pp. 1–1, 2020.
  * [16] L. Tai, G. Paolo, and M. Liu, “Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2017, pp. 31–36.
  * [17] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning.” in _ICLR_ , Y. Bengio and Y. LeCun, Eds., 2016.
  * [18] S. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” in _2017 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2017, pp. 3389–3396.
  * [19] S. Walter, “The sonar ring: Obstacle detection for a mobile robot,” in _Proceedings. 1987 IEEE International Conference on Robotics and Automation_ , vol. 4. IEEE, 1987, pp. 1574–1579.
  * [20] S. Fazli and L. Kleeman, “Wall following and obstacle avoidance results from a multi-dsp sonar ring on a mobile robot,” in _IEEE International Conference Mechatronics and Automation, 2005_ , vol. 1. IEEE, 2005, pp. 432–437.
  * [21] L. Kleeman and R. Kuc, _Sonar Sensing_. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 491–519. [Online]. Available: https://doi.org/10.1007/978-3-540-30301-5_22
  * [22] O. Robotics. (2016) Darpa subterranean challenge virtual competition. [Online]. Available: https://bitbucket.org/osrf/subt/wiki/Home
  * [23] N. Heess, J. J. Hunt, T. P. Lillicrap, and D. Silver, “Memory-based control with recurrent neural networks,” _CoRR_ , vol. abs/1512.04455, 2015. [Online]. Available: http://arxiv.org/abs/1512.04455
  * [24] T. Fan, P. Long, W. Liu, and J. Pan, “Fully distributed multi-robot collision avoidance via deep reinforcement learning for safe and efficient navigation in complex scenarios,” _CoRR_ , vol. abs/1808.03841, 2018. [Online]. Available: http://arxiv.org/abs/1808.03841
  * [25] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , July 2017.
  * [26] C. Li and M. Wand, “Precomputed real-time texture synthesis with markovian generative adversarial networks,” in _Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III_ , ser. Lecture Notes in Computer Science, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds., vol. 9907. Springer, 2016, pp. 702–716. [Online]. Available: https://doi.org/10.1007/978-3-319-46487-9_43
  * [27] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” _CoRR_ , vol. abs/1312.6114, 2014.
  * [28] A. Spurr, J. Song, S. Park, and O. Hilliges, “Cross-modal deep variational hand pose estimation,” in _CVPR_ , 2018.
  * [29] T. Shan and B. Englot, “LeGO-LOAM: Lightweight and ground-optimized lidar odometry and mapping on variable terrain,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 4758–4765.
  * [30] J. Hoffman, S. Gupta, and T. Darrell, “Learning with side information through modality hallucination,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 826–834.
  * [31] M. R. U. Saputra, P. P. B. de Gusmao, C. X. Lu, Y. Almalioglu, S. Rosa, C. Chen, J. Wahlström, W. Wang, A. Markham, and N. Trigoni, “Deeptio: A deep thermal-inertial odometry with visual hallucination,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 1672–1679, 2020.