# Underwater Acoustic Ranging Between Smartphones

Tuochao Chen,⋄ Justin Chan,⋄ Shyamnath Gollakota  ⋄Co-primary student
authorsPaul G. Allen School of Computer Science & EngineeringUniversity of
Washington, Seattle, WA, USA

††copyright: none††doi: ††isbn: ††conference: ; 2022††journalyear: ;††price:

Abstract— We present a novel underwater system that can perform acoustic
ranging between commodity smartphones. To achieve this, we design a real-time
underwater ranging protocol that computes the time-of-flight between
smartphones. To address the severe underwater multipath, we present a dual-
microphone optimization algorithm that can more reliably identify the direct
path. Our underwater evaluations show that our system has median errors of
0.48–0.86 m at distances upto 35 m. Further, our system can operate across
smartphone model pairs and works in the presence of clock drifts. While
existing underwater localization research is targeted for custom hydrophone
hardware, we believe that our work breaks new ground by demonstrating a path
to bringing underwater ranging capabilities to billions of existing
smartphones, without additional hardware.

##  1\. Introduction

Scuba diving is a popular underwater group activity that millions participate
in each year (sta, 2021). Maintaining close proximity with a dive buddy or
between a dive instructor and the other divers is critical to ensure that the
divers are able to help each other in the event of an emergency such as
failure of diving equipment, injury or being trapped by ropes or nets (fai,
2020; net, 2011). This can be challenging in low visibility situations such as
turbid waters or during a silt out, which can cause divers to become
disoriented and separated from their dive buddy or instructor (bud, 2018; mud,
2004). Estimates show that around 86% of scuba diving fatalities occurred when
the divers were diving solo or got separated from their buddy (dea, 2010).

Ideally, we need a solution where the diver’s hardware can compute the
distance from their buddy, even in turbid waters, and alert the diver if they
go beyond a pre-set range. While dive lights and other signaling hardware are
used to get the attention of other dive members (lig, 2020) and dive compasses
are used for navigation (com, 2020), they do not provide the distance
information between divers. Indeed, underwater localization is an active area
of research for dive computers, sensor networks and robotics (Tan et al.,
2010; Tomczak, 2011; Zhao et al., 2018). These systems use anchor nodes such
as floating buoys with known locations to estimate distance with accuracies of
0.6–3m (Vickery, 1998; Cario et al., 2021a; Ullah et al., 2019). This prior
work, however, requires powerful custom hardware that is not ubiquitous and
does not have economies of scale.

In this paper, we take on an under-explored research question: can we achieve
underwater ranging between commodity smartphones? Smartphones are universally
pervasive across the world and have the economy of scale that is lacking in
custom hydrophone hardware. Furthermore, smartphones are increasingly being
used with diving-proof cases for underwater video logging and as a replacement
for a dive computer (div, 2021). In principle, anyone with a smartphone should
be able to download a software app before their diving activities and use it
to track the distance from their buddy or dive instructor during underwater
activities. Smartphones also have displays and vibration motors that can be
used to alert the diver when they go beyond a pre-set distance from their
buddy. Achieving this, without the need for custom hardware, can bring
underwater ranging to billions of existing smartphones using only software and
has potential use in tens of millions of dives that occur each year.

Figure 1. Underwater acoustic ranging using smartphones. Severe reflections
and echoes from the phone’s waterproof case and the environment like the
waterbed, surface, and aquatic life create a challenging multipath environment
for time-of-flight ranging.

We introduce AquaRanger, a novel underwater ranging system that can compute
the distance between smartphones using time-of-arrival techniques. At a high
level, we use the microphones and speakers that are ubiquitous on smartphones
to enable acoustic ranging between these mobile devices. While in-air acoustic
ranging has been well-explored in the research community (Peng et al., 2007;
Jin et al., 2015), acoustic signals propagate at around 4.5x higher speed in
water (1500 m/s) than air (330 m/s), which correspondingly reduces the time-
of-flight resolution. Commodity smartphones and underwater conditions impose
three additional technical challenges.

  * •

Water-proof cases create atypical multipath profiles. A fundamental assumption
that in-air acoustic tracking algorithms make is that when two devices are in
line-of-sight and have no nearby reflecting surfaces, the multipath profile is
clean with the direct path being the strongest and earliest path (Mao et al.,
2016; Wang and Gollakota, 2019a; Peng et al., 2007; Jin et al., 2015). Water-
proof smartphone cases however result in atypical multipath due to
interactions with the case material and echoes within the case. As a result,
even when the two case-enclosed devices are operating in air, are in line-of-
sight and have no nearby reflecting surfaces, the direct path may no longer be
the strongest path (Fig. 2a,b). Further, peaks in the underwater multipath
profile can appear before the direct path due to noise and other factors (Fig.
2c). So, we can not rely on the assumption that the first non-negligible peak
in the multipath profile is the direct path.

  * •

Underwater reflections create challenging multipath. Underwater channels are
known for their challenging multipath environments where the acoustic signals
bounce back and forth between the waterbed and the surface as well as from
aquatic animals (e.g., fish) and plants. Further, particles suspended in the
water can scatter the signal. The speed of sound also spreads these
reflections across time causing a large delay spread (Ghaffarivardavagh et
al., 2020). Smartphones have a limited bandwidth of 3-4 kHz underwater (see
§2.2.2) and a low sampling rate of 44.1 kHz compared to commodity hydrophones.
This makes it challenging to disambiguate the direct path with high resolution
on a commodity smartphone.

  * •

Buffering delays and synchronization issues. Unlike hydrophones that are
custom designed for underwater operations, smartphone microphones and speakers
are primarily designed for speech. In addition to the underwater frequency
response of these sensors varying across smartphone hardware, the microphone
and speaker buffers on each smartphone are not synced with each other. These
buffers are filled in independently by the OS and so we may not know the
timestamps corresponding to the samples in the two buffers. Furthermore, some
of these phones can use a different clock for the two buffers resulting in
slightly different sampling rates. One-way in-air acoustic tracking systems
(Cai et al., 2018; Youssef et al., 2006; Wang and Gollakota, 2019b)
synchronize the clocks on the two smartphones in advance but can suffer from
clock drifts over time. Two-way systems like BeepBeep (Peng et al., 2007) and
Swordfight (Zhang et al., 2012) address this problem but use an RF channel to
synchronize between the phones by sharing the buffer timestamp information.
However, radio signals attenuate significantly underwater and cannot be used
as an out-of-band synchronization channel.

Figure 2. Acoustic channel estimates between two smartphones in open air with
(a) the plastic case and (b) the waterproof pouch. (c) shows an example
underwater channel between the smartphones.

We address the above challenges and present a novel underwater ranging system
for commodity smartphones. At a high level, to address the atypical multipath
from the water-proof case and the severe underwater reflections, we use the
two microphones on either ends of a smartphone. Smartphones use the bottom
microphone close to the user’s mouth to capture their voice while the top
microphone closer to their ear is used to perform noise cancellation. Our key
intuition is that the time difference of arrival for the direct path at the
bottom and top microphones is upper bounded by the physical distance between
them. Thus, the sample offset between the direct path in the top and the
bottom microphone channels should be lower than the acoustic propagation time
between them. Furthermore, given their spatial separation, the multipath
created by the water-proof case is different at the two microphones. Finally,
the hardware at the two microphones can have a different noise profile. Thus,
our approach identifies the direct path as the earliest non-negligible peaks
in both channels whose sample offset satisfies the physical distance
constraint between the two microphones. In §2.2.2, we present a light-weight
algorithm to perform this dual-microphone optimization and identify the direct
path.

We also design a two-way underwater ranging protocol for smartphones that does
not require explicit information exchange about timestamps. We use a self-
synchronization mechanism that synchronizes the microphone and speaker buffers
at each phone and corrects for the buffering delays introduced by the OS (see
§2.2.1). The phones auto-adjust their response time to account for the
buffering delays to eliminate the need for explicitly exchanging this
information. We show that the errors introduced by our mechanism can be
minimized and that our approach does not suffer from clock drifts between the
two smartphones.

We implemented our software system in real-time on the Android platform. We
also designed a query-response based protocol to support underwater ranging of
multiple devices (see §2.3). We evaluated our system in four different
underwater environments including a busy swimming pool in the presence of
other swimmers, an outdoor fishing dock with a depth of 9 m, a waterfront of a
park and a boathouse on a lake with people fishing and kayaking close to the
dock.

Our findings are as follows:

  * •

The median errors achieved by our system were 0.48, 0.80 and 0.86 m at 10, 20
and 35 m respectively. The 95% errors were 0.83, 1.02 and 1.51 m. When the
distance increased to 45 m, the median and 95% errors increased to 1.67 and
2.57 m.

  * •

In the busy swimming pool with other swimmers, the median and 95% errors were
0.35 and 0.84 m. When the phone itself was moving across a 5–18 m distance,
the median and 95% errors were 0.45 and 1.12 m.

  * •

Testing with Samsung Galaxy S9, Google Pixel 3a, and OnePlus 8 Pro shows that
the median and 95% errors between phone model pairs were 0.28-0.54 m and
0.41-0.75 m at 20 m.

Contributions. Existing underwater localization research has been focused on
custom hardware for sensors, robotics and dive computers (Tan et al., 2010;
Tomczak, 2011; Zhao et al., 2018). In contrast, we present a real-time
underwater system for ranging between commodity smartphones, without any
custom hardware. Our software-only system makes multiple contributions. First,
we identify the atypical multipath profiles caused by water-proof smartphone
cases that make it challenging to identify the direct path. Second, we
introduce a dual-microphone optimization technique that can more reliably
identify the direct path in the presence of underwater noise and multipath.
Third, we design a two-way underwater ranging protocol that does not require
exchanging timestamps and addresses the phone OS buffering delays. Finally, we
evaluate our system in various underwater conditions and demonstrate its
feasibility.

##  2\. AquaRanger

We first present the characterization of the channel resulting from the water-
proof case and then describe our system.

###  2.1. Multipath with water-proof case

We explore the effects of water-proof cases on the acoustic channel. We
perform in-air experiments by placing two phones 10 cm away in an open space
with no nearby reflectors. We experiment with two different case material as
shown in Fig. 2. We run three groups of experiments: the phones without case,
phones with the first case, and phones with the second case. The first phone
sends a 100 ms OFDM symbol between 1–5 kHz to the second phone where we
perform channel estimation. Figs. 2a-b show the estimated channel with and
without the two different case materials. The plots show that while the direct
path is the strongest without the case, the multipath profile looks a lot more
atypical with the cases. This is despite the devices being close to each other
and not having a close-by reflector. In addition, different case materials
lead to different channels, which makes it hard to predict and compensate for
the water-proof case since the phone can move around a bit within it.

In addition to the in-air experiments, we also performed channel estimation
underwater. We put the two phones with the first water-proof case into the
water. The phones are separated by 20 m and are at a depth of 2 m in a 9 m
deep water body. Fig. 2c shows that the multi-path profile is dense compared
to the air scenario; the former is affected by both the water-proof case and
the severe underwater reflections. Further, the multipath profile has peaks
that appear before the direct path due to noise and low SNR signals.

###  2.2. System Design

Our system uses time of arrival (ToA) techniques to achieve underwater ranging
using smartphones. At a high level, the distance $d$ between the sender and
receiver can be written as, $d=c\Delta t$, where $c$ is the propagation speed
of the signal and $\Delta t$ is the time-of-flight from the sender to the
receiver.

While time-of-flight computation is critical for this approach, accurate
ranging also requires knowing the propagation speed in the desired medium. The
speed of sound in underwater scenarios depends on multiple parameters and can
be approximated using Wilson’s speed equation (Wilson, 1960):

$c=1449+4.6T-0.055T^{2}+0.0003T^{3}+1.39(S-35)+0.017D$

Here, $T$ is the temperature in degrees Celsius, $S$ is the salinity in PSU,
and $D$ is the depth in meters. In practice, the depth limit for recreational
divers is 40 m (scu, 2021a, b). Prior work (Kuperman and Roux, 2007) shows
that at these depths, the maximum change in the speed of sound across
different seasons is around $30m/s$. This results in a $2\%$ relative error at
$1500m/s$, which is acceptable in our applications. Hence, we can fix the
underwater acoustic speed to calculate distance. While depth $D$ has
negligible impact at recreational scuba diving depths, one can improve
accuracy by configuring the temperature and salinity values that are known for
different water bodies.

In the rest of this section, we first describe our ranging protocol that
addresses synchronization issues and then describe our dual-microphone
optimization.

####  2.2.1. AquaRanger’s ranging protocol.

The key challenge in performing time-of-arrival is that different devices have
clocks that are not synchronized. Existing in-air acoustic ranging systems use
two pipelines for measuring time-of-flight: one-way ranging and two-way
ranging. In one-way ranging systems (Cai et al., 2018; Youssef et al., 2006;
Wang and Gollakota, 2019b), the two clocks at the sender and receiver are
synchronized in advance and the sender transmits its signal at the preset
timestamps. However, due to the instability of local oscillator in
smartphones, the sampling clock can drift within a few seconds (Sur et al.,
2014). To mitigate the drifting, prior work (Cai et al., 2018; Youssef et al.,
2006) use mathematical approaches to compensate for the clock drifting between
different devices (Wang and Gollakota, 2019b). In our underwater scenario,
accumulation of error due to clock drifting in underwater environment with
varying temperature, pressure, humidity, can make it challenging to achieve
robust clock drift compensation for long-term use (Walls and Gagnepain, 1992).
Researchers have also proposed two-way ranging approaches that address clock
synchronization between devices (Curtis et al., 2014; Peng et al., 2007; Zhang
et al., 2012). In these approaches (Peng et al., 2007; Zhang et al., 2012),
the sender sends a signal and the receiver replies. By exchanging the
timestamps of the receiving and sending signals at the microphones of the two
devices, the distance can be calculated. However, these systems (Peng et al.,
2007; Zhang et al., 2012) exchange the timestamp information via Wi-
Fi/Bluetooth that does not work underwater. While one could design and use an
underwater communication system, the available bandwidth is low resulting in
significant communication overhead.

Our approach. We present a two-way ranging protocol for smartphones. Our
approach has three main advantages: (1) it does not require information
exchange about timestamps, (2) it addresses the delays introduced by the
mobile OS, and (3) the ranging error does not accumulate with time. Our
protocol is illustrated in Fig. 3. At first, Phone A transmits a signal to
phone B. When Phone B detects the beginning of the signal, it waits for a
preset known time interval $t_{reply}$ and then responds back to Phone A.
After detecting the reply signal, Phone A calculates the interval $t_{send}$
between the beginning of its own preamble and the reply preamble from Phone B.
The time of flight $\Delta\tau$ can be calculated as (Peng et al., 2007),

(1) $\Delta\tau=(t_{send}-t_{reply}+\delta_{1}+\delta_{2})/2$

$t_{reply}$ is the time interval between Phone A’s preamble and Phone B’s own
reply preamble physically arriving at microphone hardware of Phone B.
$t_{sent}$ is the time interval between Phone A’s own preamble and Phone B’s
preamble at Phone A’s microphone. Here $\delta_{1}$ and $\delta_{2}$ are the
propagation delay from the speaker to the microphone on the same device. The
distance between the speaker and the microphone on the same phone is quite
small and fixed, and so we can calibrate it for each smartphone model similar
to (Peng et al., 2007).

The key challenge is in addressing the buffer delays at each of the phones.
Specifically, there are two key issues.

  * •

At each smartphone, the microphone and speaker buffers are not synced with
each other (and, 2021). These buffers are filled in independently by the OS.
As a result, we do not know the timestamps corresponding to the samples in the
two buffers.

  * •

While many Android smartphones use the same clock for the microphone and
speaker buffers, data sheets state that they can use a different clock in some
phones (and, 2021). So the sampling rates may also be slightly different
between them.

At a high level, we explore the low-level audio timing in the OS to achieve
self-synchronization between the microphone and speaker buffers on each
smartphone. The responding phone B, adjusts for its buffer delays between the
microphone and speaker to ensure that the response is transmitted at
$t_{reply}-\delta_{2}$ (in other words, ensure the response arrives at its own
microphones at $t_{reply}$). Phone A can also self-synchronize and estimate
$t_{send}$, which it can use to compute the time of flight using Eq. 1.

Figure 3. Various delays in a two-way ranging system.

Low-level audio timing. Our goal is to ensure that the replying phone (Phone
B) can send a preamble at a precise sample index in the future that
corresponds to $t_{reply}$, after the signal from the sender arrives at the
phone. To map the sample indices to time, we need to look into how Android
transmits and records sound at a low level. The low-level OpenSL ES audio
library in Android exposes access to the speaker and microphone audio sample
buffers. Specifically, the library provides the ability to directly write
audio samples to a future speaker buffer even during speaker playback. During
runtime, the library executes a CallBack function when the microphone buffer
is full or the speaker buffer is empty. In this way, we can acquire a
continuous data stream for microphone data and another continuous data stream
for speaker data. Thus, the sample indices in the microphone and speaker
streams have a linear relationship with timestamps:

(2) $\displaystyle t_{s}(n)=n/f_{s}^{s}+t_{s}^{0},$ $\displaystyle
t_{m}(m)=m/f_{s}^{m}+t_{m}^{0}$

Here $m$ and $n$ are the sample indices in the microphone and speaker data
streams. $t_{s}(n)$ is the timestamp when sample $n$ in the buffer is send out
by the speaker and $t_{m}(m)$ is the timestamp when sample $m$ arrives in the
microphone buffer. $t_{s}^{0}$ is the initial timestamp when the first sample
in the stream is sent by the speaker, and $t_{m}^{0}$ is the initial timestamp
when the first sample in the stream arrives at the microphone. $f_{s}^{s}$ and
$f_{s}^{m}$ are the sampling rates for the speaker and microphone, which may
not be exactly our desired sampling rate ($f_{s}$ = 44.1 kHz). We assume that
$f_{s}^{s}=f_{s}/(1-\alpha)$ and $f_{s}^{m}=f_{s}/(1-\beta)$, where
$|\alpha|\ll 1$ and $|\beta|\ll 1$.

Self-synchronizing speaker and microphone streams. As shown in Fig. 4, we do
not know the exact timestamp $t_{3}$ when the preamble arrived at the
microphone of Phone B. Instead, we only know the sample index $m_{2}$ of the
recorded preamble in the microphone stream. At the speaker side, we also
cannot directly know the exact timestamp $t_{4}$ when the speaker sends the
signal, but we can control the sample index $n_{2}$ in the speaker stream.
According to the definition of $t_{reply}$ in Eq. 1, we have
$t_{reply}=t_{5}-t_{3}=t_{4}+\delta_{2}-t_{3}$ from Fig. 4. Combining this
with Eq. 2, we have

(3) $\displaystyle t_{reply}$
$\displaystyle=t_{4}+\delta_{2}-t_{3}=n_{2}/f_{s}^{s}+t_{s}^{0}+\delta_{2}-m_{2}/f_{s}^{m}-t_{m}^{0}$

The microphone and speaker buffers work separately, and there is no guarantee
of the relative order between the two buffers. In other words, the initial
offsets $t_{s}^{0}$ and $t_{m}^{0}$ can be different each time we open the
streams. To address this, once we open the microphone and speaker data
streams, we do not close them so as to keep the offset, $t_{s}^{0}-t_{m}^{0}$,
constant. We write zeros to the speaker stream when we are transmitting
nothing to keep the buffer full. Further, after initializing the two streams,
the speaker sends a calibration signal (green in Fig. 4) to estimate this
offset. We write the calibration signal to the sample index $n_{1}$ in the
speaker stream. Then the microphone stream would receive this calibration
signal at sample index $m_{1}$. The propagation time of the calibration signal
from the speaker to the microphone on Phone B is $(t_{2}-t_{1})$ (i.e.,
$\delta_{2}$). By applying Eq. 2, we get:

(4) $\displaystyle
t_{2}-t_{1}=m_{1}/f_{s}^{m}+t_{m}^{0}-n_{1}/f_{s}^{s}-t_{s}^{0}=\delta_{2}$

Figure 4. Mapping buffer samples to absolute time.

Now, we compute the offset $(n1-m1)$ between the microphone and speaker after
initial calibration, which can be used for $(t_{s}^{0}-t_{m}^{0})$
compensation. After initial calibration, our goal is when phone B detects the
start of the signal from phone A at the sample index $m_{2}$ in Fig. 4, phone
B will write the reply signal at the sample index $n_{2}$ in the speaker
stream, such that they are separated in time by the desired gap,
$t_{reply}^{0}$, after adjusting for buffer delays. So, by compensating the
indices offset acquired from calibration, we set $n_{2}$ to,

(5) $\displaystyle n_{2}=m_{2}+(n_{1}-m_{1})+fs\cdot t_{reply}^{0},$

Here $f_{s}$ is the desired sampling rate. However the real reply interval
$t_{reply}$ is shown in Eq. 3. The difference between the real and desired
reply times is,

(6) $\displaystyle
t_{reply}-t_{reply}^{0}=n_{2}/f_{s}^{s}+t_{s}^{0}-m_{2}/f_{s}^{m}-t_{m}^{0}-t_{reply}^{0}+\delta_{2}$

By combining Eqs. 4 and LABEL:eq:diff and using the relationship between
$f_{s}$, $f_{s}^{m}$, $f_{s}^{s}$, we can rewrite the above equation as,

(7) $\displaystyle t_{reply}-t_{reply}^{0}$ $\displaystyle=-\alpha
t_{reply}^{0}+\frac{(m_{2}-m_{1})(\beta-\alpha)}{f_{s}}$

Figure 5. (a) Ranging error does not drift over 15 mins. (b) Underwater
frequency response between phones.

The key observation here is that the main error source is from the difference
between the actual sampling rate and the nominal sampling rate. $\alpha$ for
smartphones is around 1-80 ppm (Guggenberger et al., 2015). Since we perform
ranging every second, the timing error in the first item is 1-80 $\mu$s
(0.15-12 cm), which is lower than our target accuracies. As for the second
term, while $\beta-\alpha$ is quite small even when the two sensors do not
share a common clock, if the calibration is done only once after
initialization (green signal in Fig. 4) the time interval between initial
calibration and subsequent received signal for Phone B, $m_{2}-m_{1}$ will
increase with time, which may result in error accumulation. Our solution to
this problem is to use the reply signal from the device for continuous
calibration. In this approach, $m_{2}-m_{1}$ is limited to the interval
between sending the current reply signal and receiving the next signal (in our
design, ranging is done at a around 1 Hz frequency, so $(m_{2}-m_{1})/fs$ is
also around 1s). In this way, the second error item is negligible and would
not accumulate for a long period. Fig. 5a shows the evaluation of our
underwater ranging over a duration of around 15 minutes while the ranging is
done once every 6 s. This evaluation was done between two different smartphone
models: the Google Pixel 3A, and Samsung Galaxy S10. The plot shows that our
system does not suffer from error accumulation.

####  2.2.2. Dual-microphone joint synchronization

The above description assumes that the phones can accurately estimate the
beginning of the received signals. This is challenging because of the
multipath introduce by the water-proof case in underwater environments. §2.1
showed that the direct path can be severely attenuated and that we can not
rely on the assumption that the highest peak or the first non-negligible peak
in the multipath profile is the direct path. As shown in Fig. 6, there can be
some peaks before the direct path with amplitude greater than the average
noise level (red point with ”Wrong peak” in Fig. 6).

Figure 6. The yellow and blue curves correspond to the channel estimates at
the two microphones.

To reduce the probability of picking these wrong peaks, we use the two
microphones on the smartphone. The basic idea of our joint synchronization
algorithm is that the time difference of arrival at the bottom and top
microphones is physically constrained by the distance between them. Thus, the
sample offset between the direct path at the top microphone channel and the
direct path at the bottom microphone should be lower than the acoustic
propagation time between the two microphones (two black cross symbols in Fig.
6). Furthermore, the multipath created by the water-proof case is different at
the two microphones. Finally, the two microphones may have a different noise
profile.

Thus, our algorithm identifies the direct path as the earliest non-negligible
peaks in both channels whose sample offset satisfies the physical distance
constrain between the two microphones. Specifically, we denote the estimated
channel for the bottom microphone as $h_{1}(n)$ and the top microphones as
$h_{2}(m)$, where $n$ and $m$ are the channel tap numbers. Then, we normalized
$h_{1}$ and $h_{2}$ to be between 0 and 1. We then check whether the sample
$n$ in channel $h_{1}$ is a peak. Next, we estimate the channel noise level
for the two microphone channels by calculating the average power in the last
100 channel taps, respectively denoted as $w_{1}$ and $w_{2}$.

$\displaystyle\min_{\tau_{LOS}}\quad\tau_{LOS}=(n+m)/2,\quad\forall
m,n\in[0,L)$
$\displaystyle\begin{array}[]{r@{\quad}r@{}l@{\quad}l}s.t.&&h_{1}(n)>w_{1}+\lambda,\\\
&&h_{2}(m)>w_{2}+\lambda,\\\ &&IsPeak(n,h_{1})\cap IsPeak(m,h_{2}),\\\
&&|n-m|\leq d/c,\end{array}$

The above equations shows our optimization problem. Here $\tau_{LOS}$ is the
delay of the direct path, $n$ and $m$ are the tap numbers in the channels
$h_{1}$ and $h_{2}$. $w_{1}$ and $w_{2}$ are the estimated noise levels in
these two channels. $\lambda$ is a conservative parameter (we set it
empirically to 0.2). $d$ is the physical distance between the two microphone,
$c$ is the speed of sound and $L$ is the entire length of the channel (1260
samples). We solve this optimization problem using Algorithm. 1.

for  _$n\leftarrow 0$ to $L-1$_ do

if  _$h_{1}(n) <w_{1}+\lambda$ $\cup$ $\neg$ IsPeak(n, $h_{1}$)_ then

Continue

for  _$m\leftarrow(n-d/c)$ to $(n+d/c)$_ do

if  _$h_{2}(m) >w_{2}+\lambda$ $\cap$ IsPeak(m, $h_{2}$)_ then

$\tau_{LOS}\leftarrow(m+n)/2$

return $\tau_{LOS}$

Algorithm 1 Dual-microphone sync algorithm

In the rest of this section, we describe our preamble design and signal
processing pipeline for channel estimation.

Signal processing pipeline. We use OFDM symbols between 1-5 kHz as the
preamble. We use this frequency band given the underwater response of mobile
devices (see Fig. 5b). We fill the OFDM bins with a ZC sequence (Wen et al.,
2006) which is phase-modulated and orthogonal to its delayed version (Zhang et
al., 2020). ZC-modulated OFDM symbols can achieve much better performance than
their well-known counterpart, chirps (Sesia et al., 2011; Cai et al., 2018).
We then concatenate eight such identical OFDM symbols and multiply each with a
PN sequence with different signs ([-1, 1, 1, 1, 1, 1, -1, 1]), to increase
robustness to noise (Nasir et al., 2010). Between each OFDM symbol, we insert
a cyclic prefix to avoid inter-symbol interference.

Our preamble synchronization algorithm at the receiver is composed of there
steps: cross-correlation, auto-correlation, and channel estimation. Since the
pattern of preamble is known in advance, we can perform cross-correlation
between the microphone stream and the preamble pattern. In the presence of a
preamble, this results in a correlation peak. However, the height of the
correlation peak could vary a lot as the SNR decreases at long distances.
Meanwhile, some underwater spiky noise like bubbles would also cause high
peaks in the cross-correlation. Such noise may lead to a plenty of false
positives. To address this we use auto-correlation. Since our preamble has 8
OFDM symbols that are encoded with a 8-bit PN sequence, we split the received
signal into 8 segments corresponds to the 8 OFDM symbols, multiply each
segment by the PN sequence, and apply correlation among them (Nasir et al.,
2010). Auto-correlation is helpful because the spiky noise rarely has such
complex encoded pattern (PN sequence) and since the 8 received OFDM symbols
suffer from nearly the same multi-path, the correlation value between two
received OFDM symbols would be much higher than the correlation value between
the received and transmitted symbols. We set a threshold of 0.35 in our
design.

Due to the severe underwater multi-path profiles, the side-lobe height in the
correlation curve is usually higher than the direct path. Hence, coarse
synchronization error based on only correlation is usually hundreds of
samples, corresponding to over 6 m error. To achieve more fine-grained
synchronization, we apply channel estimation, where we leverage the channel
profiles to identify the direct path.

There are several channel estimation methods: LS (Kewen et al., 2010), MMSE
(Alihemmati and Kalantari, 2005), and other MUSIC-like estimator (Cai et al.,
2018; Xue et al., 2020). MMSE estimators can minimize the mean-square channel
estimation error, but requires some prior knowledge about noise variance and
channel covariance (Kewen et al., 2010). While MUSIC-like estimators could
achieve super-resolution channel profiles, the signal space decomposition is
difficult due to the extremely dense underwater channel and it has a high
computational complexity. Therefore, we use an LS channel estimator.

Specifically, based on the coarse synchronization of cross-correlation and
auto-correlation, we segment out 8 received OFDM symbols
$y_{1},y_{2},\dots,y_{8}$ from the microphone stream. Then we apply FFTs on
these 8 symbols to get $Y_{1},Y_{2},\dots,Y_{8}$. We denote the FFT of the
original OFDM symbol before multiplication with PN sequence by $X$ and denote
the PN sequence by $PN_{1},PN_{2},\dots,PN_{8}$. The channel model can be
written as $Y_{i}(k)=H(k)(PN_{i}\cdot X(k))+N_{i}(k)$, where $k$ represents
the $k^{th}$ frequency bin. The estimated channel is
$\hat{H}(k)=\frac{1}{8}\sum_{i=1}^{8}\frac{1}{PN_{i}}\cdot X(k)^{-1}Y_{i}(k)$.
Finally, we apply an IFFT to convert the estimated channel to the time-domain.

Figure 7. Supporting multiple diver devices. Device $i-1$ uses the the leader
query for device $i$ and its own prior response to estimate its distance from
the leader.

###  2.3. Supporting multiple divers

To support ranging of multiple devices in a dive group at the same time, we
design a query-response based system where each of the devices is queried by
the leader device in a round-robin manner. The dive instructor in the group is
assigned as the leader device which is the sender and the other divers are the
repliers. Before diving into the water, each device is designated a unique ID.

While our ranging system is designed to compute the distance value at the
sender device (i.e., the leader), adjacent query messages from the leader
device can also be used to compute the distances at the replier devices (i.e.,
other divers). Specifically, say the reply time for each replier and sender is
$\tau_{0}$. The leader first sends a preamble concatenated with the first
diver’s ID. When the phones receive the preamble, they check the ID
information. Diver 1 sends back its preamble after a reply time $\tau_{0}$,
while the devices at the other divers stay silent. When the leader receives
the reply preamble, the leader can estimate the distance from the first diver.
Then the leader sends the second preamble with the second diver’s ID after the
reply time $\tau_{0}$. When diver 1 overhears this preamble with the second
diver’s ID, the first diver can also compute the distance to the leader as,
$d_{10}=\frac{T_{10}-\tau_{0}}{2c}$. Meanwhile, the second diver would send
back its reply preamble after $\tau_{0}$ and the other phones remain silent.
When the leader receives the reply from the second diver, it would calculate
the distances to it and send the third preamble after $\tau_{0}$. In this way,
after sending the preamble to each diver, the leader can acquire the distance
of each diver. Meanwhile each diver also knows its distance to the leader by
processing the query for the next diver. We repeat this process in a round-
robin manner to compute the distance at the divers and the leader device. Note
that given the time-division nature of supporting multiple divers, this
proportionally reduces the rate at which the distance information can be
computed for each diver. One may consider designing code division multiplexing
mechanisms to query all the divers concurrently but this reduces the energy in
each of diver queries and the range. Thus, our design is focused on the time-
division based query/response protocol to support multiple diver devices.

Operation |  Runtime (ms)
---|---
Cross-correlation | 59.8 ± 10.6
Auto-correlation | 28.4 ± 4.7
Channel estimation | 1.9 ± 0.6
Total (for 500 ms buffer) | 90.1 ± 11.2
Table 1. Statistics across 100 measurements. Runtime is measured for
computation over a 500 ms buffer. Figure 8. Different underwater scenarios.
(a) Swimming pool: a busy environment with human mobility and occlusion. (b)
Dock: an outdoor location with boats, seaplanes, and animals. (c) Viewpoint:
At a park with a shallow water depth of 1.5 m. (d) Boathouse: A busy dock with
people fishing and kayaking.

We encode the ID of a user using a simple frequency-based mechanism.
Specifically, we map the user’s ID to a unique frequency from the 16
frequencies: $1500,1535,1570,...,2060\leavevmode\nobreak\ Hz$. The user’s ID
is appended to the preamble as a 100 ms frequency tone at their assigned
frequency. We empirically select this frequency range as we find that the SNR
of the tones are strongest in this range. We select a spacing of 35 Hz to
account for frequency shifts that could occur from the Doppler effect as a
result of mobility. To decode the user ID, we compute the SNR at all 16
frequency bins and select the bin with the maximum SNR.

##  3\. Implementation and Evaluation

Our ranging system has been implemented to run in the user space on Android
devices. Our implementation performs the cross-correlation, auto-correlation
and channel estimation algorithm within an average runtime of 90.1 ± 11.2 ms
over a 500 ms audio sample buffer. We use the fftw3 library to perform FFTs in
realtime. We use the OpenSL ES audio interface on Android which provides low-
level access to the speaker and microphone audio buffers. In our system, the
replying phone is configured to send a reply, one second after it has detected
a preamble from the sender.

###  3.1. Results

We evaluated our system in the four environments in Fig 8.

  * •

Swimming pool. The length of the water here is around 23 m. The depth of the
swimming pool varied from around 1 to 2.5 m. This is a busy location with
people swimming laps or engaging in other recreational water activities.

  * •

Dock. This outdoor location has a length of around 50 m with a depth of 9 m.
Boats and seaplanes would frequently sail or dock at this location with
aquatic plants and animals.

  * •

Viewpoint. By the waterfront of a park with a length of 40 m. The water had a
depth of around 1 to 1.5 m. This is a busy location with boats and strong
currents.

  * •

Boathouse. Fishing dock by the lake with a horizontal distance of 30 m. The
lake had a depth of 5 m. This is a busy location with people fishing and
kayaking close to the dock.

####  3.1.1. Accuracy versus device separation.

We first present our evaluation of the range of our system along the dock of a
lake with an average water depth of 9 m. We performed experiments using two
Samsung Galaxy S9 phones set to transmit at the maximum speaker volume. The
phones were set to transmit using the speaker at the bottom of the device, and
receive using the microphones at the bottom and top of the device. We
designate the two phones as being either a sender or replier. In this
evaluation, the experiment was repeated in each location every six seconds,
where the sender was configured to send a message to the replier. The replier
is then set to continuously run the preamble detection algorithm and respond
with its reply after it has received a preamble from the sender. At each
distance, the sender and replier are set to exchange messages up to a maximum
of 60 times. The measurements were divided into roughly three sessions, where
after 20 measurements, the phones were removed from the water and submerged
again.

Figure 9. Ranging accuracy versus device separation. CDF of absolute error as
a function of separation.

The phones were enclosed in a pouch (Hiearcool waterproof phone pouch (pou,
2021)) and attached to a selfie stick and telescopic extension pole, which was
used to submerge the phones to a depth of 2.5 m. The selfie stick and
extension pole were attached using waterproof tape and zip ties. This setup
was chosen so that the phone’s position and angle could be controlled. We used
a tape measure to mark different measurement distances up to a maximum of 45 m
along the dock. We set 1500 m/s as the speed of sound underwater.

Fig. 9 shows the CDF of the absolute error obtained by our system for four
distances of 10, 20, 35 and 45 m. The plot shows that the median error of our
system is 0.48 m at 10 m, 0.80 m at 20 m, 0.86 m at 35 m, and increases to
1.67 m at 45 m. The corresponding 95th percentile errors were 0.83 m, 1.02 m,
1.51 m and 2.57 m. The error in distance increases with separation between the
devices because the signal strength is lower at larger separation. This
results in an increased probability of confusing the direct path with the
multipath and noise. Thus, the conservative range of our system is around 35
m. We note however that this scaling of error with separation is desirable in
practice since higher resolution is required when the devices are close by
than when they are farther. A 2.5 m error when the separation is 45 m is still
meaningful and provides useful information.

We also note that in this experiments, we use a preamble with a length of 316
ms. When we use a longer preamble with a length of 479 ms, the median error of
the system decreases to 1.63 m at 45 m respectively. This shows a tradeoff
where increasing the preamble length can reduce the error at longer ranges but
will reduce the rate at which ranging can be achieved by our system. We note
that in both settings, we can provide a location value at a rate of 1 Hz,
which is sufficient for our target underwater ranging applications.

Figure 10. Effect of device depth. Errors for different depths with devices
separated horizontally by 18 m.

####  3.1.2. Accuracy versus depth.

We next present our evaluation of the ranging accuracy when the smartphones
are placed at different water depths at the dock location which had a total
depth of 9 m. We lowered the smartphones into the water using ropes marked at
approximately 2, 5, and 8 m. The phones were weighed down with a bag of
pebbles to ensure the ropes were vertically straight. The phones were
positioned at a horizontal distance of 18 m. Unlike previous experiments, the
rope would cause the phone to rotate and sway slowly, making it difficult to
control the angle of the phone during the measurement. We repeat the
measurements three times at each depth. Fig. 10 shows the CDF of the absolute
error at different depths. The plot shows that the median and 95th percentile
error is lowest at 0.28 and 0.73 m for the 5 m depth, which is around the
midpoint depth of the dock. In contrast, the median distance error at 2 and 8
m are 0.33 and 0.58 m respectively. This likely is because multipath
reflections can be stronger when the devices are close to the surface or floor
of the water body. In these experiments the phones could sway and rotate and
hence were not static.

Figure 11. Effect of shallow waters. Errors at two different locations with a
depth of 1.5 and 5 m respectively.

####  3.1.3. Evaluation in shallower waters.

The above evaluations were in a water body that had a depth of around 9 m.
Shallower waters may contribute to more severe multi-path environments because
of multiple reflections from the surface and floor. To evaluate the
performance of our system in such environments, we test our system in the
viewpoint and boathouse scenarios. At the viewpoint, the total water depth
ranged from 1 to 1.5 m and the phones were submerged to a depth of 0.5 m. At
the boathouse the total water depth was 5 m and the phones were submerged to a
depth of 2.5 m. We measured the errors at distances of 10, 20, and 30 m. Fig.
11 shows the CDF of the distance error at these locations. The plot shows that
the ranging error in the shallow water setting of the viewpoint had the
highest 95th percentile error across all natural water bodies tested, with an
error of 2.06 m at 30 m. In comparison the error at the boathouse and dock
location with deeper waters had a 95th percentile error of 1.22 and 1.48 m
respectively. We note that each of these underwater environments have a
different multipath and noise profile where shallower waters likely have more
severe multipath.

Figure 12. Trajectories computed by our smartphone ranging system when the
phone moves at different speeds. Note that underwater motion speeds are lower
than that in in-air due to much higher water resistance.

####  3.1.4. Tracking device motion.

We evaluate whether our system can be used to track the distance of the
smartphone over time. In this experiment, we moved the smartphone along
different 1D trajectories parallel to the coast at the dock location for
measurement durations of 153 to 176 s. The smartphone was moved at two times
at a slow speed of 15 cm/s, and two times at a faster speed of 32 and 56 cm/s.
We note that it was challenging to move the extension pole with the phone at
higher speeds due to the resistance of the water. At higher speeds, the range
measurements of the phone underwater would lag behind the ground truth
recordings of the the pole above water. We obtained the ground truth distance
measurement by recording a video of the smartphone’s position with respect to
a tape measure placed along the dock. The video recording was then
synchronized with the measurements obtained from the smartphones underwater,
based on the phone’s timestamp that was displayed on the screen. The video
frames corresponding to each ranging estimate was extracted, and the reading
on the tape measure was recorded. We compute a range value using our system
once every second. Fig. 12 compares the trajectories estimated by our system
with the ground truth measurements. The median and 95th percentile error
across the two slower motion patterns was 0.35 and 0.93 m. In comparison, the
median and 95th percentile errors across the faster motion patterns was
slightly higher at 0.51 and 1.17 m. These results suggest that our ranging
frequency of 1 Hz is sufficient for providing tracking capabilities at a range
of speeds.

####  3.1.5. Human mobility in the environment.

While we performed the above evaluations in natural outdoor lakes that have
aquatic life, there were no humans swimming in these water bodies. So to
understand the effect of other human mobility in the environment we evaluate
our system in a busy swimming pool. We performed measurements at a fixed
horizontal distance of 6.1 m over a period of eight minutes. During this time,
there were a number of swimmers swimming around in the pool and in adjacent
lanes. The depth of the pool was 2.5 m and the phones were held by hand at a
depth of 1 m. Note that since the experiments were performed over eight
minutes, there were slight movements in the hand over the test duration. Fig.
13a shows the CDF of the distance error at the approximate ground truth value
of 6.1 m. The median and 95th percentile error were 0.35 and 0.84 m
respectively. These low errors are likely because while a mobile human in the
environment changes the multipath in the environment, since the objective of
our algorithm is to zone in on the direct path by combining the information
across the two microphones, it can ignore these multipath variations and focus
on the direct path that does not change.

####  3.1.6. Effect of human occlusion.

Next, we measure the effect of human occlusion from the smartphone user. We
perform these experiments again in the swimming pool, where we periodically
block the speaker and microphone of one of the phones with our body. The
experiment was performed at a horizontal distance of 6.1 m over 8 minutes.
Fig. 13b shows the CDF of the distance error with and without occlusion at the
same distance. The presence of an occlusion increased the median error from
0.31 to 0.87 m. The 95th percentile error further increased from 0.89 to 2.41
m. This is expected because occlusions can sometimes reduce the amplitude of
the direct path to be close to noise. A key assumption that ranging systems
(Peng et al., 2007; Wang and Gollakota, 2019a; Mao et al., 2016; Jin et al.,
2015) make is that there exists a non-zero direct path. This is also true for
existing underwater localization systems that require custom buoys and
hardware (Tan et al., 2010; Tomczak, 2011; Zhao et al., 2018; Munafo et al.,
2015; Srinivasan et al., 2007; Maki et al., 2013; Tan et al., 2011; Moon et
al., 2009). In our system, the direct path does not need to be the strongest
path but it should be non-zero. When such a path does not exist, ranging based
systems are limited in their accuracy.

####  3.1.7. With different smartphone models.

We also evaluated our system with different smartphone model pairs. As
different smartphones use different speakers and microphones, the frequency
selectivity of an acoustic signal can vary between devices. In this
experiment, we evaluated three Android smartphones models: Samsung Galaxy S9,
Google Pixel 3a, and OnePlus 8 Pro. Fig. 14a shows the CDFs of errors between
different device pairs. The plot shows that the median error for a given
device pair ranges from 0.28 to 0.54 m. Across all device pairs and
measurements, the median and 95th percentile error is 0.41 and 0.75 m
respectively.

####  3.1.8. Effect of phone orientation.

We evaluate the effect of phone orientation on the accuracy at the dock at a
horizontal distance of 20 m and a depth of 2.5 m. We first positioned the
speaker and microphone of both phones to directly face each other so their
azimuth $\phi$ and polar angle $\theta$ is set to 0° and 180° respectively. We
measure the error when the sender phone is rotated to different azimuth and
polar angles. We first rotate the sender phone in the azimuth angle to 90° and
180° while keeping the polar angle constant. We then reposition the phone so
its speaker and microphone faces upwards with $\phi=0^{\circ}$,
$\theta=0^{\circ}$. Fig. 14b shows the CDF of the distance error for these
different rotation configurations. The median error ranges from 0.54 to 1.25 m
across different orientations. The 95th percentile error ranges from 0.78 to
1.62 m. Note that when the phone faces upwards it had the largest error likely
because the phones are closer to the water surface resulting is higher
multipath when pointing towards the surface.

Figure 13. Effect of (a) other human mobility in environment, and (b)
occlusion between the devices.

Figure 14. Effect of (a) various smartphone model pairs, and (b) orientations
with phones separated by 20 m.

####  3.1.9. Ranging of multiple devices.

We evaluate if our system can reliably support ranging updates to multiple
devices. To evaluate this, we consider a network deployment with one leader
device and three diver devices placed at the dock location. The leader and one
diver device was submerged using the extension pole, and the two remaining
diver devices were submerged using a rope. All diver devices were submerged to
a depth of approximately 3 m. Devices 1 and 2 were horizontally spaced from
the leader device by 7 m, while device 3 had a spacing of 9 m. Fig. 15a shows
the CDF of distance error for the three devices. The median error for each
device ranged from 0.21 to 0.47 m, while the 95th percentile error ranged from
0.68 to 0.79 m. Additionally, we evaluate how well our system is able to
accurately decode the device ID appended to the end of each preamble. For this
experiment, we sent between 300–480 random preamble and ID pairs at distances
up to 45 m and measured the decoding accuracy of our system across all
transmitted IDs. Fig. 15b shows that our system is able to correctly decode
the ID at a $>95\%$ accuracy at distances up to 45 m. These results show the
feasibility of our protocol working with multiple users.

####  3.1.10. Effect of multiple microphones.

Here we analyze the effect of using both the top and bottom microphones for
ranging versus using only a single microphone in isolation. Fig. 16a shows the
95th percentile distance error for these scenarios at distances of up to 45 m.
The figure reveals the following: firstly, utilizing both microphones yields
lower ranging errors at all distances. This can reduce error by as much as
4.52 m at a distance of 45 m. Secondly, when a single microphone is used in
isolation, there is no clear relationship between microphone position and
distance error. This is likely due to the different multipath and noise
profiles at different distances. These results show that identifying the
direct path using two microphones is a more accurate strategy than using a
single microphone alone.

Figure 15. Ranging of multiple devices. (a) Errors for each of the three
devices. (b) ID decoding accuracies.

##  4\. Related work

While there have been significant efforts to achieve underwater communication
(Restuccia et al., 2017; Chen et al., 2022; Sklivanitis et al., 2014; Noh et
al., 2015; Yonggang et al., 2008; Bicen et al., 2012; Mishachandar and
Vairamuthu, 2021; Jang and Adib, 2019) using OFDM modulation and rate
adaptation (ongoing, 2022) and sensing (Flores et al., 2020b, a; Foresti and
Gentili, 2000), here we focus on the prior work in localization.

Underwater tracking. There has been significant interest in achieving
underwater tracking for diving computers, sensors and robotics (Tan et al.,
2010; Tomczak, 2011; Zhao et al., 2018; Munafo et al., 2015; Srinivasan et
al., 2007; Maki et al., 2013; Tan et al., 2011; Moon et al., 2009; Barberá et
al., 2021). While our work builds on this prior art, we are the first to
demonstrate an end-to-end underwater acoustic ranging system between
smartphones. Below we describe these prior works in detail.

(Kuch et al., 2012) proposes the design of a custom diving computer that can
communicate with a surface buoy for underwater mapping and tracking. (Anjangi
et al., 2020) uses hydrophone devices on the surface as beacons with known
locations. The diver then uses a hand-held display connected to an acoustic
communication module. (Alcocer et al., 2004) proposes the use of an underwater
pinger that emits acoustic signals periodically. The pinger is synchronized
with a very high precision clock that is synchronized to GPS, prior to
deployment. The acoustic signals are tracked from the surface by using a
system of four buoys that measures the times of arrival of the acoustic
signals emitted by the pinger. The surface buoys may be equipped with GPS that
can be translated to underwater GPS (Alcocer et al., 2006). (Cheng et al.,
2008) proposes the reverse process where the time difference of arrivals is
measured at a sensor to detect range differences from the sensor to four
anchor nodes. These range differences are averaged over multiple beacon
intervals to estimate the 3-D sensor location. (Casey et al., 2007) uses a
single beacon and compute the position based on time of arrival measurements.
(Luo et al., 2008) proposes the use of directional beacons for localization.
(Cario et al., 2021b) presents the results of experiments designed in a
virtual environment used to simulate real acoustic underwater localization
systems. (Rauchenstein et al., 2018; Pottinger, 2012) propose to use machine
learning to improve localization for custom time-difference of arrival based
hydrophones. In contrast to this prior work that is designed for custom
hardware, our work achieves a novel underwater ranging system between
smartphones.

A recent short paper presents two quick experiments underwater using
smartphone hydrophones (Monteiro and Marti, 2020). This prior paper however
does not provide details about the smartphone used, algorithms or the
underwater scenarios it was tested in. In addition, it does not consider any
practice issues like clock drifting, underwater multipath, OS buffering or
mobility. In contrast, we design real-time algorithms, explore the challenges
of using smartphones underwater and provide extensive evaluation in various
underwater scenarios.

Recent works also propose the use of an autonomous marine surface (Miskovic et
al., 2015) or underwater vehicles (Nađ et al., 2020) to increase diver safety
by enabling navigation and reliable monitoring from the surface. (de Langis
and Sattar, 2020) proposes the use of autonomous underwater robots to track
divers in real time and to re-identify them. (DeMarco et al., 2013) proposes
the detection and tracking of a diver with a high-frequency sonar. (Zhu et
al., 2016a; Bayat et al., 2015) design algorithms to achieve underwater
tracking in the presence of ocean current uncertainty on the surface buoy.
(Zhu et al., 2016b) designs algorithms to account for variations in sound
velocity that is dependent on water temperature, pressure and salinity.
(Ghaffarivardavagh et al., 2020) explores the challenges of achieving
localization for underwater backscatter sensors. Our work instead is focused
on enabling underwater ranging between commodity smartphones.

Figure 16. (a) 95% errors using both microphones, the bottom and top
microphone only. (b) Smartphone compass bearings measured underwater and in
air.

In-air acoustic tracking. Our work builds on prior work that achieve in-air
acoustic device ranging, tracking, localization and health applications
(Nandakumar et al., 2015; Nandakumar et al., 2019; Chan et al., 2019a;
Nandakumar et al., 2016; Wang et al., 2019, 2021; Chan et al., 2019b).
BeepBeep (Peng et al., 2007) and Swordfight (Zhang et al., 2012) achieves
ranging between phones. These require a secondary RF channel between the two
phones to exchange synchronization information between the two devices. Radio
signals at 2.4 GHz can attenuate as much as 169 dB per meter in seawater (Dong
et al., 2016; Lin et al., 2021). Sonoloc (Erdélyi et al., 2018) can achieve
distributed acoustic localization in-air between multiple devices but requires
more than ten devices to achieve reasonable accuracies. (Zhang et al., 2018)
achieves acoustic in-air tracking by assuming that there is no significant
multipath; underwater scenarios in contrast are known for their severe
multipath challenges. ALPS (Lazik et al., 2015) and Tracko (Jin et al., 2015)
use a combination of Bluetooth and ultrasonic; Bluetooth however cannot be
used underwater. CAT (Mao et al., 2016), Millisonic (Wang and Gollakota,
2019a) and SoundTrak (Zhang et al., 2017) have been proposed to improve the
acoustic accuracies. CAT combines IMU sensor data and frequency modulated
continuous wave (FMCW) localization to address in-air multipath. SoundTrak
(Zhang et al., 2017) is designed for a smart watch and a customized finger
ring using phase tracking where the area of movement is limited to a
$20cm\times 16cm\times 11cm$ area in-air. Millisonic (Wang and Gollakota,
2019a) uses a microphone array and time difference of arrival algorithms.
While we build on this prior work on in-air acoustic tracking, our focus is on
achieving underwater ranging between smartphones.

##  5\. Discussion and Conclusion

We present a novel real-time underwater system that can achieve acoustic
ranging between commodity smartphones. We evaluate our design in various
underwater settings and demonstrate its efficacy. We believe that this work
explores a new underwater research direction by bringing ranging capabilities
to commodity smartphones. Here we discuss design considerations and avenues
for future work.

Using phone compass underwater. If the compass bearings of a smartphone placed
underwater correspond to those of a smartphone in open air, it can be used for
underwater navigation. To check this, we attached two phones to the bottom and
top of the extension pole. The smartphone at the bottom end of the pole was
submerged directly underwater to a depth of 3 m while the smartphone at the
top end of the pole remained in air. We twisted the pole on its axis in random
clockwise and anticlockwise motions and recorded the compass bearing
measurements on both smartphones. Fig. 16b shows the unwrapped compass bearing
angles measured by the smartphone underwater and in the air, showing a match
between the two measurements.

Frequency range. Our smartphone ranging system operates at 1-5 kHz which is
audible to human hearing. We note that multiple acoustic underwater modems
both in the industry and the academic community (Jang and Adib, 2019; low,
2018; Borden and DeArruda, 2012) also operate in the sub-20 kHz human-audible
frequency range. Providing user feedback underwater. An important question is
how do we provide feedback to the diver when they are go beyond a pre-set
distance from their buddy? To prevent the need for the diver to keep checking
on the phone, we can use the vibration motor to provide feedback to the diver,
which would be an interesting underwater U/I design to explore.

Relative distance to absolute location. Ranging provides the relative distance
from one device to the other but not the absolute location. Computing the
absolute location requires three devices with known locations to perform
triangulation. These devices can either be floating buoys or other underwater
smartphones in known locations. Designing this is an interesting research
avenue for future work.

## References

  * (1)
  * mud (2004) 2004\.  Muddy Waters: Techniques for Low-Vis Diving.  https://dtmag.com/thelibrary/muddy-waters-techniques-low-vis-diving/. (2004).
  * dea (2010) 2010\.  Why divers die? Cyprus Federation of underwater activities.  http://www.cfua.org/Divers-Death.htm. (2010).
  * net (2011) 2011\.  Diver drowns in fishing net.  https://scubaboard.com/community/threads/diver-drowns-in-fishing-net-bc-canada.391393/. (2011).
  * bud (2018) 2018\.  Buddy Diving For Safety and Support.  https://www.divein.com/diving/buddy-diving-for-safety/. (2018).
  * low (2018) 2018\.  Evo Logics 7/17 communication and positioning devices. https://evologics.de/acoustic-modem/7-17.  (2018).
  * com (2020) 2020\.  Best Dive Compass for Scuba Diving.  https://oceanscubadive.com/best-dive-compass/. (2020).
  * lig (2020) 2020\.  Best Scuba Diving Lights.  https://oceanscubadive.com/best-dive-compass/. (2020).
  * fai (2020) 2020\.  Scuba Equipment Issues Get This Diver in Deep Trouble.  https://www.scubadiving.com/scuba-equipment-issues-get-this-diver-in-deep-trouble. (2020).
  * and (2021) 2021\.  Audio Latency. https://developer.android.com/ndk/guides/audio/audio-latency.  (2021).
  * scu (2021a) 2021a.  Become a Certified Scuba Diver FAQs.  (2021).  https://www.padi.com/help/scuba-certification-faq
  * div (2021) 2021\.  Diveroid: turn your smartphone into an all-in-one dive gear.  https://diveroid.com/. (2021).
  * sta (2021) 2021\.  Global Scuba Diving Equipment Market - Growth, Trends, COVID-19 Impact, and Forecasts (2022 - 2027), Mordor Intelligence.  (2021).
  * scu (2021b) 2021b.  How Deep Can You Scuba Dive?  (2021).  https://www.scubadiving.com/why-is-130-feet-depth-limit-for-recreational-scuba-diving
  * pou (2021) 2021\.  Universal Waterproof Case, Waterproof Phone Pouch Compatible for iPhone 13 12 11 Pro Max XS Max XR X 8 7 Samsung Galaxy s10/s9 Google Pixel 2 HTC Up to 7.0”, IPX8 Cellphone Dry Bag -2 Pack.  (2021).  https://www.amazon.com/gp/product/B08S3SG5KF/ref=ppx_yo_dt_b_asin_title_o02_s00?ie=UTF8&psc=1
  * Alcocer et al. (2004) Alex Alcocer, Paulo Oliveira, and Antonio Pascoal. 2004\.  Study and implementation of an EKF GIB-based underwater positioning system.  IFAC Proceedings Volumes 37 (07 2004), 383–390.  https://doi.org/10.1016/S1474-6670(17)31762-7
  * Alcocer et al. (2006) Alex Alcocer, Paulo Oliveira, and Antonio Pascoal. 2006\.  Underwater acoustic positioning systems based on buoys with GPS.  (01 2006).
  * Alihemmati and Kalantari (2005) R Alihemmati and ME Kalantari. 2005.  On channel estimation and equalization in OFDM based broadband fixed wireless MAN networks. In The 7th International Conference on Advanced Communication Technology, 2005, ICACT 2005., Vol. 1. IEEE, 224–229.
  * Anjangi et al. (2020) Prasad Anjangi, Amy Gibson, Manu Ignatius, Chinmay Pendharkar, Anne Kurian, Alan Low, and Mandar Chitre. 2020.  Diver Communication and Localization System using Underwater Acoustics. In Global Oceans 2020: Singapore – U.S. Gulf Coast. 1–8.  https://doi.org/10.1109/IEEECONF38699.2020.9389462
  * Barberá et al. (2021) Humberto Barberá, Pablo Bernal-Polo, and David Herrero-Perez. 2021\.  Sensor Modeling for Underwater Localization Using a Particle Filter.  Sensors 21 (02 2021), 1549.  https://doi.org/10.3390/s21041549
  * Bayat et al. (2015) Behzad Bayat, Naveen Crasta, A. Pedro Aguiar, and Antonio Pascoal. 2015.  Range-Based Underwater Vehicle Localization in the Presence of Unknown Ocean Currents: Theory and Experiment.  Control Systems Technology, IEEE Transactions on (04 2015).  https://doi.org/10.1109/TCST.2015.2420636
  * Bicen et al. (2012) A. Ozan Bicen, A. Behzat Sahin, and Ozgur B. Akan. 2012\.  Spectrum-Aware Underwater Networks: Cognitive Acoustic Communications.  IEEE Vehicular Technology Magazine 7, 2 (2012), 34–40.  https://doi.org/10.1109/MVT.2012.2190176
  * Borden and DeArruda (2012) Joe Borden and Jeffery DeArruda. 2012.  Long range acoustic underwater communication with a compact AUV. In 2012 Oceans. 1–5.  https://doi.org/10.1109/OCEANS.2012.6405091
  * Cai et al. (2018) Chao Cai, Menglan Hu, Xiaoqiang Ma, Kai Peng, and Jiangchuan Liu. 2018.  Accurate ranging on acoustic-enabled IoT devices.  IEEE Internet of Things Journal 6, 2 (2018), 3164–3174.
  * Cario et al. (2021a) Gianni Cario, Alessandro Casavola, Gianfranco Gagliardi, Marco Lupia, and Umberto Severino. 2021a.  Accurate Localization in Acoustic Underwater Localization Systems.  Sensors 21, 3 (2021), 762.
  * Cario et al. (2021b) Gianni Cario, Alessandro Casavola, Gianfranco Gagliardi, Marco Lupia, and Umberto Severino. 2021b.  Accurate Localization in Acoustic Underwater Localization Systems.  Sensors 21 (01 2021).  https://doi.org/10.3390/s21030762
  * Casey et al. (2007) Thomas Casey, Brian Guimond, and James Hu. 2007.  Underwater Vehicle Positioning Based on Time of Arrival Measurements from a Single Beacon. In OCEANS 2007. 1–8.  https://doi.org/10.1109/OCEANS.2007.4449186
  * Chan et al. (2019a) Justin Chan, Sharat Raju, Rajalakshmi Nandakumar, Randall Bly, and Shyamnath Gollakota. 2019a.  Detecting middle ear fluid using smartphones.  Science Translational Medicine 11 (05 2019), eaav1102.  https://doi.org/10.1126/scitranslmed.aav1102
  * Chan et al. (2019b) Justin Chan, Thomas Rea, Shyamnath Gollakota, and Jacob Sunshine. 2019b.  Contactless cardiac arrest detection using smart devices.  npj Digital Medicine 2 (06 2019), 52.  https://doi.org/10.1038/s41746-019-0128-7
  * Chen et al. (2022) Tuochao Chen, Justin Chan, and Shyamnath Gollakota. 2022\.  Underwater Messaging Using Mobile Devices. In Proceedings of the ACM SIGCOMM 2022 Conference (SIGCOMM ’22). Association for Computing Machinery, New York, NY, USA, 545–559.  https://doi.org/10.1145/3544216.3544258
  * Cheng et al. (2008) Xiuzhen Cheng, Haining Shu, Qilian Liang, and David Hung-Chang Du. 2008.  Silent Positioning in Underwater Acoustic Sensor Networks.  IEEE Transactions on Vehicular Technology 57, 3 (2008), 1756–1766.  https://doi.org/10.1109/TVT.2007.912142
  * Curtis et al. (2014) Paul Curtis, Mahesh K Banavar, Sai Zhang, Andreas Spanias, and Vitor Weber. 2014.  Android acoustic ranging. In IISA 2014, The 5th International Conference on Information, Intelligence, Systems and Applications. IEEE, 118–123.
  * de Langis and Sattar (2020) Karin de Langis and Junaed Sattar. 2020.  Realtime Multi-Diver Tracking and Re-identification for Underwater Human-Robot Collaboration. In 2020 IEEE International Conference on Robotics and Automation (ICRA). 11140–11146.  https://doi.org/10.1109/ICRA40945.2020.9197308
  * DeMarco et al. (2013) Kevin J. DeMarco, Michael E. West, and Ayanna M. Howard. 2013\.  Sonar-Based Detection and Tracking of a Diver for Underwater Human-Robot Interaction Scenarios. In 2013 IEEE International Conference on Systems, Man, and Cybernetics. 2378–2383.  https://doi.org/10.1109/SMC.2013.406
  * Dong et al. (2016) Yuhan Dong, Xuelong Mi, and Yiqing Zhou. 2016.  Spatial Channel Model for Underwater Wireless Optical Communication Links: [Extended Abstract]. In Proceedings of the 11th ACM International Conference on Underwater Networks & Systems (WUWNet ’16). Association for Computing Machinery, New York, NY, USA, Article 18, 2 pages.  https://doi.org/10.1145/2999504.3001113
  * Erdélyi et al. (2018) Viktor Erdélyi, Trung-Kien Le, Bobby Bhattacharjee, Peter Druschel, and Nobutaka Ono. 2018\.  Sonoloc: Scalable positioning of commodity mobile devices.  (2018).
  * Flores et al. (2020a) Huber Flores, Naser Hossein Motlagh, Agustin Zuniga, Mohan Liyanage, Monica Passananti, Sasu Tarkoma, Moustafa Youssef, and Petteri Nurmi. 2020a.  Toward Large-Scale Autonomous Monitoring and Sensing of Underwater Pollutants.  (05 2020).
  * Flores et al. (2020b) Huber Flores, Agustin Zuniga, Naser Hossein Motlagh, Mohan Liyanage, Monica Passananti, Sasu Tarkoma, Moustafa Youssef, and Petteri Nurmi. 2020b.  PENGUIN: Aquatic Plastic Pollution Sensing Using AUVs. In Proceedings of the 6th ACM Workshop on Micro Aerial Vehicle Networks, Systems, and Applications (DroNet ’20). Association for Computing Machinery, New York, NY, USA, Article 5, 6 pages.  https://doi.org/10.1145/3396864.3399704
  * Foresti and Gentili (2000) G.L. Foresti and Stefania Gentili. 2000.  A Vision Based System for Object Detection in Underwater Images.  IJPRAI 14 (03 2000), 167–188.  https://doi.org/10.1142/S021800140000012X
  * Ghaffarivardavagh et al. (2020) Reza Ghaffarivardavagh, Sayed Saad Afzal, Osvy Rodriguez, and Fadel Adib. 2020.  Underwater Backscatter Localization: Toward a Battery-Free Underwater GPS. In Proceedings of the 19th ACM Workshop on Hot Topics in Networks (HotNets ’20). Association for Computing Machinery, New York, NY, USA, 125–131.  https://doi.org/10.1145/3422604.3425950
  * Guggenberger et al. (2015) Mario Guggenberger, Mathias Lux, and Laszlo Böszörmenyi. 2015.  An analysis of time drift in hand-held recording devices. In International Conference on Multimedia Modeling. Springer, 203–213.
  * Jang and Adib (2019) Junsu Jang and Fadel Adib. 2019.  Underwater Backscatter Networking. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM ’19). Association for Computing Machinery, New York, NY, USA, 187–199.  https://doi.org/10.1145/3341302.3342091
  * Jin et al. (2015) Haojian Jin, Christian Holz, and Kasper Hornbæk. 2015\.  Tracko: Ad-hoc mobile 3d tracking using bluetooth low energy and inaudible signals for cross-device interaction. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, 147–156.
  * Kewen et al. (2010) Liu Kewen et al. 2010\.  Research of MMSE and LS channel estimation in OFDM systems. In The 2nd international conference on information science and engineering. IEEE, 2308–2311.
  * Kuch et al. (2012) Benjamin Kuch, Giorgio Buttazzo, Elaine Azzopardi, Martin Sayer, and Arne Sieber. 2012\.  GPS diving computer for underwater tracking and mapping.  Underwater Technology The International Journal of the Society for Underwater 189 (07 2012), 189–194.  https://doi.org/10.3723/ut.30.189
  * Kuperman and Roux (2007) William A Kuperman and Philippe Roux. 2007.  Underwater acoustics.  (2007), 149–204 pages.
  * Lazik et al. (2015) Patrick Lazik, Niranjini Rajagopal, Oliver Shih, Bruno Sinopoli, and Anthony Rowe. 2015.  ALPS: A bluetooth and ultrasound platform for mapping and localization. In Proceedings of the 13th ACM conference on embedded networked sensor systems. ACM, 73–84.
  * Lin et al. (2021) Chi Lin, Yongda Yu, Jie Xiong, Yichuan Zhang, Lei Wang, Guowei Wu, and Zhongxuan Luo. 2021.  Shrimp: A Robust Underwater Visible Light Communication System. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. Association for Computing Machinery, New York, NY, USA, 134–146.  https://doi-org.offcampus.lib.washington.edu/10.1145/3447993.3448616
  * Luo et al. (2008) Hanjiang Luo, Yiyang Zhao, Zhongwen Guo, Siyuan Liu, Pengpeng Chen, and Lionel M. Ni. 2008\.  UDB: Using Directional Beacons for Localization in Underwater Sensor Networks. In 2008 14th IEEE International Conference on Parallel and Distributed Systems. 551–558.  https://doi.org/10.1109/ICPADS.2008.31
  * Maki et al. (2013) Tara Maki, Takumi Matsuda, Takashi Sakamaki, Tamaki Ura, and Junichi Kojima. 2013.  Navigation Method for Underwater Vehicles Based on Mutual Acoustical Positioning With a Single Seafloor Station.  Oceanic Engineering, IEEE Journal of 38 (01 2013), 167–177.  https://doi.org/10.1109/JOE.2012.2210799
  * Mao et al. (2016) Wenguang Mao, Jian He, and Lili Qiu. 2016.  CAT: high-precision acoustic motion tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 69–81.
  * Mishachandar and Vairamuthu (2021) B Mishachandar and S Vairamuthu. 2021.  An underwater cognitive acoustic network strategy for efficient spectrum utilization.  Applied Acoustics 175 (2021), 107861.  https://doi.org/10.1016/j.apacoust.2020.107861
  * Miskovic et al. (2015) Nikola Miskovic, Dula Nad, and Ivor Rendulic. 2015\.  Tracking Divers: An Autonomous Marine Surface Vehicle to Increase Diver Safety.  IEEE Robotics Automation Magazine 22, 3 (2015), 72–84.  https://doi.org/10.1109/MRA.2015.2448851
  * Monteiro and Marti (2020) Martín Monteiro and Arturo C. Marti. 2020.  Using smartphones as hydrophones: two experiments in underwater acoustics.  arXiv: Physics Education (2020).
  * Moon et al. (2009) Hyun Moon, Chang Ho Yu, and Jae Weon Choi. 2009.  Performance of Sensor Localization in Underwater Wireless Sensor Networks.  https://doi.org/10.13140/2.1.5126.4649
  * Munafo et al. (2015) Andrea Munafo, Jan Sliwka, and Joao Alves. 2015\.  Dynamic placement of a constellation of surface buoys for enhanced underwater positioning. 1–6.  https://doi.org/10.1109/OCEANS-Genova.2015.7271663
  * Nandakumar et al. (2019) Rajalakshmi Nandakumar, Shyamnath Gollakota, and Jacob Sunshine. 2019.  Opioid overdose detection using smartphones.  Science Translational Medicine 11 (01 2019), eaau8914.  https://doi.org/10.1126/scitranslmed.aau8914
  * Nandakumar et al. (2015) Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaniel Watson. 2015.  Contactless Sleep Apnea Detection on Smartphones. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys ’15). Association for Computing Machinery, New York, NY, USA, 45–57.  https://doi.org/10.1145/2742647.2742674
  * Nandakumar et al. (2016) Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016.  Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1515–1525.
  * Nasir et al. (2010) Ali A Nasir, Salman Durrani, and Rodney A Kennedy. 2010\.  Performance of coarse and fine timing synchronization in OFDM receivers. In 2010 2nd International Conference on Future Computer and Communication, Vol. 2. IEEE, V2–412.
  * Nađ et al. (2020) Đula Nađ, Filip Mandić, and Nikola Miskovic. 2020\.  Using Autonomous Underwater Vehicles for Diver Tracking and Navigation Aiding.  Journal of Marine Science and Engineering 8 (06 2020), 413\.  https://doi.org/10.3390/jmse8060413
  * Noh et al. (2015) Youngtae Noh, Dustin Torres, and Mario Gerla. 2015\.  Software-defined underwater acoustic networking platform and its applications.  Ad Hoc Networks 34 (01 2015).  https://doi.org/10.1016/j.adhoc.2015.01.010
  * ongoing (2022) ongoing. 2022.  Underwater Messaging Using Mobile Devices.  under submission (2022).
  * Peng et al. (2007) Chunyi Peng, Guobin Shen, Yongguang Zhang, Yanlin Li, and Kun Tan. 2007.  Beepbeep: a high accuracy acoustic ranging system using cots mobile devices. In Proceedings of the 5th international conference on Embedded networked sensor systems. ACM, 1–14.
  * Pottinger (2012) Mark Gerard Pottinger. 2012\.  LOCALISATION OF UNDERWATER SENSOR NODES IN CONFINED SPACES.
  * Rauchenstein et al. (2018) Lynn T. Rauchenstein, Abhinav Vishnu, Xinya Li, and Zhiqun Deng. 2018.  Improving Underwater Localization Accuracy with Machine Learning.  Review of Scientific Instruments 89, 7 (7 2018).  https://doi.org/10.1063/1.5012687
  * Restuccia et al. (2017) Francesco Restuccia, Emrecan Demirors, and Tommaso Melodia. 2017\.  ISonar: Software-Defined Underwater Acoustic Networking for Amphibious Smartphones. In Proceedings of the International Conference on Underwater Networks & Systems (WUWNET’17). Association for Computing Machinery, New York, NY, USA, Article 15, 9 pages.  https://doi.org/10.1145/3148675.3148710
  * Sesia et al. (2011) Stefania Sesia, Issam Toufik, and Matthew Baker. 2011\.  LTE-the UMTS long term evolution: from theory to practice.  John Wiley & Sons.
  * Sklivanitis et al. (2014) George Sklivanitis, Emrecan Demirors, Stella N. Batalama, Tommaso Melodia, and Dimitris A. Pados. 2014.  Receiver configuration and testbed development for underwater cognitive channelization. In 2014 48th Asilomar Conference on Signals, Systems and Computers. 1594–1598.  https://doi.org/10.1109/ACSSC.2014.7094734
  * Srinivasan et al. (2007) Ramji Srinivasan, S. Rajesh, N.R. Ramesh, S.M. Babu, Raju Abraham, Deepak Raphael, G.A. Ramadass, and Ma Atmanand. 2007\.  Design and testing of Control and Positioning System for Underwater mining Machine. 1 – 5.  https://doi.org/10.1109/OCEANS.2007.4449146
  * Sur et al. (2014) Sanjib Sur, Teng Wei, and Xinyu Zhang. 2014.  Autodirective audio capturing through a synchronized smartphone array. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. 28–41.
  * Tan et al. (2011) Hwee Tan, Roee Diamant, Winston Seah, and Marc Waldmeyer. 2011\.  A Survey of Techniques and Challenges in Underwater Localization.  Ocean Engineering - OCEAN ENG 38 (10 2011), 1663–1676.  https://doi.org/10.1016/j.oceaneng.2011.07.017
  * Tan et al. (2010) H.-P. Tan, A. F. Gabor, Z. A. Eu, and W. K. G. Seah. 2010\.  A Wide Coverage Positioning System (WPS) for Underwater Localization. In 2010 IEEE International Conference on Communications. 1–5.  https://doi.org/10.1109/ICC.2010.5501950
  * Tomczak (2011) Arkadiusz Tomczak. 2011\.  MODERN METHODS OF UNDERWATER POSITIONING APPLIED IN SUBSEA MINING.  AGH Journals of Mining and Geoengineering 35 (01 2011), 381–394.
  * Ullah et al. (2019) Inam Ullah, Yiming Liu, Xin Su, and Pankoo Kim. 2019\.  Efficient and accurate target localization in underwater environment.  IEEE Access 7 (2019), 101415–101426.
  * Vickery (1998) Keith Vickery. 1998\.  Acoustic positioning systems. A practical overview of current systems. In Proceedings of the 1998 Workshop on Autonomous Underwater Vehicles (Cat. No. 98CH36290). IEEE, 5–17.
  * Walls and Gagnepain (1992) Fred L Walls and J-J Gagnepain. 1992.  Environmental sensitivities of quartz oscillators.  IEEE transactions on ultrasonics, ferroelectrics, and frequency control 39, 2 (1992), 241–249.
  * Wang and Gollakota (2019a) Anran Wang and Shyamnath Gollakota. 2019a.  MilliSonic: Pushing the Limits of Acoustic Motion Tracking. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11.  https://doi.org/10.1145/3290605.3300248
  * Wang and Gollakota (2019b) Anran Wang and Shyamnath Gollakota. 2019b.  Millisonic: Pushing the limits of acoustic motion tracking. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.
  * Wang et al. (2021) Anran Wang, Dan Nguyen, Arun Sridhar, and Shyamnath Gollakota. 2021\.  Using smart speakers to contactlessly monitor heart rhythms.  Communications Biology 4 (03 2021), 319\.  https://doi.org/10.1038/s42003-021-01824-9
  * Wang et al. (2019) Anran Wang, Jacob E. Sunshine, and Shyamnath Gollakota. 2019\.  Contactless Infant Monitoring Using White Noise. In The 25th Annual International Conference on Mobile Computing and Networking (MobiCom ’19). Association for Computing Machinery, New York, NY, USA, Article 52, 16 pages.  https://doi.org/10.1145/3300061.3345453
  * Wen et al. (2006) Yang Wen, Wei Huang, and Zhongpei Zhang. 2006.  CAZAC sequence and its application in LTE random access. In 2006 IEEE Information Theory Workshop-ITW’06 Chengdu. IEEE, 544–547.
  * Wilson (1960) Wayne D Wilson. 1960\.  Equation for the speed of sound in sea water.  The Journal of the Acoustical Society of America 32, 10 (1960), 1357–1357.
  * Xue et al. (2020) Hua Xue, Jiadi Yu, Feng Lyu, and Minglu Li. 2020\.  Push the limit of multipath profiling using commodity WiFi devices with limited bandwidth.  IEEE Transactions on Vehicular Technology 69, 4 (2020), 4142–4154.
  * Yonggang et al. (2008) Wang Yonggang, Tang Jiansheng, Pan Yue, and Huangfu Li. 2008\.  Underwater communication goes cognitive. In OCEANS 2008. 1–4.  https://doi.org/10.1109/OCEANS.2008.5151898
  * Youssef et al. (2006) Moustafa Youssef, Adel Youssef, Chuck Rieger, Udaya Shankar, and Ashok Agrawala. 2006.  Pinpoint: An asynchronous time-based location determination system. In Proceedings of the 4th international conference on Mobile systems, applications and services. 165–176.
  * Zhang et al. (2017) Cheng Zhang, Qiuyue Xue, Anandghan Waghmare, Sumeet Jain, Yiming Pu, Sinan Hersek, Kent Lyons, Kenneth A Cunefare, Omer T Inan, and Gregory D Abowd. 2017\.  Soundtrak: Continuous 3d tracking of a finger using active acoustics.  Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (2017), 30.
  * Zhang et al. (2020) Yongzhao Zhang, Wei-Hsiang Huang, Chih-Yun Yang, Wen-Ping Wang, Yi-Chao Chen, Chuang-Wen You, Da-Yuan Huang, Guangtao Xue, and Jiadi Yu. 2020.  Endophasia: Utilizing acoustic-based imaging for issuing contact-free silent speech commands.  Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1–26.
  * Zhang et al. (2018) Yunting Zhang, Jiliang Wang, Weiyi Wang, Zhao Wang, and Yunhao Liu. 2018.  Vernier: Accurate and Fast Acoustic Motion Tracking Using Mobile Devices. In INFOCOM. IEEE.
  * Zhang et al. (2012) Zengbin Zhang, David Chu, Xiaomeng Chen, and Thomas Moscibroda. 2012\.  Swordfight: Enabling a new class of phone-to-phone action games on commodity phones. In Proceedings of the 10th international conference on Mobile systems, applications, and services. ACM, 1–14.
  * Zhao et al. (2018) Shuang Zhao, Zhenjie Wang, Kaifei He, and Ning Ding. 2018\.  Investigation on underwater positioning stochastic model based on acoustic ray incidence angle.  Applied Ocean Research 77 (06 2018), 69–77.  https://doi.org/10.1016/j.apor.2018.05.011
  * Zhu et al. (2016a) Zhongben Zhu, Sau-Lon Hu, and Huajun Li. 2016a.  Effect on Kalman based underwater tracking due to ocean current uncertainty. 131–137.  https://doi.org/10.1109/AUV.2016.7778660
  * Zhu et al. (2016b) Zhongben Zhu, Sau-Lon Hu, and Huajun Li. 2016b.  Kalman-based underwater tracking with unknown effective sound velocity. 1–9.  https://doi.org/10.1109/OCEANS.2016.7761086


# CHESSIoT support of event-based modeling for the Internet of Things
applications

Felicien Ihirwe  Address
Email<EMAIL_ADDRESS>
Innovation and technology Service (ITS) Lab
Intecs Solutions Spa
Pisa, Italy

###### Abstract

Internet of Things systems design and development suffers from heterogeneity
in different aspects. The component behaviors also change due to events being
internal or external and the system needs to take action subsequently. In this
paper, we demo the event-based modeling capabilities of the CHESSIoT tool on a
smart parking application. Different components was been decomposed with
functional and behavioral specifications following the component-based
approach. In the end, we have given a brief description of the future work.

###### keywords:

MDE, CHESSIoT

##  1 Background

Finding a vacant parking slot in a big parking lot can be complex and time-
consuming especially during peak periods. More complexes offer indoor parking
for their clients but few of them provide an easy way to identify a vacant
post for parking. Hence the clients are forced to drive through the parking
lot looking for a vacant space in which they could end up not finding any at
all. To address this issue, this document proposes a sensor-based smart
parking system.

A typical indoor smart parking system is composed of several nodes deployed
into different parking spaces with a consolidated central unit. The deployed
nodes collect the parking information to be used in direct decision-making or
for further analysis purposes. The two main technologies involved in the
modern indoor parking system car detection and identification are visual and
sensor-based. The more recent technologies are adopting computer vision
technologies but it is very expensive. Depending on the technology used, the
deployable nodes can be installed at a parking lot or at each parking slot in
a parking lot. For this exercise, a sensor-based technology is used. The next
figure 1 gives an show a parkeasy 111https://www.exportersindia.com/xaimen-
parkeasy/ sensor-based smart parking system.

Figure 1: Multi-level indoor parking system

From the figure 1, the system is composed of many different sensor nodes
deployed on top of parking slots. Each parking slot is composed of an
ultrasonic sensor fixed on the ceiling or on the floor of each parking slot.
Ultrasonic sensors work based on echolocation technology in which a sensor
transmits a sound wave to a given space and waits for an echo wave back. The
time between the sent pulse and the returned echo is used to calculate
distance and can give an idea if a slot is vacant or not. In a vacant space,
the time between transmitted sound and reflection is longer than in an
occupied space, hence the sensor can detect when space is occupied or not. The
object’s distance is recalculated taking into account the speed of a sound
wave. A node also has two separate LED indicators (RED and GREEN) for
indicating the occupancy of the slot, so, in case GREEN is ON means the
parking slot is vacant while RED being ON means the slot is occupied.

The modeling of a system’s node was done using the CHESSIoT environment, a
CHESS extension to cover the modeling and analysis of IoT systems. The full
description of CHESSIoT can be found in our previous work (Ihirwe et al.,,
2021). CHESS is a cross-domain, model-driven, component-based, and open-source
tool for the development of high-integrity systems (Debiasi. et al.,, 2021).
For the sake of simplicity, this paper showcase the capability of CHESSIoT
event-based behavioral modeling on a single slot.

Figure 2: CHESS editor structure

##  2 CHESSIoT modeling editor

CHESSIoT adds different IoT-related modeling functionalities on top of CHESS
such as ”IoT sub-view”, ”palletes” and several CHESSIoT specific commands to
cover IoT modeling activities in CHESS. The following figure depicts the CHESS
editor with the CHESSIoT extension.

##  3 Software modeling

The modeling of software components in CHESSIoT follows the component-based
modeling approach enforced by CHESS. In this manner, the software main
components are decomposed separately with their internal functionalities and
behaviors and later be linked together to fulfill a certain task. The
following figure depicts the physical structure of a node.

Figure 3:  The physical structure of a node

###  3.1 Components definition

Modeling the software components is done in CHESS’s component view. In
CHESSIoT, the software components such as IoTElement, VirtualBoard,
VirtualEntity are the main modeling elements. They are used to encapsulate the
system’s main part structure, operations, and behaviours. The process starts
by defining the system’s main components, interfaces, and operations. This is
done by using UML class diagrams. Figure 4. shows different main components
involved in a node. From the figure 4, and as mentioned in the previous
section, the following components are defined as the main components for a
node.

Figure 4: A node software components and interfaces

RedLED component is of <<IoTElement>>type and it represents an red LED
indicator. This element implements the ”IBlinkRed” interface which imposes the
“blinkRed()” operation.

GreenLED: this is of type <<IoTElement>>and it represents an green indicator.
The same as the RedLED, GreenLED implements the “IBlinkGreen” interface which
imposes the “blinkGreen()” operation. (The above two elements will act as
consumers, so they won’t be initiating any event by themselves)

UltrasonicSensor: this is of type <<IoTElement>>and it represents an sensor.
This element implements the “ISense” interface which imposes the “sense()”
operation.

Board represents a computing device such as Arduino, Raspberry Pi,… It is of
type <<Board>>and realizes 3 different interfaces to interact with the LEDs
and a sensor. Each interface exposes different operations depending on the
action required.

###  3.2 Modeling components internal structure

To abide by the component-based approach each component’s internal structure
needs to be decomposed separately by specifying their corresponding internal
sub-components (if any) and ports. Ports are used to support the communication
between two or more components exposing or requiring the interfaces from other
components. In CHESSIoT, the component’s messages are passed through the port
using the required or provided interface operations. The component also has an
associated initial state. The modeling of behavior information is modeled
using a UML state machine and will be discussed later in this section. The
modeling of internal structure is performed in the component view using their
custom composite structure diagram.

####  3.2.1 Red/Green LED

The Red/Green LEDs have only one unique port to interact with the Board. The
ports provide the blinking interface but they also require the “ISendBlinkRed
& ISendBlinkGreen” from the board respectively. The following figure depicts
the internal structure of an LED showcasing the internal structure of ports.
Figure 5 shows the internal structure of an LED component.

Figure 5: LED internal structure

####  3.2.2 Ultrasonic sensor

The same as the LEDs, the sensor port also will provide the “ISense” interface
and require the “IReceiveSense” from the board. The figure 6 diagram shows how
the ultrasonic sensor internal structure.

Figure 6: Ultrasonic sensor internal structure

####  3.2.3 A node

Modeling the internal structure of a node is done on top of a board and reuse
the IoTElements that have already been modeled before. Figure 7 shows the
internal structure of a board.

Figure 7: Node internal structure

Note: The instantiating other IoTElements inside the board component is done
by dragging them from the model explorer to the board component in the editor.
After dragging them, it is more likely that the ports won’t show. The easiest
way to show the component’s content is to: Select the component → Right Click→
Filter→ Show/Hide content. From there, select the border ports you want to
show.

###  3.3 Modeling components behaviors

Modeling of component’s behaviours is done using UML state machines, events,
guards, and actions. In our approach, where it is necessary, each component
should have its own internal state machine describing its behaviour. To start
with, each component always has an initial declared state while each state
should have at least one event of type “IoTEvent” to be executed either
“OnEntry, OnExit or continuously”. In the following part, we will go over all
the components’ behaviour specifications. But before continuing, let us first
familiarize ourselves with different behavioral terminologies proposed by
CHESSIoT.

Payload: This is a standalone and straightforward object to carry information
to be passed between components. In CHESSIoT, the payload can have zero or
many primitive or derived properties to be defined in a message. For instance,
suppose a component message to be communicated among components contains a
string value, an integer, or even an instance of another payload.

IoTState: This serves to keep the component state from its initial
participation until its disposal in the system. IoTState extends actual UML
states but in addition to that, IoTState carries information related to what
events need to be taken care of at a certain point in time. For instance,
OnEntry or OnExit events are triggered when entering or exiting a state. An
IoT state can also trigger an internal event, which corresponds to an internal
action to be taken.

IoTEvent: Events in CHESSIoT are triggered in a different manner depending on
the state of the component. An IoTvent can be incoming, outgoing, or generic,
which means it can come from the inside-out, from the outside, or internal. A
GenericEvent is an event that gets triggered internally to the component, for
example, changing variable value.

IoTAction: IoTAction(s) can be of different types depending on the kind of
action to be performed. For instance, the SendPayload action is referred to
when an OutgoingEvent is triggered to send the payload through a specified
port while ReceivePayload is used on IncomingEvents to receive messages from
another components. A GenericAction does not require to access the component’s
ports, for example, changing the component’s property value.

State Transition: In CHESSIoT, state transitions enable transiting from the
source state to a target state, abiding the trigger from the guard value.
Guard expressions are boolean expressions defined based on state values. They
serve to initiate a state transition by checking whether the OnExit event has
been performed correctly.

Two stages are involved when modeling the behaviour of the component. First,
the definition of payload, events, and actions that a component will use. This
is done by using the component inner class diagram. The second step is to
create an internal state machine diagram which will then make use of events
and actions defined above. The next part details the behavior specification of
the node components using a state machine.

####  3.3.1 LED’s state machine, event and actions

As shown in the figure 8, an LED has only two continuous states, “ON” and
“OFF”. To trigger transition from one state to another or to self, a guard
condition must be fulfilled.

Figure 8: LED state machine

The only IoTEvent an LED executes is an incoming event to receive the payload
from the board containing the state of either being low or high (OFF/ON).
Figure 9 shows the LED’s events and action definition.

Figure 9: LED event and actions

####  3.3.2 Ultrasonic Sensor state machine, event, and actions

From the figure 10, the Ultrasonic sensor contains 2 continuous states “SENSE”
for sensing and “SEND” for sending the payload to the board. So it will use
the ”internalEvent” event type and ”genericAction” action type for sensing and
later use an ”outgoinEvent” and “SendPayload” action for sending the value to
the board. The state transitions are triggered by the fulfilling their
corresponding guards. Figure 11 shows the sensor’s events and action
definition.

Figure 10: Sensor state machine Figure 11: Sensor event and actions

####  3.3.3 Node state machine, event, and actions

The figures 13 depict the node’s behavioural specification (which in turn can
be simply referred to as a board). The three main states of a node are
“ACQUISITION”, ”RED_ON/GREEN_OFF” and “RED_OFF/GREEN_ON”. Figure 13 shows the
board’s events and action definition. The main triggering condition for the
state change is the duration value from the payload. When the time taken by
the echo wave to come is greater or equal to 300ms then the space is vacant,
while when the echo-wave arrival duration is less than 300ms that means there
is a blocking object then the slot is occupied. The 300ms value is arbitrary,
it may be adjusted depending on the position of the sensor.

Figure 12: Node state machine Figure 13: Node event and actions

The purpose of modeling the behavioral aspect of a system in CHESSIoT is to
late generated the platform-specific code. This will be achieved by employing
the ThingML platform. The ThingML language combines well-proven software
modeling constructs aligned with UML (state-charts and components) and an
imperative platform-independent action language to construct the intended IoT
applications (Ihirwe et al.,, 2020). ThingML code generator targets many
popular programming languages such as C/C++, Java, and Javascript, and about
ten different target platforms (ranging from tiny 8bit microcontrollers to
servers) and ten different communication protocols (Harrand et al.,, 2016).

##  4 Conclusion

Event-based modeling in IoT systems helps to discover the system’s hidden
behaviors before the development of the system by itself. In this paper, we
showcased the capability of the CHESSIoT extension on modeling the behavioral
aspects of an IoT system. In the future, CHESSIoT will support the generation
of platform-specific code by employing the ThingML tool. Different IoT-
specific real-time analyses will be supported too.

## Acknowledgments

This work has received funding from the Lowcomote project under European
Union’s Horizon 2020 research and innovation program under the Marie
Skłodowska-Curie grant agreement n$\mathrm{\SIUnitSymbolDegree}$ 813884.

## References

  * Debiasi. et al., (2021) Debiasi., A., Ihirwe., F., Pierini., P., Mazzini., S., and Tonetta., S. (2021).  Model-based analysis support for dependable complex systems in chess.  In Proceedings of the 9th International Conference on Model-Driven Engineering and Software Development - MODELSWARD,, pages 262–269. INSTICC, SciTePress.
  * Harrand et al., (2016) Harrand, N., Fleurey, F., Morin, B., and Husa, K. E. (2016).  Thingml: A language and code generation framework for heterogeneous targets.  In Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems, MODELS ’16, page 125–135, New York, NY, USA. Association for Computing Machinery.
  * Ihirwe et al., (2021) Ihirwe, F., Di Ruscio, D., Mazzini, S., and Pierantonio, A. (2021).  Towards a modeling and analysis environment for industrial iot systems.  STAF’21, Bergen, Norway. International workshop on MDE for Smart IoT Systems (MESS’21) with STAF2021.
  * Ihirwe et al., (2020) Ihirwe, F., Di Ruscio, D., Mazzini, S., Pierini, P., and Pierantonio, A. (2020).  Low-code engineering for internet of things: A state of research.  In Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, MODELS ’20, New York, NY, USA. Association for Computing Machinery.