# Deep Reinforcement Learning for Power Control in Next-Generation WiFi
Network Systems††thanks: DISTRIBUTION STATEMENT A. Approved for public
release.

Ziad El Jamous1, Kemal Davaslioglu2, and Yalin E. Sagduyu3 1BlueHalo,
Rockville, MD, USA  2University Technical Services, Inc., Greenbelt, MD, USA
3National Security Institute, Virginia Tech, Arlington, VA, USA  Email:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

This paper presents a deep reinforcement learning (DRL) solution for power
control in wireless communications, describes its embedded implementation with
WiFi transceivers for a WiFi network system, and evaluates the performance
with high-fidelity emulation tests. In a multi-hop wireless network, each
mobile node measures its link quality and signal strength, and controls its
transmit power. As a model-free solution, reinforcement learning allows nodes
to adapt their actions by observing the states and maximize their cumulative
rewards over time. For each node, the state consists of transmit power, link
quality and signal strength; the action adjusts the transmit power; and the
reward combines energy efficiency (throughput normalized by energy
consumption) and penalty of changing the transmit power. As the state space is
large, Q-learning is hard to implement on embedded platforms with limited
memory and processing power. By approximating the Q-values with a deep
Q-network (DQN), DRL is implemented for the embedded platform of each node
combining an ARM processor and a WiFi transceiver for 802.11n. Controllable
and repeatable emulation tests are performed by inducing realistic channel
effects on RF signals. Performance comparison with benchmark schemes of fixed
and myopic power allocations shows that power control with DRL provides major
improvements to energy efficiency and throughput in WiFi network systems.

###### Index Terms:

Power control, WiFi, deep reinforcement learning, energy efficiency,
throughput, embedded implementation, edge computing, emulation tests.

##  I Introduction

Machine learning (ML) provides powerful means to learn from rich data
representations and solve complex tasks. Recent algorithmic and computational
advances have supported the broad use of deep neural networks for diverse
applications ranging from computer vision to natural language processing. As
high-fidelity training data with labels is not always readily available,
reinforcement learning (RL) has emerged as a viable solution for taking
actions to maximize a cumulative reward over a time horizon while interacting
with and learning from the environment. As a model-free RL algorithm,
Q-learning aims to learn the value of an action in a particular state, namely
the Q-value. However, the number of Q-value evaluations grows with the state
and action space, making the maintenance of Q-tables computationally
infeasible for a large-scale problem as well as an embedded implementation (as
needed for edge computing in wireless communication applications). Deep
reinforcement learning (DRL) has emerged as a computationally efficient
solution by approximating the Q-values by training a deep Q-network (DQN).

As the spectrum data associated with wireless communication networks is highly
complex due to network topology, channel, interference, and traffic effects,
deep learning (DL) finds diverse sets of wireless applications including RF
signal classification, waveform design, and security [1]. Wireless networks
involve dynamic effects with temporal correlations such as those induced by
network mobility and waveform patterns. Therefore, RL has the strong potential
in supporting wireless communication systems to learn from the network
environment and optimize the network performance over time without relying on
the availability of training data or any other prior knowledge on the
underlying model to learn [2]. The RL solutions have been studied for general
communication systems with various tasks including channel access [3, 4, 5, 6,
7, 8, 9, 10, 11, 12, 13] and power control [14, 15, 16, 17, 18, 19]. The
typical approach in previous works has been to use analytical models and
evaluate the performance via theoretical evaluations or simulations. Instead,
the real protocol stack of WiFi communications is used in this paper to derive
the states, actions, and rewards of DRL (that is needed to tackle the
complexity of the underlying optimization problem), and high-fidelity
experiments with real radios are performed to show the benefits of DRL for
WiFi network systems.

In this paper, a multi-hop wireless network system is considered, where each
node is equipped with an embedded platform and a WiFi (in particular, 802.11n)
transceiver. Each node may act as the source, the destination, or the relay
for network packet traffic. With the emergence of smart WiFi systems such as
in WiFi 7 [20], it is necessary to optimize the communication functionalities
for WiFi to meet the growing demand of emerging applications, including
extended reality (XR), gaming, video conferencing and streaming. These
applications require high-rate communications and benefit from energy
efficiency with prolonged network lifetime (as needed for battery-operated
networked devices).

There are many configuration parameters for next-generation WiFi systems that
involve complex dependencies between parameters and their joint optimization
especially in dense deployments and coexistence in shared bands. ML has been
considered as viable means to optimize next-generation WiFi systems [21].
Potential scenarios include 5G/6G-WiFi convergence, Multiple Radio Access
Technology (multi-RAT) including WiFi, and Internet of Things (IoT) devices
connected with WiFi (potentially in multi-hop network setting such as in smart
warehouses, where multiple hops are needed to reach IoT devices over an
extended area) and mesh WiFi networks such as Google Nest and Amazon Echo. All
these applications can benefit from improvements in both energy consumption
(that translates to longer battery and network lifetime) and throughput [22].
DL has been used for adaptation of channel access mechanisms in WiFi to
spectrum dynamics [23] and DRL has been used for coexistence of WiFi with
cellular communications [24].

The backoff mechanism of carrier-sense multiple access with collision
avoidance (CSMA/CA) as used in 802.11 protocols generates temporal
correlations in the channel access pattern. Also, network mobility induces
temporal correlations in observed spectrum data. Note that no training data is
assumed to be available in advance for the use of supervised ML for the
adaptation of WiFi communications to spectrum dynamics. Similarly, no prior
model is assumed to be available to formulate an optimization problem in
advance to solve. Therefore, RL is considered in this paper for WiFi nodes to
adapt themselves to protocol and network dynamics, and optimize their energy
efficiency which is a critical concern especially for battery-operated
devices. In addition, a distributed RL approach is pursued without considering
a centralized node to prevent communication and computation bottlenecks and
avoid a single point of failure.

In this RL formulation, the states are the transmit power, link quality and
signal strength that are either maintained or measured by each node. The
actions adjust the transmit power. As the corresponding Q-table is large, a
DRL solution is used in this paper and implemented on the embedded platform of
each node. The objectives of maximizing the network throughput and minimizing
the energy consumption (that is critical for edge devices operating on battery
power) are integrated in an energy efficiency objective. A reward function is
built to combine the energy efficiency with the penalty on changing the
transmit power excessively (to reduce the extra processing burden and avoid
performance fluctuations).

For the system-level assessment, the proposed DRL solution for power control
is implemented on the ARM processor of each node’s embedded platform that
connects to an Alfa AWUS036NHA USB wireless adapter for 802.11n communications
at the 2.4 GHz band (while the algorithm is independent of frequency band).
The performance is measured in terms of energy efficiency and throughput.

Performance evaluation is carried out with high-fidelity emulation tests by
employing real transceivers and embedded implementation of real network
protocol stack (including the proposed power control solution as well as other
network protocols like routing for multi-hop network operations) and real
packet traffic. Channel effects among transceivers are emulated with a network
channel emulator. This evaluation approach is more realistic than simulations
that typically use simplistic models of physical layer that ignore effects
such as nonlinearity, filtering, and inter modulation that are caused by
hardware. An alternative is making over the air (OTA) transmissions in a
traditional testbed environment. However, channel effects change over time and
OTA tests cannot guarantee repeatable and controllable evaluation of the
proposed scheme and benchmark schemes under the exact same scenario
corresponding to the same network topology/mobility and channel effects [25,
26].

For the emulation tests in this paper, RFnest [27] is used as the network
channel emulator. WiFi transceivers are connected to RFnest with cables.
RFnest controls channel effects on RF signals according to the network
topology and mobility pattern, and allows channel sensing and interference, if
any, among nodes. For that purpose, RFnest converts the signals from each
transmitter to baseband, processes it with FPGA to impose channel effects, and
then converts it back to 2.4 GHz before sending it to the corresponding
receiver.

Our emulation tests for a multi-hop mobile network show that DRL effectively
interacts with and learns from the network environment, and builds the best
strategy to control the transmit power for WiFi communications. This way, DRL
provides major improvements to energy efficiency (up to $24\%$) and throughput
(up to $22\%$) compared to the benchmark schemes of fixed and myopic power
allocations.

The contributions of the paper are summarized below:

  * •

A novel DRL-based power control scheme is presented to improve throughput and
energy performance of WiFi communications in a multi-hop network of mobile
nodes.

  * •

This algorithmic solution is implemented in an embedded platform that
integrates the real network protocol stack with WiFi transceiver for each
node.

  * •

Emulation tests are executed with real RF devices in a controllable and
repeatable experimentation environment, such that the proposed solution is
compared with the benchmark scheme under the same channel conditions.

  * •

Performance improvement in terms of throughput and energy efficiency is
demonstrated through emulation tests compared to the benchmark schemes of (i)
using a fixed transmit power (randomly selected over time) and (ii) solving
the optimization myopically.

The rest of the paper is organized as follows. Section II describes the system
model. Section III presents the DRL-based power control solution. Section IV
provides the performance evaluation results. Section V concludes the paper.

##  II System Model

###  II-A Network Model

A multi-hop wireless network of mobile nodes is considered. Each node equipped
with a WiFi transceiver (802.11n) communicates with each other (potentially
over multiple hops) through WiFi. Nodes are moving according to the random
waypoint model. The time is indexed with respect to the WiFi frame duration.
At any given time, each node measures channel conditions, namely the link
quality and the signal strength, and (if transmitting) adjusts its transmit
power over time to adapt to dynamics of network topology and channel as well
as dynamics of the network protocol itself. DRL is used to determine the level
of transmit power. Two benchmark schemes of fixed and myopic power allocations
are considered for comparison purposes.

Each node may act as a source, a destination, or a relay for the multi-hop
network traffic. Each node generates random unicast traffic with the randomly
selected destination. For multi-hop routing, the Babel routing algorithm [28]
is used as a loop-avoiding distance-vector routing protocol for IPv6 and IPv4
with fast convergence properties. With computationally light implementation,
it is suitable for embedded platforms. Expected transmission count (ETX) is
used as the link cost.

###  II-B Embedded Radio Implementation Approach

A system-level implementation is considered. For each node, the proposed power
control scheme with DRL is implemented on the ARM processor as an embedded
solution that controls the Alfa AWUS036NHA USB wireless adapter for 802.11n
communications at $2.4$ GHz band. The transmit power ranges from $0$ dBm to
$20$ dBm. The routing protocol is also implemented on the ARM for fast
operation at the edge device. The structure of a node with WiFi transceiver
and embedded processor is shown in Fig 1.

Figure 1: The structure of a node with WiFi transceiver and embedded
processor.

###  II-C Performance Evaluation Approach based on Emulation Tests with Real
Transceivers

The performance is measured by emulation tests using real WiFi transceivers,
real network protocol stack, and real network packet traffic. The emulation
environment is illustrated in Fig 2. Instead of over-the-air transmissions,
transmissions are made over emulated channels. RFnest is used as the network
channel emulator [27]. The antenna of each transceiver is removed and
connected via an RF cable to the network channel emulator. Signals out of each
transmitting node at 2.4 GHz are fed to the network channel emulator that are
converted to the baseband and processed by the FPGA of RFnest to induce the
desired channel effects (such as pathloss and multipath), and then converted
back to 2.4 GHz and fed to the antenna ports of the receiver node.

Channels are controlled by the network channel emulator to reflect the
mobility effects. In experiments, random waypoint model is used for node
mobility. A full mesh network channel emulation is considered such that
interference effects among nodes are also induced in addition to the channel
effects. The emulation of interference effects drives the CSMA/CA-based
backoff mechanism of WiFi. These emulation tests allow us to compare the
proposed solution with the benchmark scheme under the same channel conditions.

Figure 2: Emulation testbed environment with the network channel emulator
controlling channels among nodes corresponding to real transceivers and
embedded processors.

##  III Deep Reinforcement Learning for Controlling WiFi Power

Each node adjusts its transmit power at any given time. For that purpose, a
DRL solution is used. As the network conditions change over time, DRL provides
the mechanism to adapt to the network environment (including network topology,
channel, and traffic effects) compared to supervised learning approaches when
a model is learned from a predefined labeled dataset. With RL, nodes can
interact with the environment, perform actions, and learn over time by trial
and error.

In RL, nodes acting as agents take actions and receive rewards from the
environment, thereby adapting their decisions in real time to the changes in
the environment. However, the computational overhead involving the maintenance
of the Q-table grows significantly with the state and action space [7], posing
a practical limitation for implementation. DRL reduces the computational
burden of conventional RL, leading to a more practical implementation. One way
of implementing DRL is based on training of a DQN. The DQN involves a deep
neural network to approximate the Q-function in Q-learning. The objective in
Q-learning is to maximize the expected discounted reward. The drawback of
Q-learning is that the magnitude of Q-value evaluations grows exponentially
with the state and action space. Thus, the DQN overcomes this drawback by
approximating the Q value with the training of a deep neural network.

Q-learning is formulated as follows. The reward at time $t$ is $r_{t}$ and the
discounted reward is $R_{t}=r_{t}+\gamma r_{t+1}+\gamma^{2}r_{t+2}+\cdots$,
where $\gamma\in[0,1]$ is the discount factor. The Q-function constructs a
policy $\pi$ such that $Q_{\pi}(s,a)=\mathbb{E}[R_{t}|s_{t}=s,a_{t}=a]$, where
$a_{t}$ and $s_{t}$ denote the action and the state, respectively, at time
$t$. The optimal action-value function $Q^{\star}(s,a)=\max_{\pi}Q_{\pi}(s,a)$
obeys the following Bellman equation:

$\displaystyle
Q^{\star}(s,a)=\mathbb{E}_{s^{\prime}}\left[r+\gamma\max_{a^{\prime}}Q^{\star}(s^{\prime},a^{\prime})|s,a\right],$
(1)

where $s^{\prime}$ denotes the next state and $a^{\prime}$ is the action that
maximizes the $Q^{\star}(s^{\prime},a^{\prime})$ expression. The deep
Q-learning uses a deep neural network, parameterized by ${\theta}$ to
represent $Q(s,a;{\theta})$, which is iteratively optimized [29] by maximizing

$\displaystyle\mathbb{E}_{s,a,r,s^{\prime}}\left[r\>|\>\gamma\max_{a^{\prime}}Q(s^{\prime},a^{\prime};\theta_{i}^{-})-Q(s,a;\theta_{i})\right],$
(2)

where $\theta_{i}$ is the set of parameters for the deep neural network at
iteration $i$ and $\theta_{i}^{-}$ represents the parameters of the target
network that is frozen for a number of iterations while updating the value
(online) network. In deciding which actions to take, $\epsilon$-greedy policy
is used to select the action that maximizes the Q-value with probability
$1-\epsilon$ and a random action with probability $\epsilon$ that balances the
exploration-exploitation tradeoff. The DQN agent [30] stores the experiences
$(s,a,r,s^{\prime})$ in a replay memory to train the deep neural network by
sampling mini-batches of experiences, which is known as experience replay.

For any node $n$ at time $t$, state $s_{n}(t)$, action $a_{n}(t)$, and reward
$r_{n}(t)$ are defined as follows.

_State_ $s_{n}(t)$: The state corresponds to the tuple of
$\\{P_{n}(t),L_{n}(t),S_{n}(t)\\}$, where $P_{n}(t)$, $L_{n}(t)$, and
$S_{n}(t)$ are the transmit power level, the link quality and the signal
strength, respectively, for node $n$ at given time $t$. In our experiments,
based on the WiFi transceiver properties, $P_{n}(t)$ takes values from $0$ dBm
to $20$ dBm with $1$ dB increments, $L_{n}(t)$ takes values from $0$ to $70$
with $1$ increments, and $S_{n}(t)$ takes values from $-110$ dBm and $-40$ dBm
with $1$ dB increments.

_Action_ $a_{n}(t)$: The action of node $n$ at any given time $t$ is changing
the transmit power, namely selecting $\Delta P_{n}(t)=P_{n}(t)-P_{n}(t-1)$,
where $\Delta P_{n}(t)\in\\{\Delta,0,-\Delta\\}$. There are three possible
actions. In experiments, we set $\Delta=1$ (measured in dBm).

_Reward_ $r_{n}(t)$: The reward of node $n$ at time $t$ is computed as

$r_{n}(t)=\frac{T_{n}(t)}{E_{n}(t)}-c\cdot\Delta P_{n}(t),$ (3)

where $T_{n}(t)$ is the throughput received by node $n$ at time $t$,
$E_{n}(t)$ is the total energy consumption normalized with respect to the
packet duration that corresponds to one time increment, and $c$ is a positive
constant ($c=0.1$ in our experiments). The unit for throughput is Mbps and the
unit of energy efficiency is Mbps/Joule. The second term in the reward
corresponds to the penalty of changing the transmit power (it is needed to
avoid excessive fluctuations in decision space). When node $n$ is not
transmitting at time $t$, $r_{n}(t)=0$ is imposed.

Note that if conventional Q-learning is used, the corresponding size of state
space is $21\times 71\times 71=105,861$ (there are $21$ levels for the
transmit power, $71$ levels for the link quality, and $71$ levels for the
signal strength). As the number of possible actions is $3$, the size of
Q-table in case of conventional RL would be $105,861\times 3=317,583$. It is
practically difficult (if not infeasible) to maintain and operate over such a
large Q-table at an edge device that relies on limited memory and processing
capabilities. Hence, a DRL approach is adopted to reduce the computational
complexity and memory requirements for the support of edge applications.

To approximate the Q value in DQN, a deep neural network is trained. The deep
neural network maps the input states to the pairs of actions and Q-values. For
that purpose, a three-layer feedforward neural network (FNN) is used. The
input layer takes the states as the input. There are 140 and 70 neurons in the
hidden layers. The output layer has 3 neurons corresponding to three possible
actions. A rectifying linear unit (ReLU) activation function is used for the
hidden layers. The FNN is trained by minimizing the mean squared error (MSE)
loss function given by

$\displaystyle\ell(\theta_{i})=\left(r_{n}(t)+\gamma\max_{a^{\prime}}Q^{\star}(s^{\prime},a^{\prime},\theta_{i}^{-})-Q(s,a,\theta_{i})\right)^{2}.$
(4)

The overall DRL solution is summarized in Fig. 3. The DRL agent for each node
is trained by the help of its neighbors in a distributed setting without
relying on a centralized controller. Each node observes its state and acquires
states of other nodes from local messages to form its observation vector
$\bm{s}_{i}(t)$, selects and performs its action, receives reward
$r_{i}(\bm{s}_{i}(t),a_{i}(t))$, broadcasts its action $a_{i}(t)$ to its
neighbors, stores transition in replay memory, and updates weights for its
DQN. The details of the deep Q-learning algorithm are given in Algorithm 1.

Figure 3: Illustration of the overall DRL solution.

1 Initialize replay memory $\mathcal{M}$ ;

2 Initialize action-value function $Q$ with random weights;

3 for  _episode = $1,...,M$ _ do

4

5 for  _episode = $1,...,M$ _ do

6 for  _each node $i$ (in parallel)_ do

  1. 71.

Observe its state and acquire states of other

8 nodes from local messages to form observation

vector $\bm{s}_{i}(t)$.

* 92.

Select action

10 $a_{i}(t)=\arg\max_{a_{i}(t)\in\mathcal{A}}Q(\bm{s}_{i}(t),a_{i}(t))$

w.p. $1-\epsilon$ or a random action $a_{i}(t)\in\mathcal{A}$ w.p. $\epsilon$.

* 3.

Perform an action $a_{i}(t)$.  4. Receive reward
$r_{i}(\bm{s}_{i}(t),a_{i}(t))$.  5. Broadcast its action $a_{i}(t)$ to its
neighbors.  6. Store transition in replay memory $\mathcal{M}$.  117. Update
weights.

12 end for

13

14 end for

15

16 end for

Algorithm 1 Deep Q-learning algorithm. Figure 4: The testbed for hardware-in-
the-loop emulation tests.

##  IV Performance Evaluation with Emulation Tests

For system-level evaluation, emulation tests are performed with embedded
platforms, each (corresponding to one node) equipped with an Alfa AWUS036NHA
USB wireless adapter for 802.11n and an ARM processor hosted on a software-
defined radio (SDR). The ARM processor hosts and executes the full network
protocol stack including the WiFi transmit power control based on DRL.

The emulation test setup is shown in Fig. 4. The frequency of communications
is set as 2.4 GHz. The power control allows each node to adjust its transmit
power automatically from $0$ dBm to $20$ dBm. Algorithm 1 involves distributed
operation of nodes, each computing their actions and exchanging information
with neighbor nodes. Therefore, it is independent of the number of nodes and
network mobility. A multi-hop mobile network of $5$ nodes is considered in
emulations tests, where each node moves according to the random waypoint model
with $5$ m/s speed. Network channel emulator (RFnest) imposes the changes in
the network topology on channel conditions and controls the RF signals
accordingly by accounting for the distances among nodes that change over time.
The performance is measured in terms of two metrics:

  * •

_Throughput_ : total number of bits received by all destinations per second).

  * •

_Energy efficiency_ : throughput per unit energy consumption, where energy is
measured by total (transmit and processing) power consumption over the packet
duration.

Network traffic is generated with iPerf3. The throughput by each node is
measured and energy efficiency is computed. In this setting, the node that is
training becomes the iPerf client and neighbors become the iPerf servers. This
approach makes training more communication-efficient. Fig. 5 shows an example
of how transmit power levels change over time as determined by the DRL agent
of a node that reacts to changes in the network (including topology, channel
and traffic effects) and adjusts the transmit power levels accordingly. Each
episode corresponds to one MAC frame. In 802.11n, the frame size is bounded by
a limit of about $5.5$ ms. The reward of the corresponding DRL agent is shown
in Fig. 6 as a function of time. Note that the DRL environment considered in
this paper differs from most RL environments, where the maximum reward is a
constant. Since the link distances change, so does the maximum reward that an
agent can achieve in the formulation considered in this paper.

Figure 5: Transmit power of DRL. Figure 6: Reward of DRL. TABLE I: Average performance measured in emulation tests. Algorithm | Energy Efficiency | Throughput
---|---|---
| (Mbps/J) | (Mbps)
Fixed power | $2.601$ | $5.723$
Myopic allocation | $2.893$ | $5.983$
DQN | 3.232 | 6.964

The average performance of the proposed scheme (based on DRL) collected over
more than 30 epochs is shown in Table I. Two benchmark schemes are considered:
(i) random selection of transmit power from $0$ dBm, $10$ dBm, or $20$ dBm and
(ii) myopic allocation of transmit power (by solving (1) with $\gamma=0$).
Experiments with real radios and embedded implementation show that DRL
achieves up to $24\%$ performance gain in terms of energy efficiency and
$22\%$ performance gain in terms of throughput.

The DRL model that is used by the ARM processor takes only 20.8 KFlops to make
a decision, which is fairly small compared to the models typically used in
computer vision. The ARM processor (such as the one available on the Raspberry
Pi-4B (1GB)) consumes 1.50 GFlops/W [31]. Then, the power control algorithm
takes only 13.9 $\mu W$ (or -18.6 dBm), which is fairly small compared to the
amount of power required to transmit or receive. Also, power control updates
can be made less frequently than transmit decisions, further reducing the
power consumption for computing.

##  V Conclusion

A DRL solution for transmit power control was presented for WiFi network
systems. In a multi-hop mobile network, each node observes the states of
transmit power, link quality and signal strength, takes actions of selecting
its transmit power, and optimizes a cumulative reward based on energy
efficiency following a distributed operation. For each node, this solution was
implemented on an embedded platform that integrates an ARM processor for the
network protocol stack (including power control with DRL) and a WiFi
transceiver. High-fidelity emulation tests with real packet traffic showed
that the DRL-based power control can achieve up to $24\%$ improvement in
energy efficiency while maintaining $22\%$ improvement in throughput compared
to the fixed and myopic power allocation schemes, thereby prolonging network
lifetime as well as serving the ever-increasing demand for high communication
rates.

## References

  * [1] T. Erpek, T. O’Shea, Y. E. Sagduyu, Y. Shi, and T. C. Clancy, “Deep Learning for Wireless Communications,” _Development and Analysis of Deep Learning Architectures_ , Springer, 2020.
  * [2] A. Feriani and E. Hossain, “Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial,” _IEEE Communications Surveys & Tutorials_, 2021.
  * [3] Z. Xu, Y. Wang, J. Tang, J. Wang, and M. C. Gursoy, ”A deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs,” _IEEE International Conference on Communications (ICC)_ , 2017.
  * [4] S. Wang, H. Liu, P. H. Gomes and B. Krishnamachari, “Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Network,” _IEEE Transactions on Cognitive Communications and Networking_ , June 2018.
  * [5] O. Naparstek and K. Cohen, “Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access,” _IEEE Transactions on Wireless Communications_ , Jan. 2019.
  * [6] Y. Yu, T. Wang and S. C. Liew, “Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks,” _IEEE Journal on Selected Areas in Communications_ , June 2019.
  * [7] Y. Yu, S. C. Liew and T. Wang, “Carrier-Sense Multiple Access for Heterogeneous Wireless Networks Using Deep Reinforcement Learning,” _IEEE Wireless Communications and Networking Conference (WCNC)_ , 2019.
  * [8] R. Ali, N. Shahin, Y. B. Zikria, B. Kim and S. W. Kim, “Deep Reinforcement Learning Paradigm for Performance Optimization of Channel Observation–Based MAC Protocols in Dense WLANs,” _IEEE Access_ , 2019.
  * [9] N. Abu Zainab, T. Erpek, K. Davaslioglu, Y. E. Sagduyu, Y. Shi, S. Mackey, M. Patel, F. Panettieri, M. Qureshi, V. Isler, A. Yener, “QoS and Jamming-Aware Wireless Networking Using Deep Reinforcement Learning,” _IEEE Military Communications Conference (MILCOM)_ , 2019.
  * [10] Y. Shi, T. Erpek, and Y. E. Sagduyu, “Reinforcement Learning for Dynamic Resource Optimization in 5G Radio Access Network Slicing,” _IEEE International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD)_ , 2020.
  * [11] Y. Shi, Y. E. Sagduyu, T. Erpek, M. C. Gursoy, “How to Attack and Defend NextG Radio Access Network Slicing with Reinforcement Learning,” _arXiv preprint arXiv:2101.05768_ , 2021.
  * [12] F. Wang, M. C. Gursoy, S. Velipasalar, and Y. E. Sagduyu, “Robust Deep Reinforcement Learning Based Network Slicing Under Adversarial Jamming Attacks,” _IEEE Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)_ , 2022.
  * [13] Y. Shi and Y. E. Sagduyu, T. Erpek, and M. C. Gursoy, “Jamming Attacks on NextG Radio Access Network Slicing with Reinforcement Learning,” _IEEE Future Networks World Forum (FNWF)_ , 2022.
  * [14] X. Li, J. Fang, W. Cheng, H. Duan, Z. Chen and H. Li, “Intelligent Power Control for Spectrum Sharing in Cognitive Radios: A Deep Reinforcement Learning Approach,” _IEEE Access_ , 2018.
  * [15] Y. S. Nasir and D. Guo, “Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks,” _IEEE Journal on Selected Areas in Communications_ , Oct. 2019.
  * [16] M. K. Sharma, A. Zappone, M. Assaad, M. Debbah, and S. Vassilaras “Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach,” _IEEE Transactions on Cognitive Communications and Networking_ , Dec. 2019.
  * [17] Z. Lu and M. C. Gursoy, “Dynamic Channel Access and Power Control via Deep Reinforcement Learning,” _IEEE Vehicular Technology Conference_ (VTC2019-Fall), 2019.
  * [18] H. Zhang, N. Yang, W. Huangfu, K. Long and V. C. M. Leung, “Power Control Based on Deep Reinforcement Learning for Spectrum Sharing,” _IEEE Transactions on Wireless Communications_ , June 2020.
  * [19] F. B. Mismar, B. L. Evans and A. Alkhateeb, “Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination,” _IEEE Transactions on Communications_ , Mar. 2020.
  * [20] E. Khorov, I. Levitsky and I. F. Akyildiz, “Current Status and Directions of IEEE 802.11be, the Future Wi-Fi 7,” _IEEE Access_ , 2020.
  * [21] S. Szott, et al., “WiFi Meets ML: A Survey on Improving IEEE 802.11 Performance with Machine Learning,” _arXiv preprint arXiv:2109.04786_ , 2021.
  * [22] X. Liu and N. Ansari, “Toward Green IoT: Energy Solutions and Key Challenges,” _IEEE Communications Magazine_ , Mar. 2019. vol. 57, no. 3, pp. 104–110, Mar. 2019.
  * [23] K. Davaslioglu, S. Soltani, T. Erpek and Y. E. Sagduyu, “DeepWiFi: Cognitive WiFi with Deep Learning,” _IEEE Transactions on Mobile Computing_ , Feb. 2021.
  * [24] J. Tan, L. Zhang, Y. -C. Liang and D. Niyato, “Intelligent Sharing for LTE and WiFi Systems in Unlicensed Bands: A Deep Reinforcement Learning Approach,” _IEEE Transactions on Communications_ , May 2020.
  * [25] S. Soltani, Y. E. Sagduyu, Y. Shi, J. Li, J. Feldman, and J. Matyjas, “Distributed Cognitive Radio Network Architecture, SDR Implementation and Emulation Testbed,” _IEEE Military Communications Conference (MILCOM)_ , 2015.
  * [26] Y. E. Sagduyu, S. Soltani, T. Erpek, Y. Shi and J. Li, “A Unified Solution to Cognitive Radio Programming, Test and Evaluation for Tactical Communications,” _IEEE Communications Magazine_ , Oct. 2017.
  * [27] J. Yackoski, B. Azimi-Sadjadi, A. Namazi, J. H. Li, Y. E. Sagduyu, and R. Levy, “RF-NEST: Radio Frequency Network Emulator Simulator Tool,” _IEEE Military Communications Conference (MILCOM)_ , 2011.
  * [28] The Babel Routing Protocol, https://datatracker.ietf.org/doc/html/rfc8966.
  * [29] V. Mnih _et al._ , “Playing Atari with Deep Reinforcement Learning,” _NeurIPS Deep Learning Workshop_ , 2013.
  * [30] V. Mnih _et al._ , “Human-level Control through Deep Reinforcement Learning,” _Nature_ , 2015.
  * [31] The Weaver Computer Engineering Research Group, “The GFLOPS/W of the various machines in the VMW Research Group,” web.eece.maine.edu/~vweaver/group/green_machines.html. (accessed June 26, 2022).


# Acousto-optic reconstruction of exterior sound field based on concentric
circle sampling with circular harmonic expansion

Phuc Duc Nguyen , Kenji Ishikawa , Noboru Harada , and Takehiro Moriya
Manuscript received xx, 2023; revised xx, 2023. (_Corresponding author: Kenji
Ishikawa, ke.ishikawa@ntt.com_)P. D. Nguyen, K. Ishikawa, N. Harada, T. Moriya
are with NTT Communication Science Laboratories, NTT Corporation. Atsugi R&D
Center. 3-1, Morinosato-Wakamiya, Atsugi, Kanagawa, 243-0198, Japan.
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>0000-0002-7136-8924 0000-0003-0423-566X
0000-0002-1759-4533 0000-0003-4591-1273

###### Abstract

Acousto-optic sensing provides an alternative approach to traditional
microphone arrays by shedding light on the interaction of light with an
acoustic field. Sound field reconstruction is a fascinating and advanced
technique used in acousto-optics sensing. Current challenges in sound-field
reconstruction methods pertain to scenarios in which the sound source is
located within the reconstruction area, known as the exterior problem.
Existing reconstruction algorithms, primarily designed for interior scenarios,
often exhibit suboptimal performance when applied to exterior cases. This
paper introduces a novel technique for exterior sound-field reconstruction.
The proposed method leverages concentric circle sampling and a two-dimensional
exterior sound-field reconstruction approach based on circular harmonic
extensions. To evaluate the efficacy of this approach, both numerical
simulations and practical experiments are conducted. The results highlight the
superior accuracy of the proposed method when compared to conventional
reconstruction methods, all while utilizing a minimal amount of measured
projection data.

###### Index Terms:

Acousto-optic sensing, sound field reconstruction, tomographic, wave
expansion.

##  I Introduction

Acousto-optics sensing methodologies utilize light as the principal sensing
modality to non-intrusively sample acoustic fields within a given volume
through a sparse array of optical measurements performed remotely. In contrast
to conventional electromechanical transducers, acousto-optic methods do not
yield localized acoustic pressure measurements due to the acousto-optic
interaction taking place throughout the entire optical path [1]. Among
numerous acousto-optic sensing methods developed so far [2], precision optical
interferometry has played a central role in measuring audible sound, such as
laser Doppler vibrometry [3, 4, 1], parallel phase-shifting interferometry and
digital holography [5, 6, 7, 8, 9, 10, 11], and midfringe locked
interferometry [12].

Computed tomography, frequently observed in medical imaging or nondestructive
testing domains, enables the reconstruction of the spatial distribution of
physical property from its projection measurements [13]. In acousto-optic
sensing, the tomographic reconstruction of the sound field is essential to
obtain sound pressure distributions from the measured laser projection data.

The most common tomographic reconstruction technique is filtered back-
projection (FBP) [14]. Although this technique is simple and frequently
employed [15, 1, 16, 17], the reconstruction accuracy may not be optimal for
sound fields. Recently, physical model-based sound-field reconstruction
methods have demonstrated their superiority over conventional FBP methods [18,
19, 20, 21] because they ensure that the reconstructed field satisfies the
physical properties of sound. A typical reconstruction algorithm in the
physical model-based group is the plane-wave expansion (PWE) [19, 21].

A challenge in the existing sound-field reconstruction methods is the so-
called exterior problem where the sound source is located inside the
reconstruction area. Fig. 1 illustrates the two sound-field reconstruction
problems, the interior problem reconstructs the sound field inside a closed
space (represented by a dashed line) where a sound source is located outside
the space, whereas the exterior problem reconstructs the sound field outside a
closed space where a sound source is located inside the space. Since the
existing reconstruction algorithms are optimized for the interior
reconstruction problem, they exhibit suboptimal performance for the exterior
situation. Nevertheless, the reconstruction of the external sound field is a
critical problem set that includes essential applications such as directivity
measurement of sound sources and estimating the radiated sound field to an
infinite region.

Figure 1: Illustration of (a) interior and (b) exterior sound-field
reconstruction problems.

In this study, we present a novel exterior reconstruction method founded upon
a concentric circle sampling scheme, utilizing circular harmonics as the
expansion coefficients. To the best of our knowledge, this research represents
the pioneering effort to facilitate the acousto-optic reconstruction of
exterior sound fields, even in cases where the sound source is situated within
the confines of the reconstruction area.

The rest of this paper is structured as follows: Section II introduces the
proposed method comprising of concentric circle sampling technique and
circular harmonic expansion of the sound field. Section III presents the
simulation setups and the corresponding simulated results. This is followed by
the exposition of the experiment setups and the actual experimental results.
Section IV then meticulously presents and analyzes the experimental findings.
Section V engages in relevant discussions to shed light on the implications of
the results. Finally, Section V encapsulates the research with a conclusive
summary of key findings and their broader significance.

##  II Proposed method

###  II-A Principle of acousto-optic sensing

In Fig. 2, an exemplification of the Laser Doppler Vibrometer (LDV) is
presented, which serves to gauge acoustically induced phase shifts of light.
The functioning of this apparatus involves the generation of a laser beam,
subsequently bifurcated into two distinct branches: the sensing branch and the
reference branch. The sensing beam is then directed through the sound field
$p(r)$, wherein it undergoes backscattering upon interacting with the mirror
surface and is eventually retrieved at the LDV optics system. The measurement
principle can be found in the literature [1]. In short, the sensing beam
experiences the phase modulation caused by sound due to the acousto-optic
effect, which is detected by the LDV. A notable characteristic of this
measurement method is that the detected signal is proportional to the line
integral of sound pressure along the sensing beam.

###  II-B Concentric circular sampling

For exterior reconstruction problems, since a sound source should be located
in the center part of the reconstruction area (as depicted in Fig. 1), sensing
laser paths must avoid this part. Considering practical experimental
implementation, we propose the concentric circle sampling scheme as shown in
Fig. 3. The projections $S$ are sampled by laser beams surrounding the circle
with radius $R$ and rotation $\phi$. This sampling scheme is easily achieved
by rotating the sound source while keeping the laser beam fixed. In order to
enhance the fidelity of reconstructed sound fields, it is possible to augment
the acquisition of projections by incorporating numerous circles with varying
radii, such as $R_{1}$, $R_{2}$, and $R_{3}$ as depicted in Fig. 3, or by
employing smaller angular rotation increments. In this context, Fig. 3
illustrates a demonstration with a representation of 72 projections derived
from a single concentric circle when the angular interval is 5∘. This choice
of the number of projections was made to strike a judicious balance between
the quality of reconstruction and the computational complexity inherent in the
reconstruction process. Another important factor to consider is the optimal
number of concentric circles to use in sound field reconstruction. In an
upcoming section, we will share initial findings and discuss this issue.

Figure 2: Acousto-optic sensing utilizing a Laser Doppler Vibrometer (LDV)
device.

Figure 3: The proposed concentric circle sampling samples a sound field along
the tangent line of a circle of radius $R_{i}$ at given angular intervals. The
lines in the images represent measurement laser trajectories when the angular
interval is 5∘.

###  II-C Circular harmonic expansion of sound field

In this section, we expound upon the sound field reproduction technique
employing circular harmonics, with a focus on the global exterior sound field
as the area of interest. The acoustic field generated by an arbitrary sound
source within a two-dimensional exterior space can be accurately depicted
through the application of circular harmonic expansion (CHE), as exemplified
in the work of E. G. Williams [22]. This representation can be expressed as
follows:

$p(r,\phi,\omega)=\sum\limits_{n=-\infty}^{\infty}{{a_{n}(\omega)}}{H_{n}^{(2)}}(kr){e^{jn\phi}},$
(1)

where ${a_{n}(\omega)}$ represent the sound field coefficient for the order
$n$ and ${H_{n}^{(2)}}\left(\cdot\right)$ is the Hankel function of the second
kind. The symbols $\omega$ and $k$ correspond to the angular frequency and
wave number, respectively, with the relationship between them given by
$k=\omega/c$, where $c$ denotes the velocity of propagation.

Figure 4: The polar coordinate representation of $e^{jn\phi}$, which serves as
the foundational basis for circular harmonic expansion, is exemplified for $N$
values of 0, 1, 2, and 3.

Figure 4 shows the polar coordinate representation of $e^{jn\phi}$. Within our
reconstruction method, the selection of the parameter $n$ was made in a
flexible manner, contingent upon the specific frequency of the sound signal
and the minimal radius of the concentric circle. This choice was crucial to
ensure the comprehensive representation of the reconstructed sound field.

###  II-D Reconstruction algorithm

According to acousto-optic theorem and the circular harmonic expansion of the
sound field, the projection $s_{m}$ can be written as

${s_{m}}=\int{p(r,\phi,\omega)d{l_{m}}\approx\sum\limits_{n=-N}^{N}{{a_{n}}\int{{H_{n}^{(2)}}(kr){e^{jn\phi}}d{l_{m}}}}},$
(2)

where $l_{m}$ represents the sensing laser trajectory of the $m$th measurement
and the infinite sum of circular harmonics is truncated by order $N$. Equation
(2) can be expressed algebraically as follows.

$\mathbf{s}\approx\mathbf{H}\mathbf{a},$ (3)

where $\mathbf{H}\in{\mathbb{C}^{M\times(2N+1)}}$ is a matrix with elements
${h_{m,n}}=\int{{H_{n}^{(2)}}(kr){e^{jn\phi}}d{l_{m}}}$.
$\mathbf{a}\in{\mathbb{C}^{N}}$ are the expansion coefficients of the $N$
circular harmonic waves in the expansion. These coefficients can be determined
through the regularized least squares method.

$\mathbf{\tilde{a}}=\arg\mathop{\min}\limits_{\mathbf{a}\in{\mathbb{C}^{N}}}{\left\|\mathbf{a}\right\|_{2}}$
(4)

subject to
${\left\|{\mathbf{s}-\mathbf{H}\mathbf{a}}\right\|_{2}}\leq\varepsilon$, where
$\varepsilon$ is the permitted discrepancy between measurement $\mathbf{s}$
and $\mathbf{H}\mathbf{a}$. After obtaining the expansion coefficients
$\mathbf{\tilde{a}}$, the reconstructed sound field is calculated as

$\mathbf{\tilde{p}}=\mathbf{G}\mathbf{\tilde{a}}$ (5)

where $\mathbf{\tilde{p}}\in{\mathbb{C}^{L}}$ denote estimated pressure at
positions $r_{1},...,r_{L}$, and $\mathbf{G}\in{\mathbb{C}^{L\times(2N+1)}}$
includes elements
${g_{l,n}}=a_{n}{H_{n}^{(2)}}(kr_{l}){e^{jn\phi_{l}\cdot{r_{l}}}}$.

##  III Simulation

###  III-A Reference multi-source complex sound field

The 2D sound field generated by a point source is given as follows

$p(x,y,k)=A\frac{i}{4}H_{0}^{(1)}(k\left|{r-{r_{0}}}\right|),$ (6)

where $i=\sqrt{-1}$, $A$ is the amplitude of the sound field,
$\left|{r-{r_{0}}}\right|=\sqrt{{{(x-{x_{0}})}^{2}}+{{(y-{y_{0}})}^{2}}}$ is
the distance between $r$ and $r_{0}$. $r_{0}$ and $r$ are positions of a point
source and position to calculate sound pressure, respectively.
$k=\frac{{2\pi}}{\lambda}$ is the wavenumber of sound. $\lambda$ is the
wavelength of sound, and $H_{0}^{(1)}(.)$ is the Hankel function of the first
kind of order 0 [23].

As reference sound fields for simulating the exterior reconstruction problem,
we generated sound fields within the area of 0.7 m $\times$ 0.7 m by the
superposition of multiple point sources located within a circular area of
minimum radius $R_{\mathrm{min}}=0.3\,$m. The generated reference sound fields
are shown in the tops of Figs. 5 and 6, which are created from five sound
sources at different phases and positions. The five sound sources have the
phases of $\phi=\\{\pi/6,2\pi/6,3\pi/6,4\pi/6,5\pi/6\\}$ and located at
$\\{[0,0],[-0.05,0],[0,-0.05],[0.05,0],[0,0.05]\\}$, respectively.

Figure 5: A comparative analysis was conducted on sound fields reconstructed
by distinct reconstruction algorithms, namely FBP, PWE, and the proposed CHE.
The reference sound fields were constructed using multiple sound point sources
distributed in the proximity of a central location and across a range of
frequencies. Figure 6: A comparative evaluation was conducted to assess the
sound field reconstruction quality of various reconstruction algorithms,
specifically FBP, PWE, and the proposed CHE. The reference sound fields were
meticulously composed, featuring multiple sound point sources positioned at
locations distanced from the central point. The specified locations for the
groups of sound sources were as follows: at 1kHz: [0,0.12], 2 kHz: [0.2,0], 4
kHz: [-0.2,0], 8 kHz: [-0.2,0.1], and 16 kHz: [0.2,-0.1]. These sound fields
were generated at distinct frequencies for the express purpose of evaluation.

###  III-B Simulated sound-field reconstruction

In order to assess the quality of the reconstructions, we used reconstruction
error [24] which is calculated using
$\varepsilon(r)=10{\log_{10}}\left\\{{{{{{{{\left|{\tilde{p}(r)-p(r)}\right|}^{2}}}\mathord{\left/{\vphantom{{{{\left|{\tilde{p}(r)-p(r)}\right|}^{2}}}{\left|{p(r)}\right|}}}\right.\kern-1.2pt}{\left|{p(r)}\right|}}}^{2}}}\right\\}$,
where $\tilde{p}(r)$ and $p(r)$ are the reconstructed and reference sound
pressure at position $r$, respectively. We computed the reconstruction error
on a pixel-wise basis and juxtaposed the resulting error images alongside each
corresponding reconstructed sound field solely for evaluative purposes. In
order to enable a rigorous comparative analysis, we conducted simulations
involving sound fields generated at different frequencies. For each frequency,
we assessed two distinct scenarios wherein sound sources were positioned both
at the center and off-center locations. To ensure equitable comparisons, we
maintained a uniform number of projections (M = 2232) across three
reconstruction algorithms: FBP, PWE, and the proposed CHE-based approach. In
the case of the PWE method, our simulations entailed the use of 200 plane
waves.

Figure 5 depicts the reconstructed sound fields utilizing three distinct
methods: FBP, PWE, and CHE. These reconstructions were performed with the
sound sources located at the central coordinates [0, 0]. We employed identical
quantities of circular harmonics $N=15$ for sound fields at various
frequencies: 1 kHz, 2 kHz, 4 kHz, 8 kHz, and 16 kHz. Examination of the
reconstructed sound fields and accompanying reconstruction error images
highlights certain noteworthy findings. Specifically, the PWE method exhibited
difficulties in achieving accurate sound field reconstruction, while the FBP
approach displayed suboptimal reconstruction quality. In contrast, the
proposed CHE approach consistently yielded the highest quality reconstructions
among the three methods under consideration.

Figure 6 illustrates the reconstruction outcomes in scenarios where sound
sources are positioned at varying distances from the central location. In this
particular scenario, the sound fields exhibit a greater degree of complexity,
necessitating a larger number of circular harmonics compared to the scenario
wherein the sound sources are situated at the center position. More
specifically, we employed $N\,$=$\,20$ for sound fields at 1 kHz, 2 kHz, and 4
kHz, while for sound fields at 8 kHz and 16 kHz, we utilized $N=30$ and $N=40$
for the reconstruction process, respectively. The outcomes reveal that FBP and
PWE exhibit suboptimal reconstruction performance within the frequency bands
of 1 kHz, 2 kHz, and 4 kHz, characterized by either amplitude distortion or
misalignment in the reproduced sound source positions when compared to the
reference sound field. At higher frequency ranges, both FBP and PWE fail to
effectively reconstruct the sound fields. Conversely, across all frequency
ranges, the CHE-based method consistently delivers the highest reconstruction
quality.

Figure 7: An evaluation is conducted to assess the quality of 2 kHz sound-
field reconstruction achieved by three distinct reconstruction methods
(namely, FBP, PWE, and proposed CHE) under the influence of AWGN at varying
levels of signal-to-noise ratio (SNR).

To investigate the reconstruction performances against noise, different
amounts of white Gaussian noises are added to the reference sound field of 2
kHz shown in Fig. 5, and the reconstruction errors among the three methods are
compared. For comparing the overall errors across the entire reconstruction
area, the following normalized mean square error (NMSE) in decibels (dB) is
used;
$\mathrm{NMS{E_{dB}}}=20{\log_{10}}({{{{\left\|{\tilde{p}-p}\right\|}_{2}}}\mathord{\left/{\vphantom{{{{\left\|{\tilde{p}-p}\right\|}_{2}}}{{{\left\|p\right\|}_{2}}}}}\right.\kern-1.2pt}{{{\left\|p\right\|}_{2}}}})$.
Fig. 7 illustrates the comparison of NMSEs for the three methods. The findings
illustrate the persistent supremacy of the proposed CHE algorithm in
comparison to the FBP and PWE techniques across various noise levels.
Significantly, the sound fields reconstructed by the FBP and PWE methods reach
saturation points of NMSE at -1.3 dB, and -3.8 dB, respectively, when the SNR
exceeds or equals 14. In contrast, the sound fields reconstructed by the
proposed CHE algorithm consistently exhibit ongoing quality improvement as SNR
increases, approaching sound field quality levels obtained in a noise-free
environment, with an NMSE of -28.1 dB.

##  IV Experimental results

###  IV-A Experiment setup

TABLE I: Experiment settings

| Parameter | Value
---|---|---
Microphone | Type | Brüel Kjær 4939-A-011
Microphone amplifier | Type | BK NEXUS
| Signal delay |  $\approx 10\,\mu$s
Loudspeaker | Type | Yamaha MS101III
Stage controller | Type | OptoSigma SHOT-302GS
| Movement step | 10 mm
| Rotation step | 5∘
LDV | Type | Polytec VibroFlex
| Decoder | Displacement
| Bandwidth | 20 kHz
| Sensitivity | 5 nm/V @M$\Omega$
| Range | 10 nm
| Signal delay | 900$\,\mu$s
Anechoic chamber | Temperature | 16.9∘C
| Humidity | 28.4 %
| Pressure | 1000.4 hPa

Figure LABEL:fig:8a illustrates our experimental setup, while Fig.
LABEL:fig:8b provides a view of the experimental setup within the anechoic
chamber. The sound field was generated by a loudspeaker positioned laterally
in relation to the LDV. Instead of adjusting the position and angle of the
laser path, we rotated and moved the loudspeaker while keeping the laser beam
fixed in place to achieve the concentric circle sampling. A mirror was
positioned at a distance of 289 cm from the laser window of the LDV. We
designated the distance from the center of the loudspeaker to the laser path
as $R$. To adjust this distance, we utilized a motor under the control of a
LabVIEW program to manage both the rotation of the loudspeaker and its one-
dimensional movement. Specifically, the distance $R$ was incremented by 10 mm
for each step, ranging from $R_{\mathrm{min}}\,$=$\,0.3\,$m to
$R_{\mathrm{max}}\,$=$\,0.6\,$m. The rotation step $\delta_{\theta}$ was fixed
at 5 degrees, and a total of 72 projections were obtained for each circle
sampling, denoted as $M\,$=$\,72$. The comprehensive information regarding the
equipment and experimental parameters are listed in Table I.

###  IV-B Harmonic sould-field reconstruction

Figure 9: Experimental sound field reconstruction results from LDV
projections. Sound fields are created at different frequencies from 1 kHz to
16 kHz The reference sound pressure is recorded by the microphone placed
nearest the laser path.

To validate the effectiveness of the proposed method for experimental data, we
generated harmonic sound fields at frequencies 1 kHz, 2 kHz, 4 kHz, 8 kHz, and
16 kHz and compared the reconstructed sound fields by CHE with the fields
measured by a microphone. The results shown in Fig. 9 indicate that the
proposed CHE algorithm effectively reconstructs the sound fields from LDV
projections. Notably, it should be emphasized that our LDV device introduces a
signal latency of 900$\,\mu$s, which necessitates compensation when comparing
the reconstructed sound fields with the reference sound fields recorded by the
microphone. The precision of sound field reconstruction is demonstrated by
visualizing images depicting the difference between microphone and LDV
reconstructions. These images are presented in the third row of Fig. 9. We
estimated the difference using the same method as estimating reconstruction
errors, as discussed in Section III-B. The experimental results indicate
variations in the discrepancy between the reconstructed LDV sound field and
the microphone signal across different frequencies. At 1 kHz, a relatively
substantial difference is observed, whereas, at 16 kHz, the difference is
reduced, though still present. In contrast, at 2 kHz, 4 kHz, and 8 kHz, the
difference falls within the range of [-20,-10], underscoring the acceptable
quality of the reconstructed sound fields. These findings reveal the
persistence of minor reconstruction errors, which can be attributed to
uncertainties in phase delays and amplitude losses in the signals recorded by
both the microphone and LDV devices.

Figure 10: Time-domain sound field reconstruction.

###  IV-C Time-domain sound-field reconstruction

In order to assess the performance of time-domain sound field reconstruction,
a 2 kHz sinusoidal burst wave with a duration of 10 ms was emitted from the
loudspeaker. The experimental configuration utilized for this evaluation is
consistent with the parameters defined in Section IV-A. To reconstruct the
sound field in the time domain from experimentally acquired data, the
procedure comprises the following steps:

  * •

Step 1: Perform a Fast Fourier Transform (FFT) on all time-domain LDV
projection data, to convert projections from the time domain to the frequency
domain.

  * •

Step 2: Apply the proposed CHE algorithm to reconstruct the sound field in the
frequency domain from the FFT output data obtained in Step 1, independently
for each frequency.

  * •

Step 3: Combine the 2-dimensional (2D) reconstructed sound fields at all
frequencies into a 3-dimensional (3D) array. The third dimension represents
frequencies ranging from 0 Hz to the maximum frequency, with a length
identical to the FFT length in Step 1.

  * •

Step 4: Perform an inverse FFT on the 3D array of reconstructed sound fields
in the frequency domain to yield the results of the time-domain sound field
reconstruction.

Fig. 10 illustrates multiple frames extracted from time-domain sound fields
reconstructed using LDV projections at various time points: 1.1 ms, 1.36 ms,
1.5 ms, 2.28 ms, and 2.6 ms. Concurrently, time-domain microphone signals were
acquired synchronously with the LDV device for reference. To enable a
meaningful comparison with the microphone signals, a delay of 900$\,\mu$s in
the LDV signal was corrected. Additionally, we empirically determined the
response delay of the microphone system as approximately 10$\,\mu$s through
experimental measurements. Upon scrutinizing the data, it is evident that
minor temporal disparities of approximately 90$\,\mu$s persist between the
sound fields captured by the microphone and those recorded by the LDV device.
These slight variations are attributed to undetermined signal delays within
the LDV or microphone circuitry. Nevertheless, it is noteworthy that the sound
fields reconstructed using the CHE algorithm from LDV projections exhibit a
remarkable similarity to the corresponding microphone signals.

##  V Discussions

In order to expedite the process of sound field reconstruction, two key
aspects are considered: reducing the number of required projections $M$ and
determining an appropriate number of circular harmonics $N$.

###  V-A Influence of expansion order

The appropriate selection of the value $N$ is visually depicted in Fig.
LABEL:fig:11a. This figure illustrates the number of circular harmonic waves
required to fully represent the amplitude of the expansion coefficients,
denoted as $\left|\mathbf{a}\right|$. When we examine our experimental data,
we determine that the suitable $N$ values are 3, 5, 10, 20, and 30 for sound
fields at 1 kHz, 2 kHz, 4 kHz, 8 kHz, and 16 kHz, respectively. This selection
is based on the fact that the amplitude of expansion coefficients converges to
zero when the circular harmonics index falls within the range of [-N: N]. In
contrast, computational simulations suggest that the appropriate $N$ values
are 7, 10, and 20 for sound fields at 1 kHz, 2 kHz, and 4 kHz, respectively.
However, for sound fields at 8 kHz and 16 kHz, the amplitudes of the CHE
coefficients do not converge to zero even when $N$ equals 30 (for 8 kHz) or 40
(for 16 kHz). It is worth noting that higher frequencies necessitate larger
values for $N$ which in turn require more computational time for the
reconstruction process. Therefore, in our simulations, we opted for $N$ values
of 30 for 8 kHz, and 40 for 16 kHz, as these values adequately represent the
majority of the CHE coefficient amplitudes.

Furthermore, Fig. LABEL:fig:11b elucidates the experimental sound-field
reconstruction scenarios in instances where varying numbers of circular
harmonic waves are employed, leading to cases of good fit, underfitting, and
overfitting. A good fit is achieved when the minimum number of circular
harmonics is adequate to fully represent the amplitude of the expansion
coefficient. Underfitting occurs when there is a loss of amplitude due to the
utilization of an insufficient number of circular harmonics. Conversely,
overfitting occurs when an excessive number of circular harmonics are utilized
to represent the expansion coefficient, resulting in the emergence of complex
patterns extending beyond the boundaries of the sound source area. This can be
observed in the overfitting cases with N = 20, and N = 25. At a frequency of 2
kHz, a good fit is achieved when the parameter falls within the range of
$5\leq N\leq 15$. Underfitting occurs when $N\,<\,5$, and overfitting arises
when $N>15$. To determine an appropriate value for a good model fit at
frequencies other than 2 kHz, a reference can be made to Fig. LABEL:fig:11a.

###  V-B Influence of number of projections

In order to determine the optimal number of projections necessary for sound
field reconstruction, we conducted an assessment of the reconstructed sound
field quality employing NMSE analysis. A 2 kHz sound signal was emitted from a
loudspeaker placed in an anechoic chamber, with the sound source positioned at
an off-center location [0.12, 0]. During this analysis, we explored a range of
projection quantities, ranging from 72 projections originating from the
innermost concentric circle with a radius of $0.3\,$m to 2232 projections
obtained from 31 concentric circles, with radii varying from $0.3\,$m to
$0.6\,$m. Specifically, this encompassed 2 concentric circles with a 15 cm
increment for each step, resulting in 144 projections; 4 concentric circles
with an 8 cm increment for each step, yielding 288 projections; 8 concentric
circles with a 4 cm increment for each step, giving rise to 576 projections;
16 concentric circles with a 2 cm increment for each step, resulting in 1152
projections; and finally, 31 concentric circles with a 1 cm increment for each
step, producing a total of 2232 projections. Fig. 12a shows the NMSE values
for different numbers of the concentric circles. The highest quality is for
the 31 concentric circles with NMSE of -11.458 dB. Additionally, Fig. 12a
illustrates that the NMSE nearly reaches convergence when the number of
concentric circles is set to 16.

To minimize the number of projections employed in the reconstruction process,
it is essential to identify concentric circles that yield optimal
reconstruction performance among all available concentric circles. Fig. 12b
demonstrates that the choice of concentric circle radius has a significant
impact on reconstruction quality. Notably, utilizing 72 projections from a
circle with a radius of $R\,$=$\,0.38\,$m yields superior results, with the
NMSE of -9.39 dB, surpassing other radii. Fig. 12c demonstrates the
reconstruction quality when considering only 144 projections from two
concentric circles while the one is fixed at $R\,$=$\,0.38\,$m. It is evident
that the two concentric circles with radii of $R_{1}\,$=$\,0.38\,$m and
$R_{2}\,$=$\,0.6\,$m outperform the other pairs, achieving almost the same
reconstruction quality as when using all 31 circles. This outcome demonstrates
that by employing a combination of projections from specific concentric
circles that excel in precise sound field reconstruction, it is possible to
reduce the overall number of required projections. The determination of the
optimal concentric circle radius and the appropriate quantity of concentric
circles is beyond the scope of this paper. However, we envision that our
discoveries may provide a foundational basis for future research endeavors
aimed at investigating these intriguing inquiries.

Figure 12: The assessment of the NMSE in decibels (dB) for 2 kHz reconstructed
sound fields involves two scenarios: (a) the NMSE is evaluated by varying the
number of projections acquired from concentric circles with radii ranging from
$R_{\mathrm{min}}\,$=$\,0.3\,$m to $R_{\mathrm{max}}\,$=$\,0.6\,$m; (b) the
NMSE is assessed using 72 projections from a single concentric circle; (c)
NMSE evaluation when considering only 144 projections from two concentric
circles $R=0.38\,$m and $R_{2}$.

##  VI Conclusion

This paper introduces an innovative approach involving concentric circle
sampling and a two-dimensional exterior sound-field reconstruction technique
based on the extension of circular harmonics. The proposed reconstruction
method effectively reconstructs the sound field, particularly when the sound
source is located within the reconstruction area. To evaluate the
effectiveness of the proposed approach, comprehensive assessments are
conducted through numerical simulations and real-world experiments. The
outcomes of these simulations and experiments compellingly establish that the
proposed technique outperforms conventional reconstruction methods, exhibiting
superior accuracy, and concurrently minimizing the number of required measured
projections for the reconstruction process. This study introduces significant
advancements in acoustic data measurement and analysis, allowing for
comprehensive spatiotemporal sound field capture across extensive spatial
regions. Moreover, this methodology holds potential value in diverse
disciplines that focus on wave field measurement and visualization.

## Acknowledgment

The authors thank Dr. Samuel A. Verburg for his helpful comments in early
discussions on the plane wave expansion reconstruction method.

## References

  * [1] A. Torras-Rosell, S. Barrera-Figueroa, and F. Jacobsen, “Sound field reconstruction using acousto-optic tomography,” _The Journal of the Acoustical Society of America_ , vol. 131, no. 5, pp. 3786–3793, 2012.
  * [2] S. A. Verburg, K. Ishikawa, E. Fernandez-Grande, and Y. Oikawa, “A century of acousto-optics- from early discoveries to modern sensing of sound with light,” _Acoust. Today_ , vol. 19, pp. 54–62, 2023.
  * [3] K. Nakamura, M. Hirayama, and S. Ueha, “Measurements of air-borne ultrasound by detecting the modulation in optical refractive index of air,” in _Proc. IEEE Ultrasonics Symposium_ , 2002, pp. 609–612.
  * [4] A. Harland, J. Petzing, and J. Tyrer, “Non-invasive measurements of underwater pressure fields using laser doppler velocimetry,” _J. Sound Vib._ , vol. 252, pp. 169–177, 4 2002.
  * [5] K. Ishikawa, K. Yatabe, N. Chitanont, Y. Ikeda, Y. Oikawa, T. Onuma, H. Niwa, and M. Yoshii, “High-speed imaging of sound using parallel phase-shifting interferometry,” _Opt. Express_ , vol. 24, p. 12922, 6 2016.
  * [6] O. Matoba, H. Inokuchi, K. Nitta, and Y. Awatsuji, “Optical voice recorder by off-axis digital holography,” _Opt. Lett._ , vol. 39, p. 6549, 11 2014.
  * [7] K. Ishikawa, R. Tanigawa, K. Yatabe, Y. Oikawa, T. Onuma, and H. Niwa, “Simultaneous imaging of flow and sound using high-speed parallel phase-shifting interferometry,” _Opt. Lett._ , vol. 43, p. 991, 3 2018.
  * [8] S. K. Rajput, O. Matoba, and Y. Awatsuji, “Holographic multi-parameter imaging of dynamic phenomena with visual and audio features,” _Opt. Lett._ , vol. 44, p. 995, 2 2019.
  * [9] Y. Takase, K. Shimizu, S. Mochida, T. Inoue, K. Nishio, S. K. Rajput, O. Matoba, P. Xia, and Y. Awatsuji, “High-speed imaging of the sound field by parallel phase-shifting digital holography,” _Appl. Opt._ , vol. 60, p. A179, 2 2021.
  * [10] S. K. Rajput, O. Matoba, M. Kumar, X. Quan, and Y. Awatsuji, “Sound wave detection by common-path digital holography,” _Opt. Lasers Eng._ , vol. 137, p. 106331, 2 2021.
  * [11] Z. Zhong, C. Wang, L. Liu, Y. Liu, L. Yu, B. Liu, and M. Shan, “Visual measurement of instable sound field using common-path off-axis digital holography,” _Opt. Lasers Eng._ , vol. 158, p. 107129, 11 2022.
  * [12] K. Ishikawa, Y. Shiraki, T. Moriya, A. Ishizawa, K. Hitachi, and K. Oguri, “Low-noise optical measurement of sound using midfringe locked interferometer with differential detection,” _J. Acoust. Soc. Am._ , vol. 150, pp. 1514–1523, 8 2021.
  * [13] B. P. Flannery, H. W. Deckman, W. G. Roberge, and K. L. D’Amico, “Three-dimensional x-ray microtomography,” _Science_ , vol. 237, no. 4821, pp. 1439–1444, 1987.
  * [14] A. C. Kak and M. Slaney, _Principles of computerized tomographic imaging_. SIAM, 2001.
  * [15] Y. Oikawa, Y. Ikeda, M. Goto, T. Takizawa, and Y. Yamasaki, “Sound field measurements based on reconstruction from laser projections,” in _Proc. (ICASSP ’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005._ , vol. 4. IEEE, pp. 661–664.
  * [16] E. Koponen, J. Leskinen, T. Tarvainen, and A. Pulkkinen, “Acoustic pressure field estimation methods for synthetic schlieren tomography,” _The Journal of the Acoustical Society of America_ , vol. 145, no. 4, pp. 2470–2479, 2019.
  * [17] D. Hermawanto, K. Ishikawa, K. Yatabe, and Y. Oikawa, “Determination of frequency response of mems microphone from sound field measurements using optical phase-shifting interferometry method,” _Applied Acoustics_ , vol. 170, p. 107523, 2020.
  * [18] K. Yatabe, K. Ishikawa, and Y. Oikawa, “Acousto-optic back-projection: Physical-model-based sound field reconstruction from optical projections,” _Journal of Sound and Vibration_ , vol. 394, pp. 171–184, 2017.
  * [19] S. A. Verburg and E. Fernandez-Grande, “Acousto-optical volumetric sensing of acoustic fields,” _Physical Review Applied_ , vol. 16, no. 4, p. 044033, 2021\.
  * [20] K. Ishikawa, K. Yatabe, and Y. Oikawa, “Physical-model-based reconstruction of axisymmetric three-dimensional sound field from optical interferometric measurement,” _Measurement Science and Technology_ , vol. 32, no. 4, p. 045202, 2021.
  * [21] S. A. Verburg, E. G. Williams, and E. Fernandez-Grande, “Acousto-optic holography,” _J. Acoust. Soc. Am._ , vol. 152, pp. 3790–3799, 12 2022.
  * [22] E. G. Williams and J. A. Mann III, “Fourier acoustics: sound radiation and nearfield acoustical holography,” 2000.
  * [23] Y. Ren and Y. Haneda, “2D local exterior sound field reproduction using an addition theorem based on circular harmonic expansion,” in _2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_. IEEE, 2021, pp. 271–275.
  * [24] Y. Ren and Y. Haneda, “Two-dimensional exterior sound field reproduction using two rigid circular loudspeaker arrays,” _The Journal of the Acoustical Society of America_ , vol. 148, no. 4, pp. 2236–2247, 2020.