# LinEasyBO: Scalable Bayesian Optimization Approach for Analog Circuit
Synthesis via One-Dimensional Subspaces
††thanks: *Corresponding authors: {shuhanzhang16, yangfan<EMAIL_ADDRESS>
Shuhan Zhang1∗, Fan Yang1∗, Changhao Yan1, Dian Zhou2, Xuan Zeng1∗
1State Key Lab of ASIC & System, Microeletronics Department, Fudan University,
China
2University of Texas at Dallas, Dallas, USA

###### Abstract

A large body of literature has proved that the Bayesian optimization framework
is especially efficient and effective in analog circuit synthesis. However,
most of the previous research works only focus on designing informative
surrogate models or efficient acquisition functions. Even if searching for the
global optimum over the acquisition function surface is itself a difficult
task, it has been largely ignored. In this paper, we propose a fast and robust
Bayesian optimization approach via one-dimensional subspaces for analog
circuit synthesis. By solely focusing on optimizing one-dimension subspaces at
each iteration, we greatly reduce the computational overhead of the Bayesian
optimization framework while safely maximizing the acquisition function. By
combining the benefits of different dimension selection strategies, we
adaptively balancing between searching globally and locally. By leveraging the
batch Bayesian optimization framework, we further accelerate the optimization
procedure by making full use of the hardware resources. Experimental results
quantitatively show that our proposed algorithm can accelerate the
optimization procedure by up to $9\times$ and $38\times$ compared to LP-EI and
REMBOpBO respectively when the batch size is 15.

##  I Introduction

As the development of the integrated circuit (IC) technology, the circuit
scale increases dramatically and the unwanted parasitic effect complicates the
circuit design process significantly. The increasing demands on speed,
reliability, low power consumption and short time-to-market further aggravate
the difficulty of designing circuit manually, especially for the analog
circuit [1]. Sophisticated analog circuit design automation tools are in great
need. The analog circuit design procedure generally can be divided into two
stages: topology selection and device sizing. In this paper, we mainly focus
on the device sizing problem.

The analog circuit device sizing problem typically can be formulated as a
noisy, multimodal, expensive and derivative-free black-box function. A large
body of literature has been published on designing an efficient optimization
algorithm to solve this problem [2, 3, 4, 1, 5, 6, 7]. The traditional
optimization algorithms for analog circuit device sizing problem can be
divided into two groups: the model-based and simulation-based approaches. The
model-based approaches try to approximate the circuit performance and provide
a cheap-to-evaluate model for space exploration. Instead of searching for the
global optimum with expensive circuit simulator, the constructed model can
greatly reduce time consumption on observation, thus, accelerate the
optimization procedure. One popular model-based approach is the geometric
programming [8, 9, 10]. However, there is no theoretical guarantee that the
constructed model is accurate over the whole design space. Therefore, the
obtained global optimum may deviate from the real global optimum.

Instead of constructing a surrogate model to capture the behavior pattern of
the circuit performance, the simulation-based approaches simply search the
state space by mimicking the physical or biological phenomenon. By treating
the circuit performance as a black-box function, the simulation-based
approaches guide the search with a selection engine and invoke the circuit
simulator on the fly. There are several popular simulation-based approaches,
including simulated annealing (SA) [11, 12], multiple starting point (MSP)
algorithm [6, 13], evolutionary algorithm [4, 14] and particle swarm
optimization (PSO) algorithm [15, 16]. The limitation of the simulation-based
approaches is their relatively low convergence rate.

Inspired from above, the Bayesian optimization framework has been proposed to
pave a way out of the dilemma by combining the benefits of both model-based
and simulation-based methods [17, 18]. Generally, there are two key elements
in the Bayesian optimization framework: the statistical surrogate model and
the acquisition function. Injected with our prior beliefs about the latent
function, the surrogate model provides both the predictive mean and
uncertainty estimation for each location based on the collected data points.
By leveraging the posterior distribution provided by the surrogate model, the
acquisition function works as a utility function to measure the potential of
each location over the design space and observe the most promising data points
in an iterative and greedy manner. After a limited number of iterations, the
global optimum can be reached with a theoretical guarantee [19, 20]. Compared
to the model-based methods that only build the model once and searches for the
global optimum offline, the Bayesian optimization approach incrementally
updates the surrogate model and iteratively invokes the circuit simulator on
the fly. Compared to the simulation-based methods that solely explores the
state space with a heuristically designed selection engine, the Bayesian
optimization approach makes full use of the previously collected information
and facilitates the searching procedure more economically. Prior researches
have demonstrated the efficiency and effectiveness of the Bayesian
optimization framework for analog circuit synthesis [2, 3, 21, 7].

Although lots of efforts have spent on designing an informative surrogate
model and efficient acquisition function [22, 23, 24, 3, 7, 25, 19, 5, 26],
the acquisition function optimization procedure itself is always assumed to be
relatively easy and largely ignored. However, the global optimum can only be
theoretically guaranteed for the Bayesian optimization framework when it
successfully finds the global optimum of the acquisition function at each
iteration. The efficiency of the Bayesian optimization framework can be
significantly hurt if less promising data points are accumulated across
iterations. In other words, the acquisition function optimization procedure is
important both theoretically and practically. With the increase of the
dimensionality, this problem only deteriorates and further sabotages the
efficiency of the Bayesian optimization framework. In this paper, we propose a
fast and robust Bayesian optimization approach via one-dimensional subspaces
[27] for analog circuit synthesis. Unlike the traditional Bayesian
optimization methods that optimize the acquisition function over the entire
design space, we solely focus on one dimension at a time. In this way, our
proposed algorithm makes solid progress at each iteration and ensures
information gain regardless of the dimensionality. We also leverage the
carefully designed batch Bayesian optimization framework to further increase
sampling efficiency. Experimental results quantitatively demonstrate that our
proposed algorithm can accelerate the optimization procedure by up to
$9\times$ and $38\times$ compared to LP-EI [28] and REMBOpBO [29] respectively
while obtaining the same optimization results when the batch size is 15.

The rest of the paper is organized as follows. In §II, we formulate the device
sizing problem and briefly review the knowledge background of our proposed
algorithm. In §III, we analytically highlight the challenges of the
acquisition function optimization procedure and our proposed method of
handling these problems. In §IV, we conduct experiments on two real-world
analog circuits and quantitatively demonstrate the efficiency and
effectiveness of our proposed algorithm. Finally, we conclude the paper in §V.

##  II Background

In this section, we present both the problem formulation of device sizing
(§II-A) and a brief review of the Bayesian optimization framework (§II-B).

###  II-A Problem Formulation

Given a topology selection, the analog circuit device sizing problem can be
formulated into a black-box function optimization problem, which can be
expressed as follow:

$\text{minimize}\quad f(\bm{x}).$ (1)

By searching the $d$-dimensional design space, our target is to find an
optimal circuit design $\bm{x}$ that minimizes the Figure of Merit (FOM)
$f(\cdot)$ over the design space $\mathcal{X}$. For each design $\bm{x}$, we
regard the simulation results generated by circuit simulators including
commercial HSPICE and Spectre software as the ground-truth circuit
performances $f(\bm{x})$.

###  II-B Review of the Bayesian Optimization Framework

There are two basic elements in the Bayesian optimization framework: the
surrogate model and the acquisition function [17]. The surrogate model
captures our prior beliefs about the latent function and provides an
informative posterior distribution for each location over the design space.
The posterior distribution means both the predictive mean and uncertainty
estimation are provided. The predictive mean describes the behavior pattern of
the latent function predicted by the surrogate model. The uncertainty
estimation shows how much confidence the model has on its predictive mean. In
other words, the surrogate model not only tells us what it believes but also
how much we can trust its beliefs. Compared to deterministic models that only
provide predictive mean and guide the search in a greedy manner, the
stochastic statistical surrogate model employed by the Bayesian optimization
framework can better facilitate the searching process and help to make more
rational decisions.

One commonly used surrogate model is the Gaussian process regression (GPR)
model. Assume the we have an initial dataset $D=\\{X,\bm{y}\\}$, where
$X=\\{\bm{x}_{1},\cdots,\bm{x}_{N}\\}$, $\bm{y}=\\{y_{1},\cdots,y_{N}\\}$, and
$N$ is the size of the dataset. By injecting our prior assumption about the
latent function with mean function $m(\cdot)$ and kernel function
$k(\cdot,\cdot)$, the posterior distribution for each location $\bm{x}_{\ast}$
can be expressed as [30]:

$\begin{cases}\begin{aligned}
\mu(\bm{x}_{\ast})&=m(\bm{x}_{\ast})+k(\bm{x}_{\ast})[K+\sigma_{n}^{2}I]^{-1}(\bm{y}-\bm{m})\\\
\sigma^{2}(\bm{x}_{\ast})&=k(\bm{x}_{\ast},\bm{x}_{\ast})-k(\bm{x}_{\ast},X)[K+\sigma^{2}_{n}I]^{-1}k(X,\bm{x}_{\ast}),\end{aligned}\end{cases}$
(2)

where $\bm{m}=(m(\bm{x}_{1}),\cdots,m(\bm{x}_{N}))^{T}$,
$K=(k(\bm{x}_{1},X),\cdots,k(\bm{x}_{N},X))^{T}$, and $\sigma^{2}_{n}$ is the
noise variance. In this paper, we fix the mean function as $m(\bm{x})=0$ and
the kernel function as the square exponential (SE) covariance function.

By taking both the predictive mean and uncertainty estimation into
consideration, the acquisition function works as a utility function to
prioritize each location over the design space. By favoring the potential area
with high uncertainty estimation, the acquisition function explores the
unknown region to avoid getting stuck in the local optimum. By selecting
locations with high probability to be optimal, the acquisition function
exploits the collected information to avoid too much sampling around the less
promising area. In other words, the acquisition function trades off between
searching globally (exploration) and locally (exploitation). Popular
acquisition functions include the probability of improvement (PI) [31],
expected improvement (EI) [32], lower confidence bound (LCB) [19], Thompson
sampling (TS) [33] and entropy search (ES) [34]. A portfolio of several
acquisition functions is also possible [35, 25].

The Bayesian optimization framework starts the optimization procedure by first
randomly sampling an initial dataset. By embedding our prior beliefs about the
unknown objective function, the surrogate model builds upon the collected data
points and provides the posterior distribution for each location over the
design space. By taking both exploration and exploitation into consideration,
the acquisition function helps to select the query points and traverse the
design space efficiently. At each iteration, selected data points are observed
to incrementally update the dataset. With a limited number of iterations, the
Bayesian optimization framework can theoretically obtain the global optimum.
The corresponding framework of the Bayesian optimization algorithm is
presented in Algorithm 1.

Algorithm 1 Bayesian Optimization Framework

Randomly sample an initial dataset $D_{0}=\\{X,\bm{y}\\}$

for $t=1$ to $T$ do

Build a Gaussian process regression model with $D_{t-1}$

Select $\bm{x}_{t}$ that optimizes the acquisition function

Observe the selected data $y_{t}=f(\bm{x}_{t})$

Update the dataset $D_{t}=\\{D_{t-1},\\{\bm{x}_{t},y_{t}\\}\\}$

end for

return The best $f(\bm{x})$ recorded during optimization

##  III Proposed Approach

In this section, we first highlight the fundamental challenges of the
acquisition function optimization procedure and how it affects the efficiency
of the Bayesian optimization framework (§III-A). Then, we explicitly describe
our propose algorithm to handle this problem (§III-B and §III-C). Finally, we
summarize the proposed method (§III-D).

###  III-A Challenges of Acquisition Function Optimization

A large body of literature has been published to improve the efficiency of the
Bayesian optimization framework. However, most of the previous researches
mainly focus on designing a more informative surrogate model or an efficient
acquisition function. The inner optimization procedure (a.k.a the acquisition
function optimization procedure) has been assumed to relatively easy and often
neglected.

From a theoretical point of view, the global optimum is only granted with a
limited number of iterations for the Bayesian optimization framework, if the
global optimum of the acquisition function is obtained at each iteration. In
other words, the success of the Bayesian optimization framework crucially
relies on successfully optimizing the acquisition function. The less promising
data points accumulated across iterations can violate the theoretical
justification of the Bayesian optimization framework. Therefore, it is
theoretically important to optimize the acquisition function optimally.

From a practical point of view, the efficiency of the acquisition function
optimization procedure suffers greatly from the curse of dimensionality. With
the increase of the dimensionality, the state space for exploration increases
exponentially. To search for the global optimum globally and make full
coverage of the design space, the required number of acquisition function
estimations can be prohibitively large. Take the grid search as an example,
let us assume that we evenly select $M$ data points for acquisition function
estimation at each dimension to make full coverage of the design space. The
expected number of data points for acquisition function estimation is $M^{d}$
when the design space is $d$-dimensional. Considering that the computational
complexity of the training and prediction procedure for GPR model are
$O(N^{3})$ and $O(N^{2})$ respectively, the computational complexity for
acquisition function optimization is $O(N^{2}M^{d})$. Therefore, even if a
large body of literature claims that the theoretical computational bottleneck
of the Bayesian optimization framework is the model training process, the
acquisition function optimization procedure easily dominants the computational
overhead in practice. Although other algorithms have also been studied to
facilitate the acquisition function optimization procedure including MSP
algorithm [7], stochastic gradient ascent algorithm [36], evolutionary
strategy [37], DIRECT [38], etc. This problem is only alleviated not solved.

In summary, the acquisition function optimization procedure is theoretically
important and practically difficult. We tackle this problem by proposing a
scalable Bayesian optimization framework in this paper.

###  III-B Optimization in One-Dimensional Subspaces

Although optimizing the acquisition function over the whole design space is
practically expensive or even intractable, searching for the global optimum of
the acquisition function over the one-dimensional subspace at a time is
relatively easy [39]. Instead of optimizing the acquisition function all over
the design space, we constrain the searching space to a one-dimensional
subspace $\mathcal{L}_{i}=\\{\bm{x}_{\ast}+\beta
l_{i}:\beta\in\mathbb{R}\\}\cap\mathcal{X}$ that contains the previously
sampled best point $\bm{x}_{\ast}$ [27], where $\mathcal{L}_{i}$ represents
the $i$-th one-dimensional subspace and $l_{i}$ denotes the $i$-th direction.
In this way, the global optimum of the acquisition function can be effectively
obtained by grid search. Without surrendering the efficiency of the
acquisition function optimization procedure, our proposed algorithm makes
solid progress and ensures information gain at each iteration.

With the increase of the dimensionality, the traditional way of handling the
acquisition function optimization procedure tends to follow the same behavior
pattern as random search. Our proposed method instead makes solid and safe
progress and maximizes information gain in the selected one-dimensional
subspace, thus, improves the sampling efficiency.

###  III-C Effective Dimension Selection

According to the above, another interesting question arises: how to select the
effective dimension for optimization?

Random Selection: The intuitive and naive way of dimension selection is random
selection. By randomly selecting the dimension for optimization, the design
space at each one-dimensional subspace can be equally searched, thus, globally
explored. However, this dimension selection policy greatly neglects the
characteristics of different optimization problems and tends to evenly sample
around the low and high potential subspaces.

Descent Direction Estimation: Another dimension selection policy is to
calculate the descent direction heuristically based on the Thompson sampling
[33]. By evaluating the utility of the best data point $\bm{x}_{\ast}$ and the
$d$ data points around it with a small deviation $\bm{x}_{\ast}+\hat{g}l_{i}$
in each direction, we can calculate the gradient $\hat{g}$ at each dimension
and focus on the most crucial one. However, this dimension selection strategy
tends to constantly sample around several most effective subspaces and easily
get stuck in the local optimum.

In this paper, we propose a mixture of selection strategy to combine the
benefits of random selection and descent direction estimation. With
probability $p$, we randomly select the dimension for optimization. With
probability $1-p$, we select the dimension with descent direction estimation
based on the Thompson sampling. In this way, parameter $p$ helps to balance
between searching globally or locally. And we fix $p$ as 0.8 in this paper.

###  III-D Summary

Instead of optimizing the acquisition function over the whole design space, we
propose to focus on a single dimension at a time to ensure information gain at
each iteration. By combining the random selection strategy and the descent
direction estimation policy, we adaptively avoid sampling too much in the low
potential subspaces while globally exploring the design space. To fully
utilize the hardware resources and further accelerate the optimization
procedure, we also leverage the asynchronous batch Bayesian optimization
framework EasyBO [40] to enable parallelism. By randomly selecting the
exploration parameter, we achieve a delicate exploration-exploitation tradeoff
and maximize the information gain per data point. The heuristically observed
data points are employed to naturally penalize around the previously selected
data points that are still under observation. The overall framework of our
proposed algorithm LinEasyBO is presented in Algorithm 2.

Algorithm 2 LinEasyBO Framework

Randomly sample an initial dataset $D_{0}=\\{X,\bm{y}\\}$, and the number of
works $B$ to evaluate the design in parallel

for t = 1 to T do

Wait for a worker to finish

Query the observed data $D_{t}=\\{D_{t-1},\\{\bm{x}_{t},y_{t}\\}\\}$

Build a Gaussian process regression model with
$D_{t}\cup\\{\hat{X},\hat{\bm{y}}\\}$, where
$\hat{X}=\\{\hat{\bm{x}}_{1},\cdots,\hat{\bm{x}}_{B-1}\\}$ is the previously
selected data points remaining to be observed, and
$\hat{\bm{y}}=\\{\mu(\hat{\bm{x}}_{1}),\cdots,\mu(\hat{\bm{x}}_{B-1})\\}$ is
the predictive mean of the $\hat{X}$ that works as the heuristically
observated data points

Get the current best design $\bm{x}_{\ast}=\text{argmax}_{\bm{x}}f(\bm{x})$

Select the dimension $l_{t}$ for optimization (§III-C)

Select $\bm{x}_{t}$ that optimizes the acquisition function (§III-B)

Observe the selected data with the idle worker $y_{t}=f(\bm{x}_{t})$

end for

return The best $f(\bm{x})$ recorded during optimization

##  IV Experimental Results

In this section, we conduct experiments on two real-world analog circuits: the
charge pump circuit (§IV-A) and the class-E power amplifier circuit (§IV-B).
To demonstrate the efficiency of our proposed method in both sequential and
batch mode, we compared the performances of LinEasyBO with several state-of-
the-art algorithms in terms of both optimization results and wall-clock time.
To distinguish between the sequential and batch mode, we label each algorithm
with its corresponding batch size. In general, the tested algorithms can be
divided into the following four categories.

  1. C1

Random search, a perfect algorithm to measure the difficulty of optimization
problems and the efficiency of the optimization algorithms [41].

  2. C2

Sequential Bayesian optimization algorithms that work as baselines.

  3. C3

Sequential Bayesian optimization algorithms that selects the query points in
one-dimensional subspaces including our proposed algorithm.

  4. C4

Bayesian optimization algorithms that work in the batch mode.

For the sequential Bayesian optimization algorithms that work as baselines
(C2), we run experiments on three algorithms: (1) EI [32], an improvement-
based acquisition function that helps to explore the design space. (2) LCB
[19], an optimistic policy that searches the design space in sequence. (3)
EasyBO [40], an efficient Bayesian optimization algorithm that selects the
exploration parameter randomly.

To demonstrate the effectiveness of optimizing the inner loop in the one-
dimensional subspaces (C3), we run experiments on three alternative
algorithms: (1) Line-EI, an improvement-based policy that optimizes the
acquisition function in one-dimensional subspaces. (2) Line-LCB, an optimistic
strategy that searches the next query points in line. (3) LinEasyBO, which is
our proposed algorithm in sequential mode.

Finally, we comprehensively instantiate C4 with experiments on four
algorithms: (1) LP-EI [28], an improvement-based strategy that maintains
diversity in the batch with Lipschitz constant. (2) LP-LCB [28], an optimistic
policy that introduces a local repulsive term to reduce redundantly sampling
around the same area. (3) REMBOpBO [29], an efficient Bayesian optimization
framework that handles the high-dimensional problems by projecting the design
variables into a high effective subspace. (4) LinEasyBO, which is our proposed
asynchronous algorithm in batch mode.

To ensure a fair comparison and reduce implementation bias, we implement the
Gaussian process regression model with GPyOpt [28] library for all test
algorithms. For REMBOpBO, we follow the experimental setting of [29] and
optimize the acquisition function with NLopt [42] library. For the rest of the
algorithms, we implement the inner optimization procedure with Scipy [43]
library. For each algorithm, we limit the maximum number of circuit simulation
as 350 and set the size of the initial dataset as 20. We also run each
algorithm 20 times to reduce random fluctuations. All circuit performances are
generated with the commercial HSPICE circuit simulator. And all experiments
are conducted on a Linux workstation with two Intel Xeon CPUs and 128GB
memory.

###  IV-A Charge Pump

Figure 1: The schematic of the charge pump circuit, which is reproduced from
[6].

The schematic of the charge pump circuit is presented in Figure 1. Implemented
in an SMIC 40nm process, there are a total of 36 design variables, including
lengths and widths of the transistors, the resistance of the resistors and the
capacitance of the capacitors. The target of design is to ensure the current
of M1 and M2 stay close to 40$\mu A$ across 18 PVT corners [6]. The design
specification of the charge pump is as follows:

$\text{minimize}\quad 0.3\times\mathit{diff}+0.5\times\mathit{deviation},$ (3)

where

$\begin{cases}\begin{aligned} \mathit{diff_{1}}&=\text{max}_{\forall
PVT}(I_{M1,max}-I_{M1,avg}),\\\ \mathit{diff_{2}}&=\text{max}_{\forall
PVT}(I_{M1,avg}-I_{M1,min}),\\\ \mathit{diff_{3}}&=\text{max}_{\forall
PVT}(I_{M2,max}-I_{M2,avg}),\\\ \mathit{diff_{4}}&=\text{max}_{\forall
PVT}(I_{M2,avg}-I_{M2,min}),\\\
\mathit{diff}&=\sum^{4}_{i=1}\mathit{diff_{i}},\\\
\mathit{deviation}&=\text{max}_{\forall PVT}(I_{M1,avg}-40\mu A)\\\
&+\text{max}_{\forall PVT}(I_{M2,avg}-40\mu A).\\\ \end{aligned}\end{cases}$
(4)

And the experimental results are presented in Table I.

TABLE I: The optimization results of the charge pump circuit. Algo | Best | Worst | Mean | Std | Time
---|---|---|---|---|---
Random Search | 21.16 | 46.39 | 30.49 | 7.52 | -
EI | 3.42 | 4.67 | 4.09 | 0.35 | 7h45m32s
LCB | 3.45 | 27.92 | 5.22 | 5.22 | 6h34m11s
EasyBO | 5.09 | 7.43 | 6.15 | 0.69 | 7h57m51s
Line-EI | 3.11 | 4.40 | 3.37 | 0.29 | 7h56m52s
Line-LCB | 3.16 | 3.77 | 3.39 | 0.17 | 7h53m1s
LinEasyBO | 3.16 | 4.06 | 3.39 | 0.22 | 7h39m49s
LP-EI-5 | 3.49 | 5.13 | 4.06 | 0.39 | 2h10m54s
LP-LCB-5 | 3.63 | 5.08 | 4.17 | 0.36 | 1h48m6s
REMBOpBO-5 | 9.86 | 32.75 | 14.70 | 4.87 | 2h21m48s
LinEasyBO-5 | 3.18 | 4.11 | 3.44 | 0.25 | 1h45m58s
LP-EI-10 | 3.77 | 5.63 | 4.59 | 0.45 | 1h33m38s
LP-LCB-10 | 3.53 | 4.89 | 4.24 | 0.30 | 53m52s
REMBOpBO-10 | 8.76 | 27.68 | 17.51 | 5.12 | 1h43m2s
LinEasyBO-10 | 3.32 | 4.71 | 3.71 | 0.34 | 49m58s
LP-EI-15 | 4.02 | 5.85 | 5.02 | 0.49 | 1h22m23s
LP-LCB-15 | 3.60 | 4.93 | 4.29 | 0.33 | 42m31s
REMBOpBO-15 | 11.35 | 32.12 | 18.31 | 5.62 | 1h28m2s
LinEasyBO-15 | 3.23 | 4.90 | 3.95 | 0.41 | 33m33s

Let us first focus on the sequential mode. It is worth notice that the
performances of EI, LCB and EasyBO significantly outperform that of the random
search algorithm. This reveals that the design specification of the charge
pump circuit is hard to optimization and the Bayesian optimization framework
can guide the search efficiently and effectively. Another interesting
observation is that those optimization algorithms that only focus on exploring
one dimension at a time (Line-EI, Line-LCB and LinEasyBO) consistently
outperform their traditional counterparts. This clearly suggests that
searching the design space in one-dimensional subspaces is beneficial
regardless of the acquisition function. This also emphasizes that the
efficiency of the Bayesian optimization framework can be largely impacted by
the inner optimization procedure. By solely focusing on one dimension at a
time, space exploration efficiency can be greatly improved.

Figure 2: The optimization results with respect to the wall clock time for the
charge pump circuit, when the batch size is 15.

Now let us move on to analyzing the optimization results in batch mode.
Compared to LP-EI, LP-LCB, and REMBOpBO, LinEasyBO consistently demonstrates
better performances in terms of the optimization results and time consumption
on average regardless of the batch size. This clearly shows the efficiency and
effectiveness of our proposed algorithm. The performances of LP-EI, LP-LCB,
REMBOpBO, and LinEasyBO deteriorate with the increase of the batch size. This
implies that the design specification of the charge pump circuit is rather
sensitive to the exploration parameter of the acquisition function. In other
words, even if there is only a small change in the exploration-exploitation
tradeoff of the acquisition function, the final optimization results can
fluctuate greatly for this circuit. The fact that LinEasyBO constantly
outperforms EI, LCB and EasyBO in both sequential and batch mode again
emphasizes the importance of the inner optimization procedure. The fact that
REMBOpBO is always outperformed to a large extend by LP-EI, LP-LCB, and
LinEasyBO suggests that the low effective dimensionality preassumption doesn’t
hold for the charge pump circuit. The strong assumption that a low-dimensional
effective subspace exists for the optimization problem can greatly limit the
application of REMBOpBO, while LinEasyBO is not affected by the characteristic
of the problem and can be widely and safely used without much consideration of
the application scenario. As is shown in Figure 2, compared with LP-LCB, LP-
EI, and REMBOpBO, LinEasyBO reduces 56.1%, 84.9%, and 96.6% of the time
consumption respectively when the batch size is 15, while achieving the same
optimization results. This translates to $2\times$, $6\times$, and $29\times$
optimization procedure acceleration respectively.

###  IV-B Class-E Power Amplifier

Figure 3: The schematic of the class-E power amplifier circuit, which is
reproduced from [25].

The schematic of the class-E power amplifier circuit is presented in Figure 3.
Implemented in a TSMC 180nm process, there are a total of 12 design variables.
The target of design for this circuit is to maximize the power-added
efficiency (PAE) and the output power (Pout) simultaneously [25]. The
corresponding design specification is as follows:

$\text{maximize}\quad 3\times PAE+Pout.$ (5)

And the optimization results are shown in Table II.

As usual, we start with looking at the sequential mode. Again, the Bayesian
optimization algorithms with our proposed acquisition function optimizer
consistently outperform their traditional counterparts. This again
demonstrates the efficiency and effectiveness of searching the design space
one dimension at a time and the importance of successfully optimizing the
acquisition function at each iteration. Another interesting and unexpected
phenomenon is that random search algorithm obtains better optimization results
than EI and comparable performance with LCB. This strongly stresses the fact
that a less efficient inner optimization strategy can guide the search in the
wrong direction and greatly hurt the performance of the optimization
algorithm. By only focusing on a single dimension and optimizing the
acquisition function optimally at each iteration, the Bayesian optimization
algorithm can iteratively make solid progress and search the state space in a
safe and efficient manner. Our proposed acquisition function optimization
method can be easy-to-use and effective regardless of the acquisition function
and beneficial across different dimensionality.

Figure 4: The optimization results on average with respect to the wall-clock
time for the class-E power amplifier circuit, when the batch size is 15.

Let’s now analyze the optimization results in batch mode. One interesting
phenomenon is that REMBOpBO is constantly outperformed by random search. The
assumption that only a small group of design variables have a great impact on
the circuit performance can significantly hurt the performance of REMBOpBO
when the preassumption is not met. This makes REMBOpBO a risky choice when we
have no information at hand about the black-box optimization problem. The fact
that LinEasyBO consistently outperforms LP-EI, LP-LCB, and REMBOpBO with less
time consumption again demonstrates the efficiency and effectiveness of our
proposed algorithm. By focusing on a single dimension at a time, the
acquisition function optimization procedure can help to make solid and safe
decisions and maximize the information gain in the long run. The comparable
optimization results of LinEasyBO across different batch size show the
robustness of our proposed algorithm. As is presented in Figure 4, LinEasyBO
reduces the simulation time (accelerates the optimization procedure) by 88.8%
($9\times$), 91.7% ($11\times$), and 97.4% ($38\times$) compared to LP-EI, LP-
LCB, and REMBOpBO respectively when the batch size is 15, while obtaining the
same optimization results.

TABLE II: The optimization results of class-E power amplifier circuit. Algo | Best | Worst | Mean | Std | Time
---|---|---|---|---|---
Random Search | 4.16 | 3.34 | 3.70 | 0.21 | -
EI | 4.36 | 1.60 | 3.68 | 0.52 | 12h22m8s
LCB | 4.12 | 3.42 | 3.79 | 0.21 | 9h10m31s
EasyBO | 4.51 | 3.69 | 4.01 | 0.15 | 6h49m46s
Line-EI | 4.83 | 3.21 | 4.29 | 0.49 | 6h4m45s
Line-LCB | 4.81 | 2.82 | 4.03 | 0.52 | 6h4m46s
LinEasyBO | 5.44 | 3.35 | 4.34 | 0.58 | 6h17m15s
LP-EI-5 | 4.17 | 3.35 | 3.83 | 0.21 | 1h45m3s
LP-LCB-5 | 4.26 | 3.42 | 3.79 | 0.26 | 2h1m38s
REMBOpBO-5 | 4.08 | 3.04 | 3.57 | 0.21 | 2h39m12s
LinEasyBO-5 | 5.53 | 3.19 | 4.32 | 0.47 | 1h21m41s
LP-EI-10 | 4.60 | 3.39 | 3.84 | 0.30 | 1h17m14s
LP-LCB-10 | 4.21 | 3.45 | 3.77 | 0.20 | 1h34m13s
REMBOpBO-10 | 3.92 | 1.73 | 3.54 | 0.46 | 2h44m57s
LinEasyBO-10 | 5.32 | 3.16 | 4.25 | 0.57 | 40m43s
LP-EI-15 | 4.34 | 3.49 | 3.86 | 0.23 | 1h6m3s
LP-LCB-15 | 4.08 | 3.41 | 3.81 | 0.15 | 1h23m40s
REMBOpBO-15 | 4.10 | 2.92 | 3.55 | 0.22 | 2h33m23s
LinEasyBO-15 | 5.96 | 3.21 | 4.30 | 0.74 | 27m23s

##  V Conclusion

In this paper, we proposed a fast and robust Bayesian optimization framework
for analog circuit synthesis via one-dimensional subspaces. Instead of
searching the global optimum for acquisition function over the whole design
space, our proposed method focuses on one dimension at a time. We also
introduce an effective dimension selection strategy to balance searching
globally and locally. In this way, our proposed algorithm guarantees
information gain at each step and searches the design space safely and
efficiently. To further accelerate the optimization procedure, we leverage the
batch Bayesian optimization framework to fully utilize the computational
resources. Compared to the traditional acquisition function optimization
procedure, our method not only reduces the computational overhead for the
Bayesian optimization framework but also increases the information gain per
observation. We experimentally compare the performance of our proposed
algorithm with several state-of-the-art optimization algorithms. Experimental
results quantitatively show that our method accelerates the optimization
procedure by up to $9\times$ and $38\times$ compared to LP-EI and REMBOpBO
when the batch size is 15.

## References

  * [1] R. A. Rutenbar, G. G. Gielen, and J. Roychowdhury, “Hierarchical modeling, optimization, and synthesis for system-level analog and rf designs,” _Proceedings of the IEEE_ , vol. 95, no. 3, pp. 640–669, 2007.
  * [2] B. Liu, Y. Wang, Z. Yu, L. Liu, M. Li, Z. Wang, J. Lu, and F. V. Fernandez, “Analog circuit optimization system based on hybrid evolutionary algorithms,” _Integration_ , vol. 42, no. 2, pp. 137–148, 2009.
  * [3] B. Liu, D. Zhao, P. Reynaert, and G. Gielen, “Gaspad: A general and efficient mm-wave integrated circuit synthesis method based on surrogate model assisted evolutionary algorithm,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 33, no. 2, pp. 169–182, 2014.
  * [4] R. Phelps, M. Krasnicki, R. A. Rutenbar, L. R. Carley, and J. R. Hellums, “Anaconda: simulation-based synthesis of analog circuits via stochastic pattern search,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 19, no. 6, pp. 703–717, 2000.
  * [5] H. Hu, P. Li, and J. Z. Huang, “Parallelizable bayesian optimization for analog and mixed-signal rare failure detection with high coverage,” in _Proceedings of the International Conference on Computer-Aided Design_ , 2018, pp. 1–8.
  * [6] Y. Yang, H. Zhu, Z. Bi, C. Yan, D. Zhou, Y. Su, and X. Zeng, “Smart-msp: A self-adaptive multiple starting point optimization approach for analog circuit synthesis,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 37, no. 3, pp. 531–544, 2018.
  * [7] W. Lyu, P. Xue, F. Yang, C. Yan, Z. Hong, X. Zeng, and D. Zhou, “An efficient bayesian optimization approach for automated optimization of analog circuits,” _IEEE Transactions on Circuits and Systems I: Regular Papers_ , vol. 65, no. 6, pp. 1954–1967, 2017.
  * [8] S. P. Boyd and S. J. Kim, “Geometric programming for circuit optimization,” in _Proceedings of the 2005 international symposium on Physical design_ , 2005, pp. 44–46.
  * [9] S. Boyd, S.-J. Kim, L. Vandenberghe, and A. Hassibi, “A tutorial on geometric programming,” _Optimization and engineering_ , vol. 8, no. 1, p. 67, 2007\.
  * [10] L. Hannah and D. Dunson, “Ensemble methods for convex regression with applications to geometric programming based circuit design,” _arXiv preprint arXiv:1206.4645_ , 2012.
  * [11] G. Gielen, H. Walscharts, and W. Sansen, “Analog circuit design optimization based on symbolic simulation and simulated annealing,” _IEEE Journal of Solid-state Circuits_ , vol. 25, no. 3, pp. 707–713, 1990.
  * [12] D. Grzechca, T. Golonek, and J. Rutkowski, “The use of simulated annealing with fuzzy objective function to optimal frequency selection for analog circuit diagnosis,” in _2007 14th IEEE International Conference on Electronics, Circuits and Systems_. IEEE, 2007, pp. 899–902.
  * [13] B. Peng, F. Yang, C. Yan, X. Zeng, and D. Zhou, “Efficient multiple starting point optimization for automated analog circuit optimization via recycling simulation data,” in _Proceedings of the 2016 Conference on Design, Automation & Test in Europe_. EDA Consortium, 2016, pp. 1417–1422.
  * [14] G. Alpaydin, S. Balkir, and G. Dundar, “An evolutionary approach to automatic synthesis of high-performance analog integrated circuits,” _IEEE Transactions on Evolutionary Computation_ , vol. 7, no. 3, pp. 240–252, 2003.
  * [15] M. Fakhfakh, Y. Cooren, A. Sallem, M. Loulou, and P. Siarry, “Analog circuit design optimization through the particle swarm optimization technique,” _Analog Integrated Circuits and Signal Processing_ , vol. 63, no. 1, pp. 71–82, 2010.
  * [16] C. Wu, D. Wang, A. W. H. Ip, D. Wang, C. Chan, and H. Wang, “A particle swarm optimization approach for components placement inspection on printed circuit boards,” _Journal of Intelligent Manufacturing_ , vol. 20, no. 5, pp. 535–549, 2009.
  * [17] B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas, “Taking the human out of the loop: A review of bayesian optimization,” _Proceedings of the IEEE_ , vol. 104, no. 1, pp. 148–175, 2015.
  * [18] E. Brochu, V. M. Cora, and N. De Freitas, “A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning,” _arXiv preprint arXiv:1012.2599_ , 2010\.
  * [19] N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger, “Gaussian process optimization in the bandit setting: No regret and experimental design,” _arXiv preprint arXiv:0912.3995_ , 2009.
  * [20] Z. Wang and S. Jegelka, “Max-value entropy search for efficient bayesian optimization,” _arXiv preprint arXiv:1703.01968_ , 2017.
  * [21] B. Liu, S. Koziel, and Q. Zhang, “A multi-fidelity surrogate-model-assisted evolutionary algorithm for computationally expensive optimization problems,” _Journal of computational science_ , vol. 12, pp. 28–37, 2016.
  * [22] S. Zhang, W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, “Bayesian optimization approach for analog circuit synthesis using neural network,” in _2019 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2019, pp. 1463–1468.
  * [23] S. Zhang, W. Lyu, F. Yang, C. Yan, D. Zhou, X. Zeng, and X. Hu, “An efficient multi-fidelity bayesian optimization approach for analog circuit synthesis,” in _Proceedings of the 56th Annual Design Automation Conference 2019_. ACM, 2019, pp. 1–6.
  * [24] S. Zhang, F. Yang, D. Zhou, and X. Zeng, “An efficient asynchronous batch bayesian optimization approach for analog circuit synthesis,” in _2020 57th ACM/IEEE Design Automation Conference (DAC)_. IEEE, 2020, pp. 1–6.
  * [25] W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, “Batch bayesian optimization via multi-objective acquisition ensemble for automated analog circuit design,” in _International Conference on Machine Learning_ , 2018, pp. 3312–3320.
  * [26] B. He, S. Zhang, F. Yang, C. Yan, D. Zhou, and X. Zeng, “An efficient bayesian optimization approach for analog circuit synthesis via sparse gaussian process modeling,” in _2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2020, pp. 67–72.
  * [27] J. Kirschner, M. Mutnỳ, N. Hiller, R. Ischebeck, and A. Krause, “Adaptive and safe bayesian optimization in high dimensions via one-dimensional subspaces,” _arXiv preprint arXiv:1902.03229_ , 2019.
  * [28] J. González, Z. Dai, P. Hennig, and N. Lawrence, “Batch bayesian optimization via local penalization,” in _Artificial intelligence and statistics_ , 2016, pp. 648–657.
  * [29] H. Hu, P. Li, and J. Z. Huang, “Enabling high-dimensional bayesian optimization for efficient failure detection of analog and mixed-signal circuits,” in _2019 56th ACM/IEEE Design Automation Conference (DAC)_. IEEE, 2019, pp. 1–6.
  * [30] C. E. Rasmussen, “Gaussian processes in machine learning,” _Lecture Notes in Computer Science_ , pp. 63–71, 2003.
  * [31] H. J. Kushner, “A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise,” 1964.
  * [32] J. Mockus, V. Tiesis, and A. Zilinskas, “The application of bayesian methods for seeking the extremum,” _Towards global optimization_ , vol. 2, no. 117-129, p. 2, 1978.
  * [33] W. R. Thompson, “On the likelihood that one unknown probability exceeds another in view of the evidence of two samples,” _Biometrika_ , vol. 25, no. 3/4, pp. 285–294, 1933.
  * [34] P. Hennig and C. J. Schuler, “Entropy search for information-efficient global optimization,” _The Journal of Machine Learning Research_ , vol. 13, no. 1, pp. 1809–1837, 2012.
  * [35] M. D. Hoffman, E. Brochu, and N. de Freitas, “Portfolio allocation for bayesian optimization.” in _UAI_. Citeseer, 2011, pp. 327–336.
  * [36] J. Wilson, F. Hutter, and M. Deisenroth, “Maximizing acquisition functions for bayesian optimization,” in _Advances in Neural Information Processing Systems_ , 2018, pp. 9884–9895.
  * [37] W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, “Multi-objective bayesian optimization for analog/rf circuit synthesis,” in _Proceedings of the 55th Annual Design Automation Conference_. ACM, 2018, pp. 1–6.
  * [38] D. R. Jones, C. D. Perttunen, and B. E. Stuckman, “Lipschitzian optimization without the lipschitz constant,” _Journal of optimization Theory and Applications_ , vol. 79, no. 1, pp. 157–181, 1993.
  * [39] J. Scarlett, “Tight regret bounds for bayesian optimization in one dimension,” _arXiv preprint arXiv:1805.11792_ , 2018.
  * [40] S. Zhang, F. Yang, D. Zhou, and X. Zeng, “Bayesian methods for the yield optimization of analog and sram circuits,” in _2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)_. IEEE, 2020, pp. 440–445.
  * [41] J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” _The Journal of Machine Learning Research_ , vol. 13, no. 1, pp. 281–305, 2012.
  * [42] S. G. Johnson, “The nlopt nonlinear-optimization package,” 2014.
  * [43] E. Jones, T. Oliphant, P. Peterson _et al._ , “Scipy: Open source scientific tools for python,” 2001.


#  All-optical sub-Kelvin sympathetic cooling of a levitated microsphere in
vacuum

Yoshihiko Arita<EMAIL_ADDRESS>SUPA, School of Physics & Astronomy,
University of St Andrews, North Haugh, St Andrews, KY16 9SS, United Kingdom
Graham D. Bruce  SUPA, School of Physics & Astronomy, University of St
Andrews, North Haugh, St Andrews, KY16 9SS, United Kingdom  Ewan M. Wright
Wyant College of Optical Sciences, The University of Arizona, 1630 East
University Boulevard, Tucson, Arizona 85721, USA  SUPA, School of Physics &
Astronomy, University of St Andrews, North Haugh, St Andrews, KY16 9SS, United
Kingdom  Stephen H. Simpson  Institute of Scientific Instruments of the Czech
Academy of Science, v.v.i., Královopolská 147, 612 64 Brno, Czech Republic
Pavel Zemánek  Institute of Scientific Instruments of the Czech Academy of
Science, v.v.i., Královopolská 147, 612 64 Brno, Czech Republic  Kishan
Dholakia<EMAIL_ADDRESS>SUPA, School of Physics & Astronomy, University
of St Andrews, North Haugh, St Andrews, KY16 9SS, United Kingdom  Wyant
College of Optical Sciences, The University of Arizona, 1630 East University
Boulevard, Tucson, Arizona 85721, USA  Department of Physics, College of
Science, Yonsei University, Seoul 03722, South Korea  School of Biological
Sciences, The University of Adelaide, Adelaide, South Australia, Australia

###### Abstract

We demonstrate all-optical sympathetic cooling of a laser-trapped microsphere
to sub-Kelvin temperatures, mediate by optical binding to a feedback-cooled
adjacent particle. Our study opens prospects for multi-particle quantum
entanglement and sensing in levitated optomechanics.

_This article was published by Optica Publishing Group under the terms of
theCreative Commons Attribution 4.0 License. Further distribution of this work
must maintain attribution to the author(s) and the published article’s title,
journal citation, and DOI https://doi.org/10.1364/OPTICA.466337_

Single, levitated mesoscopic objects are providing an excellent platform for
sensing weak forces Ranjit  _et al._ (2016) and exploring the classical-
quantum boundary with massive objects Delić  _et al._ (2020) due to their weak
coupling to the environment and the ability to cool their centre of mass (CoM)
motion. Intriguing possibilities have been proposed for interacting systems
comprising multiple levitated particles, including quantum gravity
measurements Marletto and Vedral (2017), dark matter detection Moore and
Geraci (2021) and quantum friction measurement Zhao  _et al._ (2012), although
there are currently few experimental demonstrations trapping multiple
particles Arita  _et al._ (2018); Svak  _et al._ (2021); Penny  _et al._
(2021); Rieser  _et al._ (2022).

The path to systems of interacting, massive quantum objects requires
simultaneous cooling of multiple trapped particles. We take inspiration from
experiments with cold neutral atoms Myatt  _et al._ (1997) and atomic ions
Larson  _et al._ (1986), where an actively cooled object can be used to
sympathetically cool another. Recently, sympathetic cooling of two charged
particles held in a Paul trap was shown Penny  _et al._ (2021). While Coulomb
forces provide strong interactions between particles, they also couple
strongly to the environment. Hence, an all-optical alternative is desirable,
especially for compatibility with state-of-the-art ground state cooling
methods Delić  _et al._ (2020).

Figure 1:  Sympathetic cooling scheme and numerical simulations. (a) Optical
binding couples the centre-of-mass motion of two microspheres (depicted as a
spring between the particles). When feedback cooling is applied to the left
particle, the right particle is sympathetically cooled. (b-c) Time evolution
of simulated centre-of-mass temperatures $T_{1}$ (solid lines) and $T_{2}$
(dashed lines) for the feedback-cooled and sympathetically cooled particle
respectively, versus gas pressure for (b) $\xi/\kappa=0.01$ and (c)
$\xi/\kappa=0.1$. (d) Simulated steady-state temperatures $T_{1}$ (blue) and
$T_{2}$ (red) as a function of gas pressure for different binding strengths.

Towards this goal, we have developed an optical tweezers system (Fig. 1(a))
that can trap more than one particle, mediate inter-particle separation and
perform parametric feedback (PFB) cooling Gieseler  _et al._ (2012) on one
particle. Two rotating microparticles confined in separate but close-proximity
traps exhibit optical binding, a light scattering mediated interaction whose
strength is dependent on the inter-particle separation Arita  _et al._ (2018).
We show, for the first time, the use of this optical binding to perform
sympathetic cooling: by applying PFB cooling to one particle, the adjacent
particle is sympathetically cooled to sub-Kelvin temperatures.

We modelled the motion of two particles, optically bound along the x-axis and
labelled $j=1,2$, using a Langevin equation for each particle Arita  _et al._
(2018). The deterministic forces acting on the particles along the $x$-axis
are $F_{j}=-\kappa x_{j}+\xi x_{3-j}$. Here, $\kappa$ is the trap stiffness
for each individual particle, and $\xi$ describes the inter-particle coupling
due to optical binding. The ratio $\xi/\kappa$ (typically $\gtrsim 0.1$ for
vaterite microspheres) characterises the optical binding strength. To the
above system of Langevin equations we have added PFB cooling applied to
particle $j=1$ using the theoretical approach of Ref. Vovrosh  _et al._
(2017). PFB cooling in this case involves time-dependent modulation of the
trap stiffness experienced by particle $j=1$, and is characterised by the
ratio $\eta=\Delta\kappa/\kappa$ (typically $\lesssim 0.05$), $\Delta\kappa$
being the modulation depth.

Figures 1(b,c) show the time evolution of the simulated CoM temperatures for
different binding strengths. PFB cooling decreases $T_{1}$ (solid lines), with
the time to reach steady-state and the final temperature both depending on the
pressure, due to the competition between PFB cooling and reheating from
collisions with gas molecules. We quantify the sympathetic cooling effect
using a measure of the temperature defined here as the ratio of the mean
square displacement relative to that at room temperature. Temperatures below
$1\,\mathrm{K}$ are attainable at $0.001\,\mathrm{mbar}$ gas pressure.
Similarly, the steady-state temperature of the sympathetically cooled
particle, $T_{2}$ (dashed lines) depends on competition between sympathetic
cooling and gas reheating. While the steady-state temperature is lowered for
both optical binding strengths shown, for weak binding (Figure 1(b)) with
$\xi/\kappa=0.01$ the particles do not equilibrate and $T_{2}>T_{1}$ at higher
gas pressure. In contrast, for stronger optical binding (Figure 1(c)) with
$\xi/\kappa=0.1$, $T_{2}\approx T_{1}$ at all pressures below
$1\,\mathrm{mbar}$. Both particles can be cooled to below $1\,\mathrm{K}$ at
$0.001\,\mathrm{mbar}$ gas pressure. Figure 1(d) depicts the steady-state CoM
temperatures $T_{1}$ (blue) and $T_{2}$ (red) for PFB cooled and
sympathetically cooled particles respectively, for different binding strengths
$\xi/\kappa$.

In our experiment, a continuous-wave (CW) laser at $1070\,\mathrm{nm}$ is used
to create two circularly polarized optical traps (formed using a Nikon
100$\times$, NA=1.25 objective, total optical power $25\,\mathrm{mW}$) inside
a vacuum chamber. The intensity of one of these traps is controlled by an
acousto-optical deflector (AOD, IntraAction DTD-274HD6M) to allow PFB cooling.
The position of the second trap, which holds the sympathetically cooled
particle, is controlled by a spatial light modulator (SLM, Hamamatsu LCOS
X10468-03) to set the optical binding strength. Two birefringent vaterite
microspheres ($2.2\,\upmu\mathrm{m}$ in radius) are trapped at
$9.8\,\upmu\mathrm{m}$ separation, where $\xi/\kappa\gtrsim 0.1$. An avalanche
photodiode (APD, Thorlabs APD410C) measures the CoM motion of _only_ the
particle in the AOD trap. The APD signal is processed by a lock-in-amplifier
(Zurich Instruments HF2LI) to modulate the AOD trap intensity by $\lesssim\pm
5\,\%$ to realise PFB cooling of the CoM motion of this particle Gieseler  _et
al._ (2012). The sympathetically cooled particle’s CoM motion is measured by a
photomultiplier tube (PMT, Thorlabs PMM01) using a separate CW beam at
$532\,\mathrm{nm}$.

Figure 2:  (a) Experimental observations of power spectral density of the
sympathetically cooled particle at different gas pressures. These temperatures
are measured while the PFB cooled particle is at $T=105\,\mathrm{K}$,
$9.8\,\mathrm{K}$, $1.1\,\mathrm{K}$ and $0.4\,\mathrm{K}$, respectively, for
each gas pressure. (b) Sympathetic cooling of a microsphere (red asterisks)
via optical binding to the particle that is PFB cooled (blue crosses). The
dashed line is a baseline curve for a single, rotating microsphere in the
absence of PFB cooling. Inset: two trapped vaterite microspheres, where the
scale bar shows $5\,\upmu\mathrm{m}$.

Figure 2(a) shows the power spectral densities (PSD) associated with the CoM
motion of the sympathetically cooled particle. The area under the PSD curve is
proportional to the CoM temperature, which decreases as gas pressure lowers.
Figure 2 (b) shows the experimental results of the two particles’ CoM
temperatures depending on gas pressure: one cooled by PFB control (blue
crosses) and the other sympathetically cooled via optical binding (red
asterisks). We also show the baseline temperature for a single particle
without PFB cooling (black dashed line). CoM temperatures are calibrated at
room temperature with a gas pressure $>10\,\mathrm{mbar}$ without PFB. Both
particles undergo significant cooling as the pressure is reduced below $\sim
0.1\,\mathrm{mbar}$. In this regime, the rotation rate far exceeds the
translational oscillation frequencies, and heating effects associated with
resonant rotational-translational coupling are removed Arita  _et al._ (2013).
Both the PFB cooled particle and the sympathetically cooled particle can be
seen to be cooled by two orders of magnitude compared to the baseline. While
the lowest temperature of $\lesssim 1\,\mathrm{K}$ agrees with the predictions
in Figs. 1(b-d), non-conservative azimuthal forces can have significant
effects on the dynamics of rotating levitated particles Arita  _et al._
(2022), and a more detailed model will be the subject of future work.

In summary, we demonstrate all-optical sympathetic cooling of optically
trapped microspheres to sub-Kelvin motional temperatures. In principle, future
work with lower pressures and detection noise should allow the particles to be
cooled even further. Such simultaneous cooling is a significant step towards
the generation of cooled arrays of optically trapped particles, which will
allow multi-particle studies at the quantum-classical boundary, including
rotational quantum friction Zhao  _et al._ (2012).

The authors acknowledge funding from Engineering and Physical Sciences
Research Council (EP/P030017/1); Australian Research Council (DP220102303);
Czech Science Agency (19-17765S); Ministerstvo Školství, Mládeže a Tělovýchovy
(CZ.02.1.01/0.0/0.0/15_003/0000476).

The underlying data is available at
https://doi.org/10.17630/39f20d0d-74f1-4875-82b5-c2f61b038142.

## References

  * Ranjit  _et al._ (2016) Gambhir Ranjit, Mark Cunningham, Kirsten Casey, and Andrew A. Geraci, “Zeptonewton force sensing with nanospheres in an optical lattice,” Phys. Rev. A 93, 053801 (2016).
  * Delić  _et al._ (2020) Uroš Delić, Manuel Reisenbauer, Kahan Dare, David Grass, Vladan Vuletić, Nikolai Kiesel, and Markus Aspelmeyer, “Cooling of a levitated nanoparticle to the motional quantum ground state,” Science 367, 892–895 (2020).
  * Marletto and Vedral (2017) C. Marletto and V. Vedral, “Gravitationally induced entanglement between two massive particles is sufficient evidence of quantum effects in gravity,” Phys. Rev. Lett. 119, 240402 (2017).
  * Moore and Geraci (2021) David C Moore and Andrew A Geraci, “Searching for new physics using optically levitated sensors,” Quantum Science and Technology 6, 014008 (2021).
  * Zhao  _et al._ (2012) Rongkuo Zhao, Alejandro Manjavacas, F Javier Garcia de Abajo, and JB Pendry, “Rotational quantum friction,” Phys. Rev. Lett. 109, 123604 (2012).
  * Arita  _et al._ (2018) Yoshihiko Arita, Ewan M Wright, and Kishan Dholakia, “Optical binding of two cooled micro-gyroscopes levitated in vacuum,” Optica 5, 910 (2018).
  * Svak  _et al._ (2021) Vojtěch Svak, Jana Flajšmanová, Lukáš Chvátal, Martin Šiler, Alexandr Jonáš, Jan Ježek, Stephen H Simpson, Pavel Zemánek, and Oto Brzobohatỳ, “Stochastic dynamics of optically bound matter levitated in vacuum,” Optica 8, 220–229 (2021).
  * Penny  _et al._ (2021) TW Penny, A Pontin, and PF Barker, “Sympathetic cooling and squeezing of two co-levitated nanoparticles,” arXiv:2111.03123 (2021), 10.48550/arXiv.2111.03123.
  * Rieser  _et al._ (2022) Jakob Rieser, Mario A Ciampini, Henning Rudolph, Nikolai Kiesel, Klaus Hornberger, Benjamin A Stickler, Markus Aspelmeyer, and Uroš Delić, “Tunable light-induced dipole-dipole interaction between optically levitated nanoparticles,” Science 377, 987–990 (2022).
  * Myatt  _et al._ (1997) C. J. Myatt, E. A. Burt, R. W. Ghrist, E. A. Cornell, and C. E. Wieman, “Production of two overlapping bose-einstein condensates by sympathetic cooling,” Phys. Rev. Lett. 78, 586–589 (1997).
  * Larson  _et al._ (1986) DJ Larson, James C Bergquist, John J Bollinger, Wayne M Itano, and David J Wineland, “Sympathetic cooling of trapped ions: A laser-cooled two-species nonneutral ion plasma,” Phys. Rev. Lett. 57, 70 (1986).
  * Gieseler  _et al._ (2012) Jan Gieseler, Bradley Deutsch, Romain Quidant, and Lukas Novotny, “Subkelvin parametric feedback cooling of a laser-trapped nanoparticle,” Phys. Rev. Lett. 109, 103603 (2012).
  * Vovrosh  _et al._ (2017) Jamie Vovrosh, Muddassar Rashid, David Hempston, James Bateman, Mauro Paternostro, and Hendrik Ulbricht, “Parametric feedback cooling of levitated optomechanics in a parabolic mirror trap,” J. Opt. Soc. Am. B 34, 1421–1428 (2017).
  * Arita  _et al._ (2013) Yoshihiko Arita, Michael Mazilu, and Kishan Dholakia, “Laser-induced rotation and cooling of a trapped microgyroscope in vacuum,” Nat. Commun. 4, 2374 (2013).
  * Arita  _et al._ (2022) Yoshihiko Arita, Stephen H Simpson, Graham D Bruce, Ewan M Wright, Pavel Zemánek, and Kishan Dholakia, “Cooling the optical-spin driven limit cycle oscillations of a levitated gyroscope,” arXiv:2204.06925 (2022), 10.48550/arXiv.2204.06925.


0 Research [] [] An outside view of a dense 3D reconstructed scene from the
Gibson Environment [50] dataset using (a) NDF [15], and (b) our proposed
approach. Note that our method produces significantly less outliers.

# Automated Reconstruction of 3D Open Surfaces
from Sparse Point Clouds

Mohammad Samiul Arshad
The University of Texas at Arlington  e-mail:
<EMAIL_ADDRESS>William J. Beksi
The University of Texas at Arlington e-mail<EMAIL_ADDRESS>
###### Abstract

Real-world 3D data may contain intricate details defined by salient surface
gaps. Automated reconstruction of these open surfaces (_e.g_., non-watertight
meshes) is a challenging problem for environment synthesis in mixed reality
applications. Current learning-based implicit techniques can achieve high
fidelity on closed-surface reconstruction. However, their dependence on the
distinction between the inside and outside of a surface makes them incapable
of reconstructing open surfaces. Recently, a new class of implicit functions
have shown promise in reconstructing open surfaces by regressing an unsigned
distance field. Yet, these methods rely on a discretized representation of the
raw data, which loses important surface details and can lead to outliers in
the reconstruction. We propose IPVNet, a learning-based implicit model that
predicts the unsigned distance between a surface and a query point in 3D space
by leveraging both raw point cloud data and its discretized voxel counterpart.
Experiments on synthetic and real-world public datasets demonstrates that
IPVNet outperforms the state of the art while producing far fewer outliers in
the reconstruction.

Automated/semi-automated reconstructionOpen-surface reconstructionUnsigned
distance field

Introduction

Capturing detailed point cloud data from the real world is a difficult and
expensive task. Moreover, due to the limitations of 3D sensor technologies
(_e.g_., LiDAR, RGB-D, etc.), data can be sparse (_i.e_., missing details) and
incomplete (_i.e_., noisy with holes and outliers) [19, 26]. Automated
reconstruction of the missing parts and the reintroduction of surface details
is not a trivial task. Researchers have looked into a myriad of ways [20, 1,
2, 3, 10, 36, 24, 52, 31, 17, 21, 9, 25, 44, 40] to complete 3D data.
Recently, learning-based implicit functions [37, 32, 12, 18, 8, 15, 14] have
become popular among 3D reconstruction techniques due to their ability to
generate data in arbitrary resolutions.

One set of implicit learning techniques [4, 5] makes use of raw point cloud
data to learn a signed distance field. Since traditional convolutions cannot
be applied on permutation invariant point clouds, such methods often depend on
linear feature aggregation through a multilayer perceptron [39] or they define
a dynamic kernel and perform a neighborhood search to mimic convolutions [45].
Other implicit methods [14, 15] discretize the raw point clouds into voxel
grids. However, voxel grids lose information since multiple points within the
boundary of a grid are merged together. Moreover, the computational costs and
memory requirements increase cubically using this approach. Implicit functions
that learn an SDF via extraction of a zero level set must distinguish between
the inside and outside of the surface. As a result, the reconstruction is
produced as a closed surface even if the target shape includes surface gaps
[5]. However, real-world point cloud data can consist of salient open
surfaces. Closing the surface of such data often leads to the introduction of
outliers and lost details.

To reconstruct accurate geometry and preserve surface details, we propose
IPVNet, an implicit model that learns a unsigned distance field (UDF) by
accumulating features from raw point clouds and voxel grids jointly to
reconstruct open surfaces. As shown in Automated Reconstruction of 3D Open
Surfaces  from Sparse Point Clouds, our approach produces significantly less
outliers compared to the state of art [15]. To the best of our knowledge, our
work is the first approach on combining point-voxel features to learn implicit
functions. Our key contributions can be summarized as follows.

  * •

We introduce IPVNet, a novel approach for implicitly learning from raw point
cloud and voxel features to automatically reconstruct complex open surfaces.

  * •

We develop an inference module that extracts a zero level set from a UDF and
drastically lowers the amount of outliers in the reconstruction.

  * •

We show that IPVNet outperforms the state-of-the art on both synthetic and
real-world public datasets.

##  1 Related Work

3D reconstruction is a well researched area with a number of different
approaches and algorithms [20, 1, 2, 3, 10, 36, 24, 52, 31, 21, 9, 25, 44, 40,
17]. In this section, we review and compare our work with learning-based
implicit approaches. For a more comprehensive review, we refer to contemporary
surveys on 3D reconstruction [6, 53].

###  1.1 Implicit Function Learning

Instead of explicitly predicting a surface, implicit feature learning methods
try to either predict if a particular point in 3D space is inside or outside
of a target surface (occupancy), or determine how far the point is from the
target surface (distance). To reconstruct 3D data in arbitrary resolutions and
learn a continuous 3D mapping, Mescheder _et al_. [32] presented a network
that predicts voxel occupancy. Peng _et al_. [38] improved the occupancy
network by incorporating 2D and 3D convolutions. An encoder-decoder
architecture was used by Chen _et al_. [12] to learn voxel occupancy.
Michalkiewicz _et al_. [33] estimates oriented level set to extract 3D
surface. Littwin and Wolf [29] used encoded feature vectors as the network
weights to predict voxel occupancy. Park _et al_. [37] introduced DeepSDF, an
encoder-decoder based architecture that predicts a signed distance to the
surface instead of voxel occupancy. Genova _et al_. [18] divided an object’s
surface into a set of shape elements and used an encoder-decoder to learn
occupancy. Chibane _et al_. [14] used 3D feature tensors to predict voxel
occupancy. Rather than transforming point clouds into a occupancy grid, Atzmon
and Lipman used the raw point clouds to learn and predict an SDF to the target
surface in [4], and incorporated derivatives in regression loss to further
improve the reconstruction accuracy in [5].

All of the aforementioned works either predict a voxel occupancy or signed
distance value for a given query point, which is inadequate to reconstruct
open surfaces. To address this problem, we task IPVNet to learn an unsigned
distance field. Prior to our work, Chibane _et al_. [15] predicted a UDF from
an input voxel occupancy map. A similar technique to learn a UDF for single-
view garment reconstruction was used by Zhao _et al_. [55]. Venkatesh _et al_.
[47] proposed a closest surface point representation to reconstruct both open
and close surfaces. However, preceding work on UDFs often discretize the raw
point cloud data into voxel grids, which results in lost surface details. In
contrast, we make use of the raw point cloud jointly with voxel occupancy,
thus enabling us to accumulate improved features and reconstruct finer details
with less outliers.

###  1.2 Learning from Points and Voxels

Recently, fusion between features extracted from point cloud and voxel
representations has shown to improve the performance of 3D computer vision
methods. Liu _et al_. [30] introduced PVCNN to perform classification and
segmentation by extracting features from both point clouds and voxel grids via
voxelization and de-voxelization. Fusion between voxel and point features for
3D classification was used by Li _et al_. [27]. Shi _et al_. [41] gathered
multi-scale voxel features which were combined into keypoint features from a
point cloud for object detection and further improved the results by
incorporating local vector pooling in [42]. Using PVCNN as a backbone, Zhu _et
al_. [56] performed shape completion and generation through point-voxel
diffusion. Point-voxel fusion to detect 3D objects was used by Cui _et al_.
[16] and Tang _et al_. [43] learned a 3D model via sparse point-voxel
convolution.

Noh _et al_. [35] accumulated point-voxel features in a single representation
for 3D object detection. PVT, a transformer-based architecture that learns
from point-voxel features for point cloud segmentation was introduced by Zhang
_et al_. [54]. Wei _et al_. [48] used point-voxel correlation for scene flow
estimation and Li _et al_. [28] used point-voxel convolution for 3D object
detection. Xu _et al_. [51] introduced RPVNet for point cloud segmentation via
point-voxel fusion. Cherenkova _et al_. [13] utilized point-voxel
deconvolution for point cloud encoding/decoding. In contrast to the preceding
research, we focus on applying point-voxel fusion to the task of learning
implicit functions and open-surface reconstruction. To the best of our
knowledge, our approach is the first attempt to understand the effectiveness
of point-voxel fusion on these tasks.

##  2 Implicit Learning with Point-Voxel Features

Figure 1: Given a sparse point cloud $x\in\mathcal{X}$ of an object, we use a
novel encoding scheme to extract and aggregate point-voxel features from both
the raw point cloud $(x)$ and the voxel occupancy $(v)$. From the accumulated
features, a decoder module regresses the unsigned distance $UD(p,S)$ from
query point $p$ to the surface $S$. By querying the decoder multiple times,
the inference sub-module can reconstruct the surface of any target shape.

An overview of our network is presented in Fig. 1. Given a sparse point cloud
$x\in\mathcal{X}\subset\mathbb{R}^{N\times 3}$ of an object, we use a novel
encoding scheme to extract and aggregate point-voxel features from both the
raw point cloud $(x)$ and the voxel occupancy $(v)$. From the accumulated
features, a decoder module regresses the unsigned distance $UD(p,S)$ from any
query point $p$ to the surface $S$. In the following subsections we describe
the elements of our approach.

###  2.1 Point-Voxel Features

To extract a set of multilevel features from a point cloud $x$, we define a
neural function

$\Theta(x):=(z_{x}^{1},\ldots,z_{x}^{j})\>|\>\Theta\colon\mathbb{R}^{N\times
3}\rightarrow\mathbb{Z},$ (1)

where $z_{x}\in\mathbb{Z}\subset\mathbb{R}$ corresponds to the extracted
feature vector from the raw point cloud $x$, and $j$ is the total number of
layers in $\Theta$. ReLu [34] non-linearity is used for all layers except the
output layer of the point encoder.

Instead of limiting the encoded features to a single dimensional vector, a
voxel representation allows for the construction of a multidimensional latent
matrix. However, such an encoding scheme requires the input point cloud $x$ to
be discretized into a voxel grid $v$, _i.e_., $x\approx
v\colon\mathbb{R}^{N\times 3}\rightarrow\mathbb{R}^{M\times M\times M}$ where
$M\in\mathbb{N}$ is the grid resolution. Due to the discretization process,
voxel grids lose information since multiple points may lie within the same
voxel. To reintroduce lost details, we combine voxel features with point
features $z_{x}$.

Let $\Phi\colon\mathbb{R}^{M\times M\times M}\rightarrow\mathbb{Z}^{M\times
M\times M}$ be a neural function that encodes the combined point-voxel
features into a set of multidimensional feature grid $z_{xv}$ of monotonically
decreasing dimension. Then,

$\Phi(v\odot\Theta(x)):=(z_{xv}^{k\times k\times k},\ldots,z_{xv}^{l\times
l\times l}),$ (2)

where $k,l\in\mathbb{N}$ represents the dimensional upper and lower bound of
the feature grid $(M>k\gg l>1)$, the subscript ${xv}$ denotes the dependency
on both points and voxels. Similar to its point counterpart, the voxel encoder
is more directed towards local details at the early stages. However, as the
dimensionality is reduced and the receptive field grows larger, the aim shifts
to the global structure. ReLu is utilized to ensure non-linearity and batch
normalization [22] provides stability while training. The latent point
($z_{x}^{j}$) from the point encoder, along with multidimensional features
($z_{xv}$) from the point-voxel encoder and the discretized voxel grid ($v$),
are then used to construct the latent point-voxel

$z=\\{z_{x}^{j},\Phi(v\odot\Theta(x)),v\\}.$ (3)

###  2.2 Implicit Decoding

Given a query point $p\in\mathbb{R}^{3}$, a set of deep features $F_{p}$ is
sampled from the latent point-voxel features $z$ via a spatial grid sampling
[23] function $\Omega$. Specifically,

$\Omega(z,p):=(F_{p}^{1}\times\dots\times F_{p}^{n}),$ (4)

where $n=\lvert z\rvert$. Similar to [14], we extract features from a
neighborhood of distance $d\in\mathbb{R}$ along the Cartesian axes centered at
$p$ to obtain rich features. More formally,

$p:=\\{p+q\cdot c_{i}\cdot
d\\}\in\mathbb{R}^{3}\;|\;q\in\\{1,0,-1\\},i\in\\{1,2,3\\},$ (5)

where $c_{i}\in\mathbb{R}^{3}$ is the $ith$ Cartesian axis unit vector. We
define a neural function $\Psi$ that regresses the unsigned distance to the
surface $S$ of $x$ from the deep features ($F_{p}$). Concretely,

$\Psi(F_{p}^{1},\dots,F_{p}^{n})\approxeq
UD(p,S)\>|\>\Psi\colon\mathbb{Z}\rightarrow\mathbb{R}_{+},$ (6)

where $UD(\cdot)$ is a function that returns the unsigned distance from $p$ to
the ground-truth surface $S$ for any $p\in\mathbb{R}^{3}$. Therefore, the
implicit decoder to regress the unsigned distance at a given query point $p$
is defined as

$f_{x}(z,p):=(\Omega\circ\Psi)(p)\>|\>f_{x}\colon\mathbb{Z}\times\mathbb{R}^{3}\rightarrow\mathbb{R}_{+}.$
(7)

###  2.3 Training

IPVNet requires a pair $\\{X_{i},S_{i}\\}^{T}_{i=1}$ associated with input
$X_{i}$ and corresponding ground-truth surface $S_{i}$ for implicit learning.
Parameterized by the neural parameter $w$, the point-encoder, voxel-encoder,
and decoder are jointly trained with a mini-batch loss

$\mathtt{L}_{\mathtt{B}}:=\Sigma_{x\in
B}\Sigma_{p\in\mathtt{P}}|\min(f_{x}^{w}(p),\delta)-\min(UD(p,S_{x}),\delta)|,$
(8)

where $\mathtt{B}$ is a mini-batch of input and
$\mathtt{P}\in\mathbb{R}^{Q\times 3}$ is a set of query points within distance
$\delta$ of $S_{i}$. Similar to [15], we use a clamped distance $0<\delta<10$
(cm) to improve the models capacity to represent the vicinity of the surface
accurately.

###  2.4 Surface Inference

Analogous to [15], we use an iterative strategy to extract surface points from
$f_{x}$. More specifically, given a perfect approximator $f_{x}(p)$ of the
true unsigned distance $UD(p,S_{i})$, the projection of $p$ onto the surface
$S_{i}$ can be obtained by

$q:=p-f_{x}(p)\cdot\nabla_{p}f_{x}(p),q\in S_{i}\subset\mathbb{R}^{d},\forall
p\in\mathbb{R}^{d}/C.$ (9)

In (9), $C$ is the cut locus [49], _i.e_., a set of points that are
equidistant to at least two surface points. The negative gradient indicates
the direction of the fastest decrease in distance. In addition, we can move a
distance of $f_{x}(p)$ to reach $q$ if the norm of the gradient is one. By
projecting a point multiple times via (9), the inaccuracies due to $f_{x}(p)$
being an imperfect approximator can be reduced. Furthermore, filtering the
projected points to a maximum distance threshold ($max\\_thresh$) and re-
projecting them onto the surface after displacement by
$d\sim\mathcal{N}(0,\delta/3)$ can ensure higher point density within a
maximum distance ($\delta$).

Instead of uniformly sampling query points within the bounding box of $S_{i}$
[15], we use the input points $X_{i}\in\mathbb{R}^{3}$ as guidance for the
query points. In particular, we apply a random uniform jitter
$\mathtt{J}_{a}^{b}\in\mathbb{R}^{3}$ within bounds $a$ and $b$ to displace
the input points $X_{i}$. Due to the inclusion of point features in learning,
this procedure allows our model to infer more accurate surface points while
restricting the number of outliers. The details of the inference procedure are
provided in Alg. 1.

Algorithm 1 Surface Point Inference

1:procedure Inference($\mathtt{X}$)

2: $np$ $\leftarrow$ Total number of projections

3: $T$ $\leftarrow$ Maximum threshold distance

4: $R$ $\leftarrow$ Output resolution of the point cloud

5: $\mathtt{J}\leftarrow m$ points from $\mathcal{U}(a,b)$

6: $\mathtt{P}_{init}\leftarrow\\{x+j\\},\forall x\in\mathtt{X},\forall
j\in\mathtt{J}$

7: for $i=1$ to $np$ do

8: $p\leftarrow
p-f_{x}(p)\cdot\frac{\nabla_{p}f_{x}(p)}{||\nabla_{p}f_{x}(p)||},\forall
p\in\mathtt{P}_{init}$

9: end for

10:
$\mathtt{P}_{filtered}\leftarrow\\{p\in\mathtt{P}_{init}\,|\,f_{x}(p)<T\\}$

11: $\mathtt{P}_{filtered}$: draw $R$ points with replacement

12:
$\mathtt{P}_{filtered}\leftarrow\\{p+d\\}\,|\,p\in\mathtt{P}_{filtered},d\sim\mathcal{N}(0,\delta/3)$

13: for $i=1$ to $np$ do

14: $p\leftarrow
p-f_{x}(p)\cdot\frac{\nabla_{p}f_{x}(p)}{||\nabla_{p}f_{x}(p)||},\forall
p\in\mathtt{P}_{filtered}$

15: end for

16: return $\\{p\in\mathtt{P}_{filtered}\,|\,f_{x}(p)<T\\}$

17:end procedure

##  3 Experiments

To validate the performance of IPVNet we concentrate on the task of 3D object
and scene reconstruction from sparse point clouds. In this section, we present
the details of the experimental setup and provide an analysis of our results.

(a) Input (b) NDF (c) IPVNet (d) IPVNet (Inner View) (e) GT (f)  (g)  (h)  (i)
(j)  (k)  (l)  (m)  (n)  (o)

Figure 2: Object reconstruction using NDF [15], IPVNet, and the ground truth
(GT) from the ShapeNet Cars [11] test set. IPVNet performs better on
reconstructing thin structures and preserving small gaps (inset images).

###  3.1 Baseline and Metric

We utilize a neural distance field (NDF) [15] as the baseline to compare the
reconstruction quality of IPVNet. To the best of our knowledge, NDF [15] and
CSP [47] are the only approaches available for open surface reconstruction.
However, due to the unavailability of the codebase for multi-shape learning
via CSP, we compare our results only against NDF. Note that, to be fair, we do
not compare against methods that are limited to closed surface reconstruction
as our main objective is the reconstruction of open surfaces. For an unbiased
comparison, we trained an NDF following the directions from [15] on our train-
test split until a minimum validation accuracy was achieved.

To quantitatively measure the reconstruction quality, we use the
chamfer-$L_{2}$ distance (CD) to measure the accuracy and completeness of the
surface. The CD is defined as

$d_{CD}(Y,Y_{gt})=\sum_{i\in Y}\min_{j\in Y_{gt}}{||i-j||}^{2}+\sum_{j\in
Y_{gt}}\min_{i\in Y}{||j-i||}^{2},$ (10)

where $Y_{gt}\in\mathbb{R}^{\mathcal{O}\times 3}$ is the ground-truth point
cloud, $Y\in\mathbb{R}^{\mathcal{O}\times 3}$ is the reconstructed point
cloud, and $\mathcal{O}\in\mathbb{N}$ is the point density of the ground truth
and the output. In addition, precision and recall are two metrics that have
been extensively used to evaluate 3D reconstruction results. Precision
quantifies the accuracy while recall assesses the completeness of the
reconstruction. For the ground truth $Y_{gt}$ and reconstructed point cloud
$Y$, the precision of an outcome at a threshold $d$ can be calculated as

$P(d)=\sum_{i\in Y}[\min_{j\in Y_{gt}}||i-j||<d].$

Similarly, the recall for a given $d$ may be computed as

$R(d)=\sum_{j\in Y_{gt}}[\min_{i\in Y}||j-i||<d].$

The F-score, proposed in [46] as a comprehensive evaluation, combines
precision and recall to quantify the overall reconstruction quality. In
detail, the F-score at $d$ is given by

$F(d)=\frac{2\cdot P(d)\cdot R(d)}{P(d)+R(d)}.$

An F-score of 1 indicates perfect reconstruction.

###  3.2 Object Reconstruction

Due to the abundance of surface openings, we have chosen the “Cars” subset
from the ShapeNet [11] dataset for our object reconstruction experiment. We
used a random split of $70\%$-$10\%$-$20\%$ for training, validation, and
testing, respectively. To prepare the ground truth and input points we
followed the data preparation procedure outlined in [15]. Additionally, we
fixed the output point density $\mathcal{O}=1$ million to extract a smooth
mesh from the point cloud using a naive algorithm (_e.g_., [7]).

To understand the effects of sparse input on the reconstruction quality, we
evaluated IPVNet and the baseline using an input density of
$N\sim\\{300,3000,10000\\}$ points while fixing the voxel resolution to
$M=256$. In contrast to the baseline, IPVNet can reconstruct thin structures
more accurately and preserve small gaps (see the inset images in Fig. 2) while
quantitatively outperforming the reconstruction with different input densities
(Table 1).

| Chamfer-$L_{2}\downarrow$ |  $F$-score $\uparrow$
---|---|---
|  $N=300$ |  $N=3000$ |  $N=10000$ |  $d=0.1\%$ |  $d=0.05\%$
NDF |  $1.550$ |  $0.324$ |  $0.092$ |  $0.711$ |  $0.460$
IPVNet |  $\mathbf{1.217}$ |  $\mathbf{0.119}$ |  $\mathbf{0.068}$ |  $\mathbf{0.785}$ |  $\mathbf{0.542}$
Table 1: A quantitative comparison between IPVNet and the baseline (NDF [15])
on the ShapeNet Cars [11] dataset for object reconstruction from different
input densities. IPVNet outperforms the baseline on all input densities. The
chamfer-$L_{2}$ results are of order $\times 10^{-4}$ and the reconstruction
results using an input density of $N=10000$ were used to calculate the
$F$-score.

###  3.3 Real-World Scene Reconstruction

We evaluate the reconstruction of complex real-world scenes through the use of
the Gibson Environment dataset [50]. The dataset consists of RDG-D scans of
indoor spaces. A subset of 35 and 100 scenes were prepared following the
procedure from [15] for training and testing respectively. We used a sliding
window scheme and reconstructed the surface bounded by each window. Since the
sliding window may frequently consist of a very small area of the scene with
only few points, we used an output density five times as large the input
density (_i.e_., $\mathcal{O}=5\times N$) to save time. The grid resolutions
were kept fixed at $M=256$ for both IPVNet and the baseline. The
reconstruction results are highlighted in Fig. 3. In addition to improving the
preservation of structural details, IPVNet produces significantly fewer
outliers than the baseline due to the use of point features during training
and inference.

(a) Input

(b) NDF

(c) IPVNet

(d) GT

(e)

(f)

(g)  (h)

(i)

(j)

(k)

(l)

(m)

(n)  (o)

(p)

Figure 3: Scene reconstruction on the test set of the Gibson Environment [50]
dataset using NDF [15], IPVNet, and the respective ground truth (GT). Each odd
row represents an outside view of a scene while the even rows depict inside
views. In contrast to the baseline, IPVNet produces significantly less
outliers (outside view) and improves the preservation of geometric features
(inset images).

##  4 Conclusion

In this work we have introduced IPVNet, a novel architecture for automatically
reconstructing complex open surfaces. To do this, we make use of raw point
cloud data jointly with voxels to learn local and global features. Not only
have we showed that IPVNet outperforms the state of the art on both synthetic
and real-world data, but we also demonstrated the effectiveness of point
features on 3D reconstruction through ablation studies. Furthermore, we
developed an inference module that extracts a zero level set from a UDF and
drastically reduces the amount of outliers in the reconstruction. IPVNet is an
important step towards reconstructing open surfaces without losing details and
introducing outliers. We believe that our work will inspire more research on
this topic.

## References

  * [1] N. Amenta, M. Bern, and M. Kamvysselis.  A new voronoi-based surface reconstruction algorithm.  In Proceedings of the Conference on Computer Graphics and Interactive Techniques, pp. 415–421, 1998.
  * [2] N. Amenta, S. Choi, T. K. Dey, and N. Leekha.  A simple algorithm for homeomorphic surface reconstruction.  In Proceedings of the Symposium on Computational Geometry, pp. 213–222, 2000.
  * [3] N. Amenta, S. Choi, and R. K. Kolluri.  The power crust.  In Proceedings of the ACM Symposium on Solid Modeling and Applications, pp. 249–266, 2001.
  * [4] M. Atzmon and Y. Lipman.  Sal: Sign agnostic learning of shapes from raw data.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2565–2574, 2020.
  * [5] M. Atzmon and Y. Lipman.  Sald: Sign agnostic learning with derivatives.  arXiv preprint arXiv:2006.05400, 2020.
  * [6] M. Berger, A. Tagliasacchi, L. M. Seversky, P. Alliez, G. Guennebaud, J. A. Levine, A. Sharf, and C. T. Silva.  A survey of surface reconstruction from point clouds.  Computer Graphics Forum, 36(1):301–329, 2017.
  * [7] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin.  The ball-pivoting algorithm for surface reconstruction.  IEEE Transactions on Visualization and Computer Graphics, 5(4):349–359, 1999.
  * [8] B. L. Bhatnagar, C. Sminchisescu, C. Theobalt, and G. Pons-Moll.  Combining implicit function learning and parametric models for 3d human reconstruction.  In Proceedings of the European Conference on Computer Vision, pp. 311–329. Springer, 2020.
  * [9] F. Calakli and G. Taubin.  Ssd: Smooth signed distance surface reconstruction.  Computer Graphics Forum, 30(7):1993–2002, 2011.
  * [10] J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J. Mitchell, W. R. Fright, B. C. McCallum, and T. R. Evans.  Reconstruction and representation of 3d objects with radial basis functions.  In Proceedings of the Conference on Computer Graphics and Interactive Techniques, pp. 67–76, 2001.
  * [11] A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, et al.  Shapenet: An information-rich 3d model repository.  arXiv preprint arXiv:1512.03012, 2015.
  * [12] Z. Chen and H. Zhang.  Learning implicit fields for generative shape modeling.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939–5948, 2019.
  * [13] K. Cherenkova, D. Aouada, and G. Gusev.  Pvdeconv: Point-voxel deconvolution for autoencoding cad construction in 3d.  In Proceedings of the IEEE International Conference on Image Processing, pp. 2741–2745, 2020.
  * [14] J. Chibane, T. Alldieck, and G. Pons-Moll.  Implicit functions in feature space for 3d shape reconstruction and completion.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6970–6981, 2020.
  * [15] J. Chibane, A. Mir, and G. Pons-Moll.  Neural unsigned distance fields for implicit function learning.  arXiv preprint arXiv:2010.13938, 2020.
  * [16] Z. Cui and Z. Zhang.  Pvf-net: Point & voxel fusion 3d object detection framework for point cloud.  In Proceedings of the Conference on Computer and Robot Vision, pp. 125–133. IEEE, 2020.
  * [17] A. Gálvez, A. Cobo, J. Puig-Pey, and A. Iglesias.  Particle swarm optimization for bézier surface reconstruction.  In Proceedings of the International Conference on Computational Science, pp. 116–125. Springer, 2008.
  * [18] K. Genova, F. Cole, A. Sud, A. Sarna, and T. Funkhouser.  Local deep implicit functions for 3d shape.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4857–4866, 2020.
  * [19] P. Guerrero, Y. Kleiman, M. Ovsjanikov, and N. J. Mitra.  Pcpnet learning local shape properties from raw point clouds.  Computer Graphics Forum, 37(2):75–85, 2018.
  * [20] H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle.  Surface reconstruction from unorganized points.  In Proceedings of the Conference on Computer Graphics and Interactive Techniques, pp. 71–78, 1992.
  * [21] H. Huang, D. Li, H. Zhang, U. Ascher, and D. Cohen-Or.  Consolidation of unorganized point clouds for surface reconstruction.  ACM Transactions on Graphics, 28(5):1–7, 2009.
  * [22] S. Ioffe and C. Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In Proceedings of the International Conference on Machine Learning, pp. 448–456, 2015.
  * [23] M. Jaderberg, K. Simonyan, A. Zisserman, et al.  Spatial transformer networks.  In Proceedings of the Advances in Neural Information Processing Systems, vol. 28, 2015.
  * [24] M. Kazhdan, M. Bolitho, and H. Hoppe.  Poisson surface reconstruction.  In Proceedings of the Eurographics Symposium on Geometry Processing, vol. 7, 2006.
  * [25] M. Kazhdan and H. Hoppe.  Screened poisson surface reconstruction.  ACM Transactions on Graphics, 32(3):1–13, 2013.
  * [26] R. Li, X. Li, C.-W. Fu, D. Cohen-Or, and P.-A. Heng.  Pu-gan: A point cloud upsampling adversarial network.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7203–7212, 2019.
  * [27] Y. Li, G. Tong, X. Li, L. Zhang, and H. Peng.  Mvf-cnn: Fusion of multilevel features for large-scale point cloud classification.  IEEE Access, 7:46522–46537, 2019.
  * [28] Y. Li, S. Yang, Y. Zheng, and H. Lu.  Improved point-voxel region convolutional neural network: 3d object detectors for autonomous driving.  IEEE Transactions on Intelligent Transportation Systems, 2021.
  * [29] G. Littwin and L. Wolf.  Deep meta functionals for shape representation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1824–1833, 2019.
  * [30] Z. Liu, H. Tang, Y. Lin, and S. Han.  Point-voxel cnn for efficient 3d deep learning.  arXiv preprint arXiv:1907.03739, 2019.
  * [31] J. Manson, G. Petrova, and S. Schaefer.  Streaming surface reconstruction using wavelets.  Computer Graphics Forum, 27(5):1411–1420, 2008.
  * [32] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger.  Occupancy networks: Learning 3d reconstruction in function space.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4460–4470, 2019.
  * [33] M. Michalkiewicz, J. K. Pontes, D. Jack, M. Baktashmotlagh, and A. Eriksson.  Deep level sets: Implicit surface representations for 3d shape inference.  arXiv preprint arXiv:1901.06802, 2019.
  * [34] V. Nair and G. E. Hinton.  Rectified linear units improve restricted boltzmann machines.  In Proceedings of the International Conference on Machine Learning, 2010.
  * [35] J. Noh, S. Lee, and B. Ham.  Hvpr: Hybrid voxel-point representation for single-stage 3d object detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14605–14614, 2021.
  * [36] Y. Ohtake, A. Belyaev, M. Alexa, G. Turk, and H.-P. Seidel.  Multi-level partition of unity implicits.  In ACM SIGGRAPH 2005 Courses, pp. 173–es. 2005.
  * [37] J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove.  Deepsdf: Learning continuous signed distance functions for shape representation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165–174, 2019.
  * [38] S. Peng, M. Niemeyer, L. Mescheder, M. Pollefeys, and A. Geiger.  Convolutional occupancy networks.  In Proceedings of the European Conference on Computer Vision, pp. 523–540. Springer, 2020.
  * [39] C. R. Qi, H. Su, K. Mo, and L. J. Guibas.  Pointnet: Deep learning on point sets for 3d classification and segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 652–660, 2017.
  * [40] X. Ren, L. Lyu, X. He, W. Cao, Z. Yang, B. Sheng, Y. Zhang, and E. Wu.  Biorthogonal wavelet surface reconstruction using partial integrations.  Computer Graphics Forum, 37(7):13–24, 2018.
  * [41] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li.  Pv-rcnn: Point-voxel feature set abstraction for 3d object detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10529–10538, 2020.
  * [42] S. Shi, L. Jiang, J. Deng, Z. Wang, C. Guo, J. Shi, X. Wang, and H. Li.  Pv-rcnn++: Point-voxel feature set abstraction with local vector representation for 3d object detection.  arXiv preprint arXiv:2102.00463, 2021.
  * [43] H. Tang, Z. Liu, S. Zhao, Y. Lin, J. Lin, H. Wang, and S. Han.  Searching efficient 3d architectures with sparse point-voxel convolution.  In Proceedings of the European Conference on Computer Vision, pp. 685–702. Springer, 2020.
  * [44] Y. Tang and J. Feng.  Multi-scale surface reconstruction based on a curvature-adaptive signed distance field.  Computers & Graphics, 70:28–38, 2018.
  * [45] M. Tatarchenko, A. Dosovitskiy, and T. Brox.  Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2088–2096, 2017.
  * [46] M. Tatarchenko, S. R. Richter, R. Ranftl, Z. Li, V. Koltun, and T. Brox.  What do single-view 3d reconstruction networks learn?  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3405–3414, 2019.
  * [47] R. Venkatesh, T. Karmali, S. Sharma, A. Ghosh, R. V. Babu, L. A. Jeni, and M. Singh.  Deep implicit surface point prediction networks.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12653–12662, 2021.
  * [48] Y. Wei, Z. Wang, Y. Rao, J. Lu, and J. Zhou.  Pv-raft: point-voxel correlation fields for scene flow estimation of point clouds.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6954–6963, 2021.
  * [49] F.-E. Wolter.  Cut locus and medial axis in global shape interrogation and representation, design laboratory memorandum 92-2, 1993.
  * [50] F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese.  Gibson env: Real-world perception for embodied agents.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9068–9079, 2018.
  * [51] J. Xu, R. Zhang, J. Dou, Y. Zhu, J. Sun, and S. Pu.  Rpvnet: A deep and efficient range-point-voxel fusion network for lidar point cloud segmentation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16024–16033, 2021.
  * [52] J. Yang, Z. Wang, C. Zhu, and Q. Peng.  Implicit surface reconstruction with radial basis functions.  In Proceedings of the International Conference on Computer Vision and Computer Graphics, pp. 5–12. Springer, 2007.
  * [53] C. C. You, S. P. Lim, S. C. Lim, J. San Tan, C. K. Lee, and Y. M. J. Khaw.  A survey on surface reconstruction techniques for structured and unstructured data.  In Proceedings of the IEEE Conference on Open Systems, pp. 37–42, 2020.
  * [54] C. Zhang, H. Wan, S. Liu, X. Shen, and Z. Wu.  Pvt: Point-voxel transformer for 3d deep learning.  arXiv preprint arXiv:2108.06076, 2021.
  * [55] F. Zhao, W. Wang, S. Liao, and L. Shao.  Learning anchored unsigned distance functions with gradient direction alignment for single-view garment reconstruction.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12674–12683, 2021.
  * [56] Q. Zhu, L. Chen, Q. Li, M. Li, A. Nüchter, and J. Wang.  3d lidar point cloud based intersection recognition for autonomous driving.  In Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 456–461, 2012.