# Expensive Multi-Objective Bayesian Optimization Based on Diffusion Models

Bingdong Li  Zixiang Di  Yongfan Lu  Hong Qian  Feng Wang  Peng Yang  Ke Tang
Aimin Zhou

###### Abstract

Multi-objective Bayesian optimization (MOBO) has shown promising performance
on various expensive multi-objective optimization problems (EMOPs). However,
effectively modeling complex distributions of the Pareto optimal solutions is
difficult with limited function evaluations. Existing Pareto set learning
algorithms may exhibit considerable instability in such expensive scenarios,
leading to significant deviations between the obtained solution set and the
Pareto set (PS). In this paper, we propose a novel Composite Diffusion Model
based Pareto Set Learning algorithm, namely CDM-PSL, for expensive MOBO. CDM-
PSL includes both unconditional and conditional diffusion model for generating
high-quality samples. Besides, we introduce an information entropy based
weighting method to balance different objectives of EMOPs. This method is
integrated with the guiding strategy, ensuring that all the objectives are
appropriately balanced and given due consideration during the optimization
process; Extensive experimental results on both synthetic benchmarks and real-
world problems demonstrates that our proposed algorithm attains superior
performance compared with various state-of-the-art MOBO algorithms.

Machine Learning, ICML

##  1 Introduction

Expensive multi-objective optimization problems are commonly seen various
fields, such as neural architecture search (Zoph et al., 2018; Lu et al.,
2019), antenna structure design (Ding et al., 2019), and clinical drug trials
(Yu et al., 2019). Handling EMOPs involves optimizing multiple (often
conflicting) objectives simultaneously with the limit number of function
evaluations due to time and financial constraints.

To meet these challenges, multi-objective Bayesian optimization (MOBO)
(Laumanns & Ocenasek, 2002), an extension of single-objective Bayesian
Optimization (BO) (Močkus, 1975) for expensive multi-objective optimization
problems, has emerged as a promising paradigm. BO itself is recognized as an
exceedingly effective strategy for global optimization, particularly noted for
its success in addressing black-box optimization issues (Jones et al., 1998;
Snoek et al., 2012). The core principle of BO involves creating probabilistic
surrogate models that closely represent the black-box functions. These models
are utilized in conjunction with acquisition functions to seek out globally
optimal solutions. MOBO represents a fusion of Bayesian optimization with
multi-objective optimization. A widely adopted MOBO approach is the random
scalarization technique, which effectively translates a multi-objective
optimization problem into several single-objective optimization problems.
Another noteworthy strategy in MOBO involves the use of sophisticated
acquisition functions, such as the expected hypervolume improvement (EHVI)
(Couckuyt et al., 2014) and predictive entropy search (PES) (Hoffman &
Ghahramani, 2015). Among them, Pareto set learning (PSL) based methods (e.g.
(Lin et al., 2022)) which aims to modeling the Pareto set via machine learning
techniques, have shown promising performance.

However, effectively capturing and modeling complex distributions of limited
samples is difficult when faced with expensive multi-objective Bayesian
optimization problems (EMOBOPs). Existing PSL algorithms may exhibit
considerable instability in expensive scenarios. This instability can lead to
significant deviations between the obtained solution set and the true Pareto
set (PS). In other words, the quality of the resulting solution set is highly
influenced by the performance of the PSL model, which is largely limited on
EMOBOPs.

Diffusion model (DM), inspired by the natural diffusion of gases, is a kind of
popular deep generative models. The characteristics of DMs, including their
distribution coverage, stationary training objective, and effortless
scalability (Dhariwal & Nichol, 2021a), have empowered them to not only
outperform Generative Adversarial Networks (GANs) in image synthesis tasks
(Dhariwal & Nichol, 2021a) but also achieve great success in diverse fields
such as computer vision (Baranchuk et al., 2021; Yu et al., 2022), natural
language processing (Austin et al., 2021a), and waveform signal processing
(Leng et al., 2022). These features of DM show promise for its application in
Pareto set learning in EMOBOPs.

In this paper, we propose a novel Composite Diffusion Model based Pareto Set
Learning algorithm, namely CDM-PSL for expensive MOBO. We introduce the
Diffusion Model into Pareto Set Learning, where DM operates by simulating the
transition of data from an ordered state to disordered noise. This process
progressively learns and reveals the inherent distribution within the data.
This methodology is particularly effective in scenarios with limited sample
sizes, as it can efficiently extract substantial information from each
individual sample, thereby amplifying the learning impact (Yang et al., 2023).
This enables the effective modeling of complex distributions of high-quality
samples. Building upon this foundation, to generate samples of superior
quality, we designed a guided sampling process. This approach ultimately led
to the realization of conditional sample generation.

The major contributions of this paper are summarized as follows: 1) We
introduce a composite diffusion model based Pareto set learning method for
offspring generation for expensive MOBO, which includes both unconditional and
conditional sample generation. 2) We devise a guided sampling process to
improve the quality of solutions generated by the diffusion model, resulting
in a conditional diffusion model; 3) We introduce an information entropy based
weighting method to balance the importance of different objectives of multi-
objective problems. This method is integrated with the guiding strategy,
ensuring that all the objectives are appropriately balanced and given due
consideration during the optimization process; 4) We have conducted extensive
experiments on both synthetic benchmarks and real-world problems, clearly
demonstrating that CDM-PSL obtains superior performance compared with various
state-of-the-art MOBO algorithms.

##  2 Preliminaries

###  2.1 Expensive Multi-objective Optimization

A multi-objective optimization problem (MOP) can be universally expressed in
mathematical terms as follows:

$\displaystyle\operatorname{minimize}\textbf{
}\boldsymbol{f}(\boldsymbol{x})=(f_{1}(\boldsymbol{x}),f_{2}(\boldsymbol{x}),\ldots,f_{M}(\boldsymbol{x}))^{T}$
$\displaystyle subject\textrm{ }to\textrm{ }\boldsymbol{x}\in\Omega$ (1)

where $\boldsymbol{x}$ $=$ $(x_{1},x_{2},\ldots,x_{d})$ represents the
decision vector, $\boldsymbol{f}(\cdot)$: $\Omega\rightarrow\Lambda$ denotes a
black-box objective function, encompassing $M$ ($M\geq 2$) objectives,
$\Omega$ symbolizes the non-empty decision space, and $\Lambda$ is the
objective space. An MOP is considered expensive when the evaluation of
$\boldsymbol{f(x)}$ involves either time-intensive computations or high-cost
experimental procedures. In such contexts, the primary aim in optimizing an
MOP is to approximate the Pareto Front (PF) effectively within a limited
evaluation budget.

######  Definition 2.1 (Pareto dominance).

Considering two solutions $\boldsymbol{x}$ and $\boldsymbol{y}$ $\in$
$\Omega$, $\boldsymbol{x}$ is said to dominate $\boldsymbol{y}$ (expressed as
$\boldsymbol{x}\prec\boldsymbol{y}$) if and only if the following conditions
are met: 1) $\forall i\in\\{1,2,...,M\\}$, $f_{i}(\boldsymbol{x})\leq
f_{i}(\boldsymbol{y})$; 2) $\exists j\in\\{1,2,...,M\\}$,
$f_{j}(\boldsymbol{x})<f_{j}(\boldsymbol{y})$. This definition encapsulates
the essential criterion for determining the quality of solutions in terms of
meeting multiple objectives simultaneously. (Yu, 1974)

######  Definition 2.2 (Pareto optimal).

A solution $\boldsymbol{x}^{\ast}\in\Omega$ is Pareto optimal if there exists
no other solution $\boldsymbol{x}\in\Omega$ that can dominate it. This implies
that within the feasible region $\Omega$, $\boldsymbol{x}^{\ast}$ is
considered to be Pareto optimal if no alternative solution offers better
outcomes across all objectives without being worse in at least one of them.

######  Definition 2.3 (Pareto Set and Pareto Front).

The Pareto Set (PS) refers to the collection of all the Pareto optimal
solutions:
$PS$$=$$\\{\boldsymbol{x}\in\Omega|\forall\boldsymbol{y}\in\Omega,\boldsymbol{y}\not\prec\boldsymbol{x}\\}$.
The corresponding set of objective vectors of the PS is the Pareto Front (PF).

###  2.2 Bayesian Optimization

Bayesian Optimization (BO) is a powerful method for the efficient global
optimization of expensive black-box functions (Jones et al., 1998; Shahriari
et al., 2015). By leveraging probabilistic surrogate models to approximate the
black-box functions, in conjunction with acquisition functions, it seeks to
locate global optimal solutions with as few evaluations of the actual
objective function as possible. BO has been widely used in a variety of
fields, including hyperparameter tuning (Bergstra et al., 2011), A/B testing
(Letham et al., 2019), combinatorial optimization (Zhang et al., 2015), among
others.

###  2.3 Diffusion Models

Diffusion models are a specialized form of probabilistic generative models
that operate by learning to reverse a forward process that gradually increases
noise in the training data (Sohl-Dickstein et al., 2015; Ho et al., 2020).
They have demonstrated remarkable performance on a wide variety of tasks, such
as image generation (Austin et al., 2021b; Bao et al., 2022), voice synthesis
(Liu et al., 2022), video generation (Ho et al., 2022) and inpainting (Lugmayr
et al., 2022). Training a diffusion model involves two processes: the forward
diffusion process and the backward denoising process.

####  2.3.1 Forward Process

In the forward phase, Gaussian noise is added to the input data step by step
until a pure Gaussian noise is produced, which is a Markovian process. Given
an initial data distribution $\mathbf{x}_{0}\sim q(\mathbf{x})$, the noised
$x_{1},x_{2}\ldots,x_{T}$ can be obtained from the following equation:

$q(x_{t}|x_{t-1})=\mathcal{N}\Big{(}x_{t};\sqrt{1-\beta_{t}}\cdot
x_{t-1},\beta_{t}\cdot\mathbf{I}\Big{)},\forall t\in\\{1,\ldots,T\\},$

where $T$ is the number of diffusion steps and the step sizes are controlled
by a variance schedule $\\{\beta_{t}\in(0,1)\\}_{t=1}^{t}$. Moreover, the
properties of this recursive formula make it possible to obtain $q(x_{t})$
directly from $x_{0}$ by the following equation:

$q(x_{t}|x_{0})=\mathcal{N}\bigg{(}x_{t};\sqrt{\hat{\beta}_{t}}\cdot
x_{0},(1-\hat{\beta}_{t})\cdot\mathbf{I}\bigg{)},\forall
t\in\\{1,\ldots,T\\},$

where $\hat{\beta_{t}}=\prod_{i=1}^{t}\alpha_{i}$ and
$\alpha_{t}=1-\beta_{t}$. Thus, $x_{t}$ can be sampled from $q(x_{t}|x_{0})$
as follows:

$x_{t}=\sqrt{\hat{\beta}_{t}}\cdot x_{0}+\sqrt{(1-\hat{\beta}_{t})}\cdot
z_{t},$

where $z_{t}\sim\mathcal{N}(0,\mathbf{I})$.

####  2.3.2 Reverse Process

The reverse process recreates the true sample from a Gaussian noise input
$x_{T}\sim\mathcal{N}(0,\mathbf{I})$ by the following equation:

$q(x_{t-1}|x_{t})~{}=~{}\mathcal{N}(x_{t-1};\mu(x_{t},t),\Sigma(x_{t},t))$

However, $q(x_{t-1}|x_{t})$ cannot easily be evaluated due to the reverse
process lacking a complete dataset and therefore we need to train a neural
network
$p_{\theta}(x_{t-1}|x_{t})=\mathcal{N}(x_{t-1};\mu_{\theta}(x_{t},t),\Sigma_{\theta}(x_{t},t))$
to approximate these conditional probabilities. Specifically, The model takes
in the noisy data $x_{t}$ and the corresponding embedding at time step $t$ as
input, and is trained to predict the mean $\mu_{\theta}(x_{t},t)$ and the
covariance $\Sigma_{\theta}(x_{t},t))$. Based on this, Ho (Ho et al., 2020)
proposed to fix the covariance $\Sigma_{\theta}(x_{t},t))$ to a constant value
and reformulating the mean $\mu_{\theta}(x_{t},t)$ as a function dependent on
noise, as follows:

$\mu_{\theta}=\frac{1}{\sqrt{\alpha_{t}}}\cdot\left(x_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\hat{\beta}_{t}}}\cdot
z_{\theta}(x_{t},t)\right).$

This enables the model to predict the noise of the data rather than directly
predicting the mean and the covariance.

###  2.4 Conditional Generative Models

In the field of generative models, Conditional Generative Models have garnered
significant interest (Odena et al., 2017; Gong et al., 2019). A prominent
approach in Generative Adversarial Networks (GANs) involves incorporating a
classification component within the Discriminator network. The strategy aims
to facilitate the learning of classifier conditions, a method explored in
various studies (Zhao et al., 2020; Kang et al., 2021; Dung & Binh, 2022). The
training of these classifiers proceeds occurs concurrently with the GAN
training process.

Similarly, Deep Generative Models (DGMs) have been adapted to include
conditional capabilities. This is achieved by integrating a classification
head into diffusion models, as demonstrated in recent work (Song et al.,
2020). There are instances where DGMs are conditioned on images, enabling
style transfer across different instances, a concept explored by Preechakul
(Preechakul et al., 2022). The research by Dhariwal and Nichol (Dhariwal &
Nichol, 2021b) presents a novel method allowing for the control of
samplers/generators in DGMs without necessitating retraining. This idea is
further expanded by Liu et al. (Liu et al., 2023), who generalize this concept
to accommodate various modalities. In the realm of guidance without
classifiers, Ho and Salimans (Ho & Salimans, 2022) have shown that it’s
possible to achieve guidance properties in generative models without relying
on a classifier.

###  2.5 Diffusion Model Based Optimization Algorithms

Krishnamoorthy proposed a black-box optimization algorithm named DDOM
(Krishnamoorthy et al., 2023), based on the diffusion model. This algorithm
converts a single-objective optimization problem into a continuous diffusion
process, leveraging the inverse process of the diffusion model to efficiently
address complex problems. Subsequently, Yan and Jin’s EmoDM (Yan & Jin, 2024)
extended the application of the diffusion model to multi-objective
optimization. By learning the noise distribution in the previous evolutionary
search task, a set of non-dominated solutions can be generated for the new
multi-objective optimization problem without further evolutionary search.
Fang’s DMO (Fang et al., 2024) further demonstrates the diffusion model’s
efficacy by applying it to create a gasoline hybrid scheduling scheme,
highlighting its capability in solving practical multi-objective optimization
challenges. Building upon these contributions, this paper’s CDM-PSL advances
the field by optimizing the balance between solution convergence and diversity
through the integration of conditional and unconditional diffusion models.
Moreover, CDM-PSL incorporates gradient information, weighted by information
entropy, into the process of generating solutions, significantly enhancing
convergence performance during the early-stage iterations. The combination of
these strategies makes CDM-PSL have competitive performance in solving
expensive multi-objective Bayesian optimization problem.

##  3 Our Method

Figure 1: (left) The framework of CDM-PSL. (right) Diffusion Model Training
(DMT), Conditional Generation (CG), and Unconditional Generation (UG). DMT
involves learning from the selected samples through multiple steps; CG is
designed to create high-quality samples with an optimized distribution; UG is
used to generate diverse samples with high efficiency.

###  3.1 Overview

We present a composite diffusion model based Pareto set learning method for
EMOBO, denoted as CDM-PSL (Algorithm 1 and Figure 1). CDM-PSL contains three
components to generate offspring: data extraction, diffusion model training
and conditional generation. Initially, the process begins by initializing a
set of samples $\boldsymbol{X}_{0}\subset\mathcal{X}$, where
$\mathcal{X}\subset\mathcal{R}^{d}$. These samples are drawn using Latin
Hypercube Sampling (LHS) (McKay et al., 2000). Following this initialization,
acquisition and batch selection are then applied in an iterative manner.

Algorithm 1 The framework of MOBO with CDM-PSL

Input: black-box function $\boldsymbol{f}(\boldsymbol{x})$, number of
iterations $K$, batch size $B$, number of initial solutions $N$

Output: final solutions $\\{\boldsymbol{X}_{K},\boldsymbol{Y}_{K}\\}$, Pareto
front $\mathcal{P}_{f}$

Initialize $N$ solutions $\\{\boldsymbol{X}_{0},\boldsymbol{Y}_{0}\\}$ by LHS

for $k=0$ to $K-1$ do

Train surrogate model $GP_{i}^{k}$ based on
$\\{\boldsymbol{X}_{k},\boldsymbol{Y}_{k}\\}$ for each objective
$f_{i},i=1,\ldots,M$

${\boldsymbol{X}_{k}^{*}}$ $\leftarrow$ Data extraction based on
$\\{\boldsymbol{X}_{k},\boldsymbol{Y}_{k}\\}$ (Algorithm 2)

if $\boldsymbol{F}_{CDM}$ is True then

$\boldsymbol{S}$ $\leftarrow$ Pareto set learning based on
${\boldsymbol{X}_{k}^{*}}$ by CDM-PSL (Algorithm 3)

else

$\boldsymbol{S}$ $\leftarrow$ Generate offspring using Optimizer

end if

${\boldsymbol{X}}_{k}^{B}$ $\leftarrow$ Batch selection based on $GP^{k}$,
$\boldsymbol{S}$ and $\\{\boldsymbol{X}_{k},\boldsymbol{Y}_{k}\\}$

Evaluate and update $\boldsymbol{X}_{k+1}$ $\leftarrow$ $\boldsymbol{X}_{k}$
$\cup$ ${\boldsymbol{X}_{k}^{B}}$, $\boldsymbol{Y}_{k+1}$ $\leftarrow$
$\boldsymbol{Y}_{k}$ $\cup$ $\boldsymbol{f}(\boldsymbol{X}_{k}^{B})$

Decide whether to invert $\boldsymbol{F}_{CDM}$ based on HV

end for

Approximate the Pareto front $\mathcal{P}_{f}$ by non-dominated solutions in
$\boldsymbol{Y}_{K}$

Algorithm 2 Data Extraction

Input: all already-evaluated solutions in the $k$-th iteration
$\\{\boldsymbol{X}_{k},\boldsymbol{Y}_{k}\\}$, number of real samples $T$

Output: solutions after data extraction $\boldsymbol{X}_{k}^{*}$

$Fitness$ $\leftarrow$ Calculate the fitness of each solution in
$\\{\boldsymbol{X}_{k},\boldsymbol{Y}_{k}\\}$ by Eq. 2

$\boldsymbol{X}_{k}^{*}$ $\leftarrow$ Select top $T$ candidate solutions with
$Fitness$

###  3.2 Data Extraction

To prepare training data for Pareto set learning, we propose a data extraction
strategy summarized in Algorithm 2. Central to this strategy is the
application of shift-based density estimation (SDE) (Li et al., 2013) for
calculating fitness values, which is mathematically represented as:

$\displaystyle
Fitness(\boldsymbol{p})=\min\limits_{\boldsymbol{q}\in\boldsymbol{Y}_{k}\backslash\boldsymbol{p}}{\sqrt{\sum_{i=1}^{M}(\mathrm{max}\\{0,f_{i}(\boldsymbol{q})-f_{i}(\boldsymbol{p})\\})^{2}}}$
(2)

In this formula, $\boldsymbol{p}$ and $\boldsymbol{q}$ are solutions within
the set $\boldsymbol{Y}_{k}$, and $f_{i}(\boldsymbol{p})$ indicates the $i$-th
objective value of solution $\boldsymbol{p}$. The SDE methodology assesses the
quality of samples based on their convergence and diversity characteristics.
From $\boldsymbol{X}_{k}$, a total of $T$ candidate solutions (for instance,
$\frac{|\boldsymbol{X}_{k}|}{3}$) that demonstrate superior SDE values are
identified as Pareto optimal samples. This selection process is essential for
ensuring the quality and relevance of the data used in Pareto set learning.

Algorithm 3 Composite Diffusion Model based Generation

Input: trained model $\mathcal{M}$

Parameter: steps $T$, number of generation $N_{1}$, $N_{2}$

Output: solutions generated by CDM $\boldsymbol{S}$

$\boldsymbol{S}$ $\leftarrow$ $\emptyset$

for $i=1$ to $N_{1}$ do

for $t=T$ to $1$ do

$z\sim\mathcal{N}(\mathbf{0},\mathbf{I})$

$\hat{g}$ $\leftarrow$ Calculate weighted gradient by entropy weight method
and surrogate model

$x_{t-1}$ $\leftarrow$
$\frac{1}{\sqrt{\alpha_{t}}}(x_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\epsilon_{\theta}(x_{t},t))+\sigma_{t}^{2}\hat{g}+\sigma_{t}z$

end for

$\boldsymbol{S}$ $\leftarrow$ $\boldsymbol{S}$ $\cup$ $\\{x_{0}\\}$

end for

for $i=1$ to $N_{2}$ do

for $t=T$ to $1$ do

$z\sim\mathcal{N}(\mathbf{0},\mathbf{I})$

$x_{t-1}$ $\leftarrow$
$\frac{1}{\sqrt{\alpha_{t}}}(x_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\epsilon_{\theta}(x_{t},t))+\sigma_{t}z$

end for

$\boldsymbol{S}$ $\leftarrow$ $\boldsymbol{S}$ $\cup$ $\\{x_{0}\\}$

end for

###  3.3 Diffusion Model Training

The DM training process comprises two major steps: the diffusion process and
noise prediction.

####  3.3.1 Diffusion Process

Given set of samples $X_{k}^{*}$ and a specified step $t$, the diffusion
process involves gradually introducing Gaussian noise
$\epsilon\sim\mathcal{N}(0,I)$ to $X_{k}^{*}$ over $t$ steps:

$X_{k,t}^{*}=\sqrt{1-\beta_{t}}X_{k,t-1}^{*}+\sqrt{\beta_{t}}\epsilon$ (3)

In this equation, $X_{k,t}^{*}$ denotes the data at step $t$, and
$\beta_{t}\in[1\mathrm{e}-5,5\mathrm{e}-2]$ represents the noise level at step
$t$. The DM learning process for PSL operates as a Markov chain. This stepwise
approach simplifies the learning task compared to direct Pareto set learning
and effectively captures the distribution characteristics of optimal samples.

####  3.3.2 Noise Prediction

The noise prediction phase involves reconstructing the samples $X_{k,t}^{*}$,
which have undergone the diffusion process, back to their original state,
$X_{k}^{*}$. This reconstruction is achieved through a model $\mathcal{M}$
that predicts the noise added at each step, thereby reversing the diffusion
process. This process follows the equation 4:

$\tilde{X}_{k,t-1}^{*}=\frac{1}{\sqrt{1-\beta_{t}}}\left(X_{k,t}^{*}-\frac{\beta_{t}}{\sqrt{1-\sum_{s=1}^{t}\beta_{s}}}\epsilon_{\theta}\left(X_{k,t}^{*},t\right)\right).$
(4)

In this equation, $\tilde{X}_{k,t-1}^{*}$ represents the data after
reconstruction, and $\theta$ denotes the parameters of the model. The term
$\epsilon_{\theta}(X_{k,t}^{*},t)$ is the predicted noise by model
$\mathcal{M}$ at step $t$.
The loss function $\mathcal{L}$ for model $\mathcal{M}$ is defined as:

$\mathcal{L}=\frac{1}{\mathcal{H}}\sum_{i=1}^{\mathcal{H}}\left(\epsilon-\epsilon_{\theta}\left(x_{k,i,t}^{*},t\right)\right).$
(5)

This equation takes $\mathcal{H}$ into account, the total number of optimal
samples, where each $x_{k,i,t}^{*}$ is an instance from $X_{k,t}^{*}$, and
$\epsilon_{\theta}(X_{k,t}^{*},t)$ represents the predicted noise. The
training of $\mathcal{M}$, using Equation 5, aims to minimize the loss
$\mathcal{L}$, thereby enhancing the accuracy of noise prediction.
The entire DM training process, comprising both the diffusion process and
noise prediction, plays a crucial role in effective Pareto Set Learning for
EMOBOPs. This method presents a novel approach to learning high-quality
solutions, striking a balancing between exploration and exploitation in the
search space.

###  3.4 Conditional Generation

In order to further enhancing the quality of generated samples, we propose a
Guided Denoise Process. This process leverages gradients to guide the
denoising procedure of DM, thereby achieving the realization of a Conditional
Diffusion Model. The subsequent elucidation will provide a detailed
explanation through two components: the guided denoise process and the
weighted gradient.

####  3.4.1 Guided Denoise Process

For a given step $t$ and sample $x_{t}$, the denoising process with guidance
can be implemented through equation 6:

$X_{t-1}=\frac{1}{\sqrt{\alpha_{t}}}(X_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\epsilon_{\theta}(X_{t},t))+\sigma_{t}^{2}\hat{g}+\sigma_{t}z$
(6)

where $\alpha_{t}$ represents $1-\beta_{t}$, $\epsilon_{\theta}(X_{t},t)$
signifies the predicted noise by the trained model, $\sigma_{t}$ is the
standard deviation of the $t$-th step, and $\hat{g}$ denotes the weighted
gradients used to guide the denoising process.

To procure the gradients essential for guiding the model in sample generation,
we establish separate Gaussian Process (GP) models for each objective, as
proposed by Balandat (Balandat et al., 2020). These models are utilized to
compute the objective values for all generated samples, thus obtaining
gradients $g$ for each objective, facilitating the realization of conditional
generation.

####  3.4.2 Weighted Gradients

In addressing EMOPs, employing weighted gradients is essential. This requires
the determination of appropriate weights for each objective. Therefore, we
propose a weighting methodology grounded in information entropy, facilitating
the derivation of these weighted gradients $\hat{g}$.

In order to obtain weights based on information entropy, it is necessary to
first normalize the objective values. Let $y_{ij}$ represent the $j$-th
objective value of the $i$-th individual. The normalized objective value
$\tilde{y}_{ij}$ is computed using Equation 7 as follows:

$\tilde{y}_{ij}=\frac{y_{ij}-\min\left(y_{j}\right)}{\max(y_{j})-\min\left(y_{j}\right)}$
(7)

Subsequent to this, for each objective $j(j=1,2,\ldots,M)$, the probability
matrix $P_{ij}$ is calculated using Equation 8, where $k=1,2,\ldots,N$ and $N$
represents the number of individuals in the population.

$P_{ij}=\frac{\tilde{y}_{ij}}{\sum_{k=1}^{N}\tilde{y}_{kj}}$ (8)

Subsequently, for each objective $j(j=1,2,\ldots,M)$, the information entropy
$E_{j}$ is computed. The method of calculation is as outlined in Equation 9:

$E_{j}=-\frac{1}{\ln{(N)}}\sum_{i=1}^{N}\left(P_{ij}\times\ln(P_{ij}+\eta)\right)$
(9)

Herein, $i=1,2,\ldots,N$ and $j=1,2,\ldots,M$, where $N$ signifies the total
number of samples, and $M$ represents the quantity of objectives. To avoid the
occurrence of $\ln(0)$, a small positive number, $\eta$, is introduced. The
computation of information entropy relies on Shannon entropy (Shannon, 1948),
expressed as $H(X)=-\sum\left(P(x)\times\log\left(P(x)\right)\right)$.
Additionally, the coefficient $\frac{1}{\ln\left(N\right)}$ is employed to
guarantee that the values of information entropy are confined within the range
of 0 to 1.

Finally, for each objective $j=1,2,\ldots,M$, the weight $W_{j}$ is computed
using Equation 10, where $k=1,2,\ldots,M$, and $M$ denotes the total number of
objectives.

$W_{j}=\frac{1-E_{j}}{\sum_{k=1}^{M}(1-E_{k})}$ (10)

By employing the calculated weights $W$, we can subsequently derive the
entropy weighted gradients (EWG) $\hat{g}$. The entropy weighting method,
serving as an adaptive weight allocation approach, assigns weights based on
the information entropy of objectives. This method reduces subjectivity in
weighting gradients for different objective values, and ensures that the
algorithm places more emphasis on objectives with rich information content,
thereby achieving better performance in multi-objective optimization problems.
Furthermore, the characteristic of weight allocation based on information
entropy endows the entropy weighting method with broad applicability, makeing
it suitable for a wide range of fields and various types of multi-objective
optimization problems, demonstrating its strong universality. In the context
of guiding the denoising process, the weighted gradients also enhance the
effectiveness of the guidance for multi-objective problems, thereby enhancing
the conditional generation.

###  3.5 Selection Strategy

####  3.5.1 Batch Selection

After obtain the solutions sampled by the CDM-PSL, we employ the batch
selection strategy of PSL-MOBO (Lin et al., 2022) to select a small subset
$\boldsymbol{X}_{k}^{B}=\\{x_{b}|b=1,\ldots,B\\}$. Specifically, this strategy
use the Hypervolume (HV) indicator (Zitzler & Thiele, 1999) as the selection
criteria. The HV indicator is defined as follows:

$\displaystyle\textbf{HV}(S)=\Lambda(\\{q\in\mathbb{R}^{d}|\exists p\in
S:p\leq q\;\textbf{and}\;q\leq r\\})$ (11)

where $S$ signifies a solution set, $r$ is identified as a reference vector
set, and $\Lambda(\cdot)$ denotes the Lebesgue measure.

####  3.5.2 Operator Selection

At the end of each iteration round, we have devised a method to switch
operators based on the growth rate of the HV indicator. In algorithm 1,
$F_{CDM}$ is a flag used to determine if CDM is currently being used to
generate offspring. After passing through an iteration window, such as three
evaluation rounds, if the HV indicator’s growth rate falls below a predefined
threshold (default setting is 5%), we switch the operator for offspring
generation (including CDM-PSL and other Optimizer like the Genetic Algorithm
(GA)). This strategy is designed to prevent the algorithm from becoming
trapped in local optima specific to the current operator.

##  4 Experiments Study

Figure 2: The HV results of 10 algorithms, evaluated on synthetic test
functions and real-world problems ($d=20$). The horizontal axis denotes the
FEs after the initialization phase, similarly hereinafter.

Figure 3: Ablation results of CDM-PSL on 6 instances.

(a) ZDT1

(b) DTLZ6

(c) RE7

Figure 4: Approximate Pareto fronts obtained by CDM-PSL and MOBO w/o CDM-PSL.

Figure 5: The HV values relative to the number of FEs for CDM-PSL with
different number of steps $t$.

###  4.1 Experimental Settings

##### Instances and Baselines

To comprehensively validate the performance of CDM-PSL, experiments were
conducted on 9 benchmark problems (2- and 3-objective ZDT1-3 (Zitzler et al.,
2000) and DTLZ2-7 (Deb et al., 2005)) and 7 real-world problems (Tanabe &
Ishibuchi, 2020).

Moreover, we have compared CDM-PSL with 9 state-of-the-art and classical
algorithms, including NSGA-II (Deb et al., 2002), MOEA/D-EGO (Zhang et al.,
2009), TSEMO (Bradford et al., 2018), USeMO-EI (Belakaria et al., 2020), DGEMO
(Konakovic Lukovic et al., 2020), PSL-MOBO (Lin et al., 2022), qNparEGO
(Knowles, 2006), qEHVI (Daulton et al., 2020) and qNEHVI (Daulton et al.,
2021).

##### Instances and Parameter Settings

For fair comparison, the population size $N$ was initialized to 100 for all
the compared algorithms. Bayesian optimization algorithms were executed for 20
batches, each with a batch size of 5, across all algorithms. Each method was
randomly run 10 times. For CDM-PSL, the hyperparameter $t$ was set to 25, the
batch size $m$ was 1024, and the learning rate $\gamma$ was 0.001, with
training spanning 4000 epochs. The configurations for other methods were
aligned with those in their original publications (See the appendix).

##### Evaluation Metrics

The hypervolume (HV) indicator, as defined in Equation 11, was employed to
assess the quality of the solutions obtained. Higher HV values are indicative
of better performance.

###  4.2 Experimental Results

We conducted a series of experiments on a variety of widely recognized
synthetic multi-objective benchmarks, including ZDT1-3 (Zitzler et al., 2000)
and DTLZ2-7 (Deb et al., 2005). The problems selected for the experiments
featured 2 and 3 objectives, with the number of decision variables set at 20.
We particularly highlight the results for a specific instance where $d=20$,
and further details are available in the supplementary materials.
Additionally, the comprehensive performance of CDM-PSL was thoroughly
evaluated through the application of seven real-world problems.

Figure 2 shows a comparison of the hypervolume (HV) indicator relative to
function evaluations (FE). CDM-PSL demonstrates outstanding performance across
most synthetic benchmarks, excelling in both convergence speed and final
values. Additionally, CDM-PSL exhibits ideal performance in real-world
problems (RE). These findings decisively affirm the effectiveness and
superiority of the CDM-PSL approach.

###  4.3 Ablation Study

To verify the effectiveness of each component in CDM-PSL, ablation study
results are presented in Figure 3.

##### Entropy Weight

CDM-PSL w/o Weight represents the CDM-PSL variant using mean weighted
gradients to guide the sampling process instead of entropy weighted gradients
(EWG). CDM-PSL demonstrates superior convergence performance compared to CDM-
PSL w/o Weight on all the six tested problems and attains better final values
in ZDT3, DTLZ3 and DLTZ6. This indicates that EWG can gauge the significance
of objectives more accurately, thereby offering more effective guidance for
the sampling process.

##### Conditional Generation

CDM-PSL w/o Condition refers to the variant from which the conditional
generation component has been omitted. In a similar vein, CDM-PSL consistently
and significantly outperforms CDM-PSL w/o Condition across all the tested
problems. This clearly validates the critical role and effectiveness of the
conditional generation component in the CDM-PSL.

##### Switching Strategy

CDM-PSL w/o Switch is a variation of CDM-PSL that lacks the switching
strategy. The results of the ablation experiments on ZDT3 and DTLZ3
demonstrate that the inclusion of this strategy prevents the PSL model from
falling into local optima, achieved through alternating between various
optimization operators. The incorporation of the switching strategy in CDM-PSL
results in enhanced convergence performance.

##### Diffusion Model

CDM-PSL w/o DM refers to the variant that employs the Genetic Algorithm (GA)
instead of the DM-based PSL model as its optimizer, utilizing Simulated Binary
Crossover (SBX) to generate new solutions. It is evident that this version’s
performance is markedly inferior compared to that of the standard CDM-PSL.
This disparity underscores the effectiveness of DM based Pareto set learning.

###  4.4 Validity of CDM-PSL

Figure 4 displays the Pareto fronts approximated by CDM-PSL and by MOBO
without CDM-PSL, based on the posterior mean. Clearly, CDM-PSL more
effectively captures the essential features of the true PF, surpassing MOBO
without CDM-PSL, in both synthetic benchmarks and real-world problems. For
instance, MOBO without CDM-PSL is difficult to approach the true PF in a
limited number of evaluations, yet CDM-PSL can effectively capture nearly all
characteristics of ZDT1. In the three-objective DLTZ6 problem, the use of CDM-
PSL can approximate the true PF faster in a limited number of evaluations.
Additionally, our methodology demonstrates commendable exploitation
capabilities on complex problems, such as the rocket injector design (RE7)
(Vaidyanathan et al., 2003), which is characterized by a complex PF. This
complexity arises from Pareto optimal solutions being distributed across
multiple regions.

###  4.5 Parameter Sensitivity Study

The main parameter of CDM-PSL, step $t$, is studied in this subsection. Figure
5 depicts the influence of varying $t$ values on CDM-PSL’s performance. The
default value of $t$ is set at 25 in CDM-PSL. A smaller $t$ yields suboptimal
experimental outcomes; although near-excellent final values can be achieved on
certain problems, there is a noticeable decline in convergence performance,
attributed to the inadequate learning of the distribution of superior
solutions. On the other hand, a larger $t$, such as 50, results in less stable
convergence performance compared to $t=25$, primarily due to an increase in
learning error. Furthermore, an increased step $t$ also leads to longer
training durations. Consequently, balancing between convergence performance
and time efficienvy, $t=25$ is established as the optimal default step size
for CDM-PSL.

##  5 Conclusion and Future work

In this paper, we introduce a composite diffusion model based Pareto set
learning method, termed CDM-PSL, for addressing EMOBOPs. CDM-PSL uses both
unconditional and conditional diffusion model for generating high-quality
samples. Besides, the quality of the solutions generated by the Pareto Set
learning model is significantly enhanced by employing entropy weighted
gradients to guide the sampling process. Extensive experimental evaluations on
9 benchmark problems and 7 real-world problems verify the efficiency of CDM-
PSL. The relatively high computational overhead makes CDM-PSL challenging to
apply in very high-dimensional problems. The Monte Carlo tree approach has
proven effective in feature extraction for such complex scenarios (Song et
al., 2022). Consequently, there is an intention to incorporate Monte Carlo
trees into existing algorithms for future research to address higher-
dimensional multi-objective Bayesian optimization problems.

## Impact Statements

This paper presents work whose goal is to advance the field of Multi-Objective
Bayesian Optimization (MOBO). There are many potential societal consequences
of our work, none which we feel must be specifically highlighted here.

## References

  * Ament et al. (2024) Ament, S., Daulton, S., Eriksson, D., Balandat, M., and Bakshy, E.  Unexpected improvements to expected improvement for bayesian optimization.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Amir & Hasegawa (1989) Amir, H. M. and Hasegawa, T.  Nonlinear mixed-discrete structural optimization.  _Journal of Structural Engineering_ , 115(3):626–646, 1989.
  * Austin et al. (2021a) Austin, J., Johnson, D. D., Ho, J., Tarlow, D., and Van Den Berg, R.  Structured denoising diffusion models in discrete state-spaces.  _Advances in Neural Information Processing Systems_ , 34:17981–17993, 2021a.
  * Austin et al. (2021b) Austin, J., Johnson, D. D., Ho, J., Tarlow, D., and Van Den Berg, R.  Structured denoising diffusion models in discrete state-spaces.  _Advances in Neural Information Processing Systems_ , 34:17981–17993, 2021b.
  * Balandat et al. (2020) Balandat, M., Karrer, B., Jiang, D., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, E.  Botorch: A framework for efficient monte-carlo bayesian optimization.  _Advances in neural information processing systems_ , 33:21524–21538, 2020.
  * Bao et al. (2022) Bao, F., Li, C., Zhu, J., and Zhang, B.  Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models.  _arXiv preprint arXiv:2201.06503_ , 2022.
  * Baranchuk et al. (2021) Baranchuk, D., Rubachev, I., Voynov, A., Khrulkov, V., and Babenko, A.  Label-efficient semantic segmentation with diffusion models.  _arXiv preprint arXiv:2112.03126_ , 2021.
  * Belakaria et al. (2020) Belakaria, S., Deshwal, A., Jayakodi, N. K., and Doppa, J. R.  Uncertainty-aware search framework for multi-objective bayesian optimization.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pp. 10044–10052, 2020.
  * Bergstra et al. (2011) Bergstra, J., Bardenet, R., Bengio, Y., and Kégl, B.  Algorithms for hyper-parameter optimization.  _Advances in neural information processing systems_ , 24, 2011.
  * Bradford et al. (2018) Bradford, E., Schweidtmann, A. M., and Lapkin, A.  Efficient multiobjective optimization employing gaussian processes, spectral sampling and a genetic algorithm.  _Journal of global optimization_ , 71(2):407–438, 2018.
  * Cheng & Li (1999) Cheng, F. and Li, X.  Generalized center method for multiobjective engineering optimization.  _Engineering Optimization_ , 31(5):641–661, 1999.
  * Cornish et al. (2020) Cornish, R., Caterini, A., Deligiannidis, G., and Doucet, A.  Relaxing bijectivity constraints with continuously indexed normalising flows.  In _International conference on machine learning_ , pp. 2133–2143. PMLR, 2020.
  * Couckuyt et al. (2014) Couckuyt, I., Deschrijver, D., and Dhaene, T.  Fast calculation of multiobjective probability of improvement and expected improvement criteria for pareto optimization.  _Journal of Global Optimization_ , 60:575–594, 2014.
  * Daulton et al. (2020) Daulton, S., Balandat, M., and Bakshy, E.  Differentiable expected hypervolume improvement for parallel multi-objective bayesian optimization.  _Advances in Neural Information Processing Systems_ , 33:9851–9864, 2020.
  * Daulton et al. (2021) Daulton, S., Balandat, M., and Bakshy, E.  Parallel bayesian optimization of multiple noisy objectives with expected hypervolume improvement.  _Advances in Neural Information Processing Systems_ , 34:2187–2200, 2021.
  * Deb & Srinivasan (2006) Deb, K. and Srinivasan, A.  Innovization: Innovating design principles through optimization.  In _Proceedings of the 8th annual conference on Genetic and evolutionary computation_ , pp. 1629–1636, 2006.
  * Deb et al. (2002) Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T.  A fast and elitist multiobjective genetic algorithm: Nsga-ii.  _IEEE transactions on evolutionary computation_ , 6(2):182–197, 2002.
  * Deb et al. (2005) Deb, K., Thiele, L., Laumanns, M., and Zitzler, E.  _Scalable test problems for evolutionary multiobjective optimization_.  Springer, 2005.
  * Dhariwal & Nichol (2021a) Dhariwal, P. and Nichol, A.  Diffusion models beat gans on image synthesis.  _Advances in Neural Information Processing Systems_ , 34:8780–8794, 2021a.
  * Dhariwal & Nichol (2021b) Dhariwal, P. and Nichol, A.  Diffusion models beat gans on image synthesis.  _Advances in neural information processing systems_ , 34:8780–8794, 2021b.
  * Ding et al. (2019) Ding, D., Li, D., Li, Z., and Yang, L.  Compact circularly-polarized microstrip antenna for hand-held rfid reader.  In _2019 8th Asia-Pacific Conference on Antennas and Propagation (APCAP)_ , pp. 181–182. IEEE, 2019.
  * Dung & Binh (2022) Dung, D. A. and Binh, H. T. T.  Gdegan: Graphical discriminative embedding gan for tabular data.  In _2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA)_ , pp. 1–11. IEEE, 2022.
  * Fang et al. (2024) Fang, W., Du, W., He, R., Tang, Y., Jin, Y., and Yen, G. G.  Diffusion model-based multiobjective optimization for gasoline blending scheduling.  _arXiv preprint arXiv:2402.14600_ , 2024.
  * Gong et al. (2019) Gong, M., Xu, Y., Li, C., Zhang, K., and Batmanghelich, K.  Twin auxilary classifiers gan.  _Advances in neural information processing systems_ , 32, 2019.
  * Ho & Salimans (2022) Ho, J. and Salimans, T.  Classifier-free diffusion guidance.  _arXiv preprint arXiv:2207.12598_ , 2022.
  * Ho et al. (2020) Ho, J., Jain, A., and Abbeel, P.  Denoising diffusion probabilistic models.  _Advances in neural information processing systems_ , 33:6840–6851, 2020.
  * Ho et al. (2022) Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al.  Imagen video: High definition video generation with diffusion models.  _arXiv preprint arXiv:2210.02303_ , 2022.
  * Hoffman & Ghahramani (2015) Hoffman, M. W. and Ghahramani, Z.  Output-space predictive entropy search for flexible global optimization.  In _NIPS workshop on Bayesian Optimization_ , pp. 1–5, 2015.
  * Jin et al. (2021) Jin, C., Netrapalli, P., Ge, R., Kakade, S. M., and Jordan, M. I.  On nonconvex optimization for machine learning: Gradients, stochasticity, and saddle points.  _Journal of the ACM (JACM)_ , 68(2):1–29, 2021.
  * Jones et al. (1998) Jones, D. R., Schonlau, M., and Welch, W. J.  Efficient global optimization of expensive black-box functions.  _Journal of Global optimization_ , 13(4):455, 1998.
  * Kang et al. (2021) Kang, M., Shim, W., Cho, M., and Park, J.  Rebooting acgan: Auxiliary classifier gans with stable training.  _Advances in neural information processing systems_ , 34:23505–23518, 2021.
  * Kannan & Kramer (1994) Kannan, B. and Kramer, S. N.  An augmented lagrange multiplier based method for mixed integer discrete continuous optimization and its applications to mechanical design.  1994\.
  * Knowles (2006) Knowles, J.  Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems.  _IEEE Transactions on Evolutionary Computation_ , 10(1):50–66, 2006.
  * Konakovic Lukovic et al. (2020) Konakovic Lukovic, M., Tian, Y., and Matusik, W.  Diversity-guided multi-objective bayesian optimization with batch evaluations.  _Advances in Neural Information Processing Systems_ , 33:17708–17720, 2020.
  * Krishnamoorthy et al. (2023) Krishnamoorthy, S., Mashkaria, S. M., and Grover, A.  Diffusion models for black-box optimization.  In _International Conference on Machine Learning_ , pp. 17842–17857. PMLR, 2023.
  * Laumanns & Ocenasek (2002) Laumanns, M. and Ocenasek, J.  Bayesian optimization algorithms for multi-objective optimization.  In _Parallel Problem Solving from Nature—PPSN VII: 7th International Conference Granada, Spain, September 7–11, 2002 Proceedings 7_ , pp. 298–307. Springer, 2002.
  * Leng et al. (2022) Leng, Y., Chen, Z., Guo, J., Liu, H., Chen, J., Tan, X., Mandic, D., He, L., Li, X., Qin, T., et al.  Binauralgrad: A two-stage conditional diffusion probabilistic model for binaural audio synthesis.  _Advances in Neural Information Processing Systems_ , 35:23689–23700, 2022.
  * Letham et al. (2019) Letham, B., Karrer, B., Ottoni, G., and Bakshy, E.  Constrained bayesian optimization with noisy experiments.  2019\.
  * Li et al. (2013) Li, M., Yang, S., and Liu, X.  Shift-based density estimation for pareto-based algorithms in many-objective optimization.  _IEEE Transactions on Evolutionary Computation_ , 18(3):348–365, 2013.
  * Lin et al. (2022) Lin, X., Yang, Z., Zhang, X., and Zhang, Q.  Pareto set learning for expensive multi-objective optimization.  In _36th Conference on Neural Information Processing Systems (NeurIPS 2022)_ , 2022.
  * Liu et al. (2022) Liu, J., Li, C., Ren, Y., Chen, F., and Zhao, Z.  Diffsinger: Singing voice synthesis via shallow diffusion mechanism.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 36, pp. 11020–11028, 2022.
  * Liu et al. (2023) Liu, X., Park, D. H., Azadi, S., Zhang, G., Chopikyan, A., Hu, Y., Shi, H., Rohrbach, A., and Darrell, T.  More control for free! image synthesis with semantic diffusion guidance.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pp. 289–299, 2023.
  * Lu et al. (2019) Lu, Z., Whalen, I., Boddeti, V., Dhebar, Y., Deb, K., Goodman, E., and Banzhaf, W.  Nsga-net: neural architecture search using multi-objective genetic algorithm.  In _Proceedings of the genetic and evolutionary computation conference_ , pp. 419–427, 2019.
  * Lugmayr et al. (2022) Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., and Van Gool, L.  Repaint: Inpainting using denoising diffusion probabilistic models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 11461–11471, 2022.
  * McKay et al. (2000) McKay, M. D., Beckman, R. J., and Conover, W. J.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code.  _Technometrics_ , 42(1):55–61, 2000.
  * Močkus (1975) Močkus, J.  On bayesian methods for seeking the extremum.  In _Optimization Techniques IFIP Technical Conference: Novosibirsk, July 1–7, 1974_ , pp. 400–404. Springer, 1975.
  * Odena et al. (2017) Odena, A., Olah, C., and Shlens, J.  Conditional image synthesis with auxiliary classifier gans.  In _International conference on machine learning_ , pp. 2642–2651. PMLR, 2017.
  * Preechakul et al. (2022) Preechakul, K., Chatthee, N., Wizadwongsa, S., and Suwajanakorn, S.  Diffusion autoencoders: Toward a meaningful and decodable representation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 10619–10629, 2022.
  * Ray & Liew (2002) Ray, T. and Liew, K.  A swarm metaphor for multiobjective design optimization.  _Engineering optimization_ , 34(2):141–153, 2002.
  * Shahriari et al. (2015) Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and De Freitas, N.  Taking the human out of the loop: A review of bayesian optimization.  _Proceedings of the IEEE_ , 104(1):148–175, 2015.
  * Shannon (1948) Shannon, C. E.  A mathematical theory of communication.  _The Bell system technical journal_ , 27(3):379–423, 1948.
  * Snoek et al. (2012) Snoek, J., Larochelle, H., and Adams, R. P.  Practical bayesian optimization of machine learning algorithms.  _Advances in neural information processing systems_ , 25, 2012.
  * Sohl-Dickstein et al. (2015) Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S.  Deep unsupervised learning using nonequilibrium thermodynamics.  In _International conference on machine learning_ , pp. 2256–2265. PMLR, 2015.
  * Song et al. (2022) Song, L., Xue, K., Huang, X., and Qian, C.  Monte carlo tree search based variable selection for high dimensional bayesian optimization.  _Advances in Neural Information Processing Systems_ , 35:28488–28501, 2022.
  * Song et al. (2020) Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B.  Score-based generative modeling through stochastic differential equations.  _arXiv preprint arXiv:2011.13456_ , 2020.
  * Tanabe & Ishibuchi (2020) Tanabe, R. and Ishibuchi, H.  An easy-to-use real-world multi-objective optimization problem suite.  _Applied Soft Computing_ , 89:106078, 2020.
  * Vaidyanathan et al. (2003) Vaidyanathan, R., Tucker, K., Papila, N., and Shyy, W.  Cfd-based design optimization for single element rocket injector.  In _41st Aerospace sciences meeting and exhibit_ , pp. 296, 2003.
  * Wu et al. (2020) Wu, H., Köhler, J., and Noé, F.  Stochastic normalizing flows.  _Advances in Neural Information Processing Systems_ , 33:5933–5944, 2020.
  * Yan & Jin (2024) Yan, X. and Jin, Y.  Emodm: A diffusion model for evolutionary multi-objective optimization.  _arXiv preprint arXiv:2401.15931_ , 2024.
  * Yang et al. (2023) Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., and Yang, M.-H.  Diffusion models: A comprehensive survey of methods and applications.  _ACM Computing Surveys_ , 56(4):1–39, 2023.
  * Yu (1974) Yu, P.-L.  Cone convexity, cone extreme points, and nondominated solutions in decision problems with multiobjectives.  _Journal of optimization Theory and Applications_ , 14:319–377, 1974.
  * Yu et al. (2022) Yu, S., Tack, J., Mo, S., Kim, H., Kim, J., Ha, J.-W., and Shin, J.  Generating videos with dynamics-aware implicit generative adversarial networks.  _arXiv preprint arXiv:2202.10571_ , 2022.
  * Yu et al. (2019) Yu, Z., Ramakrishnan, V., and Meinzer, C.  Simulation optimization for bayesian multi-arm multi-stage clinical trial with binary endpoints.  _Journal of Biopharmaceutical Statistics_ , 29(2):306–317, 2019.
  * Zhang et al. (2009) Zhang, Q., Liu, W., Tsang, E., and Virginas, B.  Expensive multiobjective optimization by moea/d with gaussian process model.  _IEEE Transactions on Evolutionary Computation_ , 14(3):456–474, 2009.
  * Zhang et al. (2015) Zhang, Y., Sohn, K., Villegas, R., Pan, G., and Lee, H.  Improving object detection with deep convolutional networks via bayesian optimization and structured prediction.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2015.
  * Zhao et al. (2020) Zhao, Y., Li, C., Yu, P., Gao, J., and Chen, C.  Feature quantization improves gan training.  _arXiv preprint arXiv:2004.02088_ , 2020.
  * Zitzler & Thiele (1999) Zitzler, E. and Thiele, L.  Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach.  _IEEE transactions on Evolutionary Computation_ , 3(4):257–271, 1999.
  * Zitzler et al. (2000) Zitzler, E., Deb, K., and Thiele, L.  Comparison of multiobjective evolutionary algorithms: Empirical results.  _Evolutionary computation_ , 8(2):173–195, 2000.
  * Zoph et al. (2018) Zoph, B., Vasudevan, V., Shlens, J., and Le, Q. V.  Learning transferable architectures for scalable image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 8697–8710, 2018.

##  Appendix A Experimental Details

###  A.1 Parameter Settings

First, we supplement more details about training DM. DM is designed with a
straightforward architecture, incorporating two linear layers, each containing
128 hidden units, and utilizes the ReLU activation function. Additionally, we
train DM for 4000 epochs, with a batch size of 1024. The Adam optimizer is
selected for the model, set with a learning rate of 0.0001.

Moreover, the hyperparameter for the DM’s step t is configured at 25, and the
noise level is defined within the range from 1e-5 to 0.5e-1.

The maximum number of function evaluations is 200, where 100 FEs are used to
initialize the population and 100 FEs are used to evaluate new offspring after
batch selection. Besides, batch size of all the algorithms is 5, thus number
of iterations is 100 / 5 = 20.

In CDM-PSL, the initial populations are all obtained from the Latin Hypercube
Sampling (LHS) (McKay et al., 2000). To ensure fairness and allow the
comparison algorithms to perform as them originally did, the relevant settings
of the comparison algorithms are kept unchanged from the original paper.

###  A.2 Synthetic Benchmark Problems

In this subsection, we detail each synthetic benchmark problem, outlining the
dimensions of the decision space $\mathcal{X}\in\mathbb{R}^{d}$ and the
objective space $\boldsymbol{f}(\mathcal{X})\in\mathbb{R}^{M}$, where
$\boldsymbol{f}(\cdot)$ represents the black-box function. Additionally, we
discuss the reference points $\boldsymbol{r}$ utilized for computing the
hypervolume indicator. Our experiments encompass 9 synthetic benchmark
problems, with the objectives numbering 2 and 3, and the decision variables is
10, 20 and 50. Further details are available in Table 1, and the
characteristics of each problem are listed in Table 2. For the reference point
$\boldsymbol{r}\in\mathbb{R}^{M}$, we adopt a vector comprising the maximum
objective values from the initial solution set
${\boldsymbol{x}_{1},...,\boldsymbol{x}_{N}}$ (Eq. 12).

$\displaystyle\begin{aligned} &\boldsymbol{r}=(max_{1\leq i\leq
N}{\boldsymbol{f}_{1}(\boldsymbol{x}_{i})},...,max_{1\leq i\leq
N}{\boldsymbol{f}_{M}(\boldsymbol{x}_{i})})\end{aligned}$ (12)

Table 1: Description of synthetic benchmarks we used in this work. Problem | M | d |  reference point ($\boldsymbol{r}$)
---|---|---|---
ZDT1 | 10 | 2 | (0.9901, 5.8380)
| 20 | 2 | (0.9994, 6.0576)
| 50 | 2 | (0.9926, 5.8722)
ZDT2 | 10 | 2 | (0.9901, 7.4625)
| 20 | 2 | (0.9994, 6.8960)
| 50 | 2 | (0.9926, 6.2181)
ZDT3 | 10 | 2 | (0.9901, 5.8101)
| 20 | 2 | (0.9994, 6.0571)
| 50 | 2 | (0.9926, 5.8648)
DTLZ2 | 10 | 3 | (1.8753, 1.9635, 2.0503)
| 20 | 3 | (2.8390, 2.9011, 2.8575)
| 50 | 3 | (5.4059, 5.6415, 6.5147)
DTLZ3 | 10 | 3 | (1002.2772, 1173.7753, 1075.0803)
| 20 | 3 | (2421.6427, 1905.2767, 2532.9691)
| 50 | 3 | (5298.0203, 5548.5643, 6250.5483)
DTLZ4 | 10 | 3 | (2.1527, 1.2691, 0.9761)
| 20 | 3 | (3.2675, 2.6443, 2.4263)
| 50 | 3 | (6.9024, 4.2313, 3.61552)
DTLZ5 | 10 | 3 | (1.7079, 1.7896, 2.0503)
| 20 | 3 | (2.6672, 2.8009, 2.8575)
| 50 | 3 | (5.2481, 5.5483, 6.5147)
DTLZ6 | 10 | 3 | (8.1417, 8.0853, 8.5711)
| 20 | 3 | (16.8258, 16.9194, 17.7646)
| 50 | 3 | (42.4261, 44.4857, 45.2363)
DTLZ7 | 10 | 3 | (0.9902, 0.9932, 24.5440)
| 20 | 3 | (0.9984, 0.9961, 22.8114)
| 50 | 3 | (0.9925, 0.9954, 21.7584)
Table 2: Characteristics of the synthetic benchmark problems. Benchmark | Features of | Features | Separability of
---|---|---|---
problem | Pareto front | of landscape | objective functions
DTLZ1 | Linear | Multimodal | Fully Separable
DTLZ2 | Concave | Unimodal | Fully Separable
DTLZ3 | Concave | Multimodal | Fully Separable
DTLZ4 | Concave | Unimodal, biased | Fully Separable
DTLZ5 | Concave, degenerate | Unimodal | Fully Separable
DTLZ6 | Concave, degenerate | Unimodal, biased | Fully Separable
DTLZ7 | Disconnected | Multimodal | Fully Separable

###  A.3 Real-world Application Problems

Our experiments include 7 real-world application problems (Tanabe & Ishibuchi,
2020), alongside several synthetic benchmark problems. These problems were
originally proposed across various fields for distinct applications. Below are
introductions to each of these real-world problems:

##### Four Bar Truss Design (RE1).

This task aims to optimize the design of a four-bar truss, focusing on
minimizing structural volume ($f_{1}$) and joint displacement ($f_{2}$). It
considers the lengths of the four bars ($x_{1}$, $x_{2}$, $x_{3}$, and
$x_{4}$) as variables. For more information, please see (Cheng & Li, 1999).

##### Pressure Vessel Design (RE2).

The goal here is to design a cylindrical pressure vessel to minimize total
costs ($f_{1}$), including materials, forming, and welding, and to avoid
violations of three design constraints ($f_{2}$). The decision variables are
the shell thicknesses ($x_{1}$), the pressure vessel head ($x_{2}$), the inner
radius ($x_{3}$), and the length of the cylindrical section ($x_{4}$).
Additional details are in (Kannan & Kramer, 1994).

##### Disk Brake Design (RE3).

This problem focuses on designing a disc brake to minimize mass ($f_{1}$),
stopping time ($f_{2}$), and violations of four design constraints ($f_{3}$),
with four decision variables: inner radius ($x_{1}$), outer radius ($x_{2}$),
engaging force ($x_{3}$), and the number of friction surfaces ($x_{4}$).
Further details are available in (Ray & Liew, 2002).

##### Gear Train Design (RE4).

The objective is to design a gear train to minimize the deviation from the
required gear ratio ($f_{1}$), the maximum size of the gears ($f_{2}$), and
violations of design constraints ($f_{3}$), considering the number of teeth in
each of the four gears ($x_{1}$, $x_{2}$, $x_{3}$, and $x_{4}$). More
information can be found in (Deb & Srinivasan, 2006).

##### Rocket Injector Design (RE5).

The design goal for the rocket injector is to minimize the maximum temperature
on the injector face ($f_{1}$), the distance from the inlet ($f_{2}$), and the
temperature at the post tip ($f_{3}$), with decision variables including
hydrogen flow angle ($x_{1}$), hydrogen area ($x_{2}$), oxygen area ($x_{3}$),
and oxidizer post tip thickness ($x_{4}$). Detailed information is in
(Vaidyanathan et al., 2003).

##### Reinforced Concrete Beam Design (RE6).

This design problem for a reinforced concrete beam aims to minimize the total
cost of concrete and reinforcing steel ($f_{1}$) and the sum of two constraint
violations ($f_{2}$), with variables for the reinforcement area ($x_{1}$),
beam width ($x_{2}$), and beam depth ($x_{3}$). Details are available in (Amir
& Hasegawa, 1989).

##### Welded Beam Design (RE7).

The challenge is to design a welded beam to minimize the cost ($f_{1}$), end
deflection ($f_{2}$), and violations of four constraints ($f_{3}$), with
variables adjusting the beam’s size ($x_{1}$, $x_{2}$, $x_{3}$, and $x_{4}$).
More information can be found in (Ray & Liew, 2002).

To maintain consistency in evaluation, the same reference point was applied
when evaluating all the algorithms. More details are shown in the
supplementary material. Details on the real-world application problems,
including information and reference points, are presented in Table 3.

Table 3: Description of real-world application problems we used in this work. Problem | d | M |  reference point ($\boldsymbol{r}$)
---|---|---|---
RE1 | 4 | 2 | (2763.2229, 0.0369)
RE2 | 4 | 2 | (528107.1899, 1279320.8107)
RE3 | 4 | 3 | (7.6853, 7.2861, 21.5010)
RE4 | 4 | 3 | (6.7921, 60.0, 0.48)
RE5 | 4 | 3 | (0.8745, 1.05092, 1.0533)
RE6 | 3 | 2 | (749.9241, 2229.3748)
RE7 | 4 | 3 | (210.3363, 1069.9160, 39196770.1704)

###  A.4 Hardware Settings

The experiments in this paper are conducted on a server with Ubuntu 22.04 LTS
operating system, a 1.5GHz AMD EPYC 7742 CPU (64 CPU Cores), 320 GB RAM, and
two NVIDIA RTX 4090 GPU.

##  Appendix B Related Works

###  B.1 Comparing DM-Based Methods with Normalizing Flows

Normalizing flows (NF) transforms data into a prior distribution through
bijective functions, which limits the ability of modeling complex data
distribution in both practical and theoretical contexts (Cornish et al., 2020;
Wu et al., 2020). However, DM is more efficient in the task of capturing
complex data distributions, and the CDM-PSL proposed in this paper also
demonstrates this experimentally. Moreover, the random noise introduced by DM
can also naturally generates a greater diversity of candidate solutions.

Specifically, DM enhances its expressive power by introducing random noise in
the forward and backward processes, compared with NF. Considering the case of
generating candidate solutions, this property of DM enables the exploration of
more solutions in the space, thus enriching the diversity of solutions.

From the perspective of the motivation behind our proposed method, the
performance of PSL methods might deteriorate significantly due to the
complexity of distributions of Pareto optimal solutions and the limitation of
function evaluations in the face of EMOBOPs. DM can model complex
distributions of solutions step by step, and generate more diverse solutions
by introducing random noise. This gives DM an advantage over NF for
application on the PSL of EMOBOPs.

###  B.2 Comparing CDM-PSL with DDOM

In this section, we explain the novelty and advantages of CDM over DDOM.
Krishnamoorthy proposed an offline black-box optimization algorithm named DDOM
(Krishnamoorthy et al., 2023), based on the diffusion model. DDOM trains a
generative model on offline dataset and proof its effectiveness on single-
objective optimization problems. The novelty and advantages of CDM-PSL over
DDOM are as follows:

Model composition: The main difference between CDM-PSL and DDOM is that CDM
consists of two generative models, CG and UG. CG can minimize all the
objective values as much as possible in a limited number of function
evaluations, and UG can generate solutions with greater diversity.

Training data: CDM-PSL trains DM on online dataset, while DDOM is trained on
offline dataset, which means that CDMs are able to receive and learn new data
instantly, which allows them to quickly adapt to changes in data distribution.

Inference process: CDM-PSL introduces gradient information, weighted by
information entropy, into the process of generating solutions, significantly
enhancing convergence performance under limited FEs.

Research problems: CDM-PSL is targeted at solving multi-objective optimization
problems, while DDOM mainly solves single-objective optimization problems.

The combination of our proposed methods makes CDM-PSL have competitive
performance in solving EMOPs.

##  Appendix C Additional Experiments

###  C.1 The Advantages of Using Diffusion Model

Effectively modeling the complicated distribution of the PS is quite
challenging for EMOPs with limited function evaluations. Existing Pareto set
learning algorithms may exhibit considerable instability in expensive
scenarios (Lin et al., 2022). DM is meticulously engineered to represent
complex datasets through a highly flexible family of probability
distributions. It has demonstrated promising outcomes in several domains,
notably in image restoration and data synthesis (Sohl-Dickstein et al., 2015).
Thus, DM offers a promising way for Pareto set learning in expensive multi-
objective Bayesian optimization. Specifically, the proposed CDM-PSL is a
composite of CG with conditional DM and UG with unconditional DM. It takes
advantage of CG’s convergence capabilities with UG’s diversity potential.
Figure 6 illustrates the offspring produced by the SBX versus those generated
by CG and UG on DTLZ7 and ZDT1 at FE = 25. The closer to (0, 0, 0), the
better. As can be seen from the figure, both CG and UG can generate solutions
with lower objective values.

  * •

CG with conditional DM can generate offspring solutions closer to the PF than
SBX and UG by introducing performance gradient information in the denoising
process, i.e., it has stronger convergence performance.

  * •

UG with unconditional DM does not have as strong convergence performance as
CG, but since it does not rely on the gradient of the aggregated objective
values to guide the denoising process, the solutions generated by UG will have
stronger diversity.

Figure 6: The objective values of newly generated solutions obtained by GA’s
SBX method, UG and CG. ($d=20$, $FE=25$)

###  C.2 The Advantages of Entropy Weighting Method

The entropy weighting method, serving as an adaptive weight allocation
approach, assigns weights based on the information entropy of objectives. This
method reduces subjectivity in weighting gradients for different objective
values, and ensures that the algorithm places more emphasis on objectives with
rich information content, thereby achieving better performance in multi-
objective optimization problems.

To show the advantages of entropy weighting method, we visualize the offspring
to compare the convergence performance of SBX and CG under three different
weighting methods: entropy weighting, mean weighting, and random weighting.
Figure 7 illustrates the objective values of all newly generated solutions at
FE = 25. As can be seen from the figure, CG with weights can generate
solutions with lower objective values across multiple objectives. Moreover, CG
with entropy weighted gradient has better convergence performance compared to
others. This is because CG (entropy weight) pay close attention to objectives
that are relatively more important (with higher information entropy) during
the generation of offspring. The offspring produced by it exhibit stronger
convergence compared to those generated with CG (mean weight) and CG (random
weight).

Figure 7: The objective values of newly generated solutions obtained by GA’s
SBX method, CG(entropy weight), CG(mean weight) and CG(random weight).
($d=20$, $FE=25$)

###  C.3 The Advantages of CDM Over a Gradient-Based Method With Multiple
Restarts

In this section, we demonstrate through experiments the advantages of our
proposed CDM-PSL compared to the gradient-based method (GM) with multiple
restarts. The main advantage of CDM over gradient-based methods is that CDM is
based on population-wide information to generate new offsprings, while some
other general optimization methods are based on individual solutions to
generate new offsprings and they ignore the connections between solutions,
which makes CDM more advantageous in maintaining population diversity (Lin et
al., 2022).

Moreover, gradient-based methods are more likely to get stuck in local optima
or saddle points and fail to explore the global optimum, and they are
sensitive to the choice of the initial point and the step size (Jin et al.,
2021). While the problem of them falling into a local optimum can be mitigated
to some extent by restart, this can lead to significant time overhead when
dealing with high dimensional problem. However, DM can model complex
distributions of solutions, and generate more diverse solutions by introducing
random noise.

To compare the performance of CDM and GM through experiments, we implemented a
GM with multiple restarts, with the following specific settings:

  * •

The objective value gradient of the current solution is obtained in the same
manner as in CG.

  * •

The weighting of the gradient adopts the entropy weighting method, which is
the most effective in CG.

  * •

SDE is used to select good solutions for the next generation of offspring
through gradient methods.

  * •

Data augmentation is performed on the selected solutions (starting points for
the gradient method), including adding Gaussian noise/random perturbations,
etc.

The experimental results are shown in Figure 8 ($d=20,FE=25$), which validate
the advantages of CDM over GM with multiple restarts by visualizing the
offspring. GM generates solutions that are more likely to fall into local
optimality, while CDM can maintain better diversity.

Figure 9 demonstrates that CDM, compared with GM, can generate solutions with
better convergence and diversity by comparing the HV values of both.

Figure 8: The objective values of newly generated solutions obtained by GA’s
SBX method, UG, CG and GM. ($d=20$, $FE=25$)

Figure 9: The HV results of CDM-PSL and GM ($d=20$).

###  C.4 Results on Synthetic Benchmark Problems with Various Decision Space
Scales

In this section, we broaden our experimental scope to encompass problems
within 10-dimensional decision spaces, maintaining the same experimental
parameters as before, including a batch size of 5, 20 iterations, and an
initial sample size of 100. The outcomes, illustrated in Figure 10 and 11,
also demonstrate that, overall, CDM-PSL is superior to the methods compared.

Specifically, when the dimensionality of decision variables is set to 10, CDM-
PSL exhibits highly competitive performance in almost all problems compared to
the benchmark methods. Furthermore, when the dimensionality of decision
variables reaches 50, our proposed method achieves notably superior
performance on DTLZ2, DTLZ4, DTLZ5, and DTLZ6. However, the performance of
CDM-PSL shows some degradation on ZDT3, DTLZ3, and DTLZ7. As indicated by the
characteristics of benchmark problems in Table 2, these problems all possess a
multimodal landscape feature. This phenomenon occurs because our proposed
Pareto set learning approach generates new offspring based on the distribution
of the current population. In these three types of problems, the Pareto
optimal set is distributed across multiple regions, and with the increase in
decision space dimensionality, the Pareto optimal sets of these multimodal
landscape problems become more complex. As a result, our method might struggle
with adequate exploration and exploitation on these types of problems at
higher dimensions. In contrast, CDM-PSL excels in capturing unimodal Pareto
sets (such as DTLZ2, DTLZ4, DTLZ5, and DTLZ6), particularly in high-
dimensional decision spaces, showing a clear advantage over the methods
compared.

Figure 10: The HV results of 10 algorithms, evaluated on synthetic test
functions and real-world problems ($d=10$).

Figure 11: The HV results of 10 algorithms, evaluated on synthetic test
functions and real-world problems ($d=50$).

###  C.5 Results on Synthetic Benchmark Problems with Various Initial
population size

In this section, we experiment with the performance of CDM-PSL under varying
initial population sizes. We selected qEHVI (Daulton et al., 2020), qNEHVI
(Daulton et al., 2021), qLogEHVI (Ament et al., 2024), and qLogNEHVI (Ament et
al., 2024) as comparison algorithms. The initial population size for these
algorithms were set according to their default settings, specifically
$2(d+1)$, where $d$ represents the dimension of the solution. Figure 12
presents the results of these five algorithms on different problems. From the
figure, it is evident that even with a smaller initial population size, CDM-
PSL maintains good convergence performance in the early stages of the
algorithm. Furthermore, compared to the other algorithms, CDM-PSL achieves
higher HV values in the final evaluation.

Figure 12: The HV results of 5 algorithms, with the initial population size of
$2(d+1)$, where $d=20$.

###  C.6 Comparation of Log HV Differences

In this section, we include the plot of log HV difference for all the test
problems ($d=20$). Figure 13 shows the plots of log HV difference for CDM-PSL
(our method) alongside PSL-MOBO and DGEMO, which are two of the most
competitive algorithms examined in our experiments. This comparison
effectively highlights what fraction of the problem was actually solved and
showcase the superior performance of CDM-PSL.

Figure 13: The log HV difference of 3 competitive algorithms, evaluated on
synthetic test functions and real-world problems ($d=20$).

###  C.7 Ablation Study with Limited FEs

To analyze the convergence performance of our proposed method, we compare CDM-
PSL with three variants: CDM-PSL without entropy weight (CDM-PSL w/o Weight),
CDM-PSL without conditional generation (CDM-PSL w/o Condition), CDM-PSL
without diffusion model (CDM-PSL w/o DM). As can be seen from the Table 4, the
convergence performance of CDM-PSL is impaired when any of the components is
ablated. Notably, incorporating the diffusion model to generate offspring
significantly improves the model’s performance with a minimal number of
function evaluations (specifically, 25), enabling it to achieve higher HV
values within a limited set of evaluations.

Table 4: Comparison of HV Values for CDM-PSL and its variants at 25 FEs.

| ZDT1 | ZDT2 | ZDT3 | DTLZ3 | DTLZ6 | DTLZ7
---|---|---|---|---|---|---
Method |  HV $\uparrow$ |  Gap $\downarrow$ |  HV $\uparrow$ |  Gap $\downarrow$ |  HV $\uparrow$ |  Gap $\downarrow$ |  HV $\uparrow$ |  Gap $\downarrow$ |  HV $\uparrow$ |  Gap $\downarrow$ |  HV $\uparrow$ |  Gap $\downarrow$
CDM-PSL w/o Weight | 4.9542e+00 | 12.42% | 5.4062e+00 | 12.14% | 4.2659e+00 | 17.49% | 1.0391e+10 | 3.74% | 4.3509e+03 | 10.46% | 1.4604e+01 | 16.22%
CDM-PSL w/o Condition | 4.9587e+00 | 12.34% | 5.1008e+00 | 17.10% | 4.1583e+00 | 19.57% | 1.0420e+10 | 3.47% | 4.2265e+03 | 13.02% | 1.4622e+01 | 16.12%
CDM-PSL w/o DM | 3.6577e+00 | 34.35% | 3.7327e+00 | 39.34% | 3.7185e+00 | 28.08% | 9.6465e+09 | 10.64% | 2.2088e+03 | 54.54% | 9.4968e+00 | 45.52%
CDM-PSL | 5.6570e+00 | 0.00% | 6.1533e+00 | 0.00% | 5.1703e+00 | 0.00% | 1.0795e+10 | 0.00% | 4.8592e+03 | 0.00% | 1.7432e+01 | 0.00%

###  C.8 Pareto Fronts Obtained by CDM-PSL on Real-World Problems

In this section, we add more figures of the final solutions obtained by CDM-
PSL in Real-World Problems. As can be seen from the figures 14, the proposed
CDM-PSL can effectively modeling the distribution of the PS, despite that the
Pareto fronts of the real-world test problems RE1-RE7 are complicated.

Figure 14: Approximate Pareto fronts obtained by CDM-PSL in real-word
problems. ($d=20$, $FE=100$)

###  C.9 Parameter Analysis

####  C.9.1 Performance with Different Learning Rates

In this section we compare the performance of CDM-PSL under different learning
rate settings to validate the sensitivity of our proposed method to this
parameter. Figure 15 illustrates the impact of varying learning rates on the
performance of CDM-PSL, notably with $lr=0.001$ being the default setting of
our proposed method. The results depicted in Figure 15 indicate that there are
no significant differences in the performance of CDM-PSL across different
learning rates.

Figure 15: The HV results of CDM-PSL at different learning rate ($d=20$).

####  C.9.2 Study of CG and UG ratios in CDM-PSL

In this section we exploring the impact of varying ratios between conditional
generation and unconditional generation in CDM-PSL. Figure 16 displays the HV
values for CDM-PSL with different CG and UG ratios across multiple problems.
Here, ”C1-U10” signifies a CG to UG ratio of 1 to 10, which is the default
setting for CDM-PSL; ”C3-U8” indicates a ratio of 3 to 8; and so forth. From
the figure, it is observable that incrementing the proportion of CG from the
default configuration marginally improves the algorithm’s early convergence
performance, as evidenced by a slightly elevated HV value up to the 25th
function evaluation. However, with a sufficient number of function
evaluations, the DM has been able to learn the distribution of the good
solutions more accurately. Consequently, the optimal final HV value is
achievable with a minimal CG ratio and a predominately higher experimental
group UG ratio. Therefore, we opt for a CG to UG ratio of 1 to 10, which
significantly reduces the algorithm’s time overhead without compromising its
final performance.

Figure 16: The HV results of CDM-PSL with different CG and UG ratios ($d=20$).

###  C.10 Time overhead of CDM-PSL

We choose ZDT1 and DTLZ2 to discuss the time overhead of CDM-PSL, the number
of decision variables is 10 and 20 respectively, the maximum number of
function evaluations is set to 120, and the parameters for training CDM are
consistent with the default settings mentioned in the manuscript.

Table 5 lists the time overhead of each part of the CDM-PSL, including
training, CG, UG and others. We choose ZDT1 and DTLZ2 to discuss the time
overhead of the algorithms, the number of decision variables is 10 and 20
respectively, the maximum number of function evaluations is set to 120, and
the parameters for training CDM are consistent with the default settings
mentioned in the manuscript.

As we can see from the table, the time overhead of training DM is a relatively
modest fraction of the total time overhead of CDM-PSL. CG takes longer to
generate new candidate solutions compared to UG. This significant difference
is attributed to CG’s reliance on a surrogate model to calculate the objective
values during the denoising process, which is essential for obtaining the
gradient information needed to guide the denoising. However, the additional
time overhead imposed by CDM is acceptable compared to the rest of the
algorithm.

In the far right column of the table, we present the time overhead associated
with one of the latest SOTA algorithms, PSL-MOBO. A comparison between the
total time overhead of our method and that of PSL-MOBO indicates that the
additional time overhead incurred by CDM is within acceptable limits.

Table 5: Time overhead of algorithm components in CDM-PSL.

| Training | CG | UG | Other | CDM-PSL | PSL-MOBO
---|---|---|---|---|---|---
Instance | Time(s) | Proportion | Time(s) | Proportion | Time(s) | Proportion | Time(s) | Proportion | Time(s) | Time(s)
ZDT1 ($d=10$) | 12.5578 | 21.13% | 9.5404 | 16.05% | 1.7426 | 2.93% | 35.5903 | 59.88% | 59.4311 | 32.5015
ZDT1 ($d=20$) | 12.5281 | 22.11% | 12.5824 | 22.20% | 1.7389 | 3.07% | 29.8183 | 52.62% | 56.6677 | 38.7551
DTLZ2 ($d=10$) | 12.5111 | 14.32% | 12.9713 | 14.85% | 1.7146 | 1.96% | 60.1494 | 68.86% | 87.3464 | 55.2474
DTLZ2 ($d=20$) | 12.5622 | 13.33% | 18.2628 | 19.38% | 1.7364 | 1.84% | 61.6794 | 65.45% | 94.2408 | 58.8312

##  Appendix D More Explanations About the Motivation of the Proposed Method

###  D.1 Motivation of using DM in PSL-based MOBO

When addressing EMOPs, the performance of PSL methods might deteriorate
significantly due to the complexity of distributions of Pareto optimal
solutions and the limitation of function evaluations. This poses a significant
challenge to existing PSL algorithms (Lin et al., 2022). Unlike existing
methods that directly learn the distribution of the PS in a single-step way,
complex distributions of solutions can be learned step by step with the aid of
DM. The success of DM in various domains (e.g. computer vision) shows promise
for its application in PSL in EMOPs.

###  D.2 Detailed motivation

Using DM for MOBO is not trival, because one should take into account the
following two aspect: how to speed up the searching process and how to balance
convergence and diversity. Having this in mind, we proposed a PSL method with
composite diffusion models. Specifically, the proposed CDM-PSL has the
following features:

  * •

CG with conditional DM is aimed at better convergence since it introduces
gradient information to guide the searching process.

  * •

UG with unconditional DM is designed for generating diverse solutions with low
cpu-time cost.

  * •

The entropy weighting method, serving as an objective aggregation approach for
offering performance gradient information, is introduced to assign weights
based on the information entropy of objectives. This method reduces
subjectivity in weighting gradients for different objective values, and
ensures that the algorithm places more emphasis on objectives with rich
information content. Figure 7 shows that CG with entropy weighting method can
generate better solutions than CG with mean weighting or random weighting.

##  Appendix E Limitations of CDM-PSL

In this section, we will discuss some limitations of CDM-PSL. One limitation
of our approach is that CG will introduce additional time overhead. As shown
in Table 5, conditional generation increases the time overhead of generating
candidate solutions to some extent, which leads to the limitations of the
method in terms of excessive time overhead when applied to very high
dimensional problems. Another limitation of our work is that our experiments
were not conducted on real-world problems with very high dimensionality. While
we have conducted experiments on seven real-world engine design problems,
RE1-RE7, which can provide some evidence of the superiority of CDM-PSL in
facing real-world problems, these test cases do not represent the complexity
and scale of extremely high-dimensional industrial problems. We will explore
the algorithm’s performance on super large problems like real industrial
design in future work.