# A search for a muon to electron conversion in COMET

Yuki Fujii

###### Abstract

The COMET experiment at J-PARC, Japan, aims to search for muon to electron
conversion with aluminium nuclei, achieving a sensitivity four orders of
magnitude higher than the current upper limit at a 90% confidence level. The
proton beam line has recently been completed, and muons have been successfully
transported through the curved solenoid in the Phase-alpha of the experiment.
In this paper, we will present preliminary results from the Phase-alpha beam
measurement, the status of the intermediate sensitivity experiment (COMET
Phase-I), and the ultimate goal of COMET Phase-II.

##  1 Introduction

The COMET experiment searches for a transition of a muon to an electron
without emission of neutrinos, also known as a $\mu\text{--}e$ conversion.
This process violates the conservation law of lepton flavour and is strictly
forbidden in the standard model of particle physics (SM). Even when
considering the minimum SM model extension with neutrino oscillations, the
conversion rate ($CR(\mu^{-}N\to e^{-}N)$) is strongly suppressed to
approximately $10^{-54}$, which is completely undetectable. On the other hand,
a large enhancement of the $CR(\mu^{-}N\to e^{-}N)$, up to $10^{-15}$, is
predicted in many BSM models, e.g., Leptoquark models [1]. The signal electron
is essentially monochromatic with an energy $E_{\mu
e}=M_{\mu}-B_{\mu}-E_{recoil}\approx$ 105 MeV in case of muonic aluminium.
This peculiar feature together with the low background level make the
$\mu\text{--}e$ conversion an ideal tool to investigate the BSM physics. The
current upper limit on $CR(\mu^{-}N\to e^{-}N)$ is $7\times 10^{-13}$ at 90%
confidence level set by the SINDRUM II experiment [3].

In COMET, our primary target is to achieve 100 times better single event
sensitivity, $10^{-15}$, which is called COMET Phase-I [2], followed by
another two orders of magnitude improved sensitivity in Phase-II. This allows
us to indirectly explore the new physics energy scale up to 100–1000 TeV.
Furthermore, the complementary searches with other muon CLFV modes ($\mu\to
e\gamma$ and $\mu\to eee$) will provide more detailed new physics
investigation by looking at slightly different parameter space. In addition to
those two stages, we recently performed the first muon beam delivery to the
COMET experimental area with a set of detectors, which is called COMET Phase-
alpha. In this paper, we first introduce the event feature of the signal and
backgrounds, then we report details of the experimental apparatus used in
Phase-I and Phase-II, and the current preparation status. Also, the
preliminary results from the first muon beam delivery and profile measurement
in COMET Phase-alpha, are briefly presented.

##  2 Signal and Backgrounds in COMET

In the SM, a muon decays into an electron and two neutrinos to conserve the
lepton flavour; $\mu^{-}\to e^{-}\nu_{\mu}\overline{\nu_{e}}$. If this happens
while the muon is bounded to an atom, this is known as muon decay in orbit
(DIO). Due to the presence of a nucleus, DIO electrons receive the recoil
energy and they can have a broad energy spectrum with an endpoint of
$\approx$105 MeV, hence, the DIO is an intrinsic background for the signal
event. Alternatively, a muon is captured by a nucleus and emits a neutrino and
neutrons/protons/alpha particles. The capture process also includes a
radiative process called radiative muon capture; RMC. This process produces a
photon or a pair of $e^{+}e^{-}$ in the final state. However, the maximum
energy of gamma-rays is sufficiently lower than the of the conversion electron
energy to make this background negligible. The DIO events can only be
discriminated by their energy, therefore the precise momentum measurement is a
key design factor. According to our studies based on simulations, a momentum
resolution of 200 keV/$c$ is required to achieve a single event sensitivity of
$10^{-15}$. To lower the sensitivity by a factor of 100, this resolution has
to be improved further down to 150 keV/$c$. In the case of the signal event, a
muon stops inside the target and forms a muonic atom at $1s$ ground state. The
electron emitted from the muonic aluminium has a monochromatic energy of 105
MeV as already explained with a decay time of $\tau_{\mu}=$ 864 ns.

Other sources of background are the electrons produced by the in flight decays
of muons or pions. In this case, the time delay due to the muonic atom
lifetime is not present so this prompt background can be suppressed by a
timing cut. We therefore adopt a pulsed beam structure with a separation of
$\sim$1.2 $\mu$s between the pulses and use a delayed measurement window
between 700 ns and 1170 ns from the bunch arrival on the target to avoid those
prompt backgrounds in COMET. In this scheme, it is also important to maintain
a highly pulsed structure, since any leakage protons appearing inter-bunches
can create a signal-like secondary particle in this time window. We call this
an “Extinction” factor defined as; $R_{\mathrm{ext}}=\mathrm{(\\#of\ protons\
between\ bunches)}/\mathrm{(\\#of\ protons\ in\ a\ main\ bunch)}$, and this
$R_{\mathrm{ext}}$ should be less than $10^{-10}$ ($10^{-11}$) to accomplish
the Phase-I (Phase-II) sensitivity.

The last background is a cosmic-ray induced background randomly produced
inside the detector with an energy close to 105 MeV. The cosmic-ray background
will be suppressed by covering the entire detector with a cosmic-ray-veto
detector.

##  3 Apparatus

(a) Conceptual view of the staging approach

(b) Detailed Phase-I layout

Figure 1: Overview of the COMET Phase-I and Phase-II conceptual design.

### Muon beam

To produce the world’s most intense muon beam, we use the proton beam
accelerated up to 8 GeV in the main ring (MR) of the proton synchrotron at
J-PARC (Japan Proton Accelerator Research Complex), Japan. To realise an ideal
bunch separation timing, only one of two buckets is filled with protons at the
rapid cycling synchrotron which accelerates the proton beam from 400 MeV to 3
GeV before the MR. This corresponds to 4 out of 9 total buckets in the main
ring with a bunch separation of 1.17 $\mu$s. After the acceleration, protons
are slowly extracted towards the J-PARC Hadron experimental facility by using
one or two electrostatic septa (ESS) and multiple magnetic septa (SMS) while
keeping the bunched structure (BSX: Bunched Slow Extraction) [4]. The protons
are delivered to the COMET beam area and impinged into the pion production
target which is surrounded by the 5 T pion capture solenoid. Through the
hadronic interaction, pions are produced, and low momentum back scattered
pions are efficiently transferred towards the 3 T muon transportation solenoid
(MTS). Most of pions decay into muons before reaching the exit of the MTS. Due
to the curved structure of the MTS, charged particles vertically drift
depending on their charge and transverse momentum while travelling inside,
with an additional dipole field used to adjust compensate the drifting of low
momentum negative muons to keep them around the centre at the end of the MTS.

### Phase-I

In Phase-I, we will conduct the physics measurement using a Cylindrical
Detector (CyDet) system placed at the end of the first 90 degrees of the MTS
(see Figure 1) inside the 1 T solenoidal magnetic field (Detector Solenoid:
DS). The muon stopping target, a series of 200 $\mu$m thick aluminium disks,
is located inside the CyDet. The cylindrical shape helps to avoid the
remaining beam particles such as pions and their secondaries that concentrate
on the beam axis with small transverse momentum, as well as filtering the most
of DIO electrons whose momentum is below 70 MeV/$c$. The CyDet system consists
of a Cylindrical Drift Chamber (CDC) and a set of Cylindrical Trigger
Hodoscope (CTH) detectors as shown in Figure 2(a). The CDC, filled with a
He:$i$C5H10 (90:10) gas mixture, contains almost 5,000 all stereo anode wires,
and is used to reconstruct the momentum of charged particles [5]. The wires
are read out by 105 RECBE boards 111Readout Electronics for CDC in BElle
(RECBE) [6] with minor adaptations. The expected momentum resolution is about
200 keV/$c$, which is good enough to achieve the Phase-I physics sensitivity.
The CTH detectors are located at both ends of CDC and consist of two layers of
concentric rings containing 64 counters each [7]. Each counter is a plastic
scintillator (BC-408) with height$\times$width$\times$length of 5 $\times$ 88
$\times$ 360 (10 $\times$ 88 $\times$ 340) mm3 for the inner (outer) layer.
All scintillators are connected to fibre bundles to readout the scintillation
photons outside of the detector solenoid for using silicon photo-multipliers
(SiPM) which are relatively weak against radiation damage and have to be
placed outside of the detector solenoid where the neutron level is an order
lower, inside the cooling box. Due to an extremely high hit rate environment,
the fake trigger could be an issue especially in Phase-I physics measurement.
To suppress the trigger rate caused by non signal-like fake events as low as
possible, an online trigger system is introduced by utilising machine learning
based algorithms inside field programmable gate arrays (FPGAs) in the CyDet
system, which is called COTTRI (COmeT TRIgger) [11]. This will realise the
trigger rate of less than 13 kHz, which is within the maximum available rate
for our data acquisition (DAQ) system.

We will also perform a direct beam measurement by replacing the CyDet with a
set of planar shaped detectors called StrECAL (Straw+ECAL, see Figure 2(b)).
The beam measurement detector consists of a series of straw tube tracker
(StrawTrk) stations and a full absorption type electron calorimeter (ECAL).
The StrawTrk has five stations and each one consists of one vertical layer and
the other horizontal layer. Each layer has staggered straw tubes in order to
minimise gaps to reduce inefficiency. A straw is made of 20 $\mu$m thick
aluminised Mylar with a 10 mm diameter, filled with Ar:CO2=50:50. The ECAL is
composed of arrays of LYSO (Lu2(1-x)Y2xSiO5) crystals with dimensions $2\times
2\times 12$ cm3 [9]. This length corresponds to 10.5 radiation lengths to
contain the signal like electrons with minimum leakage. In each crystal,
scintillation photons will be read by an avalanche photodiode (APD, Hamamatsu
S8664-1010, $10\times 10$ mm2). Both StrawTrk and ECAL signals are read by
specially designed waveform digitisers [10] based on domino ring sampler chips
(DRS4). The beam measurement detectors are also regarded as detector
prototypes for Phase-II physics measurement.

(a) CyDet (b) StrECAL

Figure 2: Two detector systems used in COMET Phase-I.

The DS will be covered by the iron yoke and cosmic ray veto counters made of
plastic scintillator. Also, there is a possibility to introduce resistive
plate chambers (RPC) around the MTS to DS bridging solenoid where plastic
scintillators cannot work properly due to higher neutron levels expected.

### Phase-II

To achieve another 100 times better sensitivity than in Phase-I, the
production target will be replaced from graphite to tungsten to increase the
pion yield, and shielding materials will be reinforced. The MTS will be
extended by another 90 degrees to further reduce pions, and there will be an
additional electron spectrometer after the muon stopping target to filter most
DIO electrons. This electron spectrometer enables using the detector system
that covers almost fully 2$\pi$ forward direction, increasing the signal
acceptance. In COMET Phase-II, it is crucial to improve the momentum
resolution which is dominated by the multiple scattering effects. Therefore,
all straw tubes will be replaced by thinner ones, 12 $\mu$m wall thickness
with twice smaller diameter of 5 mm to reduce the material budget inside the
tracking volume. The ECAL will cover the 50 cm radius to increase the
acceptance with 1,920 crystals.

##  4 Physics Sensitivity

In Table 1, we summarise the breakdown of the expected signal acceptance in
each phase together with the expected number of background events.

Table 1: Signal efficiency and expected number of background events for the COMET Phase-I [2] / Phase-II [12, 13]. Items | Phase-I | Phase-II
---|---|---
Signal acceptance | 0.2 | 0.18
Trigger + DAQ efficiency | 0.8 | 0.87
Track finding efficiency | 0.99 | 0.77
Track selection | 0.9 | 0.94
Momentum window | 0.93 | 0.62
Timing window | 0.3 | 0.49
Total | 0.04 | 0.034
Sources of background |  |
Muon decay in orbit | 0.01 | 0.068
Radiative muon capture | 0.0019 | Negligible
Neutron emission after muon capture | <0.001 | Negligible
Charged particle emission after muon capture | <0.001 | Negligible
Prompt beam induced electrons | <0.0038 | 0.002
Radiative pion capture | 0.0028 | 0.001
Delayed beam induced222Assuming $R_{\mathrm{ext}}$ is $10^{-10}$ and $10^{-11}$ for Phase-I and Phase-II, respectively | Negligible | 0.001
Antiproton induced | 0.0012 | 0.30
Cosmic ray induced | <0.01 | 0.29
Total | <0.032 | 0.662

The efficiencies have been obtained from a detailed simulation using the
results of the tests on the prototypes. The estimate of the number of events
assumes 150 (230) days of data acquisition with 3.2 kW (56 kW) beam power in
Phase-I (Phase-II) and $4.7\times 10^{-4}$ ($1.6\times 10^{-3}$) stopped muons
per proton on target. The single event sensitivity is defined as:

$SES(\mu^{-}N\to e^{-}N)=\frac{1}{N_{\mu}\cdot f_{cap}\cdot f_{gnd}\cdot
A_{sig}},$ (4.1)

where $N_{\mu}$ is the total number of stopped muons, $f_{cap}=0.61$ is the
fraction of nuclear muon captures, $f_{gnd}=0.9$ is the model dependent
fraction of muons to be the ground state. Given the signal efficiencies
reported in Table 1, the expected S.E.S. for Phase-I (Phase-II) is $3\times
10^{-15}$ ($1.4\times 10^{-17}$), which is 100 (10,000) times better than the
one achieved by SINDRUM II.

##  5 Status

The tests of the 8 GeV BSX proton beam have shown that the extinction factor
is enough to keep the out of pulse background at a negligible level [14]. The
C-line construction was recently completed up to the COMET beam area and is
ready to provide the beam. The coils for the proton capture solenoid are all
winded and the cryostat cold bore part has been constructed [15]. The first 90
degrees of the muon transport solenoid, that is the full TS for Phase-I, has
been constructed and tested up to 1.5 T. The coil winding for the detector
solenoid has been completed and the cryostat will be made soon this year. The
CDC has already been constructed, and its performance has been evaluated with
cosmic rays. It meets the performance requirement, a spatial resolution of 170
$\mu$m [5]. We have almost completed the R&D for the CTH and the design has
been fixed. The support structure has been partially purchased and the
detector construction is underway. A trigger system for COMET Phase-I is being
prepared and we have recently performed a successful test of a small trigger
electronics chain.

The first station of the straw tube tracker has been completed [16] and
commissioned in Phase-alpha beam measurement very recently. The LYSO crystals
are almost fully purchased, and the ECAL support structure has been
manufactured.

The CRV design is almost finalised and the first module has recently been been
produced.

For Phase-II, the thinner straw tube will be used with 12 $\mu$m with a twice
smaller diameter. The test production of thin straw tubes was done in success,
and they are under the long term stability test [17].

In February and March 2023, the proton beam has been successfully delivered to
the COMET beam area as COMET Phase-alpha. In this measurement, pions produced
in a thin graphite target have been transported through the MTS and the muons
coming from their decays have been successfully observed by a set of dedicated
detectors. Preliminary results can be found in [18] while the final ones will
be published soon.

##  6 Summary and Prospects

The COMET experiment aims to search for the $\mu$-$e$ conversion with 100 and
10,000 better sensitivities than the present upper limit. The COMET beam line
has been completed in 2022. More recently the proton beam has successfully
beam delivered to COMET Phase-alpha and the first muons have been observed.
Phase-I preparation is well on track and first physics measurements are
expected for 2024-2025.

## Acknowledgments

This was written on behalf of, and the author acknowledges strong support
from, the COMET collaboration. We acknowledge support from JSPS, Japan; ARC,
Australia; Belarus; NSFC, China; IHEP, China; IN2P3-CNRS, France; CC-IN2P3,
France; SRNSF, Georgia; DFG, Germany; JINR; IBS, Korea; RFBR, Russia; STFC,
United Kingdom; and the Royal Society, United Kingdom. The views expressed
herein are those of the author and are not necessarily those of the Australian
Government or Australian Research Council.

## References

  * [1] I. Bigaran, J. Gargalionis, and R.R. Volkas, _A near-minimal leptoquark model for reconciling flavour anomalies and generating radiative neutrino masses_ , _J. High Energ. Phys._ , 2019, 106 (2019). https://doi.org/10.1007/JHEP10(2019)106
  * [2] G. Adamov, et.al. (COMET Collaboration), _COMET Phase-I techinical design report_ , _Progress of Theoretical and Experimental Physics_ , vol.2020 ,033C01 (2020).
  * [3] W. Bertl, et.al., _A search for $\mu$-$e$ conversion in muonic gold_, _Eur. Phys. J. C_ Vol 47, 337–346 (2006).
  * [4] M. Tomizawa, et.al., _Slow extraction from the J-PARC main ring using a dynamic bump_ , _Nucl. Instrum. Methods A_ , Vol 902, pp51–61 (2018).
  * [5] M. Moritsu, et.al., _Construction and performance tests of the COMET CDC_ , _Proceedings of Science_ , Vol 340, p538 (2019).
  * [6] T. Uchida, et.al., _2011 IEEE Nuclear Science Symp. Conf. Record_ , p. 694 (2011).
  * [7] Y. Fujii, et.al., _Studies of optical fibre and SiPM readout system for the Cylindrical Trigger Hodoscope in COMET Phase-I_ , _The 30th International Symposium on Lepton Photon Interactions at High Energies_ (Lepton Photon 2021), Zenodo, https://doi.org/10.5281/zenodo.7542297 (2022).
  * [8] H. Nishiguchi, et.al., _Construction on vacuum-compatible straw tracker for COMET Phase-I_ , _Nucl. Instrum. Methods A_ , Vol 958, 162800 (2020).
  * [9] K. Oishi, _Development of Electromagnetic Calorimeter Using LYSO Crystals for the COMET Experiment at J-PARC_ , _in Proceedings of the Eur. Phys. Soc. Conf. on High Ene. Phys_ , EPS-HEP, 5-12 July 2017, Venice, Italy (2018)
  * [10] K. Ueno, et.al., _Design of Radiation Tolerant Electronics for StrECal System in COMET Experiment_ , _in Proceedings of the Eur. Phys. Soc. Conf. on High Ene. Phys_ , EPS-HEP, 1-17 July 2019, Ghent, Belgium (2019)
  * [11] Y. Fujii, et.al., _Online Machine-Learning-Based Event Selection for COMET Phase-I_ , _Phys. Sci. Forum_ 2023, 8, 32. https://doi.org/10.3390/psf2023008032
  * [12] K. Oishi _Study of Sensitivity to Search for a Charged Lepton Flavor Violating Process_ , _Kyushu University, PhD thesis_ , 2021, http://epp.phys.kyushu-u.ac.jp/thesis/2021DoctorOishi.pdf
  * [13] B. Krikler _Sensitivity and background estimates for phase-II of the COMET experiment_ , _Imperial College London, PhD thesis_ , 2016, https://doi.org/10.25560/45365
  * [14] K. Noguchi, et al., _Extinction Measurement at J-PARC MR with Slow-Extracted Pulsed Proton Beam for COMET Experiment_ , _Proceedings of Science_ vol. 402, p104 (2022).
  * [15] M. Yoshida et al., _Status of Superconducting Solenoid System for COMET Phase-I Experiment at J-PARC_ , in _IEEE Transactions on Applied Superconductivity_ , vol. 25, no. 3, pp. 1–4, June (2015).
  * [16] H. Nishiguchi, et.al., _Vacuum-Compatible, Ultra-Thin-Wall Straw Tracker; Detector construction, Thinner straw R &D, and the brand-new graphite-straw development_, _Nucl. Instrum. Methods A_ , Vol 1042, 167373 (2022).
  * [17] N. Tsverava, et.al., _Development of Ultrathin 12 $\mu$m Thick Straw Tubes for the Tracking Detector of COMET Experiment_, _2019 IEEE NSS/MIC_ , Manchester, UK, 2019, pp. 1-4, doi: 10.1109/NSS/MIC42101.2019.9060032.
  * [18] K. Oishi, _COMET Phase-Alpha Experiment to Investigate COMET’s New Muon Beamline at J-PARC_ _The 31st International Symposium on Lepton Photon Interactions at High Energies_ (Lepton Photon 2023), Melbourne, Australia (2023).


# Correspondence Between the Energy Equipartition Theorem in Classical
Mechanics and its Phase-Space Formulation in Quantum Mechanics

Esteban Marulanda<EMAIL_ADDRESS>Instituto de Física,
Universidad de Antioquia, Calle 70 No. 52-21, Medellín, Colombia  Alejandro
Restrepo<EMAIL_ADDRESS>Instituto de Física, Universidad de
Antioquia, Calle 70 No. 52-21, Medellín, Colombia  Johans Restrepo
<EMAIL_ADDRESS>Instituto de Física, Universidad de Antioquia,
Calle 70 No. 52-21, Medellín, Colombia  Group pf Magnetism and Simulation.
Instituto de Física, Universidad de Antioquia. A.A. 1226, Medellín, Colombia

###### Abstract

In classical physics there is a well-known theorem in which it is established
that the energy per degree of freedom is the same. However, in quantum
mechanics due to the non-commutativity of some pairs of observables and the
possibility of having non-Markovian dynamics, the energy is not equally
distributed. We propose a correspondence between what we know about the
classical energy equipartition theorem and its possible counterpart in phase-
space formulation in quantum mechanics based on the Wigner representation.
Also, we show that in the high-temperature regime, the classical result is
recovered.

Keywords— Energy equipartition theorem, Phase-space quantum mechanics

##  1 Introduction

The energy equipartition theorem is one of the most important results in the
classical theory of statistical mechanics due to its quantitative predictions
and applicability in many areas of physics [1]. However, few references exist
for an extension of it when quantum effects become relevant, referring mainly
to low-temperature phenomena [2]. A better understanding of the energetic
distributions in this regime is then necessary for progressing in theoretical
aspects and applications of quantum systems.

Currently, the authors working in the area agree that the equipartition of
energy no longer holds in the quantum regime and the energetic distribution
follows a better-called energy partition theorem supported in the construction
of a distribution function [3, 4, 5]. Applications to a few models have been
made with satisfactory results and holding the correspondence with the
classical theorem at the high-temperature regime [4, 6, 7]. Due to the recent
exploration of this area, the works about it are very homogeneous and based on
the same conceptual and mathematical framework.

In this article a novel approach is implemented; namely, the derivation of an
analog of the classical equipartition theorem in the phase-space of quantum
mechanics through the Wigner representation. First, a review and certain
mathematical manipulations are made to the classical statement of the theorem
so that with the aid of some results in the phase-space formulation, the
version of the theorem is shown straightforward in a completely analog manner.
The results derived are tested and validated, applying them to the harmonic
oscillator in both high and low-temperature regimes for the particular case of
weak coupling limit.

##  2 Classical Version of the Theorem

Consider a system composed of $N$ particles and described by the set of
generalized coordinates {$q_{i}$, $p_{i}$} where $i=1,2,...,fN$ with $f$ being
the number of degrees of freedom per particle. The classical Hamiltonian of
the system is $H_{S}(q,p)$ where $q$ and $p$ represent all the generalized
coordinates, so the 3D energy equipartition theorem reads as follows [1]

$\left<q_{i}\frac{\partial H_{S}}{\partial q_{i}}\right>=\frac{1}{h^{3N}}\int
d^{N}\mathbf{q}\hskip 1.42262ptd^{N}\mathbf{p}\hskip
3.41432pt\rho(\mathbf{q},\mathbf{p})\hskip 1.42262ptq_{i}\frac{\partial
H_{S}}{\partial q_{i}}=\frac{1}{\beta}$ (1)

$\left<p_{i}\frac{\partial H_{S}}{\partial p_{i}}\right>=\frac{1}{h^{3N}}\int
d^{N}\mathbf{q}\hskip 1.42262ptd^{N}\mathbf{p}\hskip
3.41432pt\rho(\mathbf{q},\mathbf{p})\hskip 1.42262ptp_{i}\frac{\partial
H_{S}}{\partial p_{i}}=\frac{1}{\beta}$ (2)

here $h$ is the Planck constant, $\beta=(k_{B}T)^{-1}$ where $k_{B}$ is the
Boltzmann constant, $\mathbf{q}$ and $\mathbf{p}$ denote all the coordinates
per particle and $\rho(\mathbf{q},\mathbf{p})$ is the corresponding phase-
space density in any of the Gibbs ensembles [1]. Notice that the coordinates
$q_{i}$ and $p_{i}$ have not been merged into a single coordinate $x_{i}$ as
is customary in the demonstration of this theorem, this has a purpose as will
be shortly shown. In the particular case in which the density distribution is
given by the canonical ensemble, $\rho(\mathbf{q},\mathbf{p})=\exp(-\beta
H_{S}(q,p))/Z$ where explicitly the partition function is
$Z=\frac{1}{h^{3N}}\int d^{N}\mathbf{q}\hskip 1.42262ptd^{N}\mathbf{p}\hskip
3.41432pt\exp-(\beta H_{S}(q,p))$. It is possible, for simplicity, to think in
an alternative version of this theorem for the case in which $\beta$ does not
correspond with the known usual expression, but with a modified function of
temperature $\beta_{mod}=\beta_{mod}(T)$, then, equations 1 and 2 read as
follows:

$\left<q_{i}\frac{\partial H_{S}}{\partial
q_{i}}\right>=\frac{1}{\beta_{mod}}$ (3)

$\left<p_{i}\frac{\partial H_{S}}{\partial
p_{i}}\right>=\frac{1}{\beta_{mod}}$ (4)

where now the density distribution is given by
$\rho_{mod}(\mathbf{q},\mathbf{p})=\text{exp}(-\beta_{mod}H_{S}(\mathbf{q},\mathbf{p}))/Z_{mod}$.

Let now assume that the Hamiltonian of the system can be separated in two
functions according to:

$H_{S}(q,p)=F(q)+G(p),$ (5)

which is usually the case, so we will be able to transform equations 1 and 2
into more favorable forms for the connection with the quantum mechanical
phase-space formulation. This is a feasible assumption since many physical
systems hold it. Let’s now consider the following Hamilton equations

$\dot{q}_{i}=\frac{\partial H_{S}}{\partial p_{i}}=\\{q_{i},H_{S}\\}_{PB}$ (6)
$\dot{p}_{i}=-\frac{\partial H_{S}}{\partial q_{i}}=\\{p_{i},H_{S}\\}_{PB}$
(7)

where $\\{,\\}_{PB}$ is the Poisson bracket defined for quantities $A(q,p)$
and $B(q,p)$ as

$\\{A,B\\}_{PB}=\sum_{i}\left(\frac{\partial A}{\partial q_{i}}\frac{\partial
B}{\partial p_{i}}-\frac{\partial A}{\partial p_{i}}\frac{\partial B}{\partial
q_{i}}\right)$ (8)

Including a third function $C(q,p)$, the following identity can be
demonstrated

$\\{AB,C\\}_{PB}=A\\{B,C\\}_{PB}+\\{A,C\\}_{PB}B$ (9)

Applying these results for the quantities $q_{i}\frac{\partial H_{S}}{\partial
q_{i}}$ and $p_{i}\frac{\partial H_{S}}{\partial p_{i}}$, using the equations
5, 6, 7 and identity 9, it can be shown that

$q_{i}\frac{\partial H_{S}}{\partial
q_{i}}=-q_{i}\\{p_{i},H_{S}\\}_{PB}=-q_{i}\\{p_{i},F(q)\\}_{PB}=-\\{q_{i}p_{i},F(q)\\}_{PB}$
(10)

$p_{i}\frac{\partial H_{S}}{\partial
p_{i}}=p_{i}\\{q_{i},H_{S}\\}_{PB}=p_{i}\\{q_{i},G(p)\\}_{PB}=\\{p_{i}q_{i},G(p)\\}_{PB}$
(11)

Replacing equations 10 and 11 in 3 and 4 respectively one obtains

$\left<q_{i}\frac{\partial H_{S}}{\partial q_{i}}\right>=-\int
d^{N}\mathbf{q}\hskip 0.56905ptd^{N}\mathbf{p}\hskip
0.56905pt\frac{\exp(-\beta_{mod}H_{S}(\mathbf{q},\mathbf{p}))}{Z_{mod}}\hskip
0.56905pt\\{q_{i}p_{i},F(q)\\}_{PB}=\frac{1}{\beta_{mod}}$ (12)

$\left<p_{i}\frac{\partial H_{S}}{\partial p_{i}}\right>=\int
d^{N}\mathbf{q}\hskip 0.56905ptd^{N}\mathbf{p}\hskip
0.56905pt\frac{\exp(-\beta_{mod}H_{S}(\mathbf{q},\mathbf{p}))}{Z_{mod}}\hskip
0.56905pt\\{p_{i}q_{i},G(p)\\}_{PB}=\frac{1}{\beta_{mod}}$ (13)

These are the algebraic forms of the energy modified equipartition theorem
that will allow us henceforth to easily connect with the quantum mechanical
phase-space formulation. In the case when $\beta_{mod}=\beta$, we recover the
usual equipartition theorem.

##  3 Density Operator in Quantum Mechanics

It’s wellFor a non-factorizing initial state is known that the total state of
a system (S) and an environment (E) in quantum mechanics can be described by
the thermal equilibrium state through the density operator [8]

$\hat{\rho}={\text{exp}(-\beta\hat{H})}/{Z}$ (14)

where $\hat{H}$ is the total Hamiltonian of the system plus environment acting
on the total Hilbert space $\mathbb{H}=\mathbb{H_{S}}\otimes\mathbb{H_{E}}$
and $Z$ the total partition function. Explicitly, the total Hamiltonian is
$\hat{H}=\hat{H}_{S}+\hat{H}_{E}+\hat{V}$, where $\hat{H}_{S}$, $\hat{H}_{E}$
and $\hat{V}$ represent the Hamiltonian of the system, the environment and the
mutual interaction respectively. The interaction $\hat{V}$ can always be
written in a diagonal decomposition of a set of operators
$\left\\{S\hat{S}_{\alpha}\right\\}$ acting only on the system and a set
$\left\\{B\hat{B}_{\alpha}\right\\}$ acting only on environment, such that
$\hat{V}=\sum_{\alpha}\hat{S}_{\alpha}\otimes\hat{B}_{\alpha}$ [9]. The
description of the system of interest is given by the reduced density matrix,
this object contains all the information that can be extracted by an observer
of the system [9]. It can be shown that the reduced density matrix at second
order can be written as [10]

$\hat{\rho}_{s}\propto
e^{-\beta\hat{H}_{S}}\left[1+\frac{1}{\hbar}\sum_{\alpha}\int_{0}^{\hbar\beta}d\tau_{1}\int_{0}^{\tau_{1}}d\tau_{2}\hat{S}_{\alpha}(-i\tau_{1})\hat{S}_{\alpha}(-i\tau_{2})k_{\alpha}(\tau_{1}-\tau_{2})\right]$
(15)

where $\hbar
k_{\alpha}(\tau)=\left<\hat{B}_{\alpha}(-i\tau)\hat{B}_{\alpha}(0)\right>_{B}$
is the two-time correlation function of the environment operators. Equation 15
shows that in general, it is not possible to describe the influence of the
bath in a system by the Gibbs state as in the case of classical mechanics
[10]. However, in the limit of high temperature $\hbar\beta\rightarrow 0$,
independently of the interaction form of $\hat{V}$, tIn order to give a simple
illustration in section 6, we consider the weak coupling limit. The reduced
density matrix in this case is the Gibbs state given by

$\hat{\rho}_{S}=\text{exp}(-\beta\hat{H}_{S})/{Z_{S}}$ (16)

where in this caseWith $Z_{S}=\text{Tr}_{S}(\text{exp}(-\beta\hat{H}_{S}))$
being the canonical partition function. On the other hand, it is useful to
note that there is a relationship between the propagator
$K(q_{f},t,q_{i},0)=\left\langle
q_{f}|\text{exp}(-\frac{it}{\hbar}\hat{H}_{S})|q_{i}\right\rangle$ formulated
in terms of path integrals [11] and the matrix elements in the position basis
of equation 16. Such a relation is evident when we replace
$t\rightarrow-i\hbar\beta$ in the propagator, i.e,
$K(q_{f},-i\hbar\beta,q_{i},0)=\left\langle
q_{f}|\text{exp}(-\beta\hat{H}_{S})|q_{i}\right\rangle$, then one obtains

$\left\langle
q_{f}|\hat{\rho}_{S}|q_{i}\right\rangle=\frac{1}{Z_{S}}\left\langle
q_{f}|\text{exp}(-\beta\hat{H}_{S})|q_{i}\right\rangle=\frac{1}{Z_{S}}\int\mathfrak{D}q\hskip
2.84526pt\text{exp}(-\frac{1}{\hbar}S^{E}[q])$ (17)

Here the integral is a functional integral running over all the functions
satisfying the boundary conditions $q(0)=q_{i}$, $q(\hbar\beta)=q_{f}$ and
$S^{E}[q]=\int_{0}^{\hbar\beta}dsH_{S}(q(s),p(s))$ is the action of the
system.

##  4 Phase-Space Formulation of Quantum Mechanics

There are different approaches to phase-space formulation of quantum mechanics
that try to associate a phase-space distribution function, for example the
Glauber-Sudarshan, Kirkwood, Husimi $Q-$representations [12, 13], etc. There
is a particular approach that consists in associating every quantum observable
$\hat{O}(\hat{q},\hat{p})$ to a phase-space function
$O_{W}(\mathfrak{q},\mathfrak{p})$ (also called symbol) by means of a
bijection $\Phi$, called the Weyl transformation and defined as [14]

$\Phi(\hat{O}(\hat{q},\hat{p}))=O_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})=\int
d^{N}\mathfrak{u}\hskip
3.41432pt\exp(-\frac{i}{\hbar}\mathbf{\mathfrak{p}}\cdot\mathbf{\mathfrak{u}})\left\langle\mathbf{\mathfrak{q}}+\frac{\mathbf{\mathfrak{u}}}{2}\Big{|}\hat{O}(\hat{q},\hat{p})\Big{|}\mathbf{\mathfrak{q}}-\frac{\mathbf{\mathfrak{u}}}{2}\right\rangle$
(18)

where

$\Phi(\hat{q})=\mathfrak{q}$ (19) $\Phi(\hat{p})=\mathbf{\mathfrak{p}}$ (20)

are the corresponding symbols of position and momentum operators. The Wigner
distribution $W(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})$ defined as the
Weyl symbol of the density operator $\hat{\rho}$ by

$W(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})=\rho_{W}(\mathfrak{q},\mathfrak{p})=\frac{1}{(2\pi\hbar)^{f}}\int
d^{N}\mathfrak{u}\hskip
3.41432pt\exp(-\frac{i}{\hbar}\mathbf{\mathfrak{p}}\cdot\mathbf{\mathfrak{u}})\left\langle\mathbf{\mathfrak{q}}+\frac{\mathbf{\mathfrak{u}}}{2}\Big{|}\hat{\rho}(\hat{q},\hat{p})\Big{|}\mathbf{\mathfrak{q}}-\frac{\mathbf{\mathfrak{u}}}{2}\right\rangle$
(21)

is the corresponding phase-space distribution. This allows to compute averages
in a similar way as in classical mechanics:

$\left\langle\hat{O}\right\rangle=\int d^{N}\mathfrak{q}\hskip
1.42262ptd^{N}\mathfrak{p}\hskip 3.41432ptW(\mathfrak{q},\mathfrak{p})O_{W}.$
(22)

Since oOur purpose is to create an analogy between the classical theorem and
its counterpart in phase-space quantum mechanics, we notice that in the
particular case where we analyze the energetic distribution of the system.
Thus the Hamiltonian that appears in the classical version of the energy
equipartition theorem is the Hamiltonian of the system $\hat{H}_{S}$.
Therefore, it is of interest to calculate the same average in quantum
mechanics formulated in phase-space in order to create an analogy. To do that,
we consider $\hat{q_{i}}=\hat{q_{i}}\otimes\mathbb{I}$,
$\hat{p_{i}}=\hat{p_{i}}\otimes\mathbb{I}$ and
$\frac{\partial\hat{H}_{s}}{\partial\hat{q}_{i}}=\frac{\partial\hat{H}_{s}}{\partial\hat{q}_{i}}\otimes\mathbb{I}$,
$\frac{\partial\hat{H}_{s}}{\partial\hat{p}_{i}}=\frac{\partial\hat{H}_{s}}{\partial\hat{p}_{i}}\otimes\mathbb{I}$,
so we have

$\left<\hat{q_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}\otimes\mathbb{I}\right>=\text{Tr}_{S}\left(\hat{\rho}_{S}\hat{q_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}\right)=\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\left(\hat{q_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}\right)_{W}\hskip
1.42262pt$ (23)
$\left<\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\otimes\mathbb{I}\right>=\text{Tr}_{S}\left(\hat{\rho}_{S}\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\right)=\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\left(\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\right)_{W}\hskip
1.42262pt$ (24)

Where $W_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})$ is the Wigner
distribution associated with the reduced density matrix of the system.

A Weyl symbol of particular interest is the one associated to the product of
two operators given by [14]

$\begin{split}\Phi(\hat{A}(\hat{q},\hat{p})\hat{B}(\hat{q},\hat{p}))=A_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\star_{M}B_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\\
=A_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\text{exp}\left(\frac{i\hbar}{2}\left(\overleftarrow{\partial_{\mathfrak{q}}}\overrightarrow{\partial_{\mathfrak{p}}}-\overleftarrow{\partial_{\mathfrak{p}}}\overrightarrow{\partial_{\mathfrak{q}}}\right)\right)B_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\end{split}$
(25)

where $\overleftarrow{\partial_{\mathfrak{q}}}$ denotes the derivative with
respect to $\mathfrak{q}$ acting to the left and similarly for the other
derivatives. The $\star_{M}$ is called the Moyal product and allows one to
define the associated symbol for the commutator $\\{,\\}_{M}$ known as Moyal
bracket

$\begin{split}\Phi([\hat{A}(\hat{q},\hat{p}),\hat{B}(\hat{q},\hat{p})])=\\{A_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}}),B_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\}_{M}\\\
=\frac{2}{\hbar}A_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\text{sin}\left(\frac{\hbar}{2}\left(\overleftarrow{\partial_{\mathfrak{q}}}\overrightarrow{\partial_{\mathfrak{p}}}-\overleftarrow{\partial_{\mathfrak{p}}}\overrightarrow{\partial_{\mathfrak{q}}}\right)\right)B_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\end{split}$
(26)

which expanded is expressed as

$\begin{split}\\{A_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}}),B_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\}_{M}=2\sum_{s=0}^{\infty}\frac{(-1)^{s}}{(2s+1)!}\left(\frac{-\hbar}{2}\right)^{2s}\sum_{t=0}^{2s+1}\frac{(-1)^{t}(2s+1)!}{(2s+1-t)!t!}\\\
\times\left[\frac{\partial^{t}}{\partial\mathfrak{q}^{t}}\frac{\partial^{2s+1-t}A_{W}}{\partial\mathfrak{p}^{2s+1-t}}\right]\left[\frac{\partial^{2s+1-t}}{\partial\mathfrak{q}^{2s+1-t}}\frac{\partial^{t}B_{W}}{\partial\mathfrak{p}^{t}}\right]\\\
\\\
=\\{A_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}}),B_{W}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\}_{PB}+\mathcal{O}(\hbar^{2})\end{split}$
(27)

##  5 Version of the Theorem in Phase-Space Formulation of Quantum Mechanics

Proceeding in a similar way as in section 1, by assuming

$\hat{H}_{S}(\hat{q},\hat{p})=\hat{F}(\hat{q})+\hat{G}(\hat{p}),$ (28)

and considering the commutator for operators $\hat{A}(\hat{q},\hat{p})$ and
$\hat{B}(\hat{q},\hat{p})$ defined as

$[\hat{A},\hat{B}]=\hat{A}\hat{B}-\hat{B}\hat{A}$ (29)

the following identities can be demonstrated [15]

$[\hat{A}\hat{B},\hat{C}]=\hat{A}[\hat{B},\hat{C}]+[\hat{A},\hat{C}]\hat{B}$
(30)
$[\hat{q}_{i},\mathfrak{G}(\hat{p})]=i\hbar\frac{d\mathfrak{G}(\hat{p})}{d\hat{p}_{i}}$
(31)
$[\hat{p}_{i},\mathfrak{F}(\hat{q})]=-i\hbar\frac{d\mathfrak{F}(\hat{q})}{d\hat{q}_{i}}$
(32)

Where $\mathfrak{F}$ and $\mathfrak{G}$ are arbitrary functions of operators
$\hat{q}$ and $\hat{p}$. Thus, using equations 31 and 32, the quantum
mechanical analog of Hamilton equations can be stated as

$\frac{d\hat{q}_{i}}{dt}=\frac{\partial\hat{H}_{S}}{d\hat{p}_{i}}=\frac{1}{i\hbar}[\hat{q}_{i},\hat{H}_{S}]$
(33)
$\frac{d\hat{p}_{i}}{dt}=-\frac{\partial\hat{H}_{S}}{d\hat{q}_{i}}=\frac{1}{i\hbar}[\hat{p}_{i},\hat{H}_{S}]$
(34)

Similarly one arrives at

$\hat{q}_{i}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}=-\frac{\hat{q}_{i}}{i\hbar}[\hat{p}_{i},\hat{H}_{S}]=-\frac{\hat{q}_{i}}{i\hbar}[\hat{p}_{i},\hat{F}(\hat{q})]=-\frac{1}{i\hbar}[\hat{q}_{i}\hat{p}_{i},\hat{F}(\hat{q})]$
(35)

$\hat{p}_{i}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}=\frac{\hat{p}_{i}}{i\hbar}[\hat{q}_{i},\hat{H}_{S}]=\frac{\hat{p}_{i}}{i\hbar}[\hat{q}_{i},\hat{G}(\hat{p})]=\frac{1}{i\hbar}[\hat{p}_{i}\hat{q}_{i},\hat{G}(\hat{p})]$
(36)

By replacing in equations 23 and 24 the result is

$\begin{split}\left<\hat{q_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}\otimes\mathbb{I}\right>=-\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\left(\frac{1}{i\hbar}[\hat{q}_{i}\hat{p}_{i},\hat{F}(\hat{q})]\right)_{W}\\\
=-\int d^{N}\mathbf{\mathfrak{q}}\hskip
1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\{\mathfrak{q}_{i}\mathfrak{p}_{i},F(\mathfrak{q})\\}_{M}\end{split}$
(37)

$\begin{split}\left<\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\otimes\mathbb{I}\right>=\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\left(\frac{1}{i\hbar}[\hat{p}_{i}\hat{q}_{i},\hat{G}(\hat{p})]\right)_{W}\\\
=\int d^{N}\mathbf{\mathfrak{q}}\hskip
1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\{\mathfrak{p}_{i}\mathfrak{q}_{i},G(\mathfrak{p})\\}_{M}\end{split}$
(38)

Expanding the Moyal bracket as in 27 and noticing that the $O(\hbar^{2})$
vanish since derivatives of third-order or higher cancel out one arrives to

$\begin{split}\left<\hat{q_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}\otimes\mathbb{I}\right>=-\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\{\mathfrak{q}_{i}\mathfrak{p}_{i},F(\mathfrak{q})\\}_{M}\\\
=-\int d^{N}\mathbf{\mathfrak{q}}\hskip
1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\{\mathfrak{q}_{i}\mathfrak{p}_{i},F(\mathfrak{q})\\}_{PB}\\\
\end{split}$ (39)

$\begin{split}\left<\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\otimes\mathbb{I}\right>=\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\{\mathfrak{p}_{i}\mathfrak{q}_{i},G(\mathfrak{p})\\}_{M}\\\
=\int d^{N}\mathbf{\mathfrak{q}}\hskip
1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\{\mathfrak{p}_{i}\mathfrak{q}_{i},G(\mathfrak{p})\\}_{PB}\end{split}$
(40)

TAs we will show in section 7, these equations 39 and 40 constitute the energy
equipartition theorem for the system in the phase-space formulation of quantum
mechanics as an analogy to the formulation in classical mechanics in the weak
coupling limit as a consequence of the high-temperature regime. Notice the
explicitly the resemblance with the classical version as shown in equations 1
and 2. The second terms of equations 37 and 38 vanish since derivatives of
third-order or higher cancel out.

##  6 Application to the Quantum Harmonic Oscillator

As an illustration, let us consider the system to be a harmonic oscillator in
the high and low-temperature regime. In the first case, according to equation
15, we do not need to specify the interaction of the environment-
system.Exactly the same result can be obtained for the environment, but with
the Wigner distribution function associated to the reduced density matrix of
the bath.

##  7 Application to the Quantum Harmonic Oscillator

As an illustrative example of the proposed method in the article for
calculating the energy, let us consider the system to be a harmonic oscillator
in the high and low-temperature regimes. First, we calculate the Wigner
distribution associated with the reduced density matrix: in the weak coupling
limit

$W_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})=\frac{1}{Z_{S}}\frac{1}{2\pi\hbar}\int
du\hskip
3.41432pt\exp(-\frac{i}{\hbar}\mathbf{\mathfrak{p}}\cdot\mathbf{\mathfrak{u}})\left\langle\mathbf{\mathfrak{q}}+\frac{\mathbf{\mathfrak{u}}}{2}|\exp(-\beta\hat{H}_{S})|\mathbf{\mathfrak{q}}-\frac{\mathbf{\mathfrak{u}}}{2}\right\rangle$
(41)

with $\hat{H}_{S}=\frac{1}{2m}\hat{p}^{2}+\frac{1}{2}m\omega^{2}\hat{q}^{2}$.
As we said in section 2, the matrix elements of the density operator involved
in the integral can be calculated in the imaginary time path integral
formalism, i.e, we use the expression of the propagator of a harmonic
oscillator and we replace $t=-i\hbar\beta$. We do this in the expression
provided by [15] to get
$\left\langle\mathbf{\mathfrak{q}}+\frac{\mathbf{{\mathfrak{u}}}}{2}\Big{|}\text{exp}(-\beta\hat{H}_{S})\Big{|}\mathbf{{\mathfrak{q}}}-\frac{\mathbf{{\mathfrak{u}}}}{2}\right\rangle=\sqrt{\frac{m\omega}{2\pi\hbar\text{sinh}(\hbar\beta\omega)}}\text{exp}\big{(}-\frac{m\omega}{\hbar}q^{2}\text{tanh}(\frac{\hbar\omega\beta}{2})-\frac{m\omega}{4\hbar}\mathbf{\mathfrak{u}}^{2}\text{coth}(\frac{\hbar\omega\beta}{2}))$
and $Z_{S}=(2\text{sinh}(\frac{\hbar\omega\beta}{2}))^{-1}$. Finally, we
replace this in equation 41 and get

$\begin{split}W_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})=\frac{1}{\pi\hbar}\tanh\left(\frac{\hbar\omega\beta}{2}\right)\text{exp}\left(-\frac{\tanh(\frac{\hbar\omega\beta}{2})}{\omega\hbar}\left(\frac{\mathbf{\mathfrak{p}}^{2}}{m}+m\omega^{2}\mathbf{\mathfrak{q}}^{2}\right)\right)\end{split}$
(42)

because we have used the result of the reduced density matrix in the high-
temperature limit, we need to impose the same limit in equation 42. In that
case, we find that the Wigner distribution function is just the reduced
density matrix of the system when we replace the operators $\hat{p}$ and
$\hat{q}$ for their corresponding symbols $p$ and $q$In the high-temperature
regime, we find that the Wigner distribution function is just the reduced
density matrix of the system, i.e the classical phase-space density in the
canonical ensemble of a harmonic oscillator,

$\begin{split}W_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})=\frac{1}{Z}\text{exp}\left(-\beta\left(\frac{\mathbf{\mathfrak{p}}^{2}}{2m}+\frac{m\omega^{2}}{2}\mathbf{\mathfrak{q}}^{2}\right)\right)=\frac{\text{exp}\left(-\beta
H_{S}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\right)}{Z}\end{split}$ (43)

Where $Z=\frac{2\pi}{\beta\omega}$. Finally, because equation 43 corresponds
exactly with the probability density function for the canonical ensemble as in
the case of classical mechanics, equations 39 and 40 are equal to $k_{B}T$
using equations 12 and 13.

In the low -temperature regime, formally the description of the system is
given by equation 15 and strictly speaking the equation 42 is no longer valid.
However, if we suppose an interaction between the environment and the system
such that $\left[\hat{H}_{S},\hat{V}\right]=0$, the system is even well
described by the reduced density matrix as in the high temperature regime [10]
and therefore equation 42 is valid. Then,

$\begin{split}\left<\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\otimes\mathbb{I}\right>=\lim_{\beta\rightarrow\infty}\int
d\mathbf{\mathfrak{q}}\hskip 1.42262ptd\mathbf{\mathfrak{p}}\hskip
3.41432ptW_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})\\{\mathfrak{p}_{i}\mathfrak{q}_{i},G(\mathfrak{p})\\}_{M}\\\
=\int d\mathbf{\mathfrak{q}}\hskip 1.42262ptd\mathbf{\mathfrak{p}}\hskip
3.41432pt\frac{1}{\pi\hbar}\exp\left(-\frac{2H_{S}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})}{\omega\hbar}\right)\\{\mathfrak{p}_{i}\mathfrak{q}_{i},G(\mathfrak{p})\\}_{PB}=\frac{\hbar\omega}{2}\end{split}$
(44)

where in the last equality we have used equation 13 and where $Z_{mod}=\int
d\mathbf{q}\hskip 1.42262ptd\mathbf{p}\hskip
3.41432pt\exp(-\beta_{mod}H_{S}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}q},\mathbf{p}))=\pi\hbar$,
and $\beta_{mod}=\frac{2}{\omega\hbar}$.

This simple example shows for this particular case, that if one expects to
create a true correspondence between the classical equipartition theorem
discussed in section 1, and the theorem in quantum mechanics phase-space
formulation, it must be done in the high-temperature regime because in the
low-temperature regime one gets the modified density distribution. Further,
for a model whose coordinates are bilinearly coupled as in [16], there are
found explicit expressions for equations 23 and 24 in both regimes for the
weak coupling limit, there it is shown that the energy is not equally
distributed for the damped harmonic oscillator. This can also be seen from
[17] where the Wigner distribution function for equations 40 and 39 does not
correspond with the Gibbs distribution.

##  8 Theorem in the Classical Limit

Motivated by this example, we expect that in the classical limit the Wigner
function described by equation 41 behaves as the density probability function
in the canonical ensemble for any system. Then,In the general case of non-
factorizing initial conditions in the high-temperature regime equation, 14
takes the form of Gibbs state and it can be shown [18] that in this limit the
Wigner distribution function coincides with the same distribution as in the
classical case,

$W_{s}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}})=\frac{\text{exp}(-\beta
H_{S}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}}))}{Z}$ (45)

Where in this case we have used the fact that the Wigner distribution function
must be normalized. Using this in equations 39 and 40, we have

$\begin{split}\left<\hat{q_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}\otimes\mathbb{I}\right>\approx-\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432pt\frac{\text{exp}(-\beta
H_{S}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}}))}{Z}\\{\mathfrak{p}_{i}\mathfrak{q}_{i},F(\mathfrak{q})\\}_{PB}\end{split}$
(46)

$\begin{split}\left<\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\otimes\mathbb{I}\right>\approx\int
d^{N}\mathbf{\mathfrak{q}}\hskip 1.42262ptd^{N}\mathbf{\mathfrak{p}}\hskip
3.41432pt\frac{\text{exp}(-\beta
H_{S}(\mathbf{\mathfrak{q}},\mathbf{\mathfrak{p}}))}{Z}\\{\mathfrak{p}_{i}\mathfrak{q}_{i},G(\mathfrak{p})\\}_{PB}\end{split}$
(47)

This is according to the classical theorem equal to $k_{B}T$ (see equation 12
and 13). Then

$\left<\hat{q_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{q}_{i}}\otimes\mathbb{I}\right>\approx
k_{B}T$ (48)
$\left<\hat{p_{i}}\frac{\partial\hat{H}_{S}}{\partial\hat{p}_{i}}\otimes\mathbb{I}\right>\approx
k_{B}T$ (49)

These equations represent what we expect to get in the classical limit, i.e,
the equipartition theorem. It must be stressed that this result has some
approximations, first restrictions, and limitations. First, in general, the
total state given by equation 14 must contain some additional conditions
resulting from experimentally achievable preparations [19], second the result
is derived based on an approximation method derived in [18], and second we
approximate the product $\hbar\beta$ tends to zero to guarantee that the state
is described by the Gibbs state. The latter is guaranteed formally when
$T\rightarrow\infty$ and as long as the second-order perturbation theory is
valid [10]. In this case, no matter what kind of interaction is described by
the system and the environment, we get 48 and 49. On the other hand, the
result is only valid when one works with separable Hamiltonians which is a
particular case of more general Hamiltonians presented in the literature [20].

## Acknowledgments

One of the authors (J. R.) acknowledges University of Antioquia for the
exclusive dedication program. Financial support was provided by the CODI-UdeA
2020-34211 Simulmag2 and 2017-16253 projects. (E. M. and A. R.) thank the
professors Leonardo Pachon and Juan David Jaramillo for enjoyable and
stimulating discussions.

## References

  * [1] Walter Greiner.  Thermodynamics and statistical mechanics.  Springer-Verlag, New York, 1995.
  * [2] Christian Enss and Siegfried Hunklinger.  Low-Temperature Physics.  Springer, 1 edition, 2005.
  * [3] J. Łuczka.  Quantum counterpart of classical equipartition of energy.  J Stat Phys, 179:839–845, 2020.
  * [4] P Bialas, J Spiechowicz, and J Łuczka.  Quantum analogue of energy equipartition theorem.  Journal of Physics A: Mathematical and Theoretical, 52(15):15LT01, mar 2019.
  * [5] R. Martens and A. Verbeure.  The equipartition for quantum systems.  Letters in Mathematical Physics, 3:413–418, 1979.
  * [6] Jasleen Kaur, Aritra Ghosh, and Malay Bandyopadhyay.  Quantum counterpart of energy equipartition theorem for a dissipative charged magneto-oscillator: Effect of dissipation, memory, and magnetic field.  Phys. Rev. E, 104:064112, Dec 2021.
  * [7] Jasleen Kaur, Aritra Ghosh, and Malay Bandyopadhyay.  Quantum counterpart of energy equipartition theorem for fermionic systems.  Journal of Statistical Mechanics: Theory and Experiment, 2022(5):053105, may 2022.
  * [8] C. W. Gardiner.  Quantum noise : a handbook of Markovian and non-Markovian quantum stochastic methods with applications to quantum optics.  Springer, Berlin, 2000.
  * [9] Maximilian Schlosshauer.  Decoherence and the Quantum-To-Classical Transition.  Springer Berlin Heidelberg, 2007.
  * [10] Leonardo A. Pachón, Johan F. Triana, David Zueco, and Paul Brumer.  Influence of non-markovian dynamics in equilibrium uncertainty-relations.  The Journal of Chemical Physics, 150(3):034105, jan 2019.
  * [11] Coherent Evolution in Noisy Environments.  Springer Berlin Heidelberg, December 2002.
  * [12] Russell P. Rundle and Mark J. Everitt.  Overview of the phase space formulation of quantum mechanics with application to quantum technologies.  Advanced Quantum Technologies, 4(6):2100016, 2021.
  * [13] Cosmas K Zachos, David B Fairlie, and Thomas L Curtright.  Quantum Mechanics in Phase Space.  WORLD SCIENTIFIC, 2005.
  * [14] Robert S. Wezeman.  Weyl quantization and wigner distributions on phase space.  2014\.
  * [15] J. J. Sakurai and Jim Napolitano.  Modern Quantum Mechanics.  Cambridge University Press, sep 2020.
  * [16] P. Bialas and J. Spiechowicz and J. Łuczka.  Partition of energy for a dissipative quantum oscillator.  Scientific Reports, 8(1), oct 2018.
  * [17] L.A. Pachón and G.-L. Ingold and T. Dittrich.  Nonclassical phase-space trajectories for the damped harmonic quantum oscillator.  Chemical Physics, 375(2-3):209–215, oct 2010.
  * [18] Yun-An Yan, Jian Liu, and Jiushu Shao.  A semiclassical initial-value representation for quantum propagator and boltzmann operator.  Journal of Computational Chemistry, 40(10):1161–1171, dec 2018\.
  * [19] Hermann Grabert and Peter Schramm and Gert-Ludwig Ingold.  Quantum Brownian motion: The functional integral approach.  Physics Reports, 168(3):115–207, oct 1988.
  * [20] F Nicacio.  Weyl – Wigner representation of canonical equilibrium states.  Journal of Physics A: Mathematical and Theoretical, 54(5):055004, jan 2021.


# GLoD: Composing Global Contexts and Local Details in Image Generation

Moyuru Yamada Fujitsu Limited<EMAIL_ADDRESS>
###### Abstract

Diffusion models have demonstrated their capability to synthesize high-quality
and diverse images from textual prompts. However, simultaneous control over
both global contexts (e.g., object layouts and interactions) and local details
(e.g., colors and emotions) still remains a significant challenge. The models
often fail to understand complex descriptions involving multiple objects and
reflect specified visual attributes to wrong targets or ignore them. This
paper presents Global-Local Diffusion (GLoD), a novel framework which allows
simultaneous control over the global contexts and the local details in text-
to-image generation without requiring training or fine-tuning. It assigns
multiple global and local prompts to corresponding layers and composes their
noises to guide a denoising process using pre-trained diffusion models. Our
framework enables complex global-local compositions, conditioning objects in
the global prompt with the local prompts while preserving other unspecified
identities. Our quantitative and qualitative evaluations demonstrate that GLoD
effectively generates complex images that adhere to both user-provided object
interactions and object details.

Figure 1: Global-Local Diffusion (GLoD) takes multiple prompts as an input
(e.g., a global prompt: ’a man is talking with a woman’ and two local prompts:
’a man with white beard’ and ’a woman is wearing a necklace and smiling’)
along with their layout and assigns noises obtained from them into
corresponding layers with a pre-trained diffusion model. Then, the noises are
effectively composed to generate an image. Details of objects in the global
prompt are guided with the corresponding local prompts.

##  1 Introduction

Text-to-image generative models have emerged recently and demonstrated their
amazing capabilities in synthesizing high-quality and diverse images from text
prompts. Diffusion models Dhariwal and Nichol (2021); Ho et al. (2020); Nichol
and Dhariwal (2021) are currently one of the state-of-the-art methods and
widely used for the image generation. Despite their impressive advances in
image generation, lack of control over the generated images is a crucial
limitation in deploying them to real-world applications.

To provide further controllability over diffusion models, researchers have put
a lot of effort into control of object layouts, object interactions, and
composition of objects. Training-free layout control Chen et al. (2023) takes
a text prompt along with the object layout as an input and control the object
position based on a loss between the input layout and attention maps.
MultiDiffusion Bar-Tal et al. (2023) places an object with specified details
on a certain region using segmentation masks and a prompt for each segment.
These methods work without requiring any additional training; however, they
struggle to control both the global contexts (e.g., object interactions) and
the local details (e.g., object colors and emotions) simultaneously. With a
complex prompt containing multiple objects, the models often misinterpret
specified local details, directing them to the wrong target or ignoring them,
similar to the issues observed in Stable Diffusion Rombach et al. (2022).
While splitting the complex prompt into multiple prompts allows the model to
depict each object more accurately, handling the prompts independently poses
limitations in addressing a global context that describes interactions and
relationships between the multiple objects.

Another trial is training a model from scratch or finetuning a given diffusion
model for better controllable generation with a task-specific annotation Li et
al. (2023b); Avrahami et al. (2023b); Xue et al. (2023); Zhang et al. (2023).
Some researchers have leveraged a scene graph as an input and introduced
models which generate images from the scene graph to control the object
interactions in the generated images Farshad et al. (2023) and Yang et al.
(2022). These methods have shown superior results, while they often require a
high computation cost and a long development period. This makes it difficult
to leverage various pre-trained diffusion models.

This paper proposes Global-Local Diffusion (GLoD), a novel diffusion framework
that controls both global contexts and local details simultaneously using a
pre-traind diffusion model without requiring any additional training or
finetuning. GLoD takes as input global prompts that describe entire image
including object interactions, and local prompts that specify object details
along with their position in the form of a bounding box. The diffusion model
predicts noises from the prompts separetely and then the noises are composed
to guide the denoising process. For example, instead of giving a complex
prompt such as ’a man with white beard is talking with a smiling woman wearing
a necklace’, we decompose it into multiple prompts, a global context ’a man is
talking with a woman’ and two local details ’a man with white beard’ and ’a
woman is wearing a necklace and smiling’. The details of the man and the woman
in the global prompt are guided with each local prompt in the image generation
process. GLoD also controls the object layout with the given bounding boxes.
The generated examples are as shown in Fig. 1.

Our framework enables both global-global compositions and global-local
compositions. The global-global composition composes foreground and background
similar to the existing method Liu et al. (2022), while the global-local
composition composes the global context and the object details. Since a local
prompt can be a global prompt for other local prompts, GLoD allows us to
compose more than two layers. In addition, unlike the existing methods that
may changes object identities even by just adding a single attribute, GLoD
only changes the object details specified by the corresponding local prompts
while preserving other identities. This feature enables users to control the
generated image interactively.

In order to assess the effectiveness of our proposed method quantitatively, we
build a new test set to evaluate the controllability over the global contexts
and local details in the image generation rather than the image quality.

Our key contributions are summarized as follows111The code will be made
publicly available upon acceptance.:

  * •

We propose Global-Local Diffusion (GLoD), a simple and yet effective framework
for diffusion-based image synthesis and editing which enables controlling both
global contexts and local details simultaneously.

  * •

Through quantitative and qualitative evaluations, we demonstrate that our
proposed method can effectively generate complex images by composing the
multi-prompts describing object interactions and the details.

Figure 2: GLoD enables controlling global contexts (interaction between a dog
and a man, their layouts) and local details (the dog is black, the man is
wearing a blue shirt) independently. Local details can be specified (black dog
$\rightarrow$ Husky dog) while preserving the global contexts. Note that this
is not image editing. We generate images from the text prompts and the layout.

##  2 Related Work

###  2.1 Diffusion Models

Diffusion models Dhariwal and Nichol (2021); Ho et al. (2020); Nichol and
Dhariwal (2021) has attracted a lot of attention as a promising class of
generative models that formulates the data generation process as an iterative
denoising procedure. The models take a Gaussian noise input
${\bm{x}}_{T}\sim\mathcal{N}(\textbf{0},{\bm{I}})$ and transform it into a
sample ${\bm{x}}_{0}$ through the series of gradual denoising steps $T$. The
sample should be distributed according to a data distribution $q$. Many
research works focus on improving the diffusion process to speed up the
sampling process while maintaining high sample quality Nichol and Dhariwal
(2021); Karras et al. (2022). The latent diffusion model Rombach et al. (2022)
has also been developed to address this issue and applied the diffusion
process in latent space instead of pixel space to enable an efficient
sampling. While the diffusion models have originally shown great performance
in image generation, enabling effective image editing and image inpainting
Meng et al. (2022); Avrahami et al. (2023a), these models been successfully
used in various domains, including video Ho et al. (2022), audio Chen et al.
(2021), 3D scenes Müller et al. (2023), and motion sequences Tevet et al.
(2023). Although this paper focuses on image generation, our proposed
framework may further be applied in such domains.

###  2.2 Controllable Image Generation with Diffusion

Diffusion models are first applied to text-to-image generative models, which
generate an image conditioned on a free-form text description as an input
prompt. Classifier-free guidance Ho and Salimans (2021) plays important role
in conditioning the generated images to the input prompt. Recent text-to-image
diffusion models such as DALL-E 2 Ramesh et al. (2022), Imagen Saharia et al.
(2022), and Stable Diffusion Rombach et al. (2022) has shown remarkable
capabilities in image generation. On the other hand, recent studies Chen et
al. (2023); Bar-Tal et al. (2023); Zheng et al. (2023); Patashnik et al.
(2023) have stressed the inherent difficulty in controlling generated images
with a text description, especially in the control over (i) object layout and
(ii) visual attributes of objects. To gain more control over the object
layout, some works have leveraged bounding boxes or segmentation masks as an
additional input along with text prompts. Training-free layout control Chen et
al. (2023) takes a single prompt along with a layout of objects appeared in
the prompt as shown in Fig. 2 (b). Object layout is given in a form of the
bounding. Layout control extracts attention maps from a pre-trained diffusion
model and updates the latent embeddings of the image based on an error between
the input bounding boxes and the attention maps. Since this method simply uses
the pre-trained diffusion model to generate images, it inherits the
difficulties in control over the visual attributes of objects, i.e., the local
details. Also, the generated image may largely change even if a single word is
added or replaced in the prompt due to the fact that the input prompt
describes both the global contexts and the local details. MultiDiffusion Bar-
Tal et al. (2023) and SceneComposer Zeng et al. (2023) take multiple prompts
along with their corresponding segmentation masks as a region as shown in Fig.
2 (c). They effectively control the object layout and visual attributes of
each object. However, they cannot handle a prompt describing interactions
between those objects, i.e., the global contexts. Basically, they just place a
specific object described by the prompt in a certain region. Thus, if the
input prompt is replaced (black dog $\rightarrow$ Husky dog), the new object
(Huskey dog) does not inherit the contexts (e.g., posture) from the replaced
object (black dog). Unlike these methods, our method aims to control both the
global contexts and the local details simultaneously. Since we treat the
global contexts and the local details separately, the global contexts are
preserved even if the local details are changed, as shown in Fig. 2 (d). Some
studies Farshad et al. (2023); Yang et al. (2022) focused on image synthesis
from scene graphs for better control over the complex relations between
multiple objects in the generated images. However, these works require costly
extensive training on curated datasets. They regard a complex scene graph as
an input prompt, while our approach decomposes the complex prompts into
multiple simple prompts and does not require any training or finetuning.

Figure 3: GLoD composes multiple layers. Unconditional noise and noises
conditioned on global contexts (e.g., interactions) or local details (e.g.,
color) are assigned to separate layers ($l_{0}$, $l_{1}$, $l_{2}$). Those
layers are then composed with global guidance $g_{g}$ and local guidance
$g_{l}$.

###  2.3 Layered Image Generation and Editing

Some recent works Zhang et al. (2023); Li et al. (2023a); Liao et al. (2023)
have proposed layered image generation and editing. They considers two layers,
foreground and background, and enables to control them individually with a
segmentation mask of the foreground object. Their models are needed to be
trained with proposed losses. Unlike them, our goal is to control the global
contexts and the local details simultaneously without requiring the training
and the accurate segmentation masks as shown in Fig. 1. Our framework enables
to control local details while keeping the global contexts by composing the
multiple layers, where the global layer may represent the foreground or
background and the local layer may represent the details of the objects in the
global layer. The global layer and the local layer are not independent but
have a whole-part relationship. Our framework can also handle more than two
layers as shown in Fig. 7.

###  2.4 Compositional Generation

The compositional generation is an approach to generate the complex images by
composing a set of diffusion models, with each of them modeling a certain
component of the image. This approach has been an essential direction for
image-to-text models because it is difficult for the current models to handle
complex prompts where multiple concepts are squeezed. Recently, Liu et al.
(2022) has demonstrated successful composition of independent concepts (e.g.,
“a bear” and “in a forest”) by adding estimated score for each concept. Feng
et al. (2023) has also proposed another approach which can be directly merged
into the cross-attention layers. Inspired by the first approach, we propose a
novel method to compose whole-part concepts (e.g., ”a bear is eating an apple”
and ”the apple is green”).

##  3 Method

Our goal is to generate images where given global contexts and local details
are reflected. In this section, we introduce Global-Local Diffusion (GLoD) to
compose the global context and the local detail with pre-trained diffusion
models.

###  3.1 Compositions of Diffusion Models

We consider a pre-trained diffusion model, which takes a text prompt
$y\in\mathcal{Y}$ as a condition and generates a intermediate image
${\bm{x}}_{t}\in\mathcal{I}=\mathbb{R}^{H{\times}W{\times}C}$:

${\bm{x}}_{t-1}=\Phi({\bm{x}}_{t}|y).$ (1)

The diffusion models are also regarded as Denoising Diffusion Probabilistic
Models (DDPMs) where generation is modeled as a denoising process. The
objective of this model is to remove a noise gradually by predicting the noise
at a timestep $t$ given a noisy image $x_{t}$. To generate a less noisy image,
we sample $x_{t-1}$ until it becomes realistic over multiple iterations:

${\bm{x}}_{t-1}={\bm{x}}_{t}-\epsilon_{\theta}({\bm{x}}_{t},t)+\mathcal{N}(0,{\sigma_{t}}^{2}I),$
(2)

where $\epsilon_{\theta}({\bm{x}}_{t},t)$ is the denoising network. Liu et al.
(2022) has revealed that the denoising network or score function can be
expressed as a compositions of multiple score functions corresponding to an
individual condition $c_{i}$.

$\hat{\epsilon}({\bm{x}}_{t},t)=\epsilon_{\theta}({\bm{x}}_{t},t)+\sum_{i=0}^{n}{w_{i}(\epsilon_{\theta}({\bm{x}}_{t},t|c_{i})-\epsilon_{\theta}({\bm{x}}_{t},t))}.$
(3)

where $\epsilon_{\theta}({\bm{x}}_{t},t|c_{i})$ predicts a noise conditioned
on $c_{i}$ and $\epsilon_{\theta}({\bm{x}}_{t},t)$ outputs an unconditional
noise. This equation only focuses on composing individual conditions over
entire image, e.g., a foreground condition like ’a boat at the sea’ and a
background condition like ’a pink sky’.

We regard
$\epsilon_{\theta}({\bm{x}}_{t},t|c_{i})-\epsilon_{\theta}({\bm{x}}_{t},t)$ as
a guidance $g_{i}$, which guides the unconditional noise toward the noise
conditioned on a given condition $c_{i}$. Then, the composed denoising network
is viewed as a composition of the guidance.

$\hat{\epsilon}({\bm{x}}_{t},t)=\epsilon_{\theta}({\bm{x}}_{t},t)+\sum_{i=0}^{n}{w_{i}g_{i}}.$
(4)

Algorithm 1 GLoD sampling.

1:Diffusion model $\epsilon_{\theta}({\bm{x}}_{t},t)$, global scales $w_{i}$,
local scales $w_{j}$, global conditions $c_{gi}$, local conditions $c_{lj}$,
object region masks $M_{j}$

2:Initialize sample ${\bm{x}}_{t}\sim\mathcal{N}(\textbf{0},{\bm{I}})$

3:for $t=T,\ldots,1$ do

4: ${\bm{x}}_{t}\leftarrow f({\bm{x}}_{t},c_{gi},M)$ $\triangleright$ apply
layout control $f$

5: $\epsilon_{i}\leftarrow\epsilon_{\theta}({\bm{x}}_{t},t|c_{gi})$
$\triangleright$ scores for global condition $c_{gi}$

6: $\epsilon_{j}\leftarrow\epsilon_{\theta}({\bm{x}}_{t},t|c_{lj})$
$\triangleright$ scores for local condition $c_{lj}$

7: $\epsilon\leftarrow\epsilon_{\theta}({\bm{x}}_{t},t)$$\triangleright$
unconditional score

8: $\epsilon_{b}\leftarrow\epsilon_{i},\epsilon_{j}$ $\triangleright$ Assign
$\epsilon_{i}$ and $\epsilon_{j}$ to $\epsilon_{b}$

9:
$g_{g}\leftarrow\sum_{i=0}^{k}{w_{i}(\epsilon_{i}-\epsilon)}.$$\triangleright$
global guidance-Eq. 5

10:
$g_{l}\leftarrow\sum_{j=0}^{m}{w_{j}M_{j}(\epsilon_{j}-\epsilon_{b})}.$$\triangleright$
local guidance-Eq. 6

11:
${\bm{x}}_{t-1}\sim\mathcal{N}({\bm{x}}_{t}-(\epsilon+g_{g}+g_{l}),{\sigma_{t}}^{2}I$)
$\triangleright$ sampling

12:end for

###  3.2 Layer Composition

We propose GLoD to extend the above concept to a composition of a global
condition and a local conditions, i.e., interactions between objects and
object details. We consider a set of global conditions
$c_{g}=(c_{g1},...,c_{gk})$ and a set of local conditions
$c_{l}=(c_{l1},...,c_{lm})$. We also introduce a diffusion layer
$l=(l_{0},...,l_{t})$, where each layer contains one or more noises derived
from the corresponding prompt as shown in Fig. 3. For example, with given a
global prompt ($c_{g1}$) and two local prompts ($c_{l1}$ and $c_{l2}$), an
unconditional noise can be assigned on a layer $l_{0}$, a layer $l_{1}$
contains a noise derived from the global prompt, and layer $l_{2}$ contains
noises obtained from the local prompts. We compose the assigned noises with
two ways of guidance: (i) global guidance, which guides the image with global
conditions by the following equation:

$g_{g}=\epsilon_{\theta}({\bm{x}}_{t},t|c_{g})-\epsilon_{\theta}({\bm{x}}_{t},t),$
(5)

where the unconditional noise is always a base noise $\epsilon_{b}$. This is
also well known as Classifier-free guidance Ho and Salimans (2021). With two
global conditions, their global guidance is summed as a global-global
composition. The classifier-free guidance works well on the global-global
compositions, while it does not work effectively when we compose the global
condition and the local condition since their conditions have some overlap.
Thus, we newly propose (ii) local guidance, which guides an object on the base
layer $b$ conditioned on a condition $c_{b}$ with a local condition $c_{j}$.

$g_{l}=M_{j}(\epsilon_{\theta}({\bm{x}}_{t},t|c_{j})-\epsilon_{\theta}({\bm{x}}_{t},t|c_{b})),$
(6)

where $M_{j}\subset\\{0,1\\}^{H{\times}W}$ is a region mask of $j$-th region
corresponding to the condition $c_{j}$. In Fig. 3, two local guidance is added
to the global guidance as a global-local composition. The intuition behind the
local guidance is that an image region guided with a word ’dog’ in a global
prompt can be regarded as an unconditional ’dog’ and guided with a local
prompt by emphasizing the difference between the global and local conditions.
In the end, decomposed global prompts and local prompts are effectively
composed by our proposed guidance.

Figure 4: GLoD for a single object. The images in the first column and ’Local’
columns are sampled only from the global context (global images) and the local
detail (local images) as an input prompt, respectively. The images in
’Composed’ columns are sampled using our method, which effectively applies
local detail (e.g., long-haired) to the object in the image while preserving
the global contexts (i.e., object layouts and object postures).  Figure 5:
GLoD for multiple objects. Our method (e) can control attributes of each
sheep, while the other methods fail to reflect the specified attributes to the
correct targets.

###  3.3 Layout Control

Without any layout control, objects described in a given prompt appear
somewhere in a generated image. To effectively compose the global noise and
the local noise, we use Training-free layout control Chen et al. (2023). More
specifically, we use the backward guidance to control the layout of the
objects in their layer before computing the global noise.

Algorithm 1 provides the pseudo-code for composing diffusion noises with GLoD.
Our method composes noises obtained with pre-trained diffusion models during
inference without any additional training or finetuning.

##  4 Results

###  4.1 Evaluation Metrics

We build a new test set to evaluate the controllability over the global
contexts and local details in the image generation rather than the image
quality. The test set contains 2500 samples where each sample contains a full
text (e.g., ’a beard man is talking to a woman with earrings.’), a global text
(i.e., ’a man is talking to a woman.’), local texts for a subject and an
object (’a beard man.’ and ’a woman with earrings.’), and a layout of the
subject and the object. This test set design allows us to compute an alignment
score using CLIP similarity Radford et al. (2021). We compute a global
alignment score $S_{g}$ from an entire image and the global text, and
similarly a local alignment score for the subject $S_{ls}$ and the object
$S_{lo}$ from a region of them in the image and the corresponding local text.
We also introduce an infection score $S_{i}$ that indicates how much
undesirable effects the subject and object have. This can be an important
metric since a prompt for an object A may also unintentionally affect another
object B. We compute the similarity between a region of A and a prompt for B,
similarly for the region of B and the prompt for A, and average them. See more
details in Appendix A.

###  4.2 Implementation Details

We evaluate our method on the following conditions. In all experiments, we
used Stable Diffusion Rombach et al. (2022) as our diffusion model, where the
diffusion process is defined over a latent space $I=R^{64\times 64\times 4}$,
and a decoder is trained to reconstruct natural images in higher resolution
$[0,1]^{512\times 512\times 3}$. We use the public implementation of Stable
Diffusion by HuggingFace, specifically the Stable Diffusion v2.1 trained on
the LAION-5B dataset Schuhmann et al. (2022) as the pre-trained image
generation model. We also set Euler Discrete Scheduler Karras et al. (2022) as
the noise scheduler. As our layout control (see 3.3) we use the backward
guidance Chen et al. (2023). All the experiments are running on one A30 GPU.

We compare our GLoD with other state-of-the-art training-free methods,
including Training-free layout control Chen et al. (2023) and MultiDiffusion
Bar-Tal et al. (2023), and strong baselines, including OpenAI DALL-E 2 Ramesh
et al. (2022) and Adobe Firefly Adobe (2023). We use the publicly available
official codes and websites, and follow their instructions.

###  4.3 Image Generation with GLoD

GLoD for a single object. We first demonstrate global-local compositions for a
single object using GLoD for a better understanding, while our main targets
are more complex scenes including multiple objects as shown in Fig. 1. In Fig.
4, our method generates diverse samples which comply with compositions of a
global context (e.g., a cat is walking) and a local detail (e.g., a walking
cat is pink). The images in the first column and ’Local’ columns are sampled
only from the global context (global images) and the local detail (local
images) as an input prompt, respectively. The images in ’Composed’ columns are
sampled using our method, where the goal is to apply local detail (i.e.,
specified visual attribute) to the object in the image while preserving the
global contexts. Although we generated all the images with the same seed, the
posture of the cat is largely different in the corresponding global image and
local image (e.g., the image of ’a cat is sitting’ vs the image of ’a sitting
cat is pink’). Our method can effectively generate the images from the global
context and the local detail along with the layout (see ’Composed’). Note that
this is image generation not the editing. The generated image retains most of
the global contexts, including postures and head directions. In a few cases,
the visual attribute of the object changes only partially (e.g., composition
of ’a cat is lying’ and ’a lying cat is yellow’).

GLoD for multiple objects. We then compare GLoD with the other state-of-the-
art methods in generating more complex scene including multiple objects. In
Fig. 5, we try to generate images of a complex scene where there are multiple
objects in the same category (sheep in this case) and each of them has
different attributes (a sheep is black and standing, another sheep is white
and sitting). We show first four images generated by official web application
of (a) DALL-E 2 and (b) Firefly at first and second column, respectively. They
can generate high-quality images, but they often fail to reflect the specified
attributes to the correct targets. We then find four seeds which generate
failure samples of Training-free layout control and generate images using
Stable diffusion and our method with those seeds. Both the layout control and
our method use the same object layout as an additional input. We set ’a black
sheep and another white sheep’ as a global context, ’a sheep is black and
standing’ as a local detail of a sheep, and ’a sheep is white and sitting’ as
a local detail of the another sheep. We make the global context similar to the
original prompt to compare the generated images easily. Figure 5 shows that
our method effectively controls the attributes of each object in the image.

Quantitative evaluation. Table 1 shows quantitative evaluation of
controllability over the global context and the local details. Our method
improves the local alignment scores $S_{ls}$ and $S_{lo}$ while keeping the
global alignment score $S_{g}$ almost the same. Compared to Multi Diffusion,
the proposed method shows a superior overall alignment score $S_{gl}$. Note
that Multi Diffusion cannot handle the global prompt describing the object
interaction, and thus shows significantly lower global alignment score. The
local alignment score for an object $S_{lo}$ lags considerably behind that for
a subject $S_{ls}$. We found that the subjects (e.g., woman) are often turning
their backs on as a result of complying with the given global prompt (e.g., a
man is talking with a woman). Therefore, the alignment score $S_{lo}$ becomes
low because some of specified attributes are not visible in such cases. GLoD
also improves the infection score $S_{i}$ against the baselines. This result
indicates that our method reduces the mis-alignment between the prompt and the
generated image.

GLoD for complex scenes. Figure 1 shows other samples depicting more complex
scenes, where we give an interaction between the objects as a global context
and also specify the local details. Instead of giving a complex prompt such as
’a Husky wearing glasses is playing with a black short-hair cat’, we decompose
and handle them separately (i.e., ’a Husky is playing with a cat’, ’a Husky
wearing glasses’, and ’a cat is short-hair and black’) to effectively
synthesis complex visual scenes.

Methods |  $S_{g}$ $\uparrow$ |  $S_{ls}$ $\uparrow$ |  $S_{lo}$ $\uparrow$ |  $S_{gl}$ $\uparrow$ |  $S_{i}$ $\downarrow$
---|---|---|---|---|---
Stable Diffusion | 24.2 | $-$ | $-$ | $-$ | $-$
Layout control | 24.4 | 21.2 | 20.5 | 22.6 | 15.6
GLoD (ours) | 24.6 | 23.3 | 20.9 | 23.3 | 14.5
MultiDiffusion | 21.2 | 24.5 | 24.2 | 22.7 | 14.9
Table 1: Evaluation of controllability over the global context ($S_{g}$) and
the local details for a subject and an object ($S_{ls}$ and $S_{lo}$).
$S_{gl}$ represents an average of the global alignment score $S_{g}$ and the
local alignment scores $S_{l\\{s,o\\}}$. $S_{i}$ denotes an infection score
that indicates how much undesirable effects the subject and the object have.

###  4.4 Global-Local Composition

Figure 6: Comparison between our global-local composition (bottom) and the
conventional composition (top). Our method can change the detail of the object
while preserving the global context by composing two prompts, whereas the
conventional method often fails because they regard the prompts as two
independent concepts.

We compare our global-local composition with a conventional composition Liu et
al. (2022). Since the conventional composition aims to compose independent
concepts (e.g., foreground and background) by adding estimated score for each
concept, it often fails to compose overlapped concepts (e.g., ’running cat’
and ’white cat’) as shown in Fig. 6 (top). Our layer composition can
effectively compose such overlapped concepts as shown in Fig. 6 (bottom).

###  4.5 Image Editing with GLoD

Figure 7: GLoD also enables layered image editing, where new objects can be
added on a certain region using additional prompts. Final image (right end)
can be generated in one inference by composing six prompts.

Figure 7 shows edited image samples with GLoD. GLoD enables layered image
editing, where new objects can be added on a certain region using additional
prompts. Details of the objects on a base layer (e.g., earring) can be guided
with the prompts on the upper layer (e.g., adding a diamond) with our layer
composition. Final image (right end) can be generated in one inference by
composing multiple prompts (six in this case).

##  5 Conclusion

Image generation with simultaneous control over global contexts and local
details is still an open challenge. We proposed GLoD, a simple and yet
effective framework which composes a global prompt describing an entire image
and local prompts specifying object details with a pre-trained diffusion
model. Our framework can handle both global-global compositions and global-
local compositions without requiring any additional training or finetuning.
Through the qualitative and quantitative evaluations, we demonstrated that
GLoD effectively generates images that include interactions between objects
with detailed visual control, improving the alignment scores and reducing the
undesirable affects. A limitation we found is that the object appearance may
change only partially when the latent of the object is significantly different
between the global and the local.

## References

  * Adobe [2023] Adobe.  Adobe firefly, 2023.
  * Avrahami et al. [2023a] Omri Avrahami, Ohad Fried, and Dani Lischinski.  Blended latent diffusion.  ACM Transactions on Graphics, 42(4):1–11, jul 2023.
  * Avrahami et al. [2023b] Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, and Xi Yin.  Spatext: Spatio-textual representation for controllable image generation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18370–18380, June 2023.
  * Bar-Tal et al. [2023] Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel.  Multidiffusion: Fusing diffusion paths for controlled image generation.  In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 1737–1752. PMLR, 2023.
  * Chen et al. [2021] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan.  Wavegrad: Estimating gradients for waveform generation.  In International Conference on Learning Representations, 2021.
  * Chen et al. [2023] Minghao Chen, Iro Laina, and Andrea Vedaldi.  Training-free layout control with cross-attention guidance.  arXiv preprint arXiv:2304.03373, 2023.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Quinn Nichol.  Diffusion models beat GANs on image synthesis.  In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
  * Farshad et al. [2023] Azade Farshad, Yousef Yeganeh, Yu Chi, Chengzhi Shen, Björn Ommer, and Nassir Navab.  Scenegenie: Scene graph guided diffusion models for image synthesis, 2023.
  * Feng et al. [2023] Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang.  Training-free structured diffusion guidance for compositional text-to-image synthesis.  In The Eleventh International Conference on Learning Representations, 2023.
  * Ho and Salimans [2021] Jonathan Ho and Tim Salimans.  Classifier-free diffusion guidance.  In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6840–6851. Curran Associates, Inc., 2020.
  * Ho et al. [2022] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans.  Imagen video: High definition video generation with diffusion models, 2022.
  * Karras et al. [2022] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine.  Elucidating the design space of diffusion-based generative models.  In Proc. NeurIPS, 2022.
  * Li et al. [2023a] Pengzhi Li, QInxuan Huang, Yikang Ding, and Zhiheng Li.  Layerdiffusion: Layered controlled image editing with diffusion models, 2023.
  * Li et al. [2023b] Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee.  Gligen: Open-set grounded text-to-image generation.  CVPR, 2023.
  * Liao et al. [2023] Ting-Hsuan Liao, Songwei Ge, Yiran Xu, Yao-Chih Lee, Badour AlBahar, and Jia-Bin Huang.  Text-driven visual synthesis with latent diffusion prior, 2023.
  * Liu et al. [2022] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum.  Compositional visual generation with composable diffusion models.  In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVII, pages 423–439. Springer, 2022.
  * Meng et al. [2022] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.  SDEdit: Guided image synthesis and editing with stochastic differential equations.  In International Conference on Learning Representations, 2022.
  * Müller et al. [2023] Norman Müller, Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulo, Peter Kontschieder, and Matthias Nießner.  Diffrf: Rendering-guided 3d radiance field diffusion.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4328–4338, 2023.
  * Nichol and Dhariwal [2021] Alexander Quinn Nichol and Prafulla Dhariwal.  Improved denoising diffusion probabilistic models.  In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8162–8171. PMLR, 18–24 Jul 2021.
  * Patashnik et al. [2023] Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch-Elor, and Daniel Cohen-Or.  Localizing object-level shape variations with text-to-image diffusion models, 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision.  In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748–8763. PMLR, 18–24 Jul 2021.
  * Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents, 2022.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo-Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi.  Photorealistic text-to-image diffusion models with deep language understanding.  In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
  * Schuhmann et al. [2022] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev.  LAION-5b: An open large-scale dataset for training next generation image-text models.  In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
  * Tevet et al. [2023] Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano.  Human motion diffusion model.  In The Eleventh International Conference on Learning Representations, 2023.
  * Xue et al. [2023] Han Xue, Zhiwu Huang, Qianru Sun, Li Song, and Wenjun Zhang.  Freestyle layout-to-image synthesis.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14256–14266, June 2023.
  * Yang et al. [2022] Ling Yang, Zhilin Huang, Yang Song, Shenda Hong, Guohao Li, Wentao Zhang, Bin Cui, Bernard Ghanem, and Ming-Hsuan Yang.  Diffusion-based scene graph to image generation with masked contrastive pre-training, 2022.
  * Zeng et al. [2023] Yu Zeng, Zhe Lin, Jianming Zhang, Qing Liu, John Collomosse, Jason Kuen, and Vishal M. Patel.  Scenecomposer: Any-level semantic image synthesis.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22468–22478, June 2023.
  * Zhang et al. [2023] Xinyang Zhang, Wentian Zhao, Xin Lu, and Jeff Chien.  Text2layer: Layered image generation using latent diffusion model, 2023.
  * Zheng et al. [2023] Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li.  Layoutdiffusion: Controllable diffusion model for layout-to-image generation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22490–22499, June 2023.

Figure 8: An example in our test set. We give a subject and an object with
their location, their attributes, and an interaction between them.

##  Appendix A Test set

We generated 2500 samples for testing the controllability over the global
contexts and local details in the image generation. Fig. 8 shows an example in
our test set. Each sample contains a full text, a global text, and a local
text for a subject and an object along with their layout. The samples are
automatically generated based on a template, where we give a subject text, an
object text, their details, a layout, and an interaction between them as an
input. Since the object layout control is not our main focus, we fixed the
object layout the same as show in Fig. 8.

Figure 9: Some examples where an object is turning their backs on as a result
of complying with the given global context (Left). This does not happen with
MultiDiffusion since it does not consider the global context (Right). Figure
10: GLoD controls the local detail while keeping the global context and the
unspecified identities. The visual attribute of the man changes depending on
the local text, but the object layout and his posture are almost the same.

##  Appendix B Generated samples

Figure 9 shows some examples where the alignment score for the object
significantly drops. As shown on the left, the subject (i.e., woman) in the
generated images are naturally turning their back on since the models try to
comply with the given global context (”is talking to”). On the other hand,
this does not happen with MultiDiffusion (right) since it does not handle the
global context. Therefore, MultiDiffusion achieves the higher alignment score
for the objects in Table 1.

Figure 10 shows some examples where GLoD controls the local details. In those
images, only the local text for the subject (i.e., man) changes while keeping
the global text (”is walking with”) and the local text for the object (i.e.,
woman) the same. As shown in Fig. 10, the visual attribute of the man changes
depending on the local text, but the object layout and his posture are almost
preserved.