# Measuring Turbulence with Young Stars in the Orion Complex

Trung Ha Department of Physics, University of North Texas, Denton, TX 76203,
USA<EMAIL_ADDRESS>Yuan Li Department of Physics, University of North
Texas, Denton, TX 76203, USA  Department of Astronomy, University of
California, Berkeley, CA 94720, USA  Siyao Xu Institute for Advanced Study, 1
Einstein Drive, Princeton, NJ 08540, USA  Marina Kounkel Department of Physics
and Astronomy, Western Washington University, 516 High St, Bellingham, WA
98225, USA  Hui Li Department of Physics, Kavli Institute for Astrophysics and
Space Research, Massachusetts Institute of Technology, Cambridge, MA 02139,
USA  Department of Astronomy, Columbia University, 550 West 120th Street, New
York, NY 10027, USA

###### Abstract

Stars form in molecular clouds in the interstellar medium (ISM) with a
turbulent kinematic state. Newborn stars therefore should retain the turbulent
kinematics of their natal clouds. Gaia DR2 and APOGEE-2 surveys in combination
provide three-dimensional (3D) positions and 3D velocities of young stars in
the Orion Molecular Cloud Complex. Using the full 6D measurements, we compute
the velocity structure functions (VSFs) of the stars in six different groups
within the Orion Complex. We find that the motions of stars in all diffuse
groups exhibit strong characteristics of turbulence. Their first-order VSFs
have a power-law exponent ranging from $\sim 0.2-0.5$ on scales of a few to a
few tens of pc, generally consistent with Larson’s relation. On the other
hand, dense star clusters, such as the Orion Nebula Cluster (ONC), have
experienced rapid dynamical relaxation, and have lost the memory of the
initial turbulent kinematics. The VSFs of several individual groups and the
whole Complex all show features supporting local energy injection from
supernovae. The measured strength of turbulence depends on the location
relative to the supernova epicenters and the formation history of the groups.
Our detection of turbulence traced by young stars introduces a new method of
probing the turbulent kinematics of the ISM. Unlike previous gas-based studies
with only projected measurements accessible to observations, we utilize the
full 6D information of stars, presenting a more complete picture of the 3D
interstellar turbulence.

††thanks: NASA Hubble Fellow††thanks: NASA Hubble Fellow

##  1 Introduction

The interstellar medium (ISM) is turbulent. More specifically, the cold dense
star-forming molecular clouds are turbulent. For example, Larson (1981) finds
a power law relation between velocity dispersion and cloud size with an
exponent of 0.38, similar to the Kolmogorov law for incompressible turbulence
(Kolmogorov, 1941). Turbulence plays an important role in regulating the cloud
dynamics and star formation (Mac Low & Klessen, 2004; McKee & Ostriker, 2007;
Hennebelle & Falgarone, 2012), from cloud scales all the way down to the
scales much smaller than molecular cores (Hull et al., 2017).

The ubiquitous interstellar turbulence induces velocity fluctuations on all
scales in both diffuse interstellar phases and dense star-forming gas.
Statistical measurements of velocity fluctuations in the multi-phase ISM can
be done using the Velocity Channel Analysis (VCA), the Velocity Coordinate
Spectrum (VCS) (Lazarian & Pogosyan, 2000, 2006), the Delta-variance technique
(Stutzki et al., 1998; Ossenkopf & Mac Low, 2002), or the Principal Component
Analysis (Heyer & Brunt, 2004; Roman-Duval et al., 2011) based on
spectroscopic data. These analyses generally reveal power-law spectra of
turbulent velocities (Lazarian, 2009a; Chepurnov et al., 2010).

Many processes can drive turbulence in the ISM, such as gravity, shear, and
various forms of stellar feedback including supernova explosions (Krumholz &
Burkhart, 2016; Padoan et al., 2016). Turbulent motions shape the filamentary
density structures in molecular clouds (e.g., Federrath 2016; Xu et al. 2019).
Dense cores and protostars subsequently form in dense filaments (André, 2017).

Besides gas tracers that are commonly used to probe turbulence in the volume
filling ISM, turbulent velocities can also be sampled using point sources such
as molecular cores formed at the density peaks generated in highly supersonic
turbulence. For example, Qian et al. (2012) developed statistical measurements
of core-to-core velocity dispersion over a range of length scales , i.e., the
core velocity dispersion technique. Qian et al. (2018) find signatures for a
turbulent power-law spectrum in the Taurus Molecular Cloud, and demonstrate
that the statistics of turbulent velocities are imprinted in the velocities of
dense cores (Xu, 2020).

Because prestellar cores are nurseries of stars, newborn stars should
naturally inherit the velocities of cores and hold the fossil information on
the turbulent velocities of their natal clouds, as long as dynamical
interactions of young stars have not erased that information.

In this work, we carry out the first statistical analysis of star-to-star
velocity structure function (VSF) across the Orion complex over a range of
length scales, from $\sim 200$ pc down to $<1$ pc. These newly formed stars
are expected to still “remember” the local gas kinematics at their birth, and
can be used to recover the turbulent motions in the parent cloud. With direct
measurements on 3D velocities and 3D positions, the statistical study of
turbulent velocities with stars do not suffer from the complex convolution
between density and velocity and the projection effect as that with the
commonly used gas tracers.

In Section 2, we describe the data used in this analysis and how we compute
the VSFs. We present the VSF for each group in the Orion complex in Section 3,
and discuss how the slope and amplitude is related to the location and history
of the stellar system. In Section 4, we discuss the uncertainties and biases
of the analysis, and how our method and results compare with previous works on
turbulence in the ISM.

##  2 Data Processing

Figure 1: Foreground: each dot represents the projected position of a star in
the Orion Complex that we include in our analysis. The star’s color
corresponds to its main group. Background: greyscale optical image of the
Orion Molecular Cloud Complex, courtesy of Rogelio Bernal Andreo.

###  2.1 Data Acquisition

We obtain the locations, parallax distances, and proper motions of stars in
the Orion Molecular Cloud Complex from Gaia’s second data release (Gaia
Collaboration et al., 2018). The Gaia catalog includes five-parameter
astrometric data from over 1.3 billion stars. The line-of-sight velocities of
the stars were observed in the near infrared during the Apache Point
Observatory Galactic Evolution Experiment 2 (APOGEE-2). The APOGEE
spectrograph is mounted on the 2.5 m Sloan Foundation Telescope of the Sloan
Digital Sky Survey (SDSS), and is used to collect high resolution spectral
images of over 400,000 stars in the Milky Way (Gunn et al., 2006; Blanton et
al., 2017). In total, 8891 stars from APOGEE-2, and 16754 stars from Gaia DR2
were observed towards the Orion Complex, as specified by Kounkel et al.
(2018). Combining these two surveys provides six-dimensional information of a
subset of stars.

Kounkel et al. (2018) developed a hierarchical clustering algorithm to assess
membership of the Orion Complex and its structure based on the position of
stars in the phase space (position+velocity). They identify five main groups
inside the Complex, namely Orion A, Orion B, Orion C, Orion D, and $\lambda$
Ori. We require all stars in our sample to be a part of one of these groups,
excluding all of the stars that could not be clustered together (i.e., field
stars, or other members of Orion that may have peculiar velocity - such as
spectroscopic binaries or runaways). Furthermore, we require a complete
information on six-dimensional position and velocity for each star, i.e, a
star needs to have both astrometric solutions from Gaia DR2 as well as radial
velocities from APOGEE. Through this, we obtain a sample of 1439 stars.

We note that the Orion Nebula Cluster (ONC) is a part of the Orion A molecular
cloud, however, it is very unique, being a singularly most massive young
cluster in the solar neighborhood. The environment that is found in the ONC is
different from the more diffuse environment in the southern part of Orion A.
Thus, for the purpose of our analysis, we separate ONC and the rest of the
Orion A. Throughout the paper, Orion A refers only to the “tail” of the cloud
(L1641 and L1647), excluding stars in ONC. The reasoning behind this split is
discussed in Section 3. The term “group” in this paper refers to all “stellar
systems” that are spatially and kinematically associated, while “cluster” only
refers to open clusters such as the ONC.

Table 1 lists the number of stars and the mean age of each main group and the
ONC. We plot the projected positions of the stars in our analysis over a
greyscale image of the Orion Complex in Figure 1.

###  2.2 Method of analysis

Figure 2: Velocities of stars in the six clouds in our analysis in Cartesian
coordinate. The color map denotes stars’ velocity in the y direction, which
closely resembles their radial velocity. All clouds show some level of
randomness in the velocities of the stars.

To test our hypothesis that young star clusters “remember” their local gas
kinematics at the time of birth, we compute the VSFs for all identified
groups, as well as the VSF of the combined Orion Complex catalog, both with
and without the ONC. We first convert the radial velocity and proper motion of
stars into velocity in Cartesian coordinate, in the Local Standard of Rest
(LSR). Figure 2 shows the 3D velocity information of stars used in this study
in each cloud.

For each cloud, we compute the first-order VSF and its uncertainties. The VSF
for a sample of stars can be calculated in the following way: for each pair of
stars, we first record the physical separation $\ell$ of the pair and compute
the magnitude of the vector velocity difference $\delta\vec{v}$ as the
velocity difference $|\delta v|$. We then compute the average of the velocity
differences $\langle|\delta v|\rangle$ within logarithmic bins of $\ell$.

The VSF is related to the kinetic energy power spectrum of turbulence. It
describes how velocity difference ($|\delta v_{ij}|$) relates to the physical
separation ($\ell_{ij}$) for given pairs of points in space ($i$, $j$). In a
turbulent flow, kinetic energy cascades down from large scales to small
scales, and the velocity differences become smaller toward smaller scales. If
the turbulence is in-compressible (subsonic), we expect the first-order VSF to
be $|\delta v_{ij}|\sim\ell_{ij}^{1/3}$ within the inertial range (Kolmogorov,
1941). For compressible (supersonic) turbulence, $|\delta
v_{ij}|\sim\ell_{ij}^{1/2}$ (e.g., Federrath, 2013).

To estimate uncertainties of the VSF, we first perform random sampling of the
measurements of stars’ properties based on the observed uncertainties, and
obtain 1000 realizations for each star following a Gaussian distribution. We
only consider measurement uncertainties of the parallax distances, proper
motions, and radial velocities. We ignore the uncertainties in RA and Dec
because they are typically much smaller than the uncertainties of other
quantities. For each group, we then obtain 1000 VSFs, with each iteration
excluding a random star within the group. This allows us to account for both
measurement errors and uncertainties due to small sampling sizes in some bins
of $\ell$. We take the mean and standard deviation of the computed VSFs to be
the group’s VSF and its corresponding error. We choose 1000 iterations to
ensure accuracy even though the results are already converged at 100
iterations.

We also compute the second and third-order VSFs of the groups, and discuss
them in the Appendix.

Table 1: Properties of molecular clouds in the Orion Complex

| Orion A | Orion B | Orion C | Orion D | Lambda Ori | ONC
---|---|---|---|---|---|---
Number of Starsa | 117 | 51 | 162 | 354 | 170 | 585
Median Age (Myr) | 2.2 | 1.3 | 4.1 | 4.9 | 3.7 | 2.7
Crossing time (Myr) | 16.1 | 13.6 | 8.0 | 12.9 | 12.8 | 6.9b
Fitting Rangec (pc) | 10 - 90 | 8 - 60 | 9 - 50 | 15 - 58 | 10 - 32 | 5 - 40
Slope of VSF | $0.18$ | $0.14$ | $0.19$ | $0.53$ | $0.36$ | $0.05$

Note. a. Number of stars obtained from the combined Gaia DR2 and APOGEE
surveys with full 6D information (Section 2.1). b. This is the crossing time
for the entire ONC cluster. The inner core has a much shorter crossing time
and relaxation time (Hillenbrand & Hartmann, 1998). c. The slope of the VSFs
depends on the exact range of separation taken into account. We choose the
fitting range such that the uncertainties are low and there is a consistent
trend in the slope of the VSF.

Figure 3: Left to right, top to bottom: first-order VSF of stars in the Orion
A, Orion B, Orion C, Orion D, Lambda Ori, ONC, all Orion without ONC, and all
Orion with ONC. The error bars are obtained through a random sampling
analysis, and represent the uncertainties from both observational errors and
uncertainties due to small sample size. As reference, we also plot a black
dot-dashed line of slope 1/3 for the Kolmogorov turbulence, a pink dashed line
of slope 1/2 for supersonic turbulence, and a solid blue line for Larson’s
velocity dispersion correlation for molecular clouds: $\sigma(km\cdot
s^{-1})=1.1\cdot L(pc)^{0.38}$ (Larson, 1981). All panels use 40 logarithmic
bins of $\ell$ from 1 pc to 170 pc.

##  3 Results

Figure 3 shows the first-order VSF of stars in all six individual groups in
the Orion Complex. The last two panels show the combined VSF of all groups
without and with the ONC, respectively. To guide the eye, we also plot in each
panel the expected scaling for supersonic and subsonic turbulence, along with
Larson’s law measured for turbulence in molecular clouds in the Milky Way
(Larson, 1981).

The VSFs of all six individual groups except ONC exhibit a power-law scaling,
characteristic of a turbulent flow. The amplitudes and slopes vary from group
to group. The best-fit VSF slopes are listed in Table 1.

The Orion A group (top left panel of Figure 3) is located just outside of
Barnard’s Loop (the shell-like feature to the left of Orion A-D in Figure 1,
see also Ochsendorf et al. (2015)), which is proposed to have originated from
a supernova that erupted $\sim$6 Myr ago (Kounkel, 2020). Due to its location,
Orion A is largely unaffected by the supernova’s shock wave. Indeed we observe
that its VSF slope generally follows that of subsonic turbulence. The overall
amplitude is slightly below Larson’s relation.

The Orion B cloud is connected to Orion A but closer to the center of the
supernova explosion. Our analysis of the Orion B group suffers from high
uncertainties due to low number of stars (51) available in the catalog. The
slope of the VSF of Orion B (top middle panel) is not well constrained, but
the VSF generally follows the turbulence-like kinematics present in other
clouds. The amplitude is roughly consistent with Larson’s relation.

Orion C lies within the supernova bubble, and as Kounkel (2020) suggested, was
driven apart from Orion D at the opposite front of the supernova shock wave.
Most of the stars in Orion C and D formed after the supernova explosion that
created the Barnard’s Loop. We observe this local energy injection effect on
the VSF of Orion C (top right panel) with a higher-than-Larson amplitude. The
slope of the VSF is flatter than the Kolmogorov turbulence due to the presence
of $\sigma$ Ori, a dense cluster of age $\sim$2 Myr. We discuss the effects of
dense clusters on the VSF in more detail later in this section.

Stars in the Orion D group exhibit a curious VSF (middle left panel), with a
slope more in line with supersonic turbulence (slope $1/2$) than Kolmogorov at
distance scale $10<\ell<60$ pc and absolute velocity difference higher than
that of Larson’s law. We attribute these properties of Orion D’s VSF to its
position inside the supernova bubble. Located close by and at the front of the
bubble (in our line-of-sight) as it expands, motion of stars in the Orion D
cloud is additionally driven by the shock wave of the supernova, hence its
turbulent kinetic energy is amplified.

The last independent group we look at is $\lambda$ Ori (center panel of Figure
3). In the $10<\ell<32$ pc range, the VSF of stars in $\lambda$ Ori closely
exhibits the 0.38 slope of Larson’s law. The amplitude also exactly follows
Larson’s relation (Larson, 1981), and is higher than Orion A and B, but lower
than C and D. This result can be understood given $\lambda$ Ori’s history and
isolated position in the Orion Complex. The $\lambda$ Orion bubble (the
bubble-like structure surrounding $\lambda$ Ori in Figure 1) is proposed to
have been created by a supernova explosion $\sim 1$ Myr ago (Dolan & Mathieu,
2002). Although Kounkel (2020) estimated the age of the remnant to be $\sim 4$
Myr. Regardless, the explosion happened after many of the stars have already
formed in $\lambda$ Ori where the stellar age has a wide spread over $\sim 6$
Myr (Kounkel et al., 2018). Thus the effect of local energy injection is not
as prominent as in Orion C and D.

The ONC’s VSF (middle right panel) has a distinctively flatter slope compared
to other groups in the Orion Complex. The ONC is a dense star cluster located
at one end of the Orion A molecular cloud (Figure 1). Unlike the loose star
groups discussed previously, the ONC has a dense inner core ($\sim 2$ pc) with
an estimated relaxation time of $\tau_{r}\sim$ 0.6 Myr (Hillenbrand &
Hartmann, 1998), shorter than the cluster’s mean age of $\sim$ 2.6 Myr.
Therefore, a significant fraction of the stars in the ONC have dynamically
relaxed, and have hence “forgotten” the turbulence kinematics of their natal
clouds. The residual non-zero slope likely originates from the stars in the
outer parts of the cluster that have not relaxed yet. It is worth noting that
if we simply look at the velocity distributions of stars in different groups
(Figure 2), ONC does not appear drastically different from the rest of the
groups apart from its compactness in the velocity space. Yet our analysis
reveals its distinct kinematics due to relaxation.

In the last two panels of Figure 3, we compute the VSF for all the stars
within the Orion Complex without and with the ONC (bottom left and bottom
right panels, respectively). Without the ONC, the VSF of the Orion Complex has
similar characteristics to that of Orion D, with a steeper-than-Kolmogorov
slope and an amplitude higher than Larson’s law in the $10<\ell<60$ pc scale
range. This is because D has the highest number of stars amongst the 5 groups
(Table 1).

When the ONC is included, on small scales ($2<\ell<20$ pc), the VSF flattens
because it is heavily influenced by the cluster due to its dominant number of
stars in the catalog (585 stars vs. a total of 1439 stars). In fact, this is
why we separate the ONC from Orion A in our analysis. If we include ONC in
Orion A, the resulting VSF is significantly flatter. All the other star groups
also contain star clusters. For example, Orion C has $\sigma$ Ori. We do not
separate these clusters because they are much smaller than the ONC, and
removing them degrades the statistics of our analysis. We note, however, that
they may have contributed to the flatter VSF of Orion C and the flattening of
the VSF in the other groups on small scales. We discuss other uncertainties in
more detail in Section 4.

The combined VSF shows a bump at $\sim 60$ pc, similar to Orion D, suggesting
energy injection at that scale. The mean age of the stars in this study is
$\sim 3.7$ Myr. Given the current size of the Barnard’s loop ($\sim 80$ pc)
and its age ($\sim 6$ Myr), the energy injection scale is roughly consistent
with the size of the supernova bubble when the stars first formed. This again
shows the effects of local energy injection of supernova explosions on
turbulence.

##  4 Discussions

###  4.1 Biases and Uncertainties

Gas in molecular clouds such as the Orion Complex is turbulent by nature. The
theoretical foundation of our analysis is that newborn stars retain this
turbulence information. However, after the stars are born, they decouple from
the hydro-dynamical forces of the gas, and no longer follow the turbulent
flow. In other words, stars are only perfect tracers of turbulence at birth.
Since star formation happens over a period of time, all of the star groups
contain stars that are over a few Myr old. We have identified two mechanisms
through which stars can lose their memory of the ISM turbulence, causing
biases in our analysis.

The first mechanism is dynamical relaxation, which we discussed in the
analysis of the ONC in Section 3. Due to open clusters’ high density, close
stellar encounters are common, which alter the stars’ original orbits and lead
to dynamical relaxation. The relaxation time is estimated as a function of the
number of stars and the crossing time of the cluster:

$t_{relax}=t_{cross}\cdot\frac{N}{6\cdot\ln{N/2}}\,,$ (1)

Where N is the total number of stars in the cluster and $t_{cross}$ is the
crossing time, defined as the typical time for a star to travel a distance
equal to the half-mass size of the cluster:

$t_{cross}=\frac{2\cdot R_{hm}}{\sigma_{v}}\,.$ (2)

Here, $R_{hm}$ is the radius within which half of the cluster’s stars reside,
and $\sigma_{v}$ is the bulk-motion-removed characteristic velocity of the
cluster’s stars. It is computed as the standard deviation of the velocity
distribution with respect to the median
$\sigma_{v}\equiv\sigma[|v_{i}-v_{median}|]$.

After the stars are relaxed, they no longer hold the turbulent kinematics of
gas at the time of their formation. For loose groups of stars such as those we
analyze, the crossing times are significantly longer than the group’s current
age (Table 1). Furthermore, since each group possesses a number of stars in
the order of N $\sim 10^{3}$ (eg. N = 2800 stars within the dense core of the
ONC was used by Hillenbrand & Hartmann (1998)), the estimated relaxation time
is an order of magnitude longer than the crossing time. Therefore, dynamical
relaxation has most likely not affected the shape of the VSF for the five
groups in the Orion Complex, except for the ONC. This is in agreement with Da
Rio et al. (2017) who find that young stars in Orion A are kinematically
associated with the molecular gas but the ONC appears more dynamically evolved
and virialized. Kounkel et al. (2018) also found consistent kinematics between
young stars and gas in the whole Orion Complex where the gas has not been
dispersed.

In addition to the ONC, there are other clusters within each group: $\sigma$
Ori in Orion C, 25 Ori in Orion D, $\lambda$ Orionis Cluster in $\lambda$ Ori,
and two small clusters NGC 2024 and NGC 2068 in the Orion B group. These
clusters have much smaller physical sizes than the ONC and contain only a
small fraction of stars of the group. This property of the clusters can be
visually confirmed from Figure 1. Thus, their existence does not significantly
affect our overall results. However, they may have contributed to the
flattening of the VSFs on small scales that we observe in almost all
individual groups.

Another potential source of bias in our analysis comes simply from the
drifting of stars from their original positions. Assuming no strong stellar
encounters, stars with higher velocity differences drift apart at a faster
rate than stars with smaller velocity differences, causing the VSF to steepen
with time. This steepening happens over roughly $t_{cross}$.

In our analysis, most groups are relatively young, with their ages much
shorter than $t_{cross}$ (Table 1), so drifting should not have had time to
erase turbulence-driven velocity differences. Although drifting may have
steepened the VSF of Orion D, which has the oldest median age and the steepest
slope amongst all the groups analyzed here. We plan to further explore the
effects of relaxation and drifting in our future works both with a larger
observed sample and with numerical simulations (Li et al., 2019).

There are also other sources of uncertainties in our analysis, such as binary
systems. While the barycenter should follow the turbulent velocity of the
natal cloud, binaries also have orbital motion contributing to the overall
velocity measurement. While most systems with very large orbital speeds have
been excluded from the analysis through the initial hierarchical clustering,
systems with orbital speeds of only a few km s-1 likely remain.

False positives in hierarchical clustering, such as contamination from the
field stars can also affect our results. Kounkel et al. (2018) estimated that
false positive fraction for the clustering algorithm can be as high as
$\sim$6%. In addition, drifting can result in close pairs of uncorrelated
stars with high relative velocity. Binaries, drifting, and mis-grouped stars,
coupled with small sample sizes at small separations would explain the
flattening of the VSFs at small $\ell$.

###  4.2 Comparison with other Methods to Measure Turbulence

Most of the previous studies on turbulence in astrophysical environments rely
on the observations of gas. Although only velocity statistics can directly
reflect the dynamics of turbulence and turbulent energy cascade, it is the
density statistics that is widely used for measuring turbulence as it is much
more easily accessible to observations. For example, the delta-variance method
has been used to analyze the structure of observed molecular cloud images in
the ISM (Stutzki et al., 1998). The same method and a modification of it
(Arévalo et al., 2012) have also been used to infer turbulence in the hot
intra-cluster medium based on X-ray images (e.g., Zhuravleva et al., 2014).

Velocity centroids can recover the turbulent velocity spectra in subsonic
turbulence (Esquivel & Lazarian, 2005). The velocity channel analysis (VCA)
using channel maps and velocity coordinate spectrum (VCS) using the
fluctuations measured along the velocity axis of the Position–Position
Velocity (PPV) cubes can also be used to extract turbulent velocity spectra in
supersonic turbulence (see Lazarian (2009b) for a review). In particular, the
VCS technique only requires measurements along a few directions to obtain a
reliable velocity spectrum (Chepurnov & Lazarian, 2009). Spectroscopic data
has also been used in the Principal Component Analysis to extract the size-
line width relation and recover the structure function within molecular clouds
(e.g., Heyer & Brunt, 2004; Roman-Duval et al., 2011).

Recently, statistical measurements of the velocities of gas phase point
sources have been developed for, e.g., dense molecular cores embedded in the
background turbulent flow (Qian et al., 2012) and filaments in the centers of
galaxy clusters (Li et al., 2020). These analyses use the 1D line-of-sight
velocities and 2D positions projected on the plane of the sky. The bias due to
the projection effect depends on the thickness of the structure (Qian et al.,
2015), which is usually unknown.

Our analysis in spirit is similar to that of gas phase point sources, but
instead of using projected information, we take advantage of the full 6D (3D
velocity + 3D position) information of stars. Thus our results do not suffer
from the poorly constrained projection uncertainties. Observing 6D information
of the gas, especially the motions in the plane of the sky, is prohibitively
challenging (although see O’Dell & Henney, 2008). Thus our method has a unique
advantage, and can potentially be used to help better understand the biases in
the analyses based on gas.

###  4.3 Comparison with Theories and Other Observations

There has been a lot of previous work on turbulence in Orion based on the
observations of emission lines of the ionized gas (Castaneda, 1988; O’dell &
Wen, 1992; McLeod et al., 2016; Arthur et al., 2016). Most of them are focused
on small scales ($<1$ pc) and therefore can not be directly compared with our
results. But they do generally find VSFs consistent with turbulence, in
agreement with our results.

On larger scales (a few to $\sim 100$ pc), we have compared our results with
Larson’s Law in Section 3. Individual groups may show slightly higher or lower
amplitude than the mean Larson’s relation, with slightly steeper or shallower
slopes, but overall, our results are in good agreement with Larson’s Law for
molecular clouds in the Milky Way. Notably, Larson (1981) also finds a
slightly lower-than-average amplitude for Orion B, consistent with our
findings here even though we are using different tracers of turbulence.

Our results show that regions heavily influenced by supernova explosions can
exhibit a higher level of turbulence, and even a steeper VSF. This is
consistent with numerical simulations showing that supernovae can drive
turbulence in molecular clouds (Padoan et al., 2016). Additionally, the
steepened slope of Orion D is consistent with Federrath (2013) finding that
supernova explosions excite more compressible than solenoidal velocity modes
in the gas, which tends to steepen the VSFs, and thus exhibit the 1/2 power
law of supersonic turbulence. Observations have also shown the local effects
of supernovae on turbulence in other systems (e.g., in the Large Magellanic
Clouds (Szotkowski et al., 2019)).

##  5 Concluding Remarks and Future Work

We have introduced a new method to study turbulence in the ISM using motions
of young stars. Our results demonstrate that stars in young star groups retain
the memory of turbulence of their natal molecular clouds. Our analysis uses
the full 6D information of stars to trace turbulence, and does not suffer from
projection uncertainties. Our results can be used to provide independent
constraints on the turbulent properties of molecular clouds in addition to
analysis of gas kinematics, and also shed light on studies of the dynamical
evolution of star clusters in the Galaxy (Kamdar et al., 2019).

We plan to apply our analysis to a much larger sample of star groups in the
future. Meanwhile, we will perform and analyze tailored numerical simulations
of star formation in molecular clouds to better understand biases and
uncertainties in our analysis, as well as constraining subgrid models for star
formation (Li et al., 2019).

## Acknowledgments

We would like to thank Anna McLeod and Yuan-Sen Ting for helpful discussions.
This work was partly performed at the Aspen Center for Physics, which is
supported by National Science Foundation grant PHY-1607611. HL is supported by
NASA through the NASA Hubble Fellowship grant HST-HF2-51438.001-A awarded by
the Space Telescope Science Institute, which is operated by the Association of
Universities for Research in Astronomy, Incorporated, under NASA contract
NAS5-26555. S.X. acknowledges the support provided by NASA through the NASA
Hubble Fellowship grant $\\#$ HST-HF2-51473.001-A awarded by the Space
Telescope Science Institute, which is operated by the Association of
Universities for Research in Astronomy, Incorporated, under NASA contract
NAS5-26555.

We present the second and third order VSFs in Figure 4. High order VSFs are
usually used to evaluate the intermittency (She & Leveque, 1994; Boldyrev et
al., 2002). Different behaviors of the high-order VSFs measured with different
gas tracers, e.g., $H_{2}$, $CO$, were found in numerical simulations of
molecular clouds, which are related to their different volume filling factors
and spatial distribution (Bertram et al., 2015). The second and third order
VSFs that we present in Figure 4 are generally consistent with the Kolmogorov
scaling. This finding agrees with the measurement using the velocities of
dense cores in Taurus cloud (Qian et al., 2018). It suggests that both the
dense cores and the stars used here sample the turbulence in the relatively
diffuse region with a large volume filling factor, and thus the Kolmogorov
scaling is expected (Xu, 2020).

Figure 4: First ($p=1$, red), second ($p=2$, pink), and third ($p=3$, blue)
order VSFs of the stars in the entire Orion Complex. The second and third
order VSFs have been shifted lower for easy comparison.

## References

  * André (2017) André, P. 2017, Comptes Rendus Geoscience, 349, 187, doi: 10.1016/j.crte.2017.07.002
  * Arévalo et al. (2012) Arévalo, P., Churazov, E., Zhuravleva, I., Hernández-Monteagudo, C., & Revnivtsev, M. 2012, MNRAS, 426, 1793, doi: 10.1111/j.1365-2966.2012.21789.x
  * Arthur et al. (2016) Arthur, S. J., Medina, S. N. X., & Henney, W. J. 2016, MNRAS, 463, 2864, doi: 10.1093/mnras/stw2165
  * Bertram et al. (2015) Bertram, E., Konstandin, L., Shetty, R., Glover, S. C. O., & Klessen, R. S. 2015, MNRAS, 446, 3777, doi: 10.1093/mnras/stu2372
  * Blanton et al. (2017) Blanton, M. R., Bershady, M. A., Abolfathi, B., et al. 2017, AJ, 154, 28, doi: 10.3847/1538-3881/aa7567
  * Boldyrev et al. (2002) Boldyrev, S., Nordlund, Å., & Padoan, P. 2002, Phys. Rev. Lett., 89, 031102, doi: 10.1103/PhysRevLett.89.031102
  * Castaneda (1988) Castaneda, H. O. 1988, ApJS, 67, 93, doi: 10.1086/191268
  * Chepurnov & Lazarian (2009) Chepurnov, A., & Lazarian, A. 2009, ApJ, 693, 1074, doi: 10.1088/0004-637X/693/2/1074
  * Chepurnov et al. (2010) Chepurnov, A., Lazarian, A., Stanimirović, S., Heiles, C., & Peek, J. E. G. 2010, ApJ, 714, 1398, doi: 10.1088/0004-637X/714/2/1398
  * Da Rio et al. (2017) Da Rio, N., Tan, J. C., Covey, K. R., et al. 2017, ApJ, 845, 105, doi: 10.3847/1538-4357/aa7a5b
  * Dolan & Mathieu (2002) Dolan, C. J., & Mathieu, R. D. 2002, AJ, 123, 387, doi: 10.1086/324631
  * Esquivel & Lazarian (2005) Esquivel, A., & Lazarian, A. 2005, ApJ, 631, 320, doi: 10.1086/432458
  * Federrath (2013) Federrath, C. 2013, MNRAS, 436, 1245, doi: 10.1093/mnras/stt1644
  * Federrath (2016) —. 2016, MNRAS, 457, 375, doi: 10.1093/mnras/stv2880
  * Gaia Collaboration et al. (2018) Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2018, A&A, 616, A1, doi: 10.1051/0004-6361/201833051
  * Gunn et al. (2006) Gunn, J. E., Siegmund, W. A., Mannery, E. J., et al. 2006, AJ, 131, 2332, doi: 10.1086/500975
  * Hennebelle & Falgarone (2012) Hennebelle, P., & Falgarone, E. 2012, A&A Rev., 20, 55, doi: 10.1007/s00159-012-0055-y
  * Heyer & Brunt (2004) Heyer, M. H., & Brunt, C. M. 2004, ApJ, 615, L45, doi: 10.1086/425978
  * Hillenbrand & Hartmann (1998) Hillenbrand, L. A., & Hartmann, L. W. 1998, ApJ, 492, 540, doi: 10.1086/305076
  * Hull et al. (2017) Hull, C. L. H., Mocz, P., Burkhart, B., et al. 2017, ApJ, 842, L9, doi: 10.3847/2041-8213/aa71b7
  * Kamdar et al. (2019) Kamdar, H., Conroy, C., Ting, Y.-S., et al. 2019, ApJ, 884, 173, doi: 10.3847/1538-4357/ab44be
  * Kolmogorov (1941) Kolmogorov, A. 1941, Akademiia Nauk SSSR Doklady, 30, 301
  * Kounkel (2020) Kounkel, M. 2020, The Astrophysical Journal, 902, 122, doi: 10.3847/1538-4357/abb6e8
  * Kounkel et al. (2018) Kounkel, M., Covey, K., Suárez, G., et al. 2018, AJ, 156, 84, doi: 10.3847/1538-3881/aad1f1
  * Krumholz & Burkhart (2016) Krumholz, M. R., & Burkhart, B. 2016, MNRAS, 458, 1671, doi: 10.1093/mnras/stw434
  * Larson (1981) Larson, R. B. 1981, MNRAS, 194, 809, doi: 10.1093/mnras/194.4.809
  * Lazarian (2009a) Lazarian, A. 2009a, Space Science Reviews, 143, 357, doi: 10.1007/s11214-008-9460-y
  * Lazarian (2009b) —. 2009b, Space Sci. Rev., 143, 357, doi: 10.1007/s11214-008-9460-y
  * Lazarian & Pogosyan (2000) Lazarian, A., & Pogosyan, D. 2000, ApJ, 537, 720, doi: 10.1086/309040
  * Lazarian & Pogosyan (2006) —. 2006, ApJ, 652, 1348, doi: 10.1086/508012
  * Li et al. (2019) Li, H., Vogelsberger, M., Marinacci, F., & Gnedin, O. Y. 2019, MNRAS, 487, 364, doi: 10.1093/mnras/stz1271
  * Li et al. (2020) Li, Y., Gendron-Marsolais, M.-L., Zhuravleva, I., et al. 2020, ApJ, 889, L1, doi: 10.3847/2041-8213/ab65c7
  * Mac Low & Klessen (2004) Mac Low, M.-M., & Klessen, R. S. 2004, Reviews of Modern Physics, 76, 125, doi: 10.1103/RevModPhys.76.125
  * McKee & Ostriker (2007) McKee, C. F., & Ostriker, E. C. 2007, ARA&A, 45, 565, doi: 10.1146/annurev.astro.45.051806.110602
  * McLeod et al. (2016) McLeod, A. F., Weilbacher, P. M., Ginsburg, A., et al. 2016, MNRAS, 455, 4057, doi: 10.1093/mnras/stv2617
  * Ochsendorf et al. (2015) Ochsendorf, B. B., Brown, A. G. A., Bally, J., & Tielens, A. G. G. M. 2015, ApJ, 808, 111, doi: 10.1088/0004-637X/808/2/111
  * O’Dell & Henney (2008) O’Dell, C. R., & Henney, W. J. 2008, AJ, 136, 1566, doi: 10.1088/0004-6256/136/4/1566
  * O’dell & Wen (1992) O’dell, C. R., & Wen, Z. 1992, ApJ, 387, 229, doi: 10.1086/171074
  * Ossenkopf & Mac Low (2002) Ossenkopf, V., & Mac Low, M. M. 2002, A&A, 390, 307, doi: 10.1051/0004-6361:20020629
  * Padoan et al. (2016) Padoan, P., Pan, L., Haugbølle, T., & Nordlund, Å. 2016, ApJ, 822, 11, doi: 10.3847/0004-637X/822/1/11
  * Qian et al. (2018) Qian, L., Li, D., Gao, Y., Xu, H., & Pan, Z. 2018, ApJ, 864, 116, doi: 10.3847/1538-4357/aad780
  * Qian et al. (2012) Qian, L., Li, D., & Goldsmith, P. F. 2012, ApJ, 760, 147, doi: 10.1088/0004-637X/760/2/147
  * Qian et al. (2015) Qian, L., Li, D., Offner, S., & Pan, Z. 2015, ApJ, 811, 71, doi: 10.1088/0004-637X/811/1/71
  * Roman-Duval et al. (2011) Roman-Duval, J., Federrath, C., Brunt, C., et al. 2011, ApJ, 740, 120, doi: 10.1088/0004-637X/740/2/120
  * She & Leveque (1994) She, Z.-S., & Leveque, E. 1994, Phys. Rev. Lett., 72, 336, doi: 10.1103/PhysRevLett.72.336
  * Stutzki et al. (1998) Stutzki, J., Bensch, F., Heithausen, A., Ossenkopf, V., & Zielinsky, M. 1998, A&A, 336, 697
  * Szotkowski et al. (2019) Szotkowski, S., Yoder, D., Stanimirović, S., et al. 2019, ApJ, 887, 111, doi: 10.3847/1538-4357/ab53df
  * Xu (2020) Xu, S. 2020, MNRAS, 492, 1044, doi: 10.1093/mnras/stz3092
  * Xu et al. (2019) Xu, S., Ji, S., & Lazarian, A. 2019, ApJ, 878, 157.  https://arxiv.org/abs/1802.00987
  * Zhuravleva et al. (2014) Zhuravleva, I., Churazov, E., Schekochihin, A. A., et al. 2014, Nature, 515, 85, doi: 10.1038/nature13830


# Few-Shot Musical Source Separation

###### Abstract

Deep learning-based approaches to musical source separation are often limited
to the instrument classes that the models are trained on and do not generalize
to separate unseen instruments. To address this, we propose a few-shot musical
source separation paradigm. We condition a generic U-Net source separation
model using few audio examples of the target instrument. We train a few-shot
conditioning encoder jointly with the U-Net to encode the audio examples into
a conditioning vector to configure the U-Net via feature-wise linear
modulation (FiLM). We evaluate the trained models on real musical recordings
in the MUSDB18 and MedleyDB datasets. We show that our proposed few-shot
conditioning paradigm outperforms the baseline one-hot instrument-class
conditioned model for both seen and unseen instruments. To extend the scope of
our approach to a wider variety of real-world scenarios, we also experiment
with different conditioning example characteristics, including examples from
different recordings, with multiple sources, or negative conditioning
examples.

Index Terms—  few-shot learning, source separation, music, FiLM conditioning.

##  1 Introduction

Musical source separation (MSS) is a well-studied problem which has seen rapid
progress in recent years [1], where the goal is typically to separate the
sound of a particular instrument from a mixture recording. Systems using
various deep learning-based approaches [2, 3, 4, 5, 6, 7, 8, 9] have achieved
impressive results, in particular for singing voice separation. However, these
models are typically trained to separate one particular instrument class (e.g.
vocals, drums). In order to separate more than one instrument, more than one
model is required. More recent models have aimed to overcome this limitation
and support the separation of various instruments using a single model via
instrument class conditioning mechanisms [10, 11, 12, 13, 14]. However, these
models are still limited to the instrument classes that the models were
trained on and do not generalize to unseen instruments.

A few initial attempts at building deep learning-based query-by-example MSS
have been made, where the model is designed or has an auxiliary benefit to
perform separation based on an audio query [15, 16, 12, 17]. However, these
models have not been thoroughly evaluated on how they generalize to unseen
classes. A similar recent effort has been made to develop a one-shot source
separation model for general sounds [18]. While this system is evaluated on
how well it separates seen and unseen sound classes given one audio example
from the target, it has not been compared with any baseline, such as an
existing class label-conditioning approach. Furthermore, it is evaluated on
random mixtures of two sources, which are both less realistic and arguably
much easier to separate than real musical recordings. A meta-learning approach
was also recently proposed to achieve one-shot speech separation [19].
However, it has a very specific task definition of separating two speakers
given one audio example per speaker, which is less applicable to general MSS.
In addition, all of the systems mentioned above are relatively limited in
their flexibility – they consider using only a single query example which is
isolated (single-sourced) and in some cases drawn from the same recording as
the target, which may not always be available in real-world scenarios.

Fig. 1: High-level illustration of the few-shot MSS paradigm.

In this work, we present a few-shot MSS paradigm illustrated in Figure 1. We
condition a generic U-Net separation model using few audio examples of the
target instrument. We first embed the conditioning audio examples into a set
of example-wise conditioning vectors using the few-shot conditioning encoder,
and aggregate these vectors into a single conditioning vector by taking the
average. We use Feature-wise Linear Modulation (FiLM) [20] as the conditioning
mechanism, which is inserted at the U-Net bottleneck layer. This mechanism
allows us to incorporate side information, the conditioning vector, to
configure the U-Net to separate different target instruments. We jointly train
the few-shot conditioning encoder and the conditioned U-Net. Note that our
proposed model is similar to [18], but we extend it from one-shot to few-shot
with a simpler conditioning mechanism, where we apply the conditioning to the
bottleneck layer only [10, 11], instead of to multiple layers in both the
encoder and decoder of the U-Net.

We systematically compare our few-shot conditioned model against a class
label-conditioned baseline approach. We quantitatively evaluate both
approaches on seen and unseen instruments within real musical recordings. In
addition, we explore how much the constraints on the conditioning scenarios in
previous work [18] can be relaxed. Besides extending from one-shot to few-
shot, in particular, we consider providing (1) examples that are not drawn
from the target recording, (2) examples where the desired target is not
isolated, but is present within a mixture, and (3) negative examples of what
not to separate in addition to positive examples. We show that these
relaxations result in systems which can be applied in a wider variety of
applications. Audio examples from our few-shot MSS model are publicly
available111https://wangyu.github.io/few-shot-mss/.

##  2 Model Configurations

We base our separation model on the well-studied U-Net architecture [4, 11,
21] in order to complement related work on conditioned source separation [11,
15, 10]. While models with better separation performance exist [13], the
purpose of this study is to explore the effects of different types of
conditioning approaches and design choices, which can generally translate to
better performance, rather than to strive for state-of-the-art results.

###  2.1 Instrument class-conditioned baseline

We use a U-Net architecture very similar to that described in [11] as the
baseline. The model is conditioned directly on instrument classes (via one-hot
encoding) and can be viewed as a standard supervised learning baseline. Here,
we briefly describe our variant of the model. For the U-Net architecture, each
of the 6 encoder layers consists of $n_{i}$ 5x5 2-D convolutions with a 2x2
stride, followed by batch normalization and a Leaky ReLU activation, where
$n_{0}=16$ and $n_{i}=2n_{i-1}$ for each subsequent layer $i$. The decoder
mirrors the encoder by replacing 2-D convolutions with 2-D transpose
convolutions. Additional skip connections are added between layers at the same
hierarchical level in the encoder and decoder.

A one-hot vector that directly encodes the target instrument class is used as
the conditioning vector $z_{\textit{class}}$. A FiLM layer is inserted after
the U-Net bottleneck layer to transform the bottleneck feature $x$ via
$FiLM(x)=\gamma(z_{\textit{class}})\cdot x+\beta(z_{\textit{class}})$, where
$\gamma$ and $\beta$ are learned parameters that scale and shift $x$ based on
the conditioning information $z_{\textit{class}}$. In our work,
$z_{\textit{class}}$ is 18-dimensional to support a wide variety of instrument
classes. The y-axis in Figure 2 shows 14 of these instrument classes, and the
remaining 4 are mallets, pipe organ, bagpipes, and whistling.

The model takes three seconds of mono audio with 22.05 kHz sample rate as
input and first computes a spectrogram with a hop size of 256 and a FFT size
of 1024. The spectrogram is log-compressed, and input into the U-Net encoder.
The output of the decoder is treated as a two-channel complex mask, where a
sigmoid activation is applied on the magnitude as in [22]. The mask is applied
to the compressed spectrogram via complex multiplication, followed by
decompression. Finally, the Inverse STFT is computed to retrieve the separated
audio signal. The loss is computed as the sum of the SDR Loss [23] in the time
domain and mean absolute error on the magnitudes in the time-frequency domain.

###  2.2 Few-shot conditioning

In our proposed few-shot MSS paradigm, the external information is based on
few audio examples of the target instrument instead of its class label. We
train a few-shot conditioning encoder jointly with the U-Net to generate a
learned conditioning vector $z_{\textit{few-shot}}$ based on few target
examples.

The conditioning encoder consists of four convolution blocks, each of which
has a 64-filter 3x3 convolution, a batch normalization layer, a ReLU
activation layer, and a 2x2 max-pooling layer. To allow our model to handle
variable-length input, we apply max-pooling along the time dimension to the
output of the convolution blocks. Finally, we flatten the feature map to
output an embedding of size 512. When multiple audio examples are used as
conditioning (few-shot instead of one-shot), each example is used to generate
a conditioning vector, and the example-wise vectors are averaged into a single
conditioning vector. We keep the conditioned U-Net architecture, FiLM
conditioning mechanism, and loss terms the same as in the baseline model to
systematically compare the instrument class conditioning and few-shot
conditioning paradigms.

##  3 Experimental Design

To compare the proposed few-shot MSS with the baseline approach conditioned on
instrument class, we first train the models using a combination of several
datasets, and evaluate the trained models on real music datasets, MUSDB18 [24]
and MedleyDB [25]. For the few-shot model, we further experiment with
different evaluation setups with different conditioning example
characteristics as well as including negative conditioning examples.

###  3.1 Training Datasets

In order to support a wide variety of instrument classes, we train the models
on a combination of several multitrack datasets consisting of solo-instrument
stems that are linearly mixed to create full song recordings. The first
dataset, $D_{\textit{real}}$, is a private dataset containing 313 real-world
multi-tracks, with an instrument bias towards pop/rock music meaning that
vocals, drums, bass, guitars, and synthesizers dominate the distribution. The
second, $D_{\textit{synth}}$, is the public Slakh2100-redux [26] dataset,
containing 1710 multi-tracks synthesized from MIDI with high-quality
synthesizers. This dataset contains no vocals, and by its nature only contains
synthetic versions of each instrument. $D_{\textit{solo}}$ is a private
dataset of 627 real solo instrument recordings, roughly evenly distributed
across the 18 instrument classes. These recordings are randomly combined to
generate multi-tracks with up to 5 instruments.

###  3.2 Training setup

To generate a training example, we randomly sample a dataset $D$ from the set
$\\{D_{\textit{real}},D_{\textit{synth}},D_{\textit{solo}}\\}$, a multi-track
$M$ from $D$, a chunk from $M$ as the input mixture, and finally an instrument
track from the mixture as the target output. To train the few-shot models, we
additionally sample $n$ random chunks from the original track of the target
instrument as the conditioning examples. Note that we make sure the random
chunks do not overlap with the target output. We experiment with $n$ between
one and five (1-shot to 5-shot) to examine how this choice correlates with
model performance. All models are trained using a batch size of 16 and a
learning rate of 0.001 with the Adam optimizer and early stopping.

###  3.3 Evaluation setup

We evaluate the trained MSS models on the 50 test tracks in the MUSDB18
dataset. For each track, we have its mixture and isolated tracks of vocals,
drums, bass, and other in stereo format with a 44.1k Hz sample rate. An other
track includes all instruments in a mixture other than vocals, drums, and
bass. It can vary greatly over time with different instruments present at
different times, making conditioning using random examples potentially
incoherent. Therefore, we do not evaluate on separating other in this work.
Note that we did not include the training set of MUSDB18 into our training
data. The training and testing data are therefore from different
distributions, resulting in a more challenging setup compared to most of the
previous works on MSS.

To evaluate a test track, we downsample the audio to 22 kHz and divide the
mixture into 3-second input segments with 50% overlap. We sample $n$ random
chunks from the target instrument track as the conditioning examples for the
few-shot models, and run inference on each 3-second chunk. We reconstruct the
final signal using overlap-add, and upsample to 44.1 kHz. We compute the
signal-to-distortion ratio (SDR) [27] per track using the museval package as
our evaluation metric, and average the SDR across tracks. We repeat this
process 10 times for each track with different random conditioning examples
when evaluating few-shot models, reporting the mean and standard deviation of
means over each of the 10 test iterations.

While MUSDB18 is the most common MSS benchmarking dataset, it only contains
isolated tracks for three instrument classes. Since our baseline model
supports up to 18 instruments, we additionally evaluate the baseline and the
best performing few-shot model on the MedleyDB dataset. We filter out multi-
tracks with bleed, map all instruments to the predefined 18-instrument
vocabulary, and generate sub-mixes in which we keep only one stem per
instrument class. The resulting MedleyDB evaluation set contains 90 tracks
with 14 instrument classes.

Method | Vocals | Drums | Bass
---|---|---|---
Baseline | $\footnotesize 4.07$ | $4.35$ | $3.06$
1-shot | $4.23\pm 0.88$ | $4.34\pm 0.60$ | $2.71\pm 0.86$
2-shot | $4.64\pm 0.38$ | $4.54\pm 0.39$ | $3.27\pm 0.41$
3-shot | $4.80\pm 0.32$ | $4.71\pm 0.38$ | $3.16\pm 0.45$
4-shot | $4.58\pm 0.39$ | $4.61\pm 0.32$ | $3.08\pm 0.50$
5-shot | $4.72\pm 0.35$ | $4.66\pm 0.25$ | $3.26\pm 0.38$
Table 1: Mean SDR (dB) on MUSDB18 for the baseline and few-shot conditioned
models with different n. The standard deviation of the means for few-shot
models are computed over 10 test iterations.

###  3.4 Conditioning example characteristics

In our basic setup of training and evaluating the few-shot model, the selected
conditioning examples have two main constraints: (1) they are drawn from the
target recording, (2) they are single-sourced audio examples that only contain
the target instrument. This is a relatively ideal scenario when considering
real-world applications, where isolated examples from the target recording may
not be available. To see how a few-shot model generalizes to more challenging
and practical scenarios, we perform a set of experiments where we evaluate the
model with one constraint relaxed at a time.

To relax the first constraint, we draw $n$ conditioning examples from $n$
different recordings of the same instrument class within the test data. To
relax the second constraint, we allow some conditioning examples to contain
one additional non-target instrument, where each example has a 0.5 probability
of being multi-sourced. To get a multi-source example from a multitrack, we
randomly choose a non-target instrument and mix it with the target before we
sample an example chunk. Note that we make sure only one consistent
instrument, the target, is present across all conditioning examples.

###  3.5 Negative conditioning examples

Besides providing target instrument examples to specify the desired output, we
further explore if specifying unwanted instruments through the same
conditioning mechanism can help improve the separation. To do so, we extend
our basic few-shot model to leverage additional negative conditioning
examples. During both training and evaluation, we draw $n$ additional negative
examples, each of which contains a random non-target instrument that also
exists in the input mixture. All conditioning examples are first embedded via
the same few-shot conditioning encoder. Then we average the example-wise
conditioning vectors over positive and negative examples separately. We
concatenate the aggregated positive and negative conditioning vectors along
the feature dimension, and convert the concatenated vector to a final
conditioning vector via a fully-connected layer with a ReLU activation. The
sizes of the resulting conditioning vector and the conditioned U-Net model are
the same as in our basic setup.

##  4 Results

###  4.1 Instrument class conditioning vs. few-shot conditioning

In Table 1, we show the performance of the instrument class conditioned
baseline and few-shot conditioned models with different numbers of
conditioning examples $n$ (the number of shots).

First, the 1-shot model, conditioned on only one audio example, does not show
a significant advantage over the baseline. It achieves higher SDR on vocals,
but performs worse on bass. While we can provide more direct and specific
information about the target via audio examples for conditioning, one random
example may not be sufficient to capture the variations of the target within
the entire song. On the other hand, as $n$ increases, the few-shot model
begins to outperform the baseline across all instruments. This shows the
advantage of extending existing query-based or one-shot MSS to few-shot MSS.
We can now provide more conditioning examples to better capture the instrument
variations in pitch, timbre, and playing technique, to achieve better
separation. Next, we see a general trend that as $n$ increases, the
performance of the few-shot model initially increases and then stays
relatively stable, while the standard deviation continues to decrease. Since
we aggregate example-wise conditioning vectors by taking an average, we can
expect these diminishing returns from including more examples once enough
representative examples are present. On the other hand, conditioning on more
audio examples can mitigate the randomness from sampling and leads to more
stable results. Therefore, in the remainder of our experiments, we focus on a
5-shot model for more robust performance.

In addition to MUSDB18, we evaluate the baseline and 5-shot models on
MedleyDB, which contains many more instrument classes. Figure 2 (left) first
shows that the 5-shot model outperforms the baseline on 13 instruments out of
the total of 14 with an overall SDR of 1.96 dB (0.56 dB for the baseline).
Next, we sort the instruments by their occurrences in the training data based
on the distribution shown in Figure 2 (right). We see that the 5-shot model
can not only effectively separate instruments that are relatively rare in our
training data, but also generally outperforms the baseline by larger margins
on these rare classes, for example, Keyboards, Woodwinds, and Accordion. This
suggests that the advantage of few-shot MSS is more significant on rare
instruments. From these results, we see that few-shot MSS is applicable to
many instruments across different datasets. In the remainder of our
experiments, for brevity we focus on the MUSDB18 dataset.

Fig. 2: (Left) Median SDR on MedleyDB dataset for the baseline and 5-shot
models. We report median SDR here due to high variance between tracks. (Right)
Training data distribution.

###  4.2 Separating unseen instruments

One of the advantages of the few-shot MSS model is that it is able to handle
new, unseen instruments at inference time without being limited by the
training vocabulary. However, our training data has a large instrument
vocabulary while MUSDB18 only contains three common instruments. To
quantitatively evaluate how well models can separate unseen instruments, we
remove all vocals, drums, and bass target examples from our training data and
retrain the baseline and few-shot models with the updated training set. This
ensures that the model does not see instruments from the test set during
training.

The results in Table 2 first show that the baseline model conditioned on
instrument classes, unsurprisingly, does not perform well on unseen
instruments as it is effectively untrained for these classes. On the other
hand, both few-shot models with $n\in\\{1,5\\}$ outperform the baseline by
large margins across all instruments (except the 1-shot model for drums). Note
that the performance on drums is significantly lower than the other two
instruments. We conjecture that the high variance of drum sounds makes it
harder for the model to separate based on just one or few examples at
inference time. In addition, if we compare Table 2 to Table 1, first, we see
the drops in SDR resulting from not seeing the testing classes during
training. Next, we see that the 5-shot model outperforms both the baseline and
the 1-shot model more significantly on unseen instruments. This aligns with
our observations in Section 4.1 that the advantage of few-shot MSS is more
significant on rare and even unseen instruments.

Method | Vocals | Drums | Bass
---|---|---|---
Baseline | $0.35$ | $0.05$ | $-0.10$
1-shot | $2.65\pm 0.74$ | $-0.01\pm 0.64$ | $1.40\pm 0.88$
5-shot | $3.22\pm 0.44$ | $1.18\pm 0.40$ | $2.88\pm 0.35$
Table 2: Mean SDR (dB) on MUSDB18 for the baseline and few-shot conditioned
models with $n\in\\{1,5\\}$. Vocals, drums, and bass are held out from the
training data. The standard deviations of the means for the few-shot models
are computed over 10 test iterations.

###  4.3 Conditioning examples

In the previous experiments, we provided few-shot models with single-sourced
conditioning examples drawn from the target recording. To see if few-shot
models are able to generalize to more practical scenarios, we evaluate a
5-shot model with relaxed constraints on conditioning examples as discussed in
Section 3.4.

Table 3 first shows the performance of the basic 5-shot model where (as
before) we apply both constraints to the conditioning examples. Then, if we
draw conditioning examples from different tracks instead of the target track,
SDR drops between $\approx$0.4 to 1.1 dB across instruments. Even with this
drop, the model still achieves comparable and in some cases better SDRs
compared to the baseline model (shown in Table 1). On the other hand, if we
provide multi-sourced conditioning examples with additional non-target
instruments instead of single-sourced ones, the model performance drops
significantly. This is not surprising given that the model is trained only on
isolated audio. To see if changing the training conditions would make this
scenario possible, we retrain the 5-shot model with multi-sourced conditioning
examples. The retrained model (shown as MS in Table 3), while still performing
worse than both the basic 5-shot model and the baseline, achieves about 0.6 to
2.8 dB SDR improvement by matching the training and testing scenarios. These
results indicate that the proposed few-shot MSS model can adapt to a wide
variety of applications where the conditioning examples can be drawn from
different recordings. The model has a harder time generalizing to multi-
sourced conditioning examples, but if that is the desired scenario, we can
match the training and inference objectives to achieve better results.

Next, we experiment with including five additional negative conditioning
examples, each containing a random non-target instrument, as discussed in
Section 3.5. The results at the bottom of Table 3 show that providing
additional information about what not to separate during training and
inference helps to further improve the separation compared to the basic 5-shot
model conditioned on positive examples only. This setup could be particularly
useful when labeling non-target instruments is much easier than labeling the
target one.

Test conditioning constraints |  Model variant |  Vocals |  Drums |  Bass
---|---|---|---|---
Single-sourced |  Same track |  |  |  |
✓ |  ✓ |  |  4.72 |  4.66 |  3.26
✓ |  |  |  4.34 |  4.16 |  2.20
|  ✓ |  |  1.00 |  2.03 |  -0.63
|  ✓ |  MS |  1.63 |  3.52 |  2.21
✓ |  ✓ |  +Neg |  4.73 |  5.21 |  3.63
Table 3: Mean SDR (dB) on MUSDB18 for the 5-shot model with different
constraints applied to the conditioning examples at test time. MS: Model
trained with multi-sourced conditioning examples. +Neg: Model trained with
both positive and negative single-sourced conditioning examples.

##  5 Conclusion

In this work, we proposed a few-shot MSS paradigm where we condition a generic
U-Net source separation model on few audio examples of the target instrument.
We evaluated the trained models on real musical recordings in MUSDB18 and
MedleyDB datasets. We first quantitatively show that our proposed few-shot MSS
model outperformed the baseline model, conditioned on the instrument class
labels, on both seen and unseen instruments. Additionally, we saw a more
significant advantage of few-shot MSS on rare and unseen instruments. Next, we
further experimented with different characteristics of the conditioning
examples, including relaxing the constraints for examples and providing
additional negative conditioning. The results indicate the potential of
applying few-shot MSS to a wider variety of real-world scenarios, where the
conditioning examples may come from different recordings or contain non-target
sounds. Future work could explore the structure of the conditioning embedding
space and ways to further improve model performance such as different
conditioning mechanisms, additional loss terms, and pre-training the
conditioning encoder on few-shot classification tasks.

## References

  * [1] Estefania Cano, Derry FitzGerald, Antoine Liutkus, Mark D Plumbley, and Fabian-Robert Stöter,  “Musical source separation: An introduction,”  IEEE Signal Processing Magazine, vol. 36, no. 1, pp. 31–40, 2018\.
  * [2] Andrew J. R. Simpson, Gerard Roma, and Mark D. Plumbley,  “Deep karaoke: Extracting vocals from musical mixtures using a convolutional deep neural network.,”  CoRR, vol. abs/1504.04658, 2015.
  * [3] Stefan Uhlich, Marcello Porcu, Franck Giron, Michael Enenkl, Thomas Kemp, Naoya Takahashi, and Yuki Mitsufuji,  “Improving music source separation based on deep neural networks through data augmentation and network blending,”  in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 261–265.
  * [4] Andreas Jansson, Eric Humphrey, Nicola Montecchio, Rachel Bittner, Aparna Kumar, and Tillman Weyde,  “Singing voice separation with deep u-net convolutional networks,”  in Proc. of the 18th International Society for Music Information Retrieval Conference (ISMIR), 2017, pp. 23–27.
  * [5] Yi Luo, Zhuo Chen, John R. Hershey, Jonathan Le Roux, and Nima Mesgarani,  “Deep clustering and conventional networks for music separation: Stronger together,”  in IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2017.
  * [6] Daniel Stoller, Sebastian Ewert, and Simon Dixon,  “Wave-u-net: A multi-scale neural network for end-to-end audio source separation,”  in Proc. of the 19th International Society for Music Information Retrieval Conference (ISMIR), 2018, pp. 334–340.
  * [7] Naoya Takahashi, Nabarun Goswami, and Yuki Mitsufuji,  “MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation,”  in 16th International Workshop on Acoustic Signal Enhancement, IWAENC. 2018, pp. 106–110, IEEE.
  * [8] Romain Hennequin, Anis Khlif, Felix Voituret, and Manuel Moussallam,  “Spleeter: a fast and efficient music source separation tool with pre-trained models,”  Journal of Open Source Software, p. 2154, 2020.
  * [9] Naoya Takahashi and Yuki Mitsufuji,  “Densely connected multi-dilated convolutional networks for dense prediction tasks,”  in IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2021, pp. 993–1002.
  * [10] Olga Slizovskaia, Leo Kim, Gloria Haro, and Emilia Gomez,  “End-to-end sound source separation conditioned on instrument labels,”  in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 306–310.
  * [11] Gabriel Meseguer-Brocal and Geoffroy Peeters,  “Conditioned-u-net: Introducing a control mechanism in the u-net for multiple source separations,”  in Proc. of the 20th International Society for Music Information Retrieval Conference (ISMIR), 2019.
  * [12] Prem Seetharaman, Gordon Wichern, Shrikant Venkataramani, and Jonathan Le Roux,  “Class-conditional embeddings for music source separation,”  in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 301–305.
  * [13] Woosung Choi, Minseok Kim, Jaehwa Chung, and Soonyoung Jung,  “Lasaft: Latent source attentive frequency transformation for conditioned source separation,”  in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 171–175.
  * [14] David Samuel, Aditya Ganeshan, and Jason Naradowsky,  “Meta-learning extractors for music source separation,”  in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 816–820.
  * [15] Jie Hwan Lee, Hyeong-Seok Choi, and Kyogu Lee,  “Audio query-based music source separation,”  in Proc. of the 20th International Society for Music Information Retrieval Conference, (ISMIR), 2019, pp. 878–885.
  * [16] Rajath Kumar, Yi Luo, and Nima Mesgarani,  “Music Source Activity Detection and Separation Using Deep Attractor Network,”  in Interspeech, 2018, pp. 347–351.
  * [17] Ethan Manilow, Gordon Wichern, and Jonathan Le Roux,  “Hierarchical musical instrument separation,”  in Proc. of the 21st International Society for Music Information Retrieval (ISMIR) Conference, 2020, pp. 376–383.
  * [18] Beat Gfeller, Dominik Roblek, and Marco Tagliasacchi,  “One-shot conditional audio filtering of arbitrary sounds,”  in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 501–505.
  * [19] Yuan-Kuei Wu, Kuan-Po Huang, Yu Tsao, and Hung-yi Lee,  “One shot learning for speech separation,”  in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 5769–5773.
  * [20] Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron C. Courville,  “Film: Visual reasoning with a general conditioning layer,”  in AAAI, 2018.
  * [21] Alice Cohen-Hadria, Axel Roebel, and Geoffroy Peeters,  “Improving singing voice separation using deep u-net and wave-u-net with data augmentation,”  in 27th European Signal Processing Conference (EUSIPCO). IEEE, 2019, pp. 1–5.
  * [22] Hyeong-Seok Choi, Jang-Hyun Kim, Jaesung Huh, Adrian Kim, Jung-Woo Ha, and Kyogu Lee,  “Phase-aware speech enhancement with deep complex u-net,”  in International Conference on Learning Representations, 2018.
  * [23] Shrikant Venkataramani, Jonah Casebeer, and Paris Smaragdis,  “Adaptive front-ends for end-to-end source separation,”  in Proc. NIPS, 2017.
  * [24] Zafar Rafii, Antoine Liutkus, Fabian-Robert Stöter, Stylianos Ioannis Mimilakis, and Rachel Bittner,  “The MUSDB18 corpus for music separation,” 2017.
  * [25] Rachel Bittner, Justin Salamon, Mike Tierney, Matthias Mauch, Chris Cannam, and Juan Bello,  “Medleydb: A multitrack dataset for annotation-intensive mir research,”  in Proc. of the 15th International Society for Music Information Retrieval Conference (ISMIR), 2014.
  * [26] Ethan Manilow, Gordon Wichern, Prem Seetharaman, and Jonathan Le Roux,  “Cutting music source separation some Slakh: A dataset to study the impact of training data quality and quantity,”  in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2019.
  * [27] E. Vincent, R. Gribonval, and C. Fevotte,  “Performance measurement in blind audio source separation,”  IEEE Transactions on Audio, Speech, and Language Processing, pp. 1462–1469, 2006.