The Sørensen–Dice Coefficient has recently seen rising popularity as a loss function (also known as Dice loss) due to its robustness in tasks where the number of negative samples significantly exceeds that of positive samples, such as semantic segmentation, natural language processing, and sound event detection. Conventional training of polyphonic sound event detection systems with binary cross-entropy loss often results in suboptimal detection performance as the training is often overwhelmed by updates from negative samples. In this paper, we investigated the effect of the Dice loss, intra- and inter-modal transfer learning, data augmentation, and recording formats, on the performance of polyphonic sound event detection systems with multichannel inputs. Our analysis showed that polyphonic sound event detection systems trained with Dice loss consistently outperformed those trained with cross-entropy loss across different training settings and recording formats in terms of score and error rate. We achieved further performance gains via the use of transfer learning and an appropriate combination of different data augmentation techniques, which mitigate the problem of lacking training data.

Polyphonic sound event detection, microphone array, first-order ambisonics, dice loss, deep learning

§ INTRODUCTION

Sound event detection (SED) is the task of jointly detecting the onset, offset, and class of a sound event. Polyphonic SED refers specifically to the task of such detection for multiple, potentially overlapping sound events simultaneously. In the past decade, there have been significant advances in the applications of deep learning for SED [1]. The state-of-the-art SED models have been built from convolutional neural networks (CNN) [2], convolutional recurrent neural networks (CRNN) [3], and more recently, Conformer [4].

Majority of works in SED targeted single-channel inputs, due to the availability of large-scale datasets such as AudioSet [5] and FSD50k [6], and ease of practical deployment. For multichannel SED (MSED), there are only a few publicly-available labelled datasets, often small-scale, e.g., TUT Sound Events 2016 [7], and TAU-NIGENS Spatial Sound Events 2021 [8]. In addition, many MSED works naturally focused on multichannel and spatial feature engineering [9, 10] due to the different multichannel formats available. As a result, there is a severe lack of literature concerning the impact of different data augmentation techniques, network architectures, and loss functions on the performance of MSED models that are trained on small datasets.

In this paper, we present a thorough investigation on the factors affecting the performance of deep polyphonic MSED systems. We demonstrated the effectiveness of using pretrained weights from single-channel model for MSED to tackle the lack of training data. We also investigated the use of dice loss, which has been shown to improve single-channel SED performance [11]. Several experiments on the data augmentation techniques, training chunk durations, pretraining modalities, and multichannel audio formats were performed and their results were analyzed and discussed to better inform our understanding of polyphonic MSED systems. The rest of the paper is organized as follows. <Ref> describes our experiment setups. <Ref> presents the experimental results and discussion. Finally, we conclude the paper in <Ref>.

§ PRELIMINARIES

§.§ Dataset

In this paper, we used the TAU-NIGENS Spatial Sound Events 2021 (SSE21) dataset [8], which provides two four-channel array formats: first-order ambisonics (FOA) and microphone array (MIC). Each format has 400, 100, and 100 60-second recordings for training, validation, and testing, respectively. Both formats were used to train and evaluate the performance of the SED networks. Since SSE21 is a sound event localization and detection (SELD) dataset with directional interferences, we only take the SED labels. Hence, we have strongly-labeled sound events from the $L=12$ target classes and unlabelled out-of-class interferences.

§.§ Input features

Polyphonic  audio can generally be modelled in the time-frequency (TF) domain by

\begin{equation}
\textstyle\mbf{X}[t, f] = \sum_i \mbf{H}_i[t, f]S_i[t, f] + \mbf{N}[t, f]\in\mathbb{C}^C,
\end{equation}

where $S_i[t, f]\in\mathbb{C}$ is the $i^{th}$ sound source, $\mbf{H}_i[t, f]\in\mathbb{C}^{C}$ is the corresponding array response vector from the source to the sensor array, and $\mbf{N}[t, f]\in\mathbb{C}^{C}$ is the noise vector.

This representation is typically achieved by applying the short-time Fourier transform (STFT) on the multichannel time-domain audio data. A Hann window of size $R=1024$ with hop size 300 was used for this paper.

For the FOA format, the array response vector is given by

\begin{equation}
\mbf{H}^\text{(FOA)}_i[t, f]
= \begin{bmatrix}
H_{i, \text{w}}[t] \\
H_{i, \text{y}}[t] \\
H_{i, \text{z}}[t] \\
H_{i, \text{x}}[t]
\end{bmatrix} = \begin{bmatrix}
1 \\
\sin(\phi_i[t]) \cos(\theta_i[t])\\
\sin(\theta_i[t]) \\
\cos(\phi_i[t]) \cos(\theta_i[t])
\end{bmatrix} \in \mathbb{R}^{4},
\end{equation}
where $\phi_i[t]$ and $\theta_i[t]$ are the time-dependent azimuth and elevation angles of the $i^{th}$ sound source with respect to the array, respectively.

For the microphone format, SSE21 provides a tetrahedral array of microphones mounted on a spherical baffle, whose response has a very complex analytical expression <cit.>. From our experience, we have found that a generic far-field array response can usually approximates the theoretical response satisfactorily. The far-field response for the $c^{th}$ channel at time $t$ is given by

\begin{equation}
H^\text{(MIC)}_{i,c}[t, f]
= \exp\left(-\dfrac{\jmath 2\pi f d_{i,c}[t]}{Rv/f_\text{s}}\right) \in \mathbb{C},
\end{equation}

up to some channel-dependent scaling, where $v \approx \SI{343}{\meter\per\second}$ is the speed of sound, and $d_{i,c}[t]$ is the projected difference in distances, in metres, travelled by the $i^{th}$ sound source arriving the $c^{th}$ microphone relative to the reference microphone.

In this paper, 128-bin log-mel spectrograms were used as input features, with cutoff frequencies at 50 and 12. To facilitate learning, z-score normalization was performed on each channel of the spectrograms along the mel-frequency bin axis.

§.§ Data augmentation

As with many audio datasets, SSE21 is relatively small, thus requiring data augmentation. In this paper, we experiment with combinations of four data augmentation techniques: mixup with soft label [13], frequency shifting, channel swapping [14], and a composite technique of cutout [15] and SpecAugment [16].

§.§ Loss function

In SED, the number of negative samples often significantly exceeds that of positive samples. Hence, training polyphonic SED systems with binary cross-entropy (BCE) loss often results in suboptimal score. An alternative loss function based on the Sørensen–Dice Coefficient (SDC) [17, 18], called Dice loss, was proposed in [19]. Variations of the Dice loss have since become popular as loss functions in semantic segmentation [20] and natural language processing. A variant was also recently used in single-channel SED [11].

The SDC between two sets $\mathfrak{A}$ and $\mathfrak{B}$ is defined by

\begin{equation}
\text{SDC}(\mathfrak{A}, \mathfrak{B})
= {2\lvert\mathfrak{A}\cap\mathfrak{B}\rvert}/{\left(\lvert\mathfrak{A}\rvert+\lvert\mathfrak{B}\rvert\right)}
\end{equation}

where $\lvert\cdot\rvert$ is the set cardinality operator. The SDC is used to gauge the similarity between two sets. In binary classification, the SDC is identical to the F1 score. Generalizing the SDC to tensors with elements in the unit interval, the cardinality operator can be replaced by the $L_1$-norm, $\|\cdot\|_1$, while the intersection operator can be replaced by the Hadamard multiplication, $\circ$. As such, a differentiable Dice loss for a batch of $N$ predictions is given by

\begin{equation}
\mathcal{L}_\text{Dice}(\hat{\mbf{Y}}, \mbf{Y})
= \dfrac{1}{N}\sum_n \left[1 - \dfrac{2\|\widehat{\mbf{Y}}_n \circ  \mbf{Y}_n\|_1}{\|\widehat{\mbf{Y}}_n+\mbf{Y}_n\|_1 + \epsilon}\right]
\end{equation}

where $\widehat{\mbf{Y}}_n\in[0,1]^{T\times L}$ is the prediction tensor, $\mbf{Y}_n\in\{0,1\}^{T\times L}$ is the target tensor, $\epsilon>0$ is a small stabilizing constant, $T$ is the number of label segments, and $L$ is the number of target sound classes.

In this paper, we experiment with the BCE-Dice loss, which is a combination of the BCE and Dice losses, given by

\begin{equation}
\mathcal{L}_\text{BCE-Dice}(\hat{\mbf{Y}}, \mbf{Y})
= \mathcal{L}_\text{BCE}(\hat{\mbf{Y}}, \mbf{Y}) + \mathcal{L}_\text{Dice}(\hat{\mbf{Y}}, \mbf{Y}).
\end{equation}

§.§ Network architecture

The general architecture of the SED network used in this paper is that of a CRNN, a de facto standard architecture for SED [3, 1]. The input is passed through a CNN backbone and pooled in the frequency axis before being passed into an RNN and a fully-connected (FC) output layer.

In this paper, we experiment with a number of different CNN backbones from both audio and vision domains, such as Pretrained Audio Neural Networks (PANNs) [2], ResNet [21], EfficientNet [22], DenseNet [23]. For ease of comparison, we only use a single-layer 256-unit bidirectional gated recurrent unit (BiGRU) as the RNN network. PyTorch Image Models implementations [24] were adapted for all CNN backbones except PANNs.

§.§ Transfer learning

Early research works into the applications of deep learning in the audio domain suffered severely from the lack of data.

Many early audio systems relied on the AudioSet-pretrained VGGish model [25, 5]. The use of intra-modal (audio-for-audio) transfer learning remains popular as seen by the frequent use of PANNs [2] in many audio systems today. Recently, more works have successfully attempted inter-modal (vision-for-audio) transfer learning by fine-tuning networks with CNN backbones pretrained on ImageNet [26, 27, 28, 29]. Unfortunately, research into the effect of modalities on transfer learning for audio systems remains very limited [30].

In this paper, we investigate the effect of modalities in transfer learning for SED, using CNN backbones pretrained on either single-channel AudioSet [5] or three-channel ImageNet [29]. Since SSE21 provides four-channel inputs, the weight replication technique from [24] is used on the first convolutional layer in order to match the original pretrained weights to the four-channel inputs.

§.§ Training procedures

Recordings from the training set are chunked into four-second segments with a hop size of 0.5 unless otherwise stated. All models are trained for 30 epochs with a batch size of 32. We use the Adam optimizer [31] with a learning rate schedule shown in <Ref>.

xmin=-5, xmax=105,
ymin=-4.5, ymax=-2.5,
xtick=0, 10, 70, 100,
ytick=-3, -4,
yticklabels=e-3, e-4,
axis lines=left,

ylabel near ticks,
xlabel near ticks
[blue,fill=blue] (axis cs:0,-4) circle (1pt) – (axis cs:10,-3) circle (1pt) – (axis cs:70,-3) circle (1pt) – (axis cs:100,-4) circle (1pt);

Learning rate schedule w.r.t. training epoch (%)

§.§ Evaluation metrics

We evaluate the SED performance using the segment-based score and error rate (ER) with a segment length of 1, which have become the standard metrics for SED [32, 1].

We combine the ER and score into a single metric for checkpoint selection as follows: $\text{SEDE} = 0.5 \cdot\text{ER} + 0.5\cdot(1 - \text{\Fl})$. A lower SEDE generally indicates better model performance. All reported results were evaluated on the test set with a binarization threshold of 0.3.

§ EXPERIMENTS, RESULTS, AND DISCUSSION

§.§ Experiment 1: Data augmentation

The first experiment investigates the effect of data augmentation techniques on the MSED performance. All models in this experiment used PANN ResNet22 backbone and were trained from scratch with the BCE loss. All 16 combinations of the four augmentation techniques in <Ref> were tested. All techniques except mixup were each applied with an independent activation probability of 0.5. Mixup used an activation probability of 0.8.

Mixup (MU) uses soft label mixing with the weight drawn from $\text{Beta}(0.5, 0.5)$ and no mixing is applied if the weight lies in $[0.3, 0.7]$. Frequency shifting (FS) randomly shifts the spectrogram up or down by up to 10 mel bins. Channel swapping (CS) randomly selects one of the 24 permutations of the four channels.

The composite cutout-SpecAugment technique (CO) goes as follows. First, one of the these three techniques is chosen at random: single cutout, multiple cutouts, and SpecAugment. Single cutout fills a same-ratio rectangular section randomly located on the spectrogram, sized between 2 and 30 of the total area, with a constant value drawn randomly from the support of the original input. Multiple cutouts is similar but with 8 randomly located sections of size 8 bins by 8 frames each. SpecAugment uses a time stripe with a width up to 15 of the frames and a frequency strip with a width up to 20 of the mel bins.

The results are shown in <Ref>. For the FOA format, the ER is the lowest with all four augmentation techniques applied, while the score is the highest when all except mixup was applied, although the score for the all-four setting is very close to the no-mixup setting. For the MIC format, the best performance is achieved using the combination of all techniques except mixup. In fact, adding mixup seems to generally worsen the performance compared to the same combination without mixup. FS seems particular useful for both format. CS is useful for FOA format while CO is useful for MIC format. Majority of methods work better in combination than standalone. Interestingly, with or without data augmentation, MIC format provides better performance than FOA format.

We suspect that the difference in results is due to the difference in how spatial information is stored in the FOA and MIC formats. All channels of MIC formats tend to have similar magnitude information since spatial information is mostly encoded in the phase. The drop in performance when channel swapping was applied on MIC format supports this; we suspect swapping in the MIC format introduces little to no new information in the augmented samples. Moreover, sound event overlaps on spectrograms in the MIC format are often irresolvable without phase information. Hence, mixup likely made the overlapping sound events too difficult to detect by the network. On the other hand, the FOA format encodes spatial information via the relative intensity across the channels with little spatial information in the phase. As such, when mixup is applied, sound events originating from distinct directions are more likely to be resolvable, adding variety to the sound samples and allowing better detection performance. For the subsequent experiments, all four data augmentation techniques were used for FOA format while all except mixup was applied for MIC format.

4cData Augmentation

$\text{ER} \downarrow$
$\text{F\textsubscript{1}} \uparrow$
$\text{ER} \downarrow$
$\text{F\textsubscript{1}} \uparrow$

0.459     0.670
0.420     0.714

0.452     0.681
0.438     0.697

0.464     0.665
0.398     0.725

0.427     0.694
0.395     0.721

0.446     0.679
0.458     0.675

0.449     0.689
0.445     0.683

0.424     0.695
0.404     0.723

0.471     0.658
0.479     0.649

0.404     0.720
0.344     0.775

0.470     0.658
0.458     0.666

0.389     0.737
0.413     0.702

0.410     0.700
0.432     0.687

0.434     0.684
0.432     0.687

0.434     0.684
0.405     0.707

0.390     0.736
0.339     0.775

0.384     0.734
0.424     0.698

SED performance w.r.t. data augmentation.

§.§ Experiment 2: Loss function

Loss function
$\text{ER} \ \downarrow$
$\text{F\textsubscript{1}} \uparrow$
$\text{ER} \downarrow$
$\text{F\textsubscript{1}} \uparrow$

0.384     0.734
0.339     0.775

0.371     0.739
0.353     0.760

BCE + Dice
0.372     0.744
0.377     0.728

BCE + Dice
0.337     0.762
0.332     0.779

SED performance w.r.t. loss function and transfer learning.

The second experiment investigates the effect of loss functions on the performance of SED models with and without transfer learning.

The experiment settings were identical to that of Experiment 1.

The results for this experiment are shown in <Ref>. For both formats, the best performance was achieved with both transfer learning and the BCE-Dice loss function.
This result is somewhat expected as the Dice loss is generally more robust to positive-negative imbalance in the dataset than the BCE loss, and can be considered a soft approximation of the binary score [33]. Interestingly, using only either the BCE-Dice loss or pretraining worsened the performance in the MIC format, but using both improved the performance.
This is not observed in the FOA format, where using either technique improved the performance, and using both improved the performance further.

§.§ Experiment 3: Training chunk size

1lChunk size ()
$\text{ER} \ \downarrow$
$\text{F\textsubscript{1}} \uparrow$
$\text{ER} \downarrow$
$\text{F\textsubscript{1}} \uparrow$

4.0     0.337     0.762     0.332     0.779

8.0     0.339     0.764     0.349     0.765

12.0     0.318     0.792     0.326     0.770

SED performance w.r.t. training chunk size.

The third experiment investigates the impact of the training chunk size on the SED performance. For reference, event lengths for SSE21 have a median of 3.2 and a mean of 8.3. We used both transfer learning and the BCE-Dice loss for training. The rest of the settings followed Experiment 2.

The results for this experiment are shown in <Ref>. For the FOA format, increasing the chunk size from 4 to 8 barely improved the performance, but increasing to 12 resulted in a some appreciable gain. The results for the MIC format are less consistent, with the 8 chunk performing the worst. Further analysis of SSE21 revealed some sound classes with a significant number of events whose durations are beyond 8. Insufficient coverage by the 8 chunk may explain the lack in performance gain, as training difficulty increased beyond the additional information gained.

§.§ Experiment 4: Input channels

The fourth experiment compares the performance of SED networks using single-channel inputs against those using the full four-channel array inputs. For the single-channel setting, the omnidirectional (W) channel was used for the FOA format while the first channel was used for the MIC format. A chunk size of 4s was used and the rest of the settings followed Experiment 3.

The results are shown in <Ref>. With the addition of more channels, the FOA format achieved very little performance gain, if any, whereas the MIC format gained significant improvement. The similar performances for the FOA format are likely because the information in the W channel was already acquired indirectly via multiple microphones in an ambisonic array (32 sensors for SSE21). On the other hands, each channel in the MIC format only acquired information from a single sensor (which is also more representative of a single-sensor signal acquisition in practice). As such, the information gained from adding more channels in the MIC format is more significant, compared to the mostly spatial information gained from more channels in the FOA format.

$\text{ER} \ \downarrow$
$\text{F\textsubscript{1}} \uparrow$
$\text{ER} \downarrow$
$\text{F\textsubscript{1}} \uparrow$

Mono        0.340     0.763     0.375     0.740

All         0.337     0.762     0.332     0.779

SED performance w.r.t. number of channels.

§.§ Experiment 5: Modalities in transfer learning

$\text{ER} \ \downarrow$
$\text{F\textsubscript{1}} \uparrow$
$\text{ER} \downarrow$
$\text{F\textsubscript{1}} \uparrow$

PANN ResNet22            16        Audio
0.337      0.762
0.332      0.779

PANN CNN14          16        Audio
0.394         0.739
0.357	        0.767

ResNet18*           16        None
0.371         0.744
0.378         0.749

PANN ResNet22            16        None
0.372         0.744
0.377         0.728

EfficientNetB0      32        Image
0.398         0.722
0.376         0.743

PANN CNN14          16        None
0.446         0.685
0.365	        0.760

ResNet18*           16        Image
0.437         0.680
0.365         0.743

ResNet18            32        None
0.430         0.701
0.415         0.720

DenseNet121         32        Image
0.434         0.688
0.421         0.731

ResNet18            32        Image
0.423         0.706
0.423         0.698

EfficientNetB0      32        None
0.463         0.673
0.454         0.692

DenseNet121         32        None
0.474         0.650
0.433         0.689

SED performance w.r.t. the CNN backbone and transfer learning, listed in decreasing average SEDE. The ResNet18* model was modified from ResNet18 by removing the first pooling layer.

In the last experiment, we investigate the impact of pretraining modalities in transfer learning (TL) on the MSED performance with various CNN backbones. The settings followed Experiment 4 with all channels as inputs.

The results for this experiment are shown in <Ref>. For both formats, the pretrained networks generally perform better than their counterparts trained from scratch. This confirms the conventional wisdom that transfer learning, from the same or a related domain, allows deep networks to utilize some previously learnt patterns which are useful to SED. The ResNet18* backbone had some interesting results: With ResNet18*, where the original weights meant for ResNet18 were used, pretraining worsened performance in the FOA format but improved performance in the MIC format.

In terms of architecture, it is clear that models with ResNet-based or CNN14 backbones outperformed those with EfficientNet or DenseNet backbone when trained from scratch. DenseNet architecture is a known to cause aliasing problems [34] which may explain the poor performance. EfficientNet has building blocks which consist of information compressing layers, potentially causing in compounding information loss when trained on small datasets.

In terms of modalities, comparing the PANNs pretrained on AudioSet to other CNNs pretrained on ImageNet, it is clear networks with AudioSet-pretrained backbones outperformed those with ImageNet-pretrained backbones. This is consistent with the findings from [30] where intra-modal transfer learning outperformed inter-modal ones. Since higher-level features in audio spectrograms are very different from higher-level features in images, we suspect that higher-level representation prelearning of audio-related patterns may only be achieved via intra-modal pretraining, even if pretraining on vision datasets gives the network a headstart in learning lower-level visual representation.

§ CONCLUSION

In this paper, we performed a thorough investigation on the factors affecting the performance of deep multichannel polyphonic SED systems: data augmentation techniques, loss functions, training chunk durations, pretraining modalities, and multichannel audio formats. A proper combination of data augmentation techniques can mitigate the problem of small datasets, and significantly improve the model performance. Further improvement can be achieved by using the BCE-Dice loss and transfer learning. We showed that both inter- and intra-model transfer learning from other datasets increases the SED performance on multi-channel datasets, even with channel mismatch, with intra-model transfer learning providing higher performance gains. Interestingly, across many settings, the best performances of MIC format are slightly higher than those of FOA format.

[1]
A. Mesaros, T. Heittola, T. Virtanen, and M. D. Plumbley, “Sound Event
Detection: A Tutorial,” IEEE Signal Processing
Magazine, vol. 38, no. 5, 2021.

[2]
Q. Kong, Y. Cao, T. Iqbal, Y. Wang, W. Wang, and M. D. Plumbley, “PANNs:
Large-Scale Pretrained Audio Neural Networks for Audio Pattern
Recognition,” IEEE/ACM Transactions on Audio
Speech and Language Processing, vol. 28, pp. 2880–2894, 2020.

[3]
E. Cakir, G. Parascandolo, T. Heittola, H. Huttunen, and T. Virtanen,
“Convolutional Recurrent Neural Networks for Polyphonic Sound Event
Detection,” IEEE/ACM Transactions on Audio
Speech and Language Processing, vol. 25, no. 6, pp. 1291–1303, 2017.

[4]
K. Miyazaki, T. Komatsu, T. Hayashi, S. Watanabe, T. Toda, K. Takeda, and
L. Corporation, “Conformer-Based Sound Event Detection With Semi-Supervised
Learning and Data Augmentation,” in Proceedings
of the 5th Workshop on Detection and Classification of Acoustic Scenes and
Events, 2020, pp. 100–104.

[5]
J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore,
M. Plakal, and M. Ritter, “Audio Set: An ontology and human-labeled dataset
for audio events,” in Proceedings of the IEEE
International Conference on Acoustics, Speech and Signal Processing, 2017,
pp. 776–780.

[6]
E. Fonseca, J. Pons, X. Favory, F. Font, D. Bogdanov, A. Ferraro, S. Oramas,
A. Porter, and X. Serra, “Freesound datasets: A platform for the creation
of open audio datasets,” in Proceedings of the
18th International Society for Music Information Retrieval Conference,
2017, pp. 486–493.

[7]
A. Mesaros, T. Heittola, and T. Virtanen, “TUT database for acoustic scene
classification and sound event detection,” in
Proceedings of the European Signal Processing
Conference, 2016, pp. 1128–1132.

[8]
A. Politis, S. Adavanne, D. Krause, A. Deleforge, P. Srivastava, and
T. Virtanen, “A Dataset of Dynamic Reverberant Sound Scenes with
Directional Interferers for Sound Event Localization and Detection,”
arXiv, 2021.

[9]
S. Adavanne, P. Pertila, and T. Virtanen, “Sound event detection using
spatial features and convolutional recurrent neural network,” in
Proceedings of the IEEE International Conference
on Acoustics, Speech and Signal Processing, 2017, pp. 771–775.

[10]
T. N. T. Nguyen, D. L. Jones, and W. S. Gan, “On the Effectiveness of Spatial
and Multi-Channel Features for Multi-Channel Polyphonic Sound Event
Detection,” in Proceedings of the 5th Workshop
on Detection and Classification of Acoustic Scenes and Events, 2020, pp.

[11]
K. Imoto, S. Mishima, Y. Arai, and R. Kondo, “Impact of Sound Duration and
Inactive Frames on Sound Event Detection Performance,” in
Proceedings of the IEEE International Conference
on Acoustics, Speech and Signal Processing, 2021, pp. 860–864.

[12]
A. Politis, S. Adavanne, and T. Virtanen, “A Dataset of Reverberant Spatial
Sound Scenes with Moving Sources for Sound Event Localization and
Detection,” arXiv, 2020.

[13]
H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “MixUp: Beyond empirical
risk minimization,” in Conference Track
Proceedings of the 6th International Conference on Learning
Representations, 2018.

[14]
L. Mazzon, Y. Koizumi, M. Yasuda, and N. Harada, “First Order Ambisonics
Domain Spatial Augmentation for DNN-based Direction of Arrival Estimation,”
in Proceedings of the 4th Workshop on Detection
and Classification of Acoustic Scenes and Events, 2019, pp. 154–158.

[15]
T. DeVries and G. W. Taylor, “Improved Regularization of Convolutional Neural
Networks with Cutout,” arXiv, 2017.

[16]
D. S. Park, W. Chan, Y. Zhang, C. C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le,
“SpecAugment: A simple data augmentation method for automatic speech
recognition,” in Proceedings of the Annual
Conference of the International Speech Communication Association, 2019, pp.

[17]
L. R. Dice, “Measures of the Amount of Ecologic Association Between
Species,” Ecology, vol. 26, no. 3, pp.
297–302, 1945.

[18]
T. Sørensen, “A method of establishing roups of equal amplitude in plant
sociology based on similarity of species and its application to analyses of
the vegetation on Danish commons,” Kongelige
Danske Videnskabernes Selskab, vol. 5, no. 4, pp. 1–34, 1948.

[19]
F. Milletari, N. Navab, and S. A. Ahmadi, “V-Net: Fully convolutional neural
networks for volumetric medical image segmentation,” in
Proceedings of the 4th International Conference
on 3D Vision, 2016, pp. 565–571.

[20]
C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M. J. Cardoso,
“Generalised Dice overlap as a deep learning loss function for highly
unbalanced segmentations,” in Deep Learning in
Medical Image Analysis and Multimodal Learning for Clinical Decision
Support.1em plus 0.5em minus 0.4emSpringer, 2017, pp.

[21]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” in Proceedings of the IEEE
Computer Society Conference on Computer Vision and Pattern Recognition,
2016, pp. 770–778.

[22]
M. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for
convolutional neural networks,” in Proceedings
of the 36th International Conference on Machine Learning, 2019, pp.
10 691–10 700.

[23]
G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely Connected
Convolutional Networks,” in Proceedings of the
30th IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp.

[24]
R. Wightman, “PyTorch Image Models,” 2019.

[25]
S. Hershey, S. Chaudhuri, D. P. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore,
M. Plakal, D. Platt, R. A. Saurous, B. Seybold, M. Slaney, R. J. Weiss, and
K. Wilson, “CNN architectures for large-scale audio classification,” in
Proceedings of the IEEE International Conference
on Acoustics, Speech and Signal Processing, 2017, pp. 131–135.

[26]
Y. Kawaguchi, K. Imoto, Y. Koizumi, N. Harada, D. Niizumi, K. Dohi, R. Tanabe,
H. Purohit, and T. Endo, “Description and Discussion on DCASE 2021
Challenge Task 2: Unsupervised Anomalous Sound Detection for Machine
Condition Monitoring under Domain Shifted Conditions,”
arXiv, 2021.

[27]
R. Giri, S. V. Tenneti, F. Cheng, and K. Helwani, “Unsupervised Anomalous
Sound Detection Using Self-Supervised Classification and Group Masked
Autoencoder for Density Estimation,” IEEE AASP
Challenge on Detection and Classification of Acoustic Scenes and Events
2020, no. 2, 2020.

[28]
S. Amiriparian, M. Gerczuk, S. Ottl, L. Stappen, A. Baird, L. Koebe, and
B. Schuller, “Towards cross-modal pre-training and learning tempo-spatial
characteristics for audio recognition with convolutional and recurrent neural
networks,” Eurasip Journal on Audio, Speech,
and Music Processing, vol. 2020, no. 1, pp. 1–11, 2020.

[29]
J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei, “ImageNet:
A large-scale hierarchical image database,” in
Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, 2010, pp. 248–255.

[30]
T. Koike, K. Qian, Q. Kong, M. D. Plumbley, B. W. Schuller, and Y. Yamamoto,
“Audio for Audio is Better? An Investigation on Transfer Learning Models
for Heart Sound Classification,” in Proceedings
of the Annual International Conference of the IEEE Engineering in Medicine
and Biology Society, 2020, pp. 74–77.

[31]
D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,”
Conference Track Proceedings of the 3rd
International Conference on Learning Representations, 2014.

[32]
A. Mesaros, T. Heittola, and T. Virtanen, “Metrics for polyphonic sound event
detection,” Applied Sciences (Switzerland),
vol. 6, no. 6, p. 162, 2016.

[33]
X. Li, X. Sun, Y. Meng, J. Liang, F. Wu, and J. Li, “Dice Loss for
Data-imbalanced NLP Tasks,” in Proceedings of
the 58th Annual Meeting of the Association for Computational Linguistics,
2020, pp. 465–476.

[34]
N. Takahashi and Y. Mitsufuji, “Densely connected multidilated convolutional
networks for dense prediction tasks,” in
Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, 2021.