# SNOBERT: A Benchmark for clinical notes entity linking
in the SNOMED CT clinical terminology

###### Abstract

The extraction and analysis of insights from medical data, primarily stored in
free-text formats by healthcare workers, presents significant challenges due
to its unstructured nature. Medical coding, a crucial process in healthcare,
remains minimally automated due to the complexity of medical ontologies and
restricted access to medical texts for training Natural Language Processing
models. In this paper, we proposed a method, ”SNOBERT,” of linking text spans
in clinical notes to specific concepts in the SNOMED CT using BERT-based
models. The method consists of two stages: candidate selection and candidate
matching. The models were trained on one of the largest publicly available
dataset of labeled clinical notes. SNOBERT outperforms other classical methods
based on deep learning, as confirmed by the results of a challenge in which it
was applied.

Index Terms—  NLP, SNOMED, BERT, Entity Linking

##  1 Introduction

Most medical data is stored in free-text documents, usually filled in by
healthcare workers. Analyzing this unstructured data can be challenging, as it
can be difficult to extract meaningful insights. Medical coding remains an
under-automated process despite being widely applicable in healthcare, medical
insurance, and medical research, mainly due to the vast amount of codes in
medical ontologies and the minimal access to medical texts for training
natural language processing systems [1]. In recent years, the problem of Named
Entity Recognition (NER) within medical texts has received increasing
attention from the research community [2]. By applying standardized
terminology, healthcare organizations can convert this free-text data into a
structured format that computers can readily analyze, stimulating the
development of new medicines, treatment pathways, and better patient outcomes.
One of the most comprehensive and multilingual clinical healthcare
terminologies in the world is Systematized Nomenclature of Medicine Clinical
Terms (SNOMED CT) [3], a systematically organized computer-processable
collection of medical terms that provides codes, terms, synonyms, and
definitions used in clinical documentation and reporting.

Annotating medical data according to SNOMED CT terminology is a time-consuming
and labor-intensive process that often requires the annotator to have prior
medical training. Automating such a process using Natural Language Processing
(NLP) methods is called an Entity Linking (EL) problem. EL is the task of
linking entities within a text to a suitable concept in a reference Knowledge
Graph [4]. This work presents a ”SNOBERT” method for linking text spans in
clinical notes with specific topics in the SNOMED CT clinical terminology that
distinguishes itself with a novel two-stage approach leveraging advanced NLP
models and a refined preprocessing strategy. The method was applied to the
”SNOMED CT Entity Linking Challenge” [5].

##  2 Related Works

With the recent advances of deep learning (DL) technologies, NLP applications
have received an unprecedented boost in performance [6]. However, DL has
rarely been used to solve the entity linking problem in SNOMED CT terminology
since the massive size of the corpus needed to train on such a large set of
classes [7]. Nevertheless, few papers have been published in the field.
Hristov et al. [1] proposed a method that integrates transformer-based models,
such as BERT, pre-trained on biomedical data, with support vector
classification using the transformer embeddings for fine-tuning and predicting
SNOMED CT codes for medical texts. This hybrid approach leverages the
strengths of both deep learning and classical machine learning techniques to
achieve high accuracy in medical text coding, particularly in morphology and
topography coding.

The KGE4SCT method [8] is a technique that utilizes Knowledge Graph Embeddings
(KGEs) to automatically post-coordinate SNOMED CT clinical terms. It does this
by using a vector space to capture the ontology’s graph-like structure. The
method uses vector similarity and analogies to derive post-coordinated
expressions for clinical terms that are not explicitly present in SNOMED CT.
This facilitates the encoding of clinical information that has been extracted
from text. The effectiveness of this method has been validated on a subset of
SNOMED CT and a set of manually post-coordinated concepts.

##  3 Data

In this work, we used the MIMIC-IV-Note dataset, which contains 331,794 de-
identified hospital discharge summaries from 145,915 patients provided by the
Beth Israel Deaconess Medical Center (BIDMC) and Massachusetts Institute of
Technology (MIT) [9]. The challenge provided annotated data, comprising up to
300 annotated discharge summaries from the original MIMIC-IV-Note dataset. The
full dataset consists of a public training subset (204 notes) and a private
test subset (around 70 notes) comprising approximately 75,000 annotations
across discharge summaries.

Each entry in the discharge dataset includes a note ID and the anonymized
discharge text. Annotations indicate the concept ID, start and end points, and
the corresponding note ID. Each note ID consists of a subject ID, note
sequence position number, and note type (in this work, we use only
discharges). Annotations consist of note IDs, starting and end points of
concepts according to SNOMED CT clinical terminology, and their corresponding
IDs. Fig.1 shows an example of an annotated part of a synthetic discharge note
according to the SNOMED CT terminology.

Fig. 1: Example of synthetic discharge note annotated according to SMOMED CT
terminology, e.g. ”blockage” corresponds to ”235918000” concept ID.

Medical notes frequently use abbreviations that can be context-dependent and
may assume prior knowledge. In addition, the knowledge bases used in medical
notes can contain hundreds of thousands of concepts, with many of these
concepts occurring infrequently. As a result, there can be a ”long tail”
effect in the distribution of concepts, Fig.2. Thus, 2162 concepts out of 5336
appear only once in the annotated data. This effect causes zero-shot learning
(ZSL), a problem in deep learning in which, at test time, a learner observes
samples from classes that were not observed during training and needs to
predict the class to which they belong.

Fig. 2: Distribution of the concepts in the annotated dataset: ”long tail”
distribution effect.

##  4 Method

This section presents the proposed method for the clinical notes EL. Fig.3
illustrates the scheme of the method that consists of two stages: Candidate
Selection and Candidate Matching. In the first stage, we solved the NER
classification problem, and in the second stage, for each classified span from
the first stage, we linked the corresponding concept ID in SNOMED terminology.
In further sections, we describe each of the stages in detail.

Fig. 3: SNOBERT scheme. The method consists of two stages. In the Candidate
Selection stage (I), the BERT model is utilized to classify the text’s tokens
into seven classes. In the Candidate Matching stage (II), the Mention Encoder
matches the extracted embeddings from the training and testing datasets within
these classes. Reranker is used to rerank the top matches and to get the final
similarity score.

Fig. 4: Training pipeline of the proposed approach. The method uses a two-
stage solution: Candidate Selection and Candidate Matching. All the models
were trained on the MIMIC-IV dataset. The model from the first stage was
trained on the annotated training subset. An optional pretrain step was done
on the full unlabeled dataset. Models were evaluated on the test annotated
subset.

###  4.1 Preprocessing

We utilized the NER pipeline [10] to address certain annotation inaccuracies,
such as those caused by shifts due to tags. Approximately 10 notes out of 204
underwent corrections, involving adjustments to around 150 annotation IDs.
These corrections specifically targeted errors resulting from shifted
annotations. Furthermore, most annotated notes are missing some labels in the
paragraphs with the following headers: ’medications on admission:,’ ’___ on
admission:,’ ’discharge medications:.’ We excluded these parts from the
training process. All HTML markup elements, such as the line break element
(’br’) or the new line (’n’), have also been removed from the notes.

While a robust baseline for EL, the dictionary method falls short when it
comes to ZSL, highlighting the need for alternative solutions [11]. We
generate a static dictionary of the most common concepts from training data
and match them with test data in the post-processing step using a string-
matching search. Levenshtein ratio or Stolois distance could be utilized as a
matching metric. However, we applied ”one-to-one” matching, linking only
complete coincidences.

###  4.2 First Stage: Candidate Selection

NER is the task of identifying rigid designators’ mentions from text belonging
to predefined semantic types such as person, location, organization, etc.
[12]. NER involves processing raw text through stages, including Sentence
Segmentation, where text is divided into sentences, and Word Tokenization,
which breaks text into individual words. Subsequent stages include Part of
Speech Tagging, assigning grammatical tags based on word roles and context,
and Entity Detection, which identifies and categorizes key elements in the
text, highlighting the core function of NER in extracting meaningful
information from unstructured data.

We proposed to separate the concepts according to the first-level hierarchy
(the first entry in the path to the concept) according to SNOMED terminology.
Thus, we can emphasize the ”Finding,” ”Procedure,” ”Body structure,” and
”None” classes. During Tokenization, the words got flags in ”B-I-O” tagging
format: ”B” (Beginning) means the first token of the first word within an
annotation; ”I” (Inside) - is the first token of a subsequent word within an
annotation; ”O” (Outside) - is as a stand-alone token or single word token.
Consequently, we have seven classes: ”I-Finding,” ”B-Finding,” ”I-Procedure,”
”B-Procedure,” ”I-Body,” ”B-Body,” and ”O”.

###  4.3 Second Stage: Candidate Matching

To link classified terms from the first stage, we match their embeddings with
embeddings of terms from SNOMED terminology by cosine similarity. For this
purpose, the whole database extracted concepts from the ”Body structure,”
”Findings,” and ”Procedure” paths, which is about 200k unique IDs with the
Mention Encoder. In this work, we applied the cambridgeltl/SapBERT-from-
PubMedBERT-fulltext-mean-token model trained with UMLS 2020AA [13], using
microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext as the base
model [14].

###  4.4 Postprocessing

A reranker is used to improve the performance of the model’s initial
predictions. The model generates a list of possible predictions, which might
not be ranked optimally in order of correctness. A re-ranker evaluates these
predictions and adjusts their ranking based on more refined or specific
criteria. Using only the top one or top five predictions may cause the
correctly predicted vector to be missed. In our method, as a re-ranker, we
applied the MedCPT model [15] trained on 18M semantic query-article pairs from
PubMed.

###  4.5 Metrics

We evaluated the proposed method on both stages. First stage was evaluated
with Macro-F1 score across all labels:

$Macro-F1=\frac{1}{N}\sum^{N}_{i}{\frac{2\times Precision_{i}\times
Recall_{i}}{Precision_{i}+Recall_{i}}},$ (1)

where $N$ is a number of classes.

The second stage was evaluated with cosine similarity:

$cos(\theta)=\frac{\sum^{n}_{i}{A_{i}B_{i}}}{\sqrt{\sum^{n}_{i}{A^{2}_{i}}}\sqrt{\sum^{n}_{i}{B^{2}_{i}}}},$
(2)

where A and B are comparing vectors.

The final results were evaluated using a class macro-averaged character
intersection-over-union (mIoU). IoU is a popular metric for measuring
localization accuracy and computing localization errors. It calculates the
amount of overlap between a prediction and a ground truth:

$IoU_{class}=\frac{P^{char}_{class}\cap G^{char}_{class}}{P^{char}_{class}\cup
G^{char}_{class}},$ (3)

$macro\>IoU=\frac{\sum_{classes\in P\cup G}IoU_{class}}{N_{classes\in P\cup
G}},$ (4)

where $P_{class}^{char}$ is the set of characters in all predicted spans for a
given class category, $G_{class}^{char}$ is the set of characters in all
ground truth spans for a given class category, and $classes\in P\cup G$ are
the set of categories present in either the ground truth or the predicted
spans.

###  4.6 Training

For the training, we employed the 4-Fold Cross-Validation. Each fold consists
of 51 discharge notes. For the first stage, we used a domain-specific
pretrained language model for Biomedical Natural Language Processing [16]. We
utilized the base version, microsoft/BiomedNLP-BiomedBERT-base-uncased-
abstract-fulltext, which was pretrained on abstracts from PubMed and full-text
articles from PubMedCentral and is available in the HuggingFace repository
[14]. We ran four experiments for each setup so that three folds were used
each time for training and one for validation. We trained each split on 100
epochs using EarlyStoppingCriteria. Therefore, 75 epochs on average were
needed. We used ADAM optimizer with $3e^{-5}$, batch size of 8, and class
weighting. Training takes 30 minutes on 4 GPUs (NVIDIA A100-SXM4-40GB). A more
detailed description of the training configuration is shown in our training
repository [17].

We found a slight improvement of 0.0005 in IoU when applying the Masked
Language Model (MLM) pretraining technique. To accomplish this, we used the
larger microsoft/BiomedNLP-BiomedBERT-large-uncased-abstract weights as an
initial model instead. This optional pretraining step takes 24 hours on 4
GPUs.

##  5 Results

Table 1 shows the averaged cross-validation evaluation results for the first
(I) stage and final evaluation. MLM pretraining slightly improves the F1 score
in the first stage, and therefore, mIoU in the second with the best score of
0.4302 training on 4 GPUs. MultiGPU outstands single due to the batch size and
its synchronization in the Bert model.

Table 1: Evaluation metrics of the proposed method Model (I) | GPUs | F1 (I) | mIoU | Epoch
---|---|---|---|---
BiomedBERT large | 1 | 0.7429 | 0.4231 | 75
BiomedBERT base | 1 | 0.7487 | 0.4199 | 76
BiomedBERT large | 4 | 0.7514 | 0.4302 | 74
BiomedBERT base | 4 | 0.7499 | 0.4257 | 72

Table 2 shows the evaluation results of the second stage before reranking for
each class. The results for the top five best candidates are significantly
higher than those of the top one, which shows the need to use a reranker in
the postprocess, as the best candidate after the second stage is not always at
the top of the similarity score.

Table 2: Evaluation metrics of the second stage. Category | Similarity@1 | Similarity@5
---|---|---
Body structure | 0.715 | 0.811
Findings | 0.614 | 0.703
Procedure | 0.478 | 0.694

Table 3 shows the final test results of the three best methods and the
baseline of the competition. The dictionary-based method achieved the highest
score. However, this is a time-consuming method that requires manual effort.

Table 3: Results on the test dataset.

Author | Method | mIoU
---|---|---
Bilu et al. | Dictionary-based ∗ | 0.4202
Ours | SNOBERT | 0.4194
Popescu et al. | Faiss + Mistral | 0.3777
Baseline | deberta-v3-large | 0.1794

∗ Time-consuming semi-manual method.

##  6 Conclusion

We proposed a ”SNOBERT” method of EL of text spans in clinical notes with
specific topics in the SNOMED CT clinical terminology, tested in practice in
the ”SNOMED CT Entity Linking Challenge.” Our method uses two stages; however,
an end-to-end approach could improve the linking score [18] but would need
more training data from the other side. The limited annotation problem could
be solved using synthetic data generated with Large Language Models [19].

## References

  * [1] Anton Hristov et al.,  “Clinical text classification to snomed ct codes using transformers trained on linked open medical ontologies,”  in Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, 2023, pp. 519–526.
  * [2] Javier Reyes-Aguillón et al.,  “Clinical named entity recognition and linking using bert in combination with spanish medical embeddings.,”  in CLEF (Working Notes), 2022, pp. 341–349.
  * [3] Tim Benson,  Principles of health interoperability HL7 and SNOMED,  Springer Science & Business Media, 2012.
  * [4] Wei Shen, Jianyong Wang, and Jiawei Han,  “Entity linking with a knowledge base: Issues, techniques, and solutions,”  IEEE Transactions on Knowledge and Data Engineering, vol. 27, no. 2, pp. 443–460, 2014.
  * [5] Will Hardman and others.,  “Snomed ct entity linking challenge,”  2024\.
  * [6] Ivano Lauriola, Alberto Lavelli, and Fabio Aiolli,  “An introduction to deep learning in natural language processing: Models, techniques, and tools,”  Neurocomputing, vol. 470, pp. 443–456, 2022.
  * [7] Christophe Gaudet-Blavignac, Vasiliki Foufi, Mina Bjelogrlic, and Christian Lovis,  “Use of the systematized nomenclature of medicine clinical terms (snomed ct) for processing free text in health care: systematic scoping review,”  Journal of medical Internet research, vol. 23, no. 1, pp. e24594, 2021.
  * [8] Javier Castell-Díaz, Jose Antonio Miñarro-Giménez, and Catalina Martínez-Costa,  “Supporting snomed ct postcoordination with knowledge graph embeddings,”  Journal of Biomedical Informatics, vol. 139, pp. 104297, 2023.
  * [9] Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al.,  “Mimic-iv, a freely accessible electronic health record dataset,”  Scientific data, vol. 10, no. 1, pp. 1, 2023.
  * [10] Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov,  “Label Studio: Data labeling software,” 2020-2022,  Open source software available from https://github.com/heartexlabs/label-studio.
  * [11] Marco Basaldella, Fangyu Liu, Ehsan Shareghi, and Nigel Collier,  “COMETA: A corpus for medical entity linking in the social media,”  in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, Eds., Online, Nov. 2020, pp. 3122–3137, Association for Computational Linguistics.
  * [12] Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li,  “A survey on deep learning for named entity recognition,”  IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 1, pp. 50–70, 2022.
  * [13] Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier,  “Self-alignment pretraining for biomedical entity representations,”  in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2021, pp. 4228–4238.
  * [14] Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon,  “Domain-specific language model pretraining for biomedical natural language processing,” 2020.
  * [15] Qiao Jin, Won Kim, Qingyu Chen, Donald C Comeau, Lana Yeganova, W John Wilbur, and Zhiyong Lu,  “Medcpt: Contrastive pre-trained transformers with large-scale pubmed search logs for zero-shot biomedical information retrieval,”  Bioinformatics, vol. 39, no. 11, pp. btad651, 2023.
  * [16] Robert Tinn et al.,  “Fine-tuning large neural language models for biomedical natural language processing,” 2021.
  * [17] Mikhail Kulyabin et al.,  “A benchmark for clinical notes entity linking in the snomed ct clinical terminology,” https://github.com/MikhailKulyabin/SNOBERT.
  * [18] Tom Ayoola, Shubhi Tyagi, Joseph Fisher, Christos Christodoulopoulos, and Andrea Pierleoni,  “ReFinED: An efficient zero-shot-capable approach to end-to-end entity linking,”  in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track, Anastassia Loukina, Rashmi Gangadharaiah, and Bonan Min, Eds., Hybrid: Seattle, Washington + Online, July 2022, pp. 209–220, Association for Computational Linguistics.
  * [19] Sunjun Kweon et al.,  “Publicly shareable clinical large language model built on synthetic clinical notes,” 2023.


# Holographic dark energy with the sign-changeable interaction term

M. Abdollahi Zadeh1, A<EMAIL_ADDRESS>H.
<EMAIL_ADDRESS>1 Physics Department and Biruni
Observatory, College of Sciences, Shiraz University, Shiraz 71454, Iran
2 Research Institute for Astronomy and Astrophysics of Maragha (RIAAM), P.O.
Box 55134-441, Maragha, Iran

###### Abstract

We use three IR cutoffs, including the future event horizon, the Hubble and
Granda-Oliveros (GO) cutoffs, to construct three holographic models of dark
energy. Additionally, we consider a Friedmann-Robertson-Walker (FRW) universe
filled by a dark matter (DM) and a dark energy that interact with each other
through a mutual sign-changeable interaction. Thereinafter, we address the
evolution of the some cosmological parameters, such as the equation of state
and dimensionless density parameters of dark energy as well as the
deceleration parameter, during the cosmic evolution from the matter dominated
era until the late time acceleration. We observe that a holographic dark
energy (HDE) model with Hubble cutoff interacting with DM may be in line with
the current universe. Our study shows that models with the future event
horizon as the IR cutoff or the GO cutoff are in good agreement with the
observational data. In fact, we find out that these obtained models can
predict the universe transition from a deceleration phase to the acceleration
one in a compatible way with observations. The three obtained models may also
allow the equation of state parameter to cross the phantom line, a result
which depends on the values of the system’s constants such as the value of the
interaction coupling constant.

##  I Introduction

Observations data from type Ia supernovae (SNIa) Riess ; Riess1 ; Riess2 ;
Riess3 , the Large Scale Structure (LSS) COL2001 ; COL20011 ; COL20012 ;
COL20013  and the cosmic microwave background (CMB) anisotropies HAN2000 ;
HAN20001 ; HAN20002  obviously signal us an accelerating universe at the
present time. From theoretical point of view, this expanding phase of the
universe is supported by an unknown source called dark energy (DE) Roos . The
simplest model, which consists of a fluid with positive energy density and
negative pressure, is called cosmological constant and suffers from some
problems such as the fine-tuning and the coincidence problems Roos . Although,
many different models, such as quintessence Wetterich1988 ; Wetterich19881 ,
phantom (ghost) field Cald2002 ; Cald20021 , k-essence Chiba2000 ; Chiba20001
; Chiba20002 , Chaplygin gas Kam2001 ; Kam20011 , agegraphic DE Cai2007 ;
Cai20071 ; Cai20072 ; Cai20073 ; Cai20074 ; Cai20075 ; Cai20076 ; Cai20077 ;
Sheykhi2009 ; Sheykhi20091 ; Sheykhi20092 ; Sheykhi20093 ; Sheykhi20094  and
ghost DE models Ohta ; CaiG ; shemov ; SheyEb ; SEM ; SM  have been proposed
to justify the current accelerating phase of the universe expansion, DE is
still a confusing topic in modern cosmology Rev1 ; Rev2 ; Rev3  and its nature
and origin is an unknown problem.

Following the developments in quantum theory of gravity, the motivation to
solve the DE problem, including its nature and behavior, has arisen a lot of
attentions Witten . An example of such effort is the HDE models Cohen1999 ;
Li2004 ; Hsu2004 ; Pavon2005 ; Sheykhi2010 ; Shey2011 ; Weipil . The area-
entropy relation is the backbone of this hypothesis claiming that black hole
relates the short distance cutoff to the long distance cutoff, which leads to
an upper bound for the zero-point energy density Cohen1999 . Based on this
theory, we have $\rho_{D}=3c^{2}M_{p}^{2}L^{-2}$ for the energy density of
HDE, where $c^{2}$ and $L$ are, respectively, a constant and the infrared (IR)
cutoff, and $M_{p}$ is the reduced Planck mass Li2004 ; Hsu2004 . One of the
basic assumption in the HDE models is the suitable IR cutoff which leads to an
accelerated universe. The simplest choice for the IR cutoff is the Hubble
radius which is defined as, $L=H^{-1}$. However, it was argued that Hubble
radius cannot lead to an accelerating universe Li2004 , unless the interaction
between DE and DM is taken into account Pavon2005 ; Sheykhi2010 . In 2008,
Granda and Oliveros (GO) proposed a combination of the Hubble parameter and
its first derivative as the IR cutoff of HDE models, namely $L=(\alpha
H^{2}+\beta\dot{H})^{-1/2}$ where $\alpha$ and $\beta$ are constant Granda2008
; Granda2009 . This cutoff which is known as the GO cutoff avoids the
causality problem and solves the coincidence problem and extensively
investigated in the literatures Jamil1 ; GODGP .

Moreover, observations indicate that the possibility of a mutual interaction
between the DM and DE, which may solve the coincidence problem, is not zero
wang2016 . The simplest form of this mutual interaction can be written as
$Q=3b^{2}H\rho$ in which $b^{2}$ is a constant and $\rho$ can be the energy
density of DM, DE or even their sum wang2016 . Since $b^{2}$ is constant and
$H$ and $\rho$ are positive quantities, the sign of the interaction term is
not changed during the cosmic evolution. However, recent investigations
confirm that the sign of the interaction between DM and DE is changed during
the history of the universe and in particular in the redshift $0\cdot 45\leq
z\leq 0\cdot 9$ Cai2010 . Clearly, the sign-changeable interaction term cannot
be described by the $Q=3b^{2}H\rho$ expression. Following Cai2010 , Wei
proposed the sign-changeable interaction term as $Q=q(\alpha\dot{\rho}+3\beta
H{\rho})$ for the mutual interaction Wei2011 ; WEI2011 . In this new
expression, both $\alpha$ and $\beta$ are dimensionless constant and
$q=-1-\dot{H}/H^{2}$ is the deceleration parameter. The key ingredient is the
deceleration parameter $q$ in the interaction $Q$, and hence the interaction
$Q$ can change its sign when our universe changes from deceleration phase
$(q>0)$ to acceleration $(q<0)$. The $\alpha\dot{\rho}$ and $\beta H{\rho}$
terms are introduced from the dimensional point of view Wei2011 ; chimen1 ;
chimen2 , and one can ignore the $\alpha\dot{\rho}$ term by considering
$\alpha=0$ Wei2011 ; chimen1 ; chimen2 . It is worth noting that the
$Q(\rho,\dot{\rho})$ form has previously been introduced to describe the
mutual interaction between the dark sectors of cosmos chimen1 ; chimen2 . Now,
bearing the Firedmann equation in mind ($H^{2}\propto\rho$), we find out that
the $Q(\rho,\dot{\rho})$ interaction can indeed be written in the
$Q(H,\dot{H})$ form. On the other hand, since $q=q(H,\dot{H})$, the
$Q=q(\alpha\dot{\rho}+3\beta H{\rho})$ interaction can also be written in the
form $Q(H,\dot{H})$, meaning that in the Wei’s approach, the appearance of the
deceleration parameter, $q$, in the interaction term $Q$ is quite acceptable
Wei2011 ; WEI2011 .

In the present work, we are interested in studying the cosmological
consequences of considering a sign-changeable interaction term in HDE model.
We shall take $Q=3b^{2}Hq{\rho}$, where $\rho=\rho_{m}+\rho_{D}$ is the total
energy density. We will consider three cutoffs, including the future event
horizon, the Hubble and GO cutoffs, to build the energy density of HDE which
helps us in providing three models for DE. Thereinafter, we investigate the
evolution of the system parameters, such as the state parameter as well as the
deceleration and dimensionless density parameters, during the cosmos evolution
from the matter dominated era to the current accelerating epoch.

The paper is organized as follows. In section II, we use the future event
horizon as the IR cutoff to build the HDE, and study the evolution of the
system’s cosmological parameters. Sections III and IV also include the HDE
models with the Hubble and GO cutoffs, respectively. To have a better
understanding of the system evolution, we plot the cosmological parameters,
such as $q$, the equation of state and dimensionless density parameters, in
terms of the redshift $z$. We finish with summary and concluding remarks in
the last section.

##  II HDE with the sign-changeable interaction with future horizon as IR
cutoff

We consider a homogeneous and isotropic flat FRW universe which is described
by the line element

$\displaystyle
ds^{2}=-dt^{2}+a^{2}\left(t\right)\left[dr^{2}+r^{2}d\Omega^{2}\right],$ (1)

where $a(t)$ is scale factor. The first Friedmann equation is written as

$H^{2}=\frac{\rho}{3M_{p}^{2}},$ (2)

where $\rho$ is the total energy density, which satisfy the conservation law

$\displaystyle\dot{\rho}+3H(\rho+p)=0.$ (3)

We assume the two dark sectors of the universe, namely pressureless DM and HDE
exchange energy and thus they no longer satisfy the conservation equations,
separately. Instead, they satisfy the semi-conservation equations as

$\displaystyle\dot{\rho}_{m}+3H\rho_{m}=Q,$ (4)
$\displaystyle\dot{\rho}_{D}+3H(1+\omega_{D})\rho_{D}=-Q.$ (5)

where $\omega_{D}\equiv{p_{D}}/{\rho_{D}}$ is the equation of state parameter
of HDE, and $Q$ denotes the interaction term between DE and DM. As it is
obvious, DE (DM) decays into DM (DE) for $Q>0$ ($Q<0$). Following Wei2011 ;
WEI2011 , we assume the interaction term has the following form

$\displaystyle Q=3b^{2}Hq(\rho_{D}+\rho_{m}),$ (6)

where $b^{2}$ is the coupling constant and $q$ is the deceleration parameter
defined as

$\displaystyle q=-\frac{\ddot{a}a}{{\dot{a}}^{2}}=-1-\frac{\dot{H}}{H^{2}}.$
(7)

From Eq. (6), it is obvious that the transition of the universe from the
deceleration $(q>0)$ to an acceleration $(q<0)$ phase, changes the sign of the
interaction term $Q$. Defining, as usual, the dimensionless density parameters
as

$\Omega_{m}=\frac{\rho_{m}}{3M_{p}^{2}H^{2}},\ \ \ \ \
~{}~{}\Omega_{D}=\frac{\rho_{D}}{3M_{p}^{2}H^{2}},$ (8)

we can rewrite the first Friedmann equation in the form

$\Omega_{m}+\Omega_{D}=1.$ (9)

Also, if we take the time derivative of the ratio of the energy densities,
$r={\rho_{m}}/{\rho_{D}},$ we find

$\dot{r}={3b^{2}Hq+6b^{2}Hqr+3b^{2}Hqr^{2}+3Hr\omega_{D}},$ (10)

where we have used Eqs. (4) and (5). Solving for $w_{D}$, we arrive at

$\omega_{D}=\frac{\dot{r}-3b^{2}Hq-6b^{2}Hqr-3b^{2}Hqr^{2}}{3Hr}.$ (11)

Figure 1: Evolution of $\Omega_{D}$ versus redshift parameter $z$ for HDE with
Future cutoff for a fixed interaction parameter with coupling interaction
$b^{2}$ but for different values of the constant $c$.

Using Eqs. (8), (9), one obtains

$r=\frac{\Omega_{m}}{\Omega_{D}}=\frac{1-\Omega_{D}}{\Omega_{D}},$ (12)

which implies

$\dot{r}=-\frac{\dot{\Omega}_{D}}{\Omega_{D}^{2}}.$ (13)

Taking into account the fact that $\dot{\Omega}_{D}={\Omega}^{\prime}_{D}H$,
after using Eq. (13), we can rewrite Eq. (11) as

$\omega_{D}=\frac{-{{\Omega}^{\prime}_{D}}/{\Omega_{D}^{2}}-3b^{2}q-6b^{2}qr-3b^{2}qr^{2}}{3r},$
(14)

where the dot and the prime stand for the derivative with respect to the time
and $x=\ln{a}$, respectively. Considering the Hubble radius as the IR cutoff,
it is easy to show that $r$ is a constant Pavon2005 ; Shey2011 . Here, we use
the future event horizon as the IR cutoff and so $r$ is no longer a constant
Li2004 . The future event horizon is defined as Li2004 ; Huang2004

$R_{h}=a(t)\int_{t}^{\infty}{\frac{dt}{a(t)}}=\frac{c\sqrt{1+r}}{H},$ (15)

where $c$ is a constant, and it leads to

$\dot{R_{h}}={HR_{h}-1}={c\sqrt{1+r}-1}.$ (16)

Taking the time derivative of Eq. (2), after using Eqs. (4), (5) and (8), we
obtain

$\frac{\dot{H}}{H^{2}}=-\frac{3}{2}{\Omega_{D}}({1+r+\omega_{D}}).$ (17)

Moreover, using (16) and (17), we obtain

$\omega_{D}=-\frac{1}{3}-\frac{2\sqrt{\Omega_{D}}}{3c}-\frac{b^{2}q}{\Omega_{D}},$
(18)

Combining Eqs. (16) and (17) we get,

$\frac{{\Omega}^{\prime}_{D}}{\Omega_{D}^{2}}=(1-\Omega_{D})\left(\frac{1}{\Omega_{D}}+\frac{2}{c\sqrt{\Omega_{D}}}-\frac{3b^{2}q}{{\Omega}_{D}(1-\Omega_{D})}\right).$
(19)

Substituting Eq. (17) into (7), one finds

$q=\frac{1-\Omega_{D}-\frac{2\Omega_{D}^{\frac{3}{2}}}{c}}{2+3b^{2}}.$ (20)

Combining this equation with Eq. (18), we find out that, for the non-
interacting case ($b^{2}=0$), we have $\omega_{D}=-1$ in late time where
$\Omega_{D}\rightarrow 1$ provided we take $c=1$. In addition, it is easy to
check that, for $b^{2}=0$ and $c=1$, we have $q=-1$ provided
$\Omega_{D}\rightarrow 1$.

Figure 2: The evolution of $\omega_{D}$ versus redshift parameter $z$ for HDE
with Future cutoff. Left panel corresponds to a fixed couplings between DE and
DM and the right panel shows the case in which $c$ is constant.

The evolution of $\Omega_{D}$ in terms of redshift parameter ($1+z$) with
respect to the constant $b^{2}$ is plotted in Fig. 1. From this figure we see
that, in the early universe where $z\rightarrow\infty$ we have
$\Omega_{D}\rightarrow 0$, while at the late time where $1+z\rightarrow 0$ we
have $\Omega_{D}\rightarrow 1$. It is also apparent that, at fixed $b^{2}$,
the DE role in the early universe will be highlighted with increasing the
value of $c$. Additionally, in the long run limit ($1+z\rightarrow 0$), DE
plays a more effective role with decreasing the value of $c$.

Figure 3: The evolution of the deceleration parameter $q$ against redshift
parameter $z$ for HDE with Future cutoff. Left panel corresponds to a fixed
couplings between DE and DM and the right panel shows the case $c$ is
constant.

The equation of state (EoS) parameter given in (18) is also plotted in Fig. 2
showing that, for a fixed $b^{2}$, the EoS parameter can cross the phantom
line for $c<1$, and when $c>1$, we always have $\omega_{D}>-1$.

The behaviour of the deceleration parameter $q$ is plotted in Fig. 3 showing
that there is a deceleration expansion at the early time followed by an
acceleration expansion. In addition, the transition from the deceleration
phase to the acceleration one happen at $z\approx 0.6$ which is consistent
with recent observations Daly ; Kom1 ; Kom2 . As Fig. 3 shows, for fixed
$b^{2}$, the universe expansion may experience a phantom phase at current time
while $c>1$. Moreover, for two universes with the same $b^{2}$ and different
values of $c$, the universe with larger amount of $c$ gets its phase
transition point earlier than the other universe. Finally, for fixed $c$ and
for the current stage of the universe, this model predicts that the universe
expansion phase may fall in the phantom era as a function of $b^{2}$.

Figure 4: The evolution of $\Omega_{D}$ versus the $1+z$ parameter for HDE
with GO cutoff and $b^{2}=0\cdot 01$. Left panel corresponds to $\alpha<1$ and
the right panel shows the $\alpha>1$ case.

##  III HDE with the sign-changeable interaction with Hubble cutoff

In this section, we use the Hubble radius ($L=H^{-1}$) as IR cutoff to study
the effects of a sign-changeable interaction in the HDE model. In this case we
have $\rho_{D}=3c^{2}M_{p}^{2}H^{2}$, where $c$ is a constant. Finally,
bearing the $\Omega_{D}={\rho_{D}}/{(3M_{p}^{2}H^{2})}$ relation in mind, we
find out $\Omega_{D}=c^{2}$ meaning that $c^{2}$ should meet the $0\leq
c^{2}\leq 1$ condition Ghaffari2015 . It is worth mentioning that, in the non-
interacting model we arrive at $\omega_{D}=0$ for the HDE with Hubble radius
as the IR cutoff Li2004 . This implies that in this case the accelerated
expansion of the universe cannot be achieved Li2004 . It is also a matter of
calculation to combine the $\rho_{D}=3c^{2}M_{p}^{2}H^{2}$ expression with
Eqs. (5) and (17), and using the $Q=3b^{2}Hq\rho=3b^{2}Hq(\rho_{D}+\rho_{m})$
relation for the mutual interaction to get

$\omega_{D}=-\frac{b^{2}q}{c^{2}(1-c^{2})},$ (21)

where

$q=-1-\frac{\dot{H}}{H^{2}}=\frac{1-c^{2}}{2(1-c^{2})+3b^{2}}.$ (22)

It is obvious that in the absence of the interaction term, where $b^{2}=0$, we
get $w_{D}=0$ and $q=1/2>0$ which refer to a decelerated universe. Now,
inserting Eq. (22) into Eq. (21), we obtain

$\omega_{D}=-\frac{b^{2}}{c^{2}[2(1-c^{2})+3b^{2}]}.$ (23)

Clearly, for $2(1-c^{2})+3b^{2}>0$, we have $\omega_{D}<0$.

Figure 5: The evolution of $\omega_{D}$ versus the $1+z$ parameter for HDE
with GO cutoff and $b^{2}=0\cdot 01$. Left panel corresponds to $\alpha<1$ and
the right panel shows the $\alpha>1$ case.

##  IV HDE with the sign-changeable interaction with GO cutoff

The third cutoff, introduced by Granda and Oliveros (GO) Granda2008 , is the
generalization of the Ricci scalar Gao  in which we have
$L=(\beta\dot{H}+\alpha H^{2})^{-1/2}$ where $\alpha$ and $\beta$ are
constant, and therefore

$\rho_{D}=3M_{p}^{2}(\alpha H^{2}+\beta\dot{H}),$ (24)

which can also be rewritten

$\frac{\dot{H}}{H^{2}}=\frac{\Omega_{D}-\alpha}{\beta}.$ (25)

Taking the time derivative of Eq. (2) and using Eqs. (4), (5), (8) and (9), we
reach

$\frac{\dot{\rho}_{D}}{3M_{p}^{2}H^{3}}=\frac{2\dot{H}}{H^{2}}+3(1-\Omega_{D})-3b^{2}q.$
(26)

From Eq. (8), we obtain

$\dot{\Omega}_{D}=\frac{\dot{\rho}_{D}}{3M_{p}^{2}H^{2}}-2\Omega_{D}\frac{\dot{H}}{H}.$
(27)

Combining Eqs. (25) and (26) with (27) to get

${\Omega}^{\prime}_{D}=(1-\Omega_{D})\left[3+\frac{2}{\beta}(\Omega_{D}-\alpha)\right]-3b^{2}q.$
(28)

Inserting Eqs. (24), (25) and (26) into (5), and combining Eqs. (25) and (7),
we find the EoS and deceleration parameter as

$\omega_{D}=-\frac{1}{\Omega_{D}}\left[1+\frac{2}{3}\left(\frac{\Omega_{D}-\alpha}{\beta}\right)\right],$
(29) $q=-1-\frac{\dot{H}}{H^{2}}=-1-\frac{\Omega_{D}-\alpha}{\beta}.$ (30)

The evolution of HDE density parameter ($\Omega_{D}$) in terms of redshift
parameter ($1+z$), whenever $b^{2}=0\cdot 01$, is shown in Fig. 4. As it is
obvious from Fig. 4, we have $\Omega_{D}\rightarrow 0$ and
$\Omega_{D}\rightarrow 1$ in the early universe ($1+z\rightarrow\infty$) and
the $1+z\rightarrow 0$ limit, respectively.

Since Eq. (29) indicates that we have $\omega_{D}=-1$ while
$\Omega_{D}\rightarrow 1$ and $\alpha=1$ simultaneously, we plot the
$\alpha<1$ and $\alpha>1$ cases separately in Fig. 5. As it is obvious from
this figure, the model can show a phantom-like behavior ($\omega_{D}<-1$) for
$\alpha<1$. Additionally, for the $\alpha>1$ case, $\omega_{D}$ is always
larger than $-1$.

Figure 6: The evolution of $q$ versus the $1+z$ parameter for HDE with GO
cutoff and $b^{2}=0\cdot 01$. Left panel corresponds to $\alpha<1$ and the
right panel shows the case $\alpha>1$.

The behavior of $q$ is also shown in Fig. 6. It is obvious that the universe
experiences a transition from a deceleration phase to an acceleration one
around $z\approx 0\cdot 6$, compatible with observation Daly . It is also
useful to note that Eq. (30) shows $q\rightarrow-1$ while
$\Omega_{D}\rightarrow 1$ and $\alpha=1$.

##  V Summary and concluding remarks

Throughout this paper, we considered a universe filled by dark matter and dark
energy, in which there is a sign-changeable interaction
[$Q=3b^{2}Hq(\rho_{D}+\rho_{m})$] between the dark sectors. Considering the
future event horizon, the Hubble radius and GO as the system IR cutoffs, we
followed the HDE idea to obtain three models for dark energy.

At first, we used the future event horizon as the IR cutoff to construct the
HDE density profile. In addition, we studied the evolution of density
parameter, the deceleration parameter and the EOS parameter. Our study shows
that the model is compatible with some recent observations, and it predicts a
transition from the deceleration phase to an acceleration one at around
$z\approx 0\cdot 6$. Our check indicates that, for fixed $b^{2}$, we have
$\omega_{D}>-1$ for $c>1$. We also saw that, for fixed $b^{2}$, the phantom-
like behavior ($\omega_{D}<-1$) may be obtained for $c<1$.

Moreover, we focused on the Hubble and the GO cutoffs and found out that these
cutoffs may also lead to HDE models compatible with observations. In fact, we
saw that, unlike the non-interacting HDE model with the Hubble cutoff Li2004 ,
it is possible to have $\omega_{D}<0$ and $q<0$ in the interacting case. In
addition, our study shows that the model with the GO cutoff can predict the
universe transition from a deceleration phase to an acceleration one at around
$z\approx 0\cdot 6$. We finally found out that the EOS of the model with GO
cutoff may also cross the phantom line.

###### Acknowledgements.

We also thank Shiraz University Research Council. This work has been supported
financially by Research Institute for Astronomy & Astrophysics of Maragha
(RIAAM), Iran.

## References

  * (1) A. G. Riess et al.,  Observational evidence from supernovae for an accelerating Universe and a cosmological constant, Astron. J. 1161009 (1998) [arXiv:astro-ph/9805201].
  * (2) S. Perlmutter et al.,  Measurements of omega and lambda from 42 high-redshift supernovae, Astrophys. J. 517 565 (1999) [arXiv:astro-ph/9812133].
  * (3) P. deBernardis, et al., A flat Universe from high-resolution maps of the cosmic microwave background radiation, Nature 404 955 (2000) [arXiv:astro-ph/0004404v1].
  * (4) S. Perlmutter,et al., New Constraints on ΩM, ΩΛ, and w from an Independent Set of 11 High-Redshift Supernovae Observed with the Hubble Space Telescope,Astrophys. J. 598102 (2003) [arXiv:astro-ph/0309368v1].
  * (5) M. Colless et al., The 2dF Galaxy Redshift Survey: Spectra and redshifts, Mon. Not. R. Astron. Soc. 328, 1039 (2001)[arXiv:astro-ph/0106498v3].
  * (6) M. Tegmark etal., Cosmological parameters from SDSS and WMAP, Phys. Rev. D 69, 103501 (2004)[ arXiv:astro-ph/0310723v2].
  * (7) S. Cole et al., The 2dF Galaxy Redshift Survey:Power-spectrum analysis of the final dataset and cosmological implications, Mon. Not. R. Astron. Soc. 362, 505 (2005)[arXiv:astro-ph/0501174v2].
  * (8) V. Springel, C. S. Frenk, and S. M. D. White, The large-scale structure of the Universe, Nature(London) 440, 1137 (2006) [arXiv:astro-ph/0604561].
  * (9) S. Hanany et al., MAXIMA-1:A Measurement of the Cosmic Microwave Background Anisotropy on Angular Scales of 10’-5°, Astrophys. J. Lett. 545, L5 (2000)[arXiv:astro-ph/0005123v3].
  * (10) C. B. Netterfield et al., A Measurement by BOOMERANG of Multiple Peaks in theAngular Power Spectrum of the Cosmic Microwave Background, Astrophys. J. 571, 604 (2002)[arXiv:astro-ph/0104460v3].
  * (11) D. N. Spergel et al., First Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Determination of Cosmological Parameters, Astrophys. J. Suppl. 148, 175 (2003)[ arXiv:astro-ph/0302209v3].
  * (12) M. Roos, Introduction to Cosmology (John Wiley and Sons, UK, 2003)
  * (13) C. Wetterich, Cosmology And The Fate Of Dilatation Symmetry, Nucl. Phys. B302, 668 (1988).
  * (14) B. Ratra and J. Peebles, Cosmological Consequences Of A Rolling Homogeneous Scalar Field, Phys. Rev. D 37, 3406 (1988).
  * (15) R. R. Caldwell,  A Phantom Menace?, Phys. Lett. B 545, 23 (2002) [astro-ph/9908168].
  * (16) R. R. Caldwell, M. Kamionkowski and N. N. Weinberg,  Phantom Energy and Cosmic Doomsday,Phys. Rev. Lett.91, 071301 (2003) [astro-ph/0302506].
  * (17) T. Chiba, T. Okabe, and M. Yamaguchi, Kinetically driven quintessence, Phys. Rev. D 62, 023511 (2000)[arXiv:astro-ph/9912463].
  * (18) C. Armendariz-Picon, V. F. Mukhanov and P. J. Steinhardt, A dynamical solution to the problem of a small cosmological constant and late-time cosmic acceleration, Phys. Rev. Lett. 85, 4438 (2000)[arXiv:astro-ph/0004134].
  * (19) C. Armendariz-Picon, V. F. Mukhanov and P. J. Steinhardt,  Essentials of k-essence, Phys. Rev.D63, 103510 (2001) [arXiv:astro-ph/0006373].
  * (20) A. Y. Kamenshchik, U. Moschella and V. Pasquier, An alternative to quintessence, Phys. Lett. B511, 265 (2001) [arXiv:gr-qc/0103004].
  * (21) M. C. Bento, O. Bertolami, and A. A. Sen, Generalized Chaplygin Gas, Accelerated Expansion and Dark Energy-Matter Unification, Phys. Rev. D 66, 043507 (2002)[arXiv:gr-qc/0202064].
  * (22) R. G. Cai,A Dark Energy Model Characterized by the Age of the Universe, Phys. Lett. B 657, 228 (2007)[arXiv:0707.4049 [hep-th]].
  * (23) H. Wei and R. G. Cai,A new model of agegraphic dark energy ,Phys. Lett. B 660, 113 (2008)[arXiv:0708.0884 [astro-ph]].
  * (24) K. Y. Kim, H.W. Lee, and Y. S. Myung,Instability of agegraphic dark energy models, Phys. Lett. B 660, 118 (2008)[arXiv:0709.2743[gr-qc]].
  * (25) H. Wei and R. G. Cai,Cosmological constraints on new agegraphic dark energy, Phys. Lett. B663, 1 (2008)[arXiv:0708.1894 [astro-ph]].
  * (26) J. P. Wu, D. Z. Ma, and Y. Ling,Quintessence reconstruction of the new agegraphic dark energy model, Phys. Lett. B663, 152 (2008)[arXiv:0805.0546 [hep-th]].
  * (27) J. Zhang, X. Zhang, and H. Liu,Agegraphic dark energy as a quintessence, Eur. Phys. J. C 54, 303 (2008)[arXiv:0801.2809 [astro-ph]].
  * (28) H. Wei and R. G. Cai, Interacting agegraphic dark energy, Eur. Phys. J. C 59, 99 (2009)[arXiv:0707.4052 [hep-th]];
H. Wei and R. G. Cai, Statefinder Diagnostic and w-w’ Analysis for the
Agegraphic Dark Energy Models without and with Interaction, Phys. Lett. B 655,
1 (2007), [arXiv:0707.4526];
H. Wei and R. G. Cai, Interacting Agegraphic Dark Energy, Eur. Phys. J. C 59,
99 (2009), [arXiv:0707.4052]

  * (29) I. P. Neupane,A note on agegraphic dark energy, Phys. Lett. B673, 111 (2009)[arXiv:0708.2910 [hep-th]].
  * (30) A. Sheykhi, Interacting agegraphic dark energy models in non-flat universe, Phys. Lett. B 680, 113 (2009)[arXiv:0907.5144 [hep-th]].
  * (31) A. Sheykhi Interacting agegraphic tachyon model of dark energy, 682, 329 (2010) [arXiv:0907.2491 [hep-th]].
  * (32) A. Sheykhi Interacting new agegraphic dark energy in non-flat Brans-Dicke cosmology, Phys. Rev. D 81, 023525 (2010) [arXiv:0908.0606 [gr-qc]].
  * (33) A. Sheykhi and M. Jamil,Interacting HDE and NADE in Brans-Dicke Chameleon Cosmology, Phys. Lett. B694, 284 (2011) [arXiv:1010.0385 [hep-th]].
  * (34) A. Sheykhi and M. R. Setare,Interacting new agegraphic viscous dark energy with varying G, Int. J. Theor. Phys.49, 2777 (2010)[arXiv:1003.1109 [physics.gen-ph]].
  * (35) N. Ohta, Dark energy and QCD ghost, Phys. Lett. B 695, 41 (2011), [arXiv:1010.1339].
  * (36) R. G. Cai, Q. Su, Z. L. Tuo, H. B. Zhang, Notes on Ghost Dark Energy, Phys. Rev. D 84,123501 (2011), [arXiv:1011.3212].
  * (37) A. Sheykhi, M. Sadegh Movahed, Interacting ghost dark energy in non-flat universe Gen. Relativ. Gravit. 44,449 (2012), [arXiv:1104.4713].
  * (38) E. Ebrahimi, A. Sheykhi, Instability of QCD ghost dark energy model, Int. J. Mod. Phys. D 20, 2369 (2011), [arXiv:1106.3504].
  * (39) A. Sheykhi, M. Sadegh Movahed, E. Ebrahimi, Tachyon Reconstruction of Ghost Dark Energy, Astrophys. Space Sci. 339, 93 (2012), [arXiv:1105.0325].
  * (40) A. Sheykhi, M. Tavayef, Ghost Dark Energy in a Cyclic Universe, Iran. J. Sci. Technol. Trans. Sci. DOI: 10.1007/s40995-016-0083-y.
  * (41) S. Nojiri and S. D. Odintsov, The new form of the equation of state for dark energy fluid and accelerating universe, Phys. Lett. B 639, 144 (2006), [arXiv:hep-th/0606025].
  * (42) K. Bamba, S. Capozziello, S. Nojiri and S. D. Odintsov, Dark energy cosmology: the equivalent description via different theoretical models and cosmography tests, Astrophys. Space Sci. 342, 155 (2012), [arXiv:1205.3421].
  * (43) M. Li. X. D. Li, S. Wang and and Y. Wang, THE UNIVERSE. 1, 4 (2013).
  * (44) E. Witten, The cosmological constant from the viewpoint of string theory,[ arXiv:hep-ph/0002297].
  * (45) A. G. Cohen, D. B. Kaplan and A. E. Nelson, Effective field theory, black holes, and the cosmological constant, Phys. Rev. Lett. 82, 4971 (1999) [arXiv:hep-th/9803132]
  * (46) M. Li, A model of holographic dark energy, Phys. Lett. B 603, 1 (2004) [arXiv:hep-th/0403127].
  * (47) S. D. H. Hsu, Entropy bounds and dark energy, Phys. Lett. B 594, 13 (2004) [arXiv:hep-th/0403052]
  * (48) D. Pavon, W. Zimdahl,Holographic dark energy and cosmic coincidence, Phys. Lett. B 628, 206 (2005) [arXiv:gr-qc/0505020].
  * (49) A. Sheykhi,Thermodynamics of interacting holographic dark energy with the apparent horizon as an IR cutoff, Classical Quantum Gravity 27, 025007 (2010). [arXiv:0910.0510 [hep-th]]
  * (50) A. Sheykhi,Holographic scalar field models of dark energy, Phys. Rev. D 84, 107302 (2011).arXiv:1106.5697 [arXiv:physics.gen-ph]
  * (51) H. Wei, Pilgrim Dark Energy, Class. Quant. Grav. 29, 175008 (2012) [arXiv:1204.4032];
H. Wei, Modified Holographic Dark Energy, Nucl. Phys. B 819, 210 (2009)
[arXiv:0902.2030].

  * (52) L. N. Granda, A. Oliveros,Infrared cut-off proposal for the holographic density, Phys. Lett. B 669 (2008) [ arXiv:0810.3149v1 [gr-qc]]
  * (53) L. N. Granda, A. Oliveros,New infrared cut-off for the holographic scalar fields models of dark energy, Phys. Lett. B 671275 (2009) 199 [arXiv:0810.3663v2 [gr-qc]]
  * (54) M. Jamil, K. karami, A. Sheykhi, E. Kazemi, Z. Azarmi, Holographic dark energy in Brans-Dicke cosmology with Granda-Oliveros cut-off, Int. J. Theor. Phys. 51, 604 (2012).
  * (55) S. Ghaffari, M. H. Dehghani and A. Sheykhi, Holographic dark energy in the DGP braneworld with GO cutoff, Phys. Rev. D 89(2014) 123009 [arXiv:1506.01676];
A. Khodam-Mohammadi, E. Karimkhani, A. Sheykhi, Best values of parameters for
interacting HDE with GO IR-cutoff in Brans-Dicke cosmology, Int. J. Mod. Phys.
D, 23, 14500813 (2014) [arXiv:1409.3115]

  * (56) B. Wang, E. Abdalla, F. Atrio-Barandela, D. Pavon, Dark Matter and Dark Energy Interactions: Theoretical Challenges, Cosmological Implications and Observational Signatures[arXiv:1603.08299 [gr-qc]]
  * (57) R. G. Cai and Q. P. Su,On the Dark Sector Interactions, Phys. Rev. D 81, 103514 (2010) [arXiv:0912.1943v2 [astro-ph.CO]].
  * (58) H. Wei, Cosmological Evolution of Quintessence and Phantom with a New Type of Interaction in Dark Sector, Nucl. Phys. B 845,381 (2011) [arXiv:1008.4968v3]
  * (59) H. Wei, Cosmological Constraints on the Sign-Changeable Interactions, Commun. Theor. Phys.56, 972 (2011) [arXiv:1010.1074 [gr-qc]]
  * (60) L. P. Chimento, Linear and nonlinear interactions in the dark sector, Phys. Rev. D 81, 043525 (2010) [arXiv:0911.5687]
  * (61) L. P. Chimento, M. Forte and G. M. Kremer, Cosmological model with interactions in the dark sector, Gen. Rel. Grav. 41, 1125 (2009) [arXiv:0711.2646].
  * (62) Q.G. Huang, M. Li, The holographic dark energy in a non-flat universe ,JCAP 0408 013 (2004) .[arXiv:astro-ph/0404229].
  * (63) R.A. Daly et al., Improved Constraints on the Acceleration History of the Universe and the Properties of the Dark Energy, Astrophys. J. 677, 1 (2008).
  * (64) E. Komatsu et al. [WMAP Collaboration], SEVEN-YEAR WILKINSON MICROWAVE ANISOTROPY PROBE (WMAP *) OBSERVATIONS: COSMOLOGICAL INTERPRETATION, Astrophys. J. Suppl. 192, 18 (2011).
  * (65) V. Salvatelli, A. Marchini, L. L. Honorez and O. Mena, New constraints on coupled dark energy from the Planck satellite experiment , Phys. Rev. D 88, 023531 (2013).
  * (66) S. Ghaffari, A. Sheykhi and M. H. Dehghani, Statefinder diagnosis for holographic dark energy in the DGP braneworld, Phys. Rev. D 912, 023007 (2015) [arXiv:1504.04008 [gr-qc]].
  * (67) C. J. Gao, X. L. Chen and Y. G. Shen, A Holographic Dark Energy Model from Ricci Scalar Curvature ,Phys. Rev. D 79, 043511 (2009)[ arXiv:0712.1394v4 [astro-ph]].


# FMSG-JLESS Submission for DCASE 2024 Task4 on Sound Event Detection with
Heterogeneous Training Dataset and Potentially Missing Labels

###### Abstract

This report presents the systems developed and submitted by Fortemedia
Singapore (FMSG) and Joint Laboratory of Environmental Sound Sensing (JLESS)
for DCASE 2024 Task 4. The task focuses on recognizing event classes and their
time boundaries, given that multiple events can be present and may overlap in
an audio recording. The novelty this year is a dataset with two sources,
making it challenging to achieve good performance without knowing the source
of the audio clips during evaluation. To address this, we propose a sound
event detection method using domain generalization. Our approach integrates
features from bidirectional encoder representations from audio transformers
and a convolutional recurrent neural network. We focus on three main
strategies to improve our method. First, we apply mixstyle to the frequency
dimension to adapt the mel-spectrograms from different domains. Second, we
consider training loss of our model specific to each datasets for their
corresponding classes. This independent learning framework helps the model
extract domain-specific features effectively. Lastly, we use the sound event
bounding boxes method for post-processing. Our proposed method shows superior
macro-average pAUC and polyphonic SED score performance on the DCASE 2024
Challenge Task 4 validation dataset and public evaluation dataset.

Index Terms—  sound event detection, semi-supervised learning, domain
generalization, mixstyle

##  1 Introduction

Sound event detection (SED) [1, 2, 3, 4, 5] involves identifying sound events
from acoustic signals and accurately classifying them into specific categories
with timestamps, considering various acoustic environments. DCASE 2024 Task 4
[6], entitled “Sound Event Detection with Heterogeneous Training Dataset and
Potentially Missing Labels,” focuses on SED. This task follows up on DCASE
2023 Task 4A [7, 8] and Task 4B [9] on the following aspects:

  * •

DCASE 2023 Task 4A evaluated systems for detecting sound events using weakly
labeled data (without timestamps) and unlabeled data in the DESED dataset
[10]. The goal was to provide both event class and event time localization,
despite multiple overlapping events in an audio recording.

  * •

DCASE 2023 Task 4B evaluated systems for detecting sound events with soft
labeled data from the MAESTRO dataset [11]. This task focused on exploring the
significance of using soft labels for SED.

DCASE 2024 Task 4 aims to unify the setups of both the tasks of 2023 edition.
Specifically, instead of training an SED model on each subtask separately with
its dataset, an intriguing approach is to just train a single model on all
available datasets. The goal is still to provide event classes along with
their time boundaries, even with multiple overlapping events. This task
explores leveraging training data with varying annotation granularity
(temporal resolution, soft/hard labels). Systems will be evaluated on labels
with different granularity to understand their behavior and robustness for
various applications. Target classes in different datasets also differ, so
sound labels present in one dataset might not be annotated in another. The
systems need to handle potentially missing target labels during training and
perform without knowing the origin of the audio clips at evaluation time.

Although previous years’ challenges, like the frequency dynamic convolutional
recurrent neural network (FDY-CRNN) [12, 13, 14], have shown notable
performance in DCASE Task 4, this year’s Task 4 introduces new challenges. The
main challenge is how to combine heterogeneous training datasets from diverse
domains with different annotations to improve performance. Deep neural
networks struggle to generalize across diverse domains, leading to poor
results in real-world scenarios. Therefore, domain generalization (DG) [15]
has become an essential research topic in fields like computer vision, audio
processing, and natural language processing. Inspired by the exploration in
DCASE Task 1 [16], which dealt with audio clips from multiple devices, we
propose using the domain generalization approach for this year’s Task 4.

In this technical report, we outline our contributions to our submission for
DCASE 2024 Task 4. The primary contributions of our submissions are as
follows:

  * •

We utilize the frame-level embeddings generated by the pretrained BEATs model
in late-fusion with the FDY-CRNN and then fed into the recurrent neural
network with the classifier.

  * •

We leverage the DG to explore the appropriate way to use the heterogeneous
training datasets from diverse domains.

  * •

We modify the baseline framework to independently compute the training loss
for our model, which is specific to each dataset for their corresponding
classes.

  * •

We employ the sound event bounding boxes method as a post-processing method to
further enhance the performance in the DESED dataset.

##  2 Dataset

The DCASE 2024 Challenge Task 4 comprises two datasets, and participants must
use both in the training phase and provide one individual model that performs
well for the two datasets.

###  2.1 DESED dataset

DESED [10] consists of 10-second audio clips either recorded in a domestic
environment or synthesized to reproduce such an environment. It features
annotated sound events from 10 classes: alarm bell ringing, blender, cat,
dishes, dog, electric shaver/toothbrush, frying, running water, speech, and
vacuum cleaner. The synthetic part of the dataset is generated with Scaper
[17] using foreground events from the Freesound datasets and backgrounds from
YouTube videos and the Freesound subset of the MUSAN [18] dataset. The real-
world recording part comes from AudioSet [19] and includes a weakly annotated
set (1,578 clips), an unlabeled set (14,412 clips), and a strongly annotated
portion (3,470 clips).

###  2.2 MAESTRO Real dataset

MAESTRO Real [11, 20], used in the DCASE 2023 Task 4B challenge, consists of a
development set (6,426 clips) and an evaluation set of long-form real-world
recordings. This dataset contains multiple temporally strong annotated events
with soft labels from 17 classes. However, in this challenge, only 11 classes
are evaluated, as the other 6 do not have confidence levels over 0.5. These
classes are: birds_singing, car, people_talking, footsteps, children_voices,
wind_blowing, brakes_squeaking, large_vehicle, cutlery_and_dishes,
metro_approaching, and metro_leaving. This data was annotated using
crowdsourcing, where temporally-weak labeling is combined with a sliding
window approach to determine events’ temporal localization. In order to obtain
the soft labels, annotations of multiple annotators are aggregated via MACE
[21]. The recordings are taken from the TUT Acoustic Scenes 2016 dataset [22]
and are between 3 to 5 minutes long.

##  3 Proposed Approach

###  3.1 Baseline

The baseline system is inherited from previous DCASE Task 4 challenges [7, 23]
and consists of a CRNN [24] that uses self-supervised features from the pre-
trained BEATs [25] model. First, the CRNN has a convolutional neural network
(CNN) encoder with 7 convolutional layers, batch normalization, gated linear
units, and dropout, followed by a bi-directional gated recurrent unit (biGRU)
layer. Then, BEATs features are concatenated with the CNN-extracted features
before the biGRU layer. Average pooling is applied to the BEATs features to
match the sequence length of the CNN encoder. Finally, Attention pooling is
used to derive clip-wise and frame-wise posteriors. During training, the BEATs
model remains frozen, and the mean-teacher framework [26, 27] is used to
leverage unlabeled and weakly labeled data. The attention pooling mechanism in
the baseline model uses the softmax function over classes. Before applying
softmax, values for unlabeled classes (not in the current clip dataset) are
masked to negative infinity.

As a preprocessing step, some DESED events are mapped to similar classes in
MAESTRO. For example, in DESED, “speech” is a super-class for “people
talking,” “children’s voices,” and “announcements” in MAESTRO. “Dishes” in
DESED corresponds to “cutlery and dishes” in MAESTRO, and “dog” in DESED is a
super-class for “dog bark”. This mapping ensures that when computing the loss
for MAESTRO, the network output for similar classes in DESED is adjusted
accordingly.

Figure 1: Architecture of (a) CRNN (Baseline) (b) FDY-CRNN. when hop length is
256.

###  3.2 Domain generalization

In DCASE 2024 Challenge Task 4, we must train one model on heterogeneous
datasets. As mentioned in Section 2, the DESED and MAESTRO Real datasets come
from different sources. Even if they share the same segment labels, their
features can be quite different, confusing model training because domain
information is not useful. domain generalization (DG) [15] aims to address
this problem by learning robust models against data distribution changes
across domains, known as domain shift. The goal is to ensure that the trained
model can generalize well to any domain by learning domain-invariant feature
representations that remain discriminative across multiple domains.

MixStyle [28] is a common DG method motivated by the observation that the
visual domain is closely related to image style. Specifically, MixStyle mixes
the feature statistics of two instances with a random convex weight to
simulate new styles. However, unlike images where 2D convolution operates
along spatial dimensions, in case of audio, 2D convolution operates on
frequency and temporal information. Therefore, domain information may not be
mainly distributed in channel statistics in audio as it is in MixStyle. In
[16], the authors analyzed the relationship between the domain and the
statistics of each feature dimension showing that the frequency feature
dimension carries more domain-relevant information than the channel dimension.
This inspired us to adopt frequency-wise MixStyle for applying it to both
internal CNN features and the mel-spectrogram before feeding data into the
CNN. After integrating frequency-dynamic attention and MixStyle, we propose a
new model, which is introduced in the next section. We note that MixStyle is
not applied during testing.

We also explored several other DG methods including the residual normalization
[29], and adapted them to the audio task. However, the experiments demonstrate
that the freqwise mixstyle has straightforward improvement for heterogeneous
training datasets.

###  3.3 Network

In this work, we also employed FDY-CRNN from [7], which uses frequency-
adaptive kernels to enforce frequency dependency in 2D convolutions. In the
baseline CRNN architecture shown in Figure 1(a), we replaced the standard 2D
convolutional blocks with FDY-convolutional blocks, as illustrated in Figure
1(b). The CNN part consists of 7 blocks with the same number of filters as in
the baseline. In the FDY-convolutional block, batch normalization and gated
linear units are used.

###  3.4 Independent loss function

The model is trained using the binary cross-entropy (BCE) loss function on
DESED real-world strongly, synthetic, and weakly labeled data, as well as on
MAESTRO soft-labeled data. Mean squared error (MSE) is used for the mean-
teacher [26, 27] pseudo-labeling loss component, which is applied to both weak
and unlabeled data from DESED. When computing the loss for both components on
a particular clip, we avoid computing the loss for the network outputs
corresponding to classes that do not belong to the clip’s original dataset.
For example, in case of MAESTRO, we do not compute the loss for DESED output
logits even for classes that have been cross-mapped as explained in the
baseline. This is the main difference between our system and the baseline in
loss computation.

###  3.5 Pretrained model

We utilize the pretrained BEATs model, which has achieved state-of-the-art
performance on AudioSet with a mean average precision (mAP) of 0.486. The
BEATs is an iterative self-supervised framework for audio representation
learning, using an acoustic tokenizer and a semi-supervised learning model.
Unlike previous models, BEATs employs a self-distilled tokenizer to convert
audio signals into discrete labels. We use it to construct frame-level
embeddings of size 768, aligning with the recently released baseline approach.

###  3.6 Data augmentation

For this year’s challenge, we used two data augmentation methods. We applied
SpecAugment-style time-wise masking [30] to the features extracted by the pre-
trained model and independently to the features extracted from the CNN
encoder. This strategy, referred to as “dropstep”, helps improve model
robustness by adding variety to the training data.

Additionally, we used the Mixup [31] strategy. This helps in linear
interpolation and improves model robustness. Mixup is applied independently on
the MAESTRO and DESED datasets.

###  3.7 Curated set

To solve the mismatch between the synthetic valid dataset and the real-world
test set, we further split the 3,470 clips of the strongly annotated AudioSet
part to get the extra real valid dataset (373 clips).

###  3.8 Sound event box-based post-processing

Existing systems [32] commonly predict sound presence confidence in short time
frames. Then, thresholding produces binary frame-level presence decisions,
with the extent of individual events determined by merging consecutive
positive frames. In the previous challenge [7], we used median filtering as
post-processing.

A recent study in [33] shows that frame-level thresholding degrades the
prediction of event extent by coupling it with the system’s sound presence
confidence. Inspired by bounding box predictions in image object detection
[34] SEBBs are one-dimensional bounding boxes defined by event onset time,
event offset time, sound class, and confidence. They represent sound event
candidates with a scalar confidence score. The final SED is derived by class-
wise event-level thresholding of the SEBBs’ confidences. SEBBs whose
confidence exceeds the threshold are accepted as detections, while the rest
are discarded. The threshold controls the sensitivity of systems. For high
sensitivity/recall (few missed hits), a low detection threshold detects events
even when the system’s confidence is low. For high precision (few false
alarms), a higher threshold detects only events with high confidence. With
SEBBs, the sensitivity of a system can be controlled without impacting the
detection of an event’s onset and offset times, which was a problem with the
previous frame-level thresholding approach.

We first tune the hyperparameters for the change-point-based predictor of
Sound Event Bounding Boxes (cSEBBs) [33] based on the strong validation
dataset, then we use cSEBBs as the post-processing method in our system.

##  4 Experimental setup

###  4.1 Feature extraction

All audio clips are resampled to a 16 kHz mono channel using Librosa. They are
segmented with a window size of 2048 samples and a hop length of either 160 or
256 samples. A short-time Fourier transform is applied to extract
spectrograms. Mel-filters are then used to create log-mel spectrograms
spanning from 0 to 8 kHz. Clips shorter than 10 seconds are padded with
silence if needed.

Table 1: Performance in PSDS and mPAUC of different single-systems on the
DESED development set (D-PSDS), DESED public evaluation set (PE-PSDS), and
MAESTRO evaluation set (mPAUC) including frequency-dynamic convolution (FDY),
domain generalization (DG), and strong validation dataset (Strong Val).
‘AdaResNorm’ and ‘ResNorm’ stands for adaptive residual normalization and
residual normalization, respectively. Based on the joint score, the sum of PE-
PSDS and mPAUC, the systems with ‘*’ are chosen for final submission.

System | DG | Hop_length | FDY | Strong Val | D-PSDS | PE-PSDS (raw) | PE-PSDS (cSEBBs) | mPAUC | Joint score
---|---|---|---|---|---|---|---|---|---
Baseline | - | 256 | - | - | 0.483 | 0.529 | - | 0.721 | 1.250
S-1* | MixStyle | 256 | - | - | 0.506 | 0.587 | 0.629 | 0.737 | 1.366
S-2 | MixStyle | 256 | ✓ | - | 0.510 | 0.590 | 0.600 | 0.753 | 1.353
S-3* | MixStyle inside FDY | 256 | ✓ | - | 0.503 | 0.590 | 0.634 | 0.737 | 1.371
S-4 | MixStyle | 256 | - | ✓ | 0.500 | 0.595 | 0.615 | 0.745 | 1.360
S-5 | AdaResNorm | 256 | - | - | 0.493 | 0.589 | 0.593 | 0.747 | 1.340
S-6 | ResNorm | 256 | - | - | 0.491 | 0.595 | 0.595 | 0.733 | 1.328
S-7* | MixStyle | 160 | - | - | 0.480 | 0.588 | 0.643 | 0.748 | 1.391
S-8 | MixStyle inside FDY | 160 | ✓ | - | 0.485 | 0.599 | 0.629 | 0.737 | 1.366

###  4.2 Training method

For all experiments, a batch size of 60 was used, comprising the strong set,
weak set, and unlabeled set, with batch size distribution: approximately 1/5
of the maestro dataset, 1/10 of the synth dataset, 1/10 of the synth+strong
dataset, 1/5 of the weak dataset, and 2/5 of the unlabeled dataset. The
training process included 50 epochs for warmup, a maximum of 300 epochs, and
an epoch decay of 100. Gradient clipping was set at 5.0, the EMA factor for
the mean teacher [26] was 0.999, validation was performed every 10 epochs, and
the maximum weight for self-supervised loss was 2. The Adam optimizer was
employed with a learning rate of 0.001. An exponential warmup was applied for
the initial 50 epochs, and no early stopping was implemented during the
training process.

###  4.3 Evaluation metric

This year, this task requires us to consider the PSDS [35, 36] for evaluation.
Event onset and offset times required for PSDS computation are only available
for DESED data and classes, so PSDS is only evaluated on this fraction of the
evaluation set. For MAESTRO, segment-based labels (one second) are provided,
and we use the segment-based mean (macro-averaged) partial area under the ROC
curve (mPAUC) as the primary metric, with a maximum FP-rate of 0.1. mPAUC is
computed with respect to hard labels (threshold = 0.5) for the 11 classes
listed. DESED and MAESTRO clips are anonymized and shuffled in the evaluation
set to prevent manual domain identification.

###  4.4 Ensemble

Ensemble modeling is a technique that leverages the strengths of multiple
models to improve overall performance and enhance the generalization
capability of a system. In our system, ensemble modeling plays a crucial role
to improve system performance. These models work together to extract the best
aspects from the highest-performing models. To generate final predictions, we
aggregate the individual predictions from all the models and calculate their
average. This approach ensures that every model contributes to the overall
performance of the ensemble system.

##  5 Results and Analysis

In this section, we first present the findings of the of the 3 single systems,
followed by 1 ensemble system that we submitted.

###  5.1 Single-systems

The PSDS measures the performance of systems on SED for the DESED subsets. In
the DESED development set, the baseline system achieves a PSDS of 0.483.
Systems S-1* (0.506), S-2 (0.510), S-3* (0.503), and S-4 (0.500) outperform
the baseline, showing significance of MixStyle to improve performance. Systems
S-5 (0.493) and S-6 (0.491) show slight improvements, while S-7* (0.480) and
S-8 (0.485) perform slightly worse, suggesting the hop length of 160 might be
less effective. For the public evaluation set, while the baseline achieves
0.529 our systems S-1* (0.587), S-2 (0.590), S-3* (0.590), S-4 (0.595), and
S-8 (0.599) outperform the baseline, with S-8 achieving the highest score,
indicating the benefit of integrating MixStyle inside FDY.

The mPAUC is another key metric that is used for MAESTRO subset. The baseline
system achieves an mPAUC of 0.721. System S-1* shows a significant improvement
with 0.737, highlighting the positive impact of MixStyle on generalization.
System S-2 achieves the highest mPAUC of 0.753, which indicates combining
MixStyle with frequency-dynamic convolution significantly enhances
performance. System S-3* also performs well with 0.737, reinforcing the
benefits of domain generalization. Systems S-5 (0.747) and S-6 (0.733) show
moderate improvements with adaptive normalization techniques, while S-7*
(0.748) demonstrates excellent performance even with a shorter hop length.
System S-8, integrating MixStyle inside FDY, shows consistent improvement with
an mPAUC of 0.737.

When combining results from both PSDS and mPAUC, it becomes more evident that
systems integrating MixStyle and frequency-dynamic convolution (FDY)
outperform the baseline across different datasets. System S-8, for instance,
achieves the highest public evaluation set PSDS (0.599) and a strong mPAUC
(0.737), demonstrating its robustness and adaptability. Systems S-1* and S-3*
also show balanced improvements across both metrics, making them reliable
choices for the final submission.

We chose systems S-1*, S-3*, and S-7* for final submission due to their
superior performance and generalization capabilities. System S-1* achieves
high scores across all metrics, indicating robust generalization. System S-3*
combines MixStyle with frequency-dynamic convolution, effectively enhancing
domain generalization and event-level detection. System S-7* shows excellent
performance with a shorter hop length, demonstrating the model’s adaptability
to different configurations. These single-systems consistently outperform the
baseline and showed significant improvements in terms of both the performance
metrics, making them the best candidates for submission.

###  5.2 Ensemble system

We used all the single-systems except S-5 and S-6 for the ensemble system.
After applying the ensemble method mentioned in the previous section, we
achieved the highest development dataset PSDS of 0.520, a raw public
evaluation dataset PSDS of 0.620, and an mPAUC of 0.762. After applying cSEBBs
post-processing, the public evaluation dataset PSDS improved to 0.656, and
thereby improved the joint score to 1.418. We submitted this system as our
only ensemble system.

##  6 Conclusion

This report presents SED systems by Fortemedia Singapore and the Joint
Laboratory of Environmental Sound Sensing as a participation to DCASE 2024
Task 4. We addressed the challenge of recognizing overlapping events from
different sources using a method that integrates bidirectional encoder
representations from audio transformers and a convolutional recurrent neural
network. Our key strategies included applying MixStyle to adapt multi-domains,
using an independent learning framework for dataset-specific training loss,
and employing sound event bounding boxes for post-processing. As a part of our
submission we submitted 3 single-systems and 1 ensemble system. Our ensemble
system achieved the highest public evaluation dataset PSDS of 0.656, and mPAUC
of 0.762, thereby showed significant improvment over the challenge baseline.

## References

  * [1] A. Mesaros, T. Heittola, T. Virtanen, and M. D. Plumbley, “Sound event detection: A tutorial,” _IEEE Signal Processing Magazine_ , vol. 38, no. 5, pp. 67–83, 2021.
  * [2] T. Khandelwal, R. K. Das, and E. S. Chng, “Sound event detection: A journey through DCASE challenge series,” _APSIPA Transactions on Signal and Information Processing_ , vol. 13, 2024.
  * [3] J. P. Bello, C. Silva, O. Nov, R. L. Dubois, A. Arora, J. Salamon, C. Mydlarz, and H. Doraiswamy, “SONYC: A system for monitoring, analyzing, and mitigating urban noise pollution,” _Communications of the ACM_ , vol. 62, no. 2, pp. 68–77, 2019.
  * [4] J. P. Bello, C. Mydlarz, and J. Salamon, “Sound analysis in smart cities,” _Springer International Publishing_ , pp. 373–397, 2018.
  * [5] C. Debes, A. Merentitis, S. Sukhanov, M. Niessen, N. Frangiadakis, and A. Bauer, “Monitoring activities of daily living in smart homes: Understanding human behavior,” _IEEE Signal Processing Magazine_ , vol. 33, no. 2, pp. 81–94, 2016.
  * [6] S. Cornell, J. Ebbers, C. Douwes, I. Martín-Morató, M. Harju, A. Mesaros, and R. Serizel, “DCASE 2024 task 4: Sound event detection with heterogeneous data and missing labels,” _arXiv preprint:2406.08056_ , 2024.
  * [7] Y. Xiao, T. Khandelwal, and R. K. Das, “FMSG submission for DCASE 2023 challenge task 4 on sound event detection with weak labels and synthetic soundscapes,” DCASE 2023 Challenge, Tech. Rep., 2023.
  * [8] Y. Xiao and R. K. Das, “Dual knowledge distillation for efficient sound event detection,” _arXiv preprint:2402.02781_ , 2024.
  * [9] H. Yin, J. Bai, S. Huang, and J. Chen, “How information on soft labels and hard labels mutually benefits sound event detection tasks,” DCASE 2023 Challenge, Tech. Rep., 2023.
  * [10] N. Turpault, R. Serizel, A. P. Shah, and J. Salamon, “Sound event detection in domestic environments with weakly labeled data and soundscape synthesis,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2019.
  * [11] I. Martín-Morató and A. Mesaros, “Strong labeling of sound events using crowdsourced weak labels and annotator competence estimation,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 31, pp. 902–914, 2023.
  * [12] H. Nam, S.-H. Kim, B.-Y. Ko, and Y.-H. Park, “Frequency dynamic convolution: Frequency-adaptive pattern recognition for sound event detection,” in _Proc. INTERSPEECH_ , 2022, pp. 2763–2767.
  * [13] T. Khandelwal, R. K. Das, A. Koh, and E. S. Chng, “FMSG-NTU submission for DCASE 2022 task 4 on sound event detection in domestic environments,” DCASE 2022 Challenge, Tech. Rep., 2022.
  * [14] ——, “Leveraging Audio-Tagging Assisted Sound Event Detection using Weakified Strong Labels and Frequency Dynamic Convolutions,” in _Proc. IEEE Statistical Signal Processing Workshop (SSP)_ , 2023, pp. 329–333.
  * [15] K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy, “Domain generalization: A survey,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 45, no. 4, pp. 4396–4415, 2022.
  * [16] B. Kim, S. Yang, J. Kim, H. Park, J. Lee, and S. Chang, “Domain generalization with relaxed instance frequency-wise normalization for multi-device acoustic scene classification,” in _Proc. INTERSPEECH_ , 2022, pp. 2393–2397.
  * [17] J. Salamon, D. MacConnell, M. Cartwright, P. Li, and J. P. Bello, “Scaper: A library for soundscape synthesis and augmentation,” in _Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_ , 2017, pp. 344–348.
  * [18] D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,” _arXiv preprint:1510.08484_ , 2015.
  * [19] J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in _Proc. IEEE international conference on acoustics, speech and signal processing (ICASSP)_ , 2017, pp. 776–780.
  * [20] I. Martín-Morató, M. Harju, P. Ahokas, and A. Mesaros, “Training sound event detection with soft labels from crowdsourced annotations,” in _Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2023, pp. 1–5.
  * [21] D. Hovy, T. Berg-Kirkpatrick, A. Vaswani, and E. Hovy, “Learning whom to trust with MACE,” in _Proc. the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)_ , 2013, pp. 1120–1130.
  * [22] A. Mesaros, T. Heittola, and T. Virtanen, “TUT database for acoustic scene classification and sound event detection,” in _Proc. 24th European Signal Processing Conference (EUSIPCO)_ , 2016, pp. 1128–1132.
  * [23] F. Ronchini, S. Cornell, R. Serizel, N. Turpault, E. Fonseca, and D. P. Ellis, “Description and analysis of novelties introduced in DCASE task 4 2022 on the baseline system,” _arXiv preprint:2210.07856_ , 2022.
  * [24] E. Çakir, G. Parascandolo, T. Heittola, H. Huttunen, and T. Virtanen, “Convolutional recurrent neural networks for polyphonic sound event detection,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 25, no. 6, pp. 1291–1303, 2017.
  * [25] S. Chen, Y. Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, and F. Wei, “BEATs: Audio pre-training with acoustic tokenizers,” in _Proc. International conference on machine learning (ICML)_ , vol. 202, 2023, pp. 5178–5193.
  * [26] A. Tarvainen and H. Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” in _Proc. Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 30, 2017.
  * [27] L. JiaKai, “Mean teacher convolution system for DCASE 2018 task 4,” DCASE 2018 Challenge, Tech. Rep., 2018.
  * [28] K. Zhou, Y. Yang, Y. Qiao, and T. Xiang, “Domain generalization with mixstyle,” in _Proc. International Conference on Learning Representations (ICLR)_ , 2020.
  * [29] H. Nam and H.-E. Kim, “Batch-instance normalization for adaptively style-invariant neural networks,” in _Proc. Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 31, 2018.
  * [30] D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, “SpecAugment: A simple data augmentation method for automatic speech recognition,” in _Proc. INTERSPEECH_ , 2019, pp. 2613–2617.
  * [31] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “Mixup: Beyond empirical risk minimization,” in _Proc. International Conference on Learning Representations (ICLR)_ , 2018.
  * [32] L. Cances, P. Guyot, and T. Pellegrini, “Evaluation of post-processing algorithms for polyphonic sound event detection,” in _Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_ , 2019, pp. 318–322.
  * [33] J. Ebbers, F. G. Germain, G. Wichern, and J. Le Roux, “Sound event bounding boxes,” _arXiv preprint:2406.04212_ , 2024.
  * [34] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in _Proc. the IEEE conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 779–788.
  * [35] C. Bilen, G. Ferroni, F. Tuveri, J. Azcarreta, and S. Krstulovic, “A framework for the robust evaluation of sound event detection,” in _Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 61–65.
  * [36] J. Ebbers, R. Haeb-Umbach, and R. Serizel, “Post-processing independent evaluation of sound event detection systems,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2023.