11institutetext: Amazon Web Services111Copyright © 2021 Amazon Web Services.
Use permitted under a CC BY 4.0 license., Seattle, WA
11email<EMAIL_ADDRESS>
# Graph? Yes! Which one? Help!

Ora Lassila  Michael Schmidt  Brad Bebee  Dave Bechberger  Willem Broekema
Ankesh Khandelwal  Kelvin Lawrence  Ronak Sharda  Bryan Thompson

###### Abstract

Amazon Neptune is a graph database service that supports two graph
(meta)models: W3C’s Resource Description Framework (RDF) [8] and Labeled
Property Graphs (LPG) [15, 13]. Customers opt in for one or the other model,
and this choice determines which data modeling features can be used, and –
perhaps more importantly – which query languages are available to query and
manipulate the graph. The choice between the two technology stacks is
difficult and requires consideration of data modeling aspects, query language
features, their adequacy for current and future use cases, as well as many
other factors (including developer preferences). Sometimes we see customers
make the wrong choice with no easy way to reverse it later.

It is therefore highly desirable that the choice of the query language can be
made without consideration of what graph model is chosen, and can be easily
revised or complemented at a later point. In this paper, we advocate and
explore the idea of a single, unified graph data model that embraces both RDF
and LPGs, and naturally supports different graph query languages on top. We
investigate obstacles towards unifying the two graph data models, and propose
an initial unifying model, dubbed “one graph” (“1G” for short), as the basis
for moving forward.

###### Keywords:

RDF Labeled Property Graphs Graph Queries.

##  1 Introduction

The Amazon Neptune graph database supports both RDF and LPG models and query
languages, but today customers are not able to “cross-use” these technologies
(e.g., a user cannot query RDF data using Gremlin). This as well as the
differences in domain modeling approaches between the two graph models cause
confusion among prospective Neptune users. These users frequently struggle
with the choice between the two models. We have even seen customers make the
wrong choice with no easy way to reverse direction later. Occasionally, we see
requests for interoperability (i.e., cross-use) from existing users who are
already proficient with RDF or LPG, or both, and from customers with use cases
that are sometimes better served via RDF and at other times better served via
LPG.

Making a good choice between the two technology stacks is complex and requires
a balanced consideration of data modeling aspects, query language features,
and their adequacy for current and future use cases. Furthermore, the choice
has impact on aspects such as ease of data maintenance, augmentation and
interchange, software ecosystems (such as explorers, visualization and
application development tools), or simply developer expertise and preferences.
We have repeatedly seen our customers struggle with this decision, and the
eventual choice is often dominated by the query language that appears most
appealing for implementing the initial use case, yet additional requirements
and new use cases that emerge over time may challenge the initial choice.

RDF offers a formal model that supports global identifiers (IRIs), well-
defined graph merging, a natural way to break a graph into subgraphs, and
federated query, making it well-suited to use cases where multiple
(potentially external) data sources are used (see, for example, the Linked
Open Data project222https://lod-cloud.net/). RDF also supports the reuse of
schemas in the form of vocabularies for ontology definition as well as logical
reasoning. Not surprisingly, we often see information architects prefer the
features of the RDF model because of a good fit with use cases for data
alignment, master data management, and data exchange. LPGs, on the other hand,
are more in line with familiar programming models, and offer good integration
with a number of programming languages. Software developers often choose an
LPG language because they find it more natural and more “compatible” with
their programming paradigm. Developers coming from the SQL world often like
that a vertex in an LPG is much like a row in a relational database.
Unfortunately, many of these developers are then challenged when they need to
expand a single graph to multiple data sources or when they have to deal with
external data. Note that the choice of LPG can also happen when RDF is
dismissed out of hand because it is viewed as complex and “academic”.
Regardless of what the reasons, we believe that the (forced) choice of graph
models slows the adoption of graphs because it creates confusion and
segmentation in the graph database space.

What if users could choose between different query languages, independent of
what graph model they have decided to use? From interactions with Neptune
customers we have noticed that there are often strong preferences for a
particular query language, and there are also situations where one query
language is simply better suited because of its particular features (e.g.,
expressiveness of graph traversals and path queries in Gremlin vs. SERVICE
federation in SPARQL [12]). We argue that one’s choice of a query language
should not dictate that one has to forego benefits of a graph model said query
language does not support.

Therefore, in this paper we examine the idea of graph interoperability. That
is, removing the obstacles that prevent us from using SPARQL [4] over LPGs,
Gremlin [14] or openCypher333https://github.com/opencypher over RDF, etc. The
goal is not merely to be able to cross-use query languages, but to be able to
do it in a manner where the user does not have to be cognizant (and careful)
about how the interoperability is achieved. In other words, we are interested
in a data model that combines both RDF and LPG into a model providing a
unified semantics that includes (and generalizes) the specifics of the
individual models. In addition to flexibility in choosing the query language,
the idea is that this unified data would also allow graph users to combine and
interlink data sets maintained in both RDF and LPG formats. More generally,
the idea of providing a unified data model abstracts away the need for
customers to choose a data format ahead of time, therefore removing a major
obstacle to graph database adoption.

In this context it should be noted that less interesting is some kind of
“qualified interoperability” where the cross-use of query languages would
require one to understand the implementation of the underlying graph models –
in other words, we want to stay away from, say, implementing RDF using an
LPG,444See, for example, this: https://www.tigergraph.com/blogs/about-
tigergraph/graph-gurus-episode-1-building-an-enterprise-knowledge-graph-from-
rdf-data/ because this simply requires the users to be knowledgeable of
multiple models instead of a single one.

##  2 The 1G Model

We start our investigation in the context of a graph database (Neptune) that
uses a “quad-based” internal representation for both RDF and for LPG, and
while this may seem to bias our thinking towards RDF-centric solutions, our
broad goal is to not be preferential to either graph model.

In the 1G data model, we represent data – RDF graphs, LPGs, or both – using a
set of so-called statements of the form

$\displaystyle\texttt{src}\xrightarrow{\text{label}}\texttt{value}:sid$

where src represents a source vertex, label denotes the edge or property
label, value denotes either the target vertex or the property value (such as a
string, number, etc.), and $sid$ assigns a globally unique ID (as will be
discussed later, the same statement might come with different identifiers).
For RDF, 1G statements map directly to SPO triples with unique ID; for LPGs,
they give us a unified mechanism to represent edges and properties alike. If
the type of value is fixed (to either a vertex or a property value), we may
refer to statements as edges and properties, respectively. The main purpose of
this formalism is to define the semantics of our model and does not imply a
particular physical implementation.

The $sid$s can be used in either src or value position, to refer to the
statements that they represent (say, to make statements about these
statements). As a concrete example for a 1G dataset, consider the following
statements:

$\displaystyle\texttt{Alice}\xrightarrow{\text{knows}}\texttt{Bob}:sid_{1}$
$\displaystyle\texttt{Alice}\xrightarrow{\text{name}}\texttt{"Alice"}:sid_{2}$
$\displaystyle\texttt{Bob}\xrightarrow{\text{name}}\texttt{"Bob"}:sid_{3}$

We can then introduce additional statements that refer to the ones above by
referencing existing $sids$, in this example $sid_{1}$, for instance:

$\displaystyle sid_{1}\xrightarrow{\text{since}}2020:sid_{4}$

Conceptually, we distinguish between ground statements (or ground edges and
properties, if their value type is known) as statements that do not contain
$sids$, and assertions, represented as statements with $sid$s in src and/or
value position.555We do not see a strong use case for supporting $sid$s in the
label position. In the previous example, our three ground statements describe
two persons Alice and Bob with their names, stating that Alice knows Bob. The
assertion over the $sid$ of the first ground statement states that the knows
relationship between Alice and Bob, represented by $sid_{1}$, exists since
2020 – this is how 1G captures what is commonly referred to as edge properties
in LPGs.

Interoperability between graphs can now be defined as “views” of the 1G model
for RDF, RDF-star [5], and LPG. Those views can be understood as specific
interpretations (or mappings) of the generic 1G model into a specific graph
data model. We sketch this idea by providing possible interpretations for our
toy graph, for RDF, RDF-star, and LPG, respectively:

    # Plain RDF without reification (no support for edge properties)
    :Alice :knows :Bob .
    :Alice :name "Alice" .
    :Bob :name "Bob" .

    # RDF-star: edge properties = statements about "reified" triples
    :Alice :knows :Bob .
    <<:Alice :knows :Bob>> :since 2020 .
    :Alice :name "Alice" .
    :Bob :name "Bob" .

With LPG, assertions over edges are viewed as edge properties, so our LPG
interpretation of the 1G model consists of the following vertices and edges:

$\displaystyle v_{1}\textrm{ with }id=\texttt{Alice}\textrm{ and properties
}\\{\texttt{name}=>\texttt{"Alice"}\\}$ $\displaystyle v_{2}\textrm{ with
}id=\texttt{Bob}\textrm{ and properties }\\{\texttt{name}=>\texttt{"Bob"}\\}$
$\displaystyle e_{1}\textrm{ from }v_{1}\textrm{ to }v_{2}\textrm{ with label
}\texttt{knows}\textrm{ and properties }\\{\texttt{since}=>2020\\}$

There exist, of course, other possible interpretations of the 1G model for
each of the existing graph data formats (and we deliberately omit details such
as mapping between 1G identifiers and RDF/LPG identifiers, which we will
discuss in Section 3.5). For instance, in our example we chose the assertion

$\displaystyle sid_{1}\xrightarrow{\text{since}}2020:sid_{4}$

to be “invisible” in the context of a plain RDF model; alternatively, we could
have represented this assertion using a set of triples that leverage the RDF
reification vocabulary. The goal of this paper is not to provide a complete
formalization of the 1G model and its interpretation, but rather leverage it
as a framework to discuss the key questions that need to be answered in order
to achieve graph interoperability, and would lead to a “natural” semantics for
the query languages over a unified graph data model.666In Section 3.4 we show
how the 1G model is used to represent RDF’s named graphs. The discussion in
this paper is limited to LPGs used by Gremlin and openCypher. Other graph
query languages and implementations may come with additional challenges to
consider. Our evaluation of RDF (wrt. our 1G model) includes features of RDF-
star.

##  3 Interoperability challenges and some proposed solutions

We discuss challenges to interoperability broadly, ranging from semantics to
potential implementation issues.777Note that the issue of schema languages is
out of scope for this paper.

###  3.1 Challenge #1: Edge properties, multiple edge instances, and
reification

One of the most fundamental perceived (structural) difference between RDF and
LPGs is that RDF does not offer built-in edge property support (except as
“statements about statements” via reification, which is generally understood
to be cumbersome and inefficient). We sketched this aspect in Section 2
already when talking about possible interpretations for edge properties in
RDF. A related issue is that LPGs support more than one instance of an edge.
The RDF-star effort is adding edge properties to RDF, but – as the draft
specification currently stands – not in a way that is completely compatible
with LPGs. In RDF-star as currently formulated, a triple $<s,p,o>$ is
understood to be unique, in the sense that there cannot exist an identical
instance of that triple with its own identity. As such, all statements
involving said $<s,p,o>$ are understood to reference the same triple (in
contrast, the original RDF reification mechanism [16, section 4.3] is more
expressive: each statement model has its own identity). In LPGs, each edge is
considered a unique object with its own identity, and it is perfectly possible
to have two edges with identical endpoints and label. The following example
illustrates the case of multiple edge identities using the 1G model:

$\displaystyle\texttt{Alice}\xrightarrow{\text{knows}}\texttt{Bob}:sid_{1}$
$\displaystyle\texttt{Alice}\xrightarrow{\text{knows}}\texttt{Bob}:sid_{2}$

Further assume there are four assertions over these two ground edges:

$\displaystyle sid_{1}\xrightarrow{\text{statedBy}}\texttt{NYTimes}:sid_{3}$
$\displaystyle sid_{1}\xrightarrow{\text{since}}2020:sid_{4}$ $\displaystyle
sid_{2}\xrightarrow{\text{statedBy}}\texttt{TheGuardian}:sid_{5}$
$\displaystyle sid_{2}\xrightarrow{\text{since}}2021:sid_{6}$

Informally speaking, the four assertions over the ground facts state that
Alice knows Bob (a) since 2020 as per the NY Times, and (b) since 2021 as per
The Guardian. This example has a natural representation in LPGs (namely, two
distinguished edges, each carrying the respective two edge properties),
whereas there is no built-in mechanism in RDF-star as defined at the time of
writing. This means that with respect to edge properties (and some other
issues notwithstanding) we can take an RDF(-star) graph and map it onto an
LPG, but there are cases where the converse is not true, at least not without
falling back on more complex modeling approaches (such as introducing
intermediate nodes or using named graphs to attach custom statement
identifiers).

Prohibiting multiple edges would seriously hamper how LPGs are used – there is
a clear use case behind the distinction of the two edges in the example above
– so it makes sense to capture this expressiveness in 1G. The question then
becomes what an RDF-star view (where the difference cannot be expressed) over
such data would look like. One option is to collapse multiple edges (in our
example, $sid_{1}$ and $sid_{2}$) into a single edge, effectively exposing a
single edge with four edge properties when looking at 1G from an RDF-star
perspective. This may still allow for some useful SPARQL-star queries (e.g., a
query such as “give me all ground facts stated by :NYTimes”) but in the
general case information would be lost. Conceptually, this could be seen as a
“dimensional reduction” when mapping 1G into RDF-star, which is accounting for
the fact that RDF-star is just not expressive enough to capture the
distinction between these two edges.

There are a couple of other, similar questions related to reification, apart
from the edge identity challenge discussed above. Those include aspects such
as multi-level reification (which can be expressed with RDF-star but not with
LPGs), questions whether the reification of a 1G statement via its $sid$
necessarily implies the presence of the underlying statement as a ground
statement or not (in our 1G model it does, and in fact the model is not
capable of referring to non-asserted statements). While those may sound like
technical details, we believe that the challenge of overcoming these
“reification gaps” is at the very heart of graph interoperability.

From our experience, the convenience of having edge properties as a built-in
construct is one of the core strengths of LPGs over RDF. Given the existing
gaps in expressiveness and the early stage of RDF-star proposals, it might
also be worth re-considering RDF-star and better aligning its expressiveness
to capture the distinction between multiple edges.

###  3.2 Challenge #2: Triples vs. graph abstraction

At the very basic level, an RDF graph is defined as a “set of triples”,
whereas LPGs are defined as (optionally labeled) nodes with properties that
can be connected via labeled edges. The LPG notion of a vertex label is very
similar to that of rdf:type. That is, LPG labels classify a vertex. While the
mapping of triples to and from our abstract 1G model is straightforward (in
the absence of reification, there is one 1G statement for every triple, with a
unique sid assigned), the mapping from LPGs into the 1G model is more
challenging. For instance, certain flavors of LPGs allow “stand-alone” nodes
without labels and properties, i.e., a node itself that neither carries any
information nor is connected to any other node.888Note that RDF, effectively,
does not have vertices, in the sense that they could exist “stand-alone”.
There is an infinite space of identifiers, any of which could be used as
vertices. This is not merely a philosophical notion, as we now see. In fact,
such nodes can be extracted via query languages (in Gremlin via g.V() or in
openCypher via MATCH (n)), but there is no natural representation for these
stand-alone nodes in our 1G model. In Gremlin and openCypher, vertices always
have at least a default label, however.999This is not unlike every node in an
RDF graph being an instance of rdfs:Resource.

Similar questions arise when querying data originating from RDF graphs via LPG
query languages. LPG languages typically follow the abstraction layer of
vertices and edges. Gremlin, for instance, allows to enumerate all vertices
using the query g.V() whereas edges can be extracted via g.E(). In the LPG
model, the sets of vertices and edges are disjoint.

The key challenge in defining the semantics of LPG query languages becomes
defining the concepts of vertices and edges over our 1G model – which would
manifest itself in a formal mapping from 1G to LPGs. Concretely, the question
becomes how we define the concepts of vertices, edges, vertex properties, and
edge properties in a 1G-to-LPG mapping. The basic idea here would be to map
identifiers from src and value position of 1G statements to vertices, and use
the sids as edge or property identifiers (depending on whether a statement
carries an identifier (such as Alice or Bob) or a literal (such as a string
"Alice" or "Bob") in its value position, respectively).

###  3.3 Challenge #3: Datatype alignment

1G requires a unified type system over RDF and LPGs. As a W3C standard it
builds upon the XML Schema definition (XSD) [2] and utilizes primitive XSD
datatypes such as strings, numbers, and dates. Since RDF is defined along an
Open World paradigm, its datatypes tend to be more extensible and flexible
than LPG types. For instance, there is no validation in place that restricts
users from adding ill-typed values such as "this is not an
integer"$\wedge\wedge$xsd:integer, and users can also provide language tags
for simple string values. Composite types such as lists, bags, and sequences
are not available as “primary” literal types but need to be modeled explicitly
using RDF containers.101010https://www.w3.org/TR/rdf-schema/#ch_container RDF
supports bags, but not sets, because the graph structure offers no easy means
to enforce set semantics while also permitting the merge of RDF data sets.

For LPGs, on the other hand, there exists (to the best of our knowledge) no
formal definition of the type system. In contrast to RDF, LPGs support
different sets of composite datatypes as built-in types. This means that,
while RDF uses the graph structure to model composite types, in LPGs the
attribute value itself is an instance of a composite type (a list, map, etc.).
This reflects the general notion of semi-structured JSON documents as
attribute values. Semantics of these datatypes, however, are opaque and are
typically “delegated” to the underlying implementation language, making it
potentially hard to unify graph representation. Generally, we have a menagerie
of datatypes to try to reconcile, and needless “baggage” because of the
reliance on implementation languages. The challenge for a unifying graph data
model is to define a meta type system that captures and aligns these different
types, giving them a concrete semantics.

One possible approach to align the type systems between RDF and LPGs more
closely – with a focus on composite types – would be to leverage RDF’s user-
defined literal mechanism, which is an extension mechanism that can be used to
represent arbitrary complex types. As an alternative to the structural
approach usually taken by RDF, a list could be represented in RDF as a literal
value whose lexical form is, say, "[1, 2, 3]"$\wedge\wedge$:OneGraphList, the
serialized form of a list, and where :OneGraphList is a reserved datatype IRI
defining the literal value to be a list with well-defined syntax that can be
used interchangeably in RDF and LPGs.111111The syntax makes no assertion about
how the data are represented internally. This approach renders these datatypes
opaque with respect to existing RDF semantics, but for processing (say, in
SPARQL) we could define some type of folding/unfolding operations to expose
the internal structure, or even specific user-defined functions such as
element projections that understand and interpret such typed literals. In some
ways these operations could be considered an extension of the “literal value”
notion that is part of the RDF specification [8, section 3.3].

###  3.4 Challenge #4: Graph partitioning

SPARQL defines the notion of named graphs, which are often used to support
subgraph management use cases. Named graphs are usually thought of as an
extension of the triple model to a quad model with the addition of a
(sub)graph identifier. Some users have chosen to treat named graphs as
containers (sometimes containers of a single triple) to make “statements about
statements” (or sets of statements) in lieu of using the reification mechanism
(this, in the absence of the proposed RDF-star scenario). This is outside the
defined formal semantics of RDF, since named graphs do not have any semantic
theory in the RDF model.

Rather than having an RDF quad like $<s,p,o,g>$, in 1G we introduce a
“membership relation” (in this paper we use inGraph as a reserved label):

$\displaystyle s\xrightarrow{\text{p}}o:sid_{1}$ $\displaystyle
sid_{1}\xrightarrow{\text{inGraph}}g:sid_{2}$

where $sid_{1}$ and $sid_{2}$ are statement identifiers. The motivation behind
this approach is to restore symmetry to the data model instead of privileging
named graphs as somehow special. Named graphs are just an application of the
statement identifier, much like aspects in [9]. Note, however, that the 1G
model does not dictate how a database physically organizes the graph. There
is, of course, a storage size consideration when using additional
triples/quads.

From the standpoint of SPARQL semantics, the membership triples are not
visible, but instead could be considered an implementation artifact. However,
with this proposal they would be part of the data in the data set. SPARQL
default graph and named graph semantics are recovered by performing the
appropriate operations over the logical model. Again, we do not imply that the
physical schema needs to have this same data organization.

Since in our proposed solution we treat named graph membership effectively as
an “edge or meta property”, this approach can be extended to LPGs. In an LPG
implementation where properties (including vertex properties) are represented
as triples/quads, this approach permits vertex properties to be assigned graph
membership. It remains a debatable issue whether the notion of subgraphs in an
LPG would extend to vertices themselves and, if not, how the LPG view would be
recovered from the 1G model. Obviously in RDF this is not an issue since RDF
vertices are merely names with no internal structure.

###  3.5 Challenge #5: Graph merging and external identifiers

This is one of the strongest benefits of RDF, and conversely one of the
weakest aspects of LPGs: RDF has a specification for graph merging [6, section
4.1]. Whenever multiple data sources are used, particularly from multiple
organizations, such as in the Linked Open Data cloud, graph merging is a key
functionality. Strong external identifiers (IRIs) are an essential part of
this mechanism, but while it is perfectly possible to use IRIs as identifiers
in LPGs, there is more to graph merging; edge properties, and particularly
multiple edge instances (see above) complicate any merging semantics.
Specifically, under which conditions could two “similar” edges in two graphs
that are to be merged be considered the “same” edge?

Allowing both RDF and LPG data to be represented in a single model requires
the co-existence of global identifiers (i.e., IRIs coming from RDF data) and
local identifiers (i.e., node and edge identifiers in the form of strings,
coming from LPG data). Such a distinction would make it possible to load data
via both RDF and LPG data formats into the same logical graph; this would give
us mere co-existence, without any (initial) overlap in vertex identifiers,
labels, etc. User-defined default namespaces could then be used to expose
local identifiers as if they were IRIs, whenever they are queried via SPARQL;
conversely, IRIs originating from RDF data sets could be shortened according
to existing namespace prefixes, to make querying and syntactic result
representation for LPG query languages more user-friendly.

The typical use case, however, goes beyond just the mere co-existence of local
and global identifiers: a common use case would be to unify elements (such as
nodes and edges) originating from RDF and LPGs. Assume, for instance, a user
who maintains internal data about countries, available as LPG data in which
countries are identified via a simple country code string, and that the user
wants to augment this data with information coming from an RDF dump of the
Geonames dataset.121212https://www.geonames.org/ When initially loading the
data sets, the LPG country identifier strings and the IRIs for the respective
countries coming from Geonames would be disjoint; in order to allow users to
align such identifiers, we would need to provide user-configurable rules that
guide the merging process (in this case, collating the country IRIs from the
Geonames RDF data with the country code strings from the LPG dataset). This
mechanism could also be used to customize RDF graph merging when blank nodes
are present (e.g., when two graphs derived from the same source are merged,
one may want to treat blank node identifiers as indeed identifying the same
blank nodes – this is something that is allowed under RDF semantics).
Identifiers could also be generated using a templating mechanism such as the
one employed in R2RML [3, section 7.3].

###  3.6 Challenge #6: Lack of formal foundation

Unlike SPARQL or SQL, LPG query languages – by and large – lack strict formal
semantics (in the form of, say, a query algebra). This makes it hard to assess
semantic compatibility. Similarly, unlike RDF, formal semantics do not exist
for LPGs (or only exist post hoc, as in [17, 10]). For LPG query languages,
semantics is typically defined informally either via documentation and
examples, or via an implementation. Our Gremlin implementation, for instance,
is largely based on our interpretation of the informal Tinkerpop
specification131313https://tinkerpop.apache.org/gremlin.html – which comes
with details that are not unambiguously defined – and its reference
implementation. To provide a simple example, consider the question of ordering
guarantees: if a Gremlin query contains an order() step in a non-terminal
position, it is ambiguous whether (and how) subsequent steps should maintain
that order. For example, would a Gremlin query such as
g.V(1,2,3,4).order().by(’age’).out() guarantee that out() nodes have to be
reported for nodes with ids 1, 2, 3, and 4 in the order implied by their age
property? Similar questions around semantics arise in the context of dynamic
typing, execution order guarantees, and runtime exceptions thrown through the
query language. And of course, such questions arise in the context of other
query languages like openCypher. While openCypher does provide an official
spec, certain details are not entirely worked out. Of course, SPARQL is not
entirely without fault either, as there is some ambiguity in its specification
as well – see, for example, [7] and also [11].

###  3.7 Challenge #7: Update query semantics

In order to be able to subsume both the RDF(-star) and the LPG models, a
unifying graph model needs to be as expressive as the “most expressive” model,
in each of the considered dimensions. As we have illustrated in previous
examples, certain extensions that are defined for the more expressive model
may not have a natural representation in the less expressive model, thus
introducing “dimensions” that are invisible when looking at the data from the
less expressive model’s perspective. As a concrete example, we discussed in
Section 3.1 that multiple edge properties modeled in our 1G model, while
visible in an LPG interpretation, may collapse into a single edge property in
an RDF-star interpretation.

While read query semantics can be unambiguously defined by mapping the 1G
model to a lower-dimensional level, the situation becomes more complex for
queries that manipulate the data. Consider, for instance, the example database
sketched in challenge #1, and assume we write a SPARQL query (non- SPARQL-
star) that requests to drop the triple :Bob :knows :Alice. From a perspective
of SPARQL (e.g., when querying), this triple would always show up once, but
the question is whether dropping it should – behind the scenes – delete both
ground statements? And if so, should deletion include the deletion of the
attached edge properties (which exposes a risk for accidental over-deletion)?
Would adding an edge for every subject with IRI :Bob to a new person introduce
one or two edges? As another scenario, when we use SPARQL-star and add a new
edge property to the toy 1G model introduced in Section 3.1, which of the two
edges would we attach it to?

The root cause for the ambiguity of all these scenarios is that we request
updates over a simplified, dimensionally reduced view of the data. While the
discussion of all these questions goes beyond the scope of this paper, the
semantics of such operations needs to be carefully designed in order to avoid
unwanted side effects and obtain a “natural” behavior in scenarios where data
is queried and manipulated via different query languages.

##  4 Conclusions and the way forward

A recent survey [1] of organizations working on (or considering to adopt)
knowledge graphs found that interoperability and standards are the highest
priority among survey respondents. Data integration was seen as the dominant
use case. These findings could be interpreted to suggest a need for RDF/LPG
compatibility and unification. While it may seem that making RDF and LPGs
fully compatible is not possible (as per the official RDF specifications and
the emerging RDF-star work), we believe there is a way forward. Minimally, we
must address the challenges of edge identity (multiple similar edges), graph
merging, and well-defined semantics for updates across languages. One way
forward would be to define some kind of “compatibility subset” to cover enough
ground so that most RDF and LPG applications would work with no or minimal
modifications. Lack of interoperability slows the overall adoption of graph
technologies, and thus should be a high-priority item to be addressed by the
broader graph community.

In this paper we have presented the 1G model as a source of ideas about where
to go next. The open questions we have posed imply lots of interesting
research, the outcomes of which will have significant practical relevance. We
now seek support from the broader community to look into these topics.

## References

  * [1] Atkin, M., Deely, T., Scharffe, F.: Knowledge Graph Benchmarking Report 2021 (version 2.0). Zenodo, http://doi.org/10.5281/zenodo.4950097 (June 2021)
  * [2] Biron, P.V., Malhotra, A.: XML Schema Part 2: Datatypes (Second Edition). W3C Recommendation, World Wide Web Consortium (Oct 2004), http://www.w3.org/TR/2004/REC-xmlschema-2-20041028/
  * [3] Das, S., Sundara, S., Cyganiak, R.: R2RML: RDB to RDF Mapping Language. W3C Recommendation, World Wide Web Consortium, Cambridge, MA (Sep 2012)
  * [4] Harris, S., Seaborne, A.: SPARQL 1.1 Query Language. W3C Recommendation, World Wide Web Consortium, Cambridge, MA (2013)
  * [5] Hartig, O., Champin, P.A., Kellogg, G., Seaborne, A., Arndt, D., Broekstra, J., DuCharme, B., Lassila, O., Patel-Schneider, P.F., Prud’hommeaux, E., Thibodeau, Jr., T., Bryan, T.: RDF-star and SPARQL-star. W3C Community Group Working Draft, World Wide Web Consortium (June 2021)
  * [6] Hayes, P.: RDF Semantics. W3C Recommendation, World Wide Web Consortium (Feb 2004), http://www.w3.org/TR/rdf-mt/
  * [7] Hernández, D., Gutierrez, C., Angles, R.: Correlation and Substitution in SPARQL. arXiv preprint arXiv:1606.01441 (2016)
  * [8] Klyne, G., Carroll, J.J.: Resource Description Framework (RDF): Concepts and Abstract Syntax. W3C Recommendation, World Wide Web Consortium, Cambridge, MA (Feb 2004), http://www.w3.org/TR/rdf-concepts/
  * [9] Lassila, O., Mannermaa, M., Oliver, I., Sabbouh, M.: Aspect-Oriented Data. Accepted submission to the W3C Workshop on RDF Next Steps, World Wide Web Consortium (June 2010)
  * [10] Marton, J., Szárnyas, G., Varró, D.: Formalising openCypher graph queries in relational algebra. In: European Conference on Advances in Databases and Information Systems. pp. 182–196. Springer (2017)
  * [11] Patel-Schneider, P.F., Martin, D.: EXISTStential Aspects of SPARQL. In: International Semantic Web Conference (Posters & Demos). vol. CEUR-WS #1690 (2016)
  * [12] Prud’hommeaux, E., Buil-Aranda, C.: SPARQL 1.1 Federated Query. W3C Recommendation, World Wide Web Consortium, Cambridge, MA (2013)
  * [13] Robinson, I., Webber, J., Eifrem, E.: Graph databases: new opportunities for connected data. O’Reilly Media, Inc. (2013)
  * [14] Rodriguez, M.A.: The Gremlin Graph Traversal Machine and Language (invited talk). In: Proceedings of the 15th Symposium on Database Programming Languages (DBPL 2015). pp. 1–10 (2015)
  * [15] Rodriguez, M.A., Neubauer, P.: Constructions from dots and lines. Bulletin of the American Society for Information Science and Technology 36(6), 35–41 (2010)
  * [16] Schreiber, G., Raimond, Y.: RDF 1.1 Primer. W3C Recommendation, World Wide Web Consortium, Cambridge, MA (2014)
  * [17] Shinavier, J., Wisnesky, R.: Algebraic property graphs. arXiv preprint arXiv:1909.04881 (2019)


# The Impact of Frequency Bands on Acoustic Anomaly Detection of Machines
using Deep Learning Based Model

Tin Nguyen1∗, Lam Pham2∗, Phat Lam3, Dat Ngo4, Hieu Tang5, Alexander
Schindler6 L. Pham and A. Schindler are with Austrian Institute of Technology,
Austria.T. Nguyen and P. Lam are with HCM University of Technology, VietnamD.
Ngo is with University of Essex, UKH. Tang is with FPT University, Vietnam(*)
Main and equal contribution into the paper.

###### Abstract

In this paper, we propose a deep learning based model for Acoustic Anomaly
Detection of Machines, the task for detecting abnormal machines by analysing
the machine sound. By conducting extensive experiments, we indicate that
multiple techniques of pseudo audios, audio segment, data augmentation,
Mahalanobis distance, and narrow frequency bands, which mainly focus on
feature engineering, are effective to enhance the system performance. Among
the evaluating techniques, the narrow frequency bands presents a significant
impact. Indeed, our proposed model, which focuses on the narrow frequency
bands, outperforms the DCASE baseline on the benchmark dataset of DCASE 2022
Task 2 Development set. The important role of the narrow frequency bands
indicated in this paper inspires the research community on the task of
Acoustic Anomaly Detection of Machines to further investigate and propose
novel network architectures focusing on the frequency bands.

Keywords— Data augmentation, pseudo audio, Gamma distribution, Euclidean
distance, Mahalanobis distance.

##  I INTRODUCTION

In industrial settings, the reliability of machinery is crucial for ensuring
uninterrupted operations, which plays a decisive role in achieving production
efficiency and cost-effectiveness. Therefore, machine condition monitoring,
which involves the continuous monitoring of machinery to ensure machine
reliability, is considered as an essential component in industrial settings.
In terms of preventing machine failures, anomalies in machine sound are
considered as one of the potential signals indicating the machinery breakdown
situation. Therefore, the task of Acoustic Anomaly Detection of Machines
(AADoM) proves valuable in the early detection of potential faults through
sound observation, serving as the primary and proactive stage of an effective
machine condition monitoring system. The AADoM has received attention from the
scientific research community in recent years and become an increasingly
challenging task in the widespread development stage of AI-based factory
automation. Since the machine typically operates without issues for the
majority of its runtime, collecting a number of anomalous sounds seems to be
challenging and unrealistic. Therefore, the self-supervised/unsupervised
approaches proposed for AADoM task have become popular. Another challenge of
the AADoM task is the issue of domain shifting when there are variations in
acoustic characteristics between training and test data caused by some factors
such as operational speed, different environmental settings, noise, etc. This
leads models to fail to detect anomalies within different domains. For this
reason, the pursuit of developing AADoM systems using domain generalization
techniques has become a common objective in real-life applications. To deal
with these two main challenges, a wide range of unsupervised/self-supervised
and deep learning based approaches combining with domain generalization
techniques have been proposed that successfully explore the sound features and
distinguish normal or abnormal behavior of the machine sound. For example, [1,
2, 3] utilized Autoencoder to learn the normal audio data from various domains
and extract the latent space representing normal audio. Then, the anomaly
score is calculated based on construction error between the evaluating audio
data with the latent space. Another approach involves leveraging self-
supervised based models that learn different attributes of available normal
data across various domains such as machine operation conditions, machine
types, noise conditions and use them as feature extractors to get audio
embeddings in the representation of normal data. Some deep neural network
based architectures are proposed to extract audio embeddings in a self-
supervised manner such as EfficientNet-B0 [4], ResNet [5], Efficient Residual
Net [6], MobileNetV2 [7], Mobile Facenet [8, 9], etc. Then, anomaly detectors
such as Gaussian mixture models (GMM) [10], local outlier factor (LOF) [11],
or k-nearest neighbors (k-NN) are manipulated to calculate anomaly score from
the extracted audio embeddings using different metrics such as Euclidean
distance or Cosine similarity.

Generally, the existing AADoM systems mostly focus on three following
strategies to enhance the model performance: improving network architectures
to learn audio features better from given audio data across domains and then
extract a generalized representation for normal audio data; improving anomaly
detector algorithms and the estimated distribution of normal data to better
differentiate anomalous samples among normal ones; improving domain
generalization techniques to handle domain shifting problem. However, most of
the existing AADoM systems have not focused on exploring feature extraction
where spectrograms are generated and represent the audio input. In particular,
these systems used the entire spectrogram which provides a vast amount of
information, but not all of the information in the spectrogram is necessarily
indicative of anomalies. (i.e. white noise at low frequency, irrelevant
spectral lines at very high frequency). Second, different types of machines
exhibit distinctive characteristic sounds related to their components and
mechanisms. The relevant patterns of normality and anomalies may be
concentrated in specific frequency bands for a particular machine. Therefore,
analyzing all frequency bands within the entire spectrogram for all machine
types may overlook localized anomaly patterns, making it more challenging to
detect and interpret insightful features. Finally, training a model on the
entire spectrogram can be more complex, requiring a larger amount of labeled
data and storage capacity, which is unsuitable for real-time applications or
systems with limited computational resources. In this paper, we therefore
focus on exploring feature extraction and then indicate the main factors
influencing the model performance. Initially, we propose a baseline system for
the AADoM task based on the self-supervised and the deep neural network based
approach. To analyze feature extraction, we conducted extensive experiments to
assess the influence of various factors of pseudo audios, audio segments, data
augmentation and frequency bands. The experimental results on the benchmark
dataset of DCASE 2022 Task 2 Development set prove that focusing on the narrow
frequency bands significantly facilitates AADoM task.

##  II The Proposed Baseline System

We first propose a baseline system for the task of Acoustic Anomaly Detection
of Machines (AADoM), referred to as the ProBaseline. As Fig. 1 shows, the
ProBaseline comprises three main components: feature extraction,
classification model, Gamma distribution.

Figure 1: The proposed baseline system for AADoM

Feature extraction: The raw audio recordings from both ‘train’ and ‘test’
subsets are first transformed into STFT spectrograms with the window size and
the hop size set to 2048 and 1024, respectively. Then, we apply 128 Mel
filters on STFT spectrograms to generate the Mel spectrograms.

Classification Model: Given the Mel-spectrograms represented the audio input,
we establish the classification task in which Mel-spectrograms are classified
into certain classes based on different operating conditions of the machines.
In this paper, we proposed a Mobile-FaceNet based architecture for the
classification task which is presented in Table I. Our proposed Mobile-FaceNet
network is inspired by [12] which presents an inverted residual structure with
linear bottleneck. By leveraging this architecture, we add some layers such as
LinearGDConv2d, LinearConv2d and Dense to create our deep learning model.
Notably, only audio data from the ‘train’ subset is used for training the
proposed Mobile-FaceNet network.

Gamma Distribution: After the training process, we achieve the pre-trained
Mobile-FaceNet network. We then feed the Mel-spectrograms from both the
‘train’ and ‘test’ subsets into the pre-trained network to extract the ‘train’
audio embeddings and ‘test’ audio embeddings, respectively. The ‘train’ and
‘test’ audio embeddings are the output of the Softmax layer in the pre-trained
Mobile-FaceNet network. Given the ‘train’ audio embeddings, we apply the
Euclidean distance measurement [13]to compute the mean of the ‘train’ audio
embeddings. We then establish the Gamma distribution in which the difference
between one ‘train’ audio embedding and the mean of the ‘train’ audio
embeddings is considered as one variable of the distribution, which is
computed by:

$d_{e}=||\mathbf{x}-\mathbf{m}||^{2}_{2}$ (1)

where $||.||$ denotes Euclidean norm, $\mathbf{x}$ presents one ‘train’ audio
embedding, and $\mathbf{m}$ presents the mean of the ‘train’ audio embeddings.

For the evaluation process on the ‘test’ subset, the difference between one
‘test’ audio embedding and the mean of the ‘train’ audio embeddings is first
computed. Then, the difference value is compared with the given Gamma
distribution with a certain threshold (e.g. 0.9) to decide whether the ‘test’
audio embedding is normal or abnormal.

TABLE I: The proposed Mobile-FaceNet based network architecture for the classification Operations | t | c | n | s
---|---|---|---|---
Conv2d [3x3] | - | 64 | - | 2
Conv2d [3x3] | - | 64 | - | 1
Bottleneck | 2 | 128 | 2 | 2
Bottleneck | 4 | 128 | 2 | 2
Bottleneck | 4 | 128 | 2 | 2
Conv2d [1x1] | - | 512 | - | 1
LinearGDConv2d | - | 512 | - | 1
LinearConv2d [1x1] | - | 512 | - | 1
GlobalAvgPooling2d | - | - | - | -
Dense (activation=’relu’) | - | 1024 | - | -
Dropout (0.3) | - | - | - | -
Dense (activation=’softmax’) | - | C | - | -

##  III Further Improve the Proposed Baseline

As the proposed baseline system (ProBaseline) is recently described, we assume
that there are three main factors affecting the performance: the feature
extraction to generate spectrogram input, the network architecture to extract
audio embeddings, the distance measurement method to generate the Gamma
distribution. In this paper, we focus on improving the quality of spectrogram
input and the Gamma distribution rather than the network architecture. We then
propose our improvement methods which are described in below sections.

###  III-A Generate Pseudo Audio Samples

We are inspired that if pseudo audio recordings can be generated and grouped
into one class, the proposed Mobile-FaceNet based architecture is enforced to
learn audio features and generate diverse audio embeddings. More diverse audio
embeddings have the potential to present a well-performed Gamma distribution
that leads to improve the performance. In this paper, we therefore apply the
pitch-shifting technique to synthesize pseudo audio recordings.

###  III-B Working on Audio Segment Level

Instead of using the entire 10-second audio recording to generate one
spectrogram and then extract one audio embedding, we split the entire audio
recording into many audio segments with the length of 2.5 seconds per segment
with an overlap of 50%. By approaching the audio segments, we achieve a number
of audio embeddings that enrich the Gamma distribution and then potentially
further improve the AADoM performance.

###  III-C Apply Online Data Augmentations

To enhance the Mobile-FaceNet network performance for the classification task
and then generate well-performed audio embeddings, we apply two data
augmentation methods, referred to as Specaugment [14], and Mixup [15]. First,
ten random and continuous temporal and frequency bins of the Mel-spectrograms
are erased (Specaugment). Then, the spectrograms are randomly mixed together
using different coefficients from both Beta and Uniform distributions (Mixup).
As these two data augmentation methods are applied on each batch of Mel-
spectrograms in the training process, we refer them to as the online data
augmentations.

Figure 2: Performance comparison (AUC) on ‘source’ test domain among the
proposed baseline (ProBaselline) and the proposed baseline with a certain
improvement (ProBaseline-Mah, ProBaseline-Mix, ProBaseline-Pse, ProBaseline-
Seg, and ProBaseline-Imp)

###  III-D Evaluate Narrow Frequency Bands

We are inspired that certain faults in machine sound are located at certain
frequency bands rather than spreading all the frequency bins. Therefore, if we
can pinpoint and evaluate certain frequency bands indicating the location of
faults in machine sounds, the AADoM system is then designed to focus on these
certain frequency bands for further enhancing the performance. In this paper,
we conduct an extensive experiment in which a wide range of frequency bands of
0 kHz to 3 kHz, 0.5 kHz to 3.5 kHz, 1 kHz to 4 kHz, 1.5 kHz to 4.5 kHz, 2 kHz
to 5 kHz, 2.5 kHz to 5.5 kHz, 3 kHz to 6 kHz, 3.5 kHz to 6.5 kHz, 4 kHz to 7
kHz are investigated. While evaluating these specific and narrow frequency
bands, the other frequency bands on STFT spectrograms are removed before
transforming into Mel-spectrograms.

###  III-E Apply Mahalanobis Distance

As the Euclidean distance [13] used to measure the distance among audio
embeddings is limited to present the correlation of the single variables in
the audio embeddings (i.e. an audio embedding mathematically presents a vector
with multiple variables) Therefore, approaching the Euclidean can be distorted
by outliers or skewed distributions as it treats variables of the audio
embeddings independent and equal importance. To tackle this limitation, we
replace the Euclidean distance with the Mahalanobis distance measurement [16].
As the Mahalanobis method computes the mean of a multivariate distribution
based on the covariance matrix of the distribution, it takes account of the
scale, the correlation, and the shape of the variables.

The Mahalanobis distance between an audio embedding and a distribution of
audio embeddings is calculated by the equation:

$d_{m}=\sqrt{(\mathbf{x}-\mathbf{m})^{T}\mathbf{S}^{-1}(\mathbf{x}-\mathbf{m})}$
(2)

where $\mathbf{x}$ is one audio embedding, $\mathbf{m}$ is the mean of the all
audio embeddings, $\mathbf{S}$ is the covariance matrix of all audio
embeddings.

Figure 3: Performance comparison (AUC) on ‘target’ test domain among the
proposed baseline (ProBaselline) and the proposed baseline with a certain
improvement (ProBaseline-Mah, ProBaseline-Mix, ProBaseline-Pse, ProBaseline-
Seg, and ProBaseline-Imp)

###  III-F Evaluate Different Thresholds of Gamma Distribution

As the threshold value set to Gamma distribution may affect the performance of
AADoM system, we evaluate a wide range of threshold values from 0.85 to 0.95
with the step of 0.1.

##  IV Experiment And Results

###  IV-A Datasets

In this paper, we evaluate our proposed models on the benchmark dataset:
Development set of DCASE 2022 Challenge Task2. This dataset presents 10-second
audio recordings which were collected from 7 machine types: ‘Bearing’, ‘Fan’,
‘Gearbox’, ‘Slider’, ‘ToyCar’, ‘ToyTrain’ and ‘Valve’. For each machine, there
are two data subsets of ‘train’ and ‘test’ for training and testing processes,
respectively. In the ‘train’ subset, audio recordings are separated into three
sections, referred to as ‘section 0’, ‘section 1’ and ‘section 2’. Three
sections present three different operating conditions of the machine. For each
section, there are 1000 audio recordings which are separated into 990 ‘source’
audio recordings and 10 ‘target’ audio recordings. The imbalanced number
between ‘source’ and ‘target’ data presents the issue of domain shift in this
challenge. As regards the ‘test’ subset, it comprises 200 audio recordings
which are separated into 100 ‘source’ audio recordings and 100 ‘target’ audio
recordings, referred to as the ‘source’ test domain and ‘target’ test domain.

###  IV-B Evaluation Metric

In this paper, we obey the DCASE 2022 Task 2 challenge [17, 18, 19], use AUC
and pAUC as the evaluation metrics for the task of Anomalous Sound Detection
on Machines.

Figure 4: The effect of frequency bands on ‘source’ test domain Figure 5: The
effect of frequency bands on ‘target’ test domain

###  IV-C Experimental Settings

We construct our proposed deep neural networks with the TensorFlow framework.
We train the proposed deep neural networks for 30 epochs. All deep neural
networks in this paper are trained with the Titan RTX 24GB GPU. We use the
Adam method [20] for the optimization. The learning rate is set to 0.0001.

###  IV-D Experimental Results and Discussion

We first compare the proposed baseline (ProBaseline) to the proposed baseline
with a certain improvement: the ProBaseline with Mahalanobis distance
(ProBaseline-Mah), the ProBaseline with Mixup data augmentation (ProBaseline-
Mix), the ProBaseline using audio segment (ProBaseline-Seg), the ProBaseline
using pseudo samples (ProBaseline-Pse), and the ProBaseline using all
improvements (ProBaseline-Imp).

Figure 6: The effect of Gamma threshold on ‘source’ test domain Figure 7: The
effect of Gamma threshold on ‘target’ test domain

As Fig. 2 and Fig. 3 show, all proposed improvement methods help to enhance
the proposed baseline (ProBaseline) on all machines and for both ‘source’ and
‘target’ test generally. Among improvement methods, using segments of
spectrogram instead of an entire spectrogram significantly helps to improve
the AUC scores (63,5%/62.74% from ProBaseline-Seg compared to 56.04%/55.74%
from ProBasline in ‘source’/‘test’ domain). This proves our motivation that
using audio segments helps to generate more audio embeddings and create a
well-performed Gamma distribution, then improve the AADoM system performance.
Given the proposed baseline using all improvement methods (ProBaseline-Imp),
we evaluate whether narrow frequency bands affect the system performance. We
referred these models to as ProBaseline-Imp-Freq. Notably, the Gamma
distribution threshold is set to 0.9 to evaluate the specific frequency bands.
As Fig. 4 and Fig. 5 show, the frequency bands significantly affect the
performance regarding the type of machine. In particular, while ‘Fan’ and
‘Valve’ machines present the best AUC of 73.2%/61% and 75.7%/67.43% on the
‘source’/‘target’ domain from the low bands of 0.5 kHz to 3.5 kHz and 1 kHz to
4 kHz, the ‘Slider’ and ‘ToyCar’ achieve the best AUC at the higher bands of 2
kHz to 5 kHz and 2.5 kHz to 5.5 kHz, respectively. Both ‘Bearing’ and
‘Gearbox’ show the best AUC scores at the same middle bands of 1.5 kHz to 4.5
kHz. Regarding the ’ToyTrain’ machine, the frequency bands do not
significantly affect the performance. Notably, from 3 kHz to above, the
performance significantly drops regarding all machine types. The experimental
results indicate that faults occurring on different machine types are located
at certain and narrow frequency bands. Therefore, if we focus on exploring
frequency bands on which the corresponding faults of each machine locate, the
AADoM system performance has the potential to be improved significantly.

Given the significant effect of frequency bands on the AADoM performance, we
evaluate the role of Gamma distribution threshold over each frequency band. In
particular, we evaluate different threshold values from 0.85 to 0.95 with the
step of 0.1. As the Fig. 6 and Fig. 7 show, only ‘Bearing’ and ‘Gearbox’
machines are affected by the threshold at the frequency bands of 2 kHz - 5
kHz. It can be concluded that the threshold does not significantly affect the
AADoM system performance.

TABLE II: Performance comparison (AUC/pAUC) on ‘test’ set among DCASE
baseline, the proposed baseline (ProBaseline), the proposed baseline with all
improvement methods (ProBaseline-Imp), and the proposed baseline with all
improvement methods with focusing on frequency bands (ProBaseline-Imp-Freq)

Machine | DCASE | ProBaseline | ProBaseline | ProBaseline
---|---|---|---|---
| Baseline [17] |  | -Imp | -Imp-Freq
Bearing | 60.2/57.1 | 59.2/51.3 | 60.1/54.4 | 63.1/56.9
Fan | 59.4/56.8 | 48.8/53.3 | 67.0/62.7 | 74.3/61.7
Gearbox | 62.7/56.0 | 61.2/56.6 | 70.8/62.4 | 75.5/64.6
Slider | 51.6/54.6 | 58.0/54.9 | 73.0/63.1 | 86.1/74.5
ToyCar | 55.5/52.2 | 52.9/51.1 | 59.2/54.2 | 65.6/56.1
ToyTrain | 51.5/51.5 | 47.2/50.1 | 53.1/52.4 | 55.7/52.5
Valve | 62.1/62.4 | 65.0/57.0 | 61.5/60.2 | 71.5/62.9

As the improvement methods and the focus on certain frequency bands help to
improve the performance, we fine-tune the ProBaseline-Imp-Freq models by
training the Mobile-FaceNet networks for more than 60 epochs. For each 20
epochs, we save the model and then extract audio embeddings. The fine-tuning
process helps to create a large number of audio embeddings that leads to
achieve a well-performed Gamma distribution. Our best results are then
compared with the DCASE baseline in the Table II. As the Table II shows, our
fine-tuning ProBaseline-Imp-Freq model outperforms the DCASE baseline over all
machines. Among the machines, ‘ToyTrain’ shows the worst performance AUC of
55.7%. Meanwhile, ‘Fan’, ‘Gearbox’, ‘Slide’, and ‘Valve’ present the potential
AUC scores of 74.3%, 75.5%, 86.1%, 71.5%, respectively.

##  V Conclusion

This paper has presented a deep learning based system for the Acoustic Anomaly
Detection of Machines (AADoM). By combining multiple techniques with a focus
on feature extraction (pseudo audios, audio segment, data augmentation,
frequency bands focusing), we successfully achieved an AADoM system that
outperforms the DCASE baseline on the benchmark dataset of DCASE 2022 Task 2
Development set. Our experimental results also indicate the significantly
important role of narrow frequency bands in further improving the AADoM system
performance.

## References

  * [1] Jun’ya Yamashita, Hayato Mori, Satoshi Tamura, and Satoru Hayamizu,  “Vae-based anomaly detection with domain adaptation,”  Tech. Rep., DCASE Challenge, 2021.
  * [2] Jan Tozicka, Bezusek Marek, Durkota Karel, and Linda Michal,  “Dadaed-double anomaly detector with aediff,”  Tech. Rep., DCASE Challenge, 2022.
  * [3] Shuxian Wang et al,  “Ensemble of multiple anomaly detectors under domain generalization conditions,”  Tech. Rep., DCASE Challenge, 2022.
  * [4] Ibuki Kuroyanagi, Tomoki Hayashi, Kazuya Takeda, and Tomoki Toda,  “Two-stage anomalous sound detection systems using domain generalization and specialization techniques,”  Tech. Rep., DCASE Challenge, 2022.
  * [5] Kevin Wilkinghoff,  “An outlier exposed anomalous sound detection system for domain generalization in machine condition monitoring,”  Tech. Rep., DCASE Challenge, 2022.
  * [6] Sergey Verbitskiy, Milana Shkhanukova, and Viacheslav Vyshegorodtsev,  “Unsupervised anomalous sound detection using multiple time-frequency representations,”  Tech. Rep., DCASE Challenge, 2022.
  * [7] Ismail Nejjar, Jean Meunier-Pion, Gaetan Frusque, Olga Fink, and Gif-sur-Yvette CentraleSupélec,  “Dcase challenge 2022: Self-supervised learning pre-training, training for unsupervised anomalous sound detection,”  Tech. Rep., DCASE Challenge, 2022.
  * [8] Ying Zeng, Hongqing Liu, Lihua Xu, Yi Zhou, and Lu Gan,  “Robust anomaly sound detection framework for machine condition monitoring,”  Tech. Rep., DCASE Challenge, 2022.
  * [9] Kazuki Morita, Tomohiko Yano, and Khai Tran,  “Comparative experiments on spectrogram representation for anomalous sound detection,”  Tech. Rep., DCASE Challenge, 2022.
  * [10] Wenbo Liu, Delong Cui, Zhiping Peng, and Jihai Zhong,  “Outlier detection algorithm based on gaussian mixture model,”  in Proc. ICPICS, 2019, pp. 488–492.
  * [11] Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander,  “Lof: identifying density-based local outliers,”  in Proc. ACM SIGMOD, 2000, pp. 93–104.
  * [12] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen,  “Mobilenetv2: Inverted residuals and linear bottlenecks,”  in Proc. CVPR, 2018, pp. 4510–4520.
  * [13] Ivan Dokmanic, Reza Parhizkar, Juri Ranieri, and Martin Vetterli,  “Euclidean distance matrices: essential theory, algorithms, and applications,”  IEEE Signal Processing Magazine, vol. 32, no. 6, pp. 12–30, 2015.
  * [14] Shehu Mohammed Yusuf, EA Adedokun, MB Mu’azu, IJ Umoh, and Ahmed Abdul Ibrahim,  “A novel multi-window spectrogram augmentation approach for speech emotion recognition using deep learning,”  in Proc. ICMEAS, 2021, pp. 1–6.
  * [15] Ning Jia and Chunjun Zheng,  “Emotion speech synthesis method based on multi-channel time–frequency domain generative adversarial networks (mc-tfd gans) and mixup,”  Arabian Journal for Science and Engineering, vol. 47, no. 2, pp. 1749–1762, 2022.
  * [16] Jae Phil Chung and Hong Hee Yoo,  “Blade fault diagnosis using mahalanobis distance,”  Journal of Mechanical Science and Technology, vol. 35, pp. 1377–1385, 2021.
  * [17] Kota Dohi et al,  “Description and discussion on DCASE 2022 challenge task 2: Unsupervised anomalous sound detection for machine condition monitoring applying domain generalization techniques,”  in Proc. DCASE, 2022, pp. 1–5.
  * [18] Kota Dohi et al,  “MIMII DG: Sound dataset for malfunctioning industrial machine investigation and inspection for domain generalization task,”  in Proc. DCASE, 2022, pp. 1–5.
  * [19] Noboru Harada et al,  “ToyADMOS2: Another dataset of miniature-machine operating sounds for anomalous sound detection under domain shift conditions,”  in Proc. DCASE, 2021, pp. 1–5.
  * [20] P. K. Diederik and B. Jimmy,  “Adam: A method for stochastic optimization,”  CoRR, vol. abs/1412.6980, 2015.