# Investigating shocking events in the Ethereum stablecoin ecosystem through
temporal multilayer graph structure ††thanks: Citation: Authors. Title.
Pages…. DOI:000000/11111.

Cheick Tidiane Ba, Richard G. Clegg
Queen Mary University London
London, United Kingdom
{c.ba<EMAIL_ADDRESS>
Benjamin A. Steer
Pometry Ltd
London, United Kingdom
<EMAIL_ADDRESS>
Matteo Zignani
University of Milan
Milan, Italy
<EMAIL_ADDRESS>

###### Abstract

In the dynamic landscape of the Web, we are witnessing the emergence of the
Web3 paradigm, which dictates that platforms should rely on blockchain
technology and cryptocurrencies to sustain themselves and their profitability.
Web3 has garnered significant attention as the currencies gained considerable
monetary value. Cryptocurrencies are characterised by high market volatility
and susceptibility to substantial crashes, issues that require temporal
analysis methodologies able to tackle the high temporal resolution,
heterogeneity and scale of blockchain data. While existing research attempts
to analyse crash events, fundamental questions persist regarding the optimal
time scale for analysis, differentiation between long-term and short-term
trends, and the identification and characterisation of shock events within
these decentralised systems.

This paper addresses these issues by examining cryptocurrencies traded on the
Ethereum blockchain, with a spotlight on the crash of the stablecoin TerraUSD
(UST) and the currency LUNA designed to stabilise it. Utilising complex
network analysis and a multi-layer temporal graph allows the study of the
correlations between the layers representing the currencies and system
evolution across diverse time scales. The investigation sheds light on the
strong interconnections among stablecoins pre-crash and the significant post-
crash transformations. We identify anomalous signals before, during, and after
the collapse, emphasising their impact on graph structure metrics and user
movement across layers.

This paper pioneers temporal, cross-chain graph analysis to explore a
cryptocurrency collapse. It emphasises the importance of temporal analysis for
studies on web-derived data. In addition, the methodology shows how graph-
based analysis can enhance traditional econometric results. Overall, this
research carries implications beyond its field, for example for regulatory
agencies aiming to safeguard users from shocks and monitor investment risks
for citizens and clients.

_K_ eywords transaction network $\cdot$ temporal network $\cdot$
cryptocurrency $\cdot$ Web3 $\cdot$ multilayer graph

##  1 Introduction

Analysing multi-layer, temporal networks at scale is an important problem in
the study of complex networks. In particular, there are issues of
understanding the best time-scale to analyse, how to determine long-term and
short-term trends and how to recognise and characterise shock events in such a
system. In this paper, we look at Web3 and specifically into six
cryptocurrencies traded over the Ethereum blockchain. We use multi-layer,
temporal network analysis techniques enabled by the Raphtory software
developed at Queen Mary University of London and Pometry Ltd. The software
enables simultaneously looking at how temporal networks evolve in time and at
different time-scales while considering several network layers together.

Digital currencies remain an important but potentially volatile market. In
particular, there is the possibility that a particular currency will
completely crash, wiping the value of that coin to a tiny fraction of its
previous trade value: these events can destroy investments and savings,
impacting the lives of investors. It is, therefore, important to analyse the
events that lead up to a crash and to look at whether the effects of a crash
can destabilise other currencies. “Stablecoins" are an attempt to make
cryptocurrencies that can be used for trading rather than speculation by
“pegging" the value of the coin to a fixed exchange rate (often one dollar to
one stablecoin). The stablecoin TerraUSD (UST)111Throughout this paper we will
refer to currency by the shortened name it was commonly traded under. was
pegged to trade for $1 and was associated with a companion currency LUNA that
was intended to stabilise it. Over a few days in May 2022 both UST and LUNA
had collapsed and never regained their value. The collapse is believed to have
been precipitated by deliberate and coordinated large-scale sales of UST in an
attempt to crash UST, drive down the price of Bitcoin (BTC) and derive profit
from this.

In this paper, we use techniques from complex network analysis and, in
particular, temporal multi-layer analysis to look at the events leading up to
and immediately following the collapse of LUNA and UST. UST and LUNA were
traded on a cryptocurrency platform known as Terra and this data is not
available to us. However, the two currencies were also traded in a “wrapped"
form on the Ethereum platform, USTC (US Terra classic) and WLUNC (wrapped Luna
classic) respectively. From the available set of Ethereum trades, we look at
USTC and WLUNC trades and compare them with the other most highly traded
stablecoins on the platform. This gives us a pool of six currencies trading
over the same platform that can be compared as they evolve in the weeks before
and after the collapse, hence the analysis considers 26,209,860 transactions
occurring in a symmetrical analysis period around the day of the crash.

To our knowledge, this is the first paper to use temporal, cross-chain, graph
analysis to look at the collapse of a cryptocurrency. The analysis in this
paper can only be achieved with multi-layer temporal graph systems as it is
necessary to look at correlations between different graph layers and to
consider the evolution of the system at different time scales. We show that
the six currencies are initially highly correlated in terms of the number of
sales but, post-crash, separate into two groups where the “failed" currencies
correlate with each other but not the other four. We show that two high-volume
selling events on UST had a significant effect on the structure of sales
volumes before the crash (one event exactly coinciding with the crash)
effectively decoupling its sales volume from all other currencies. We also
characterise a significant “rally" event that temporarily caused large changes
to the UST and LUNA ecosystems shortly after the collapse. The collapse event
has significant effects on graph metrics such as graph density and average
degree but also effects on measures like clustering coefficient and weakly
connected components.

##  2 Background

Stablecoins are digital currencies that attempt to peg their value to some
other reference asset, for example the US dollar or a fixed amount of gold
[1]. This is designed to reduce the volatility of the currency as the high
volatility of some currencies can put users off adopting them for use as
currency (as opposed to speculation). Many different stablecoins exist traded
over several different blockchain systems. Digital currencies can be traded
between different blockchains by a process known as “wrapping". In this
process, a currency token originating on chain A is held in place and a
wrapped coin representing the same token is minted on chain B [2]. This
allows, for example, BTC to be spent on the Ethereum chain as Wrapped Bitcoin
(WBTC).

Acronym | Short description | Comment |
---|---|---|---
LUNA | Luna | Cryptocurrency, unwrapped version, not in our dataset. |
UST | Terra USD | Stablecoin, unwrapped version, not in our dataset. |
BTC | Bitcoin | Cryptocurrency, not part of this study. |
WLUNC | Wrapped Luna Classic | Wrapped cryptocurrency |
USTC | Terra USD Classic | Wrapped stablecoin |
USDT | Tether | Stablecoin |
USDC | USD Coin | Stablecoin |
DAI | Dai | Stablecoin |
USDP | Pax Dollar | Stablecoin |
Table 1: Currencies under study, the acronym is the label that the currency
uses for trading.

In this paper, we wish to study the ecosystem around the currencies Terra and
Luna which ran on their own blockchain known as Terra. TerraUSD(UST) was a
stablecoin pegged at one US dollar. Luna (LUNA) was a cryptocurrency designed
to ensure the stability of the Terra stablecoin. This was achieved by an
algorithmic process known as “Anchor", that automatically would swap Terra
with Luna currency to adjust their prices: in fact, the protocol kept the UST
exchange rate stable by varying the pool of Terra available [3], generating or
destroying tokens when needed. The Terra blockchain data is unavailable to us,
however, we can study wrapped versions of UST and LUNA traded on the Ethereum
blockchain. We provided a summary of the currencies under study in Table 1.

The interest in studying the Terra/Luna ecosystem comes from the sudden
complete crash of UST in May 2022. A timeline extracted from [3] and [4] is as
follows:

  * •

3rd April 2022 (S1): Anomalous UST selling event discovered during this
analysis. We denote this event as sale 1 (S1).

  * •

19th April 2022 (S2): Anomalous UST selling event discovered during this
analysis. We denote this event as sale 2 (S2).

  * •

20th April 2022: UST becomes the third largest stablecoin.

  * •

5th May 2022: The start of strong and persistent selling pressure is evident
on BTC and LUNA.

  * •

7th May 2022: Agreement is made to buy a large number of UST for BTC greatly
reducing the liquidity in the pool. Simultaneously a large number of UST
tokens are put up for sale (which caused the first UST de-pegging, below $0.99
[3]).

  * •

8th May 2022: UST loses its peg against the dollar falling to $0.99 and the
Luna Foundation group uses reserve funds in BTC to prop up the currency.

  * •

9th May 2022 (C): UST depegs again falling in value to $0.35 leading to
customers trying to sell their reserves to exit the market. We denote this
event as a crash (C).

  * •

10th May 2022: Luna Foundation Guard sells reserve Bitcoin (BTC) to attempt to
restore the peg.

  * •

12th May 2022: LUNA has collapsed by almost 99%.

  * •

13th May 2022: Terra temporarily halts its Blockchain and Binance suspends
LUNA trading.

  * •

25th May 2022: Proposal [5, 6, 7] for the launch of a new blockchain Terra 2.0
is official.

  * •

27th May 2022 (T2): Terra 2.0 is live. Original LUNA tokens became Luna
Classic (LUNC). The original UST stablecoin became TerraClassicUSD (USTC).
Unlike the original Terra network, Terra 2.0 has no algorithmic stablecoin
[8]. We denote this event as Terra 2.0 (T2).

The final consequence was not only the collapse of the Terra platform, UST and
LUNA but also a decrease in the price of BTC. The motivation speculated for
the attack is that the attackers were short-selling large amounts of BTC.
Short selling is, in effect, a bet that a traded asset will decrease in price.
The short seller contracts to borrow and sell the asset now and buy it at a
later date at market price. If the price drops the seller, has profited by the
amount of the price drop. In some markets, short selling is highly regulated
as it has been implicated in market crashes [9]. In our case study, there is a
belief that high levels of selling of Bitcoin observed in the days prior to
the crisis were linked to short selling [3].

#### Related works

The Luna crash serves as an interesting case study due to its relevance within
the stablecoins ecosystem, attracting attention from both industry and the
research community. There are many works describing the ecosystem and the
properties of the Terra system. For instance, Uhlig [10] proposes a
theoretical framework for the analysis of hourly data on prices and market
capitalisation data, delineating the gradual unfolding of the crash. Limited
research leverages on-chain data. Cho [11] investigates the underlying causes
of the de-pegging event focusing on the token economics of the Terra
Blockchain. They utilise on-chain data focusing on transactions converting
USTC to LUNC over the four days following the May 9 crash, identifying
critical design flaws in the conversion process exploited by attackers. These
works focus mainly on Terra. However, various studies explore the impact of
the crash on other currencies. De Blasis et al. [12] examine the impact of the
USTC collapse on other stablecoins by using proprietary minute-by-minute price
transaction data of various cryptocurrencies, over a symmetrical 40-day period
around the USTC crash of May 9th. Their work shows the presence of significant
contagion from the USTC collapse, which could be explained by traders’ herding
behaviour i.e. users reacting to the crash by moving to other currencies.
Briola et al. [3] provide a background on the Terra project and describe the
main events that led to the crash, before delving into an analysis focused on
the hourly price data time series. They study the evolution of dependencies
between 61 cryptocurrencies, by leveraging the price time series to define a
network structure among currencies, a correlation network: their analysis
seems to exclude the presence of herding behaviour. The importance of
analyzing cross-chain behaviour is denoted by other works as well. Barthere et
al. [13] analyses proprietary data from Terra and Ethereum to identify the
main users involved in the crash. They assert that a small subset of users
initiated the crash by leveraging system vulnerabilities, benefiting
significantly by converting USTC to other coins before the crash. Liu et al.
[14] employ a similar dataset and approach, leveraging proprietary transaction
data from Terra and Ethereum. Their extensive analysis across multiple chains
and assets reveals the complex nature of the run on Terra. At the same time,
they believe the crash was exacerbated by concerns about system sustainability
and not just a concentrated market manipulation. Although they have user
transaction data, in their study, the authors do not focus on the transaction
networks but focus more on describing the events and identifying the main
actors.

A key limitation in these works is while they provide important results on the
crash, they neglect the analysis of the transactions among users from a
network standpoint: indeed, the circulation of tokens within the ecosystem can
be effectively represented as networks or graphs [15]. In this representation,
nodes (or vertices) symbolise users, while links (or edges) denote the dynamic
interactions or relationships among them. This modelling and analysis approach
for currency systems through networks has significantly enhanced our
understanding of financial transaction networks [16, 17]. Moreover, temporal
networks [18] have proven to be a valuable framework for comprehending web-
based currency systems such as Bitcoin [19, 20, 21, 22] and Ethereum [23, 24,
25, 26], as well as in the context of new-generation social media platforms
[27, 28, 29]. However, none of these works analyses the stablecoin ecosystem
from a network standpoint. Recent works analyzing the Chartalist dataset by
Shamsi et al. [4] highlighted the importance of network structure in the
stablecoin system. For instance, Huang et al. [30] focus on change detection
on dynamic graphs, proposing a spectral method to create dynamic graph
embeddings and an associated anomaly score can be used to detect significant
changes. The main result is that they detected anomalies in the graph
embeddings three days before the crash; however, the work used a unique graph,
without distinguishing the stablecoins, hence the detected anomalies before it
may not necessarily be related to the Luna Terra system or the crash.
Similarly, Zhu et al. [31], focuses on the task of trend detection in
blockchain-based networks, with a method focused on core decomposition to
detect a change in the activity in the temporal transaction network. Among the
selected datasets for evaluation, they rely on a single dataset: the authors
can detect a perturbation in network structure caused by the crash, but they
too model the entire transaction data as a homogeneous temporal network.

While network structure seems an important factor to analyse, none of the
current works analyse the transaction network in relation to the crisis; and
those who leverage network structure failed to properly separate the different
currencies, treating the transactions across different currencies as the same
object of the study.

##  3 Research motivation

The cryptocurrency and stablecoin ecosystem here consists of several heavily
interlinked currencies that can be studied together both before and after the
shock. This paper studies the effects of the shock on the currencies most
directly affected but also on those that are related. This is consistent with
the suspicion that the attack on Terra/Luna was intended as an attack on the
price of BTC. The paper answers the following research questions:

#### What is the nature of the cryptocurrency ecosystem before and after the
attack

Can we see commonalities in the signals between the different currencies? Are
users investing in several currencies or specialising in one? We analyse the
cross-correlation of various measures (eg number of trades and number of tokes
traded) between currencies to ask whether similar patterns of selling and
buying were occurring over time within the system.

#### Can we anticipate the crash by looking at trade volumes of
cryptocurrencies in the days leading up to it? What are the effects of the
crash on trade volume?

Are the events preceding it unusual in scale or characteristics? Are they
isolated to just one currency? We look at the volume and number of trades in
each currency over the immediate time period before the attack to find those
unusual “whale" events that precede the crash and the aftershocks from the
event.

#### What are the effects of the attack on the system on the most directly
affected currencies?

We look at the size and number of trades after the crash event to consider the
effects of the crash focusing on the most affected currencies and considering
different time windows.

#### Can the after effects of the attack be seen on graph structure as a
whole?

We look at traditional graph measures such as reciprocity, clustering
coefficients, weakly connected components, mean degree and graph density on
each currency treated as an individual graph. We ask whether there are short
and longer term trends visible in these graph measures as a result of the
crash.

#### Where did user interest move post-crash?

We look at how users changed their investment strategies after the crash
event. Did users of currencies not directly affected change their behaviour?
Where did users in the most directly affected currencies move their
investments?

#### Can we locate individuals associated with the crash?

We will deepen the analysis of user activity on days characterised by
anomalous user activity. Are the anomalies detected caused by a few
individuals or the result of a widespread change of behaviour? Are there
common actors active during the anomalous events?

##  4 Methodology

In this section we give a basic description of our dataset and the methodology
we will use for analysis. It should be remembered throughout this and the next
section that the analysis of USTC and WLUNC are looking only at those
transactions that can be seen on the Ethereum blockchain. The number of
transactions on the Terra blockchain (that we cannot observe) where Terra and
Luna are traded natively are likely to be far larger in volume.

###  4.1 Dataset

The dataset we rely on is the stablecoins dataset presented in Chartalist
datasets by Shamsi et al. [4]. It covers the activity of users of the top
stablecoins traded on Ethereum ordered by market capitalisation at the time of
the crash, alongside USTC and WLUNC. The dataset consists of 70,678,297
transactions, occurring between 1st April 2022 and 1st October 2022, covering
a month prior to the crash and six months after the crash. We are focused on
specific details, namely the sender ( from_address) and the receiver
(to_address) of the currency, the transfer timestamp (time_stamp), the token
amount sent (value), and the smart contract address (contract_address) that
identifies the currency traded. The currencies under examination are
associated with the Terra Ecosystem: USTC and WLUNC222In the dataset, WLUNC is
denoted as LUNC, but the contract address points to Wrapped LUNA
https://etherscan.io/token/0xd2877702675e6ceb975b4a1dff9fb7baf4c91ea9,
currently traded as WLUNC . Additionally, the dataset encompasses user
transactions involving other stablecoins, including Tether (USDT), a
collateral-backed cryptocurrency stablecoin [32]; Dai, a USD-pegged stablecoin
maintained through smart contracts, governed by MakerDAO [33]; Pax Dollar
(USDP), a stablecoin collateralized 1:1 by USD in Paxos-owned US bank accounts
and approved by Wall Street regulators [34]; and USD Coin (USDC), managed by
the Centre consortium, backed by a dollars in reserve or other undisclosed
"approved investments" [35]. We provided a summary of the currencies in Table
1.

We also integrate the available data with currency daily prices. We recover
the price data from the web platform CoinCodex [36]. The obtained data is
presented in Figure 1. From Figure 1 we can see how Luna was extremely
valuable with a peak value surpassing $120 a token. Figure 1 presents an
alternative version of the plot that gives us a better view of the
stablecoins. The plot is normalized by the starting value observed in the
period, highlighting the fall of USTC while other stablecoins largely exhibit
their designed behaviour, holding a value very close to one dollar.

(a) Close values log

(b) Close values norm starting value

Figure 1: Daily price of the currencies under study, in the periods covered by
the full dataset. We represent the closing prices on April 1st and onward,
normalising the values of each by dividing by the starting value in the
observed period. In the insets, we provide a zoomed version around the value
of 1 USD.

###  4.2 Analysis methods

The data is analysed as a directed, multilayer temporal graph. A node
represents an Ethereum wallet. A directed edge represents a transaction
between two wallets, the weight represents the amount of currency, the
direction represents the direction money is transferred and the layer
represents the individual currency. Using the Raphtory system [37] we can
easily perform analysis using sliding windows to consider the trades within
different parts of the data and investigate how the temporal graph behaves
within a time period333Code will be made publicly available in case of
acceptance.. If we define a time window $T=\left[t_{1},t_{2}\right)$ then that
window will include trades made at all times $t\leq t_{1}$ and
$t<t_{2}$.444The closed/open interval means we do not double count trades that
occur exactly on a border if, for example, we move a window forward every hour
we do not want trades occurring exactly on the hour to be in two different
windows. The degree of a node within a time window is a measure of the number
of trading partners that the wallet has within that time window. The balance
of the weights of inbound minus outbound link weights is the node’s income
during the period.

Figure 2: Illustration of the construction of the temporal multilayer network.
Given a set of transactions, we can model them as a network, where each layer
represents one of the currencies under study.

For analysis purposes, we want to separate the data into a pre-crash and post-
crash period to investigate the normal behaviour of the market and to see if
it returns to the same situation after the crash occurs. Obviously, it is
somewhat arbitrary as to when such an event ends. For the purposes of this
paper, we have chosen the 9th May 2022 as the centre of the crash event and we
exclude that day and a week on either side. This leads to a fifteen-day
exclusion period with the pre-crash period including all data up to the end of
1st May and the post-crash period including all data from the beginning of the
17th May. This gives us thirty days of complete data before the exclusion zone
and by considering all data up to the end of 15th June we have thirty days
after the exclusion zone. The exclusion zone is plotted on all graphs as a
light red area with $C$ marking the centre. Graphs are also marked with $S1$
and $S2$ for the two major selling events on 3rd April 2022 and 19th April
2022 that preceded the crash (see the timeline in Section 2) and with $T2$ for
the launch of Terra 2.0 blockchain on 27th May 2022.

###  4.3 Basic network statistics

Table 2: Network statistics in the full graph and each layer. Active: source (out) or target (in) of at least one transaction. Sinks perform no out action, while sources perform only out actions | Full graph | USDC | DAI | USDT | USTC | WLUNC | USDP
---|---|---|---|---|---|---|---
$|V|$ | 2,908,261 | 1,025,672 | 147,966 | 1,824,419 | 39,365 | 67,756 | 6,163
$|E|$ | 5,963,549 | 2,169,372 | 288,935 | 3,451,307 | 76,016 | 106,464 | 9,115
Trades total | 26,209,860 | 10,222,791 | 1,569,005 | 13,264,318 | 394,319 | 699,345 | 60,082
Active out | 2,336,953 | 870,624 | 121,329 | 1,429,503 | 32,433 | 32,906 | 5,013
Active in | 2,547,837 | 907,951 | 126,998 | 1,580,164 | 37,074 | 66,339 | 4,289
Sources | 360,424 | 117,721 | 20,968 | 244,255 | 2,291 | 1,417 | 1,874
Sinks | 571,308 | 155,048 | 26,637 | 394,916 | 6,932 | 34,850 | 1,150

Considering the different currency transactions as layers in a multi-layer
graph we first want to establish the size of each layer/currency in terms of
nodes ($|V|$), edges ($|E|$), and total trades/transactions. This can be seen
in Table 2. The layers exhibit considerable heterogeneity in size. It can be
easily seen that USDT is by far the largest and USDP the smallest. Notably,
the two directly involved layers, USTC and WLUNC, are not among the largest
layers, but they still surpass one of the layers (USDP). This observation is
intriguing, especially considering that USTC and WLUNC are the wrapped
versions of the original currencies. Across all layers, a predominant pattern
emerges where most users are typically active either as senders or receivers,
with a higher frequency of activity as receivers. Additionally, a notable
portion of users exclusively serves as sources, solely sending out currency,
or act as sinks, exclusively receiving currency.

(a) Degree full period

(b) Degree pre crash

(c) Degree post crash

Figure 3: Probability distribution for node degree on the full data set, pre-
crash period and post-crash period. The axes are logarithmic on both scales.

Figure 3 shows the degree distribution of the different stable coins and WLUNC
pre and post crash as well as the full data set. The degree of a node is the
number of different trading partners that the node has. Currencies with a
larger userbase will tend to have higher degree nodes and hence stretch to the
bottom right of the probability distribution function (PDF). The full data is
a larger set of trades hence we expect slightly larger node degrees to result.
When comparing the pre and post crash data we can see that post crash WLUNC
has a stretched distribution indicating traders with a larger number of
trading partners in this period. By contrast, USTC has fewer trades in the
post crash period. The other coins have quite a similar distribution pre and
post crash.

##  5 Results

Using the methodology from the previous section we set out to answer various
research questions concerning the stablecoin ecosystem.

###  5.1 What is the nature of the cryptocurrency ecosystem before and after
the attack

Table 3 shows the number of users active classed by the number of layers they
are active in before and after the crash and in the full data. It can be seen
that most users trade only a single currency. The number of users trading in
all six currencies is low. One possible reason for this is that stablecoins if
they work correctly are supposed to hold a pegged value. A user who believes
this will happen might have little incentive to trade multiple stablecoins.

Table 3: Active users. We represent in how many layers are users active. Layers | Full data users |  | Pre-crash users |  | Post-crash users |
---|---|---|---|---|---|---
1 | 2,727,485 | 93.78 $\%$ | 1,201,638 | 95.43 $\%$ | 1,305,639 | 94.86 $\%$
2 | 161,004 | 5.54 $\%$ | 51,758 | 4.11 $\%$ | 64,523 | 4.69 $\%$
3 | 17,610 | 0.61 $\%$ | 5,174 | 0.41 $\%$ | 5,737 | 0.42 $\%$
4 | 1,848 | 0.06 $\%$ | 470 | 0.04 $\%$ | 362 | 0.03 $\%$
5 | 258 | 0.01 $\%$ | 65 | 0.01 $\%$ | 87 | 0.01 $\%$
6 | 56 | 0.00 $\%$ | 26 | 0.00 $\%$ | 29 | 0.00 $\%$

(a) Cross-correlation $\max_{k}(\rho(k))$ for the raw number of transactions
per day pre-crash (left), post-crash (right).

(b) Cross-correlation $\max_{k}(\rho(k))$ for the number of graph edges per
day pre-crash (left), post-crash (right).

(c) Cross-correlation $\max_{k}(\rho(k))$ for the total trade value per day
pre-crash (left), post-crash (right).

Figure 4: Cross-correlation $\max_{k}(\rho(k))$ of trade patterns for the six
currencies studied.

An obvious question to ask about the system is whether the trading patterns
within one currency correlate with those patterns within another currency. To
do this for every day we extract the raw number of edges in the graph, the raw
number of transactions and the total value of transactions converted to USD.
This gives us some different time series. We use the normalised cross-
correlation function $\rho(k)\in(-1,1)$ where $k$ is the lag in days. Here
$\rho(k)=1$ is perfect correlation, $\rho(k)=-1$ a perfect anti-correlation
and $\rho(k)=0$ no correlation at all. We plot the maximum $\max_{k}(\rho(k))$
for all $k$ as a matrix of heatmaps in Figure 4, with separate heatmaps for
the pre-crash and post-crash. In almost all cases this occurred when $k=0$.

From the analysis of Figure 4 the most obvious conclusion is that many of the
currencies have an extremely high correlation in the pre-crash period. The
number of trades and graph edges (pairs of trading partners) show extremely
high correlation in this period. The graph of total trade value shows less
correlation even in the pre-crash period but the currency USDT is a marked
anomaly having low correlation compared with the others. The likely reason for
this is a small number of large volume trade events in the pre-crash period,
that appears to be part of the attempt to destabilise the WLUNC, USDT system.
In the post crash period, the system has split into two with the remaining
four stablecoins extremely highly correlated with each other, but USDT and
WLUNC have a much lower correlation. It is also interesting to note that WLUNC
and USDT correlate well with each other in almost all analyses apart from the
pre-crash trade value which is disrupted for USDT as previously described. The
major conclusion of this analysis is that pre-crash all currencies exhibited
extremely similar trade patterns (by this measure at least) but post crash
WLUNC and USDT have become a separate eco-system.

###  5.2 Can we anticipate the crash by looking at the cryptocurrencies in the
days leading up to it?

(a) Transactions each day

(b) Edges each day

(c) Tokens transferred/traded each day

(d) Tokens transferred/traded (USD value) each day

Figure 5: Fluctuations in trading volume where each line represents one of the
currencies under study. The number of transactions is the total transactions
that day. The number of edges is the number of unique pairs of buyers
transacting. The tokens transferred are the raw number of tokens and the value
in USD uses the close price of that day’s trading to convert this.

In Figure5, various aspects of the studied currencies are monitored. Firstly,
we examine the daily number of transactions in Figure 5. The currencies
exhibit different trading volumes, reflective of the size of their active user
bases. When observing changes over time, we note the instability in values,
with clear peaks visible for some currencies during the crash. Notably,
currencies directly involved in the crash, WLUNC and USTC, demonstrate larger
variations during this period. Post-crash, both currencies show a downward
trend, although USTC displays a secondary peak in transaction volume. Similar
observations are made when considering the number of edges in Figure 5. Pre-
crash, we observe a lack of anomalous behaviour, followed by a surge during
the crash period for WLUNC and USTC. Post-crash, WLUNC and USTC experienced a
sudden increase in links to other users, interrupting a declining trend in
transaction volume. Analyzing token movements in terms of total token
transfers, presented in Figure 5, reveals noteworthy patterns. Before the
crash, two anomalous peaks in USTC token transfers were observed on the 3rd
April 2022 and the 19th April 2022 (labelled in the plot as S1 and S2,
respectively). Other layers do not exhibit significant movements before the
crash. The layers most influenced by the crash are primarily WLUNC and USTC,
with WLUNC tokens experiencing a substantial increase in movements. Post-
crash, the number of WLUNC tokens transferred remains high but eventually
decreases. This decline can be attributed to the decreasing USD value of the
tokens, as depicted in Figure 5. Despite a consistent number of tokens
transferred, their value diminishes over time. Other layers display relative
stability, with a slight rise in USDC, USDT, and USDP token movements during
the crash. In summary, suspicious anomalies can be observed before the crash,
particularly in USTC token transfers.

###  5.3 What are the effects of the attack on the system beyond the most
directly affected currencies?

(a) New Edges in WLUNC layer

(b) New Edges in USTC layer

(c) New Edges in USDT layer

Figure 6: Monitoring the new edges between unique user pairs across different
window sizes. Here new means that this is the first window in which they are
present. On the X-axis: Time. On the Y-axis: number of edges according to the
selected time window.

Our objective is to examine the driving forces behind the system, discerning
whether it is propelled by returning users who consistently contribute or by
new users who have not been encountered previously. To achieve this, we adopt
a methodology inspired by prior research [38], wherein distinctions are made
between new users or edges within a specified window. These entities encompass
users and pairwise interactions occurring within the window and have not been
active at any time prior to the selected window. The adjustment of the window
size $\tau$ enables varied definitions of an active user. We monitor, for each
layer, the number of new interactions according to daily, weekly, and bi-
weekly windows. An edge is considered new if it is the first time window it
occurs in.

We show the main currencies WLUNC and USTC, in Figures 6 and 6 respectively.
In both currencies, an intriguing pattern emerges. There are two peaks in the
number of new edges, with the first peak coinciding with the crash period.
This is likely a result of currency holders responding by engaging in trades
with new users (disposing of the coin) while new users enter the system,
likely driven by speculative motives. The second notable surge in USTC and
WLUNC plots aligns with the launch of Terra 2.0 discussed in Section 2. A
plausible explanation for this second surge stems from the criteria for
distributing new LUNA tokens on Terra 2.0 [6, 7]. The proposal specifies
eligibility criteria for users to receive tokens, based on selected snapshots
pre-attack and post-attack. The Governance chose 7th May 2022 at 11 pm for the
pre-crash snapshot and 27th May 2022 at 3.59 AM for the post-crash snapshot.
Notably, this choice favours the users who were invested in Terra or Luna
before the crash. With the voting concluding on 25th May 2022, making the plan
official, and the target launch date on 27th May 2022, users who sold during
the crash but intended to join Terra 2.0 had an incentive to purchase WLUNC
tokens before the 27th May 2022 deadline. This potentially explains the spikes
in USTC and WLUNC on 26th May 2022.

By contrast, other currencies exhibit different behaviour. Their plots
resemble that of USDT in Figure 6, with most currencies showing only one peak
immediately following the crash. While the crash induces a behaviour change,
the launch of Terra 2.0 does not significantly impact the other layers.

###  5.4 Can the after-effects of the attack be seen on graph structure as a
whole?

(a) Considered currencies (legend)

(b) Reciprocity values

(c) Fraction of users in the largest weakly connected component

(d) Clustering coefficient

(e) Average Degree

(f) Density

Figure 7: Graph measures over time. We monitor different graph measures over
time, For each day, the interactions on each currency are treated as an
individual graph. On the X-axis: Time. On the Y-axis: the selected graph
measures values over time. For clustering coefficient (c), one of the layers
exhibits outlier values, so we focus on the main currencies in the main plot,
while we provide the complete Figure as an inset.

We hypothesize that a crash could lead to changes to the graph structure.
Therefore, we delved into the presence of short and long-term trends using
traditional graph measures, including reciprocity, clustering coefficients,
weakly connected components, mean degree, and graph density. The analysis was
conducted layer by layer, treating each currency as an individual graph, and
the results are presented in Figure 7.

The reciprocity measure (see Figure 7) remains relatively stable for all
currencies. Notable spikes are observed on the WLUNC Layer before the crash,
and on the USTC layer, a rising trend is evident during the crash. However,
post-crash values return to a range similar to pre-crash levels. The other
currencies do not exhibit significant changes in their reciprocity levels.
Examining the size of the largest weakly connected component (see Figure 7),
we observe a drop in the days right after the crash (labelled C), impacting
all layers, with notable shifts in the USDC and USDT layers. However, post-
crash values revert to the pre-crash range. A small change is noted preceding
the launch of Terra 2.0 (labelled T2). The density measure (see Figure 7)
shows a significant drop right after the crash (labelled C) for Luna and USTC
layers, likely influenced by an increasing number of active users during that
time. Density values recover after the crash. Additional changes in density
values align with the launch of Terra 2.0, particularly the rise on the USTC
layer. The other currencies exhibit similar behaviour, with a drop in density
values during the crash; but are characterised by relatively stable density
values throughout the observation period. The clustering coefficient measure
(see Figure 7) showcases fewer changes. On the WLUNC layer, there’s a drop
after the crash, while the USTC layer sees a rise in the last days of the
crash, with values remaining high post-crash. The other layers do not
experience significant changes. Similarly, the mean degree (Figure 7) does not
highlight substantial changes, except for temporary rises during the crash,
aligning with the overall increase in activity.

In summary, certain graph measures confirm that the graph structure undergoes
changes during the observation period, especially during the crash. However,
users who remain active tend to re-establish a similar graph structure after
the disruptive events pass.

###  5.5 Where did user interest move post crash?

We might expect users to react to the crash in WLUNC and USTC by diversifying
into other currencies. Figure 8 shows the number of layers that users are
present in (equivalently to the number of currencies they trade that day). For
all users (see Figure 8) we can see very little difference but perhaps a small
increase immediately after the centre of the crash (label C). If we look at
only those users who trade WLUNC (see Figure 8) we can see the opposite
behaviour a decrease in the number trading multiple currencies (the apparently
smaller increase in the number of trades of a single currency is caused by the
log scale). The likely cause is the large increase in users trading WLUNC on
those days who are simply trying to sell their WLUNC and are not interested in
other trades. A similar but less pronounced trend is seen in the graph of
users who trade USTC (see Figure 8) and this figure also shows a similar trend
around event $T2$.

(a) Number of layers for all users

(b) Number of layers ((users that use WLUNC)

(c) Number of layers (users that use USTC)

Figure 8: Number of layers in which users are active. At each timestep, we
check the number of layers in which the users are active, giving them a label
(1…6). Each line represents the changes in the possible cases (1…6). On the
X-axis: Time. On the Y-axis: fraction of each possible value.

(a) Favourite layer shift for users active on WLUNC

(b) Favourite layer shift for users active on USTC

(c) Favourite layer shift for users active on DAI

(d) Favourite layer shift for users active on USDC

(e) Favourite layer shift for users active on USDP

(f) Favourite layer shift for users active on USDT

Figure 9: Changes of favourite layer for users selected pre-crash and observed
post-crash. Each graph is focused on a layer: given a layer, we select users
that had at least 1 transaction on that layer. A node is labelled with its
favourite layer i.e. the layer in which they have the most Edges.

To gain a deeper understanding of how user behaviour evolved, we conducted an
additional analysis focusing on users’ favourite layers—indicating the layer
where they are most active—before and after the crash. We presume that users
may shift their attention to different currencies post-crash, especially away
from the currencies most involved in the crash. The shifts are visualised
using Sankey diagrams in Figure 9. The Sankey diagrams reveal that users
active before the crash tend to be inactive after the crash. Across all
currencies, among those who remain active, there is a notable trend of
shifting focus to different currencies; however, there are differences among
currencies. Specifically, in the currencies directly involved in the crash,
Terra and Luna (see Figures 9 and 9) we can observe how among those who remain
active, there is a notable trend of shifting focus to different currencies.
For example, in the case of WLUNC, users predominantly migrate to USDC and
USDT. Instead, on the other layers (DAI, USDC, USDT, USDP), a higher
proportion of users remain faithful, showing less migration to other
currencies.

###  5.6 Can we locate individuals associated with the crash?

The global analysis identified anomalous selling activity on two specific
days, namely the 3rd April 2022 and the 19th April 2022. What remains
uncertain is whether this activity stems from a widespread increase in overall
activity or is driven by a few specific user accounts. Additionally, we aim to
ascertain if there is overlap in users across different days, particularly
whether the same users are involved in the anomalous days. To investigate, we
analysed the token amounts sold on these selected days, aiming to determine
each user’s contribution to the total sales for a given day. We computed the
percentage of total sales attributed to each user, visualising the top 10
users and the remaining are condensed into a single entity, the tail. This
visual representation aims to distinguish between scenarios. If the surge in
selling activity is due to a general increase in overall activity, we expect
to observe a more evenly distributed subdivision among the top 10 users, with
a significant portion of sales originating from the remaining users in the
tail. Conversely, if the increase in token movements is driven by a few users,
we anticipate a diminished impact from users outside the top 10. To validate
the anomaly, we conducted a parallel analysis on control days, specifically
the start and end of the pre-crash period (2nd April 2022, 1st May 2022), and
a day between the two peaks (11th April 2022). In the analysis, we excluded
the Zero address (0x0…0) whose outward movements represent token creation
(minting).

Figure 10: Top 10 addresses in terms of USTC tokens sold during selected days.
We represent the 2 anomalous days in terms of pre-crash USTC selling, 3rd
April 2022 and 19th April 2022, and some control days without anomalies. We
represent the percentage of tokens sold over the selected day by the top 10
users, the rest are condensed as the tail. Users from the top 10 that were
active on both anomalous days are coloured in red, while users recurrent on
other days are coloured in green.

FFigure 10 illustrates the segmentation of the top token sellers on both the
anomalous days and the control days. Observations indicate that on peak days
(3rd April 2022, 19th April 2022), a few top users dominate the token
movement, accounting for a significant percentage of total token sales —
almost $99\%$ on both days. In contrast, on control days, sales are more
evenly distributed, with the top 10 users contributing a lesser percentage to
total sales, we observe a maximum of around $65\%$. This discrepancy suggests
that a limited number of users are responsible for the peak sales, showcasing
substantial token movements not witnessed on other days.

Another important aspect is the presence of recurrent users across the various
days. In general, we observe that some users are active across different days.
Users A, B, and C consistently appear in the top 10 during the two anomalous
days, driving the anomalous peaks. However, their activities on non-peak days
are comparatively subdued. Notably, user C displays activity between peaks but
not afterwards, resurfacing only on the last control day. Other accounts
intermittently appear in the top 10 on regular days, yet their influence is
eclipsed by the substantial movements of users A, B, and C. This reveals that
among the users contributing to anomalous peaks, some exhibit coordinated
efforts, appearing active exclusively on peak days or displaying
interconnected movements.

##  6 Conclusions

This work underscores how novel temporal network analysis can reveal new
details in data from Web3, one of the novel paradigms for the Web. It
emphasises the importance of separating transactions in layers in the analysis
of temporal heterogeneous data, such as transaction data from the web. Using
the Raphtory software we were able to simultaneously look at multiple layers
(currencies), and multiple timescales and track events as time evolved.

In this study, our focus was on examining the events leading up to and
immediately following the collapse of the Luna Terra ecosystem, employing
temporal multilayer analysis. This approach enabled a layer-by-layer
exploration of the ecosystem, allowing us to assess the crash’s impact across
currencies. Our research addressed various questions related to the crash,
revealing the synchronisation of stablecoin ecosystems before the crash and
the new equilibrium found afterwards. We found that the vast majority of users
concentrate their activity on just a single layer of the network. Post-crash
we find a small, temporary rally in trading but actually very little change in
the majority of traditional graph measures considered. We show that the
majority of users trading the affected currencies simply exited trading any of
the currencies examined. Finally, we show definite evidence of collusion among
a very small number of users in coordinating events leading up to the crash.

## Acknowledgments

This work is partly funded by the Cisco research gifts scheme.

## References

  * [1] Lennart Ante, Ingo Fiedler, Jan Marius Willruth, and Fred Steinmetz.  A systematic literature review of empirical research on stablecoins.  FinTech, 2(1):34–47, 2023.
  * [2] Giulio Caldarelli.  Wrapping trust for interoperability: A preliminary study of wrapped tokens.  Information, 13(1), 2022.
  * [3] Antonio Briola, David Vidal-Tomás, Yuanrong Wang, and Tomaso Aste.  Anatomy of a stablecoin’s failure: The terra-luna case.  Finance Research Letters, 51:103358, 2023.
  * [4] Kiarash Shamsi, Friedhelm Victor, Murat Kantarcioglu, Yulia Gel, and Cuneyt G Akcora.  Chartalist: Labeled graph datasets for utxo and account-based blockchains.  Advances in Neural Information Processing Systems, 35:34926–34939, 2022.
  * [5] LUNA Go Forward Proposal - #2 by zhew2013 - Governance & Proposals - Terra Research Forum, May 2022.
  * [6] Terra Proposal 1623 to Fork The Chain Gathers 90% Support From the Community with 7 Days Left to Vote - Ethereum World News, May 2022.
  * [7] MC.  Terra 2.0 — LUNA Airdrop, May 2022.
  * [8] Terraform Labs.  Exchange migration guide | Terra Docs, July 2023.
  * [9] Haiyan Jiang, Ahsan Habib, and Mostafa Monzur Hasan.  Short selling: A review of the literature and implications for future research.  European Accounting Review, 31(1):1–31, 2022.
  * [10] Harald Uhlig.  A luna-tic stablecoin crash.  Technical report, National Bureau of Economic Research, 2022.
  * [11] Jaewoo Cho.  A token economics explanation for the de-pegging of the algorithmic stablecoin: Analysis of the case of terra.  Ledger, 8, 2023.
  * [12] Riccardo De Blasis, Luca Galati, Alexander Webb, and Robert I Webb.  Intelligent design: stablecoins (in) stability and collateral during market turbulence.  Financial Innovation, 9(1):85, 2023.
  * [13] Aurelie Barthere, Beili Baraki, Yong Li Khoo, Philip Grushyn, Xin Yi Lim, and Joshua Ho.  On-chain forensics: Demystifying terrausd de-peg.  Nansen Research, May, 27, 2022.
  * [14] Jiageng Liu, Igor Makarov, and Antoinette Schoar.  Anatomy of a run: The terra luna crash.  Technical report, National Bureau of Economic Research, 2023.
  * [15] Albert-László Barabási.  Network science.  Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1987):20120375, 2013.
  * [16] Sabeur Aridhi and Engelbert Mephu Nguifo.  Big graph mining: Frameworks and techniques.  Big Data Research, 6:1–10, 2016.
  * [17] Michele Coscia.  The atlas for the aspiring network scientist, 2021.
  * [18] Petter Holme and Jari Saramäki.  Temporal network theory, volume 2.  Springer, New York City, NY, 2019.
  * [19] Sarah Meiklejohn, Marjori Pomarole, Grant Jordan, Kirill Levchenko, Damon McCoy, Geoffrey M Voelker, and Stefan Savage.  A fistful of bitcoins: characterizing payments among men with no names.  In Proceedings of the 2013 conference on Internet measurement conference, pages 127–140, 2013.
  * [20] Kai Wang, Jun Pang, Dingjie Chen, Yu Zhao, Dapeng Huang, Chen Chen, and Weili Han.  A large-scale empirical analysis of ransomware activities in bitcoin.  ACM Transactions on the Web (TWEB), 16(2):1–29, 2021.
  * [21] Damiano Di Francesco Maesa, Andrea Marino, and Laura Ricci.  Data-driven analysis of bitcoin properties: exploiting the users graph.  International Journal of Data Science and Analytics, 6(1):63–80, 2018.
  * [22] Nicolas Gensollen and Matthieu Latapy.  Do you trade with your friends or become friends with your trading partners? a case study in the g1 cryptocurrency.  Applied Network Science, 5(1):NA–NA, 2020.
  * [23] Dongchao Guo, Jiaqing Dong, and Kai Wang.  Graph structure and statistical properties of ethereum transaction relationships.  Information Sciences, 492:58–71, 2019.
  * [24] Trishie Sharma, Rachit Agarwal, and Sandeep Kumar Shukla.  Understanding rug pulls: An in-depth behavioral analysis of fraudulent nft creators.  ACM Trans. Web, 18(1), oct 2023.
  * [25] Matthieu Nadini, Laura Alessandretti, Flavio Di Giacinto, Mauro Martino, Luca Maria Aiello, and Andrea Baronchelli.  Mapping the nft revolution: market trends, trade networks, and visual features.  Scientific reports, 11(1):1–11, 2021.
  * [26] Alessia Galdeman, Matteo Zignani, and Sabrina Gaito.  Disentangling the growth of blockchain-based networks by graph evolution rule mining.  In 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA), pages 1–10. IEEE, 2022.
  * [27] Barbara Guidi, Andrea Michienzi, and Laura Ricci.  A graph-based socioeconomic analysis of steemit.  IEEE Transactions on Computational Social Systems, 8(2):365–376, 2021.
  * [28] Cheick Tidiane Ba, Andrea Michienzi, Barbara Guidi, Matteo Zignani, Laura Ricci, and Sabrina Gaito.  Fork-based user migration in blockchain online social media.  In Proceedings of the 14th ACM conference on web science, WebSci ’22, page 174–184, New York, NY, USA, 2022. Association for Computing Machinery.
  * [29] Cheick Tidiane Ba, Matteo Zignani, and Sabrina Gaito.  Characterizing growth in decentralized socio-economic networks through triadic closure-related network motifs.  Online Social Networks and Media, 37-38:100266, 2023.
  * [30] Shenyang Huang, Jacob Danovitch, Guillaume Rabusseau, and Reihaneh Rabbany.  Fast and attributed change detection on dynamic graphs with density of states.  In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 15–26. Springer, 2023.
  * [31] Jason Zhu, Arijit Khan, and Cuneyt Gurcan Akcora.  Core-based trend detection in blockchain networks.  arXiv preprint arXiv:2303.14241, 2023.
  * [32] Tether Limited Inc.  Tether USDT whitepapers - whitepaper.io, September 2022.
  * [33] Maker Team.  The maker protocol: Makerdao’s mulfi-collateral dai (mcd) system.  White paper, 2020.
  * [34] Paxos.  USDP Whitepaper, August 2021.
  * [35] Centre.  USD Coin USDC whitepapers - whitepaper.io, May 2018.
  * [36] CoinCodex.  Crypto Prices, Charts and Cryptocurrency Market Cap | CoinCodex, November 2023.
  * [37] Ben Steer, Naomi A. Arnold, Cheick Tidiane, Renaud Lambiotte, Haaroon Yousaf, Lucas Jeub, Fabian Murariu, Shivam Kapoor, Pedro Rico, Rachel Chan, Louis Chan, James Alford, Richard G. Clegg, Felix Cuadrado, Matthew Russell Barnes, Peijie Zhong, John Pougué-Biyong, and Alhamza Alnaimi.  Raphtory: The temporal graph engine for Rust and Python.  Journal of Open Source Software, 9(95):5940, 2024.
  * [38] Naomi A Arnold, Benjamin Steer, Imane Hafnaoui, Hugo A Parada G, Raul J Mondragón, Félix Cuadrado, and Richard G Clegg.  Moving with the times: investigating the alt-right network gab with temporal interaction graphs.  Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2):1–17, 2021.


Jinhua Liang⋆, Xubo Liu∗, Haohe Liu∗, Huy Phan†, Emmanouil Benetos⋆‡,
Mark D. Plumbley∗, Wenwu Wang∗

# Adapting Language-Audio Models as Few-Shot Audio Learners

###### Abstract

Contrastive language-audio pretraining (CLAP) has become a new paradigm to
learn audio concepts with audio-text pairs. CLAP models have shown
unprecedented performance as zero-shot classifiers on downstream tasks. To
further adapt CLAP with domain-specific knowledge, a popular method is to
finetune its audio encoder with available labelled examples. However, this is
challenging in low-shot scenarios, as the amount of annotations is limited
compared to the model size. In this work, we introduce a Training-efficient
(Treff) adapter to rapidly learn with a small set of examples while
maintaining the capacity for zero-shot classification. First, we propose a
cross-attention linear model (CALM) to map a set of labelled examples and test
audio to test labels. Second, we find initialising CALM as a cosine
measurement improves our Treff adapter even without training. The Treff
adapter beats metric-based methods in few-shot settings and yields competitive
results to fully-supervised methods.

Index Terms: Contrastive language-audio pretraining, few-shot learning, domain
adaptation, audio classification

##  1 Introduction

Learning new concepts from a small set of examples is challenging in machine
learning. It is a de-facto issue in audio domains where high-quality labels
are more labor-intensive to obtain. While existing few-shot algorithms exploit
available annotations by directly learning in the few-shot setting (i.e.,
$n$-way $k$-shot problem) [1, 2], contrastive language-audio pretraining
(CLAP) provides a new paradigm to learn audio concepts using large-scale
audio-text pairs. CLAP has shown an impressive capacity for zero-shot
knowledge transfer in audio classification [3, 4, 5], largely due to the use
of a large-scale datasets and its different training objective.

To further adapt CLAP to downstream datasets, existing methods directly
finetune the audio encoder with examples in target domains [4, 5]. However,
finetuning is ill-suited for few-shot settings because the number of available
examples is extremely small with respect to the number of model parameters. In
addition, updating the parameters of CLAP's audio encoder with discriminative
learning would break the connection to its language encoder. A natural
question is: can we bootstrap a CLAP in few-shot scenarios while maintaining
its capacity as a zero-shot classifier?

Figure 1: Adaptation accuracy on the ESC-50 dataset v.s. numbers of shots
across various models. Our Treff adapters outperform other adaptation
approaches in data-scarce scenarios and are comparable to finetuning with
abundant data. We find that current approaches to domain adaptation yield a
poor performance in low-shot settings. Finetuning is even worse than zero-shot
knowledge transfer when fewer than 3 examples are available, largely due to
overfitting. This motivates our work on few-shot domain adaptation.

In this paper, we propose the Training-efficient (Treff) adapter to bootstrap
CLAP models by bridging zero-shot knowledge transfer and few-shot finetuning.
Our Treff adapter contains two core designs: a cross-attention linear model
(CALM) and cosine initialisation. First, we devise CALM where a set of support
audio clips and test audio clips are mapped to their probability
distributions. Second, we find that with a proper initialisation, using CALM
as a cosine similarity measurement yields a better performance even without
training. Figure 1 summarises performance of various adaptation approaches on
the ESC-50 dataset v.s. the number of examples (i.e., shots) per class. Our
Treff adapters outperform other adaptation approaches in few-shot scenarios
when the number of examples per classes is less than 16. As the number of
shots increases, the proposed Treff adapter can still have a competitive
result to the finetuning approach. We find that our empirical study is
consistent with some previous findings on different datasets [6]. Our
experimental results demonstrate that the proposed Treff adapter outperforms
zero-shot classification by a large margin with only a small set of annotated
data. In addition, our Treff adapter finetuned with half the number of shots
can have a comparable performance with state-of-the-art fully-supervised
learners. We speculate this work would shed light on domain adaptation in
data-limited scenarios. The contributions in this work are summarised below:

Figure 2: Overall framework of Treff adapter to classify $q$ audio clips in an
$n$-way $k$-shot setting. For the few-shot classification results $P_{FSL}$,
support embeddings $\mathbf{a}$ and test embeddings $\mathbf{\tilde{a}}$ are
extracted with a frozen audio encoder separately. Test embeddings
$\mathbf{\tilde{a}}$ then attend to support embeddings $\mathbf{a}$ and their
corresponding one-hot labels $\mathbf{y}$ in the CALM. To get the zero-shot
transferred knowledge $P_{ZSL}$, candidate labels concatenated with a fixed
prompt (e.g., ``This is a sound of ") are encoded by the frozen language
encoder to get text embeddings $\mathbf{t}$. Coefficient
$\\{\mathbf{c}_{qk}\\}$ is then measured by calculating cosine similarity
between test audio embedding $\mathbf{\tilde{a}}_{q}$ and text embedding
$\mathbf{t}_{k}$. The overall result is yielded by combining $P_{FSL}$ and
$P_{ZSL}$ with a learnable parameter $\alpha$.

  * •

We introduce the Treff adapter to further improve CLAP performance with a
small set of examples while preserving its ability to zero-shot
classification. To our knowledge, this is the first work to study audio-
language models in the few-shot audio domain.

  * •

We propose CALM to retrieve labels of the test examples by measuring the
affinity between test and support embeddings. We show that the CALM can
efficiently learn from few-shot settings while maintaining the semantic
information of the pretrained feature representation.

  * •

We devise a cosine initialisation strategy for CALM. We find that our Treff
adapter benefits from this strategy in the few-shot settings even without
training. The proposed Treff adapter in training-free version outperforms the
zero-shot classifier by over 5% accuracy on the ESC-50 dataset.

##  2 Related work

Contrastive cross-modality pretraining. CLAP [4, 5] follows the idea of
Contrastive Image-Language Pretraining (CLIP) [7] where contrastive learning
is applied to train a cross-modal retrieval model with a large number of
image-text pairs. The cross-modal retrieval model is trained so that multi-
modal embeddings from the same pair are moved closer, otherwise, pushed away.
Therefore, it can classify audio clips by measuring the similarity between
multi-modal embeddings without additional examples (referred as zero-shot
learning, ZSL). Such cross-modality models, however, cannot benefit from extra
labelled data during inference. To further adapt cross-modality retrieval
models with available examples, learnable prompts are devised to reduce CLIP's
reliance on prompt engineering [8, 9]. Still, these adaptation methods require
a large amount of labelled data for finetuning. The training-free adapter for
CLIP (TIP-adapter) was proposed to learn a cache model to map test audio to
their labels and initialise the cache model with support audio embeddings to
avoid training [10]. It initialised the weights of the cache model with the
values of support embeddings for rapid learning. The TIP-adapter, however,
inevitably jeopardises the semantic information of support examples in the
gradient-based optimisation. Cross-modal few-shot learning is designed where
the examples from other modalities are treated as additional examples [11].
Although this cross-modal method showed promising results in few-shot
classification, it requires updating linear classifiers and multiple encoders
for the best performance, which takes additional computational resources.

Metric-based few-shot learning. Few-shot learning (FSL) aims to train a
generic encoder via $n$-way $k$-shot problems where a classifier to predict
the probability distribution of test data (queries) over $n$ different classes
and each class has $k$ labelled objects (support shots or shots) for
reference. Let $(\mathbf{x}_{i},\mathbf{y}_{i})\in\mathcal{D}$ be a data point
in the dataset $\mathcal{D}$. Suppose
$\mathcal{S}=\\{(\mathbf{x}_{i},\mathbf{y}_{i})\\}_{i=1}^{nk}$ be a (small)
support set of $nk$ examples. Given a test audio clip
$\mathbf{\tilde{x}}_{q}\in\mathcal{Q}$ where $\mathcal{Q}$ is the test set, a
few-shot paradigm is expected to predict categorical probabilities
$P(\mathbf{\tilde{y}}_{q})$:

$P(\mathbf{\tilde{y}}_{q})=\mathcal{M}_{\theta}(\mathbf{\tilde{y}}_{q}|\mathbf{\tilde{x}}_{q},\mathcal{S})$
(1)

where $\mathcal{M_{\theta}}$ is any mapping function and $\theta$ represents
the model parameters. Metric-based FSL aims to cluster data points as per
their categories in the emebedding space. Matching Networks map a set of
support data and queries to their labels [1]. Prototypical networks learn
prototypes to represent different classes and measure the $l_{2}$ distance
between queries and prototypes in the embedding space [2]. Subsequent works
attempted to apply prototypical networks to multi-label cases [12]. Siamese
networks take two different data as input and learn the distance between them
explicitly [13].

##  3 Treff adapter for CLAP

###  3.1 Overall framework

We hereby introduce the Treff adapter, a simple adaptation method to boost
audio classification performance of the CLAP model by leveraging few-shot
learning.

As shown in Fig. 2, the Treff adapter makes use of audio-label pairs and zero-
shot knowledge by aggregating logits of FSL and ZSL. For FSL, the Treff
adapter extracts audio embeddings
$\mathbf{A}=[\mathbf{a}_{1},\mathbf{a}_{2},\dots,\mathbf{a}_{nk}]\in\mathbb{R}^{nk\times
d}$ and text embeddings
$\mathbf{T}=[\mathbf{t}_{1},\mathbf{t}_{2},\dots,\mathbf{t}_{n}]\in\mathbb{R}^{n\times
d}$ where $\mathbf{a}$ and $\mathbf{t}$ are column vectors of $\mathbf{A}$ and
$\mathbf{T}$, respectively, and $d$ is the dimension of the embedding in the
shared audio-language space. The test audio embedding
$\tilde{\mathbf{A}}=[\mathbf{\tilde{a}}_{1},\mathbf{\tilde{a}}_{2},\dots,\mathbf{\tilde{a}}_{q}]\in\mathbb{R}^{q\times
d}$ is likewise extracted by the audio encoder and then mapped to its label
with the support audio-label pairs using CALM. The resulted affinity matrix
$\mathbf{S}=\\{s_{ij}\\}$ is then combined with one-hot labels of support
audio for few-shot classification. In the other branch, the cosine distances
between the query audio embedding $\mathbf{\tilde{A}}$ and the text embedding
$\mathbf{T}$ are measured and utilised for zero-shot classification
explicitly. The overall inference for test audio $\mathbf{\tilde{y}}_{q}$
aggregates logits from zero-shot learning ($P_{ZSL}$) and few-shot learning
($P_{FSL}$):

$P_{overall}(\mathbf{\tilde{y}}_{q})=P_{ZSL}(\mathbf{\tilde{y}}_{q}|\mathbf{\tilde{x}}_{q},\mathbf{T})+\alpha\cdot
P_{FSL}(\mathbf{\tilde{y}}_{q}|\mathbf{\tilde{x}}_{q},\mathcal{S})$ (2)

where $\alpha$ is a trainable parameter and we initialise it with 1.0 in our
experiments.

Our Treff adapter has two core designs: CALM and cosine initialisation. We
illustrate how the CALM structure retrieves the label of a query audio clip
using a small amount of support audio examples in Section 3.2; and in Section
3.3, we show that our Treff adapter can function as a similarity measurement
without finetuning by adopting a simple initialisation.

###  3.2 CALM

Inspired by recent progress in attention mechanisms [14], we devise CALM by
using the $KQV$-attention [14]. Different from common cross-attention modules
where $K$ and $V$ are from the same modality [15, 16], CALM uses bi-modality
information from labelled examples. Figure 3 compares structures of the cache
model [10] and our CALM where $Q$ and $K$ denote the query and support audio
embeddings, respectively, and $V$ denotes the one-hot labels of the support
examples. The query embedding $\mathbf{\tilde{a}}_{i}$ and the support
embedding $\mathbf{a}_{j}$ are normalised and then fed into a trainable linear
layer, separately

$\mathbf{\tilde{e}}_{i}=f_{\mathbf{W}}(\mathbf{\tilde{a}}_{i})=\frac{\mathbf{\tilde{a}}_{i}}{|\mathbf{\tilde{a}}_{i}|}\mathbf{W}^{\mathsf{T}},$
(3)
$\mathbf{e}_{j}=f_{\mathbf{W}}(\mathbf{a}_{j})=\frac{\mathbf{\tilde{a}}_{j}}{|\mathbf{a}_{j}|}\mathbf{W}^{\mathsf{T}},$
(4)

where $\mathbf{W}$ is the weight in the trainable layer. We note that the
linear layer shares the weights across these two embeddings. The motivation is
that sharing the weights reduces the trainable parameters in the finetuning.
The linear layer maps the audio embedding to a new space where the embeddings
from the same class get closer to each other. The affinity coefficient is then
calculated by

$s_{ij}=\mathbf{\tilde{e}}_{i}^{\mathsf{T}}\mathbf{e}_{j},$ (5)

The output of the CALM is obtained by

$\mathbf{o}_{i}=\sum_{j}\varphi(s_{ij})\cdot y_{j},$ (6)

where $\mathbf{o}_{i}\in\mathbb{R}^{n}$ is the probability distribution over
the $i$-th test audio example; $\varphi(\cdot)$ is a scaling function
controlling the sharpness of the similarity coefficient

$\varphi(x)=\exp(b(1-x))$ (7)

where $b$ is a temperature factor, set empirically as 5.5 in our experiments.

Figure 3: Cache model (left) and our proposed CALM (right). The $Q$ and $K$
values are the query and support audio embeddings respectively, while the $V$
values are the one-hot labels associated to support audio examples.

###  3.3 Training-efficient adaptation

Training-free Treff adapter. The Treff adapter endows CLAP models with the
ability to learn new, domain-specific knowledge from a small set of labelled
examples. We prove that the Treff adapter can boost the model performance in a
few-shot setting even without gradient-based optimisation. Inspired by the
TIP-adapter [10], a generic encoder pretrained on a large-scale database
should be good enough to cluster data points as per their semantic
information. We replace the weights of the trainable linear layer with an
identity matrix when no fine-tuning is available. Therefore, coefficients
$s_{ij}$ in the affinity matrix would be

$s_{ij}=\biggl{(}\frac{\mathbf{\tilde{a}}_{i}}{|\mathbf{\tilde{a}}_{i}|}\biggl{)}^{\mathsf{T}}\frac{\mathbf{a}_{j}}{|\mathbf{a}_{j}|},$
(8)

In this case, the non-trainable version of CALM is actually a cosine-
similarity function between the query and support embedding (i.e.,
$\cos(\mathbf{\tilde{a}}_{i},\mathbf{a}_{j})$). We found that even the cosine-
similarity scores between the audio embeddings are capable of indicating how
close those audio examples are.

Finetuning the Treff adapter. While the training-free Treff adapter
outperforms the performance of zero-shot knowledge transfer, it is still not
comparable with finetuning and other gradient-optimization methods (e.g., the
TIP-adapter [10]). We thus optimise the weights of the Treff adapter with the
support examples only. Specifically, we initialise the weights of trainable
linear layer with the identity matrix and calculate the affinity matrix among
the support audio embeddings for the training. The weights of the linear layer
are updated using the cross-entropy between the predicted logits and the
ground truth of these support examples. Finally, we apply the updated weights
to infer the labels of the test audio clips.

###  3.4 Relations with existing works

Relation with TIP-adapter. The TIP-adapter was proposed for the image domain
whereas our Treff adapter is proposed for audio. In addition, our Treff
adapter differs from the TIP-adapter on how the trainable parameters are
finetuned. While the TIP-adapter applied a cache model to learn key values in
the few-shot knowledge retrieval, our Treff adapter trains a linear layer to
transform both query and support audio embeddings to a new space where audio
embeddings belonging to the same class get closer than those in the original
space. Therefore, the TIP-adapter will lose the information from the support
embeddings due to the gradient-optimization while our Treff adapter preserves
the semantic information of the support examples during the training. The
experimental results in Table 2 showed that the information retained is
beneficial, especially when the computational resource is limited. Meanwhile,
when initialising the learnable parameters with appropriate methods and fixing
them in the training, both our Treff adapter and TIP-adapter reduce to a
cosine similarity measure.

Relation with Matching Network. While both Matching Networks and Our Treff
adapter attempt to retrieve query labels with the affinity matrix between
query and support embedding, the matching network [1] aims to train a generic
audio encoder from scratch. Therefore, matching network keeps the affinity
calculation fixed throughout the training process. In contrast, benefiting
from pretrained audio-language models, the Treff adapter optimises the
affinity measurement in the finetuning. This makes the Treff adapter yield a
better performance than the matching network with less training resources.

Relation with Siamese network. Siamese networks [13] attempted to find a
gradient-optimised distance measure between two input examples. Our Treff
adapter, however, transforms the embedding to a semantic space where the
clusters of data points become more compact. In addition, the Siamese network
learns implicitly the feature representation for which our proposed methods
leverage a pretrained model.

##  4 Experiments and results

###  4.1 Experiment setup

We compare our methods with several sound datasets, including ESC-50 [17],
FSDKaggle2018 [18], and FSD-FS [19]. Among them, FSD-FS is a dedicated subset
of FSD50K [20] for multi-label few-shot audio classification.

To implement experiments, we resampled the audio recordings at 44.1kHz. The
window length was about 20ms with 50% overlap, and the number of Mel bank
filters was fixed to 64. Log-Mel spectrograms were used as input for few-shot
learning methods. We followed the few-shot settings as per [12, 19]. More
details can be found in our released code 111To be released after peer review.

###  4.2 Experiment results

Table 1: Accuracy (%) of different methods for the ESC-50 and FSDKaggle18K datasets. ZS denotes zero-shot performance, and FT is short for few-shot performance. * indicates our reproduced results. The best result on each dataset is marked with underline, and the best adaptation methods are highlighted in bold. Model | ESC-50 | FSDKaggle18K
---|---|---
Topline systems
AudioCLIP (FT) [3] | 97.15 | -
CLAP (FT) [4] | 95.30*/96.70 | -
HTS-AT [21] | 97.00 | -
Baseline systems
VGGish (ZS) [22] | 33.00 | -
AudioCLIP (ZS) [3] | 69.40 | -
CLAP (ZS) [4] | 82.60 | -
Treff adapter (w/o FT) | 87.75 | 71.03
TIP-adapter (w/ FT) [10] | 91.45* | 83.23*
Treff adapter (w/ FT) | 92.21 | 86.58

Comparison with fine-tuning. Table 1 shows the comparison between the proposed
Treff adapter against the state-of-the-art following both fine-tuning (topline
systems) and the zero-shot transferring (baseline systems) approach. We show
the results of CLAP adapters trained with 16 examples here, as the improvement
is getting marginal when the amount of support data increases (see more
details in Figure 1). Our Treff adapter outperforms the zero-shot learning
methods by a large margin and even achieves a comparable performance with the
fine-tuning methods. In addition, the proposed Treff adapter outperforms the
TIP-adapter, which was also adopted as a CLAP adapter, by 0.8% absolute in
terms of accuracy. This is likely because our Treff adapter benefits from the
preserved semantic information in the support embedding whereas the TIP-
adapter discards it via gradient-based optimisation.

Table 2: Accuracy (%) of various methods in the few-shot settings. The best
results are highlighted in bold.

| ESC-50 (%) | FSD-FS (%)
---|---|---
| 5-way | 12-way | 15-way
ProtoNet [2] | 88.18 | 77.70 | 33.02
MatchNet [1] | 86.83 | 71.81 | -
HPN [12] | 88.65 | 78.65 | -
Treff adapter (w/o FT) | 97.49 | 94.68 | 68.34
TIP-adapter [10] | 97.52 | 95.58 | 69.45
Treff adapter | 98.53 | 96.29 | 70.59

Comparison with few-shot learning. Table 2 shows the accuracy of the proposed
Treff adapter against the metric-based few-shot learning and the TIP-adapter.
Both our Treff adapter and the TIP-adapter achieve better results than other
metric-based learning algorithms by a large margin. The reason may be that
these two adapters utilise a large-scale pretrained model as their encoder.
Our proposed Treff adapter outperforms the TIP-adapter by 0.71 percentage
points in terms of accuracy, indicating that CALM learns task-specific
knowledge while preserving the knowledge from CLAP. Table 3 compares the
proposed Treff-adapter with other cross-modality few-shot methods on the
ImageNet-ESC [11]. It can be observed that the Treff adapter achieves the best
performance. It outperforms the cross-modality few-shot learning by a large
margin as it is able to make use of zero-shot knowledge transferring
explicitly while the cross-modality FSL discards it gradually in the parameter
optimisation.

Table 3: Accuracy (%) of cross-modality few-shot learning. The best results are highlighted in bold. Xmodal FSL denotes cross-modality FSL method. Dataset | Method | 1-shot | 2-shot | 4-shot
---|---|---|---|---
| Xmodal FSL | 35.70 | 45.90 | 51.60
ImageNet-ESC-19 | TIP-adapter | 86.45 | 87.89 | 87.11
| Treff adapter | 85.26 | 88.03 | 87.63
| Xmodal FSL | 35.00 | 43.50 | 48.50
ImageNet-ESC–27 | TIP-adapter | 86.39 | 87.31 | 87.03
| Treff adapter | 86.02 | 87.04 | 89.35

##  5 Conclusion

We presented the Treff adapter, a training-efficient adapter for CLAP, to
boost zero-shot classification performance by making use of a small set of
labelled data. Specifically, we designed CALM to retrieve the probability
distribution of text-audio clips over classes using a set of audio-label pairs
and combined it with CLAP's zero-shot classification results. Furthermore, we
designed a training-free version of the Treff adapter by using CALM as a
cosine similarity measure. Experiments showed that the proposed Treff adapter
is comparable and even better than fully-supervised methods and adaptation
methods in low-shot and data-abundant scenarios. While the Treff adapter shows
that combining large-scale pretraining and rapid learning of domain-specific
knowledge is non-trivial for obtaining generic representations for few-shot
learning, it is still limited to audio classification tasks. In the future, we
will explore how to use audio-language models in diverse audio domains.

##  6 Acknowledgments

This work was supported by the Engineering and Physical Sciences Research
Council [grant number EP/T518086/1]. The research utilised Queen Mary's
Apocrita HPC facility, supported by QMUL Research-IT,
http://doi.org/10.5281/zenodo.438045.

## References

  * [1] O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra, ``Matching Networks for One Shot Learning,'' in _Advances in Neural Information Processing Systems_ , vol. 29. Curran Associates, Inc., 2016. [Online]. Available: https://proceedings.neurips.cc/paper/2016/hash/90e1357833654983612fb05e3ec9148c-Abstract.html
  * [2] J. Snell, K. Swersky, and R. Zemel, ``Prototypical Networks for Few-shot Learning,'' _Advances in Neural Information Processing Systems_ , vol. 30, pp. 4077–4087, 2017.
  * [3] A. Guzhov, F. Raue, J. Hees, and A. Dengel, ``Audioclip: Extending Clip to Image, Text and Audio,'' in _ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , May 2022, pp. 976–980, iSSN: 2379-190X.
  * [4] B. Elizalde, S. Deshmukh, M. A. Ismail, and H. Wang, ``CLAP: Learning Audio Concepts From Natural Language Supervision,'' Jun. 2022, arXiv:2206.04769 [cs, eess]. [Online]. Available: http://arxiv.org/abs/2206.04769
  * [5] Y. Wu*, K. Chen*, T. Zhang*, Y. Hui*, T. Berg-Kirkpatrick, and S. Dubnov, ``Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation,'' in _IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP_ , 2023.
  * [6] Y. Wang, N. J. Bryan, J. Salamon, M. Cartwright, and J. P. Bello, ``Who Calls The Shots? Rethinking Few-Shot Learning for Audio,'' in _2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_. IEEE, 2021, pp. 36–40.
  * [7] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, ``Learning Transferable Visual Models From Natural Language Supervision,'' in _Proceedings of the 38th International Conference on Machine Learning_. PMLR, Jul. 2021, pp. 8748–8763, iSSN: 2640-3498. [Online]. Available: https://proceedings.mlr.press/v139/radford21a.html
  * [8] K. Zhou, J. Yang, C. C. Loy, and Z. Liu, ``Learning to Prompt for Vision-Language Models,'' _International Journal of Computer Vision_ , vol. 130, no. 9, pp. 2337–2348, Sep. 2022. [Online]. Available: https://doi.org/10.1007/s11263-022-01653-1
  * [9] ——, ``Conditional Prompt Learning for Vision-Language Models,'' in _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022.
  * [10] R. Zhang, W. Zhang, R. Fang, P. Gao, K. Li, J. Dai, Y. Qiao, and H. Li, ``Tip-Adapter: Training-Free Adaption of CLIP for Few-Shot Classification,'' in _Computer Vision – ECCV 2022_ , ser. Lecture Notes in Computer Science, S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, Eds. Cham: Springer Nature Switzerland, 2022, pp. 493–510.
  * [11] Z. Lin, S. Yu, Z. Kuang, D. Pathak, and D. Ramanan, ``Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models,'' Jan. 2023, arXiv:2301.06267 [cs, eess]. [Online]. Available: http://arxiv.org/abs/2301.06267
  * [12] J. Liang, H. Phan, and E. Benetos, ``Leveraging Label Hierachies for Few-Shot Everyday Sound Recognition,'' in _Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop (DCASE2022)_ , Nancy, France, Nov. 2022.
  * [13] G. Koch, R. Zemel, and R. Salakhutdinov, ``Siamese Neural Networks for One-shot Image Recognition,'' in _ICML Deep Learning Workshop_ , vol. 2. Lille, 2015.
  * [14] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, ``Attention Is All You Need,'' in _Advances in neural information processing systems_ , 2017, pp. 5998–6008.
  * [15] A. Jaegle, S. Borgeaud, J.-B. Alayrac, C. Doersch, C. Ionescu, D. Ding, S. Koppula, D. Zoran, A. Brock, E. Shelhamer, O. J. Henaff, M. Botvinick, A. Zisserman, O. Vinyals, and J. Carreira, ``Perceiver IO: A General Architecture for Structured Inputs & Outputs,'' Mar. 2022. [Online]. Available: https://openreview.net/forum?id=fILj7WpI-g
  * [16] J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Barreira, O. Vinyals, A. Zisserman, and K. Simonyan, ``Flamingo: a Visual Language Model for Few-Shot Learning,'' Nov. 2022, arXiv:2204.14198 [cs]. [Online]. Available: http://arxiv.org/abs/2204.14198
  * [17] K. J. Piczak, ``ESC: Dataset for Environmental Sound Classification,'' in _Proceedings of the 23rd ACM International Conference on Multimedia_. Brisbane Australia: ACM, Oct. 2015, pp. 1015–1018. [Online]. Available: https://dl.acm.org/doi/10.1145/2733373.2806390
  * [18] E. Fonseca, M. Plakal, F. Font, D. P. Ellis, X. Favory, J. Pons, and X. Serra, ``General-purpose Tagging of Freesound Audio with Audioset Labels: Task Description, Dataset, and Baseline,'' in _Proceedings of the Detection and Classification of Acoustic Scenes and Events 2018 Workshop (DCASE2018)_ , Nov. 2018, pp. 69–73.
  * [19] J. Liang, H. Phan, and E. Benetos, ``Learning from Taxonomy: Multi-label Few-Shot Classification for Everyday Sound Recognition,'' Dec. 2022, arXiv:2212.08952 [cs, eess]. [Online]. Available: http://arxiv.org/abs/2212.08952
  * [20] E. Fonseca, X. Favory, J. Pons, F. Font, and X. Serra, ``FSD50K: An Open Dataset of Human-Labeled Sound Events,'' _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 30, pp. 829–852, 2022.
  * [21] K. Chen, X. Du, B. Zhu, Z. Ma, T. Berg-Kirkpatrick, and S. Dubnov, ``HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection,'' in _ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2022, pp. 646–650, iSSN: 2379-190X.
  * [22] S. Hershey, S. Chaudhuri, D. P. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold, and others, ``CNN Architectures for Large-scale Audio Classification,'' in _2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2017, pp. 131–135.


With the ejection of dredged up material during each eruption, the white dwarf
must be losing mass each eruption. Such a system will evolve with a long
whittling away of the mass of the two stellar components, with lengthening
nova recurrence timescale and shortening orbital period, ultimately to quietly
fade away. Unfortunately, the case for U Sco as a neon nova is confused. Mason
(2011) claimed that U Sco is a neon nova, based on the strong neon lines
(compared to the [OIII] $\lambda$5007 line) and on the P-Cygni profile for
ultraviolet lines as characteristic only of neon novae. But then in a
corrigendum, Mason (2013) pointed out a case (with a high electron density of
$>$106 cm-3) where the neon abundance need not be more than mildly enriched.
This point was immediately denied by their collection of measures for the best
cases of neon novae, which have strong neon lines (compared to the oxygen
lines) yet have electron densities $>$106 cm-3. Further, I do not see how the
U Sco ejecta can have such high electron densities as $>$106 cm-3 at 104 days
past peak (of the 2010 eruption) with its very low ejecta mass and very high
expansion velocity. So we are left with U Sco having very bright neon emission
lines and P-Cygni profiles in the ultraviolet, looking just like the known
neon novae. Still, I cannot see that a confident conclusion has been reached.
In all, based on the composition of the ejecta, we are left with no-conclusion
as to whether the U Sco white dwarf is gaining or losing mass.

####  5.5.3 What is driving the accretion?

U Sco has an incredibly high accretion rate of $\sim$(5–16)$\times$10-7
M⊙/year, which places it near the maximum possible for any CV, to avoid the
steady-hydrogen-burning regime. What is driving this rare and extreme
accretion? The high accretion rate can only be caused by a rapid shrinkage of
the companion’s Roche lobe size or by a rapid expansion of the companion star
itself.

The ordinary expansion of a star after leaving the main sequence is well known
from tracks of models in the H-R diagram. With no accurate current mass for
the companion, and no useful estimate of its original main-sequence mass, the
expansion rates cannot be determined with any high accuracy. For a
characteristic path in the middle of the subgiant track, a 1.0 M⊙ star will
expand from 1.75 to 2.47 R⊙ in 0.5 Gyr, for an expansion velocity of 1.0
m/year. Such a slow expansion cannot provide the observed accretion rate for
any plausible gas density along the surface of the Roche lobe, by many orders-
of-magnitude. Choosing any other plausible evolutionary track still has the
companion’s expansion rate too small to matter by many orders-of-magnitude.
This demonstrates that the expansion of the companion star is not contributing
to driving the accretion.

The only other possibility is that the companion’s Roche lobe is shrinking at
some fast rate. A changing size of the Roche lobe is tied to a proportionate
change of the orbital period
($\dot{R}_{Roche}/R_{Roche}$=$\frac{2}{3}$$\dot{P}/P$). For U Sco, an
apparently good measure of the evolutionary $\dot{P}$ is
($+$0.20$\pm$0.05)$\times$10-9 from 1945–2022, averaging over seven eruptions.
But the trouble is that this long-term evolutionary trend is positive, with
the period increasing, and the companion’s Roche lobe getting larger. With
such evolution, U Sco cannot have any sort of a high $\dot{M}$.

Something is driving the high $\dot{M}$ in U Sco, and it can only be the rapid
shrinkage of the companion’s Roche lobe. Thus, the evolutionary $\dot{P}$ must
be negative, with an overall smoothed parabola that is concave-down in Fig. 9.
Something is wrong with the upturn in the uniform evolution prominently seen
in Fig. 9. Perhaps chaotic variations in $\Delta P$ and $\dot{P}$ only average
out to some evolutionary average rate on time-scales $\gg$77 years and $\gg$7
eruptions.

Presumably, the long-term average $\dot{P}$ will not be much greater than the
maximal period change observed during the inter-eruption intervals. That is,
the evolutionary $\dot{P}$ cannot be expected to be more negative than the
$\dot{P}$ from the 2010.1–2016.78 interval, $-$2.1$\times$10-8. For this
maximal evolutionary rate and a 2.2 R⊙ companion, the companion’s Roche lobe
would be contracting at a rate of 6.3 km/year. This is to be compared to an
atmospheric scale height of order 700 km. With such a rate, the Roche lobe
would have moved inward by close to one atmospheric scale height in the 116
years going back to 1906, the 1906 accretion would have been near a factor of
2.7$\times$ smaller than currently, and the oldest $\delta Y$ values should be
more like 30 years. This is clearly contradictory to the eruption record, and
the Roche lobe is not shrinking at a rate anywhere near to 6.3 km/year.

Another perhaps severe problem arises when trying to get accretion of
$\sim$1$\times$10-7 M⊙/year when just 6.3 km of shell crosses over the Roche
lobe each year. To get the observed rate, the density in the accretion zone
must be roughly 0.001 gm/cc. This does not sound like a dense atmosphere by
terrestrial standards, but for a 1 M⊙ star at 2.2 R⊙, such a density only
occurs far inside the expected surface. For a similar isolated star, the 0.001
gm/cc density level occurs roughly 5 per cent inward from the photosphere.
That is, the Roche lobe has shrunk far inside the companion’s surface, and the
companion star has not responded by having local expansion to achieve any
condition close to hydrostatic equilibrium. The restoration of hydrostatic
equilibrium is on the sound-crossing time-scale, which is much faster than the
time-scale for the contraction of the Roche lobe. I do not understand how it
is possible for the stellar atmosphere to have such a high density near the
Roche lobe. The vertical density structure near the Roche lobe of a fast
accreting system is a difficult problem for current astrophysics. Still
perhaps a solution can be found where U Sco can have its high $\dot{M}$ driven
by some hypothetical maximal period change. Until then, the mechanism that is
driving the very high accretion in U Sco remains mysterious.

####  5.5.4 Overview on evolution

The evolution of CVs is now the forefront issue for CV research, and is also
the higher-level goal of my program to measure eclipse times and period
changes of U Sco and other novae. CV evolution is driven by period changes.
The U Sco case shows that the evolution is controlled by both $\Delta
P_{nova}$ and $\dot{P}$, with these roughly cancelling out. Both $\Delta
P_{nova}$ and $\dot{P}$ show large variations from eruption to eruption, with
the causes being mysterious. The physical mechanism for the sudden period
changes across the nova events cannot be any of the well-known mechanisms
(e.g., mass loss, frictional angular momentum loss), but instead is apparently
caused by the ubiquitous asymmetric ejection of mass into the nova shell
pushing a jet force on to the white dwarf. The physical mechanism for the
steady period change between eruptions is not from the steady mass transfer,
but rather must be dominated by the poorly-known mechanism of magnetic braking
in a stellar wind of the companion star.

The very-high accretion rate in U Sco is not being driven by the ordinary
evolutionary expansion of the subgiant companion star (such is greatly too
small to matter in all cases), hence the driving for the very-high $\dot{M}$
can only come from a fast shrinkage of the Roche lobe and by a large negative
$\dot{P}$. Unfortunately, the 77 year $O-C$ records shows that $\dot{P}$ is
positive over 7 eruption cycles. I can only explain this discrepancy by
suggesting that the vagaries of the variations in $\Delta P_{nova}$ and
$\dot{P}$ randomly happened to produce a positive $\dot{P}$ over the 1945–2022
interval, whereas intervals $\gg$77 years would produce a steady average
negative $\dot{P}$. For any negative $\dot{P}$, the gas density at the Roche
surface is, apparently, impossibly high to allow the observed $\dot{M}$. There
is no understanding of the cause or mechanism for the very-high observed
accretion rate in U Sco.

For U Sco (and the 31 per cent of nova systems with evolved companions), the
progenitor system was a wide detached binary with a white dwarf and a $\sim$1
M⊙ secondary star. The CV was formed only after ordinary evolution of the
secondary star had it leave the main sequence and expand until coming into
contact with its Roche lobe. U Sco now has a very high mass white dwarf (1.36
M⊙) and a very high accretion rate ($\sim$(5–16)$\times$10-7 M⊙/year), with
both of these conditions required to make the system a recurrent nova (with
$\delta Y$$\sim$10 years). Whether the white dwarf is gaining or losing mass
is not known, with order-of-magnitude uncertainties in the balance between
accretion and ejection. In the end, the fate of U Sco is still a mystery.

##  6 ACKNOWLEDGEMENTS

I am grateful for the eclipse time series observations of G. Myers and W.
Cooney, with these being critical for recognizing and understanding the
2016.78 eruption. E. Christensen and K. Battams have kindly passed along CSS
and LASCO images. Further, many observers and data-handlers provided light
curves from the AAVSO, CRTS, Pan-STARRS, DECam, Kepler satellite, ATLAS, and
ZTF.

##  7 DATA AVAILABILITY

All of the photometry data are explicitly given in Tables 2, 3, 4, and 6, or
are publicly available from the references and links in Table 1.

## References

  * Bellm et al. (2010) Banerjee D. P. K. et al., 2010, MNRAS, 408, L71
  * Bellm et al. (2019) Bellm E. C. et al., 2019, PASP, 131, 18002
  * Bruch (2021) Bruch A., 2021, MNRAS, 509, 4669
  * Chambers ert al. (2016) Chambers K. C. et al., 2016, arXiv:1612.05560
  * Diaz et al. (2010) Diaz M., Williams R., Luna G., Moraes M., Takeda L., 2010, AJ, 140, 1860
  * Drake et al. (2009) Drake A. J. et al., 2009, ApJ, 696, 870
  * Drake et al. (2014) Drake A. J. et al., 2014, MNRAS, 441, 1186
  * Heinze et al. (2018) Heinze A. N. et al., 2018, AJ, 156, 241
  * Hillman et al. (2020) Hillman Y., Shara M., Prialnik D., Kovetz A., 2020, Nature Astron., 4, 886
  * Kafka and Williams (2011) Kafka S., Williams R., 2011, A&A, 526, 83
  * Knigge, Baraffe, and Patterson (2011) Knigge C., Baraffe I., Patterson J., 2011, ApJS, 194, 28
  * Livio, Govarie, and Ritter (1991) Livio M., Govarie A., Ritter H. 1991, A&A, 246, 84
  * Lu et al. (2012) Lu Y., Xiang F.-Y., Shi X.-M., 2012, PASJ, 64, 84
  * Mason (2011) Mason E., 2011, A&A, 532, L11
  * Mason (2013) Mason E., 2013, A&A, 556, C2
  * Mason et al. (2012) Mason E., Ederoclite A., Williams R. E., Della Valle M., Setiawan J., 2012, A&A, 544, 149
  * Nanouris et al. (2011) Nanouris N., Kalimeris A., Antonopoulou E., Rovithis-Livaniou H., 2011, A&A, 535, A126
  * Ness et al. (2012) Ness J.-U. et al., 2012, ApJ, 745, 43
  * Orio et al. (2013) Orio M. et al., 2013, MNRAS, 429, 1342
  * Page et al. (2013) Page K. L. et al., 2013, ApJ, 768, L26
  * Pagnotta et al. (2015) Pagnotta A. et al., 2015, ApJ, 811, 32
  * Patterson (1984) Patterson J., 1984, ApJS, 54, 443
  * Rappaport, Joss, and Webbink (1982) Rappaport S., Joss P. C., Webbink R. F., 1982, ApJ, 254, 616
  * Siviero et al. (2006) Siviero A., Dallaporta S., & Munari U., 2006, BaltA, 15, 387
  * Schaefer (1990) Schaefer B. E., 1990, ApJ, 355, L39
  * Schaefer (2005) Schaefer B. E., 2005, ApJ, 621, L53
  * Schaefer (2009) Schaefer B. E., 2009, ApJ, 697, 721
  * Schaefer (2010) Schaefer B. E., 2010, ApJS, 187, 275
  * Schaefer (2011) Schaefer B. E., 2011, ApJ, 742, 112
  * Schaefer (2020a) Schaefer B. E., 2020a, MNRAS, 492, 3323
  * Schaefer (2020b) Schaefer B. E., 2020b, MNRAS, 492, 3343
  * Schaefer (2021) Schaefer B. E., 2021, RNAAS, 5, 148
  * Schaefer (2022) Schaefer B. E., 2022, MNRAS, in press
  * Schaefer and Patterson (1983) Schaefer B. E., Patterson, J. O., 1983, ApJ, 268, 710
  * Schaefer and Ringwald (1995) Schaefer B. E., Ringwald F. A., 1995, ApJ, 447, L45
  * Schaefer et al. (1992) Schaefer B. E. et al., 1992, ApJS, 81, 321
  * Schaefer et al. (2010) Schaefer B. E. et al., 2010, AJ, 140, 925
  * Schaefer et al. (2011) Schaefer B. E. et al., 2011, ApJ, 742, 113
  * Schaefer et al. (2013) Schaefer B. E. et al., 2013, ApJ, 773, 55
  * Schaefer et al. (2019) Schaefer B. E. et al., 2019, MNRAS, 487, 1120
  * Schaefer, Pagnotta, and Zoppelt (2022) Schaefer B. E., Pagnotta A., Zoppelt S., 2022, MNRAS, 512, 1924
  * Shara et al. (1986) Shara M. M., Livio M., Moffatt A. F. J., Orio M., 1986, ApJ, 311, 163
  * Shara et al. (1986) Shara M. M., Zurek D., De Marco O., Mizusawa T., Williams R., Livio M., 2012, AJ, 143, 143
  * Shara et al. (2018) Shara M. M., Prialnik D., Hillman Y., Kovetz A., 2018, ApJ, 860, 110
  * Shen and Bildsten (2009) Shen K. J., Bildsten L., 2009, ApJ, 699, 324
  * Takei et al. (2013) Takei D. et al., 2013, ApJ, 769, L4
  * Tonry et al. (2018) Tonry, J. L. et al., 2018, PASP, 130, 4505